 |
|
|
|
I have the same problem. No AWS connections using the plugin work correctly. They all show the same error in |
|
|
I also have the same problem. 02-Jun 10:49 bareos-sd JobId 16: Warning: stored/label.cc:389 Open device "AWS_S3_1-00" (AWS S3 Storage) Volume "S3_Full-0003" failed: ERR=stored/dev.cc:734 Could not open: AWS S3 Storage/S3_Full-0003, ERR=Success 02-Jun 10:50 bareos-sd JobId 16: Warning: stored/label.cc:389 Open device "AWS_S3_1-00" (AWS S3 Storage) Volume "S3_Full-0003" failed: ERR=stored/dev.cc:734 Could not open: AWS S3 Storage/S3_Full-0003, ERR=Success 02-Jun 10:50 bareos-sd JobId 16: Warning: stored/mount.cc:276 Open device "AWS_S3_1-00" (AWS S3 Storage) Volume "S3_Full-0003" failed: ERR=stored/dev.cc:734 Could not open: AWS S3 Storage/S3_Full-0003, ERR=Success 02-Jun 10:50 bareos-sd JobId 16: Please mount append Volume "S3_Full-0003" or label a new one for: Job: S3_backup-bareos-fd.2021-06-02_10.38.04_45 Storage: "AWS_S3_1-00" (AWS S3 Storage) Pool: S3_Full Media type: S3_Object1 02-Jun 10:56 bareos-sd JobId 16: Warning: stored/mount.cc:276 Open device "AWS_S3_1-00" (AWS S3 Storage) Volume "S3_Full-0003" failed: ERR=stored/dev.cc:734 Could not open: AWS S3 Storage/S3_Full-0003, ERR=Success I can mount via console but not label: *mount The defined Storage resources are: 1: File 2: S3_Object Select Storage resource (1-2): 2 Connecting to Storage daemon S3_Object at bareos-test:9103 ... 3001 OK mount requested. Device="AWS_S3_1-00" (AWS S3 Storage) *label The defined Storage resources are: 1: File 2: S3_Object Select Storage resource (1-2): 2 Enter new Volume name: S3_Full-0004 Defined Pools: 1: Scratch 2: S3_Incremental 3: S3_Full 4: S3_Differential 5: Incremental 6: Full 7: Differential Select the Pool (1-7): 3 Connecting to Storage daemon S3_Object at bareos-test:9103 ... Sending label command for Volume "S3_Full-0004" Slot 0 ... 3910 Unable to open device ""AWS_S3_1-00" (AWS S3 Storage)": ERR=stored/dev.cc:734 Could not open: AWS S3 Storage/S3_Full-0004, ERR=Success Label command failed for Volume S3_Full-0004. Do not forget to mount the drive!!! Centos 7.4, latest bareos |
|
|
Problem solved. Added AWS policy "AmazonS3FullAccess" to the bucket and it connected and tried to label and write to volumes. But no volumes were written: 105 2021-06-02 17:04:56 bareos-sd JobId 19: End of medium on Volume "S3_Full-0015" Bytes=104,832,224 Blocks=1,625 at 02-Jun-2021 17:04. 104 2021-06-02 17:04:56 bareos-sd JobId 19: Error: stored/block.cc:822 Write error on fd=0 at file:blk 0:104832223 on device "AWS_S3_1-00" (AWS S3 Storage). ERR=No such file or directory. 103 2021-06-02 17:04:56 bareos-sd JobId 19: Error: stored/block.cc:803 Write error at 0:104832223 on device "AWS_S3_1-00" (AWS S3 Storage). ERR=No such file or directory. 102 2021-06-02 17:04:30 bareos-sd JobId 19: New volume "S3_Full-0015" mounted on device "AWS_S3_1-00" (AWS S3 Storage) at 02-Jun-2021 17:04. 101 2021-06-02 17:04:30 bareos-sd JobId 19: Wrote label to prelabeled Volume "S3_Full-0015" on device "AWS_S3_1-00" (AWS S3 Storage) 100 2021-06-02 17:04:30 bareos-sd JobId 19: Labeled new Volume "S3_Full-0015" on device "AWS_S3_1-00" (AWS S3 Storage). 99 2021-06-02 17:04:30 bareos-dir JobId 19: Created new Volume "S3_Full-0015" in catalog. No complaints on permissions or likewise. Just nothing written. |
|
|
@perhallenborg Have you created the bucket `droplet-bareos` before running the task? One suggestion: try to test with these options (iothreads and retries): `Device Options = "profile=/etc/bareos/bareos-sd.d/device/droplet/aws.profile,bucket=droplet-bareos,chunksize=100M,iothreads=0,retries=1"` |
|
|
Bucket created, but with another name. I can connect and write to the bucket from my Mac, as a mounted device. So the keys and secrets work. And Bareos doesn't complain about not being connected, as far as I can see. Tried also with iothreads=2 Tried nightly build Bareos 21 but same issue. |
|
|
So, the name of the backet at AWS-side are matched to the Bareos settings name. Correct? When I debug my S3 storage, it is useful to enable logging on the AWS side. what does bconsole show you, if you ask something like: *show storage="AWS_S3_1-00" *status storage="AWS_S3_1-00" ? |
|
|
Buckets are a match. Tried loggin on AWS side but it is empty. Config was taken directly from Bareos wiki. Enter a period (.) to cancel a command. *show storage="AWS_S3_1_00" storage resource AWS_S3_1_00 not found. You have messages. *status storage="AWS_S3_1_00" Storage resource "AWS_S3_1_00": not found The defined Storage resources are: 1: File 2: S3_Object Select Storage resource (1-2): 2 Connecting to Storage daemon S3_Object at bareos-test.adnet.adlibris.se:9103 bareos-sd Version: 21.0.0~pre540.f3597a14d (01 June 2021) CentOS Linux release 7.9.2009 (Core) Daemon started 03-Jun-21 10:13. Jobs: run=1, running=0, pre-release version binary Sizes: boffset_t=8 size_t=8 int32_t=4 int64_t=8 bwlimit=0kB/s Running Jobs: No Jobs running. ==== Jobs waiting to reserve a drive: ==== Terminated Jobs: JobId Level Files Bytes Status Finished Name =================================================================== 23 Full 570 58.88 M Error 03-Jun-21 08:41 S3_backup-bareos-fd 24 Full 570 58.88 M Cancel 03-Jun-21 08:52 S3_backup-bareos-fd 25 Incr 0 0 Cancel 03-Jun-21 09:06 S3_backup-bareos-fd 26 Full 0 0 Cancel 03-Jun-21 09:10 S3_backup-bareos-fd 27 Incr 0 0 Cancel 03-Jun-21 09:20 S3_backup-bareos-fd 28 Incr 0 0 Error 03-Jun-21 09:23 S3_backup-bareos-fd 29 Full 54,796 2.424 G Cancel 03-Jun-21 09:26 S3_backup-bareos-fd 30 Incr 16 2.967 M Error 03-Jun-21 10:00 S3_backup-bareos-fd 31 Full 61,355 2.524 G Error 03-Jun-21 10:03 S3_backup-bareos-fd 32 Full 61,355 2.524 G Error 03-Jun-21 10:26 S3_backup-bareos-fd ==== Device status: Device "AWS_S3_1-00" (AWS S3 Storage) is not open. Backend connection is working. No pending IO flush requests. == ==== Used Volume status: ==== |
|
|
Ahhh, *show storage="AWS_S3_1-00" should be *show storage="S3_Object" > Backend connection is working. Tell u that bareos SD can connect. That is mine Device configuration ``` #cat storage/bareos-sd.d/device/WASABI_ObjectStorage.conf Device { Name = WASABI_ObjectStorage Media Type = S3_Object1 Archive Device = WASABI Object Storage Device Options = "profile=/etc/bareos/bareos-sd.d/device/droplet/wasabi_us-east-1.profile,bucket=i-clf1-bareos,chunksize=100M,iothreads=7,retries=0" Device Type = droplet Label Media = yes # lets Bareos label unlabeled media Random Access = yes Automatic Mount = yes # when device opened, read it Removable Media = no Always Open = no Maximum Concurrent Jobs = 1 } # cat storage/bareos-sd.d/device/droplet/wasabi_us-east-1.profile host = s3.us-east-1.wasabisys.com:443 use_https = true backend = s3 aws_region = us-east-1 aws_auth_sign_version = 4 access_key = "" secret_key = "" pricing_dir = "" # ``` I can see only one difference: the host contain the 443 port. > host = s3.us-east-1.wasabisys.com:443 Try to use it. |
|
|
I already do, doesn't work without it. host = <bucketname>.s3-eu-north-1.amazonaws.com:443 # This parameter is only used as baseurl and will be prepended with bucket and location set in device resource to form correct url use_https = true access_key = xx secret_key = xx pricing_dir = "/etc/bareos/pricing" # If not empty, an droplet.csv file will be created which will record all S3 operations. backend = s3 aws_auth_sign_version = 4 # Currently, AWS S3 uses version 4. The Ceph S3 gateway uses version 2. aws_region = eu-north-1 |
|
|
I have also hit this issue and one reason behind is that the SSL cert is valid for *.s3.amazonaws.com but that * only allows one level and no dots. Also SSL certs for *.*.fqdn are not allowed. curl https://some-bucket.eu-central-1.s3.amazonaws.com curl: (60) SSL: no alternative certificate subject name matches target host name 'some-bucket.eu-central-1.s3.amazonaws.com' OTOH, this says, it is valid: openssl s_client -connect some-bucket.eu-central-1.s3.amazonaws.com:443 so maybe one fix could be to let libdroplet do the verifcation the openssl way. |
|
|
consolidating droplet issues |
- Anonymous
