View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0001060bareos-corestorage daemonpublic2019-02-17 20:072024-03-20 14:25
Reporterdr80 Assigned Tojoergs  
PriorityhighSeveritymajorReproducibilityalways
Status closedResolutionno change required 
PlatformLinuxOSSLESOS Version12
Product Version18.2.5 
Summary0001060: bareos-storage-droplet: no https possible with AWS S3
DescriptionHello,

https is not possible with bareos-storage-droplet in combination with AWS S3. Configuration was created as described in the documentation (https://docs.bareos.org/chapter20/storage-backends.html).

Only when "use_https" is set to false in droplet.profile it will works.

Steps To Reproduce- Install Bareos 18.2.5 with bareos-storage-droplet
- Create S3 Bucket on AWS for Region eu-central-1 (Frankfurt)
- Configure the Storage Daemon for using droplet
- label new volume to S3 "label volume=Full-0001 pool=Full storage=S3_Object"

#bconsole:
Enter a period to cancel a command.
label volume=Full-0001 pool=Full storage=S3_Object
Automatically selected Catalog: MyCatalog
Using Catalog "MyCatalog"
Connecting to Storage daemon S3_Object at linux-o9nk.local:9103 ...
Sending label command for Volume "Full-0001" Slot 0 ...
3910 Unable to open device ""S3_ObjectStorage" (S3)": ERR=stored/dev.cc:731 Could not open: S3/Full-0001, ERR=Success

#bconsole:
Enter a period to cancel a command.
*messages
17-Feb 19:47 bareos-sd: ERROR in backends/droplet_device.cc:109 error: src/conn.c:389: init_ssl_conn: SSL connect error: 0: 0
17-Feb 19:47 bareos-sd: ERROR in backends/droplet_device.cc:109 error: src/conn.c:392: init_ssl_conn: SSL certificate verification status: 0: ok
Additional InformationTested with OS: SLES 12 SP4/CentOS 7 and version 18.2.5/18.2.6

Storage Configuration: bareos-sd.d/device/S3_ObjectStorage.conf

Device {
  Name = S3_ObjectStorage
  Media Type = S3_Object1
  Archive Device = S3 Object Storage

  Device Options = "profile=/etc/bareos/bareos-sd.d/device/droplet/aws_eu-central-1.profile,bucket=<aws_s3_bucketname>,chunksize=100M,iothreads=0,retries=1"

  Device Type = droplet
  Label Media = yes # lets Bareos label unlabeled media
  Random Access = yes
  Automatic Mount = yes # when device opened, read it
  Removable Media = no
  Always Open = no
  Description = "S3 device"
  Maximum Concurrent Jobs = 1
}

Director Configuration: bareos-dir.d\storage\S3_Object.conf

Storage {
  Name = S3_Object
  Address = linux-o9nk.local
  Password = "<pw-storage>"
  Device = S3_ObjectStorage
  Media Type = S3_Object1
}

Droplet Configuration: bareos-sd.d\device\aws_eu-central-1.profile

# Generic host, but can't access buckets younger than 24h:
#host = s3.amazonaws.com
# Region specific host name. Can access also new buckets.
host = s3.eu-central-1.amazonaws.com
use_https = true
backend = s3
aws_region = eu-central-1
aws_auth_sign_version = 4
access_key = "<ACCESS_KEY>"
secret_key = "<SECRET_KEY>"
pricing_dir = ""
Tagsaws, droplet, s3, storage

Activities

andrei693

andrei693

2019-03-13 18:32

reporter   ~0003282

Have you tried with host = s3.amazonaws.com?
Works here on 18.2.5 on Centos7 with region us-west-2.

*label volume=Full-AWS-0001 pool=Full-AWS storage=S3_Object
Connecting to Storage daemon S3_Object at bareos-sd.example.com:9103 ...
Sending label command for Volume "Full-AWS-0001" Slot 0 ...
3000 OK label. VolBytes=226 Volume="Full-AWS-0001" Device="AWS_S3_1-00" (AWS S3 Storage)
Catalog record for Volume "Full-AWS-0001", Slot 0 successfully created.
Requesting to mount AWS_S3_1-00 ...
3906 File device "AWS_S3_1-00" (AWS S3 Storage) is always mounted.
dr80

dr80

2019-03-25 17:47

reporter   ~0003299

Yes, test with "s3.amazonaws.com" and "s3.eu-central-1.amazonaws.com"

I noticed that even if use_https = true is set, it still tries to communicate via port 80 / http.

After that I set as test -> host = s3.eu-central-1.amazonaws.com:443

Afterwards the communication took place via 443 / https.

My Droplet Config now looks like this:

Droplet Configuration: bareos-sd.d/device/aws_eu-central-1.profile

host = s3.eu-central-1.amazonaws.com:443
use_https = true
backend = s3
aws_region = eu-central-1
aws_auth_sign_version = 4
access_key = "<ACCESS_KEY>"
secret_key = "<SECRET_KEY>"
pricing_dir = ""

There seems to be a bug in the Droplet Library. Here the wrong port is used for SSL communication by default.
CysiaCom

CysiaCom

2019-03-26 13:29

reporter   ~0003300

Can confirm same behavior on WEST-1 zone.
Setting HTTPS = True didn't worked in any way i did set up.

Changing the host adding the port and enabling https worked.

My Working setting right now:
host = s3-eu-west-1.amazonaws.com:443
use_https = true
backend = s3
aws_region = eu-west-1
aws_auth_sign_version = 4
access_key = "KEY"
secret_key = "KEY"
pricing_dir = ""

bconsole Storage Status says now:
Device "AWS_S3_XXXX" (S3) is not open.
Jmsg Job=*System* type=6 level=1553602883 CysiaServer-sd: info: src/droplet.c:127: dpl_init: PRNG has been seeded with enough data
Backend connection is working.
No pending IO flush requests.
Lufa

Lufa

2020-12-10 13:17

reporter   ~0004063

Have the same issue. Adding :443 to the host's address solves the problem.

  Version: 19.2.7 (16 April 2020) Linux-3.10.0-1062.18.1.el7.x86_64 redhat Red Hat Enterprise Linux Server release 7.7 (Maipo)
  CentOS Linux release 7.9.2009 (Core)
ideacloud

ideacloud

2021-03-30 06:24

reporter   ~0004106

I have the same problem. No AWS connections using the plugin work correctly. They all show the same error in
bruno-at-bareos

bruno-at-bareos

2024-03-20 14:25

manager   ~0005851

while the droplet plugin will get replaced in future 23.1.x version when you want to connect with https you need to set the port to :443 after the hostname.

Issue History

Date Modified Username Field Change
2019-02-17 20:07 dr80 New Issue
2019-02-17 20:07 dr80 Tag Attached: s3;droplet;aws;storage
2019-02-17 20:12 dr80 Tag Attached: droplet
2019-02-17 20:12 dr80 Tag Attached: s3
2019-02-17 20:12 dr80 Tag Attached: aws
2019-02-17 20:12 dr80 Tag Attached: storage
2019-02-17 20:12 dr80 Tag Detached: s3;droplet;aws;storage
2019-03-13 18:32 andrei693 Note Added: 0003282
2019-03-25 17:47 dr80 Note Added: 0003299
2019-03-26 13:29 CysiaCom Note Added: 0003300
2019-07-02 15:59 joergs Assigned To => joergs
2019-07-02 15:59 joergs Status new => acknowledged
2020-12-10 13:17 Lufa Note Added: 0004063
2021-03-30 06:24 ideacloud Note Added: 0004106
2024-03-20 14:25 bruno-at-bareos Status acknowledged => closed
2024-03-20 14:25 bruno-at-bareos Resolution open => no change required
2024-03-20 14:25 bruno-at-bareos Note Added: 0005851