 |
On the Bug Reporting Page, it says Before reporting a bug, please read the Bug Reporting Howto: https://www.bareos.org/en/HOWTO/articles/how-to-create-a-bugreport.html Here especially interesting: "Also, please keep your language polite. You want somebody to help you with your problem and solve it, that is seldomly achieved by being impolite." |
|
|
Take a look here. For a beta feature this contains more than enough documentation. https://fossies.org/linux/bareos-Release/README.droplet |
|
|
I have already seen this and it is not sufficient. I didn't succeed setting connection to AWS S3 properly and seeing posts on bareos-users mailing group, I'm not the only one. Nothing really is explained there. According to what is stated in this draft, I have set up everything properly, so following that I am either wrong or there is a bug and droplet doesn't work. But having it so barely documented I cannot say which one is right. |
|
|
Please post your configuration and the output of storage daemon running with debug level set to 250 when running a job. Maybe I can help. I will also write some kind of readme for the bareos/S3 setup down. |
|
|
/etc/bareos/bareos-sd.d/device/droplet/droplet.profile: use_https = False host = my-bucket.s3.eu-west-2.amazonaws.com access_key = "myaccesskey" secret_key = "mysecretkey" pricing_dir = "" backend = s3 aws_auth_sign_version = 4 /etc/bareos/bareos-sd.d/device/S3-droplet-lon-001.conf: Device { Name = S3Droplet-lon-001 Media Type = s3bucket Archive Device = S3 Object Storage # # Config options: # profile= - Droplet profile to use either absolute PATH or logical name (e.g. ~/.droplet/<profile>.profile # location= - AWS location (e.g. us-east etc.) # acl= - Canned ACL # storageclass= - Storage Class to use. # bucket= - Bucket to store objects in. # chunksize= - Size of Volume Chunks (default = 10 Mb) # iothreads= - Number of IO-threads to use for uploads (use blocking uploads if not set.) # ioslots= - Number of IO-slots per IO-thread (default 10) # mmap - Use mmap to allocate Chunk memory instead of malloc(). # Device Options = "profile=/etc/bareos/bareos-sd.d/device/droplet/droplet.profile,bucket=my-bucket,iothreads=1,ioslots=1,location=eu-west-2" Device Type = droplet LabelMedia = yes # lets Bareos label unlabeled media Random Access = yes AutomaticMount = yes # when device opened, read it RemovableMedia = no AlwaysOpen = no Description = "S3 Object device. A connecting Director must have the same Name and MediaType." Maximum File Size = 200M # 500 MB (Allows for seeking to small portions of the Volume) Maximum Concurrent Jobs = 1 #Maximum Spool Size = 15000M } I've tried manipulating iothreads, ioslots, bucket name, with/without location and Maximum Spool Size, different Maximum File Size and for profile I've tried different formats of URL, use_https = True and nothing has worked so far. Will upload debug level 250 log file soon, can also provide more verbose log if needed. I've also posted some debug in https://groups.google.com/forum/#!topic/bareos-users/8-qOrSvcI2E when I was trying one of the varying configs. |
|
|
|
|
|
Change your host in your droplet.profile to "s3.amazonaws.com", as the backend will insert the bucket and location parameters from your Device Options there. But as I assume that you tested this before. What you can try is to test your droplet profile with dplsh, it is a shell tool that tests your profile and direct-connects to your cloud storage. https://github.com/scality/Droplet-sh |
|
|
No, I haven't tried to set it up like you say before, as in bareos-users group which I linked previously, you can see that thraizz has posted very similar configuration (using full URL bucket) and claimed that this setup works for him. Also nothing from the short manual regarding droplet suggested that I could try this option, the manual even suggests the opposite. As for the droplet-sh you mentioned - I don't really see how could this help me verifying the droplet. I have successfully connected to that bucket with S3FS and with `aws s3 ls`, so the access to the bucket is fine. If the droplet-sh would let me test my profile file, I don't see it at all as you say that bucket name and region is taken by backend from Device Options, so the droplet-sh would not be able to get these data if the 'host' line wouldn't contain them. I have just tested following options: host = s3.eu-west-2.amazonaws.com host = s3.amazonaws.com and it doesn't work, my bucket didn't register any traffic, so bareos didn't even try to contact it. |
|
|
The "host=" argument in your droplet profile is prepended by "bucket=" and "location=" arguments in your device ressource. If the host is set as s3.eu-west-2-amazonaws.com bareos-sd will try to reach the bucket "my-bucket.s3-eu-west-2.my-bucket.s3.eu-west-2-amazonaws.com". thraizz's setup cant work. I cannot see any droplet errors in your trace file, none the less I have created my own AWS S3 bucket and it seems as the resolving is still buggy - our internal version can reach and resolve the host right, while the same configuration fails with public version and the error: "Failed to lookup hostname ".s3.amazonaws.com": Unknown host". Be sure to keep your version updated as there is already a working solution, it's just not out yet. I am also writing a plugin documentation like we have it for the vmware plugin so that the configuration will be easier. Sorry for the inconvenience and thanks for the feedback. |
|
|
ok, cool, that clarifies the setup, I will amend mine and it would be great if you updated this ticket when you roll out the version that fixes the issue so I can test it and come back with feedback on newer version. still needs documentation though, as you see, the part you've written regarding the bucket URL construction is not described anywhere apart from this ticket and it is not really clear. |
|
|
Fix committed to bareos master branch with changesetid 8966. |
|
|
Fix committed to bareos dev branch with changesetid 9356. |
- Anonymous
