View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0000617 | bareos-core | storage daemon | public | 2016-02-10 19:56 | 2019-12-18 15:25 |
Reporter | matt01 | Assigned To | mvwieringen | ||
Priority | normal | Severity | minor | Reproducibility | always |
Status | closed | Resolution | fixed | ||
Platform | Linux | OS | any | OS Version | 3 |
Product Version | 15.2.2 | ||||
Summary | 0000617: Add support for rados_striper functions in rados_device.c where it currently dosn't exist. | ||||
Description | Not all of the calls in rados_device.c will use their rados_striper alternatives if HAVE_RADOS_STRIPER is defined. rados_striper functions need to be provided m_striper. | ||||
Tags | No tags attached. | ||||
0001-add-support-for-rados_striper-functions-in-rados_dev.patch (4,039 bytes)
From fe573d6a77a1b8802c5f4d5d3b08f7d4fe6d82e5 Mon Sep 17 00:00:00 2001 From: mr245 <mr245@bath.ac.uk> Date: Wed, 10 Feb 2016 14:27:08 +0000 Subject: [PATCH] add support for rados_striper functions in rados_device.c where it currently dosn't exist --- src/stored/backends/rados_device.c | 46 +++++++++++++++++++++++++++++++++----- 1 file changed, 41 insertions(+), 5 deletions(-) diff --git a/src/stored/backends/rados_device.c b/src/stored/backends/rados_device.c index da6b64a..cee871d 100644 --- a/src/stored/backends/rados_device.c +++ b/src/stored/backends/rados_device.c @@ -210,8 +210,11 @@ int rados_device::d_open(const char *pathname, int flags, int mode) /* * See if the object already exists. */ +#ifdef HAVE_RADOS_STRIPER + status = rados_striper_stat(m_striper, getVolCatName(), &object_size, &object_mtime); +#else status = rados_stat(m_ctx, getVolCatName(), &object_size, &object_mtime); - +#endif /* * See if the O_CREAT flag is set. */ @@ -225,8 +228,8 @@ int rados_device::d_open(const char *pathname, int flags, int mode) */ #ifdef HAVE_RADOS_STRIPER if (m_stripe_volume) { - rados_striper_write(m_ctx, getVolCatName(), " ", 1, 0); - rados_striper_trunc(m_ctx, getVolCatName(), 0); + rados_striper_write(m_striper, getVolCatName(), " ", 1, 0); + rados_striper_trunc(m_striper, getVolCatName(), 0); } else { #endif rados_write(m_ctx, getVolCatName(), " ", 1, 0); @@ -323,7 +326,7 @@ ssize_t rados_device::d_write(int fd, const void *buffer, size_t count) int status; #ifdef HAVE_RADOS_STRIPER - if (m_striper) { + if (m_stripe_volume) { status = rados_striper_write(m_striper, getVolCatName(), (char *)buffer, count, m_offset); } else { #endif @@ -386,11 +389,25 @@ boffset_t rados_device::d_lseek(DCR *dcr, boffset_t offset, int whence) uint64_t object_size; time_t object_mtime; +#ifdef HAVE_RADOS_STRIPER + if (m_stripe_volume) { + if (rados_striper_stat(m_striper, getVolCatName(), &object_size, &object_mtime) == 0) { + m_offset = object_size + offset; + } else { + return -1; + } + } else { +#endif if (rados_stat(m_ctx, getVolCatName(), &object_size, &object_mtime) == 0) { m_offset = object_size + offset; } else { return -1; } + +#ifdef HAVE_RADOS_STRIPER + } +#endif + break; } default: @@ -410,7 +427,7 @@ bool rados_device::d_truncate(DCR *dcr) #ifdef HAVE_RADOS_STRIPER if (m_stripe_volume) { - status = rados_striper_trunc(m_ctx, getVolCatName(), 0); + status = rados_striper_trunc(m_striper, getVolCatName(), 0); } else { #endif status = rados_trunc(m_ctx, getVolCatName(), 0); @@ -424,7 +441,16 @@ bool rados_device::d_truncate(DCR *dcr) return false; } +#ifdef HAVE_RADOS_STRIPER + if (m_stripe_volume) { + status = rados_striper_stat(m_striper, getVolCatName(), &object_size, &object_mtime); + } else { +#endif status = rados_stat(m_ctx, getVolCatName(), &object_size, &object_mtime); +#ifdef HAVE_RADOS_STRIPER + } +#endif + if (status < 0) { Mmsg2(errmsg, _("Unable to stat volume %s. ERR=%s\n"), getVolCatName(), be.bstrerror(-status)); Dmsg1(100, "%s", errmsg); @@ -432,7 +458,17 @@ bool rados_device::d_truncate(DCR *dcr) } if (object_size != 0) { /* rados_trunc() didn't work. */ + +#ifdef HAVE_RADOS_STRIPER + if (m_stripe_volume) { + status = rados_striper_remove(m_striper, getVolCatName()); + } else { +#endif status = rados_remove(m_ctx, getVolCatName()); +#ifdef HAVE_RADOS_STRIPER + } +#endif + if (status < 0) { Mmsg2(errmsg, _("Unable to remove volume %s. ERR=%s\n"), getVolCatName(), be.bstrerror(-status)); Dmsg1(100, "%s", errmsg); -- 2.4.10 |
|
There are a couple of problems with this patch. Next to that I have some other patches pending so I took the patch and tried merging the changes. Problem with the striper stuff was that it was fully new when I added the code and docs where nowhere to be found. So it was more a proof of concept then anything that might work at all. |
|
I pushed a temporary branch to github under https://github.com/bareos/bareos/commits/mvw/ceph-changes which includes some more fixes. It may need some work but is probably a better base to start fixing. |
|
Ok Thanks, I agree there doesn't seem to be any documentation on the striper functions, Looking at the header files all of the functions appear to be the same as the non striper alternatives. Hopefully documentation will appear soon. We will be looking at using ceph for backups here and striping is desirable so I plan on testing this heavily soon. Matt |
|
Yes CEPH is kind of sloppy with documenting their API they also changed the write API in some version from returning the number of bytes and then all of a sudden return true or false. When doing the rados_create2 interface last week I run into something similar as it seems it existed before but then only had 2 arguments nowadays 4 and you have to look in the git repo when they changed it and then look forward in the commits to see when the API version number gets bumped. Honestly I have not to much interest in every possible storage platform but when we started S3 I already found out quickly that the native APIs of CEPH and Gluster are much better suited for the way we work then the S3 api which you also can use on those platforms. Now that storage backends are abstracted as part of cloud storage stuff we can easily plugin new ones. But for most we are depended somewhat on end-users as keeping them all in shape is not something that we can spend numerous hours on. You might run into some problems with the branch I mentioned as some stuff is based on the limited documentation available. |
|
Fix committed to bareos master branch with changesetid 6132. | |
bareos: master 01c224cf 2016-02-10 16:27 mr245 Committer: mvwieringen Ported: N/A Details Diff |
ceph: Add more support for rados_striper functions. Fixes 0000617: Add support for rados_striper functions in rados_device.c where it currently dosn't exist. Signed-off-by: Marco van Wieringen <marco.van.wieringen@bareos.com> |
Affected Issues 0000617 |
|
mod - src/stored/backends/rados_device.c | Diff File | ||
mod - src/stored/backends/rados_device.h | Diff File |
Date Modified | Username | Field | Change |
---|---|---|---|
2016-02-10 19:56 | matt01 | New Issue | |
2016-02-10 19:56 | matt01 | File Added: 0001-add-support-for-rados_striper-functions-in-rados_dev.patch | |
2016-02-11 10:00 | mvwieringen | Note Added: 0002189 | |
2016-02-11 10:05 | mvwieringen | Note Added: 0002190 | |
2016-02-11 10:06 | mvwieringen | Assigned To | => mvwieringen |
2016-02-11 10:06 | mvwieringen | Status | new => feedback |
2016-02-11 11:07 | matt01 | Note Added: 0002191 | |
2016-02-11 11:07 | matt01 | Status | feedback => assigned |
2016-02-11 12:49 | mvwieringen | Note Added: 0002192 | |
2016-02-11 12:50 | mvwieringen | Status | assigned => feedback |
2016-02-12 21:37 | mvwieringen | Changeset attached | => bareos master 01c224cf |
2016-02-12 21:37 | mvwieringen | Note Added: 0002194 | |
2016-02-12 21:37 | mvwieringen | Status | feedback => resolved |
2016-02-12 21:37 | mvwieringen | Resolution | open => fixed |
2019-12-18 15:25 | arogge | Status | resolved => closed |