Source Control Integration - Bareos Bug Tracker

bareos-17.2 f7d97b88 2017-07-02 01:22 mvwieringen Committer: pstorz Ported: N/A Details Diff	stored: Implement object device chunked volumes. As some of the REST protocols only allow you to write blobs in total we now offer a chunked object store device type in which the volumes are chunked into pieces and these pieces are flushed to the backing store either on close or when we reach the maximum size of the chunk and need a new one. We also implemented the reading of such chunked devices. This runs on top of Scality's libdroplet with some local fixes and changes.
	mod - autoconf/configure.in	Diff File
	mod - src/lib/Makefile.in	Diff File
	mod - src/lib/cbuf.c	Diff File
	mod - src/lib/cbuf.h	Diff File
	add - src/lib/ordered_cbuf.c	Diff File
	add - src/lib/ordered_cbuf.h	Diff File
	mod - src/stored/Makefile.in	Diff File
	mod - src/stored/backends/Makefile.in	Diff File
	add - src/stored/backends/chunked_device.c	Diff File
	add - src/stored/backends/chunked_device.h	Diff File
	mod - src/stored/backends/object_store_device.c	Diff File
	mod - src/stored/backends/object_store_device.h	Diff File
	mod - src/stored/backends/unix_tape_device.c	Diff File
	mod - src/stored/dev.c	Diff File
	mod - src/win32/stored/backends/win32_tape_device.c	Diff File
bareos-17.2 0ab6d804 2017-07-02 01:22 mvwieringen Committer: pstorz Ported: N/A Details Diff	build: Rebuild configure and config.h.in
	mod - autoconf/config.h.in	Diff File
	mod - configure	Diff File
bareos-16.2 2ef36e5f 2017-07-02 01:22 mvwieringen Ported: N/A Details Diff	stored: Use SEEK_CUR not SEEK_END to check positioning. When we want to determine if the position is set right in DCR::is_eod_valid() we need to perform a lseek(fd, 0, SEEK_CUR) and not a lseek(fd, 0, SEEK_END) as that is already done in DEVICE::eod(). We only want to determine if the lseek was done right in DCR::is_eod_valid() and by doing a SEEK_END we do the same seek twice which leads to the same result but is not correct.
	mod - src/stored/mount.c	Diff File
bareos-16.2-droplet 9c13a428 2017-07-02 01:22 mvwieringen Ported: N/A Details Diff	lib: Add edit_pthread() function to print pthread_t ids. There is no real clean way to print the pthread_t type returned by various pthreads functions. Until now things were printed as pointer addresses. The edit_pthread() method is analog to other edit functions and tries to print things in an uniform way and should be used to print the content of the pthread_t opaque structure. The benefit of this is that the representation is in one central place and we can implement specific representations for different platforms. As part of the implementation of this new function the edit.c source file was also refactored and reindented to be better readable. Further more now all modifiers are stored as static const char pointers which used to be done for some functions but also not in others.
	mod - src/lib/btimers.c	Diff File
	mod - src/lib/edit.c	Diff File
	mod - src/lib/jcr.c	Diff File
	mod - src/lib/lockmgr.c	Diff File
	mod - src/lib/protos.h	Diff File
	mod - src/lib/res.c	Diff File
	mod - src/stored/lock.c	Diff File
bareos-16.2-droplet 44bddc87 2017-07-02 01:22 mvwieringen Ported: N/A Details Diff	stored: Add device specific status trigger. This adds code that allows you to trigger a device to return specific device status information analog to the way which was already available to plugins using the bsdEventDriveStatus and bsdEventVolumeStatus events. In essence this is equivalent to the bsdEventDriveStatus which allows a plugin (for example the scsicrypto-sd plugin to return the crypto status of an LTO4+ drive.) but then for any device without the need to have a plugin loaded for the specific device.
	mod - src/stored/dev.h	Diff File
	mod - src/stored/sd_plugins.h	Diff File
	mod - src/stored/status.c	Diff File
bareos-16.2-droplet 45651202 2017-07-02 01:22 mvwieringen Ported: N/A Details Diff	stored: Implement object device chunked volumes. As some of the REST protocols only allow you to write blobs in total we now offer a chunked object store device type in which the volumes are chunked into pieces and these pieces are flushed to the backing store either on close or when we reach the maximum size of the chunk and need a new one. We also implemented the reading of such chunked devices. This runs on top of Scality's libdroplet with some local fixes and changes.
	mod - autoconf/configure.in	Diff File
	mod - src/lib/Makefile.in	Diff File
	mod - src/lib/cbuf.c	Diff File
	mod - src/lib/cbuf.h	Diff File
	add - src/lib/ordered_cbuf.c	Diff File
	add - src/lib/ordered_cbuf.h	Diff File
	mod - src/stored/Makefile.in	Diff File
	mod - src/stored/backends/Makefile.in	Diff File
	add - src/stored/backends/chunked_device.c	Diff File
	add - src/stored/backends/chunked_device.h	Diff File
	mod - src/stored/backends/object_store_device.c	Diff File
	mod - src/stored/backends/object_store_device.h	Diff File
	mod - src/stored/backends/unix_tape_device.c	Diff File
	mod - src/stored/dev.c	Diff File
	mod - src/win32/stored/backends/win32_tape_device.c	Diff File
bareos-16.2-droplet 2c893f00 2017-07-02 01:22 mvwieringen Ported: N/A Details Diff	build: Rebuild configure and config.h.in
	mod - autoconf/config.h.in	Diff File
	mod - configure	Diff File
master 42d7e719 2017-06-30 20:18 joergs Ported: N/A Details Diff	bsock: allow sending long messages While API modes 0 and 1 normally send data as small chunks (often line based), API mode 2 (JSON) collects the result of a bcommand, forms a JSON object and sends it. This can result in large messages. The maximum valid size of the result of a bcommand used to be 4 MB. The size of "llist jobs limit=1" is already close to 1 KB. So before this patch, "llist jobs" silently fails if the Bareos Director returns more than 5000 jobs. With this patch, the maximum length of a bcommand result is int32 (2GB). This is a modified version of commit 84ba7730bb4799f0793b2c0ee8572c5c75970ffe, which had to be reverted because of problems. Fixes 0000719: ERROR in output_formatter.c:822 Failed to send result as json.	Affected Issues 0000719
	mod - src/lib/bsock_tcp.c	Diff File
master 5f688789 2017-06-30 17:37 joergs Ported: N/A Details Diff	tweak help string of the update command some other tweaks
	mod - src/cats/update_bareos_tables.in	Diff File
	mod - src/defaultconfigs/bareos-dir.d/catalog/MyCatalog.conf.in	Diff File
	mod - src/dird/ua_cmds.c	Diff File
	mod - src/dird/ua_update.c	Diff File
bareos-17.2 d287bee6 2017-06-28 19:41 mvwieringen Committer: pstorz Ported: N/A Details Diff	build: Update copyright. Claim copyright for all changes done over the years. Copyright statements are already in most source files but not in the generic version header and the command output. So essentially this doesn't change anything already not in effect already just makes it more prominent. Also added the BAREOS copyright which was missing in the comment section of version.h
	mod - src/include/version.h	Diff File
bareos-16.2 56fe5d9c 2017-06-28 19:41 mvwieringen Ported: N/A Details Diff	build: Update copyright. Claim copyright for all changes done over the years. Copyright statements are already in most source files but not in the generic version header and the command output. So essentially this doesn't change anything already not in effect already just makes it more prominent. Also added the BAREOS copyright which was missing in the comment section of version.h
	mod - src/include/version.h	Diff File
bareos-17.2 4fe52fcd 2017-06-28 19:41 mvwieringen Committer: pstorz Ported: N/A Details Diff	cephfs: Don't reinitialize the connection to CEPH. The plugin event bEventRestoreCommand is fired for each file we are supposed to restore, currently we don't check if we already have setup the connection to CEPH using the function connect_to_cephfs() and as such we leak a set of socket connections for each file restored. This is bad so lets properly check if things are already setup correctly. For a backup we do the same e.g. check if p_ctx->cmount is not already initialized.
	mod - src/plugins/filed/cephfs-fd.c	Diff File
bareos-16.2 ec32843f 2017-06-28 19:41 mvwieringen Ported: N/A Details Diff	cephfs: Don't reinitialize the connection to CEPH. The plugin event bEventRestoreCommand is fired for each file we are supposed to restore, currently we don't check if we already have setup the connection to CEPH using the function connect_to_cephfs() and as such we leak a set of socket connections for each file restored. This is bad so lets properly check if things are already setup correctly. For a backup we do the same e.g. check if p_ctx->cmount is not already initialized.
	mod - src/plugins/filed/cephfs-fd.c	Diff File
bareos-17.2 b1babb6a 2017-06-28 19:41 mvwieringen Committer: pstorz Ported: N/A Details Diff	gfapi: Don't reinitialize the connection to gluster The plugin event bEventRestoreCommand is fired for each file we are supposed to restore, currently we don't check if we already have setup the connection to gluster using the function connect_to_gluster() and as such we leak a set of socket connections for each file restored. This is bad so lets properly check if things are already setup correctly. For a backup we do the same e.g. check if p_ctx->glfs is not already initialized.
	mod - src/plugins/filed/gfapi-fd.c	Diff File
bareos-16.2 a5aa2618 2017-06-28 19:41 mvwieringen Ported: N/A Details Diff	gfapi: Don't reinitialize the connection to gluster The plugin event bEventRestoreCommand is fired for each file we are supposed to restore, currently we don't check if we already have setup the connection to gluster using the function connect_to_gluster() and as such we leak a set of socket connections for each file restored. This is bad so lets properly check if things are already setup correctly. For a backup we do the same e.g. check if p_ctx->glfs is not already initialized.
	mod - src/plugins/filed/gfapi-fd.c	Diff File
bareos-17.2 48e0c779 2017-06-28 19:41 mvwieringen Committer: pstorz Ported: N/A Details Diff	gfapi: Explicitly close glfs fd on IO-open. Close the glfs fd handle on an IO_OPEN when it was not closed for whatever reason.
	mod - src/plugins/filed/gfapi-fd.c	Diff File
bareos-16.2 fde6778b 2017-06-28 19:41 mvwieringen Ported: N/A Details Diff	gfapi: Explicitly close glfs fd on IO-open. Close the glfs fd handle on an IO_OPEN when it was not closed for whatever reason.
	mod - src/plugins/filed/gfapi-fd.c	Diff File
bareos-17.2 d8036ba2 2017-06-28 19:41 mvwieringen Committer: pstorz Ported: N/A Details Diff	gfapi: Fix backups with empty glusterfind filelist. The original plugin interface always expect one or more files to be backuped. With glusterfind it can happen that no files changed since the last backup which means when we do a setup_backup we return bRC_OK instead of bRC_More which normally means there are more files to backup (for determining what file needs to be processed next we use get_next_file_to_backup() which is also used in the endBackupFile() plugin function. Problem is however returning bRC_OK has different meaning when used in bEventBackupCommand e.g. start of backup vs endBackupFile() there is a nice and smart workaround and that is that we know normally get_next_file_to_backup() should always return bRC_More and only on an empty list a bRC_OK is returned. The way the initialization works we can return anything other then bRC_OK (bRC_Skip in our case) which means the loop never gets executed. As we now never get into the backup loop we also are no longer bitten by the side effect you see on backing up an empty file list e.g. failing backups due to the fact that the backup expects one or more valid so called savepackets. When the filelist is empty a savepacket with as sp_type zero is returned and that leads to the following error: 'no type in startBackupFile packet.' So the problem manifested itself as a way different error then what actually is the real problem. Corner case closed.
	mod - src/filed/fd_plugins.c	Diff File
	mod - src/plugins/filed/gfapi-fd.c	Diff File
bareos-16.2 bc43e15b 2017-06-28 19:41 mvwieringen Ported: N/A Details Diff	gfapi: Fix backups with empty glusterfind filelist. The original plugin interface always expect one or more files to be backuped. With glusterfind it can happen that no files changed since the last backup which means when we do a setup_backup we return bRC_OK instead of bRC_More which normally means there are more files to backup (for determining what file needs to be processed next we use get_next_file_to_backup() which is also used in the endBackupFile() plugin function. Problem is however returning bRC_OK has different meaning when used in bEventBackupCommand e.g. start of backup vs endBackupFile() there is a nice and smart workaround and that is that we know normally get_next_file_to_backup() should always return bRC_More and only on an empty list a bRC_OK is returned. The way the initialization works we can return anything other then bRC_OK (bRC_Skip in our case) which means the loop never gets executed. As we now never get into the backup loop we also are no longer bitten by the side effect you see on backing up an empty file list e.g. failing backups due to the fact that the backup expects one or more valid so called savepackets. When the filelist is empty a savepacket with as sp_type zero is returned and that leads to the following error: 'no type in startBackupFile packet.' So the problem manifested itself as a way different error then what actually is the real problem. Corner case closed.
	mod - src/filed/fd_plugins.c	Diff File
	mod - src/plugins/filed/gfapi-fd.c	Diff File
bareos-17.2 8a2d9ce9 2017-06-28 19:41 mvwieringen Committer: pstorz Ported: N/A Details Diff	gfapi: Allow to use non-accurate backups with glusterfind With the introduction of glusterfind support we explicitly assumed that accurate mode was always used by the Job doing the backup. This patch allows you to run non accurate backups although best practice is to use accurate mode.
	mod - src/plugins/filed/gfapi-fd.c	Diff File
bareos-16.2 53ca1154 2017-06-28 19:41 mvwieringen Ported: N/A Details Diff	gfapi: Allow to use non-accurate backups with glusterfind With the introduction of glusterfind support we explicitly assumed that accurate mode was always used by the Job doing the backup. This patch allows you to run non accurate backups although best practice is to use accurate mode.
	mod - src/plugins/filed/gfapi-fd.c	Diff File
bareos-17.2 f7f488a7 2017-06-28 19:41 mvwieringen Committer: pstorz Ported: N/A Details Diff	debug: Print open flags as %08o and mode as %04o In debug message printing open flags as octal and 4 bytes long delivers much better readable stuff as you get things line 0640 which is sane as that is also something you can give to lets say chmod. Open flags like O_CREAT, O_READONLY etc are also defined as octal values of 8 bytes so lets also print them so you can easily grep them from the proper include files and they look somewhat familiar. Before those were sometimes printed as decimal and sometimes as hexadecimal so that always leads to converting.
	mod - src/filed/accurate.c	Diff File
	mod - src/filed/fd_plugins.c	Diff File
	mod - src/findlib/bfile.c	Diff File
	mod - src/findlib/create_file.c	Diff File
	mod - src/plugins/filed/python-fd.c	Diff File
	mod - src/stored/dev.c	Diff File
bareos-16.2-droplet 2132a59d 2017-06-28 19:41 mvwieringen Ported: N/A Details Diff	debug: Print open flags as %08o and mode as %04o In debug message printing open flags as octal and 4 bytes long delivers much better readable stuff as you get things line 0640 which is sane as that is also something you can give to lets say chmod. Open flags like O_CREAT, O_READONLY etc are also defined as octal values of 8 bytes so lets also print them so you can easily grep them from the proper include files and they look somewhat familiar. Before those were sometimes printed as decimal and sometimes as hexadecimal so that always leads to converting.
	mod - src/filed/accurate.c	Diff File
	mod - src/filed/fd_plugins.c	Diff File
	mod - src/findlib/bfile.c	Diff File
	mod - src/findlib/create_file.c	Diff File
	mod - src/plugins/filed/python-fd.c	Diff File
	mod - src/stored/dev.c	Diff File
master 8a7d72d8 2017-06-27 14:56 joergs Ported: N/A Details Diff	update_bareos_tables: show a warning if updating to 2170 With DB version 2170, the FilenameId has been merged into the File table. The result is a lot faster, however require more disk space. During the migration, it can require twice the normal database disk space. This commit adds a warning about this to the update_bareos_tables script.
	mod - src/cats/update_bareos_tables.in	Diff File
master 2c99f9a3 2017-06-26 17:06 joergs Ported: N/A Details Diff	tweak: remove compile warning
	mod - src/stored/record.c	Diff File

Reporting new Issues is disabled, please Report new Issues at https://github.com/bareos/bareos/issues

Changesets: bareos