0000632: fd-bpipe plugin merges stderr with stdout, which can result in corrupted backups. - Bareos Bug Tracker

ID	Project	Category	View Status	Date Submitted	Last Update

0000632	bareos-core	file daemon	public	2016-03-11 07:52	2019-12-18 15:25

Reporter	ben.mehlman	Assigned To
Priority	normal	Severity	major	Reproducibility	always
Status	closed	Resolution	fixed
Platform	Linux	OS	any	OS Version	3
Product Version	15.2.2

Summary	0000632: fd-bpipe plugin merges stderr with stdout, which can result in corrupted backups.
Description	In plugins/filed/bpipe-fd.c, pluginIO() calls open_bpipe() function (lib/bpipe.c) to execute the read command. This function dups stderr and stdin. This means that if any errors, warnings or other information is written to stderr it will cause the backup to be corrupted. When one is scripting with various third party programs it is not always predictable when one of them may write something to stderr and corruption could occur without the user ever knowing until it is too late. It also makes debugging of scripts very difficult since when things fail you cannot see why. I suggest that the solution for this is to use a select() or poll() to read separately from stdout and stderr and that anything read from stderr should be logged with Dmsg/Jmsg so that it appears in the job log.
Steps To Reproduce	Use following script as reader= for pipe plugin, and run backup: #!/bin/bash echo "This is STDERR" 1>&2 echo "This is STDOUT" exit 0
Tags	No tags attached.

ben.mehlman 2016-03-11 07:55 reporter ~0002211	I am sorry, I didn't mean to tag this as "major" although it is important to me :) I must have made a mistake with the bug tracker...

pstorz 2016-03-11 09:12 administrator ~0002212	As you seem to have analyzed the problem in depth, we are awaiting a patch to solve this problem. :) Thanks for using bareos

mvwieringen 2016-03-11 10:50 developer ~0002213	The original bpipe-fd used popen() which just ignores stderr. So for now I will push a stop gap patch that restores that old behavior. I the commit I also added a comment on a possible future better solution using an extra FILE named efd next to the rfd and wfd that exist nowadays in the BPIPE class. When the patch gets pushed mantis will set the bug to resolved but as this is only a partial fix we probably should reopen it.

mvwieringen 2016-03-11 12:23 developer ~0002214	Fix committed to bareos bareos-15.2 branch with changesetid 6163.

mvwieringen 2016-03-11 16:22 developer ~0002215	Fix committed to bareos bareos-15.2 branch with changesetid 6164.

mvwieringen 2016-07-08 14:34 developer ~0002310	Fix committed to bareos bareos-14.2 branch with changesetid 6417.

bareos: bareos-15.2 bd780a7f 2016-03-11 10:59 mvwieringen Ported: N/A Details Diff	bpipe-fd: stderr is merged with stdout, possible corruption. When changing from using popen() to open_bpipe() it was overlooked that popen() just ignores stderr but open_bpipe() dups both stdout and stderr onto the read pipe used. This patch adds a flag to open_bpipe() which is by default set to true to dup stderr or not. For now this fixes the problem but a better solution could probably be coded by adding an extra member to the BPIPE class with the name efd which is the error handle just as we now have a write and read handle. Fixes 0000632: fd-bpipe plugin merges stderr with stdout, which can result in corrupted backups.		Affected Issues 0000632
	mod - src/lib/bpipe.c		Diff File
	mod - src/lib/protos.h		Diff File
	mod - src/plugins/filed/bpipe-fd.c		Diff File
bareos: bareos-14.2 747d34cf 2016-03-11 10:59 mvwieringen Ported: N/A Details Diff	bpipe-fd: stderr is merged with stdout, possible corruption. When changing from using popen() to open_bpipe() it was overlooked that popen() just ignores stderr but open_bpipe() dups both stdout and stderr onto the read pipe used. This patch adds a flag to open_bpipe() which is by default set to true to dup stderr or not. For now this fixes the problem but a better solution could probably be coded by adding an extra member to the BPIPE class with the name efd which is the error handle just as we now have a write and read handle. Fixes 0000632: fd-bpipe plugin merges stderr with stdout, which can result in corrupted backups.		Affected Issues 0000632
	mod - src/lib/bpipe.c		Diff File
	mod - src/lib/protos.h		Diff File
	mod - src/plugins/filed/bpipe-fd.c		Diff File
bareos: bareos-15.2 ddb66e4b 2016-03-11 15:31 mvwieringen Ported: N/A Details Diff	win32: Fix open_bpipe() in windows compat emulation. Fixes 0000632: fd-bpipe plugin merges stderr with stdout, which can result in corrupted backups.		Affected Issues 0000632
	mod - src/win32/compat/compat.c		Diff File
bareos: bareos-14.2 6f1ce0dc 2016-03-11 15:31 mvwieringen Ported: N/A Details Diff	win32: Fix open_bpipe() in windows compat emulation. Fixes 0000632: fd-bpipe plugin merges stderr with stdout, which can result in corrupted backups.		Affected Issues 0000632
	mod - src/win32/compat/compat.c		Diff File

Date Modified	Username	Field	Change
2016-03-11 07:52	ben.mehlman	New Issue
2016-03-11 07:55	ben.mehlman	Note Added: 0002211
2016-03-11 09:12	pstorz	Note Added: 0002212
2016-03-11 09:12	pstorz	Status	new => feedback
2016-03-11 10:50	mvwieringen	Note Added: 0002213
2016-03-11 12:23	mvwieringen	Changeset attached	=> bareos bareos-15.2 bd780a7f
2016-03-11 12:23	mvwieringen	Note Added: 0002214
2016-03-11 12:23	mvwieringen	Status	feedback => resolved
2016-03-11 12:23	mvwieringen	Resolution	open => fixed
2016-03-11 14:29	maik	Relationship added	child of 0000625
2016-03-11 16:22	mvwieringen	Changeset attached	=> bareos bareos-15.2 ddb66e4b
2016-03-11 16:22	mvwieringen	Note Added: 0002215
2016-07-05 11:47	joergs	Relationship added	child of 0000584
2016-07-08 14:34	mvwieringen	Changeset attached	=> bareos bareos-14.2 747d34cf
2016-07-08 14:34	mvwieringen	Changeset attached	=> bareos bareos-14.2 6f1ce0dc
2016-07-08 14:34	mvwieringen	Note Added: 0002310
2019-12-18 15:25	arogge	Status	resolved => closed