View Issue Details

IDProjectCategoryView StatusLast Update
0000632bareos-core[All Projects] file daemonpublic2016-07-08 14:34
Reporterben.mehlmanAssigned To 
PrioritynormalSeveritymajorReproducibilityalways
Status resolvedResolutionfixed 
PlatformLinuxOSanyOS Version3
Product Version15.2.2 
Fixed in Version 
Summary0000632: fd-bpipe plugin merges stderr with stdout, which can result in corrupted backups.
DescriptionIn plugins/filed/bpipe-fd.c, pluginIO() calls open_bpipe() function (lib/bpipe.c) to execute the read command. This function dups stderr and stdin.

This means that if any errors, warnings or other information is written to stderr it will cause the backup to be corrupted. When one is scripting with various third party programs it is not always predictable when one of them may write something to stderr and corruption could occur without the user ever knowing until it is too late.

It also makes debugging of scripts very difficult since when things fail you cannot see why.

I suggest that the solution for this is to use a select() or poll() to read separately from stdout and stderr and that anything read from stderr should be logged with Dmsg/Jmsg so that it appears in the job log.
Steps To ReproduceUse following script as reader= for pipe plugin, and run backup:

#!/bin/bash
echo "This is STDERR" 1>&2
echo "This is STDOUT"
exit 0
TagsNo tags attached.
bareos-master: impactyes
bareos-master: actionfixed
bareos-19.2: impact
bareos-19.2: action
bareos-18.2: impact
bareos-18.2: action
bareos-17.2: impact
bareos-17.2: action
bareos-16.2: impact
bareos-16.2: action
bareos-15.2: impactyes
bareos-15.2: actionfixed
bareos-14.2: impactyes
bareos-14.2: actionfixed
bareos-13.2: impactyes
bareos-13.2: actionwill care
bareos-12.4: impactno
bareos-12.4: action

Relationships

child of 0000625 closedmaik Release bareos-15.2.3 
child of 0000584 closedjoergs Release bareos-14.2.7 

Activities

ben.mehlman

ben.mehlman

2016-03-11 07:55

reporter   ~0002211

I am sorry, I didn't mean to tag this as "major" although it is important to me :) I must have made a mistake with the bug tracker...
pstorz

pstorz

2016-03-11 09:12

administrator   ~0002212

As you seem to have analyzed the problem in depth, we are awaiting a patch to solve this problem. :)

Thanks for using bareos

 
mvwieringen

mvwieringen

2016-03-11 10:50

developer   ~0002213

The original bpipe-fd used popen() which just ignores stderr.
So for now I will push a stop gap patch that restores that
old behavior. I the commit I also added a comment on a possible
future better solution using an extra FILE named efd next to the
rfd and wfd that exist nowadays in the BPIPE class.

When the patch gets pushed mantis will set the bug to resolved but
as this is only a partial fix we probably should reopen it.
mvwieringen

mvwieringen

2016-03-11 12:23

developer   ~0002214

Fix committed to bareos bareos-15.2 branch with changesetid 6163.
mvwieringen

mvwieringen

2016-03-11 16:22

developer   ~0002215

Fix committed to bareos bareos-15.2 branch with changesetid 6164.
mvwieringen

mvwieringen

2016-07-08 14:34

developer   ~0002310

Fix committed to bareos bareos-14.2 branch with changesetid 6417.

Related Changesets

bareos: bareos-15.2 bd780a7f

2016-03-11 09:59:50

mvwieringen

Ported: N/A

Details Diff
bpipe-fd: stderr is merged with stdout, possible corruption.

When changing from using popen() to open_bpipe() it was overlooked that
popen() just ignores stderr but open_bpipe() dups both stdout and stderr
onto the read pipe used. This patch adds a flag to open_bpipe() which is
by default set to true to dup stderr or not. For now this fixes the
problem but a better solution could probably be coded by adding an extra
member to the BPIPE class with the name efd which is the error handle
just as we now have a write and read handle.

Fixes 0000632: fd-bpipe plugin merges stderr with stdout, which can result
in corrupted backups.
Affected Issues
0000632
mod - src/lib/bpipe.c Diff File
mod - src/lib/protos.h Diff File
mod - src/plugins/filed/bpipe-fd.c Diff File

bareos: bareos-14.2 747d34cf

2016-03-11 09:59:50

mvwieringen

Ported: N/A

Details Diff
bpipe-fd: stderr is merged with stdout, possible corruption.

When changing from using popen() to open_bpipe() it was overlooked that
popen() just ignores stderr but open_bpipe() dups both stdout and stderr
onto the read pipe used. This patch adds a flag to open_bpipe() which is
by default set to true to dup stderr or not. For now this fixes the
problem but a better solution could probably be coded by adding an extra
member to the BPIPE class with the name efd which is the error handle
just as we now have a write and read handle.

Fixes 0000632: fd-bpipe plugin merges stderr with stdout, which can result
in corrupted backups.
Affected Issues
0000632
mod - src/lib/bpipe.c Diff File
mod - src/lib/protos.h Diff File
mod - src/plugins/filed/bpipe-fd.c Diff File

bareos: bareos-15.2 ddb66e4b

2016-03-11 14:31:10

mvwieringen

Ported: N/A

Details Diff
win32: Fix open_bpipe() in windows compat emulation.

Fixes 0000632: fd-bpipe plugin merges stderr with stdout, which can result
in corrupted backups.
Affected Issues
0000632
mod - src/win32/compat/compat.c Diff File

bareos: bareos-14.2 6f1ce0dc

2016-03-11 14:31:10

mvwieringen

Ported: N/A

Details Diff
win32: Fix open_bpipe() in windows compat emulation.

Fixes 0000632: fd-bpipe plugin merges stderr with stdout, which can result
in corrupted backups.
Affected Issues
0000632
mod - src/win32/compat/compat.c Diff File

Issue History

Date Modified Username Field Change
2016-03-11 07:52 ben.mehlman New Issue
2016-03-11 07:55 ben.mehlman Note Added: 0002211
2016-03-11 09:12 pstorz Note Added: 0002212
2016-03-11 09:12 pstorz Status new => feedback
2016-03-11 10:46 mvwieringen bareos-master: impact => yes
2016-03-11 10:46 mvwieringen bareos-master: action => will care
2016-03-11 10:46 mvwieringen bareos-15.2: impact => yes
2016-03-11 10:46 mvwieringen bareos-15.2: action => will care
2016-03-11 10:46 mvwieringen bareos-14.2: impact => yes
2016-03-11 10:46 mvwieringen bareos-14.2: action => will care
2016-03-11 10:46 mvwieringen bareos-13.2: impact => yes
2016-03-11 10:46 mvwieringen bareos-13.2: action => will care
2016-03-11 10:46 mvwieringen bareos-12.4: impact => no
2016-03-11 10:50 mvwieringen Note Added: 0002213
2016-03-11 12:23 mvwieringen Changeset attached => bareos bareos-15.2 bd780a7f
2016-03-11 12:23 mvwieringen Note Added: 0002214
2016-03-11 12:23 mvwieringen Status feedback => resolved
2016-03-11 12:23 mvwieringen Resolution open => fixed
2016-03-11 14:29 maik Relationship added child of 0000625
2016-03-11 16:22 mvwieringen Changeset attached => bareos bareos-15.2 ddb66e4b
2016-03-11 16:22 mvwieringen Note Added: 0002215
2016-07-05 11:47 joergs Relationship added child of 0000584
2016-07-05 11:48 joergs bareos-master: action will care => fixed
2016-07-05 11:48 joergs bareos-15.2: action will care => fixed
2016-07-05 11:48 joergs bareos-14.2: action will care => fixed
2016-07-08 14:34 mvwieringen Changeset attached => bareos bareos-14.2 747d34cf
2016-07-08 14:34 mvwieringen Changeset attached => bareos bareos-14.2 6f1ce0dc
2016-07-08 14:34 mvwieringen Note Added: 0002310