View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0000586bareos-coreGeneralpublic2015-12-21 10:132017-09-21 22:23
ReporterHauke Wintjen Assigned Tojoergs  
PrioritynormalSeverityminorReproducibilityalways
Status closedResolutionno change required 
Product Version15.2.2 
Summary0000586: Software compression not working for some fileSets
DescriptionWhen i use the director config as attached, all backup jobs for "Linux Homes" report that no SW compression is used, although it is configured to use that.
Steps To ReproduceUse the attached config file and run a job with fileset "Linux Homes" and look into the reporting mail. No SW compression is reported for this. Use a job for "Linux OS Full" and a note appears that SW compression is used.
Additional InformationLinux Homes:
============================================================================
20-Dec 17:31 mediasrv-dir JobId 54: No prior Full backup Job record found.
20-Dec 17:31 mediasrv-dir JobId 54: No prior or suitable Full backup found in catalog. Doing FULL backup.
20-Dec 18:15 mediasrv-dir JobId 54: Start Backup JobId 54, Job=Backup_Linux_Homes_HWintjen.2015-12-20_17.31.20_31
20-Dec 18:15 mediasrv-dir JobId 54: Using Device "FileStorage" to write.
20-Dec 18:15 mediasrv-sd JobId 54: Volume "Full-0008" previously written, moving to end of data.
20-Dec 18:15 mediasrv-sd JobId 54: Ready to append to end of Volume "Full-0008" size=21135254409
20-Dec 18:22 hwintjen-fd JobId 54: Could not stat "/home/katja/.gvfs": ERR=Keine Berechtigung
20-Dec 19:20 mediasrv-sd JobId 54: Elapsed time=01:04:44, Transfer rate=2.230 M Bytes/second
20-Dec 19:20 mediasrv-dir JobId 54: Bareos mediasrv-dir 15.2.2 (16Nov15):
  Build OS: x86_64-pc-linux-gnu ubuntu Ubuntu 12.04 LTS
  JobId: 54
  Job: Backup_Linux_Homes_HWintjen.2015-12-20_17.31.20_31
  Backup Level: Full (upgraded from Incremental)
  Client: "hwintjen-fd" 15.2.2 (16Nov15) i686-pc-linux-gnu,ubuntu,Ubuntu 12.04 LTS,xUbuntu_12.04,i586
  FileSet: "Linux Homes" 2015-12-14 09:28:51
  Pool: "Full" (From Job FullPool override)
  Catalog: "MyCatalog" (From Client resource)
  Storage: "File" (From Job resource)
  Scheduled time: 20-Dec-2015 17:31:20
  Start time: 20-Dec-2015 18:15:20
  End time: 20-Dec-2015 19:20:13
  Elapsed time: 1 hour 4 mins 53 secs
  Priority: 10
  FD Files Written: 56,014
  SD Files Written: 56,014
  FD Bytes Written: 8,654,211,608 (8.654 GB)
  SD Bytes Written: 8,663,247,598 (8.663 GB)
  Rate: 2223.0 KB/s
  Software Compression: None
  VSS: no
  Encryption: no
  Accurate: no
  Volume name(s): Full-0008
  Volume Session Id: 37
  Volume Session Time: 1450166763
  Last Volume Bytes: 29,806,084,481 (29.80 GB)
  Non-fatal FD errors: 1
  SD Errors: 0
  FD termination status: OK
  SD termination status: OK
  Termination: Backup OK -- with warnings


=======================================================================
Linux OS Full:

=======================================================================
20-Dec 17:27 mediasrv-dir JobId 52: Start Backup JobId 52, Job=Backup_Linux_OS_HWintjen.2015-12-20_17.27.52_20
20-Dec 17:27 mediasrv-dir JobId 52: Using Device "FileStorage" to write.
20-Dec 17:27 mediasrv-sd JobId 52: Volume "Full-0008" previously written, moving to end of data.
20-Dec 17:27 mediasrv-sd JobId 52: Ready to append to end of Volume "Full-0008" size=18127391809
20-Dec 17:27 hwintjen-fd JobId 52: /dev is a different filesystem. Will not descend from / into it.
20-Dec 17:32 hwintjen-fd JobId 52: /run is a different filesystem. Will not descend from / into it.
20-Dec 17:32 hwintjen-fd JobId 52: /sys is a different filesystem. Will not descend from / into it.
20-Dec 17:32 hwintjen-fd JobId 52: /var is a different filesystem. Will not descend from / into it.
20-Dec 18:09 hwintjen-fd JobId 52: /home is a different filesystem. Will not descend from / into it.
20-Dec 18:10 hwintjen-fd JobId 52: /dos_c is a different filesystem. Will not descend from / into it.
20-Dec 18:11 mediasrv-sd JobId 52: Elapsed time=00:43:15, Transfer rate=1.153 M Bytes/second
20-Dec 18:12 mediasrv-dir JobId 52: Bareos mediasrv-dir 15.2.2 (16Nov15):
  Build OS: x86_64-pc-linux-gnu ubuntu Ubuntu 12.04 LTS
  JobId: 52
  Job: Backup_Linux_OS_HWintjen.2015-12-20_17.27.52_20
  Backup Level: Full
  Client: "hwintjen-fd" 15.2.2 (16Nov15) i686-pc-linux-gnu,ubuntu,Ubuntu 12.04 LTS,xUbuntu_12.04,i586
  FileSet: "Linux All OS" 2015-12-14 19:47:15
  Pool: "Full" (From command line)
  Catalog: "MyCatalog" (From Client resource)
  Storage: "File" (From Job resource)
  Scheduled time: 20-Dec-2015 17:27:52
  Start time: 20-Dec-2015 17:27:54
  End time: 20-Dec-2015 18:12:01
  Elapsed time: 44 mins 7 secs
  Priority: 10
  FD Files Written: 438,464
  SD Files Written: 438,464
  FD Bytes Written: 2,928,282,332 (2.928 GB)
  SD Bytes Written: 2,993,561,480 (2.993 GB)
  Rate: 1106.3 KB/s
  Software Compression: 56.3 % (gzip)
  VSS: no
  Encryption: no
  Accurate: no
  Volume name(s): Full-0008
  Volume Session Id: 35
  Volume Session Time: 1450166763
  Last Volume Bytes: 21,135,254,409 (21.13 GB)
  Non-fatal FD errors: 0
  SD Errors: 0
  FD termination status: OK
  SD termination status: OK
  Termination: Backup OK
========================================================================
TagsNo tags attached.

Activities

Hauke Wintjen

Hauke Wintjen

2015-12-21 10:13

reporter  

bareos-dir.conf (13,837 bytes)
joergs

joergs

2015-12-21 15:03

developer   ~0002051

maybe because you have two options blocks defined in "Linux Homes"?

FileSet {
  Name = "Linux Homes"
  Include {
    Options {
      Signature = MD5 # calculate md5 checksum per file
      One FS = Yes # change into other filessytems
      FS Type = ext2 # filesystems of given types will be backed up
      FS Type = ext3 # others will be ignored
      FS Type = ext4
      sparse=yes
      verify=pinugs5
      compression=GZIP
    }
    options {
       RegexDir = "^/home/.*/tmp"
       RegexDir = "^/home/.*/\\..*cache"
       Exclude = yes
    }
Hauke Wintjen

Hauke Wintjen

2015-12-21 16:12

reporter   ~0002053

Yes thats true. I was following an example from the docs:
================================================
FileSet {
  Name = "Full Set"
  Include {
     Options {
        wilddir = "/home/a*"
        wilddir = "/home/b*"
     }
     Options {
        RegexDir = ".*"
        exclude = yes
     }
     File = /home
  }
}
 

Configuration 9.25: Include and Exclude
================================================
See also: Configuration 9.26: Include/Exclude example

So the docu is wrong here?
Or how do i exclude some RegexDirs from a set of files?
joergs

joergs

2015-12-22 18:38

developer   ~0002057

It is possible to use multiple option blocks.

I reproduced it. You can work around if you specifiy "compression=GZIP" also in the second options block.
It is even sufficient, to specify it only in the second options block.

Thats not pretty, and not what a user expected, but at least this should work as a workaround.
joergs

joergs

2015-12-23 12:06

developer   ~0002059

I discussed this with other Bareos developers. In principle, it should be possible (and have worked in the past) to define different compression algorithms per Options block (e.g. have html files compressed with GZIP, store mp3 files uncompressed and doc files by lz4).
joergs

joergs

2017-09-21 22:23

developer   ~0002749

Generally, multiple include blocks with different options can be used.
I added a regression test for it:

https://github.com/bareos/bareos-regress/blob/master/tests/fileset-multiple-include-blocks
and
https://github.com/bareos/bareos-regress/blob/master/configs/fileset-multiple-include-blocks/bareos-dir.d/fileset/FS_TESTJOB.conf.in

I also added a test/example for using multiple option blocks:

https://github.com/bareos/bareos-regress/blob/master/tests/fileset-multiple-options-blocks
and
https://github.com/bareos/bareos-regress/blob/master/configs/fileset-multiple-options-blocks/bareos-dir.d/fileset/FS_TESTJOB.conf.in

Issue History

Date Modified Username Field Change
2015-12-21 10:13 Hauke Wintjen New Issue
2015-12-21 10:13 Hauke Wintjen File Added: bareos-dir.conf
2015-12-21 15:03 joergs Note Added: 0002051
2015-12-21 15:03 joergs Status new => feedback
2015-12-21 16:12 Hauke Wintjen Note Added: 0002053
2015-12-21 16:12 Hauke Wintjen Status feedback => new
2015-12-22 09:58 joergs Assigned To => joergs
2015-12-22 09:58 joergs Status new => assigned
2015-12-22 18:38 joergs Note Added: 0002057
2015-12-22 18:39 joergs Priority normal => low
2015-12-22 18:39 joergs Severity major => minor
2015-12-23 12:06 joergs Note Added: 0002059
2015-12-23 12:06 joergs Priority low => normal
2017-09-21 22:23 joergs Note Added: 0002749
2017-09-21 22:23 joergs Status assigned => resolved
2017-09-21 22:23 joergs Resolution open => no change required
2017-09-21 22:23 joergs Status resolved => closed