View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0000694bareos-corestorage daemonpublic2016-09-04 11:362024-03-20 13:47
Reportertigerfoot Assigned Topstorz  
PrioritynormalSeveritymajorReproducibilityalways
Status closedResolutionfixed 
PlatformLinuxOSCentOSOS Version7
Product Version15.2.5 
Summary0000694: Backup sparse file on storage with autoxflate lead to unrecoverable data
DescriptionGlobal scenario :
The idea is backuping virtual machine disk storage file (kvm/virsh/qemu qcow or raw img) to a storage which will have autoxflate plugin activated (lost's of ram, spool place, and cpu power) when the client hypervisor is already overloaded.
For each backup the VM will be shutdown and then the xml definition file and the virtual disk image is backup.
For the backup everything seems to be ok ( Backup status is ok ) but on restore we found having report (see attached) with lot of errors similar to
31-août 17:06 test-infra-bareos-sd JobId 32: Error: Compressed header version error. version=0x344c
31-août 17:06 test-infra-bareos-sd JobId 32: Error: Compressed header version error. version=0x344c
31-août 17:06 test-infra-bareos-sd JobId 32: Error: Compressed header version error. version=0x344c

My understanding is that using sparse = yes in fileset (which is true in that perticular type of files) desactivate compression on bareos-fd if compression is set to protect the sparse attribute.
If bareos-sd daemon is transparent on compression (no autoxflate on device) then the backup and the restore works as expected.

It seems we missed with bareos a regression test about sparse + sd with autoxflate.

See below the tests made
Steps To ReproduceCreate a real sparse file
With fileset Sparse run a backup on a device with autoxflate activated.
Backup will say OK
Run a restore of this fileset on same client
Restore will end on Errors

See files attached.
TagsNo tags attached.

Activities

tigerfoot

tigerfoot

2016-09-04 11:36

developer  

Restore_Error_With_Daemon_Errors_interleaved.log.xz (2,188 bytes)
tigerfoot

tigerfoot

2016-09-04 11:37

developer  

sparse_file_help.sh (2,622 bytes)    
#!/usr/bin/env bash

# This small helper file allow you to create, make content and play
# with sparse file
# Part of Bareos Training by Ioda-Net Sàrl
#

BASE="`pwd`"

if [ $UID -ne 0 ];then
  echo "sorry need to be run as root"
  return 1
fi

# Script to create sparse file example with a bit of contain in it
F="/data/samples/sparse_file.img"
M="/tmp/tmpSPARSE"
if [ ! -d ${M} ];then
  mkdir ${M}
fi

function sparse_report_size {
  du -h --apparent-size ${F}
  du -h ${F}
  if [ $(mount | grep ${M} | wc -l) -eq 1 ]; then
    df -h ${M}
  fi
}

function sparse_cleanup {
  _sparse_umount
  rm -fr ${M}
  rm -f ${F}
}

function sparse_create_512 {
  sparse_cleanup
  mkdir ${M}
  truncate -s 512M ${F}
  mkfs.ext4 -L SPARSE_TEST -m 5 ${F}
}


function sparse_resize_1G {
  dd if=/dev/zero of=${F} bs=1 count=0 seek=1G
  e2fsck -f ${F}
  resize2fs -p ${F}
  sparse_report_size
}

function sparse_resize_4G {
  dd if=/dev/zero of=${F} bs=1 count=0 seek=4G
  e2fsck -f ${F}
  resize2fs -p ${F}
  sparse_report_size
}


function _sparse_mount {
  mount -o loop ${F} ${M}
  if [ $? -ne 0 ];then
    echo "Error during mount loop"
    return 1
  fi

  chown admin:users -R ${M}
  cd ${M}

}

function _sparse_umount {
  cd ${BASE}
  if [ -f ${F} ];then
    if [ $(mount | grep ${M} | wc -l) -eq 1 ]; then
      umount ${F}
      if [ $? -ne 0 ];then
        echo "Error during umount loop"
        lsof ${F}
        return 1
      fi
    fi
  fi
}

function sparse_create_content {
  _sparse_mount
  mkdir folder
  for f in {1..5}; do touch folder/file${f}; done
  echo "This is a test to see if it works..." >> folder/file1
  dd if=/dev/urandom of=folder/random4M.dat bs=4k count=1000
  sync
  ls -Rlh ${M}/*
  _sparse_umount
  sparse_report_size
}

function sparse_create_content_more {
  _sparse_mount
  mkdir resized
  for f in {1..5}; do touch resized/file${f}; done
  echo "This is a test to see if resized works..." >> resized/file1
  dd if=/dev/urandom of=resized/random41M.dat bs=4k count=10000
  sync
  ls -Rlh ${M}/*
  _sparse_umount
  sparse_report_size
}

function sparse_create_content_ultimate {
  _sparse_mount
  mkdir resized
  for f in {10..50}; do echo "$f" > resized/file${f}; done
  echo "This is a test to see if resized works..." >> resized/file1
  dd if=/dev/urandom of=resized/random410M.dat bs=4k count=100000
  sync

  ls -Rlh ${M}/*
  _sparse_umount
  sparse_report_size
}


echo "usage : source $0"
echo "Available functions : sparse_create_512 sparse_create_content sparse_resize_1G sparse_resize_4G sparse_create_content_more sparse_create_content_ultimate sparse_report_size (sparse_cleanup)"
sparse_file_help.sh (2,622 bytes)   
tigerfoot

tigerfoot

2016-09-04 11:38

developer   ~0002343

Last edited: 2016-09-04 11:49

 With sparse_file_help.sh create a sparse file to be backuped.
create content inside.

Use Default.conf in bareos-sd.d/device/ as autoxflate device

Use Sparse.conf in bareos-dir.d/fileset as the file to be saved.
Adapt path to where sparse is created

You can also add a sparse.xml (emulating a virsh vm definition)


You should got a backup ok status (like attache mail-backup.log)

and if you try a restore you should have errors about head compression
tigerfoot

tigerfoot

2016-09-04 11:45

developer  

Default.conf (651 bytes)    
Device {
  Name = Default
  Description = "Default File device. A connecting Director must have the same Name and MediaType."
  Media Type = default
  Archive Device = /var/lib/bareos/storage/default    # No final / you broke bls otherwise
  LabelMedia = yes;                   # lets Bareos label unlabeled media
  Random Access = yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;

  Maximum Spool Size = 18G
  Maximum Job Spool Size = 16G
  Spool Directory = /var/spool/bareos

  Auto Deflate = both
  Auto Deflate Algorithm = LZ4HC
  Auto Deflate Level = 2
  Auto Inflate = both

}
Default.conf (651 bytes)   
tigerfoot

tigerfoot

2016-09-04 11:45

developer  

Sparse.conf (1,161 bytes)    
# Filset to play with sparse files
FileSet {
    Name = "Sparse"
    Description = "Backup Sparse File and non sparse file"
    Include {
        # Specific options for qcow images
      Options {
        Signature = SHA1                                        # Beware the sha1 for a sparse file will never be the one on disk.
        Compression = LZFAST
        OneFS = yes
        Portable = yes
        IgnoreCase = no
        Noatime = yes
        AclSupport = no
        xattrSupport = no
        Verify = sm1
        Accurate = mcspug
        sparse = yes                        # Default no, don't use it on non sparse file
                                            # But use it when needed (for example specific known sparse file)
        Exclude = no
        WildFile = "*sparse*"
# By scripting
#        WildFile = "|echo *%n*"
      }
      Options {                             # We exclude all the rest !
        Exclude = yes
        WildDir = "*"                       # What happen if not set ?
        WildFile = "*"
      }

     File = "/data/samples"
# By scripting
#    File = "\\|sh -c \"find /data/samples -iname '*%n*.img'\""

 }
}
Sparse.conf (1,161 bytes)   
tigerfoot

tigerfoot

2016-09-04 11:47

developer  

mail-backup.log (2,493 bytes)    
31-Aug 20:26 earth-dir JobId 1: Using Device "Default" to write.
31-Aug 20:26 earth-sd JobId 1: Labeled new Volume "Full-0001" on device "Default" (/var/lib/bareos/storage/default).
31-Aug 20:26 earth-sd JobId 1: Wrote label to prelabeled Volume "Full-0001" on device "Default" (/var/lib/bareos/storage/default)
31-Aug 20:26 earth-sd JobId 1: autodeflation: Compressor on device Default is LZ4HC
31-Aug 20:26 earth-sd JobId 1: autoxflate-sd.c: Default OUT:[SD->inflate=yes->deflate=yes->DEV] IN:[DEV->inflate=yes->deflate=yes->SD]
31-Aug 20:26 earth-sd JobId 1: Spooling data ...
31-Aug 20:26 earth-sd JobId 1: Committing spooled data to Volume "Full-0001". Despooling 45,150,031 bytes ...
31-Aug 20:26 earth-sd JobId 1: Despooling elapsed time = 00:00:01, Transfer rate = 45.15 M Bytes/second
31-Aug 20:26 earth-sd JobId 1: Elapsed time=00:00:01, Transfer rate=45.10 M Bytes/second
31-Aug 20:26 earth-sd JobId 1: Sending spooled attrs to the Director. Despooling 443 bytes ...
31-Aug 20:26 earth-sd JobId 1: autoxflate-sd.c: deflate ratio: 98.19%
31-Aug 20:26 earth-dir JobId 1: Bareos earth-dir 15.2.2 (16Nov15):
  Build OS:               x86_64-suse-linux-gnu suse openSUSE Leap 42.1 (x86_64)
  JobId:                  1
  Job:                    Default.2016-08-31_20.26.27_04
  Backup Level:           Full
  Client:                 "earth-fd" 15.2.2 (16Nov15) x86_64-suse-linux-gnu,suse,openSUSE Leap 42.1 (x86_64),openSUSE_Leap_42.1,x86_64
  FileSet:                "Sparse" 2016-08-31 20:26:27
  Pool:                   "Full" (From Job FullPool override)
  Catalog:                "MyCatalog" (From Client resource)
  Storage:                "Default" (From Job resource)
  Scheduled time:         31-Aug-2016 20:26:26
  Start time:             31-Aug-2016 20:26:30
  End time:               31-Aug-2016 20:26:31
  Elapsed time:           1 sec
  Priority:               10
  FD Files Written:       2
  SD Files Written:       2
  FD Bytes Written:       45,940,760 (45.94 MB)
  SD Bytes Written:       45,107,637 (45.10 MB)
  Rate:                   45940.8 KB/s
  Software Compression:   None
  VSS:                    no
  Encryption:             no
  Accurate:               yes
  Volume name(s):         Full-0001
  Volume Session Id:      1
  Volume Session Time:    1472667883
  Last Volume Bytes:      45,141,826 (45.14 MB)
  Non-fatal FD errors:    0
  SD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Backup OK
mail-backup.log (2,493 bytes)   
tigerfoot

tigerfoot

2016-09-04 11:47

developer  

mail-restore.log (3,552 bytes)    
From bareos@earth.bareos.local  Wed Aug 31 20:27:14 2016
Return-Path: <bareos@earth.bareos.local>
X-Original-To: bareos@earth.bareos.local
Delivered-To: bareos@earth.bareos.local
Received: from earth (localhost [IPv6:::1])
        by earth.bareos.local (Postfix) with SMTP id EF545A025C
        for <bareos@earth.bareos.local>; Wed, 31 Aug 2016 20:27:13 +0200 (CEST)
From: (Bareos) <bareos@earth.bareos.local>
Subject: Bareos: Restore Restore Error of earth-fd Full
Sender: bareos@earth.bareos.local
To: bareos@earth.bareos.local
Content-Type: text/plain; charset=UTF-8
Date: Wed, 31 Aug 2016 20:27:13 +0200 (CEST)
Message-Id: <20160831182713.EF545A025C@earth.bareos.local>

31-Aug 20:27 earth-dir JobId 2: Start Restore Job Restore.2016-08-31_20.27.11_05
31-Aug 20:27 earth-dir JobId 2: Using Device "Default" to read.
31-Aug 20:27 earth-sd JobId 2: Ready to read from volume "Full-0001" on device "Default" (/var/lib/bareos/storage/default).
31-Aug 20:27 earth-fd JobId 2: -rw-r--r--   1 root     root      1073741824 2016-08-31 20:22:13  /var/tmp/restores/data/samples/sparse_file.img
31-Aug 20:27 earth-sd JobId 2: autodeflation: Compressor on device Default is LZ4HC
31-Aug 20:27 earth-sd JobId 2: autoxflate-sd.c: Default OUT:[SD->inflate=yes->deflate=yes->DEV] IN:[DEV->inflate=yes->deflate=yes->SD]
31-Aug 20:27 earth-sd JobId 2: Forward spacing Volume "Full-0001" to file:block 0:195.
31-Aug 20:27 earth-fd JobId 2: Error: attribs.c:419 File size of restored file /var/tmp/restores/data/samples/sparse_file.img not correct. Original 1073741824, restored 1073741832.
31-Aug 20:27 earth-sd JobId 2: Error: Compressed header version error. version=0x3448
31-Aug 20:27 earth-sd JobId 2: Error: Compressed header version error. version=0x3448
31-Aug 20:27 earth-sd JobId 2: Error: Compressed header version error. version=0x3448
31-Aug 20:27 earth-sd JobId 2: Error: Compressed header version error. version=0x3448
31-Aug 20:27 earth-sd JobId 2: Error: Compressed header version error. version=0x3448
31-Aug 20:27 earth-sd JobId 2: Error: Compressed header version error. version=0x3448
31-Aug 20:27 earth-sd JobId 2: Error: Compressed header version error. version=0x3448
31-Aug 20:27 earth-sd JobId 2: Error: Compressed header version error. version=0x3448
31-Aug 20:27 earth-sd JobId 2: Error: Compressed header version error. version=0x3448
31-Aug 20:27 earth-sd JobId 2: Error: Compressed header version error. version=0x3448
31-Aug 20:27 earth-sd JobId 2: Error: Compressed header version error. version=0x3448
31-Aug 20:27 earth-sd JobId 2: Error: Compressed header version error. version=0x3448
31-Aug 20:27 earth-sd JobId 2: Error: Compressed header version error. version=0x3448
31-Aug 20:27 earth-sd JobId 2: Error: Compressed header version error. version=0x3448
31-Aug 20:27 earth-sd JobId 2: Error: Compressed header version error. version=0x3448
... x 11000 lines
31-Aug 20:27 earth-dir JobId 2: Error: Bareos earth-dir 15.2.2 (16Nov15):
  Build OS:               x86_64-suse-linux-gnu suse openSUSE Leap 42.1 (x86_64)
  JobId:                  2
  Job:                    Restore.2016-08-31_20.27.11_05
  Restore Client:         earth-fd
  Start time:             31-Aug-2016 20:27:13
  End time:               31-Aug-2016 20:27:13
  Elapsed time:           0 secs
  Files Expected:         1
  Files Restored:         1
  Bytes Restored:         45,940,760
  Rate:                   0.0 KB/s
  FD Errors:              1
  FD termination status:  Error
  SD termination status:  OK
  Termination:            *** Restore Error ***
mail-restore.log (3,552 bytes)   
tigerfoot

tigerfoot

2016-09-04 11:54

developer   ~0002344

On site, we discover that xml was not that destroyed (contain a number of null at the begining of the file, able to be manually edited)
and also the virtual disk image was not that corrupted a virtual machine can use it.
But hash signature between the source and the restored version was different, and this is not what we want :-)

From my understanding the sparse protection mechanism available in the bareos-fd should inform the storage or something like that.

Finally implementing fileset with no compression and no autoxflate device allow to do the job of backup and restore correctly. But we have to have a protection to not allow this auto-goal shooting situation.
pstorz

pstorz

2016-09-06 13:01

administrator   ~0002347

Ok, could reproduce the probelm
pstorz

pstorz

2016-09-07 11:07

administrator   ~0002348

Hello Bruno,

I have just sent you directly the patch and a autoxflate-sd.so binary via direct mail.

Please verify if the problem is solved and give feedback.

Thank you very much
tigerfoot

tigerfoot

2016-09-13 08:40

developer   ~0002351

Hi Philipp,

I tried on my local openSUSE Leap 42.1 64 bits 15.2.4 but I
still see errors during the restore after a new successful (no errors or warnings) backup on a new media.

earth-sd JobId 45: Error: Compressed header version error. version=0x3448

If I let the compression on the Fileset I also get
08-Sep 11:05 earth-sd JobId 43: Error: Compressed header version error.
version=0x465a
cause in that case inflate and deflate are used.
maik

maik

2017-02-10 10:42

administrator   ~0002557

Eventually also related to this thread about restore problems with the Percona plugin:

https://groups.google.com/forum/?fromgroups#!topic/bareos-users/4cQoJ_d8HtA
maik

maik

2017-02-13 11:00

administrator   ~0002559

Reproduced the same behavior with Percona plugin, sparse=yes, Error on restore as described in the thread mentioned above.
stephand

stephand

2017-02-13 18:51

developer   ~0002560

I've run some tests.
The combination of sparse=yes and autoxflate is broken, restore fails even with a small, not sparse testfile without any zeroes. On Restore getting

Error: Compressed header version error. version=0x3448

And the restore result has 8 extra zeroes prepended:
# ls -la /data/testfile.bin /tmp/bareos-restores/data/testfile.bin ; md5sum /data/testfile.bin /tmp/bareos-restores/data/testfile.bin; hexdump -C /data/testfile.bin ; hexdump -C /tmp/bareos-restores/data/testfile.bin
-rw-r--r--. 1 root root 9 Feb 13 12:19 /data/testfile.bin
-rw-r--r--. 1 root root 17 Feb 13 12:19 /tmp/bareos-restores/data/testfile.bin
d4907025e168b3929e89854381fcd449 /data/testfile.bin
74dfbc4dee929c853fe01bb130762c08 /tmp/bareos-restores/data/testfile.bin
00000000 54 65 73 74 46 69 6c 65 0a |TestFile.|
00000009
00000000 00 00 00 00 00 00 00 00 54 65 73 74 46 69 6c 65 |........TestFile|
00000010 0a |.|
00000011

So logically, this is implicitly broken when used with Python Plugins, too.

Python Plugins however also have an issue with sparse=yes, but that only hits when the data stream contains blocks of zeroes larger than 64K, but that's a different problem. If possible, a Python plugin that can't handle sparse should detect this and fail the job with an appropriate error.

The combination of autoxflate an Python Plugins _without_ sparse=yes works correctly.
tigerfoot

tigerfoot

2017-02-15 15:23

developer  

bareos_patches_first_results.txt (41,343 bytes)    
Modification done step by step on the sparse file

source sparse_file_help.sh
usage : source -bash
Available functions : sparse_create_512 sparse_create_content sparse_resize_1G sparse_resize_4G sparse_create_content_more sparse_create_content_ultimate sparse_report_size (sparse_cleanup)
earth:/data/Templates.Configurations/scripts # sparse_create_512
mke2fs 1.42.11 (09-Jul-2014)
Discarding device blocks: done
Creating filesystem with 131072 4k blocks and 32768 inodes
Filesystem UUID: 1a32cb7d-3592-4707-a1e1-05dba008db97
Superblock backups stored on blocks:
        32768, 98304

Allocating group tables: done
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done

earth:/data/Templates.Configurations/scripts # sparse_create_content
1000+0 records in
1000+0 records out
4096000 bytes (4.1 MB) copied, 0.323538 s, 12.7 MB/s
/tmp/tmpSPARSE/folder:
total 4.0M
-rw-r--r-- 1 root root   37 fév 15 14:00 file1
-rw-r--r-- 1 root root    0 fév 15 14:00 file2
-rw-r--r-- 1 root root    0 fév 15 14:00 file3
-rw-r--r-- 1 root root    0 fév 15 14:00 file4
-rw-r--r-- 1 root root    0 fév 15 14:00 file5
-rw-r--r-- 1 root root 4.0M fév 15 14:00 random4M.dat

/tmp/tmpSPARSE/lost+found:
total 0
512M    /data/samples/sparse_file.img
21M     /data/samples/sparse_file.img
earth:/data/Templates.Configurations/scripts # sparse_resize_4G
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.000101264 s, 0.0 kB/s
e2fsck 1.42.11 (09-Jul-2014)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
SPARSE_TEST: 18/32768 files (0.0% non-contiguous), 7259/131072 blocks
resize2fs 1.42.11 (09-Jul-2014)
Resizing the filesystem on /data/samples/sparse_file.img to 1048576 (4k) blocks.
The filesystem on /data/samples/sparse_file.img is now 1048576 blocks long.

4.0G    /data/samples/sparse_file.img
21M     /data/samples/sparse_file.img
earth:/data/Templates.Configurations/scripts # sparse_create_content_more
10000+0 records in
10000+0 records out
40960000 bytes (41 MB) copied, 3.36789 s, 12.2 MB/s
/tmp/tmpSPARSE/folder:
total 4.0M
-rw-r--r-- 1 admin users   37 fév 15 14:00 file1
-rw-r--r-- 1 admin users    0 fév 15 14:00 file2
-rw-r--r-- 1 admin users    0 fév 15 14:00 file3
-rw-r--r-- 1 admin users    0 fév 15 14:00 file4
-rw-r--r-- 1 admin users    0 fév 15 14:00 file5
-rw-r--r-- 1 admin users 4.0M fév 15 14:00 random4M.dat

/tmp/tmpSPARSE/lost+found:
total 0

/tmp/tmpSPARSE/resized:
total 40M
-rw-r--r-- 1 root root  42 fév 15 14:01 file1
-rw-r--r-- 1 root root   0 fév 15 14:01 file2
-rw-r--r-- 1 root root   0 fév 15 14:01 file3
-rw-r--r-- 1 root root   0 fév 15 14:01 file4
-rw-r--r-- 1 root root   0 fév 15 14:01 file5
-rw-r--r-- 1 root root 40M fév 15 14:02 random41M.dat
4.0G    /data/samples/sparse_file.img
60M     /data/samples/sparse_file.img
earth:/data/Templates.Configurations/scripts # sp
sparse_cleanup                  sparse_create_content_more      sparse_resize_1G                spawn_login                     split-logfile
sparse_create_512               sparse_create_content_ultimate  sparse_resize_4G                splain                          split-logfile2
sparse_create_content           sparse_report_size              spawn_console                   split                           spwd
earth:/data/Templates.Configurations/scripts # sp
sparse_cleanup                  sparse_create_content_more      sparse_resize_1G                spawn_login                     split-logfile
sparse_create_512               sparse_create_content_ultimate  sparse_resize_4G                splain                          split-logfile2
sparse_create_content           sparse_report_size              spawn_console                   split                           spwd
earth:/data/Templates.Configurations/scripts # sparse_report_size
4.0G    /data/samples/sparse_file.img
60M     /data/samples/sparse_file.img
earth:/data/Templates.Configurations/scripts # sparse_create_content_ultimate
mkdir: cannot create directory ‘resized’: File exists
100000+0 records in
100000+0 records out
409600000 bytes (410 MB) copied, 34.0737 s, 12.0 MB/s
/tmp/tmpSPARSE/folder:
total 4.0M
-rw-r--r-- 1 admin users   37 fév 15 14:00 file1
-rw-r--r-- 1 admin users    0 fév 15 14:00 file2
-rw-r--r-- 1 admin users    0 fév 15 14:00 file3
-rw-r--r-- 1 admin users    0 fév 15 14:00 file4
-rw-r--r-- 1 admin users    0 fév 15 14:00 file5
-rw-r--r-- 1 admin users 4.0M fév 15 14:00 random4M.dat

/tmp/tmpSPARSE/lost+found:
total 0

/tmp/tmpSPARSE/resized:
total 430M
-rw-r--r-- 1 admin users   84 fév 15 14:03 file1
-rw-r--r-- 1 root  root     3 fév 15 14:03 file10
-rw-r--r-- 1 root  root     3 fév 15 14:03 file11
-rw-r--r-- 1 root  root     3 fév 15 14:03 file12
-rw-r--r-- 1 root  root     3 fév 15 14:03 file13
-rw-r--r-- 1 root  root     3 fév 15 14:03 file14
-rw-r--r-- 1 root  root     3 fév 15 14:03 file15
-rw-r--r-- 1 root  root     3 fév 15 14:03 file16
-rw-r--r-- 1 root  root     3 fév 15 14:03 file17
-rw-r--r-- 1 root  root     3 fév 15 14:03 file18
-rw-r--r-- 1 root  root     3 fév 15 14:03 file19
-rw-r--r-- 1 admin users    0 fév 15 14:01 file2
-rw-r--r-- 1 root  root     3 fév 15 14:03 file20
-rw-r--r-- 1 root  root     3 fév 15 14:03 file21
-rw-r--r-- 1 root  root     3 fév 15 14:03 file22
-rw-r--r-- 1 root  root     3 fév 15 14:03 file23
-rw-r--r-- 1 root  root     3 fév 15 14:03 file24
-rw-r--r-- 1 root  root     3 fév 15 14:03 file25
-rw-r--r-- 1 root  root     3 fév 15 14:03 file26
-rw-r--r-- 1 root  root     3 fév 15 14:03 file27
-rw-r--r-- 1 root  root     3 fév 15 14:03 file28
-rw-r--r-- 1 root  root     3 fév 15 14:03 file29
-rw-r--r-- 1 admin users    0 fév 15 14:01 file3
-rw-r--r-- 1 root  root     3 fév 15 14:03 file30
-rw-r--r-- 1 root  root     3 fév 15 14:03 file31
-rw-r--r-- 1 root  root     3 fév 15 14:03 file32
-rw-r--r-- 1 root  root     3 fév 15 14:03 file33
-rw-r--r-- 1 root  root     3 fév 15 14:03 file34
-rw-r--r-- 1 root  root     3 fév 15 14:03 file35
-rw-r--r-- 1 root  root     3 fév 15 14:03 file36
-rw-r--r-- 1 root  root     3 fév 15 14:03 file37
-rw-r--r-- 1 root  root     3 fév 15 14:03 file38
-rw-r--r-- 1 root  root     3 fév 15 14:03 file39
-rw-r--r-- 1 admin users    0 fév 15 14:01 file4
-rw-r--r-- 1 root  root     3 fév 15 14:03 file40
-rw-r--r-- 1 root  root     3 fév 15 14:03 file41
-rw-r--r-- 1 root  root     3 fév 15 14:03 file42
-rw-r--r-- 1 root  root     3 fév 15 14:03 file43
-rw-r--r-- 1 root  root     3 fév 15 14:03 file44
-rw-r--r-- 1 root  root     3 fév 15 14:03 file45
-rw-r--r-- 1 root  root     3 fév 15 14:03 file46
-rw-r--r-- 1 root  root     3 fév 15 14:03 file47
-rw-r--r-- 1 root  root     3 fév 15 14:03 file48
-rw-r--r-- 1 root  root     3 fév 15 14:03 file49
-rw-r--r-- 1 admin users    0 fév 15 14:01 file5
-rw-r--r-- 1 root  root     3 fév 15 14:03 file50
-rw-r--r-- 1 root  root  391M fév 15 14:03 random410M.dat
-rw-r--r-- 1 admin users  40M fév 15 14:02 random41M.dat
4.0G    /data/samples/sparse_file.img
451M    /data/samples/sparse_file.img

Jobs done at each step

*list jobs
+-------+---------+-----------+---------------------+------+-------+----------+---------------+-----------+
| jobid | name    | client    | starttime           | type | level | jobfiles | jobbytes      | jobstatus |
+-------+---------+-----------+---------------------+------+-------+----------+---------------+-----------+
|     4 | Earth   | earth-fd  | 2017-02-15 13:59:58 | B    | F     |        2 |         8,851 | T         |
|     5 | Earth   | earth-fd  | 2017-02-15 14:00:57 | B    | I     |        1 |     4,114,219 | T         |
|     6 | Earth   | earth-fd  | 2017-02-15 14:01:37 | B    | I     |        1 |     4,125,146 | T         |
|     7 | Earth   | earth-fd  | 2017-02-15 14:02:13 | B    | I     |        1 |    45,125,208 | T         |
|     8 | Earth   | earth-fd  | 2017-02-15 14:03:53 | B    | I     |        1 |   455,065,574 | T         |
+-------+---------+-----------+---------------------+------+-------+----------+---------------+-----------+



stat /data/samples/sparse_file.img /var/tmp/restores/8/data/samples/sparse_file.img
  File: ‘/data/samples/sparse_file.img’
  Size: 4294967296      Blocks: 921728     IO Block: 4096   regular file
Device: 801h/2049d      Inode: 2099437     Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (  100/   users)
Access: 2017-02-15 14:14:50.412913092 +0100
Modify: 2017-02-15 14:03:42.463105091 +0100
Change: 2017-02-15 14:03:42.463105091 +0100
 Birth: -
  File: ‘/var/tmp/restores/8/data/samples/sparse_file.img’
  Size: 4294967296      Blocks: 891536     IO Block: 4096   regular file
Device: 801h/2049d      Inode: 917886      Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (  100/   users)
Access: 2017-02-15 14:15:22.925161092 +0100
Modify: 2017-02-15 14:03:42.000000000 +0100
Change: 2017-02-15 14:11:03.431479091 +0100
 Birth: -

du -h /data/samples/sparse_file.img /var/tmp/restores/8/data/samples/sparse_file.img
451M    /data/samples/sparse_file.img
436M    /var/tmp/restores/8/data/samples/sparse_file.img

du -h --apparent-size /data/samples/sparse_file.img /var/tmp/restores/8/data/samples/sparse_file.img
4.0G    /data/samples/sparse_file.img
4.0G    /var/tmp/restores/8/data/samples/sparse_file.img

mkdir /tmp/S /tmp/8
mount -t loop /data/samples/sparse_file.img /tmp/S
-> work

mount -t loop /var/tmp/restores/8/data/samples/sparse_file.img /tmp/S
-> error wrong fs ...


Don't know if useful
hexdump source and restore
tail -n 100 0 > 0.100
tail -n 100 8 > 8.100

diff -ru 0.100 8.100
--- 0.100       2017-02-15 14:30:47.982198828 +0100
+++ 8.100       2017-02-15 14:30:56.654207429 +0100
@@ -1,100 +1,100 @@
-c80013b0 0000 0003 0000 0000 0000 0000 2000 8a4e
-c80013c0 000e 0008 001e 0008 1c20 0008 8000 2000
-c80013d0 0000 0003 0000 0000 0000 0000 2000 2b20
-c80013e0 000f 0008 001f 0008 1e20 0008 8000 2000
-c80013f0 0000 0001 0000 0000 0000 0000 2000 4c02
-c8001400 0000 0000 0000 0000 0000 0000 0000 0000
-*
-d8000000 0000 0004 0000 0010 ccc8 0000 aac8 000f
-d8000010 ffee 0003 0000 0000 0002 0000 0002 0000
-d8000020 8000 0000 8000 0000 2000 0000 50df 58a4
-d8000030 5122 58a4 0000 ffff ef53 0000 0001 0000
-d8000040 5122 58a4 0000 0000 0000 0000 0001 0000
-d8000050 0000 0000 000b 0000 0100 001b 003c 0000
-d8000060 0242 0000 007b 0000 321a 7dcb 9235 0747
-d8000070 e1a1 db05 08a0 97db 5053 5241 4553 545f
-d8000080 5345 0054 0000 0000 742f 706d 742f 706d
-d8000090 5053 5241 4553 0000 0000 0000 0000 0000
+d7ff3fd0 5a46 5a46 0000 1d06 0000 0000 0000 0000
+d7ff3fe0 0000 0000 0000 0000 0000 0000 0000 0000
+*
+d8000000 0000 0000 0000 0000 0000 0004 0000 0010
+d8000010 ccc8 0000 aac8 000f ffee 0003 0000 0000
+d8000020 0002 0000 0002 0000 8000 0000 8000 0000
+d8000030 2000 0000 50df 58a4 5122 58a4 0000 ffff
+d8000040 ef53 0000 0001 0000 5122 58a4 0000 0000
+d8000050 0000 0000 0001 0000 0000 0000 000b 0000
+d8000060 0100 001b 003c 0000 0242 0000 007b 0000
+d8000070 321a 7dcb 9235 0747 e1a1 db05 08a0 97db
+d8000080 5053 5241 4553 545f 5345 0054 0000 0000
+d8000090 742f 706d 742f 706d 5053 5241 4553 0000
 d80000a0 0000 0000 0000 0000 0000 0000 0000 0000
 *
-d80000c0 0000 0000 0000 0000 0000 0000 0000 001f
-d80000d0 0000 0000 0000 0000 0000 0000 0000 0000
-d80000e0 0008 0000 0000 0000 0000 0000 f258 ef20
-d80000f0 66e3 224b 199f b11b 7442 e111 0101 0000
-d8000100 000c 0000 0000 0000 503c 58a4 f30a 0001
-d8000110 0004 0000 0000 0000 0000 0000 1000 0000
-d8000120 0000 0001 0000 0000 0000 0000 0000 0000
+d80000d0 0000 0000 0000 001f 0000 0000 0000 0000
+d80000e0 0000 0000 0000 0000 0008 0000 0000 0000
+d80000f0 0000 0000 f258 ef20 66e3 224b 199f b11b
+d8000100 7442 e111 0101 0000 000c 0000 0000 0000
+d8000110 503c 58a4 f30a 0001 0004 0000 0000 0000
+d8000120 0000 0000 1000 0000 0000 0001 0000 0000
 d8000130 0000 0000 0000 0000 0000 0000 0000 0000
-d8000140 0000 0000 0000 0000 0000 0000 0000 0100
-d8000150 0000 0000 0000 0000 0000 0000 001c 001c
-d8000160 0001 0000 0000 0000 0000 0000 0000 0000
-d8000170 0000 0000 0004 0000 51ad 0000 0000 0000
-d8000180 0000 0000 0000 0000 0000 0000 0000 0000
-*
-d8001000 0021 0000 0031 0000 0041 0000 5fb8 1fee
-d8001010 0003 0004 0000 0000 0000 0000 1fee a5e4
-d8001020 0022 0000 0032 0000 0241 0000 7bf6 2000
-d8001030 0000 0005 0000 0000 0000 0000 2000 a2a3
-d8001040 0023 0000 0033 0000 0441 0000 7000 2000
-d8001050 0000 0005 0000 0000 0000 0000 2000 9498
-d8001060 0024 0000 0034 0000 0641 0000 7fdf 2000
-d8001070 0000 0005 0000 0000 0000 0000 2000 096e
-d8001080 002a 0000 0035 0000 0842 0000 8000 2000
-d8001090 0000 0003 0000 0000 0000 0000 2000 4872
-d80010a0 002b 0000 0036 0000 0a42 0000 7fdf 2000
-d80010b0 0000 0001 0000 0000 0000 0000 2000 bed9
-d80010c0 002c 0000 0037 0000 0c42 0000 8000 2000
-d80010d0 0000 0003 0000 0000 0000 0000 2000 49c0
-d80010e0 002d 0000 0038 0000 0e42 0000 7fdf 2000
-d80010f0 0000 0001 0000 0000 0000 0000 2000 ff44
-d8001100 002e 0000 0039 0000 1042 0000 8000 2000
-d8001110 0000 0003 0000 0000 0000 0000 2000 4dc2
-d8001120 002f 0000 003a 0000 1242 0000 7fdf 2000
-d8001130 0000 0001 0000 0000 0000 0000 2000 bb69
-d8001140 003b 0000 003c 0000 1442 0000 8000 2000
-d8001150 0000 0003 0000 0000 0000 0000 2000 c381
-d8001160 003d 0000 003e 0000 1642 0000 8000 2000
-d8001170 0000 0003 0000 0000 0000 0000 2000 d355
-d8001180 003f 0000 0040 0000 1842 0000 8000 2000
-d8001190 0000 0003 0000 0000 0000 0000 2000 df8f
-d80011a0 1a42 0000 1a43 0000 1a44 0000 8000 2000
-d80011b0 0000 0003 0000 0000 0000 0000 2000 96c9
-d80011c0 1c44 0000 1c45 0000 1c46 0000 8000 2000
-d80011d0 0000 0003 0000 0000 0000 0000 2000 ed61
-d80011e0 1e46 0000 1e47 0000 1e48 0000 8000 2000
-d80011f0 0000 0003 0000 0000 0000 0000 2000 c0f6
-d8001200 0000 0008 0010 0008 0020 0008 5fe0 2000
-d8001210 0000 0001 0000 0000 0000 0000 2000 148d
-d8001220 0001 0008 0011 0008 0220 0008 8000 2000
-d8001230 0000 0003 0000 0000 0000 0000 2000 8ff0
-d8001240 0002 0008 0012 0008 0420 0008 8000 2000
-d8001250 0000 0003 0000 0000 0000 0000 2000 2e9e
-d8001260 0003 0008 0013 0008 0620 0008 8000 2000
-d8001270 0000 0003 0000 0000 0000 0000 2000 8e45
-d8001280 0004 0008 0014 0008 0820 0008 8000 2000
-d8001290 0000 0003 0000 0000 0000 0000 2000 2c41
-d80012a0 0005 0008 0015 0008 0a20 0008 8000 2000
-d80012b0 0000 0003 0000 0000 0000 0000 2000 8c9a
-d80012c0 0006 0008 0016 0008 0c20 0008 8000 2000
-d80012d0 0000 0003 0000 0000 0000 0000 2000 2df4
-d80012e0 0007 0008 0017 0008 0e20 0008 8000 2000
-d80012f0 0000 0003 0000 0000 0000 0000 2000 8d2f
-d8001300 0008 0008 0018 0008 1020 0008 8000 2000
-d8001310 0000 0003 0000 0000 0000 0000 2000 29ff
-d8001320 0009 0008 0019 0008 1220 0008 7fdf 2000
-d8001330 0000 0001 0000 0000 0000 0000 2000 60a7
-d8001340 000a 0008 001a 0008 1420 0008 8000 2000
-d8001350 0000 0003 0000 0000 0000 0000 2000 284a
-d8001360 000b 0008 001b 0008 1620 0008 7fdf 2000
-d8001370 0000 0001 0000 0000 0000 0000 2000 6112
-d8001380 000c 0008 001c 0008 1820 0008 8000 2000
-d8001390 0000 0003 0000 0000 0000 0000 2000 2a95
-d80013a0 000d 0008 001d 0008 1a20 0008 8000 2000
-d80013b0 0000 0003 0000 0000 0000 0000 2000 8a4e
-d80013c0 000e 0008 001e 0008 1c20 0008 8000 2000
-d80013d0 0000 0003 0000 0000 0000 0000 2000 2b20
-d80013e0 000f 0008 001f 0008 1e20 0008 8000 2000
-d80013f0 0000 0001 0000 0000 0000 0000 2000 4c02
-d8001400 0000 0000 0000 0000 0000 0000 0000 0000
+*
+d8000150 0000 0000 0000 0100 0000 0000 0000 0000
+d8000160 0000 0000 001c 001c 0001 0000 0000 0000
+d8000170 0000 0000 0000 0000 0000 0000 0004 0000
+d8000180 51ad 0000 0000 0000 0000 0000 0000 0000
+d8000190 0000 0000 0000 0000 0000 0000 0000 0000
+*
+d8001000 0000 0000 0000 0000 0021 0000 0031 0000
+d8001010 0041 0000 5fb8 1fee 0003 0004 0000 0000
+d8001020 0000 0000 1fee a5e4 0022 0000 0032 0000
+d8001030 0241 0000 7bf6 2000 0000 0005 0000 0000
+d8001040 0000 0000 2000 a2a3 0023 0000 0033 0000
+d8001050 0441 0000 7000 2000 0000 0005 0000 0000
+d8001060 0000 0000 2000 9498 0024 0000 0034 0000
+d8001070 0641 0000 7fdf 2000 0000 0005 0000 0000
+d8001080 0000 0000 2000 096e 002a 0000 0035 0000
+d8001090 0842 0000 8000 2000 0000 0003 0000 0000
+d80010a0 0000 0000 2000 4872 002b 0000 0036 0000
+d80010b0 0a42 0000 7fdf 2000 0000 0001 0000 0000
+d80010c0 0000 0000 2000 bed9 002c 0000 0037 0000
+d80010d0 0c42 0000 8000 2000 0000 0003 0000 0000
+d80010e0 0000 0000 2000 49c0 002d 0000 0038 0000
+d80010f0 0e42 0000 7fdf 2000 0000 0001 0000 0000
+d8001100 0000 0000 2000 ff44 002e 0000 0039 0000
+d8001110 1042 0000 8000 2000 0000 0003 0000 0000
+d8001120 0000 0000 2000 4dc2 002f 0000 003a 0000
+d8001130 1242 0000 7fdf 2000 0000 0001 0000 0000
+d8001140 0000 0000 2000 bb69 003b 0000 003c 0000
+d8001150 1442 0000 8000 2000 0000 0003 0000 0000
+d8001160 0000 0000 2000 c381 003d 0000 003e 0000
+d8001170 1642 0000 8000 2000 0000 0003 0000 0000
+d8001180 0000 0000 2000 d355 003f 0000 0040 0000
+d8001190 1842 0000 8000 2000 0000 0003 0000 0000
+d80011a0 0000 0000 2000 df8f 1a42 0000 1a43 0000
+d80011b0 1a44 0000 8000 2000 0000 0003 0000 0000
+d80011c0 0000 0000 2000 96c9 1c44 0000 1c45 0000
+d80011d0 1c46 0000 8000 2000 0000 0003 0000 0000
+d80011e0 0000 0000 2000 ed61 1e46 0000 1e47 0000
+d80011f0 1e48 0000 8000 2000 0000 0003 0000 0000
+d8001200 0000 0000 2000 c0f6 0000 0008 0010 0008
+d8001210 0020 0008 5fe0 2000 0000 0001 0000 0000
+d8001220 0000 0000 2000 148d 0001 0008 0011 0008
+d8001230 0220 0008 8000 2000 0000 0003 0000 0000
+d8001240 0000 0000 2000 8ff0 0002 0008 0012 0008
+d8001250 0420 0008 8000 2000 0000 0003 0000 0000
+d8001260 0000 0000 2000 2e9e 0003 0008 0013 0008
+d8001270 0620 0008 8000 2000 0000 0003 0000 0000
+d8001280 0000 0000 2000 8e45 0004 0008 0014 0008
+d8001290 0820 0008 8000 2000 0000 0003 0000 0000
+d80012a0 0000 0000 2000 2c41 0005 0008 0015 0008
+d80012b0 0a20 0008 8000 2000 0000 0003 0000 0000
+d80012c0 0000 0000 2000 8c9a 0006 0008 0016 0008
+d80012d0 0c20 0008 8000 2000 0000 0003 0000 0000
+d80012e0 0000 0000 2000 2df4 0007 0008 0017 0008
+d80012f0 0e20 0008 8000 2000 0000 0003 0000 0000
+d8001300 0000 0000 2000 8d2f 0008 0008 0018 0008
+d8001310 1020 0008 8000 2000 0000 0003 0000 0000
+d8001320 0000 0000 2000 29ff 0009 0008 0019 0008
+d8001330 1220 0008 7fdf 2000 0000 0001 0000 0000
+d8001340 0000 0000 2000 60a7 000a 0008 001a 0008
+d8001350 1420 0008 8000 2000 0000 0003 0000 0000
+d8001360 0000 0000 2000 284a 000b 0008 001b 0008
+d8001370 1620 0008 7fdf 2000 0000 0001 0000 0000
+d8001380 0000 0000 2000 6112 000c 0008 001c 0008
+d8001390 1820 0008 8000 2000 0000 0003 0000 0000
+d80013a0 0000 0000 2000 2a95 000d 0008 001d 0008
+d80013b0 1a20 0008 8000 2000 0000 0003 0000 0000
+d80013c0 0000 0000 2000 8a4e 000e 0008 001e 0008
+d80013d0 1c20 0008 8000 2000 0000 0003 0000 0000
+d80013e0 0000 0000 2000 2b20 000f 0008 001f 0008
+d80013f0 1e20 0008 8000 2000 0000 0001 0000 0000
+d8001400 0000 0000 2000 4c02 0000 0000 0000 0000
+d8001410 0000 0000 0000 0000 0000 0000 0000 0000
+*
+ffffffc0 5a46 5a46 0000 6000 0000 0000 0000 0000
+ffffffd0 0000 0000 0000 0000 0000 0000 0000 0000
 *
 100000000


Same apply to first sparse empty created file

sha256sum sparse_file.img /var/tmp/restores/4/data/samples/sparse_file.img
4fa7579c0fad0465105a8df8206b862fb245d7ad89f0f7f956bb4a0313fa7e2c  sparse_file.img
879b37608381b7b57a11d7f98968f26708d4fc16ea02e1d9581a8cbffbc68475  /var/tmp/restores/4/data/samples/sparse_file.img
earth:/data/samples # du -h /var/tmp/restores/4/data/samples/sparse_file.img
588K    /var/tmp/restores/4/data/samples/sparse_file.img
earth:/data/samples # du -h sparse_file.img /var/tmp/restores/4/data/samples/sparse_file.img
17M     sparse_file.img
588K    /var/tmp/restores/4/data/samples/sparse_file.img
earth:/data/samples # du -h --apparent-size sparse_file.img /var/tmp/restores/4/data/samples/sparse_file.img
512M    sparse_file.img
512M    /var/tmp/restores/4/data/samples/sparse_file.img


bareos jobs logs


bconsole
Connecting to Director earth.bareos.local:9101
1000 OK: earth-dir Version: 16.2.4 (01 July 2016)
Enter a period to cancel a command.
*list joblog jobid=4
Automatically selected Catalog: MyCatalog
Using Catalog "MyCatalog"
 2017-02-15 13:59:56 earth-dir JobId 4: No prior Full backup Job record found.
 2017-02-15 13:59:56 earth-dir JobId 4: No prior or suitable Full backup found in catalog. Doing FULL backup.
 2017-02-15 13:59:58 earth-dir JobId 4: Start Backup JobId 4, Job=Earth.2017-02-15_13.59.56_04
 2017-02-15 13:59:58 earth-dir JobId 4: Max configured use duration=82,800 sec. exceeded. Marking Volume "Full-0001" as Used.
 2017-02-15 13:59:58 earth-dir JobId 4: Created new Volume "Full-0002" in catalog.
 2017-02-15 13:59:58 earth-dir JobId 4: Using Device "Default" to write.
 2017-02-15 13:59:58 earth-sd JobId 4: Labeled new Volume "Full-0002" on device "Default" (/var/lib/bareos/storage/default).
 2017-02-15 13:59:58 earth-sd JobId 4: Wrote label to prelabeled Volume "Full-0002" on device "Default" (/var/lib/bareos/storage/default)
 2017-02-15 13:59:58 earth-sd JobId 4: autoxflate-sd: Compressor on device Default is LZ4HC
 2017-02-15 13:59:58 earth-sd JobId 4: autoxflate-sd: Default OUT:[SD->inflate=yes->deflate=yes->DEV] IN:[DEV->inflate=yes->deflate=yes->SD]
 2017-02-15 13:59:58 earth-sd JobId 4: Elapsed time=00:00:01, Transfer rate=590.0 K Bytes/second
 2017-02-15 13:59:58 earth-sd JobId 4: autoxflate-sd: inflate ratio: 6664.11%
 2017-02-15 13:59:58 earth-dir JobId 4: Bareos earth-dir 16.2.4 (01Jul16):
  Build OS:               x86_64-suse-linux-gnu suse openSUSE Leap 42.1 (x86_64)
  JobId:                  4
  Job:                    Earth.2017-02-15_13.59.56_04
  Backup Level:           Full (upgraded from Incremental)
  Client:                 "earth-fd" 16.2.4 (01Jul16) x86_64-suse-linux-gnu,suse,openSUSE Leap 42.1 (x86_64),openSUSE_Leap_42.1,x86_64
  FileSet:                "Sparse" 2017-02-15 13:59:56
  Pool:                   "Full" (From Job FullPool override)
  Catalog:                "MyCatalog" (From Client resource)
  Storage:                "Default" (From Job resource)
  Scheduled time:         15-Feb-2017 13:59:30
  Start time:             15-Feb-2017 13:59:58
  End time:               15-Feb-2017 13:59:58
  Elapsed time:           0 secs
  Priority:               10
  FD Files Written:       2
  SD Files Written:       2
  FD Bytes Written:       8,851 (8.851 KB)
  SD Bytes Written:       590,042 (590.0 KB)
  Rate:                   0.0 KB/s
  Software Compression:   98.5 % (lz4hc)
  VSS:                    no
  Encryption:             no
  Accurate:               yes
  Volume name(s):         Full-0002
  Volume Session Id:      1
  Volume Session Time:    1487163557
  Last Volume Bytes:      591,075 (591.0 KB)
  Non-fatal FD errors:    0
  SD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Backup OK

*list joblog jobid=5
 2017-02-15 14:00:57 earth-dir JobId 5: Start Backup JobId 5, Job=Earth.2017-02-15_14.00.55_05
 2017-02-15 14:00:57 earth-dir JobId 5: Created new Volume "Incremental-0003" in catalog.
 2017-02-15 14:00:57 earth-dir JobId 5: Using Device "Default" to write.
 2017-02-15 14:00:57 earth-dir JobId 5: Sending Accurate information.
 2017-02-15 14:00:57 earth-sd JobId 5: Labeled new Volume "Incremental-0003" on device "Default" (/var/lib/bareos/storage/default).
 2017-02-15 14:00:57 earth-sd JobId 5: Wrote label to prelabeled Volume "Incremental-0003" on device "Default" (/var/lib/bareos/storage/default)
 2017-02-15 14:00:57 earth-sd JobId 5: autoxflate-sd: Compressor on device Default is LZ4HC
 2017-02-15 14:00:57 earth-fd JobId 5:      Unchanged file skipped: /data/samples
 2017-02-15 14:00:57 earth-sd JobId 5: autoxflate-sd: Default OUT:[SD->inflate=yes->deflate=yes->DEV] IN:[DEV->inflate=yes->deflate=yes->SD]
 2017-02-15 14:00:58 earth-fd JobId 5:      Unchanged file skipped: /data/samples
 2017-02-15 14:00:58 earth-sd JobId 5: Elapsed time=00:00:01, Transfer rate=4.784 M Bytes/second
 2017-02-15 14:00:58 earth-sd JobId 5: autoxflate-sd: inflate ratio: 116.28%
 2017-02-15 14:00:58 earth-dir JobId 5: Bareos earth-dir 16.2.4 (01Jul16):
  Build OS:               x86_64-suse-linux-gnu suse openSUSE Leap 42.1 (x86_64)
  JobId:                  5
  Job:                    Earth.2017-02-15_14.00.55_05
  Backup Level:           Incremental, since=2017-02-15 13:59:58
  Client:                 "earth-fd" 16.2.4 (01Jul16) x86_64-suse-linux-gnu,suse,openSUSE Leap 42.1 (x86_64),openSUSE_Leap_42.1,x86_64
  FileSet:                "Sparse" 2017-02-15 13:59:56
  Pool:                   "Incremental" (From Job IncPool override)
  Catalog:                "MyCatalog" (From Client resource)
  Storage:                "Default" (From Job resource)
  Scheduled time:         15-Feb-2017 14:00:51
  Start time:             15-Feb-2017 14:00:57
  End time:               15-Feb-2017 14:00:58
  Elapsed time:           1 sec
  Priority:               10
  FD Files Written:       1
  SD Files Written:       1
  FD Bytes Written:       4,114,219 (4.114 MB)
  SD Bytes Written:       4,784,264 (4.784 MB)
  Rate:                   4114.2 KB/s
  Software Compression:   14.0 % (lz4hc)
  VSS:                    no
  Encryption:             no
  Accurate:               yes
  Volume name(s):         Incremental-0003
  Volume Session Id:      2
  Volume Session Time:    1487163557
  Last Volume Bytes:      4,788,421 (4.788 MB)
  Non-fatal FD errors:    0
  SD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Backup OK

*list joblog jobid=6
 2017-02-15 14:01:37 earth-dir JobId 6: Start Backup JobId 6, Job=Earth.2017-02-15_14.01.35_06
 2017-02-15 14:01:37 earth-dir JobId 6: Using Device "Default" to write.
 2017-02-15 14:01:37 earth-dir JobId 6: Sending Accurate information.
 2017-02-15 14:01:37 earth-sd JobId 6: Volume "Incremental-0003" previously written, moving to end of data.
 2017-02-15 14:01:37 earth-sd JobId 6: Ready to append to end of Volume "Incremental-0003" size=4788421
 2017-02-15 14:01:37 earth-sd JobId 6: autoxflate-sd: Compressor on device Default is LZ4HC
 2017-02-15 14:01:37 earth-fd JobId 6:      Unchanged file skipped: /data/samples
 2017-02-15 14:01:37 earth-sd JobId 6: autoxflate-sd: Default OUT:[SD->inflate=yes->deflate=yes->DEV] IN:[DEV->inflate=yes->deflate=yes->SD]
 2017-02-15 14:01:39 earth-fd JobId 6:      Unchanged file skipped: /data/samples
 2017-02-15 14:01:39 earth-sd JobId 6: Elapsed time=00:00:02, Transfer rate=2.588 M Bytes/second
 2017-02-15 14:01:39 earth-sd JobId 6: autoxflate-sd: inflate ratio: 125.51%
 2017-02-15 14:01:39 earth-dir JobId 6: Bareos earth-dir 16.2.4 (01Jul16):
  Build OS:               x86_64-suse-linux-gnu suse openSUSE Leap 42.1 (x86_64)
  JobId:                  6
  Job:                    Earth.2017-02-15_14.01.35_06
  Backup Level:           Incremental, since=2017-02-15 14:00:57
  Client:                 "earth-fd" 16.2.4 (01Jul16) x86_64-suse-linux-gnu,suse,openSUSE Leap 42.1 (x86_64),openSUSE_Leap_42.1,x86_64
  FileSet:                "Sparse" 2017-02-15 13:59:56
  Pool:                   "Incremental" (From Job IncPool override)
  Catalog:                "MyCatalog" (From Client resource)
  Storage:                "Default" (From Job resource)
  Scheduled time:         15-Feb-2017 14:01:33
  Start time:             15-Feb-2017 14:01:37
  End time:               15-Feb-2017 14:01:39
  Elapsed time:           2 secs
  Priority:               10
  FD Files Written:       1
  SD Files Written:       1
  FD Bytes Written:       4,125,146 (4.125 MB)
  SD Bytes Written:       5,177,537 (5.177 MB)
  Rate:                   2062.6 KB/s
  Software Compression:   20.3 % (lz4hc)
  VSS:                    no
  Encryption:             no
  Accurate:               yes
  Volume name(s):         Incremental-0003
  Volume Session Id:      3
  Volume Session Time:    1487163557
  Last Volume Bytes:      9,970,198 (9.970 MB)
  Non-fatal FD errors:    0
  SD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Backup OK

*list joblog jobid=7
 2017-02-15 14:02:13 earth-dir JobId 7: Start Backup JobId 7, Job=Earth.2017-02-15_14.02.11_07
 2017-02-15 14:02:13 earth-dir JobId 7: Using Device "Default" to write.
 2017-02-15 14:02:14 earth-dir JobId 7: Sending Accurate information.
 2017-02-15 14:02:14 earth-sd JobId 7: Volume "Incremental-0003" previously written, moving to end of data.
 2017-02-15 14:02:14 earth-sd JobId 7: Ready to append to end of Volume "Incremental-0003" size=9970198
 2017-02-15 14:02:14 earth-sd JobId 7: autoxflate-sd: Compressor on device Default is LZ4HC
 2017-02-15 14:02:14 earth-fd JobId 7:      Unchanged file skipped: /data/samples
 2017-02-15 14:02:14 earth-sd JobId 7: autoxflate-sd: Default OUT:[SD->inflate=yes->deflate=yes->DEV] IN:[DEV->inflate=yes->deflate=yes->SD]
 2017-02-15 14:02:15 earth-fd JobId 7:      Unchanged file skipped: /data/samples
 2017-02-15 14:02:15 earth-sd JobId 7: Elapsed time=00:00:01, Transfer rate=46.46 M Bytes/second
 2017-02-15 14:02:15 earth-sd JobId 7: autoxflate-sd: inflate ratio: 102.97%
 2017-02-15 14:02:15 earth-dir JobId 7: Bareos earth-dir 16.2.4 (01Jul16):
  Build OS:               x86_64-suse-linux-gnu suse openSUSE Leap 42.1 (x86_64)
  JobId:                  7
  Job:                    Earth.2017-02-15_14.02.11_07
  Backup Level:           Incremental, since=2017-02-15 14:01:37
  Client:                 "earth-fd" 16.2.4 (01Jul16) x86_64-suse-linux-gnu,suse,openSUSE Leap 42.1 (x86_64),openSUSE_Leap_42.1,x86_64
  FileSet:                "Sparse" 2017-02-15 13:59:56
  Pool:                   "Incremental" (From Job IncPool override)
  Catalog:                "MyCatalog" (From Client resource)
  Storage:                "Default" (From Job resource)
  Scheduled time:         15-Feb-2017 14:02:11
  Start time:             15-Feb-2017 14:02:13
  End time:               15-Feb-2017 14:02:15
  Elapsed time:           2 secs
  Priority:               10
  FD Files Written:       1
  SD Files Written:       1
  FD Bytes Written:       45,125,208 (45.12 MB)
  SD Bytes Written:       46,465,217 (46.46 MB)
  Rate:                   22562.6 KB/s
  Software Compression:   2.9 % (lz4hc)
  VSS:                    no
  Encryption:             no
  Accurate:               yes
  Volume name(s):         Incremental-0003
  Volume Session Id:      4
  Volume Session Time:    1487163557
  Last Volume Bytes:      56,470,255 (56.47 MB)
  Non-fatal FD errors:    0
  SD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Backup OK

*list joblog jobid=8
 2017-02-15 14:03:52 earth-dir JobId 8: Start Backup JobId 8, Job=Earth.2017-02-15_14.03.50_08
 2017-02-15 14:03:52 earth-dir JobId 8: Using Device "Default" to write.
 2017-02-15 14:03:53 earth-dir JobId 8: Sending Accurate information.
 2017-02-15 14:03:53 earth-sd JobId 8: Volume "Incremental-0003" previously written, moving to end of data.
 2017-02-15 14:03:53 earth-sd JobId 8: Ready to append to end of Volume "Incremental-0003" size=56470255
 2017-02-15 14:03:53 earth-sd JobId 8: autoxflate-sd: Compressor on device Default is LZ4HC
 2017-02-15 14:03:53 earth-fd JobId 8:      Unchanged file skipped: /data/samples
 2017-02-15 14:03:53 earth-sd JobId 8: autoxflate-sd: Default OUT:[SD->inflate=yes->deflate=yes->DEV] IN:[DEV->inflate=yes->deflate=yes->SD]
 2017-02-15 14:03:58 earth-fd JobId 8:      Unchanged file skipped: /data/samples
 2017-02-15 14:03:58 earth-sd JobId 8: Elapsed time=00:00:05, Transfer rate=91.29 M Bytes/second
 2017-02-15 14:03:58 earth-sd JobId 8: autoxflate-sd: inflate ratio: 100.31%
 2017-02-15 14:03:58 earth-dir JobId 8: Bareos earth-dir 16.2.4 (01Jul16):
  Build OS:               x86_64-suse-linux-gnu suse openSUSE Leap 42.1 (x86_64)
  JobId:                  8
  Job:                    Earth.2017-02-15_14.03.50_08
  Backup Level:           Incremental, since=2017-02-15 14:02:13
  Client:                 "earth-fd" 16.2.4 (01Jul16) x86_64-suse-linux-gnu,suse,openSUSE Leap 42.1 (x86_64),openSUSE_Leap_42.1,x86_64
  FileSet:                "Sparse" 2017-02-15 13:59:56
  Pool:                   "Incremental" (From Job IncPool override)
  Catalog:                "MyCatalog" (From Client resource)
  Storage:                "Default" (From Job resource)
  Scheduled time:         15-Feb-2017 14:03:50
  Start time:             15-Feb-2017 14:03:53
  End time:               15-Feb-2017 14:03:58
  Elapsed time:           5 secs
  Priority:               10
  FD Files Written:       1
  SD Files Written:       1
  FD Bytes Written:       455,065,574 (455.0 MB)
  SD Bytes Written:       456,458,434 (456.4 MB)
  Rate:                   91013.1 KB/s
  Software Compression:   None
  VSS:                    no
  Encryption:             no
  Accurate:               yes
  Volume name(s):         Incremental-0003
  Volume Session Id:      5
  Volume Session Time:    1487163557
  Last Volume Bytes:      513,267,549 (513.2 MB)
  Non-fatal FD errors:    0
  SD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Backup OK

*
*
*
*
*list joblog jobid=9
 2017-02-15 14:10:54 earth-dir JobId 9: Start Restore Job Restore.2017-02-15_14.10.52_09
 2017-02-15 14:10:55 earth-dir JobId 9: Using Device "Default" to read.
 2017-02-15 14:10:55 earth-sd JobId 9: Ready to read from volume "Incremental-0003" on device "Default" (/var/lib/bareos/storage/default).
 2017-02-15 14:10:55 earth-fd JobId 9: -rw-r--r--   1 root     users     4294967296 2017-02-15 14:03:42  /var/tmp/restores/8//data/samples/sparse_file.img
 2017-02-15 14:10:55 earth-sd JobId 9: autoxflate-sd: Compressor on device Default is LZ4HC
 2017-02-15 14:10:55 earth-sd JobId 9: autoxflate-sd: Default OUT:[SD->inflate=yes->deflate=yes->DEV] IN:[DEV->inflate=yes->deflate=yes->SD]
 2017-02-15 14:10:55 earth-sd JobId 9: Forward spacing Volume "Incremental-0003" to file:block 0:56470255.
 2017-02-15 14:11:03 earth-sd JobId 9: End of Volume at file 0 on device "Default" (/var/lib/bareos/storage/default), Volume "Incremental-0003"
 2017-02-15 14:11:03 earth-sd JobId 9: End of all volumes.
 2017-02-15 14:11:03 earth-sd JobId 9: autoxflate-sd: deflate ratio: 99.70%
 2017-02-15 14:11:03 earth-dir JobId 9: Bareos earth-dir 16.2.4 (01Jul16):
  Build OS:               x86_64-suse-linux-gnu suse openSUSE Leap 42.1 (x86_64)
  JobId:                  9
  Job:                    Restore.2017-02-15_14.10.52_09
  Restore Client:         earth-fd
  Start time:             15-Feb-2017 14:10:54
  End time:               15-Feb-2017 14:11:03
  Elapsed time:           9 secs
  Files Expected:         1
  Files Restored:         1
  Bytes Restored:         456,402,584
  Rate:                   50711.4 KB/s
  FD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Restore OK

*list joblog jobid=10
 2017-02-15 14:12:43 earth-dir JobId 10: Start Restore Job Restore.2017-02-15_14.12.41_11
 2017-02-15 14:12:43 earth-dir JobId 10: Using Device "Default" to read.
 2017-02-15 14:12:43 earth-sd JobId 10: Ready to read from volume "Incremental-0003" on device "Default" (/var/lib/bareos/storage/default).
 2017-02-15 14:12:43 earth-fd JobId 10: -rw-r--r--   1 root     users     4294967296 2017-02-15 14:02:02  /var/tmp/restores/7/data/samples/sparse_file.img
 2017-02-15 14:12:43 earth-sd JobId 10: autoxflate-sd: Compressor on device Default is LZ4HC
 2017-02-15 14:12:43 earth-sd JobId 10: autoxflate-sd: Default OUT:[SD->inflate=yes->deflate=yes->DEV] IN:[DEV->inflate=yes->deflate=yes->SD]
 2017-02-15 14:12:43 earth-sd JobId 10: Forward spacing Volume "Incremental-0003" to file:block 0:9970198.
 2017-02-15 14:12:44 earth-sd JobId 10: autoxflate-sd: deflate ratio: 97.09%
 2017-02-15 14:12:44 earth-dir JobId 10: Bareos earth-dir 16.2.4 (01Jul16):
  Build OS:               x86_64-suse-linux-gnu suse openSUSE Leap 42.1 (x86_64)
  JobId:                  10
  Job:                    Restore.2017-02-15_14.12.41_11
  Restore Client:         earth-fd
  Start time:             15-Feb-2017 14:12:43
  End time:               15-Feb-2017 14:12:44
  Elapsed time:           1 sec
  Files Expected:         1
  Files Restored:         1
  Bytes Restored:         46,459,416
  Rate:                   46459.4 KB/s
  FD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Restore OK

*list joblog jobid=11
 2017-02-15 14:13:31 earth-dir JobId 11: Start Restore Job Restore.2017-02-15_14.13.29_12
 2017-02-15 14:13:31 earth-dir JobId 11: Using Device "Default" to read.
 2017-02-15 14:13:31 earth-sd JobId 11: Ready to read from volume "Incremental-0003" on device "Default" (/var/lib/bareos/storage/default).
 2017-02-15 14:13:31 earth-fd JobId 11: -rw-r--r--   1 root     users     4294967296 2017-02-15 14:01:22  /var/tmp/restores/6/data/samples/sparse_file.img
 2017-02-15 14:13:31 earth-sd JobId 11: autoxflate-sd: Compressor on device Default is LZ4HC
 2017-02-15 14:13:31 earth-sd JobId 11: autoxflate-sd: Default OUT:[SD->inflate=yes->deflate=yes->DEV] IN:[DEV->inflate=yes->deflate=yes->SD]
 2017-02-15 14:13:31 earth-sd JobId 11: Forward spacing Volume "Incremental-0003" to file:block 0:4788421.
 2017-02-15 14:13:31 earth-sd JobId 11: autoxflate-sd: deflate ratio: 79.47%
 2017-02-15 14:13:31 earth-dir JobId 11: Bareos earth-dir 16.2.4 (01Jul16):
  Build OS:               x86_64-suse-linux-gnu suse openSUSE Leap 42.1 (x86_64)
  JobId:                  11
  Job:                    Restore.2017-02-15_14.13.29_12
  Restore Client:         earth-fd
  Start time:             15-Feb-2017 14:13:31
  End time:               15-Feb-2017 14:13:31
  Elapsed time:           0 secs
  Files Expected:         1
  Files Restored:         1
  Bytes Restored:         5,176,776
  Rate:                   0.0 KB/s
  FD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Restore OK

*list joblog jobid=12
 2017-02-15 14:13:44 earth-dir JobId 12: Start Restore Job Restore.2017-02-15_14.13.42_13
 2017-02-15 14:13:44 earth-dir JobId 12: Using Device "Default" to read.
 2017-02-15 14:13:44 earth-sd JobId 12: Ready to read from volume "Incremental-0003" on device "Default" (/var/lib/bareos/storage/default).
 2017-02-15 14:13:44 earth-fd JobId 12: -rw-r--r--   1 root     users      536870912 2017-02-15 14:00:15  /var/tmp/restores/5/data/samples/sparse_file.img
 2017-02-15 14:13:44 earth-sd JobId 12: autoxflate-sd: Compressor on device Default is LZ4HC
 2017-02-15 14:13:44 earth-sd JobId 12: autoxflate-sd: Default OUT:[SD->inflate=yes->deflate=yes->DEV] IN:[DEV->inflate=yes->deflate=yes->SD]
 2017-02-15 14:13:44 earth-sd JobId 12: Forward spacing Volume "Incremental-0003" to file:block 0:205.
 2017-02-15 14:13:44 earth-sd JobId 12: autoxflate-sd: deflate ratio: 85.84%
 2017-02-15 14:13:44 earth-dir JobId 12: Bareos earth-dir 16.2.4 (01Jul16):
  Build OS:               x86_64-suse-linux-gnu suse openSUSE Leap 42.1 (x86_64)
  JobId:                  12
  Job:                    Restore.2017-02-15_14.13.42_13
  Restore Client:         earth-fd
  Start time:             15-Feb-2017 14:13:44
  End time:               15-Feb-2017 14:13:44
  Elapsed time:           0 secs
  Files Expected:         1
  Files Restored:         1
  Bytes Restored:         4,783,552
  Rate:                   0.0 KB/s
  FD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Restore OK

*list joblog jobid=13
 2017-02-15 14:14:06 earth-dir JobId 13: Start Restore Job Restore.2017-02-15_14.14.04_14
 2017-02-15 14:14:06 earth-dir JobId 13: Using Device "Default" to read.
 2017-02-15 14:14:06 earth-sd JobId 13: Ready to read from volume "Full-0002" on device "Default" (/var/lib/bareos/storage/default).
 2017-02-15 14:14:06 earth-sd JobId 13: autoxflate-sd: Compressor on device Default is LZ4HC
 2017-02-15 14:14:06 earth-sd JobId 13: autoxflate-sd: Default OUT:[SD->inflate=yes->deflate=yes->DEV] IN:[DEV->inflate=yes->deflate=yes->SD]
 2017-02-15 14:14:06 earth-sd JobId 13: Forward spacing Volume "Full-0002" to file:block 0:191.
 2017-02-15 14:14:06 earth-fd JobId 13: -rw-r--r--   1 root     users      536870912 2017-02-15 13:57:32  /var/tmp/restores/4/data/samples/sparse_file.img
 2017-02-15 14:14:06 earth-sd JobId 13: autoxflate-sd: deflate ratio: 0.76%
 2017-02-15 14:14:06 earth-dir JobId 13: Bareos earth-dir 16.2.4 (01Jul16):
  Build OS:               x86_64-suse-linux-gnu suse openSUSE Leap 42.1 (x86_64)
  JobId:                  13
  Job:                    Restore.2017-02-15_14.14.04_14
  Restore Client:         earth-fd
  Start time:             15-Feb-2017 14:14:06
  End time:               15-Feb-2017 14:14:06
  Elapsed time:           0 secs
  Files Expected:         1
  Files Restored:         1
  Bytes Restored:         589,760
  Rate:                   0.0 KB/s
  FD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Restore OK
bareos_patches_first_results.txt (41,343 bytes)   
tigerfoot

tigerfoot

2017-02-15 15:24

developer   ~0002571

Tested today new patched (wip) version.
There's no error in backup nor restore, but unfortunately the file is corrupted.
See attachments about the process.

Even the full just tested again from scratch is broken.
bruno-at-bareos

bruno-at-bareos

2024-03-20 13:47

manager   ~0005843

Sound this has been fixed with https://github.com/bareos/bareos/pull/1013

Issue History

Date Modified Username Field Change
2016-09-04 11:36 tigerfoot New Issue
2016-09-04 11:36 tigerfoot File Added: Restore_Error_With_Daemon_Errors_interleaved.log.xz
2016-09-04 11:37 tigerfoot File Added: sparse_file_help.sh
2016-09-04 11:38 tigerfoot Note Added: 0002343
2016-09-04 11:45 tigerfoot File Added: Default.conf
2016-09-04 11:45 tigerfoot File Added: Sparse.conf
2016-09-04 11:47 tigerfoot File Added: mail-backup.log
2016-09-04 11:47 tigerfoot File Added: mail-restore.log
2016-09-04 11:49 tigerfoot Note Edited: 0002343
2016-09-04 11:54 tigerfoot Note Added: 0002344
2016-09-06 13:01 pstorz Note Added: 0002347
2016-09-06 13:01 pstorz Assigned To => pstorz
2016-09-06 13:01 pstorz Status new => acknowledged
2016-09-07 11:07 pstorz Note Added: 0002348
2016-09-07 11:07 pstorz Status acknowledged => feedback
2016-09-13 08:40 tigerfoot Note Added: 0002351
2016-09-13 08:40 tigerfoot Status feedback => assigned
2017-02-10 10:42 maik Note Added: 0002557
2017-02-13 11:00 maik Note Added: 0002559
2017-02-13 18:51 stephand Note Added: 0002560
2017-02-15 15:23 tigerfoot File Added: bareos_patches_first_results.txt
2017-02-15 15:24 tigerfoot Note Added: 0002571
2024-03-20 13:47 bruno-at-bareos Status assigned => closed
2024-03-20 13:47 bruno-at-bareos Resolution open => fixed
2024-03-20 13:47 bruno-at-bareos Note Added: 0005843