View Issue Details

IDProjectCategoryView StatusLast Update
0000460bareos-core[All Projects] file daemonpublic2015-12-03 11:34
ReporterqwertyAssigned Topstorz 
PrioritynormalSeverityminorReproducibilityN/A
Status assignedResolutionopen 
PlatformSlackware LinuxOSanyOS Version3
Product Version14.2.4 
Target VersionFixed in Version 
Summary0000460: SHA1 checksum sometimes wrong
DescriptionFor some files the checksum is wrong when doing incremental backup in accurate mode.
Note that is just for some files. And always the same files.
These files are not touched in any way.
They get backed up on every backup, full and incremental.
The problem is that it eat up backup media.

The FileSet setup look like this:
FileSet {
  Name = "Server1-data_opt_u"
  Include {
    Options {
      signature = SHA1
      compression = GZIP
      accurate = pins1u
      basejob = pins1u
      verify = pins1
      sparse = yes
      noatime = yes
      checkfilechanges = no
      exclude = yes

      @/opt/bareos/etc/clients.d/srv1.files/data_opt_u.wild_exclude
    }

    @/opt/bareos/etc/clients.d/srv1.files/data_opt_u.include
  }

  Exclude {
    @/opt/bareos/etc/clients.d/srv1.files/data_opt_u.exclude
  }
}

Shorted debug output from the file daemon.
One file as example.
.
.
.
srv1-fd: accurate_htable.c:97-0 add fname=</opt/u/Slackware/13.0/slackware-13.0-install-dvd.iso> lstat=P0E Jxf IHk B H0 H0 A DpqbgA BAA dPIV BUCE69 BMKTCp BU2xiJ A A g delta_seq=0 chksum=qCVteboqSK23Btz+1XrFVVb+KpA
.
.
.
srv1-fd: verify.c:322-0 === read_digest
srv1-fd: accurate.c:55-0 lookup </opt/u/Slackware/13.0/slackware-13.0-install-dvd.iso> ok
srv1-fd: verify.c:262-0 === digest_file
srv1-fd: bfile.c:1101-0 bopen: fname /opt/u/Slackware/13.0/slackware-13.0-install-dvd.iso, flags 262144, mode 0, rdev 0
srv1-fd: verify.c:322-0 === read_digest
srv1-fd: verify.c:437-0 /opt/u/Slackware/13.0/slackware-13.0-install-dvd.iso SHA1 chksum diff. Cat: qCVteboqSK23Btz+1XrFVVb+KpA File: WNmTLFTpUj5hZwRbIWmuqsx6n50
.
.
.
TagsNo tags attached.
bareos-master: impact
bareos-master: action
bareos-18.2: impact
bareos-18.2: action
bareos-17.2: impact
bareos-17.2: action
bareos-16.2: impact
bareos-16.2: action
bareos-15.2: impact
bareos-15.2: action
bareos-14.2: impact
bareos-14.2: action
bareos-13.2: impact
bareos-13.2: action
bareos-12.4: impact
bareos-12.4: action

Relationships

related to 0000462 closed windows file daemon and zero-length files in Inc backup 

Activities

pstorz

pstorz

2015-05-26 12:00

administrator   ~0001741

Hello,

we fixed some problems with the digest code for 0000462, and I think that this should also solve this problem.
(see
https://github.com/bareos/bareos/commit/b75dbb84551b93a4f2359db71dea7527edfd0541 )

Please check with the newest master code if your problem is now fixed.

Thank you
qwerty

qwerty

2015-06-02 21:00

reporter   ~0001765

No. The problem is still there.
backelj

backelj

2015-09-09 10:49

reporter   ~0001822

Hello, I'm experiencing the same thing with these settings:

FileSet {
  Name = test-data-fileset
  Ignore FileSet Changes = yes
  Include {
    Options {
      Compression = GZIP
      Signature = MD5
      BaseJob = pmugcs5
      Accurate = mcs5
      Verify = pins5
      sparse = yes
      aclsupport = yes
      xattrsupport = yes
      hfsplussupport = yes
      checkfilechanges = yes
    }

The initial checksum is different from all subsequent checksums which are all the same.
That is, given the output above, the checksum that is shown in the debug output for accurate_htable.c is different from the one in the debug output of verify.c. In subsequent backups, the checksum in the debug output of verify.c does not change.

I'm wondering how these checksums are computed (are they computed on the original file or on the gzipped-file?) and why the initial checksum is different...
backelj

backelj

2015-09-10 11:12

reporter   ~0001828

Hello again,

I've found an old thread on the bacula mailing lists that is about a similar issue:

http://sourceforge.net/p/bacula/mailman/message/20282084/

I followed the thread and did the following:
- Create a full backup.
- Run a VolumeToCatalog verify job = OK.
- Run a DiskToCatalog verify job = OK.
- Run a incremental backup = files are still copied.

At the end they talk about the "sparse = yes" setting.
If I remove this setting, all is fine again!
Moreover, the checksums in the initial full backup are different from the one with the setting "sparse=yes".
In this thread, they mention that "the Verify code that creates the hash
does not take account of the Sparse flag". Maybe this is the case for the bareos code as well???

So qwerty, would you be able and willing to test this as well (i.e., without the "sparse=yes" setting)?

Many thanks,

Franky
qwerty

qwerty

2015-09-11 14:01

reporter   ~0001830

Hi

Yes, with "Sparse = no" it works.
I missed that bacula thread.
It looks like that old bug is still there.
Thanks Franky.
backelj

backelj

2015-12-03 11:34

reporter   ~0002025

I'm wondering: will there be some action taken to resolve this issue, or will using the "sparse=no" the only solution to this problem?

Issue History

Date Modified Username Field Change
2015-04-27 15:01 qwerty New Issue
2015-05-21 11:30 pstorz Relationship added related to 0000462
2015-05-26 12:00 pstorz Note Added: 0001741
2015-05-26 12:00 pstorz Assigned To => pstorz
2015-05-26 12:00 pstorz Status new => feedback
2015-06-02 21:00 qwerty Note Added: 0001765
2015-06-02 21:00 qwerty Status feedback => assigned
2015-09-09 10:49 backelj Note Added: 0001822
2015-09-10 11:12 backelj Note Added: 0001828
2015-09-11 14:01 qwerty Note Added: 0001830
2015-12-03 11:34 backelj Note Added: 0002025