View Issue Details

IDProjectCategoryView StatusLast Update
0001069bareos-coreGeneralpublic2023-10-11 16:23
Reportergslongo Assigned Tobruno-at-bareos  
PriorityhighSeverityblockReproducibilityalways
Status closedResolutionno change required 
PlatformLinuxOSCentOSOS Version7
Product Version18.2.5 
Summary0001069: Always Incremental Jobs possible bug ? Not backing up same files as Incremental jobs..
DescriptionHi everyone,

We are planing to implement AI jobs for one of our projects and we decided to make tests before on a running infrastructure (Version: 18.2.6).

The problem is AI jobs are backing up less files than original incr job. To do our test we copied a normal job and here is the result of execution :

*list job=denver-always-incremental
Using Catalog "MyCatalog"
+--------+---------------------------+--------+---------------------+------+-------+----------+---------------+-----------+
| jobid | name | client | starttime | type | level | jobfiles | jobbytes | jobstatus |
+--------+---------------------------+--------+---------------------+------+-------+----------+---------------+-----------+
| 21,197 | denver-always-incremental | denver | 2019-03-15 15:03:10 | B | F | 147,998 | 2,079,207,003 | T |
| 21,217 | denver-always-incremental | denver | 2019-03-16 01:51:21 | B | D | 341 | 32,224,492 | T |
| 21,254 | denver-always-incremental | denver | 2019-03-18 01:15:39 | B | I | 17 | 1,524,336 | T |
| 21,290 | denver-always-incremental | denver | 2019-03-19 01:15:34 | B | I | 20 | 4,419,477 | T |
+--------+---------------------------+--------+---------------------+------+-------+----------+---------------+-----------+
*list job=denver
....

| 21,216 | denver | denver | 2019-03-16 01:50:56 | B | D | 341 | 32,223,670 | T |
| 21,253 | denver | denver | 2019-03-18 01:14:34 | B | I | 1,291 | 184,462,873 | T |
| 21,289 | denver | denver | 2019-03-19 01:14:43 | B | I | 2,413 | 132,643,290 | T |
+--------+--------+--------+---------------------+------+-------+----------+---------------+-----------+


As you can see, starting from same base (same differential job on 2019-03-16) does not provides same results. There is a lot of files missing in the AI job.
However, File set is same and "estimate" provides same results.
Only Incr jobs seems to be "buggy"

Here is a copy of our config files :

job
{
  Name = "denver"
  FileSet = "denver"
  Schedule = "WeeklyCycle"
  Client = "denver"
  JobDefs = "DefaultJob"
  Write bootstrap = "/var/lib/bareos/bstraps/denver.bsr"
  Accurate = yes
}

Job
{
  Name = "denver-always-incremental"
  FileSet = "denver"
  Schedule = "WeeklyCycleAlwaysIncremental"
  Client = "denver"
  JobDefs = "DefaultJob"
  Write bootstrap = "/var/lib/bareos/bstraps/denver-always-incremental.bsr"
 
    # Always incremental settings
    AlwaysIncremental = yes
    AlwaysIncrementalJobRetention = 7 days
    Always Incremental Keep Number = 6
   
    Accurate = yes
 
    Pool = AI-Incremental
    Full Backup Pool = AI-Consolidated
}

 
Job {
    Name = "denver-always-incremental-consolidate"
    Type = "Consolidate"
    Accurate = "yes"
    JobDefs = "DefaultJob"
}
 

FileSet
{
  Name = "denver"
  Ignore FileSet Changes = yes
  Include {
    Options {
      signature = MD5
      compression = gzip
      noatime = yes
    }
    File = /
    File = /boot
    File = /home
    File = /var/log
  }
  Exclude {
    File = /proc
    File = /sys
    File = /tmp
    File = /.journal
    File = /.fsck
  }
}


JobDefs {
  Name = "DefaultJob"
  Type = Backup
  Level = Full
  Client = baloo
  FileSet = "Full Set"
  Schedule = "NONE"
  Storage = File
  Messages = Standard
  Pool = Month
  Priority = 10
  Accurate = yes
}


Do is it a bad configuration or it could be a bug ?

Thank you !
TagsNo tags attached.

Activities

gslongo

gslongo

2019-03-21 15:49

reporter   ~0003296

Hi,

I think I've found the error. This looks like implementation error. This looks related to the accurate implementation and the situation where 2 incrementals (different jobs but on same client, sharing same accurate informations) are run sequentially on the same Client.
As a try, I've inverted the backup order : Launching the AI before the original Incremental, and... The result is also inverted !

So, I created a "testmodif1.txt" file, then launching AI Job then Incr job with tracelog enabled and the result is :

root@pve-support:~# grep testmo /var/lib/bareos/pve-support-fd.trace.*
/var/lib/bareos/pve-support-fd.trace.ai:pve-support-fd (100): filed/accurate.cc:186-21381 accurate /root/testmodif1.txt (not found)
/var/lib/bareos/pve-support-fd.trace.incr:pve-support-fd (100): filed/accurate_htable.cc:87-21382 add fname=</root/testmodif1.txt> lstat=P0A DAAKQ IGk B A A A A BAA A Bck55J Bck55J Bck55J A A d delta_seq=0 chksum=
/var/lib/bareos/pve-support-fd.trace.incr:pve-support-fd (100): filed/accurate.cc:101-21382 lookup </root/testmodif1.txt> ok

As you can see, the file is not backed up on the second incr job.


Again, modified the testmodif1.txt file and launching AI then Incr job :

root@pve-support:~# grep testmod /var/lib/bareos/pve-support-fd.trace*
/var/lib/bareos/pve-support-fd.trace.ai:pve-support-fd (100): filed/accurate_htable.cc:87-21383 add fname=</root/testmodif1.txt> lstat=P0A DAAKQ IGk B A A A A BAA A Bck55J Bck55J Bck55J A A d delta_seq=0 chksum=
/var/lib/bareos/pve-support-fd.trace.ai:pve-support-fd (100): filed/accurate.cc:101-21383 lookup </root/testmodif1.txt> ok
/var/lib/bareos/pve-support-fd.trace.ai:pve-support-fd (99): filed/accurate.cc:276-21383 /root/testmodif1.txt st_mtime differs
/var/lib/bareos/pve-support-fd.trace.ai:pve-support-fd (100): findlib/bfile.cc:1101-21383 bopen: fname /root/testmodif1.txt, flags 00000000, mode 0000, rdev 0
/var/lib/bareos/pve-support-fd.trace.incr:pve-support-fd (100): filed/accurate_htable.cc:87-21384 add fname=</root/testmodif1.txt> lstat=P0A DAAKW IGk B A A A F BAA I Bck6Aw Bck6Aw Bck6Aw A A d delta_seq=0 chksum=
/var/lib/bareos/pve-support-fd.trace.incr:pve-support-fd (100): filed/accurate.cc:101-21384 lookup </root/testmodif1.txt> ok


Then as you can see, as soon as the file have been backed up a first time, it will never be backed up again even if the job is not the same. As a result : Inconcistent backups

I know it should not happen with a proper jobs setup but in some cases, it could.
bruno-at-bareos

bruno-at-bareos

2023-09-13 11:40

manager   ~0005432

Hello cleaning our bug db entries, we would like to know if you can still reproduce this with a recent version like 22.1.0?
Regards
bruno-at-bareos

bruno-at-bareos

2023-09-13 11:47

manager   ~0005433

Should work normally if you adapt your configuration to the rule
one-client-one-fileset-> give one job, one accurate entity.

So if you double your fileset, then inc and AI inc should react the same.
bruno-at-bareos

bruno-at-bareos

2023-10-11 16:23

manager   ~0005465

configuration issue

Issue History

Date Modified Username Field Change
2019-03-20 14:22 gslongo New Issue
2019-03-21 15:49 gslongo Note Added: 0003296
2023-09-13 11:40 bruno-at-bareos Assigned To => bruno-at-bareos
2023-09-13 11:40 bruno-at-bareos Status new => feedback
2023-09-13 11:40 bruno-at-bareos Note Added: 0005432
2023-09-13 11:47 bruno-at-bareos Note Added: 0005433
2023-10-11 16:23 bruno-at-bareos Status feedback => closed
2023-10-11 16:23 bruno-at-bareos Resolution open => no change required
2023-10-11 16:23 bruno-at-bareos Note Added: 0005465