View Issue Details
|ID||Project||Category||View Status||Date Submitted||Last Update|
|0000907||bareos-core||[All Projects] file daemon||public||2018-02-09 10:59||2018-11-20 13:25|
|Priority||normal||Severity||major||Reproducibility||have not tried|
|Target Version||Fixed in Version|
|Summary||0000907: Incremental Job backups same files|
|Description||Director is running on CentOS 7, FD on Windows Server 2016.|
Jobtype: always incremental;
Bug: Some files (~ 180) were copied on every job.
22401 Incr 2,187 105.1 G OK 04-Feb-18 01:24 FS01-AI
22421 Incr 1,143 103.8 G OK 05-Feb-18 08:35 FS01-AI
22495 Incr 2,515 105.3 G OK 09-Feb-18 07:13 FS01-AI
In the database you can see that the files have not changed between backups:
bareos=# select * from file where pathid='5305263' and name='foo.bar' and jobid=22495;
fileid | fileindex | jobid | pathid | deltaseq | markid | fhinfo | fhnode | lstat | md5 | name
252600154 | 2292 | 22495 | 5305263 | 0 | 0 | 0 | 0 | A A IH/ A A A BAAAYg ePLa A A BOh02e BN3PJg BZFeCr A A f | VsVr09AEXA85Ywpk6aW+8w | foo.bar
252163936 | 1010 | 22421 | 5305263 | 0 | 0 | 0 | 0 | A A IH/ A A A BAAAYg ePLa A A BOh02e BN3PJg BZFeCr A A f | VsVr09AEXA85Ywpk6aW+8w | foo.bar
252090699 | 1934 | 22401 | 5305263 | 0 | 0 | 0 | 0 | A A IH/ A A A BAAAYg ePLa A A BOh02e BN3PJg BZFeCr A A f | VsVr09AEXA85Ywpk6aW+8w | foo.bar
|Tags||No tags attached.|
Anything special you know about the affected files?
Some filesystems behave oddly, react on access time ...
You can configure, what attributes are evaluated on accurate mode by the fileset include option "accurate", see http://doc.bareos.org/master/html/bareos-manual-main-reference.html#FileSetIncludeRessource
If you are only interested in the file contents, setting
accurate = 5
might be an option.
Joerg, thanks for your reply.
Affected files were pst(s), media files and binaries but no M$ office files (which are the most on this server / fileset). The File system is a ntfs on a Server 2016.
I understand that it is a possible workaround to reduce the attribut set which decides about if a file has changed or not. I will give it a try.
Please correct me if I am wrong but from my understanding of bareos the decision of backingup a file or not is done by the catalog entries. The entries in the catalog (see my database dump in the bug report) for the files which were all the time backed up were the same so there must be a bug in the decision process.
Normal backup is based only on timestamps (backup all files that have been modified after the last backup).
As you are using Always Incremental, I assume you are using accurate backup, where the backup decision is really based on the catalog data.
My proposal can be seen as workaround, but it has been intended as a way to track down the problem. If it works with some settings, we can determine which attributes are treated wrong.
Anyhow, there are also other possible reason:
maybe there have been job runs in between, where the data have not been present (volume not mounted, ...). In this case the in between job has find out that foo.bar has not been available in between. As it had been available afterwards, it has been backuped again.
Hi, i have same problem on two servers with Ubuntu 16.04 and BareOS 17.2.4.
I have 2 milions files, which are backuped with incremental job again (ALL) after update Bareos from version 15.2 to 17.2.
But this do only on BIG backups (2 and 3 milions files).
I saw, that upgrade of database from version 2004 to new 2171 merge somehow tables File and Filename. Maybe here is some problem.
In restored environment back to version 15.2 i have in table File 2 848 506 rows (size 1,2 GiB) and in table FileName 2 297 827 rows (337 MiB). After upgrade to 17.2 has table File 2GiB and incremental backup did backup all.
In another environment in version 17.2 i have 14 600 000 rows (size 3,6GiB) - there i have problem.
I have the same problem, I have a simple incremental backup job for a web server's almost static files (php, js, pictures). Every time all files are backup up. There is 1,7 millions files.
The environment is:
OS: Debian GNU/Linux 9.4 (stretch)
Bareos: Version: 17.2.4 (21 Sep 2017)
We have 99 Jobs on quite identical machines. All Linux. All ext3 or ext4.
*Two* of these jobs fail and do a full backup each incremental "accurate" backup. Both have over 4 million files. All other 97 jobs have < 2 million files and work just fine!
Disabling accurate fixes this.
I have looked into the source code (filed/accurate.c) and found that the init for the MDB accurate database is called and then the return code (which could false) is *completely ignored*:
jcr->accurate = true;
It should read:
if (!jcr->file_list->init(jcr, nb))
// emit error message, possibly informing director: dir->fsend(_("2991 Failed to allocate memory');
I don't know if this fixes this issue, it just points to a missing error handling...
OS: Gentoo Linux, Ubuntu Linux, Debian Linux, ...
Bareos: Various versions, most of them 17.2.4 and 17.2.7
|2018-02-09 10:59||mbr||New Issue|
|2018-02-09 19:58||joergs||Note Added: 0002903|
|2018-02-09 19:58||joergs||Status||new => feedback|
|2018-02-12 14:38||mbr||Note Added: 0002906|
|2018-02-12 14:38||mbr||Status||feedback => new|
|2018-02-12 16:39||joergs||Note Added: 0002907|
|2018-02-12 16:39||joergs||Status||new => feedback|
|2018-02-12 16:40||joergs||bareos-master: impact||=> yes|
|2018-02-12 16:40||joergs||bareos-17.2: impact||=> yes|
|2018-02-12 16:40||joergs||Category||director => file daemon|
|2018-02-12 16:40||joergs||OS||CentOS => Windows|
|2018-02-12 16:40||joergs||OS Version||7 => 2016|
|2018-03-05 16:17||romor||Note Added: 0002938|
|2018-07-05 23:25||beckzg||Note Added: 0003062|
|2018-11-20 13:25||kilroy||Note Added: 0003152|