View Issue Details
|ID||Project||Category||View Status||Date Submitted||Last Update|
|0001194||bareos-core||file daemon||public||2020-02-13 08:53||2021-12-21 14:04|
|Fixed in Version||21.0.0|
|Summary||0001194: Incremental backup does backup all files instead of changed files|
|Description||On 30.1.2020 the incremental backup on my Windows Server 2016 (64bit) suddenly did not backup the changed files anymore but did backup up all files instead. This happened ever since except one time on 4.2.2020 where it did a correct incremental backup.|
This change of behaviour does not correlate with Windows updates - the last time I installed Windows updates on the server was on 17.1.2020.
Also the Bareos server was not changed.
I am using the Accurate option for my backups.
in this bug https://bugs.bareos.org/view.php?id=907
I found the hint to use only MD5 hash for the accurate option so I tried this and changed my fileset configuration to "accurate = 5"
but it did not change the behaviour. The incremental backup is still backing up all files.
Backups of clients with Windows 10 still work correctly, so I assume that issues lies with the file daemon on Windows Server 2016.
Any ideas what this could be caused by?
Or any hint where to debug?
|Tags||No tags attached.|
First of all: please don't run 17.2 if you can help it.
As you have accurate enabled, it could be that the accurate flags are not transferred to the client correctly.
- can you successfully open the restore browser in bconsole using the restore command and then selecting option 5, and the client in question?
- how many files are "all files"?
Thank you for the quick feedback!
Upgrading from 17.2 is on my todo list but I didn't have the time to tackle that so far.
I can restore successfully for this client with bconsole.
All files are 8,786,819 files at the moment. The number of files didn't increase drastically, it was always about this amount for this client.
It would be great if 17.2 would work again, this would give me time to plan and execute the upgrade to 19.2 in a relaxed manner.
If you think an upgrade to 19.2 is the only solution for this problem I will have to reorganize my tasks and give this highest priority.
Since I didn't get any further feedback about how to proceed, I decided myself and worked over the weekend to update Bareos to the latest version 19.2.6
So far (for the last two nights) the incremental and differential backups work correctly again, so the update seems to have solved the issue.
But I still don't understand why Bareos changed it's behaviour all of a sudden in the first place...
I don't understand it either - but I'm also not eager to dig into old code to find an issue that has been fixed in the meantime.
While 19.2 has its own issues (and we're working on that) it is probably a much better choice than 17.2.
Is your problem fixed now? Can I close the issue?
Let me monitor the backup behaviour until end of the week just to be sure ...
I will give you feedback on Friday.
As the problem doesn't seem to occur in 19.2 anymore, I'll close the issue.
You can just reopen it if the issue resurfaces. Thank you!
I have to reopen this issue.
After upgrading to version 19.2.6 and then to version 19.2.7 (to fix issue 1192) a month ago this issue did not happen again until two days ago.
Now I see the same behaviour with version 19.2.7 as described before - the Incremental backup on my Windows Server 2016 (64bit) does backup all files instead of changed files.
There are two things that I could imagine happened:
1. the accurate filelist generated by the director and sent to the fd was truncated or empty
2. the previous incremental marked all files as deleted
As 2.) is easier to check, I'd like to rule that out first. Run a restore command to restore up to the latest incremental using "restore client before specified time" and take a look at the restore filetree that is generated. Do you see all files that you'd expect, or are (all) files missing?
If all files are there, then you're probably having an issue related to 1.), if files are missing then probably one of your previous incrementals went wrong.
I tested your option 2 and got this error from bconsole:
Query failed: DROP TABLE temp1 : ERR=ERROR: table "temp1" does not exist
For one or more of the JobIds selected, no files were found, so file selection is not possible.
Most likely your retention policy pruned the files.
This pointed me in the right direction. The partition of the Postgres database was low on disk space and therefore the table "temp1" could not be created.
After increasing the disk space for the Postgres database the incremental backup is working correctly again.
thank you very much!
Maybe the error logging of Bareos could be improved to log an error to the messages of the backup job when the creation of the accurate filelist fails?
I'm glad you could fix the issue.
I'm not sure there is a simple way to detect that situation. After all, if the database hits an anomalous condition (like a filled up filesystem) it can fails in any way it sees fit.
But I'll see if we can do something to improve the situation.
|The issue should be resolved (i.e. the job will now fail in that situation) in Bareos 21.|
|2020-02-13 08:53||Int||New Issue|
|2020-02-13 09:37||arogge||Assigned To||=> arogge|
|2020-02-13 09:37||arogge||Status||new => feedback|
|2020-02-13 09:37||arogge||Note Added: 0003805|
|2020-02-13 09:37||arogge||Relationship added||related to 0000907|
|2020-02-13 12:05||Int||Note Added: 0003809|
|2020-02-13 12:05||Int||Status||feedback => assigned|
|2020-02-18 09:08||Int||Note Added: 0003821|
|2020-02-18 10:32||arogge||Note Added: 0003822|
|2020-02-18 10:43||Int||Note Added: 0003823|
|2020-02-18 15:42||arogge||Status||assigned => resolved|
|2020-02-18 15:42||arogge||Resolution||open => unable to reproduce|
|2020-02-18 15:42||arogge||Note Added: 0003829|
|2020-06-04 10:15||Int||Status||resolved => new|
|2020-06-04 10:15||Int||Resolution||unable to reproduce => reopened|
|2020-06-04 10:15||Int||Note Added: 0003998|
|2020-06-04 10:20||arogge||Product Version||17.2.4 => 19.2.7|
|2020-06-04 10:55||arogge||Note Added: 0003999|
|2020-06-04 10:56||arogge||Status||new => feedback|
|2020-06-05 11:21||Int||Note Added: 0004002|
|2020-06-05 11:21||Int||Status||feedback => assigned|
|2020-06-08 10:46||arogge||Note Added: 0004003|
|2021-12-21 13:57||arogge||Relationship added||related to 0001289|
|2021-12-21 14:04||arogge||Status||assigned => resolved|
|2021-12-21 14:04||arogge||Fixed in Version||=> 21.0.0|
|2021-12-21 14:04||arogge||Note Added: 0004398|
|2021-12-21 14:04||arogge||Status||resolved => closed|