View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0001574 | bareos-core | General | public | 2023-11-28 08:06 | 2023-11-29 09:32 |
Reporter | irekpias | Assigned To | bruno-at-bareos | ||
Priority | normal | Severity | minor | Reproducibility | have not tried |
Status | closed | Resolution | no change required | ||
Platform | Linux | OS | RHEL (and clones) | OS Version | 9 |
Summary | 0001574: After switch to bareos 23.0.0~pre1269.3ddd57cb0 - incremential backup job backups all files in file set | ||||
Description | I have upgrade to bareos 23.0.0~pre1269.3ddd57cb0 then first incremental job of Windows client (with the same version 23.0.0~pre1269.3ddd57cb0) took all files from file set (like full backup). Log states that was Incremential backup. Then i ESTIMATE next incremential backup and it will take 0 files (as intended). In Saturday there will be differential backup, and ESTIMATE commands shows, that it will take all files (like full backup). I didn't change file sets or source structure of folders to backup - is the same as always. In bareos log i have this as new thing: JobId 827: (D:\)\\?\Volume{646a421e-93d2-4fb9-9e35-94be49251c94}\ -> \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy22 - this is normal ? | ||||
Steps To Reproduce | - | ||||
Additional Information | - | ||||
Tags | No tags attached. | ||||
Maybe sharing the job log, also the estimate command and its output with option listing would help Don't forget to re-add the level when you issue an estimate command (estimate will not check what the default level will be). estimate job=XXX level=Differential listing |
|
Here You go: 2023-11-27 22:12:06 bareos-dir JobId 826: Start Backup JobId 826, Job=backup-serwer.2023-11-27_22.12.04_39 2023-11-27 22:12:06 bareos-dir JobId 826: Connected Storage daemon at 192.168.x.x:9103, encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3 2023-11-27 22:12:06 bareos-dir JobId 826: Encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3 2023-11-27 22:12:06 bareos-dir JobId 826: Connected Client: serwer-fd at 192.168.100.251:9102, encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3 2023-11-27 22:12:06 bareos-dir JobId 826: Handshake: Immediate TLS 2023-11-27 22:12:06 bareos-dir JobId 826: Encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3 2023-11-27 22:12:06 bareos-dir JobId 826: Sending Accurate information. 2023-11-27 22:12:08 bareos-dir JobId 826: Using Device "FileStorage3" to write. 2023-11-27 22:12:06 serwer-fd JobId 826: Created 26 wildcard excludes from FilesNotToBackup Registry key 2023-11-27 22:12:08 serwer-fd JobId 826: Connected Storage daemon at 192.168.x.x:9103, encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3 2023-11-27 22:12:08 serwer-fd JobId 826: Encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3 2023-11-27 22:12:08 bareos-sd JobId 826: Recycled volume "Inc-backup-serwer-11" on device "FileStorage3" (/mnt/storage1/BareOS-data/serwer), all previous data lost. 2023-11-27 22:12:09 serwer-fd JobId 826: Generate VSS snapshots. Driver="Win64 VSS" 2023-11-27 22:12:09 serwer-fd JobId 826: VolumeMountpoints are not processed as onefs = yes. 2023-11-27 22:12:11 serwer-fd JobId 826: (D:\)\\?\Volume{646a421e-93d2-4fb9-9e35-94be49251c94}\ -> \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy21 2023-11-27 22:54:14 bareos-sd JobId 826: User defined maximum volume capacity 85,899,345,920 exceeded on device "FileStorage3" (/mnt/storage1/BareOS-data/serwer). 2023-11-27 22:54:14 bareos-sd JobId 826: End of medium on Volume "Inc-backup-serwer-11" Bytes=85,898,297,522 Blocks=81,919 at 27-lis-2023 22:54. 2023-11-27 22:54:15 bareos-dir JobId 826: Volume "Inc-backup-serwer-12" has Volume Retention of 10022400 sec. and has 1 jobs that will be pruned 2023-11-27 22:54:15 bareos-dir JobId 826: Purging the following 1 JobIds: 357 2023-11-27 22:54:15 bareos-dir JobId 826: Created new Volume "Inc-backup-serwer-18" in catalog. 2023-11-27 22:54:15 bareos-sd JobId 826: Labeled new Volume "Inc-backup-serwer-18" on device "FileStorage3" (/mnt/storage1/BareOS-data/serwer). 2023-11-27 22:54:15 bareos-sd JobId 826: Wrote label to prelabeled Volume "Inc-backup-serwer-18" on device "FileStorage3" (/mnt/storage1/BareOS-data/serwer) 2023-11-27 22:54:15 bareos-sd JobId 826: New volume "Inc-backup-serwer-18" mounted on device "FileStorage3" (/mnt/storage1/BareOS-data/serwer) at 27-lis-2023 22:54. 2023-11-27 23:16:00 serwer-fd JobId 826: VSS Writer (BackupComplete): "Task Scheduler Writer", State: 0x1 (VSS_WS_STABLE) 2023-11-27 23:16:00 serwer-fd JobId 826: VSS Writer (BackupComplete): "VSS Metadata Store Writer", State: 0x1 (VSS_WS_STABLE) 2023-11-27 23:16:00 serwer-fd JobId 826: VSS Writer (BackupComplete): "Performance Counters Writer", State: 0x1 (VSS_WS_STABLE) 2023-11-27 23:16:00 serwer-fd JobId 826: VSS Writer (BackupComplete): "System Writer", State: 0x1 (VSS_WS_STABLE) 2023-11-27 23:16:00 serwer-fd JobId 826: VSS Writer (BackupComplete): "FSRM Writer", State: 0x1 (VSS_WS_STABLE) 2023-11-27 23:16:00 serwer-fd JobId 826: VSS Writer (BackupComplete): "ASR Writer", State: 0x1 (VSS_WS_STABLE) 2023-11-27 23:16:00 serwer-fd JobId 826: VSS Writer (BackupComplete): "WMI Writer", State: 0x1 (VSS_WS_STABLE) 2023-11-27 23:16:00 serwer-fd JobId 826: VSS Writer (BackupComplete): "DFS Replication service writer", State: 0x1 (VSS_WS_STABLE) 2023-11-27 23:16:00 serwer-fd JobId 826: VSS Writer (BackupComplete): "MSSearch Service Writer", State: 0x1 (VSS_WS_STABLE) 2023-11-27 23:16:00 serwer-fd JobId 826: VSS Writer (BackupComplete): "Dhcp Jet Writer", State: 0x1 (VSS_WS_STABLE) 2023-11-27 23:16:00 serwer-fd JobId 826: VSS Writer (BackupComplete): "Registry Writer", State: 0x1 (VSS_WS_STABLE) 2023-11-27 23:16:00 serwer-fd JobId 826: VSS Writer (BackupComplete): "Shadow Copy Optimization Writer", State: 0x1 (VSS_WS_STABLE) 2023-11-27 23:16:00 serwer-fd JobId 826: VSS Writer (BackupComplete): "NTDS", State: 0x1 (VSS_WS_STABLE) 2023-11-27 23:16:00 serwer-fd JobId 826: VSS Writer (BackupComplete): "COM+ REGDB Writer", State: 0x1 (VSS_WS_STABLE) 2023-11-27 23:16:00 bareos-sd JobId 826: Releasing device "FileStorage3" (/mnt/storage1/BareOS-data/serwer). 2023-11-27 23:16:08 bareos-sd JobId 826: Elapsed time=01:03:59, Transfer rate=35.57 M Bytes/second 2023-11-27 23:16:08 bareos-dir JobId 826: Insert of attributes batch table with 238764 entries start 2023-11-27 23:16:08 bareos-dir JobId 826: Insert of attributes batch table done 2023-11-27 23:16:08 bareos-dir JobId 826: Bareos bareos-dir 23.0.0~pre1269.3ddd57cb0 (27Nov23): Build OS: Red Hat Enterprise Linux release 9.1 (Plow) JobId: 826 Job: backup-serwer.2023-11-27_22.12.04_39 Backup Level: Incremental, since=2023-11-26 16:00:03 Client: "serwer-fd" 23.0.0~pre1269.3ddd57cb0 (27Nov23) Microsoft Windows Server 2012 Standard Edition (build 9200), 64-bit,Cross-compile FileSet: "serwer-Pliki" 2023-05-08 22:30:39 Pool: "Inc-serwer" (From Job IncPool override) Catalog: "MyCatalog" (From Client resource) Storage: "FileStorage3" (From Job resource) Scheduled time: 27-lis-2023 22:11:45 Start time: 27-lis-2023 22:12:06 End time: 27-lis-2023 23:16:08 Elapsed time: 1 hour 4 mins 2 secs Priority: 10 Allow Mixed Priority: no FD Files Written: 238,765 SD Files Written: 238,765 FD Bytes Written: 136,492,552,648 (136.4 GB) SD Bytes Written: 136,554,979,330 (136.5 GB) Rate: 35526.4 KB/s Software Compression: 43.4 % (lz4hc) VSS: yes Encryption: no Accurate: yes Volume name(s): Inc-backup-serwer-11|Inc-backup-serwer-18 Volume Session Id: 7 Volume Session Time: 1701086140 Last Volume Bytes: 50,680,044,428 (50.68 GB) Non-fatal FD errors: 0 SD Errors: 0 FD termination status: OK SD termination status: OK Bareos binary info: Bareos community build (UNSUPPORTED): Get professional support from https://www.bareos.com Job triggered by: User Termination: Backup OK When i switch back to version 22.1.1-pre and for test i ran ESTIMATE command incremental job, again bareos want in INCREMENTAL job backup again all files (nothing changes in files from the last incremental backup) - not only changes files. This is wired for me. |
|
We retried here, and when a full job is present when starting a new incremental after doing an upgrade from 22 to 23, the incremental backup only the changes and not everything. So we completely are unable to reproduce your issue. If you have, please share the output of "list jobs job=backup-serwer" |
|
| 723 | backup-serwer | serwer-fd | 2023-11-05 16:00:02 | 01:40:59 | B | F | 237,991 | 138,328,726,452 | T | | 726 | backup-serwer | serwer-fd | 2023-11-06 12:00:02 | 00:00:21 | B | I | 121 | 69,924,040 | T | | 727 | backup-serwer | serwer-fd | 2023-11-06 18:00:02 | 00:00:19 | B | I | 67 | 5,845,997 | T | | 730 | backup-serwer | serwer-fd | 2023-11-07 12:00:02 | 00:00:18 | B | I | 151 | 9,927,264 | T | | 731 | backup-serwer | serwer-fd | 2023-11-07 18:00:02 | 00:00:18 | B | I | 120 | 4,371,643 | T | | 734 | backup-serwer | serwer-fd | 2023-11-08 12:00:02 | 00:00:20 | B | I | 21 | 1,777,733 | T | | 735 | backup-serwer | serwer-fd | 2023-11-08 18:00:02 | 00:00:18 | B | I | 52 | 5,016,521 | T | | 738 | backup-serwer | serwer-fd | 2023-11-09 12:00:03 | 00:00:18 | B | I | 26 | 4,478,792 | T | | 739 | backup-serwer | serwer-fd | 2023-11-09 18:00:03 | 00:00:19 | B | I | 37 | 13,712,270 | T | | 742 | backup-serwer | serwer-fd | 2023-11-10 12:00:02 | 00:00:20 | B | I | 11 | 1,970,901 | T | | 743 | backup-serwer | serwer-fd | 2023-11-10 18:00:03 | 00:00:18 | B | I | 46 | 4,212,333 | T | | 746 | backup-serwer | serwer-fd | 2023-11-11 12:00:02 | 00:00:21 | B | I | 1 | 0 | T | | 747 | backup-serwer | serwer-fd | 2023-11-11 18:00:02 | 00:00:21 | B | I | 1 | 0 | T | | 750 | backup-serwer | serwer-fd | 2023-11-12 16:00:02 | 00:00:30 | B | D | 438 | 108,267,598 | T | | 756 | backup-serwer | serwer-fd | 2023-11-13 12:00:02 | 00:00:19 | B | I | 37 | 17,303,089 | T | | 757 | backup-serwer | serwer-fd | 2023-11-13 18:00:03 | 00:00:18 | B | I | 60 | 11,636,589 | T | | 760 | backup-serwer | serwer-fd | 2023-11-14 12:00:02 | 00:00:19 | B | I | 147 | 4,840,049 | T | | 761 | backup-serwer | serwer-fd | 2023-11-14 18:00:03 | 00:00:19 | B | I | 78 | 2,222,576 | T | | 764 | backup-serwer | serwer-fd | 2023-11-15 12:00:02 | 00:00:23 | B | I | 57 | 7,306,432 | T | | 765 | backup-serwer | serwer-fd | 2023-11-15 18:00:02 | 00:00:20 | B | I | 66 | 8,407,306 | T | | 768 | backup-serwer | serwer-fd | 2023-11-16 12:00:02 | 00:00:24 | B | I | 38 | 3,766,539 | T | | 769 | backup-serwer | serwer-fd | 2023-11-16 18:00:02 | 00:00:19 | B | I | 71 | 6,292,296 | T | | 772 | backup-serwer | serwer-fd | 2023-11-17 12:00:02 | 00:00:22 | B | I | 44 | 25,440,405 | T | | 773 | backup-serwer | serwer-fd | 2023-11-17 18:00:02 | 00:00:19 | B | I | 26 | 11,066,763 | T | | 777 | backup-serwer | serwer-fd | 2023-11-18 12:00:02 | 00:00:19 | B | I | 1 | 0 | T | | 778 | backup-serwer | serwer-fd | 2023-11-18 18:00:02 | 00:00:18 | B | I | 1 | 0 | T | | 781 | backup-serwer | serwer-fd | 2023-11-19 16:00:03 | 00:00:40 | B | D | 796 | 190,538,546 | T | | 784 | backup-serwer | serwer-fd | 2023-11-20 12:00:02 | 00:00:20 | B | I | 53 | 7,549,470 | T | | 785 | backup-serwer | serwer-fd | 2023-11-20 18:00:02 | 00:00:18 | B | I | 49 | 18,784,724 | T | | 788 | backup-serwer | serwer-fd | 2023-11-21 12:00:02 | 00:00:20 | B | I | 144 | 8,470,658 | T | | 789 | backup-serwer | serwer-fd | 2023-11-21 18:00:02 | 00:00:18 | B | I | 33 | 6,674,134 | T | | 792 | backup-serwer | serwer-fd | 2023-11-22 12:00:02 | 00:00:28 | B | I | 47 | 34,222,084 | T | | 793 | backup-serwer | serwer-fd | 2023-11-22 18:00:02 | 00:00:19 | B | I | 39 | 2,851,189 | T | | 796 | backup-serwer | serwer-fd | 2023-11-23 12:00:02 | 00:00:28 | B | I | 46 | 40,461,990 | T | | 797 | backup-serwer | serwer-fd | 2023-11-23 18:00:02 | 00:00:21 | B | I | 40 | 20,681,952 | T | | 800 | backup-serwer | serwer-fd | 2023-11-24 12:00:02 | 00:00:25 | B | I | 163 | 57,059,142 | T | | 801 | backup-serwer | serwer-fd | 2023-11-24 18:00:02 | 00:00:22 | B | I | 133 | 22,588,994 | T | | 812 | backup-serwer | serwer-fd | 2023-11-25 12:00:02 | 00:00:31 | B | I | 3 | 1,278,262 | T | | 813 | backup-serwer | serwer-fd | 2023-11-25 18:00:03 | 00:00:18 | B | I | 1 | 0 | T | | 816 | backup-serwer | serwer-fd | 2023-11-26 16:00:03 | 00:00:54 | B | D | 1,340 | 363,681,471 | T | | 826 | backup-serwer | serwer-fd | 2023-11-27 22:12:06 | 01:04:02 | B | I | 238,765 | 136,492,552,648 | T | | 827 | backup-serwer | serwer-fd | 2023-11-27 23:48:42 | 00:00:11 | B | I | 1 | 0 | T | | 828 | backup-serwer | serwer-fd | 2023-11-28 12:00:02 | 00:00:13 | B | I | 35 | 2,531,113 | T | +-------+-------------------+---------------+---------------------+----------+------+-------+----------+-----------------+-----------+ And estimate job as Differential: *estimate jobid=backup-serwer type=differential Connecting to Client serwer-fd at 192.168.x.x:9102 2000 OK estimate files=227,846 bytes=241,150,017,585 So in Sunday i will have a "Full" backup in Differential job. This is strange: 2023-11-27 22:12:11 serwer-fd JobId 826: (D:\)\\?\Volume{646a421e-93d2-4fb9-9e35-94be49251c94}\ -> \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy21 - never sen before. Maybe problem lies in VSS of bareos-client in this version ? |
|
Don't worry about the line it was existing but only show for drive, now we increase the level of details. So that is completely expected. Could you please share your fileset, maybe some parameters in have their default changed, like the hardlink like state in the release note https://docs.bareos.org/Appendix/ReleaseNotes.html#breaking-changes Hard Links (Dir->Fileset->Include->Options) = no (was yes) it would be really interesting to check the difference between files backup during the full, the diff and then the incremental. To identify such a file you can extract the list of files for the job bconsole <<< "@out /tmp/full_files.list list file jobid=723" bconsole <<< "@out /tmp/diff_files.list list file jobid=816" bconsole <<< "@out /tmp/incr_files.list list file jobid=826" In those lists identify a few files present in the 3 jobs For example if you find something like D:/samples/tests/fileA the query will be the following bconsole <<< "@out /tmp/file_differences.log sqlquery select f.jobid, p.path, f.name, ls.*, md5 from file f, path p, decode_lstat(f.lstat) ls where f.jobid in (723,816,826) and f.pathid = p.pathid and f.name = 'fileA' order by ; " Attach the resulting log file here (can be set to private). |
|
Thanks for your report. Yes Windows tools often have the tendency to change file content and or attribute while it would not be expected by users, and administrator. To Summarize your incident, activating Windows search on user data, after making a full, will create changes on file, and as it is its design Bareos will backup that new version of the files. We wish you nice backups & restores. |
|
Date Modified | Username | Field | Change |
---|---|---|---|
2023-11-28 08:06 | irekpias | New Issue | |
2023-11-28 09:22 | bruno-at-bareos | Note Added: 0005539 | |
2023-11-28 10:31 | irekpias | Note Added: 0005540 | |
2023-11-28 13:51 | bruno-at-bareos | Note Added: 0005541 | |
2023-11-28 14:41 | irekpias | Note Added: 0005542 | |
2023-11-28 16:00 | bruno-at-bareos | Note Added: 0005543 | |
2023-11-29 09:32 | bruno-at-bareos | Assigned To | => bruno-at-bareos |
2023-11-29 09:32 | bruno-at-bareos | Status | new => closed |
2023-11-29 09:32 | bruno-at-bareos | Resolution | open => no change required |
2023-11-29 09:32 | bruno-at-bareos | Note Added: 0005546 |