View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0001539 | bareos-core | General | public | 2023-06-10 20:19 | 2023-06-20 15:25 |
| Reporter | cmlara | Assigned To | bruno-at-bareos | ||
| Priority | normal | Severity | minor | Reproducibility | unable to reproduce |
| Status | closed | Resolution | fixed | ||
| Product Version | 22.0.3 | ||||
| Summary | 0001539: Incorrect startfile stored in jobmedia table prevents backup restoration or consolidation | ||||
| Description | When using S3 based storage and creating a full backup(or consolidation) an incorrect startfile may be written to the catalog preventing the recovery or restoration of a backup. Unknown at this time if this is a director or sd issue and if it is limited to just S3 storage. | ||||
| Steps To Reproduce | This is still a work in progress, however the pattern appears to be as follows currently: Start a backup will be stored to a S3 volume, the volume is to be created automatically during the job and labeled automatically. The backup for this job should fail such as a consolidation backup where the first source backup is corrupt (in my case it is the 'full' backup has the wrong startfile making this a self perpetuating error when combined with the next step) Create a new full backup (or a consolidated backup that success) that will be written to the same volume as above. Review the jobmedia table for the full backup and observe it has an incorrect startfile. Attempt to restore and receive a volume data error. Modify the startfile field in the database and have a successful restore. | ||||
| Additional Information | Partial Logs: 10-Jun 05:57 bareos-dir JobId 6704: Using Device "s3-us-east-2-01" to write. 10-Jun 05:57 s3-us-east-2-sd JobId 6704: Volume "S3-AI-Consolidated-0282" previously written, moving to end of data. 10-Jun 05:57 s3-us-east-2-sd JobId 6704: Ready to append to end of Volume "S3-AI-Consolidated-0282" size=258 .... 10-Jun 06:04 bareos-dir JobId 6704: Bareos bareos-dir 22.0.1~pre (21Dec22): Build OS: Ubuntu 20.04.4 LTS JobId: 6704 ... SD Files Written: 10,445 SD Bytes Written: 8,415,763,778 (8.415 GB) ... Last Volume Bytes: 8,424,262,058 (8.424 GB) SD Errors: 0 SD termination status: OK Accurate: yes Bareos binary info: Bareos community build (UNSUPPORTED): Get professional support from https://www.bareos.com Job triggered by: User Termination: Backup OK select * from jobmedia where jobid=6704 jobmediaid | jobid | mediaid | firstindex | lastindex | startfile | endfile | startblock | endblock | jobbytes | volindex ------------+-------+---------+------------+-----------+-----------+---------+------------+------------+----------+---------- 5168 | 6704 | 282 | 1 | 10445 | 1 | 1 | 258 | 4129294761 | 0 | 1 Key aspects on the issue is the startfile=1 where this should be startfile=0 because the backup starts at 258 bytes into the data file. I have attached a partially redacted hexdump of 0x0 through 0x01DF of the backup as stored in S3. Trying to restore from this job will return an error similar to: Error: stored/block.cc:290 Volume data error at 1:258! Wanted ID: "BB02", got "RANDOM_BINARY_DATA_HERE_AS_ASCII_TEXT". Buffer discarded. The "got" portion changes based on what is at the incorrect offset. If I manually update the jobmedia table to set startfile=0 a restore (or consolidation) will now work, which aligns to the fact that the backup appended to the end of the volume at size=258 Maybe related: 0000829 | ||||
| Tags | No tags attached. | ||||
 |
bareos-backup-headers-redacted.txt (2,371 bytes)
00000000 b5 d1 d0 03 00 00 01 02 00 00 00 00 42 42 30 32 |............BB02|
00000010 00 00 00 3d 64 76 7b 24 ff ff ff fe 00 00 00 00 |...=dv{$........|
00000020 00 00 00 de 42 61 72 65 6f 73 20 32 2e 30 20 69 |....Bareos 2.0 i|
00000030 6d 6d 6f 72 74 61 6c 0a 00 00 00 00 14 00 05 fd |mmortal.........|
00000040 b5 4d 1e 79 f2 00 05 fd b5 4d 1f ac 93 00 00 00 |.M.y.....M......|
00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 53 33 2d |.............S3-|
00000060 41 49 2d 43 6f 6e 73 6f 6c 69 64 61 74 65 64 2d |AI-Consolidated-|
00000070 30 32 38 32 00 00 53 33 2d 41 49 2d 43 6f 6e 73 |0282..S3-AI-Cons|
00000080 6f 6c 69 64 61 74 65 64 00 42 61 63 6b 75 70 00 |olidated.Backup.|
00000090 53 33 2d 75 73 2d 65 61 73 74 2d 32 2d XX XX XX |S3-us-east-2-XXX|
000000a0 XX XX XX XX XX 00 69 70 2d XX XX XX XX XX XX XX |XXXXX.ip-XXXXXXX|
000000b0 XX XX XX XX XX XX 00 73 33 2d 75 73 2d 65 61 73 |XXXXXX.s3-us-eas|
000000c0 74 2d 32 2d 73 64 00 56 65 72 2e 20 32 32 2e 30 |t-2-sd.Ver. 22.0|
000000d0 2e 31 7e 70 72 65 20 32 31 20 44 65 63 65 6d 62 |.1~pre 21 Decemb|
000000e0 65 72 20 32 30 32 32 00 42 75 69 6c 64 20 32 30 |er 2022.Build 20|
000000f0 32 32 2d 31 32 2d 32 31 20 31 30 3a 30 38 3a 35 |22-12-21 10:08:5|
00000100 38 00 88 b9 e3 02 00 00 fc 00 00 00 00 00 42 42 |8.............BB|
00000110 30 32 00 00 00 3e 64 76 7b 24 ff ff ff fc 00 00 |02...>dv{$......|
00000120 1a 30 00 00 00 b7 42 61 72 65 6f 73 20 32 2e 30 |.0....Bareos 2.0|
00000130 20 69 6d 6d 6f 72 74 61 6c 0a 00 00 00 00 14 00 | immortal.......|
00000140 00 1a 30 00 05 fd c0 27 89 30 29 00 00 00 00 00 |..0....'.0).....|
00000150 00 00 00 53 33 2d 41 49 2d 43 6f 6e 73 6f 6c 69 |...S3-AI-Consoli|
00000160 64 61 74 65 64 00 42 61 63 6b 75 70 00 65 6d 61 |dated.Backup.ema|
00000170 69 6c 62 61 63 6b 65 6e 64 32 00 65 6d 61 69 6c |ilbackend2.email|
00000180 62 61 63 6b 65 6e 64 32 2d 66 64 00 65 6d 61 69 |backend2-fd.emai|
00000190 6c 62 61 63 6b 65 6e 64 32 2e 32 30 32 33 2d 30 |lbackend2.2023-0|
000001a0 36 2d 31 30 5f 30 35 2e 35 36 2e 35 34 5f 33 30 |6-10_05.56.54_30|
000001b0 00 45 6d 61 69 6c 20 53 65 72 76 65 72 00 00 00 |.Email Server...|
000001c0 00 42 00 00 00 46 49 2b 6b 56 47 54 2f 39 48 32 |.B...FI+kVGT/9H2|
000001d0 68 31 56 44 6c 52 5a 34 2f 36 6f 43 00 00 00 00 |h1VDlRZ4/6oC....|
|
|
|
More complex, I was doing a set of COPY jobs and had the same happen for a new volume that did not previously exist and had no errors on it when the job rolled-over from one volume to another (max size on original volume) 10-Jun 18:50 s3-us-east-2-sd JobId 6727: End of medium on Volume "S3-AI-Longterm-0283" Bytes=26,843,526,939 Blocks=416,103 at 10-Jun-2023 18:50. 10-Jun 18:50 bareos-dir JobId 6727: Created new Volume "S3-AI-Longterm-0284" in catalog. 10-Jun 18:50 s3-us-east-2-sd JobId 6727: Labeled new Volume "S3-AI-Longterm-0284" on device "s3-us-east-2-02" (s3-us-east-2-02). 10-Jun 18:50 s3-us-east-2-sd JobId 6727: Wrote label to prelabeled Volume "S3-AI-Longterm-0284" on device "s3-us-east-2-02" (s3-us-east-2-02) 10-Jun 18:50 s3-us-east-2-sd JobId 6727: New volume "S3-AI-Longterm-0284" mounted on device "s3-us-east-2-02" (s3-us-east-2-02) at 10-Jun-2023 18:50. (6727 is the 'copy' job and 6728 is the job record for the created copy) select * from jobmedia where jobid='6728'; jobmediaid | jobid | mediaid | firstindex | lastindex | startfile | endfile | startblock | endblock | jobbytes | volindex ------------+-------+---------+------------+-----------+-----------+---------+------------+------------+----------+---------- 5174 | 6728 | 283 | 1 | 867 | 5 | 6 | 3273856821 | 1073723162 | 0 | 1 5175 | 6728 | 284 | 867 | 1682 | 1 | 1 | 250 | 908951103 | 0 | 2 I am having trouble finding a consistent pattern for reproduction. I do have trace logs from the SD if there is anything in them that I should be specifically looking for. I unfortunately did not have the Director trace logging enabled. I have attached a redacted copy of the askdir log lines as seen from the SD related to jobid 6727/6728 as they appear to be the most relevant for the catalog being updated. To me they do not appear different compared to a job that completes with correct StartFile/EndFile values. Perhaps this is a Director fault? bareos-copy-job-sd-dir-logs.txt (24,197 bytes)
Jun 10 18:48:27 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (199): stored/dir_cmd.cc:297-0 <dird: JobId=6727 job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 job_name=Copy#001S3#001AI#001Full client_name=s3-us-east-2-fd type=99 level=73 FileSet=Bigtime#001Personal NoAttr=0 SpoolAttr=0 FileSetMD5=E++89g/msB9jI++nEisctB SpoolData=0 PreferMountedVols=1 SpoolSize=0 rerunning=0 VolSessionId=0 VolSessionTime=0 Quota=0 Protocol=0 BackupFormat=Native Jun 10 18:48:27 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (100): stored/job.cc:89-0 <dird: JobId=6727 job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 job_name=Copy#001S3#001AI#001Full client_name=s3-us-east-2-fd type=99 level=73 FileSet=Bigtime#001Personal NoAttr=0 SpoolAttr=0 FileSetMD5=E++89g/msB9jI++nEisctB SpoolData=0 PreferMountedVols=1 SpoolSize=0 rerunning=0 VolSessionId=0 VolSessionTime=0 Quota=0 Protocol=0 BackupFormat=Native Jun 10 18:48:27 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/job.cc:166-6727 >dird jid=6727: 3000 OK Job SDid=74 SDtime=1685486372 Authorization=REDACTED Jun 10 18:48:27 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (199): stored/dir_cmd.cc:297-6727 <dird: bootstrap Jun 10 18:48:27 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (199): stored/dir_cmd.cc:297-6727 <dird: getSecureEraseCmd Jun 10 18:48:27 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (199): stored/dir_cmd.cc:297-6727 <dird: use storage=s3-us-east-2-01 media_type=S3-us-east-2-XXXXXXXX pool_name=S3-AI-Consolidated pool_type=Backup append=0 copy=0 stripe=0 Jun 10 18:48:27 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (150): stored/reserve.cc:206-6727 <dird: use storage=s3-us-east-2-01 media_type=S3-us-east-2-XXXXXXXX pool_name=S3-AI-Consolidated pool_type=Backup append=0 copy=0 stripe=0 Jun 10 18:48:27 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (150): stored/reserve.cc:235-6727 <dird device: use device=s3-us-east-2-01 Jun 10 18:48:27 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (150): stored/reserve.cc:791-6727 >dird: 3000 OK use device device=s3-us-east-2-01 Jun 10 18:48:27 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (199): stored/dir_cmd.cc:297-6727 <dird: use storage=s3-us-east-2-02 media_type=S3-us-east-2-XXXXXXXX pool_name=S3-AI-Longterm pool_type=Backup append=1 copy=0 stripe=0 Jun 10 18:48:27 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (150): stored/reserve.cc:206-6727 <dird: use storage=s3-us-east-2-02 media_type=S3-us-east-2-XXXXXXXX pool_name=S3-AI-Longterm pool_type=Backup append=1 copy=0 stripe=0 Jun 10 18:48:27 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (150): stored/reserve.cc:235-6727 <dird device: use device=s3-us-east-2-02 Jun 10 18:48:27 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:231-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 FindMedia=1 pool_name=S3-AI-Longterm media_type=S3-us-east-2-XXXXXXXX unwanted_volumes= Jun 10 18:48:27 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Longterm-0283 VolJobs=6 VolFiles=5 VolBlocks=383631 VolBytes=24748693301 VolMounts=6 VolErrors=0 VolWrites=383632 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=423581718 EndFile=5 EndBlock=3273856820 LabelType=0 MediaId=283 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:48:27 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (150): stored/reserve.cc:791-6727 >dird: 3000 OK use device device=s3-us-east-2-02 Jun 10 18:48:28 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (199): stored/dir_cmd.cc:297-6727 <dird: runs3-us-east-2-sd (200): stored/dir_cmd.cc:311-6727 Do command: run Jun 10 18:48:28 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:184-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 GetVolInfo VolName=S3-AI-Longterm-0283 write=1 Jun 10 18:48:28 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Longterm-0283 VolJobs=6 VolFiles=5 VolBlocks=383631 VolBytes=24748693301 VolMounts=6 VolErrors=0 VolWrites=383632 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=423581718 EndFile=5 EndBlock=3273856820 LabelType=0 MediaId=283 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:48:30 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:184-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 GetVolInfo VolName=S3-AI-Longterm-0283 write=1 Jun 10 18:48:30 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Longterm-0283 VolJobs=6 VolFiles=5 VolBlocks=383631 VolBytes=24748693301 VolMounts=6 VolErrors=0 VolWrites=383632 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=423581718 EndFile=5 EndBlock=3273856820 LabelType=0 MediaId=283 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:48:32 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:318-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 UpdateMedia VolName=S3-AI-Longterm-0283 VolJobs=6 VolFiles=5 VolBlocks=383631 VolBytes=24748693301 VolMounts=7 VolErrors=0 VolWrites=383632 MaxVolBytes=26843545600 EndTime=1686422912 VolStatus=Append Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=423581718 VolFirstWritten=0 Jun 10 18:48:32 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Longterm-0283 VolJobs=6 VolFiles=5 VolBlocks=383631 VolBytes=24748693301 VolMounts=7 VolErrors=0 VolWrites=383632 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=423581718 EndFile=5 EndBlock=3273856820 LabelType=0 MediaId=283 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:48:32 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:318-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 UpdateMedia VolName=S3-AI-Longterm-0283 VolJobs=7 VolFiles=5 VolBlocks=383631 VolBytes=24748693301 VolMounts=7 VolErrors=0 VolWrites=383632 MaxVolBytes=26843545600 EndTime=1686422912 VolStatus=Append Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=423581718 VolFirstWritten=0 Jun 10 18:48:32 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Longterm-0283 VolJobs=7 VolFiles=5 VolBlocks=383631 VolBytes=24748693301 VolMounts=7 VolErrors=0 VolWrites=383632 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=423581718 EndFile=5 EndBlock=3273856820 LabelType=0 MediaId=283 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:48:32 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:184-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 GetVolInfo VolName=S3-AI-Consolidated-0195 write=0 Jun 10 18:48:32 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Consolidated-0195 VolJobs=2 VolFiles=1 VolBlocks=114815 VolBytes=7406913160 VolMounts=2 VolErrors=0 VolWrites=114816 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Used Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=210 VolWriteTime=143555767 EndFile=1 EndBlock=3111945863 LabelType=0 MediaId=195 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:49:16 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (199): stored/dir_cmd.cc:297-0 <dird: setdebug=255 trace=0 timestamp=-1 Jun 10 18:49:18 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:367-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 CreateJobMedia FirstIndex=1 LastIndex=800 StartFile=5 EndFile=5 StartBlock=3273856821 EndBlock=4273792801 Copy=0 Strip=0 MediaId=283 Jun 10 18:49:18 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:375-6727 <dird 1000 OK CreateJobMedia Jun 10 18:49:18 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:318-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 UpdateMedia VolName=S3-AI-Longterm-0283 VolJobs=7 VolFiles=5 VolBlocks=399131 VolBytes=25748629282 VolMounts=7 VolErrors=0 VolWrites=399132 MaxVolBytes=26843545600 EndTime=1686422958 VolStatus=Append Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=439350663 VolFirstWritten=0 Jun 10 18:49:18 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Longterm-0283 VolJobs=7 VolFiles=5 VolBlocks=399131 VolBytes=25748629282 VolMounts=7 VolErrors=0 VolWrites=399132 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=439350663 EndFile=5 EndBlock=4273792801 LabelType=0 MediaId=283 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:50:02 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:367-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 CreateJobMedia FirstIndex=800 LastIndex=858 StartFile=5 EndFile=6 StartBlock=4273792802 EndBlock=978761498 Copy=0 Strip=0 MediaId=283 Jun 10 18:50:02 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:375-6727 <dird 1000 OK CreateJobMedia Jun 10 18:50:02 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:318-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 UpdateMedia VolName=S3-AI-Longterm-0283 VolJobs=7 VolFiles=6 VolBlocks=414631 VolBytes=26748565275 VolMounts=7 VolErrors=0 VolWrites=414632 MaxVolBytes=26843545600 EndTime=1686423002 VolStatus=Append Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=457102244 VolFirstWritten=0 Jun 10 18:50:02 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Longterm-0283 VolJobs=7 VolFiles=6 VolBlocks=414631 VolBytes=26748565275 VolMounts=7 VolErrors=0 VolWrites=414632 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=457102244 EndFile=6 EndBlock=978761498 LabelType=0 MediaId=283 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:50:02 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:367-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 CreateJobMedia FirstIndex=858 LastIndex=867 StartFile=6 EndFile=6 StartBlock=978761499 EndBlock=1073723162 Copy=0 Strip=0 MediaId=283 Jun 10 18:50:03 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:375-6727 <dird 1000 OK CreateJobMedia Jun 10 18:50:03 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:318-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 UpdateMedia VolName=S3-AI-Longterm-0283 VolJobs=7 VolFiles=6 VolBlocks=416103 VolBytes=26843526939 VolMounts=7 VolErrors=0 VolWrites=416104 MaxVolBytes=26843545600 EndTime=1686423003 VolStatus=Full Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=457156712 VolFirstWritten=0 Jun 10 18:50:03 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Longterm-0283 VolJobs=7 VolFiles=6 VolBlocks=416103 VolBytes=26843526939 VolMounts=7 VolErrors=0 VolWrites=416104 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Full Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=457156712 EndFile=6 EndBlock=1073723162 LabelType=0 MediaId=283 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:50:04 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:231-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 FindMedia=1 pool_name=S3-AI-Longterm media_type=S3-us-east-2-XXXXXXXX unwanted_volumes= Jun 10 18:50:04 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Longterm-0284 VolJobs=0 VolFiles=0 VolBlocks=0 VolBytes=0 VolMounts=0 VolErrors=0 VolWrites=0 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=0 EndFile=0 EndBlock=0 LabelType=0 MediaId=284 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:50:04 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:318-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 UpdateMedia VolName=S3-AI-Longterm-0284 VolJobs=0 VolFiles=0 VolBlocks=0 VolBytes=0 VolMounts=0 VolErrors=0 VolWrites=0 MaxVolBytes=26843545600 EndTime=1686423004 VolStatus=Append Slot=0 relabel=1 InChanger=0 VolReadTime=0 VolWriteTime=0 VolFirstWritten=0 Jun 10 18:50:04 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Longterm-0284 VolJobs=0 VolFiles=0 VolBlocks=0 VolBytes=0 VolMounts=0 VolErrors=0 VolWrites=0 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=0 EndFile=0 EndBlock=0 LabelType=0 MediaId=284 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:50:04 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:318-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 UpdateMedia VolName=S3-AI-Longterm-0284 VolJobs=0 VolFiles=0 VolBlocks=0 VolBytes=250 VolMounts=1 VolErrors=0 VolWrites=1 MaxVolBytes=26843545600 EndTime=1686423004 VolStatus=Append Slot=0 relabel=1 InChanger=0 VolReadTime=0 VolWriteTime=6 VolFirstWritten=1686423004 Jun 10 18:50:05 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Longterm-0284 VolJobs=0 VolFiles=0 VolBlocks=0 VolBytes=250 VolMounts=1 VolErrors=0 VolWrites=1 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=6 EndFile=0 EndBlock=0 LabelType=0 MediaId=284 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:50:05 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:318-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 UpdateMedia VolName=S3-AI-Longterm-0284 VolJobs=1 VolFiles=0 VolBlocks=0 VolBytes=250 VolMounts=1 VolErrors=0 VolWrites=1 MaxVolBytes=26843545600 EndTime=1686423005 VolStatus=Append Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=6 VolFirstWritten=0 Jun 10 18:50:05 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Longterm-0284 VolJobs=1 VolFiles=0 VolBlocks=0 VolBytes=250 VolMounts=1 VolErrors=0 VolWrites=1 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=6 EndFile=0 EndBlock=0 LabelType=0 MediaId=284 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:50:52 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:367-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 CreateJobMedia FirstIndex=867 LastIndex=973 StartFile=0 EndFile=0 StartBlock=250 EndBlock=999936215 Copy=0 Strip=0 MediaId=284 Jun 10 18:50:52 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:375-6727 <dird 1000 OK CreateJobMedia Jun 10 18:50:52 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:318-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 UpdateMedia VolName=S3-AI-Longterm-0284 VolJobs=1 VolFiles=0 VolBlocks=15500 VolBytes=999936216 VolMounts=1 VolErrors=0 VolWrites=15501 MaxVolBytes=26843545600 EndTime=1686423052 VolStatus=Append Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=16367331 VolFirstWritten=0 Jun 10 18:50:52 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Longterm-0284 VolJobs=1 VolFiles=0 VolBlocks=15500 VolBytes=999936216 VolMounts=1 VolErrors=0 VolWrites=15501 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=16367331 EndFile=0 EndBlock=999936215 LabelType=0 MediaId=284 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:51:33 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:367-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 CreateJobMedia FirstIndex=973 LastIndex=1069 StartFile=0 EndFile=0 StartBlock=999936216 EndBlock=1999872195 Copy=0 Strip=0 MediaId=284 Jun 10 18:51:33 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:375-6727 <dird 1000 OK CreateJobMedia Jun 10 18:51:33 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:318-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 UpdateMedia VolName=S3-AI-Longterm-0284 VolJobs=1 VolFiles=0 VolBlocks=31000 VolBytes=1999872196 VolMounts=1 VolErrors=0 VolWrites=31001 MaxVolBytes=26843545600 EndTime=1686423093 VolStatus=Append Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=33447828 VolFirstWritten=0 Jun 10 18:51:33 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Longterm-0284 VolJobs=1 VolFiles=0 VolBlocks=31000 VolBytes=1999872196 VolMounts=1 VolErrors=0 VolWrites=31001 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=33447828 EndFile=0 EndBlock=1999872195 LabelType=0 MediaId=284 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:52:14 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:375-6727 <dird 1000 OK CreateJobMedia Jun 10 18:52:14 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:318-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 UpdateMedia VolName=S3-AI-Longterm-0284 VolJobs=1 VolFiles=0 VolBlocks=46500 VolBytes=2999808196 VolMounts=1 VolErrors=0 VolWrites=46501 MaxVolBytes=26843545600 EndTime=1686423134 VolStatus=Append Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=49995402 VolFirstWritten=0 Jun 10 18:52:14 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Longterm-0284 VolJobs=1 VolFiles=0 VolBlocks=46500 VolBytes=2999808196 VolMounts=1 VolErrors=0 VolWrites=46501 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=49995402 EndFile=0 EndBlock=2999808195 LabelType=0 MediaId=284 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:52:57 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:367-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 CreateJobMedia FirstIndex=1155 LastIndex=1214 StartFile=0 EndFile=0 StartBlock=2999808196 EndBlock=3999744182 Copy=0 Strip=0 MediaId=284 Jun 10 18:52:57 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:375-6727 <dird 1000 OK CreateJobMedia Jun 10 18:52:57 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:318-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 UpdateMedia VolName=S3-AI-Longterm-0284 VolJobs=1 VolFiles=0 VolBlocks=62000 VolBytes=3999744183 VolMounts=1 VolErrors=0 VolWrites=62001 MaxVolBytes=26843545600 EndTime=1686423177 VolStatus=Append Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=67183143 VolFirstWritten=0 Jun 10 18:52:57 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Longterm-0284 VolJobs=1 VolFiles=0 VolBlocks=62000 VolBytes=3999744183 VolMounts=1 VolErrors=0 VolWrites=62001 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=67183143 EndFile=0 EndBlock=3999744182 LabelType=0 MediaId=284 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:53:38 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:367-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 CreateJobMedia FirstIndex=1214 LastIndex=1289 StartFile=0 EndFile=1 StartBlock=3999744183 EndBlock=704712877 Copy=0 Strip=0 MediaId=284 Jun 10 18:53:38 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:375-6727 <dird 1000 OK CreateJobMedia Jun 10 18:53:38 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:318-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 UpdateMedia VolName=S3-AI-Longterm-0284 VolJobs=1 VolFiles=1 VolBlocks=77500 VolBytes=4999680174 VolMounts=1 VolErrors=0 VolWrites=77501 MaxVolBytes=26843545600 EndTime=1686423218 VolStatus=Append Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=83033724 VolFirstWritten=0 Jun 10 18:53:38 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Longterm-0284 VolJobs=1 VolFiles=1 VolBlocks=77500 VolBytes=4999680174 VolMounts=1 VolErrors=0 VolWrites=77501 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=83033724 EndFile=1 EndBlock=704712877 LabelType=0 MediaId=284 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:53:47 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:367-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 CreateJobMedia FirstIndex=1289 LastIndex=1682 StartFile=1 EndFile=1 StartBlock=704712878 EndBlock=908951103 Copy=0 Strip=0 MediaId=284 Jun 10 18:53:47 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:375-6727 <dird 1000 OK CreateJobMedia Jun 10 18:53:47 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:318-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 UpdateMedia VolName=S3-AI-Longterm-0284 VolJobs=1 VolFiles=1 VolBlocks=80666 VolBytes=5203918400 VolMounts=1 VolErrors=0 VolWrites=80667 MaxVolBytes=26843545600 EndTime=1686423227 VolStatus=Append Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=86301635 VolFirstWritten=0 Jun 10 18:53:47 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Longterm-0284 VolJobs=1 VolFiles=1 VolBlocks=80666 VolBytes=5203918400 VolMounts=1 VolErrors=0 VolWrites=80667 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=86301635 EndFile=1 EndBlock=908951103 LabelType=0 MediaId=284 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 Jun 10 18:53:48 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:318-6727 >dird CatReq Job=Copy_S3_AI_Full.2023-06-10_18.35.46_51 UpdateMedia VolName=S3-AI-Consolidated-0195 VolJobs=2 VolFiles=1 VolBlocks=114815 VolBytes=7406913160 VolMounts=2 VolErrors=0 VolWrites=114816 MaxVolBytes=26843545600 EndTime=1686423228 VolStatus=Used Slot=0 relabel=0 InChanger=0 VolReadTime=118752889 VolWriteTime=143555767 VolFirstWritten=0 Jun 10 18:53:48 ip-XXXXXXXXXXXXX bareos-sd[1420637]: s3-us-east-2-sd (50): stored/askdir.cc:113-6727 <dird 1000 OK VolName=S3-AI-Consolidated-0195 VolJobs=2 VolFiles=1 VolBlocks=114815 VolBytes=7406913160 VolMounts=2 VolErrors=0 VolWrites=114816 MaxVolBytes=26843545600 VolCapacityBytes=0 VolStatus=Used Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=118752889 VolWriteTime=143555767 EndFile=1 EndBlock=3111945863 LabelType=0 MediaId=195 REDACTED_ENCRYPTION_KEY_SECTION MinBlocksize=0 MaxBlocksize=0 |
|
|
Did you consider to upgrade to the last version available in current It look like this is directly related to https://github.com/bareos/bareos/pull/1347 which was fixed in 22.0.1 |
|
|
I'm use to the days where no new binaries were published in a major release so I didn't think to check for a binary update to apply first and didn't see the startfile issue in this tracker when considering to self compile a new release. Looks like that is probably the solution. Updating on my side now. |
|
|
Fixed normally in 22.1.0 Reopen if needed |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2023-06-10 20:19 | cmlara | New Issue | |
| 2023-06-10 20:19 | cmlara | File Added: bareos-backup-headers-redacted.txt | |
| 2023-06-11 08:33 | cmlara | Note Added: 0005067 | |
| 2023-06-11 08:33 | cmlara | File Added: bareos-copy-job-sd-dir-logs.txt | |
| 2023-06-13 16:07 | bruno-at-bareos | Assigned To | => bruno-at-bareos |
| 2023-06-13 16:07 | bruno-at-bareos | Status | new => feedback |
| 2023-06-13 16:07 | bruno-at-bareos | Note Added: 0005068 | |
| 2023-06-14 01:40 | cmlara | Note Added: 0005070 | |
| 2023-06-14 01:40 | cmlara | Status | feedback => assigned |
| 2023-06-20 15:25 | bruno-at-bareos | Status | assigned => closed |
| 2023-06-20 15:25 | bruno-at-bareos | Resolution | open => fixed |
| 2023-06-20 15:25 | bruno-at-bareos | Note Added: 0005089 |
