View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0000636 | bareos-core | storage daemon | public | 2016-03-30 08:50 | 2023-12-13 15:34 |
Reporter | axestr | Assigned To | bruno-at-bareos | ||
Priority | normal | Severity | minor | Reproducibility | always |
Status | closed | Resolution | unable to reproduce | ||
Platform | Linux | OS | CentOS | OS Version | 6 |
Product Version | 15.2.2 | ||||
Summary | 0000636: Error on Copy Job / Maybe simmilar to 0000361 | ||||
Description | For an copy job, I become allways the error Fatal error: append.c:191 FI=51 from SD not positive or sequential=0 see additional information for full job log. All other jobs and copy jobs are running fine. Maybe, this is simmialar to 0000361? How can I help to resolve this Issue? | ||||
Steps To Reproduce | Scheduler reproduces this error daily. | ||||
Additional Information | 30-Mar 01:40 home-dir JobId 9703: Copying using JobId=6613 Job=MsWs1-Users.2016-02-18_22.30.00_32 30-Mar 01:40 home-dir JobId 9703: Bootstrap records written to /var/lib/bareos/home-dir.restore.9.bsr 30-Mar 01:40 home-dir JobId 9703: Start Copying JobId 9703, Job=Copy-HomeLocal-Q-to-Backup1-Q.2016-03-30_01.40.03_21 30-Mar 01:40 home-dir JobId 9703: Using Device "Home-Store1" to read. 30-Mar 01:40 home-dir JobId 9704: Using Device "Backup1-Store1" to write. 30-Mar 01:40 backup1-sd JobId 9704: Volume "Backup1-Q-4" previously written, moving to end of data. 30-Mar 01:40 home-sd JobId 9703: Ready to read from volume "HomeLocal-Q-2" on device "Home-Store1" (/home/system-shares/bareos-storage/store1). 30-Mar 01:40 backup1-sd JobId 9704: Ready to append to end of Volume "Backup1-Q-4" size=6782764324 30-Mar 01:40 home-sd JobId 9703: Forward spacing Volume "HomeLocal-Q-2" to file:block 2:2081572941. 30-Mar 01:40 home-sd JobId 9703: End of Volume at file 2 on device "Home-Store1" (/home/system-shares/bareos-storage/store1), Volume "HomeLocal-Q-2" 30-Mar 01:40 home-sd JobId 9703: Ready to read from volume "HomeLocal-Q-3" on device "Home-Store1" (/home/system-shares/bareos-storage/store1). 30-Mar 01:40 backup1-sd JobId 9704: Fatal error: append.c:191 FI=51 from SD not positive or sequential=0 30-Mar 01:40 backup1-sd JobId 9704: Elapsed time=00:00:01, Transfer rate=0 Bytes/second 30-Mar 01:40 home-sd JobId 9703: Forward spacing Volume "HomeLocal-Q-3" to file:block 0:11067481. 30-Mar 01:40 home-sd JobId 9703: Error: bsock_tcp.c:422 Write error sending -1 bytes to Storage daemon:backup1.pmit.cc:9103: ERR=Connection reset by peer 30-Mar 01:40 home-sd JobId 9703: Fatal error: mac.c:537 Network send error to SD. ERR=Connection reset by peer 30-Mar 01:40 home-dir JobId 9703: Error: Bareos home-dir 15.2.2 (16Nov15): Build OS: x86_64-redhat-linux-gnu redhat CentOS release 6.6 (Final) Prev Backup JobId: 6613 Prev Backup Job: MsWs1-Users.2016-02-18_22.30.00_32 New Backup JobId: 9704 Current JobId: 9703 Current Job: Copy-HomeLocal-Q-to-Backup1-Q.2016-03-30_01.40.03_21 Backup Level: Full Client: Dummy FileSet: "Dummy" Read Pool: "HomeLocal-Q" (From Job resource) Read Storage: "Home-Store1" (From Pool resource) Write Pool: "Backup1-Q" (From Job Pool's NextPool resource) Write Storage: "Backup1-Store1" (From Storage from Pool's NextPool resource) Next Pool: "Backup1-Q" (From Job Pool's NextPool resource) Catalog: "MyCatalog" (From Client resource) Start time: 30-Mar-2016 01:40:05 End time: 30-Mar-2016 01:40:05 Elapsed time: 0 secs Priority: 10 SD Files Written: 2 SD Bytes Written: 3,078 (3.078 KB) Rate: 0.0 KB/s Volume name(s): Volume Session Id: 50 Volume Session Time: 1459228020 Last Volume Bytes: 0 (0 B) SD Errors: 1 SD termination status: Fatal Error Termination: *** Copying Error *** | ||||
Tags | No tags attached. | ||||
Found a nasty workaround. The problem was the copy of a job which resits on two volumen (HomeLocal-Q-2 and HomeLocal-Q-3). Deleted the job, now everything is fine. But deleting the job is not a clean solution ;-) |
|
I think this is a corner case where there is not much on the first volume (e.g. not even a full data record) and as such an ASSERT is triggered that makes sure that the FI (FileIndex) is progressing. Probably a serious difficult one to reproduce in a reliable way to be able to create a workaround. As you deleted the Job we also have not really a way to get some higher debug output to see if my hunch is right. |
|
If the needed Information is in the PostgreSQL database, I can get backups from this database. Also, I have this information from BAT and following reports: Error: Bareos home-dir 15.2.2 (16Nov15): Build OS: x86_64-redhat-linux-gnu redhat CentOS release 6.6 (Final) Prev Backup JobId: 6613 Prev Backup Job: MsWs1-Users.2016-02-18_22.30.00_32 New Backup JobId: 10355 Current JobId: 10354 Current Job: Copy-HomeLocal-Q-to-Backup1-Q.2016-04-07_01.40.02_26 Backup Level: Full Client: Dummy FileSet: "Dummy" Read Pool: "HomeLocal-Q" (From Job resource) Read Storage: "Home-Store1" (From Pool resource) Write Pool: "Backup1-Q" (From Job Pool's NextPool resource) Write Storage: "Backup1-Store1" (From Storage from Pool's NextPool resource) Next Pool: "Backup1-Q" (From Job Pool's NextPool resource) Catalog: "MyCatalog" (From Client resource) Start time: 07-Apr-2016 01:40:05 End time: 07-Apr-2016 01:40:05 Elapsed time: 0 secs Priority: 10 SD Files Written: 2 SD Bytes Written: 3,078 (3.078 KB) Rate: 0.0 KB/s Volume name(s): Volume Session Id: 510 Volume Session Time: 1459228020 Last Volume Bytes: 0 (0 B) SD Errors: 1 SD termination status: Fatal Error Termination: *** Copying Error *** 27-Mar 01:20 home-dir JobId 9457: Copying using JobId=6613 Job=MsWs1-Users.2016-02-18_22.30.00_32 27-Mar 01:20 home-dir JobId 9457: Bootstrap records written to /var/lib/bareos/home-dir.restore.4.bsr 27-Mar 01:20 home-dir JobId 9457: Start Copying JobId 9457, Job=Copy-HomeLocal-Q-to-Backup1-Q.2016-03-27_01.20.02_58 27-Mar 01:20 home-dir JobId 9457: Using Device "Home-Store1" to read. 27-Mar 01:20 home-dir JobId 9458: Using Device "Backup1-Store1" to write. 27-Mar 01:20 backup1-sd JobId 9458: Volume "Backup1-Q-4" previously written, moving to end of data. 27-Mar 01:20 home-sd JobId 9457: Ready to read from volume "HomeLocal-Q-2" on device "Home-Store1" (/home/system-shares/bareos-storage/store1). 27-Mar 01:20 backup1-sd JobId 9458: Ready to append to end of Volume "Backup1-Q-4" size=6143665867 27-Mar 01:20 home-sd JobId 9457: Forward spacing Volume "HomeLocal-Q-2" to file:block 2:2081572941. 27-Mar 01:20 home-sd JobId 9457: End of Volume at file 2 on device "Home-Store1" (/home/system-shares/bareos-storage/store1), Volume "HomeLocal-Q-2" 27-Mar 01:20 home-sd JobId 9457: Ready to read from volume "HomeLocal-Q-3" on device "Home-Store1" (/home/system-shares/bareos-storage/store1). 27-Mar 01:20 backup1-sd JobId 9458: Fatal error: append.c:191 FI=51 from SD not positive or sequential=0 27-Mar 01:20 backup1-sd JobId 9458: Elapsed time=00:00:01, Transfer rate=0 Bytes/second 27-Mar 01:20 home-sd JobId 9457: Forward spacing Volume "HomeLocal-Q-3" to file:block 0:11067481. 27-Mar 01:20 home-sd JobId 9457: Error: bsock_tcp.c:422 Write error sending -1 bytes to Storage daemon:backup1.pmit.cc:9103: ERR=Connection reset by peer 27-Mar 01:20 home-sd JobId 9457: Fatal error: mac.c:537 Network send error to SD. ERR=Connection reset by peer 27-Mar 01:20 home-dir JobId 9457: Error: Bareos home-dir 15.2.2 (16Nov15): Build OS: x86_64-redhat-linux-gnu redhat CentOS release 6.6 (Final) Prev Backup JobId: 6613 Prev Backup Job: MsWs1-Users.2016-02-18_22.30.00_32 New Backup JobId: 9458 Current JobId: 9457 Current Job: Copy-HomeLocal-Q-to-Backup1-Q.2016-03-27_01.20.02_58 Backup Level: Full Client: Dummy FileSet: "Dummy" Read Pool: "HomeLocal-Q" (From Job resource) Read Storage: "Home-Store1" (From Pool resource) Write Pool: "Backup1-Q" (From Job Pool's NextPool resource) Write Storage: "Backup1-Store1" (From Storage from Pool's NextPool resource) Next Pool: "Backup1-Q" (From Job Pool's NextPool resource) Catalog: "MyCatalog" (From Client resource) Start time: 27-Mar-2016 01:20:03 End time: 27-Mar-2016 01:20:03 Elapsed time: 0 secs Priority: 10 SD Files Written: 2 SD Bytes Written: 3,078 (3.078 KB) Rate: 0.0 KB/s Volume name(s): Volume Session Id: 44 Volume Session Time: 1458977449 Last Volume Bytes: 0 (0 B) SD Errors: 1 SD termination status: Fatal Error Termination: *** Copying Error *** 28-Mar 01:40 home-dir JobId 9533: Copying using JobId=6613 Job=MsWs1-Users.2016-02-18_22.30.00_32 28-Mar 01:40 home-dir JobId 9533: Bootstrap records written to /var/lib/bareos/home-dir.restore.3.bsr 28-Mar 01:40 home-dir JobId 9533: Start Copying JobId 9533, Job=Copy-HomeLocal-Q-to-Backup1-Q.2016-03-28_01.40.00_30 28-Mar 01:40 home-dir JobId 9533: Using Device "Home-Store1" to read. 28-Mar 01:40 home-dir JobId 9534: Using Device "Backup1-Store1" to write. 28-Mar 01:40 backup1-sd JobId 9534: Volume "Backup1-Q-4" previously written, moving to end of data. 28-Mar 01:40 home-sd JobId 9533: Ready to read from volume "HomeLocal-Q-2" on device "Home-Store1" (/home/system-shares/bareos-storage/store1). 28-Mar 01:40 backup1-sd JobId 9534: Ready to append to end of Volume "Backup1-Q-4" size=6592042035 28-Mar 01:40 home-sd JobId 9533: Forward spacing Volume "HomeLocal-Q-2" to file:block 2:2081572941. 28-Mar 01:40 home-sd JobId 9533: End of Volume at file 2 on device "Home-Store1" (/home/system-shares/bareos-storage/store1), Volume "HomeLocal-Q-2" 28-Mar 01:40 home-sd JobId 9533: Ready to read from volume "HomeLocal-Q-3" on device "Home-Store1" (/home/system-shares/bareos-storage/store1). 28-Mar 01:40 backup1-sd JobId 9534: Fatal error: append.c:191 FI=51 from SD not positive or sequential=0 28-Mar 01:40 backup1-sd JobId 9534: Elapsed time=00:00:01, Transfer rate=0 Bytes/second 28-Mar 01:40 home-sd JobId 9533: Forward spacing Volume "HomeLocal-Q-3" to file:block 0:11067481. 28-Mar 01:40 home-sd JobId 9533: Error: bsock_tcp.c:422 Write error sending -1 bytes to Storage daemon:backup1.pmit.cc:9103: ERR=Connection reset by peer 28-Mar 01:40 home-sd JobId 9533: Fatal error: mac.c:537 Network send error to SD. ERR=Connection reset by peer 28-Mar 01:40 home-dir JobId 9533: Error: Bareos home-dir 15.2.2 (16Nov15): Build OS: x86_64-redhat-linux-gnu redhat CentOS release 6.6 (Final) Prev Backup JobId: 6613 Prev Backup Job: MsWs1-Users.2016-02-18_22.30.00_32 New Backup JobId: 9534 Current JobId: 9533 Current Job: Copy-HomeLocal-Q-to-Backup1-Q.2016-03-28_01.40.00_30 Backup Level: Full Client: Dummy FileSet: "Dummy" Read Pool: "HomeLocal-Q" (From Job resource) Read Storage: "Home-Store1" (From Pool resource) Write Pool: "Backup1-Q" (From Job Pool's NextPool resource) Write Storage: "Backup1-Store1" (From Storage from Pool's NextPool resource) Next Pool: "Backup1-Q" (From Job Pool's NextPool resource) Catalog: "MyCatalog" (From Client resource) Start time: 28-Mar-2016 01:40:32 End time: 28-Mar-2016 01:40:33 Elapsed time: 1 sec Priority: 10 SD Files Written: 2 SD Bytes Written: 3,078 (3.078 KB) Rate: 3.1 KB/s Volume name(s): Volume Session Id: 27 Volume Session Time: 1459067007 Last Volume Bytes: 0 (0 B) SD Errors: 1 SD termination status: Fatal Error Termination: *** Copying Error *** 29-Mar 01:40 home-dir JobId 9607: Copying using JobId=6613 Job=MsWs1-Users.2016-02-18_22.30.00_32 29-Mar 01:40 home-dir JobId 9607: Bootstrap records written to /var/lib/bareos/home-dir.restore.1.bsr 29-Mar 01:40 home-dir JobId 9607: Start Copying JobId 9607, Job=Copy-HomeLocal-Q-to-Backup1-Q.2016-03-29_01.40.02_44 29-Mar 01:40 home-dir JobId 9607: Using Device "Home-Store1" to read. 29-Mar 01:40 home-dir JobId 9608: Using Device "Backup1-Store1" to write. 29-Mar 01:40 backup1-sd JobId 9608: Volume "Backup1-Q-4" previously written, moving to end of data. 29-Mar 01:40 home-sd JobId 9607: Ready to read from volume "HomeLocal-Q-2" on device "Home-Store1" (/home/system-shares/bareos-storage/store1). 29-Mar 01:40 backup1-sd JobId 9608: Ready to append to end of Volume "Backup1-Q-4" size=6592238917 29-Mar 01:40 home-sd JobId 9607: Forward spacing Volume "HomeLocal-Q-2" to file:block 2:2081572941. 29-Mar 01:40 home-sd JobId 9607: End of Volume at file 2 on device "Home-Store1" (/home/system-shares/bareos-storage/store1), Volume "HomeLocal-Q-2" 29-Mar 01:40 home-sd JobId 9607: Ready to read from volume "HomeLocal-Q-3" on device "Home-Store1" (/home/system-shares/bareos-storage/store1). 29-Mar 01:40 backup1-sd JobId 9608: Fatal error: append.c:191 FI=51 from SD not positive or sequential=0 29-Mar 01:40 backup1-sd JobId 9608: Elapsed time=00:00:01, Transfer rate=0 Bytes/second 29-Mar 01:40 home-sd JobId 9607: Forward spacing Volume "HomeLocal-Q-3" to file:block 0:11067481. 29-Mar 01:40 home-sd JobId 9607: Error: bsock_tcp.c:422 Write error sending -1 bytes to Storage daemon:backup1.pmit.cc:9103: ERR=Connection reset by peer 29-Mar 01:40 home-sd JobId 9607: Fatal error: mac.c:537 Network send error to SD. ERR=Connection reset by peer 29-Mar 01:40 home-dir JobId 9607: Error: Bareos home-dir 15.2.2 (16Nov15): Build OS: x86_64-redhat-linux-gnu redhat CentOS release 6.6 (Final) Prev Backup JobId: 6613 Prev Backup Job: MsWs1-Users.2016-02-18_22.30.00_32 New Backup JobId: 9608 Current JobId: 9607 Current Job: Copy-HomeLocal-Q-to-Backup1-Q.2016-03-29_01.40.02_44 Backup Level: Full Client: Dummy FileSet: "Dummy" Read Pool: "HomeLocal-Q" (From Job resource) Read Storage: "Home-Store1" (From Pool resource) Write Pool: "Backup1-Q" (From Job Pool's NextPool resource) Write Storage: "Backup1-Store1" (From Storage from Pool's NextPool resource) Next Pool: "Backup1-Q" (From Job Pool's NextPool resource) Catalog: "MyCatalog" (From Client resource) Start time: 29-Mar-2016 01:40:05 End time: 29-Mar-2016 01:40:05 Elapsed time: 0 secs Priority: 10 SD Files Written: 2 SD Bytes Written: 3,078 (3.078 KB) Rate: 0.0 KB/s Volume name(s): Volume Session Id: 79 Volume Session Time: 1459067007 Last Volume Bytes: 0 (0 B) SD Errors: 1 SD termination status: Fatal Error Termination: *** Copying Error *** 30-Mar 01:40 home-dir JobId 9703: Copying using JobId=6613 Job=MsWs1-Users.2016-02-18_22.30.00_32 30-Mar 01:40 home-dir JobId 9703: Bootstrap records written to /var/lib/bareos/home-dir.restore.9.bsr 30-Mar 01:40 home-dir JobId 9703: Start Copying JobId 9703, Job=Copy-HomeLocal-Q-to-Backup1-Q.2016-03-30_01.40.03_21 30-Mar 01:40 home-dir JobId 9703: Using Device "Home-Store1" to read. 30-Mar 01:40 home-dir JobId 9704: Using Device "Backup1-Store1" to write. 30-Mar 01:40 backup1-sd JobId 9704: Volume "Backup1-Q-4" previously written, moving to end of data. 30-Mar 01:40 home-sd JobId 9703: Ready to read from volume "HomeLocal-Q-2" on device "Home-Store1" (/home/system-shares/bareos-storage/store1). 30-Mar 01:40 backup1-sd JobId 9704: Ready to append to end of Volume "Backup1-Q-4" size=6782764324 30-Mar 01:40 home-sd JobId 9703: Forward spacing Volume "HomeLocal-Q-2" to file:block 2:2081572941. 30-Mar 01:40 home-sd JobId 9703: End of Volume at file 2 on device "Home-Store1" (/home/system-shares/bareos-storage/store1), Volume "HomeLocal-Q-2" 30-Mar 01:40 home-sd JobId 9703: Ready to read from volume "HomeLocal-Q-3" on device "Home-Store1" (/home/system-shares/bareos-storage/store1). 30-Mar 01:40 backup1-sd JobId 9704: Fatal error: append.c:191 FI=51 from SD not positive or sequential=0 30-Mar 01:40 backup1-sd JobId 9704: Elapsed time=00:00:01, Transfer rate=0 Bytes/second 30-Mar 01:40 home-sd JobId 9703: Forward spacing Volume "HomeLocal-Q-3" to file:block 0:11067481. 30-Mar 01:40 home-sd JobId 9703: Error: bsock_tcp.c:422 Write error sending -1 bytes to Storage daemon:backup1.pmit.cc:9103: ERR=Connection reset by peer 30-Mar 01:40 home-sd JobId 9703: Fatal error: mac.c:537 Network send error to SD. ERR=Connection reset by peer 30-Mar 01:40 home-dir JobId 9703: Error: Bareos home-dir 15.2.2 (16Nov15): Build OS: x86_64-redhat-linux-gnu redhat CentOS release 6.6 (Final) Prev Backup JobId: 6613 Prev Backup Job: MsWs1-Users.2016-02-18_22.30.00_32 New Backup JobId: 9704 Current JobId: 9703 Current Job: Copy-HomeLocal-Q-to-Backup1-Q.2016-03-30_01.40.03_21 Backup Level: Full Client: Dummy FileSet: "Dummy" Read Pool: "HomeLocal-Q" (From Job resource) Read Storage: "Home-Store1" (From Pool resource) Write Pool: "Backup1-Q" (From Job Pool's NextPool resource) Write Storage: "Backup1-Store1" (From Storage from Pool's NextPool resource) Next Pool: "Backup1-Q" (From Job Pool's NextPool resource) Catalog: "MyCatalog" (From Client resource) Start time: 30-Mar-2016 01:40:05 End time: 30-Mar-2016 01:40:05 Elapsed time: 0 secs Priority: 10 SD Files Written: 2 SD Bytes Written: 3,078 (3.078 KB) Rate: 0.0 KB/s Volume name(s): Volume Session Id: 50 Volume Session Time: 1459228020 Last Volume Bytes: 0 (0 B) SD Errors: 1 SD termination status: Fatal Error Termination: *** Copying Error *** 07-Apr 01:40 home-dir JobId 10354: Copying using JobId=6613 Job=MsWs1-Users.2016-02-18_22.30.00_32 07-Apr 01:40 home-dir JobId 10354: Bootstrap records written to /var/lib/bareos/home-dir.restore.170.bsr 07-Apr 01:40 home-dir JobId 10354: Start Copying JobId 10354, Job=Copy-HomeLocal-Q-to-Backup1-Q.2016-04-07_01.40.02_26 07-Apr 01:40 home-dir JobId 10354: Using Device "Home-Store1" to read. 07-Apr 01:40 home-dir JobId 10355: Using Device "Backup1-Store1" to write. 07-Apr 01:40 backup1-sd JobId 10355: Volume "Backup1-Q-0" previously written, moving to end of data. 07-Apr 01:40 home-sd JobId 10354: Ready to read from volume "HomeLocal-Q-2" on device "Home-Store1" (/home/system-shares/bareos-storage/store1). 07-Apr 01:40 backup1-sd JobId 10355: Ready to append to end of Volume "Backup1-Q-0" size=3490511342 07-Apr 01:40 home-sd JobId 10354: Forward spacing Volume "HomeLocal-Q-2" to file:block 2:2081572941. 07-Apr 01:40 home-sd JobId 10354: End of Volume at file 2 on device "Home-Store1" (/home/system-shares/bareos-storage/store1), Volume "HomeLocal-Q-2" 07-Apr 01:40 home-sd JobId 10354: Ready to read from volume "HomeLocal-Q-3" on device "Home-Store1" (/home/system-shares/bareos-storage/store1). 07-Apr 01:40 backup1-sd JobId 10355: Fatal error: append.c:191 FI=51 from SD not positive or sequential=0 07-Apr 01:40 backup1-sd JobId 10355: Elapsed time=00:00:01, Transfer rate=0 Bytes/second 07-Apr 01:40 home-sd JobId 10354: Forward spacing Volume "HomeLocal-Q-3" to file:block 0:11067481. 07-Apr 01:40 home-sd JobId 10354: Error: bsock_tcp.c:422 Write error sending -1 bytes to Storage daemon:backup1.pmit.cc:9103: ERR=Connection reset by peer 07-Apr 01:40 home-sd JobId 10354: Fatal error: mac.c:537 Network send error to SD. ERR=Connection reset by peer 07-Apr 01:40 home-dir JobId 10354: Error: Bareos home-dir 15.2.2 (16Nov15): Build OS: x86_64-redhat-linux-gnu redhat CentOS release 6.6 (Final) Prev Backup JobId: 6613 Prev Backup Job: MsWs1-Users.2016-02-18_22.30.00_32 New Backup JobId: 10355 Current JobId: 10354 Current Job: Copy-HomeLocal-Q-to-Backup1-Q.2016-04-07_01.40.02_26 Backup Level: Full Client: Dummy FileSet: "Dummy" Read Pool: "HomeLocal-Q" (From Job resource) Read Storage: "Home-Store1" (From Pool resource) Write Pool: "Backup1-Q" (From Job Pool's NextPool resource) Write Storage: "Backup1-Store1" (From Storage from Pool's NextPool resource) Next Pool: "Backup1-Q" (From Job Pool's NextPool resource) Catalog: "MyCatalog" (From Client resource) Start time: 07-Apr-2016 01:40:05 End time: 07-Apr-2016 01:40:05 Elapsed time: 0 secs Priority: 10 SD Files Written: 2 SD Bytes Written: 3,078 (3.078 KB) Rate: 0.0 KB/s Volume name(s): Volume Session Id: 510 Volume Session Time: 1459228020 Last Volume Bytes: 0 (0 B) SD Errors: 1 SD termination status: Fatal Error Termination: *** Copying Error *** |
|
Hi Bareos Team, Same error pop's up again daily. What could I do/track to identify? 04-Nov 01:20 home-dir JobId 64023: Copying using JobId=62888 Job=Leela-Users.2017-10-22_22.30.00_57 04-Nov 01:20 home-dir JobId 64023: Bootstrap records written to /var/lib/bareos/home-dir.restore.108.bsr 04-Nov 01:20 home-dir JobId 64023: Start Copying JobId 64023, Job=Copy-HomeLocal-Y-to-Backup1-Y.2017-11-04_01.20.02_23 04-Nov 01:20 home-dir JobId 64023: Using Device "Home-Store1" to read. 04-Nov 01:20 home-dir JobId 64024: Using Device "Backup1-Store1" to write. 04-Nov 01:20 home-sd JobId 64023: Ready to read from volume "HomeLocal-Y-10" on device "Home-Store1" (/home/system-shares/bareos-storage/store1). 04-Nov 01:20 backup1-sd JobId 64024: Volume "Backup1-Y-11" previously written, moving to end of data. 04-Nov 01:20 backup1-sd JobId 64024: Ready to append to end of Volume "Backup1-Y-11" size=3495845162 04-Nov 01:20 home-sd JobId 64023: Forward spacing Volume "HomeLocal-Y-10" to file:block 2:2100121434. 04-Nov 01:20 home-sd JobId 64023: End of Volume at file 2 on device "Home-Store1" (/home/system-shares/bareos-storage/store1), Volume "HomeLocal-Y-10" 04-Nov 01:20 home-sd JobId 64023: Ready to read from volume "HomeLocal-Y-11" on device "Home-Store1" (/home/system-shares/bareos-storage/store1). 04-Nov 01:20 backup1-sd JobId 64024: Fatal error: append.c:191 FI=217 from SD not positive or sequential=0 04-Nov 01:20 home-sd JobId 64023: Forward spacing Volume "HomeLocal-Y-11" to file:block 0:21805270. 04-Nov 01:20 home-sd JobId 64023: Error: bsock_tcp.c:422 Write error sending 14208 bytes to Storage daemon:backup1.pmit.cc:9103: ERR=Connection reset by peer 04-Nov 01:20 home-sd JobId 64023: Fatal error: mac.c:326 Network send error to SD. ERR=Connection reset by peer 04-Nov 01:20 home-sd JobId 64023: Error: bsock_tcp.c:357 Socket has errors=1 on call to Storage daemon:backup1.pmit.cc:9103 04-Nov 01:20 backup1-sd JobId 64024: Elapsed time=00:00:01, Transfer rate=0 Bytes/second 04-Nov 01:20 home-dir JobId 64023: Error: Bareos home-dir 15.2.2 (16Nov15): Build OS: x86_64-redhat-linux-gnu redhat CentOS release 6.6 (Final) Prev Backup JobId: 62888 Prev Backup Job: Leela-Users.2017-10-22_22.30.00_57 New Backup JobId: 64024 Current JobId: 64023 Current Job: Copy-HomeLocal-Y-to-Backup1-Y.2017-11-04_01.20.02_23 Backup Level: Full Client: Dummy FileSet: "Dummy" Read Pool: "HomeLocal-Y" (From Job resource) Read Storage: "Home-Store1" (From Pool resource) Write Pool: "Backup1-Y" (From Job Pool's NextPool resource) Write Storage: "Backup1-Store1" (From Storage from Pool's NextPool resource) Next Pool: "Backup1-Y" (From Job Pool's NextPool resource) Catalog: "MyCatalog" (From Client resource) Start time: 04-Nov-2017 01:20:04 End time: 04-Nov-2017 01:20:05 Elapsed time: 1 sec Priority: 10 SD Files Written: 1 SD Bytes Written: 3,606 (3.606 KB) Rate: 3.6 KB/s Volume name(s): Volume Session Id: 315 Volume Session Time: 1509371965 Last Volume Bytes: 0 (0 B) SD Errors: 2 SD termination status: Fatal Error Termination: *** Copying Error *** BR, Axel |
|
I have same problem. Bareos version 17.2.4-9.1 installed from http://download.bareos.org/bareos/release/latest/xUbuntu_16.04. First it looks like network problem, but when I've made dump I've found that TCP connection closed by storage without FIN flag, eventually it just started to sent RST flag. In debug output from storage I've found next: 23-May-2018 04:52:49.623885 ow-backup03-sd (850): message.c:858-6475 Enter dispatch_message type=3 msg=ow-backup03-sd JobId 6475: Fatal error: append.c:192 FI=5 from SD not positive or sequential=0 23-May-2018 04:52:49.623899 ow-backup03-sd (850): message.c:1129-6475 DIRECTOR for following msg: ow-backup03-sd JobId 6475: Fatal error: append.c:192 FI=5 from SD not positive or sequential=0 It only occurs when I try to run copy job. |
|
Copy job is not completely broken, it can't copy only jobs which share volume with another job: *list jobs volume=ord-uploadcdr01-fd-mia-backup03_WV-CDRs_full-20180514-5851-486 +-------+----------------------------+--------------------+---------------------+------+-------+----------+-------------+-----------+ | jobid | name | client | starttime | type | level | jobfiles | jobbytes | jobstatus | +-------+----------------------------+--------------------+---------------------+------+-------+----------+-------------+-----------+ | 5,926 | ow-cdrupload01_CDRs_daily | ow-cdrupload01-fd | 2018-05-15 21:00:02 | B | F | 108 | 809,354,467 | T | | 5,927 | ord-uploadcdr01_CDRs_daily | ord-uploadcdr01-fd | 2018-05-15 21:00:03 | B | F | 70 | 334,296,463 | T | +-------+----------------------------+--------------------+---------------------+------+-------+----------+-------------+-----------+ *list volumes jobid=5926 Jobid 5926 used 1 Volume(s): ord-uploadcdr01-fd-mia-backup03_WV-CDRs_full-20180514-5851-486 *list volumes jobid=5927 Jobid 5927 used 1 Volume(s): ord-uploadcdr01-fd-mia-backup03_WV-CDRs_full-20180514-5851-486 In webui job looks like it uses two volumes. |
|
|
|
can't be reproduced with recent version. If you are such a case, please don't delete the job or volumes and reopen an new issue. | |
Date Modified | Username | Field | Change |
---|---|---|---|
2016-03-30 08:50 | axestr | New Issue | |
2016-04-08 07:35 | axestr | Note Added: 0002230 | |
2016-05-09 19:13 | mvwieringen | Note Added: 0002265 | |
2016-05-09 19:13 | mvwieringen | Status | new => feedback |
2016-05-10 06:21 | axestr | Note Added: 0002266 | |
2016-05-10 06:21 | axestr | Status | feedback => new |
2017-11-04 08:37 | axestr | Note Added: 0002811 | |
2018-05-23 11:43 | IvanBayan | Note Added: 0003014 | |
2018-05-23 14:12 | IvanBayan | Note Added: 0003015 | |
2018-05-23 14:13 | IvanBayan | File Added: bareos_volumes.png | |
2023-12-13 15:34 | bruno-at-bareos | Assigned To | => bruno-at-bareos |
2023-12-13 15:34 | bruno-at-bareos | Status | new => closed |
2023-12-13 15:34 | bruno-at-bareos | Resolution | open => unable to reproduce |
2023-12-13 15:34 | bruno-at-bareos | Note Added: 0005636 |