View Issue Details

IDProjectCategoryView StatusLast Update
0001091bareos-coredirectorpublic2019-12-18 15:21
Reporterisi Assigned Toarogge  
PrioritynormalSeverityminorReproducibilityalways
Status closedResolutionfixed 
PlatformLinuxOSDebianOS Version9
Product Version18.2.5 
Summary0001091: NDMP to NDMP Copy Job Fails
DescriptionBackup to NDMPFile Pool works.
Backup to NDMPTape Pool works.

Copy from NDMPTape to NDMPCopy Pool gives error:

Automatically selected Catalog: MyCatalog
Using Catalog "MyCatalog"
 bareos-dir JobId 253: Start Copying JobId 253, Job=copy-asterix-lka-copypool.2019-06-11_13.23.24_16
 bareos-dir JobId 253: Fatal error: Connect failure: ERR=error:1408F10B:SSL routines:ssl3_get_record:wrong version number
 bareos-dir JobId 253: Error: TLS shutdown failure.: ERR=error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
 bareos-dir JobId 253: Fatal error: TLS negotiation failed
 bareos-dir JobId 253: Error: Bareos bareos-dir 18.2.5 (30Jan19):
  Build OS: Linux-4.4.92-6.18-default debian Debian GNU/Linux 9.7 (stretch)
  Prev Backup JobId: 242
  Prev Backup Job: asterix-lka-ndmp-job.2019-06-11_11.53.23_48
  New Backup JobId: 254
  Current JobId: 253
  Current Job: copy-asterix-lka-copypool.2019-06-11_13.23.24_16
  Backup Level: Full
  Client: asterix-ndmp
  FileSet: "asterix-lka-ndmp-fs"
  Read Pool: "NDMPTape" (From Job resource)
  Read Storage: "NDMPTape" (From Pool resource)
  Write Pool: "NDMPCopy" (From Job Pool's NextPool resource)
  Write Storage: "NDMPCopy" (From Storage from Pool's NextPool resource)
  Next Pool: "NDMPCopy" (From Job Pool's NextPool resource)
  Catalog: "MyCatalog" (From Default catalog)
  Start time: 11-Jun-2019 13:23:26
  End time: 11-Jun-2019 13:23:26
  Elapsed time: 0 secs
  Priority: 10
  SD Files Written: 0
  SD Bytes Written: 0 (0 B)
  Rate: 0.0 KB/s
  Volume name(s):
  Volume Session Id: 0
  Volume Session Time: 0
  Last Volume Bytes: 0 (0 B)
  SD Errors: 0
  SD termination status:
  Bareos binary info: bareos.org build: Get official binaries and vendor support on bareos.com
  Termination: *** Copying Error ***

No TLS config settings in any config File present. TLS Config is 'default' settings.
TagsNo tags attached.

Activities

arogge

arogge

2019-07-09 09:04

manager   ~0003421

Is this a local (on the same sd) or a remote (to a different sd) copy?

Can you please try of the issue still exists in our nightly build https://download.bareos.org/bareos/experimental/nightly/
isi

isi

2019-07-09 16:24

reporter   ~0003428

It's local, on same sd. Will try nightly and report back. Thanks.
isi

isi

2019-07-10 12:29

reporter   ~0003429

No difference with nightly Version.

09-Jul 16:31 bareos-dir JobId 431: Copying using JobId=413 Job=asterix-lka-ndmp-job.2019-07-07_09.00.00_16
09-Jul 16:31 bareos-dir JobId 431: Bootstrap records written to /var/lib/bareos/bareos-dir.restore.1.bsr
09-Jul 16:31 bareos-dir JobId 431: Start Copying JobId 431, Job=copy-asterix-lka-copypool.2019-07-09_16.31.30_10
09-Jul 16:31 bareos-dir JobId 431: Fatal error: Connect failure: ERR=error:1408F10B:SSL routines:ssl3_get_record:wrong version number
09-Jul 16:31 bareos-dir JobId 431: Error: TLS shutdown failure.: ERR=error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
09-Jul 16:31 bareos-dir JobId 431: Fatal error: TLS negotiation failed
09-Jul 16:31 bareos-dir JobId 431: Error: Bareos bareos-dir 19.1.2 (01Feb19):
  Build OS: Linux-4.4.175-89-default debian Debian GNU/Linux 9.9 (stretch)
  Prev Backup JobId: 413
  Prev Backup Job: asterix-lka-ndmp-job.2019-07-07_09.00.00_16
  New Backup JobId: 432
  Current JobId: 431
  Current Job: copy-asterix-lka-copypool.2019-07-09_16.31.30_10
  Backup Level: Full
  Client: asterix-ndmp
  FileSet: "asterix-lka-ndmp-fs"
  Read Pool: "NDMPTape" (From Job resource)
  Read Storage: "NDMPTape" (From Pool resource)
  Write Pool: "NDMPCopy" (From Job Pool's NextPool resource)
  Write Storage: "NDMPCopy" (From Storage from Pool's NextPool resource)
  Next Pool: "NDMPCopy" (From Job Pool's NextPool resource)
  Catalog: "MyCatalog" (From Default catalog)
  Start time: 09-Jul-2019 16:31:32
  End time: 09-Jul-2019 16:31:32
  Elapsed time: 0 secs
  Priority: 10
  SD Files Written: 0
  SD Bytes Written: 0 (0 B)
  Rate: 0.0 KB/s
  Volume name(s):
  Volume Session Id: 0
  Volume Session Time: 0
  Last Volume Bytes: 0 (0 B)
  SD Errors: 0
  SD termination status:
  Bareos binary info: bareos.org build: Get official binaries and vendor support on bareos.com
  Termination: *** Copying Error ***
arogge

arogge

2019-07-10 17:02

manager   ~0003430

That's weird. Our testsuite should have tested this before release.
Can you provide your storage configuration for the two storages and paired storages, so I can test this?
isi

isi

2019-07-11 06:31

reporter   ~0003437

Storage {
  Name = NDMPCopy
  Address = 10.210.0.88 # N.B. Use a fully qualified name here
  Port = 10000
  Protocol = NDMPv4 # Need to specify protocol before password as protocol determines password encoding used.
  Auth Type = Clear # Clear == Clear Text, MD5 == Challenge protocol
  Username = ndmp # username of the NDMP user on the TAPE AGENT e.g. the Bareos SD but accessed via the NDMP protocol.
  Password = xxxxxed # password of the NDMP user on the TAPE AGENT e.g. the Bareos SD but accessed via the NDMP protocol.
  Device = scalar-i500
  Media Type = LTO-5
  PairedStorage = i500
  Maximum Concurrent Jobs = 6
}
Storage {
  Name = NDMPTape
  Address = 10.210.0.88 # N.B. Use a fully qualified name here
  Port = 10000
  Protocol = NDMPv4 # Need to specify protocol before password as protocol determines password encoding used.
  Auth Type = Clear # Clear == Clear Text, MD5 == Challenge protocol
  Username = ndmp # username of the NDMP user on the TAPE AGENT e.g. the Bareos SD but accessed via the NDMP protocol.
  Password = xxxxxed # password of the NDMP user on the TAPE AGENT e.g. the Bareos SD but accessed via the NDMP protocol.
  Device = scalar-i500
  Media Type = LTO-5
  PairedStorage = i500
  Maximum Concurrent Jobs = 6
}

Storage {
  Name = i500
  Address = bb8.elkb.de # N.B. Use a fully qualified name here (do not use "localhost" here).
  Password = xxxxxxed
  Device = scalar-i500
  Media Type = LTO-5
  AutoChanger = yes
  Maximum Concurrent Jobs = 6
}
isi

isi

2019-07-11 06:39

reporter   ~0003438

OK, If I look at it now, I will change the IP Adresses into names and test again.
arogge

arogge

2019-07-11 14:43

manager   ~0003443

As far as I can tell you should have only two storages: the Bareos storage "i500" and *one* paired NDMP storage.
I think the second NDMP storage leads to confusion inside of Bareos.
isi

isi

2019-07-12 06:41

reporter   ~0003445

When I try to copy my Tape Backups to a copy Tape to tranfer to vault
I get this:

12-Jul 06:22 bareos-dir JobId 452: Fatal error: JobId 413 cannot Copying using the same read and write storage.
12-Jul 06:22 bareos-dir JobId 452: Error: Bareos bareos-dir 19.1.2 (01Feb19):
 
Thats why I added a second NDMP Storage in the first place.

I tested with a i500-copy paired to NDMPCopy. It gives also the TLS Error.
So it looks like its not possible at the moment to have a NDMP Copy Pool ? Or am I doing it wrong ?
arogge

arogge

2019-07-12 13:32

manager   ~0003454

Our Testsuite uses the following:
Configure two NDMP Storages in director (copy source and copy target) each with its own Bareos Storage as Paired Storage.
The Bareos Storages may point to the same Device on the same SD.

This is what our automated test does. As 18.2.5 has of course been tested with the testsuite before release, I'm quite sure this is some kind of configuration issue (and maybe a lack of proper documentation for NDMP).
isi

isi

2019-07-12 13:47

reporter   ~0003456

Thanks for your time and your support. I will try as you suggested.
Case can be closed at your convenience.
arogge

arogge

2019-07-12 13:53

manager   ~0003457

I'd really appreciate your feedback whether you got it working or not, so we know what to add to the documentation exactly.
isi

isi

2019-07-15 08:37

reporter   ~0003460

Can't get this to work. If I disable TLS I get different error Messages like:

15-Jul 08:27 bareos-dir JobId 531: Fatal error: Password encoding is not MD5. You are probably restoring a NDMP Backup with a restore job not configured for NDMP protocol.
15-Jul 08:27 bareos-dir JobId 531: Fatal error: Director unable to authenticate with Storage daemon at "bb8-storage.elkb.de:10000". Possible causes:
Passwords or names not the same or TLS negotiation problem or Maximum Concurrent Jobs exceeded on the SD or SD networking messed up (restart daemon).

Changed encoding to MD5, double checked Username and Password and Maximum Concurrent Jobs.

Have to give up for now.
arogge

arogge

2019-07-15 10:00

manager   ~0003464

Could you also provide your Pool, Job (and if applicable JobDefs) configuration?
This should be simple and work out of the box.
isi

isi

2019-07-15 10:13

reporter   ~0003468

sure:
Pool {
  Name = NDMPTape
     Pool Type = Backup
     Recycle = yes
     AutoPrune = yes
     Storage = NDMPTape
     Recycle Pool = Scratch
     Maximum Block Size = 1048576
     Volume Retention = 6 Month
     Next Pool = NDMPCopy
}
Pool {
  Name = NDMPCopy
     Pool Type = Backup
     Recycle = yes
     AutoPrune = yes
     Storage = NDMPCopy
     Recycle Pool = Scratch
     Maximum Block Size = 1048576
     Volume Retention = 6 Month
}

Job {
  Name = copy-asterix-lka-copypool
  client = None
  Type = Copy
  Messages = Standard
  Pool = NDMPTape
  File Set = None
  Schedule = sun-22
  Selection Type = SQL Query
  Selection Pattern = "
    SELECT MAX(jobid)
          FROM job
          WHERE name='asterix-lka-ndmp-job'
      AND type='B'
      AND level='F'
      AND jobstatus='T';
"
  Enabled = no
}
# Fake client for copy jobs
Client {
  Name = None
  Address = localhost
  Protocol = NDMPv4
  Auth Type = Clear
  Username = ndmp-copy
  Password = xxxxed
  Catalog = MyCatalog
  Enabled = yes
}

# Fake fileset for copy jobs
Fileset {
  Name = None
  Include {
    Options {
    }
  }
}
# ndmp copy agent in bareos-sd.d/ndmp
Ndmp {
  Name = None
  Username = ndmp-copy
  Password = xxxxed
  AuthType = Clear
}
arogge

arogge

2019-07-15 10:31

manager   ~0003469

A copy Job doesn't require a Client, so you can remove the directive.
If you want to keep the dummy, remove the NDMP settings. Even though this is an NDMP Backup copying it is not a NDMP operation.
isi

isi

2019-07-15 11:06

reporter   ~0003473

Last edited: 2019-07-15 11:59

>> Even though this is an NDMP Backup copying it is not a NDMP operation. <<

This was the right hint !!! If it's not NDMP, the Copy Storage must look like this:

Storage {
  Name = NDMPCopy
  Address = bb8.elkb.de # N.B. Use a fully qualified name here
  Password = "xxxxxed"
  Device = scalar-i500
  Media Type = LTO-5
  PairedStorage = i500-copy
  Maximum Concurrent Jobs = 6
}

You made my day !!!
It's loading Tapes and copying Blocks. So Cool, thank you very much.

isi

isi

2019-07-15 12:13

reporter   ~0003474

One don'T even need a different Paired Storage, it also works with:
PairedStorage = i500
arogge

arogge

2019-07-15 12:28

manager   ~0003475

Do you have any idea what we could add to the documentation so nobody else runs into this issue?
isi

isi

2019-07-15 12:52

reporter   ~0003477

I was so focused on NDMP, that for me it was not clear that copying
an NDMP Backup is not an NDMP Operation and that the Copy Storage
is just a "normal" Bareos Storage.

No Need for a Fake Client and a Fake Fileset, and no need for a 2nd
ndmp_tape_agent. No NDMP at all involved here.
arogge

arogge

2019-07-15 13:13

manager   ~0003478

Added note to NDMP documentation so this case becomes clearer.
arogge

arogge

2019-07-15 13:22

manager   ~0003479

Fix committed to bareos master branch with changesetid 11595.

Related Changesets

bareos: master 6bc2093d

2019-07-15 15:00

arogge

Ported: N/A

Details Diff
docs: Add note for NDMP copy/migrate jobs

Fixes 0001091: NDMP to NDMP Copy Job Fails

Previously the docs weren't clear concerning copying and migrating
NDMP jobs. This patch adds at least a short note.
Affected Issues
0001091
mod - docs/manuals/source/TasksAndConcepts/NdmpBackupsWithBareos.rst Diff File

Issue History

Date Modified Username Field Change
2019-06-11 13:41 isi New Issue
2019-07-09 09:04 arogge Assigned To => arogge
2019-07-09 09:04 arogge Status new => feedback
2019-07-09 09:04 arogge Note Added: 0003421
2019-07-09 09:04 arogge Assigned To arogge =>
2019-07-09 16:24 isi Note Added: 0003428
2019-07-09 16:24 isi Status feedback => new
2019-07-10 12:29 isi Note Added: 0003429
2019-07-10 17:02 arogge Assigned To => arogge
2019-07-10 17:02 arogge Status new => feedback
2019-07-10 17:02 arogge Note Added: 0003430
2019-07-10 17:44 arogge Assigned To arogge =>
2019-07-11 06:31 isi Note Added: 0003437
2019-07-11 06:31 isi Status feedback => new
2019-07-11 06:39 isi Note Added: 0003438
2019-07-11 14:43 arogge Assigned To => arogge
2019-07-11 14:43 arogge Status new => feedback
2019-07-11 14:43 arogge Note Added: 0003443
2019-07-12 06:41 isi Note Added: 0003445
2019-07-12 06:41 isi Status feedback => assigned
2019-07-12 13:32 arogge Assigned To arogge =>
2019-07-12 13:32 arogge Status assigned => feedback
2019-07-12 13:32 arogge Note Added: 0003454
2019-07-12 13:47 isi Note Added: 0003456
2019-07-12 13:47 isi Status feedback => new
2019-07-12 13:53 arogge Status new => feedback
2019-07-12 13:53 arogge Note Added: 0003457
2019-07-15 08:37 isi Note Added: 0003460
2019-07-15 08:37 isi Status feedback => new
2019-07-15 10:00 arogge Status new => feedback
2019-07-15 10:00 arogge Note Added: 0003464
2019-07-15 10:13 isi Note Added: 0003468
2019-07-15 10:13 isi Status feedback => new
2019-07-15 10:31 arogge Status new => feedback
2019-07-15 10:31 arogge Note Added: 0003469
2019-07-15 11:06 isi Note Added: 0003473
2019-07-15 11:06 isi Status feedback => new
2019-07-15 11:59 isi Note Edited: 0003473
2019-07-15 12:13 isi Note Added: 0003474
2019-07-15 12:28 arogge Assigned To => arogge
2019-07-15 12:28 arogge Status new => feedback
2019-07-15 12:28 arogge Note Added: 0003475
2019-07-15 12:52 isi Note Added: 0003477
2019-07-15 12:52 isi Status feedback => assigned
2019-07-15 13:13 arogge Status assigned => resolved
2019-07-15 13:13 arogge Resolution open => fixed
2019-07-15 13:13 arogge Note Added: 0003478
2019-07-15 13:22 arogge Changeset attached => bareos master 6bc2093d
2019-07-15 13:22 arogge Note Added: 0003479
2019-12-18 15:21 arogge Status resolved => closed