View Issue Details

Related ChangesetsJump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0000345bareos-coreGeneralpublic2014-10-01 11:432015-03-25 19:18
Reporterompsg Assigned To 
PrioritynormalSeveritymajorReproducibilityalways
Status closedResolutionfixed 
PlatformLinuxOSUbuntuOS Version14.04
Product Version14.2.1 
Summary0000345: NDMP restore does not work
DescriptionWhen trying to restore a successful NDMP backup the following error occurs:

01-Okt 09:57 bareos-dir JobId 65: Start Restore Job Restore_netapp2040.2014-10-01_09.57.15_11
01-Okt 09:57 bareos-dir JobId 65: Using Device "FileStorage" to read.
01-Okt 09:57 bareos-dir JobId 65: Opening tape drive GJLD-ENOO-KOAN-EELP-BEAA-NPJC-CKMM-IMGH@/vol/_test1 read-only
01-Okt 09:57 bareos-dir JobId 65: Commanding tape drive to rewind
01-Okt 09:57 bareos-sd JobId 65: Ready to read from volume "Full-0001" on device "FileStorage" (/var/lib/bareos/storage).
01-Okt 09:57 bareos-sd JobId 65: Forward spacing Volume "Full-0001" to file:block 3:1922472554.
01-Okt 09:57 bareos-dir JobId 65: ERR NDMP4_DATA_START_RECOVER NDMP4_ILLEGAL_ARGS_ERR
01-Okt 09:57 bareos-dir JobId 65: Waiting for operation to halt
01-Okt 09:57 bareos-dir JobId 65: LOG_MESSAGE: 'Nlist of 0 entries. No restore to perform.'
01-Okt 09:57 bareos-dir JobId 65: Async request NDMP4_NOTIFY_DATA_HALTED
01-Okt 09:57 bareos-dir JobId 65: Async request NDMP4_LOG_MESSAGE
01-Okt 09:57 bareos-dir JobId 65: LOG_MESSAGE: 'Aborted by client'
01-Okt 09:57 bareos-dir JobId 65: Commanding tape drive to rewind
01-Okt 09:57 bareos-dir JobId 65: Closing tape drive GJLD-ENOO-KOAN-EELP-BEAA-NPJC-CKMM-IMGH@/vol/_test1
01-Okt 09:57 bareos-dir JobId 65: Operation halted, stopping
01-Okt 09:57 bareos-dir JobId 65: Operation ended questionably
01-Okt 09:57 bareos-dir JobId 65: Error: Bareos bareos-dir 14.2.1 (12Sep14):
  Build OS: x86_64-pc-linux-gnu ubuntu Ubuntu 14.04 LTS
  JobId: 65
  Job: Restore_netapp2040.2014-10-01_09.57.15_11
  Restore Client: netapp2040-ndmp
  Start time: 01-Okt-2014 09:57:17
  End time: 01-Okt-2014 09:57:26
  Elapsed time: 9 secs
  Files Expected: 1
  Files Restored: 0
  Bytes Restored: 0
  Rate: 0.0 KB/s
  SD termination status: Running
  Termination: *** Restore Error ***
Steps To ReproduceRestore a NDMP Backup.
Additional InformationClient to Backup is a NetApp FAS 2040.

Bareos configuration for the client:

Client {
  Name = netapp2040-ndmp
  Address = netapp2040.domain.local
  Port = 10000
  Protocol = NDMPv4
  Auth Type = MD5
  Username = "username"
  Password = "password"
}

JobDefs {
  Name = "DefaultNDMPJob"
  Type = Backup
  Protocol = NDMP
  Backup Format = smtape
  Level = Incremental
  Schedule = "WeeklyCycle"
  Storage = NDMPFile
  Messages = Standard
  Pool = Incremental
  Priority = 10
  Write Bootstrap = "/var/lib/bareos/%c.bsr"
  Full Backup Pool = Full
  Differential Backup Pool = Differential
  Incremental Backup Pool = Incremental
}


Job {
  Enabled = no
  Name = "Backup_netapp2040"
  JobDefs = "DefaultNDMPJob"
  Client = netapp2040-ndmp
  FileSet = "netapp2040"
  Backup Format = smtape
}

FileSet {
  Name = netapp2040
  Include {
    Options {
       meta = "SMTAPE_DELETE_SNAPSHOT=Y"
    }
    File = /vol/_test1
  }
}

Job {
  Name = "Restore_netapp2040"
  client = "netapp2040-ndmp"
  Type = Restore
  Protocol = NDMP
  Backup Format = smtape
  Pool = Incremental
  FileSet = "netapp2040 restore"
  Storage = NDMPFile
  Messages = Standard
  Where = /
}

Fileset {
  Name = "netapp2040 restore"
  Include {
    Options {
       meta = "SMTAPE_BREAK_MIRROR=Y"
    }
    File = /vol/_test1
  }
}
TagsNo tags attached.

Relationships

related to 0000330 closed NDMP protocol error 
related to 0000374 closed NDMP SMTAPE restores fail with "SMTAPE: Error: Read Socket received EOF." 

Activities

mvwieringen

mvwieringen

2014-10-06 11:19

developer   ~0000993

From the error its seems that SMTAPE needs a namelist on restore.
A simple patch for that is attached to this bug report. So if you
compile the code yourself you can check out if that a least solves
this particular problem. For that you need the current 14.2 code and
the patch attached to this bug (a change from false to true in a
mapping table that contains NDMP specific parameters for certain
types of NDMP backup.)

Warning however 14.2.1 is beta and we added quite some new stuff for NDMP
which has problems especially with NetAPP as it seems to send data out
of order which we currently don't handle well. I will attach a link
to the bug describing that problem.

For now I would use the stable 13.2 version. Given that NDMP is not really
a standard and every implementation is different it may take some time until
we support each platform right.
mvwieringen

mvwieringen

2014-10-06 11:19

developer  

0001-NDMP-SMTAPE-restore-does-need-namelist.patch (912 bytes)    
From a5bf15878e4a3760cfe4a5197dce60a8ffc72fb0 Mon Sep 17 00:00:00 2001
From: Marco van Wieringen <marco.van.wieringen@bareos.com>
Date: Mon, 6 Oct 2014 11:12:59 +0200
Subject: [PATCH] NDMP SMTAPE restore does need namelist.

Fixes #345: NDMP restore does not work
---
 src/dird/ndmp_dma.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/src/dird/ndmp_dma.c b/src/dird/ndmp_dma.c
index 7319343..0ab6396 100644
--- a/src/dird/ndmp_dma.c
+++ b/src/dird/ndmp_dma.c
@@ -179,7 +179,7 @@ struct ndmp_backup_format_option {
 static ndmp_backup_format_option ndmp_backup_format_options[] = {
    { (char *)"dump", true, true, true, true },
    { (char *)"tar", true, false, true, true },
-   { (char *)"smtape", false, false, false, false },
+   { (char *)"smtape", false, false, false, true },
    { (char *)"zfs", false, true, false, true },
    { NULL, false, false, false }
 };
-- 
1.7.3.2
0001-NDMP-SMTAPE-restore-does-need-namelist.patch (912 bytes)   
ompsg

ompsg

2014-10-09 10:28

reporter   ~0000997

For information on version 13.2.2 of the error also occurs. The NetApp reports: 'Nlist of 0 entries. No restore to perform.'
ompsg

ompsg

2014-10-09 12:44

reporter   ~0000998

I've tried the patch. The restore still does not work. The error message is slightly different:

09-Okt 12:23 baroes-src-dir JobId 2: Start Restore Job Restore_netapp2040.2014-10-09_12.23.56_05
09-Okt 12:23 baroes-src-dir JobId 2: Using Device "FileStorage" to read.
09-Okt 12:23 baroes-src-dir JobId 2: Opening tape drive OAFJ-PLFI-BOEE-MDCJ-DMOE-GPDD-CABG-MMDL@/vol/_test1 read-only
09-Okt 12:23 baroes-src-dir JobId 2: Commanding tape drive to rewind
09-Okt 12:23 baroes-src-sd JobId 2: Ready to read from volume "Full-0001" on device "FileStorage" (/tmp).
09-Okt 12:23 baroes-src-sd JobId 2: Forward spacing Volume "Full-0001" to file:block 0:203.
09-Okt 12:23 baroes-src-dir JobId 2: ERR NDMP4_DATA_START_RECOVER NDMP4_IO_ERR
09-Okt 12:23 baroes-src-dir JobId 2: Waiting for operation to halt
09-Okt 12:23 baroes-src-dir JobId 2: LOG_MESSAGE: 'Storing of nlist entries failed.'
09-Okt 12:24 baroes-src-dir JobId 2: Async request NDMP4_NOTIFY_DATA_HALTED
09-Okt 12:24 baroes-src-dir JobId 2: Async request NDMP4_LOG_MESSAGE
09-Okt 12:24 baroes-src-dir JobId 2: LOG_MESSAGE: 'Aborted by client'
09-Okt 12:24 baroes-src-dir JobId 2: Commanding tape drive to rewind
09-Okt 12:24 baroes-src-dir JobId 2: Closing tape drive OAFJ-PLFI-BOEE-MDCJ-DMOE-GPDD-CABG-MMDL@/vol/_test1
09-Okt 12:24 baroes-src-dir JobId 2: Operation halted, stopping
09-Okt 12:24 baroes-src-dir JobId 2: Operation ended questionably
09-Okt 12:24 baroes-src-dir JobId 2: Error: Bareos baroes-src-dir 14.2.1 (12Sep14):
  Build OS: x86_64-unknown-linux-gnu ubuntu Ubuntu 14.04.1 LTS
  JobId: 2
  Job: Restore_netapp2040.2014-10-09_12.23.56_05
  Restore Client: netapp2040-ndmp
  Start time: 09-Okt-2014 12:23:58
  End time: 09-Okt-2014 12:24:07
  Elapsed time: 9 secs
  Files Expected: 1
  Files Restored: 0
  Bytes Restored: 0
  Rate: 0.0 KB/s
  SD termination status: Running
  Termination: *** Restore Error ***
mvwieringen

mvwieringen

2014-10-09 14:51

developer   ~0000999

Hmm seems we are not the only ones having those problems with the
'Storing of nlist entries failed'

http://www.pickysysadmin.ca/2012/09/12/symantec-backup-exec-ndmp-backups-fail-the-verify-stage-when-backing-up-from-a-netapp/

Given that the report is on a old version I guess this is some other problem
but given that NetAPP is so much closed source its going to be hard to trouble
shoot this problem.

You could try cranking up the debugging level of the NDMP protocol but you
are mostly on your own for this particular problem. For a community problem
NDMP is probably a bit out of scope.

You are not trying to restore to the original volume right ? e.g. you should
specify an alternative restore point in the where of the NDMP restore as
otherwise it will try to restore to the original volume and that will
probably blow up.
ompsg

ompsg

2014-10-09 16:27

reporter   ~0001000

I tried the original volume to restore. But the attempts to change the recovery path also failed.

On the linked page is written, the error was corrected with the release 7.3.7p1. I am at version 7.3.7. I'm considering if I perform the update and will continue to perform tests.

Thank you for your help.
mvwieringen

mvwieringen

2014-10-09 17:47

developer   ~0001001

Fix committed to bareos master branch with changesetid 2201.
mvwieringen

mvwieringen

2014-10-10 21:42

developer   ~0001004

Fix committed to bareos bareos-13.2 branch with changesetid 2236.
mvwieringen

mvwieringen

2014-10-11 21:10

developer   ~0001005

Ok the patch is applied to all supported branches that had this bug.

B.T.W. the docs have something interesting on NDMP restore of the NetAPP.

The destination volume for restore must be in a "restricted" state.
Refer to your Network Appliance documentation on how to set a volume to a
restricted state.
mvwieringen

mvwieringen

2015-03-25 16:51

developer   ~0001407

Fix committed to bareos2015 bareos-14.2 branch with changesetid 4599.
joergs

joergs

2015-03-25 19:18

developer   ~0001558

Due to the reimport of the Github repository to bugs.bareos.org, the status of some tickets have been changed. These tickets will be closed again.
Sorry for the noise.

Related Changesets

bareos: master f085b9c6

2014-10-06 11:12

mvwieringen

Ported: N/A

Details Diff		 NDMP SMTAPE restore does need namelist.

Fixes 0000345: NDMP restore does not work		 Affected Issues
0000345
mod - src/dird/ndmp_dma.c Diff File

bareos: bareos-13.2 adc874f8

2014-10-06 11:12

mvwieringen

Ported: N/A

Details Diff		 NDMP SMTAPE restore does need namelist.

Fixes 0000345: NDMP restore does not work		 Affected Issues
0000345
mod - src/dird/ndmp_dma.c Diff File

bareos2015: bareos-13.2 7690a205

2014-10-06 13:12

mvwieringen

Ported: N/A

Details Diff		 NDMP SMTAPE restore does need namelist.

Fixes 0000345: NDMP restore does not work		 Affected Issues
0000345
mod - src/dird/ndmp_dma.c Diff File

bareos2015: bareos-14.2 32659c88

2014-10-06 13:12

mvwieringen

Ported: N/A

Details Diff		 NDMP SMTAPE restore does need namelist.

Fixes 0000345: NDMP restore does not work		 Affected Issues
0000345
mod - src/dird/ndmp_dma.c Diff File

Issue History

Date Modified Username Field Change
2014-10-01 11:43 ompsg New Issue
2014-10-06 11:19 mvwieringen Note Added: 0000993
2014-10-06 11:19 mvwieringen Status new => feedback
2014-10-06 11:19 mvwieringen File Added: 0001-NDMP-SMTAPE-restore-does-need-namelist.patch
2014-10-06 11:21 mvwieringen Relationship added related to 0000330
2014-10-09 10:28 ompsg Note Added: 0000997
2014-10-09 10:28 ompsg Status feedback => new
2014-10-09 12:44 ompsg Note Added: 0000998
2014-10-09 14:51 mvwieringen Note Added: 0000999
2014-10-09 14:52 mvwieringen Assigned To => mvwieringen
2014-10-09 14:52 mvwieringen Status new => feedback
2014-10-09 16:27 ompsg Note Added: 0001000
2014-10-09 16:27 ompsg Status feedback => assigned
2014-10-09 17:47 mvwieringen Changeset attached => bareos master f085b9c6
2014-10-09 17:47 mvwieringen Note Added: 0001001
2014-10-09 17:47 mvwieringen Status assigned => resolved
2014-10-09 17:47 mvwieringen Resolution open => fixed
2014-10-09 17:50 mvwieringen Status resolved => feedback
2014-10-09 17:50 mvwieringen Resolution fixed => reopened
2014-10-10 21:42 mvwieringen Changeset attached => bareos bareos-13.2 adc874f8
2014-10-10 21:42 mvwieringen Note Added: 0001004
2014-10-10 21:42 mvwieringen Status feedback => resolved
2014-10-11 21:10 mvwieringen Note Added: 0001005
2014-10-11 21:10 mvwieringen Status resolved => feedback
2014-12-18 14:31 mvwieringen Relationship added related to 0000376
2014-12-18 14:31 mvwieringen Relationship deleted related to 0000376
2014-12-18 14:32 mvwieringen Relationship added related to 0000374
2015-02-10 14:51 mvwieringen Assigned To mvwieringen =>
2015-02-10 14:51 mvwieringen Status feedback => closed
2015-02-10 14:51 mvwieringen Resolution reopened => unable to reproduce
2015-03-25 16:51 mvwieringen Changeset attached => bareos2015 bareos-13.2 7690a205
2015-03-25 16:51 mvwieringen Changeset attached => bareos2015 bareos-14.2 32659c88
2015-03-25 16:51 mvwieringen Note Added: 0001407
2015-03-25 16:51 mvwieringen Status closed => resolved
2015-03-25 16:51 mvwieringen Resolution unable to reproduce => fixed
2015-03-25 19:18 joergs Note Added: 0001558
2015-03-25 19:18 joergs Status resolved => closed