View Issue Details

IDProjectCategoryView StatusLast Update
0001056bareos-core[All Projects] documentationpublic2019-12-12 13:28
ReporterOweiAssigned Toarogge 
PriorityhighSeveritymajorReproducibilityalways
Status closedResolutionfixed 
PlatformLinuxOSCentOSOS Version7
Product Version17.2.4 
Fixed in Version 
Summary0001056: NDMP restore on 16.2.5 and above does not fill NDMP environment correctly
DescriptionHi,

We recently upgraded our Bareos 16.2.4 installation to 17.2.4. Since then we can no longer restore our backups. This is very critical as we have to do a restore right now and we can’t. I tested several hours and restores work fine on 16.2.4 but not under 17.2.4, 17.2.5 and also 18.2.4. Have not yet tested nightly. I can imagine that since there were heavy modifications in 17.2.4 for NDMP Native Support something is now broken. Looking at the client NDMP Debug log there is one strange behaviour on releases > 16.2.4. ZFS_FILE_SIZE is shown as NULL.

I will attach the debug logs later as I’m currently not at my workstation.
TagsNDMP, Restore broken in 17.2.4, ZFS
bareos-master: impactyes
bareos-master: actionfixed
bareos-19.2: impact
bareos-19.2: action
bareos-18.2: impactyes
bareos-18.2: actionfixed
bareos-17.2: impactyes
bareos-17.2: actionnone
bareos-16.2: impactyes
bareos-16.2: actionnone
bareos-15.2: impactno
bareos-15.2: actionnone
bareos-14.2: impactno
bareos-14.2: actionnone
bareos-13.2: impactno
bareos-13.2: actionnone
bareos-12.4: impactno
bareos-12.4: actionnone

Relationships

related to 0001152 new Release Bareos 18.2.7 

Activities

Owei

Owei

2019-02-16 17:18

reporter   ~0003266

I investigated further comparing the client ndmp debug logs between restore under 16.2.4 and 17.2.5. It seems that the restore running under 17.2.5 some very important info is not passed on to the client. Under 16.2.4 the following entries are logged in the client NDMP debug log file:

5][ndmpd_save_env:526]:env(FILESYSTEM): "/fridge/testsnap@monthly03"
 2/16 17:06:03 DEBUG: [5][ndmpd_save_env:526]:env(HIST): "n"
 2/16 17:06:03 DEBUG: [5][ndmpd_save_env:526]:env(LEVEL): "0"
 2/16 17:06:03 DEBUG: [5][ndmpd_save_env:526]:env(TYPE): "zfs"
 2/16 17:06:03 DEBUG: [5][ndmpd_save_env:526]:env(UPDATE): "y"
 2/16 17:06:03 DEBUG: [5][ndmpd_save_env:526]:env(USER): "root"
 2/16 17:06:03 DEBUG: [5][ndmpd_save_env:526]:env(ZFS_BACKUP_SIZE): "2152710680"

Under 17.2.5 only the following are shown:

 2/16 17:04:53 DEBUG: [5][ndmpd_save_env:526]:env(TYPE): "zfs"
 2/16 17:04:53 DEBUG: [5][ndmpd_save_env:526]:env(USER): "root"

I guess these are very important for the restore.
Owei

Owei

2019-02-16 17:20

reporter   ~0003267

Bareos 17.2.5 Client NDMP Debug Log

ndmplog.0_172_rst (6,518 bytes)
Owei

Owei

2019-02-16 17:21

reporter   ~0003268

Bareos 16.2.4 Client NDMP Debug Log

ndmplog.0_162_rst (6,810 bytes)
arogge

arogge

2019-03-13 10:14

developer   ~0003278

This issue was introduced with "NDMP: fix restore problems with jobs with multiple BSRs" (see https://github.com/bareos/bareos/commit/f733669829db24e69e2ac6977a46f07e6a85ab62)
arogge_adm

arogge_adm

2019-04-10 18:58

administrator   ~0003323

Fix committed to bareos bareos-18.2 branch with changesetid 11105.

Related Changesets

bareos: master a2014835

2019-03-14 09:28:25

arogge_adm

Ported: N/A

Details Diff
dird: redesign GetNdmpEnvironmentString() API

Fixes 0001056: NDMP restore on 16.2.5 and above does
not fill NDMP environment correctly

Previously one overload of the function
GetNdmpEnvironmentString() wanted a JobDbRecord*
and expected jr->VolSessionId and jr->VolSessionTime
to contain the values for the volume from which the
restore happens. These had to be filled manually
before calling GetNdmlEnvironmentString() which had
not been done since 16.2.5 resulting in 0001056.

This patch now redesigns the API for all overloads
of GetNdmpEnviromentString() to make it harder
to misuse.

We also add a new struct VolumeSessionInfo to wrap
a pair of VolumeSessionId and VolumeSessionTime.
These two numbers are only meaningful together, so
they now have their own container.
Affected Issues
0001056
mod - core/src/cats/cats.h Diff File
mod - core/src/cats/sql_get.cc Diff File
mod - core/src/dird/ndmp_dma_restore_NDMP_BAREOS.cc Diff File
mod - core/src/dird/ndmp_dma_restore_NDMP_NATIVE.cc Diff File
add - core/src/lib/volume_session_info.h Diff File

bareos: bareos-18.2 e514ec5b

2019-03-14 09:28:25

arogge_adm

Ported: N/A

Details Diff
dird: redesign GetNdmpEnvironmentString() API

Fixes 0001056: NDMP restore on 16.2.5 and above does
not fill NDMP environment correctly

Previously one overload of the function
GetNdmpEnvironmentString() wanted a JobDbRecord*
and expected jr->VolSessionId and jr->VolSessionTime
to contain the values for the volume from which the
restore happens. These had to be filled manually
before calling GetNdmlEnvironmentString() which had
not been done since 16.2.5 resulting in 0001056.

This patch now redesigns the API for all overloads
of GetNdmpEnviromentString() to make it harder
to misuse.

We also add a new struct VolumeSessionInfo to wrap
a pair of VolumeSessionId and VolumeSessionTime.
These two numbers are only meaningful together, so
they now have their own container.
Affected Issues
0001056
mod - core/src/cats/cats.h Diff File
mod - core/src/cats/sql_get.cc Diff File
mod - core/src/dird/ndmp_dma_restore_NDMP_BAREOS.cc Diff File
mod - core/src/dird/ndmp_dma_restore_NDMP_NATIVE.cc Diff File
add - core/src/lib/volume_session_info.h Diff File

bareos: master 020ca8e3

2019-03-27 12:05:19

arogge_adm


Committer: GitHub

Ported: N/A

Details Diff
Merge pull request 0000160 from bareos/dev/arogge/master/fix-1056

dird: redesign GetNdmpEnvironmentString() API
Affected Issues
0001056
mod - core/src/cats/cats.h Diff File
mod - core/src/cats/sql_create.cc Diff File
mod - core/src/cats/sql_get.cc Diff File
mod - core/src/dird/ndmp_dma_restore_NDMP_BAREOS.cc Diff File
mod - core/src/dird/ndmp_dma_restore_NDMP_NATIVE.cc Diff File
mod - core/src/lib/edit.h Diff File
add - core/src/lib/volume_session_info.h Diff File
mod - core/src/ndmp/ndmos.h Diff File
mod - core/src/ndmp/ndmp4_pp.c Diff File

Issue History

Date Modified Username Field Change
2019-02-14 08:02 Owei New Issue
2019-02-16 17:14 Owei Tag Attached: NDMP
2019-02-16 17:14 Owei Tag Attached: ZFS
2019-02-16 17:14 Owei Tag Attached: Restore broken in 17.2.4
2019-02-16 17:18 Owei Note Added: 0003266
2019-02-16 17:20 Owei File Added: ndmplog.0_172_rst
2019-02-16 17:20 Owei Note Added: 0003267
2019-02-16 17:21 Owei File Added: ndmplog.0_162_rst
2019-02-16 17:21 Owei Note Added: 0003268
2019-03-13 10:12 arogge Assigned To => arogge
2019-03-13 10:12 arogge Status new => assigned
2019-03-13 10:14 arogge Status assigned => confirmed
2019-03-13 10:14 arogge Note Added: 0003278
2019-03-13 10:19 arogge Summary Restore of ZFS NDMP Backup broken in all versions > 16.2.x!!! => NDMP restore on 16.2.5 and above does not fill NDMP environment correctly
2019-03-13 10:19 arogge bareos-master: impact => yes
2019-03-13 10:19 arogge bareos-master: action => will care
2019-03-13 10:19 arogge bareos-18.2: impact => yes
2019-03-13 10:19 arogge bareos-18.2: action => will care
2019-03-13 10:19 arogge bareos-17.2: impact => yes
2019-03-13 10:19 arogge bareos-17.2: action => will care
2019-03-13 10:19 arogge bareos-16.2: impact => yes
2019-03-13 10:19 arogge bareos-16.2: action => none
2019-03-13 10:19 arogge bareos-15.2: impact => no
2019-03-13 10:19 arogge bareos-15.2: action => none
2019-03-13 10:19 arogge bareos-14.2: impact => no
2019-03-13 10:19 arogge bareos-14.2: action => none
2019-03-13 10:19 arogge bareos-13.2: impact => no
2019-03-13 10:19 arogge bareos-13.2: action => none
2019-03-13 10:19 arogge bareos-12.4: impact => no
2019-03-13 10:19 arogge bareos-12.4: action => none
2019-03-14 17:13 arogge bareos-17.2: action will care => none
2019-04-10 15:23 arogge_adm Changeset attached => bareos master 020ca8e3
2019-04-10 18:58 arogge_adm Changeset attached => bareos master a2014835
2019-04-10 18:58 arogge_adm Changeset attached => bareos bareos-18.2 e514ec5b
2019-04-10 18:58 arogge_adm Note Added: 0003323
2019-05-13 09:37 arogge Status confirmed => resolved
2019-05-13 09:37 arogge Resolution open => fixed
2019-05-13 09:37 arogge bareos-master: action will care => fixed
2019-05-13 09:37 arogge bareos-18.2: action will care => fixed
2019-05-13 09:37 arogge Status resolved => closed
2019-12-12 13:28 arogge Relationship added related to 0001152