View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0001413bareos-corebconsolepublic2021-12-27 15:292021-12-28 09:38
Reporterjcottin Assigned Tobruno-at-bareos  
PriorityhighSeveritymajorReproducibilityalways
Status resolvedResolutionno change required 
PlatformLinuxOSDebianOS Version10
Product Version21.0.0 
Summary0001413: While restoring a directory using always incremental scheme, Bareos looks for a volume in the wrong storage.
DescriptionI configured the Always Incremental scheme using 2 different storage (FILE) as advised in the documentation.
-----------------------------------
https://docs.bareos.org/TasksAndConcepts/AlwaysIncrementalBackupScheme.html?highlight=job#storages-and-pools

While restoring a directory using always incremental scheme, Bareos looks for a volume in the wrong storage.

he job will require the following
   Volume(s) Storage(s) SD Device(s)
===========================================================================

    AI-Consolidated-vm-aiqi-linux-test-backup-0006 FileStorage-AI-Consolidated FileStorage-AI-Consolidated
    AI-Incremental-vm-aiqi-linux-test-backup0012 FileStorage-AI-Incremental FileStorage-AI-Incremental

It looks for AI-Incremental-vm-aiqi-linux-test-backup0012 in FileStorage-AI-Consolidated.
it should look for it in FileStorage-AI-Incremental.

Is there a problem with my setup ?
Steps To ReproduceUsing bconsole I target a backup before : 2021-12-27 19:00:00
I can find 3 backup (1 Full, 2 Incremental)
=======================================================
+-------+-------+----------+----------------+---------------------+------------------------------------------------+
| jobid | level | jobfiles | jobbytes | starttime | volumename |
+-------+-------+----------+----------------+---------------------+------------------------------------------------+
| 24 | F | 108,199 | 13,145,763,765 | 2021-12-25 08:06:41 | AI-Consolidated-vm-aiqi-linux-test-backup-0006 |
| 27 | I | 95 | 68,530 | 2021-12-25 20:00:04 | AI-Incremental-vm-aiqi-linux-test-backup0008 |
| 32 | I | 40 | 1,322,314 | 2021-12-26 20:00:09 | AI-Incremental-vm-aiqi-linux-test-backup0012 |
+-------+-------+----------+----------------+---------------------+------------------------------------------------+
-----------------------------------
$ cd /var/lib/mysql.dumps/wordpressdb/
cwd is: /var/lib/mysql.dumps/wordpressdb/
-----------------------------------
$ dir
-rw-r--r-- 1 0 (root) 112 (bareos) 1830 2021-12-25 07:55:02 /var/lib/mysql.dumps/wordpressdb/%create.gz
-rw-r--r-- 1 0 (root) 112 (bareos) 149 2021-12-25 07:55:02 /var/lib/mysql.dumps/wordpressdb/%tables
-rw-r--r-- 1 0 (root) 112 (bareos) 783 2021-12-25 07:55:02 /var/lib/mysql.dumps/wordpressdb/wp_commentmeta.sql.gz
-rw-r--r-- 1 0 (root) 112 (bareos) 1161 2021-12-25 07:55:02 /var/lib/mysql.dumps/wordpressdb/wp_comments.sql.gz
-rw-r--r-- 1 0 (root) 112 (bareos) 869 2021-12-25 07:55:02 /var/lib/mysql.dumps/wordpressdb/wp_links.sql.gz
-rw-r--r-- 1 0 (root) 112 (bareos) 235966 2021-12-25 07:55:02 /var/lib/mysql.dumps/wordpressdb/wp_options.sql.gz
-rw-r--r-- 1 0 (root) 112 (bareos) 830 2021-12-25 07:55:02 /var/lib/mysql.dumps/wordpressdb/wp_postmeta.sql.gz
-rw-r--r-- 1 0 (root) 112 (bareos) 3470 2021-12-25 07:55:02 /var/lib/mysql.dumps/wordpressdb/wp_posts.sql.gz
-rw-r--r-- 1 0 (root) 112 (bareos) 770 2021-12-25 07:55:02 /var/lib/mysql.dumps/wordpressdb/wp_term_relationships.sql.gz
-rw-r--r-- 1 0 (root) 112 (bareos) 838 2021-12-25 07:55:02 /var/lib/mysql.dumps/wordpressdb/wp_term_taxonomy.sql.gz
-rw-r--r-- 1 0 (root) 112 (bareos) 780 2021-12-25 07:55:02 /var/lib/mysql.dumps/wordpressdb/wp_termmeta.sql.gz
-rw-r--r-- 1 0 (root) 112 (bareos) 814 2021-12-25 07:55:02 /var/lib/mysql.dumps/wordpressdb/wp_terms.sql.gz
-rw-r--r-- 1 0 (root) 112 (bareos) 1404 2021-12-25 07:55:02 /var/lib/mysql.dumps/wordpressdb/wp_usermeta.sql.gz
-rw-r--r-- 1 0 (root) 112 (bareos) 983 2021-12-25 07:55:02 /var/lib/mysql.dumps/wordpressdb/wp_users.sql.gz
-----------------------------------
$ cd ..
cwd is: /var/lib/mysql.dumps/
-----------------------------------
I mark the folder:
$ mark /var/lib/mysql.dumps/wordpressdb
15 files marked.
$ done
-----------------------------------
The job will require the following
   Volume(s) Storage(s) SD Device(s)
============================================

    AI-Consolidated-vm-aiqi-linux-test-backup-0006 FileStorage-AI-Consolidated FileStorage-AI-Consolidated
    AI-Incremental-vm-aiqi-linux-test-backup0012 FileStorage-AI-Incremental FileStorage-AI-Incremental

Volumes marked with "*" are online.
18 files selected to be restored.

Using Catalog "MyCatalog"
Run Restore job
JobName: RestoreFiles
Bootstrap: /var/lib/bareos/bareos-dir.restore.2.bsr
Where: /tmp/bareos-restores
Replace: Always
FileSet: LinuxAll
Backup Client: vm-aiqi-linux-test-backup-fd
Restore Client: vm-aiqi-linux-test-backup-fd
Format: Native
Storage: FileStorage-AI-Consolidated
When: 2021-12-27 22:10:13
Catalog: MyCatalog
Priority: 10
Plugin Options: *None*
OK to run? (yes/mod/no): yes

I get these two messages.
============================================
27-Dec 22:15 bareos-sd JobId 43: Warning: stored/acquire.cc:286 Read open device "FileStorage-AI-Consolidated" (/var/lib/bareos/storage-AI-Consolidated) Volume "AI-Incremental-vm-aiqi-linux-test-backup0012" failed: ERR=stored/dev.cc:716 Could not open: /var/lib/bareos/storage-AI-Consolidated/AI-Incremental-vm-aiqi-linux-test-backup0012, ERR=No such file or directory

27-Dec 22:15 bareos-sd JobId 43: Please mount read Volume "AI-Incremental-vm-aiqi-linux-test-backup0012" for:
    Job: RestoreFiles.2021-12-27_22.15.29_31
    Storage: "FileStorage-AI-Consolidated" (/var/lib/bareos/storage-AI-Consolidated)
    Pool: Incremental-BareOS
    Media type: File
============================================

Bareos try to find AI-Incremental-vm-aiqi-linux-test-backup0012 in the wrong storage.
Additional Information
===========================================
Job {
  Name = vm-aiqi-linux-test-backup-job
  Client = vm-aiqi-linux-test-backup-fd

  Accurate = yes
  Always Incremental = yes
  Always Incremental Job Retention = 30 days
  Always Incremental Keep Number = 15
  Always Incremental Max Full Age = 60 days

  Level = Incremental
  Type = Backup
  FileSet = "LinuxAll-vm-aiqi-linux-test-backup" # LinuxAll fileset (0000013)
  Schedule = "WeeklyCycleCustomers"
  Storage = FileStorage-AI-Incremental
  Messages = Standard
  Pool = AI-Incremental-vm-aiqi-linux-test-backup
  Priority = 10
  Write Bootstrap = "/var/lib/bareos/%c.bsr"

  Full Backup Pool = AI-Consolidated-vm-aiqi-linux-test-backup # write Full Backups into "Full" Pool (0000005)
  Incremental Backup Pool = AI-Incremental-vm-aiqi-linux-test-backup # write Incr Backups into "Incremental" Pool (0000011)

  Enabled = yes

  RunScript {
    FailJobOnError = Yes
    RunsOnClient = Yes
    RunsWhen = Before
    Command = "sh /SCRIPTS/mysql/pre.mysql.sh"
  }

  Run Script {
    Console = ".bvfs_update jobid=%i"
    RunsWhen = After
    RunsOnClient = No
  }
}
===========================================
Pool {
  Name = AI-Consolidated-vm-aiqi-linux-test-backup
  Pool Type = Backup
  Recycle = yes # Bareos can automatically recycle Volumes
  AutoPrune = yes # Prune expired volumes
  Volume Retention = 360 days # How long should the Full Backups be kept? (0000006)
  Maximum Volume Bytes = 50G # Limit Volume size to something reasonable
  Label Format = "AI-Consolidated-vm-aiqi-linux-test-backup-" # Volumes will be labeled "Full-<volume-id>"
  Storage = FileStorage-AI-Consolidated
}
===========================================
Pool {
  Name = AI-Incremental-vm-aiqi-linux-test-backup
  Pool Type = Backup
  Recycle = yes # Bareos can automatically recycle Volumes
  AutoPrune = yes # Prune expired volumes
  Volume Retention = 360 days # How long should the Incremental Backups be kept? (0000012)
  Maximum Volume Bytes = 50G # Limit Volume size to something reasonable
  Label Format = "AI-Incremental-vm-aiqi-linux-test-backup" # Volumes will be labeled "Incremental-<volume-id>"
  Volume Use Duration = 23h
  Storage = FileStorage-AI-Incremental
  Next Pool = AI-Consolidated-vm-aiqi-linux-test-backup
}

Both volume are available in their respective storage.

root@vm-aiqi-testbackup:~# ls -l /var/lib/bareos/storage-AI-Consolidated/AI-Consolidated-vm-aiqi-linux-test-backup-0006
-rw-r----- 1 bareos bareos 26349467738 Dec 25 08:09 /var/lib/bareos/storage-AI-Consolidated/AI-Consolidated-vm-aiqi-linux-test-backup-0006

root@vm-aiqi-testbackup:~# ls -l /var/lib/bareos/storage-AI-Incremental/AI-Incremental-vm-aiqi-linux-test-backup0012
-rw-r----- 1 bareos bareos 1329612 Dec 26 20:00 /var/lib/bareos/storage-AI-Incremental/AI-Incremental-vm-aiqi-linux-test-backup0012
Tagsalways incremental, storage

Activities

jcottin

jcottin

2021-12-27 15:53

reporter  

Bareos-always-incremental-restore-fail.txt (7,259 bytes)    
*restore

First you select one or more JobIds that contain files
to be restored. You will be presented several methods
of specifying the JobIds. Then you will be allowed to
select which files from those JobIds are to be restored.

To select the JobIds, you have the following choices:
 1: List last 20 Jobs run
 2: List Jobs where a given File is saved
 3: Enter list of comma separated JobIds to select
 4: Enter SQL list command
 5: Select the most recent backup for a client
 6: Select backup for a client before a specified time
 7: Enter a list of files to restore
 8: Enter a list of files to restore before a specified time
 9: Find the JobIds of the most recent backup for a client
10: Find the JobIds for a backup for a client before a specified time
11: Enter a list of directories to restore for found JobIds
12: Select full restore to a specified Job date
13: Cancel
Select item:  (1-13): 6
The restored files will the most current backup
BEFORE the date you specify below.

Enter date as YYYY-MM-DD HH:MM:SS :2021-12-27 22:00:00
Defined Clients:
1: bareos-fd
2: vm-aiqi-linux-test-backup-fd
3: vm-aiqiit-web-fd
4: vm-aiqiit-windows-test-backup-fd
Select the Client (1-4): 2
The defined FileSet resources are:
1: LinuxAll-vm-aiqi-linux-test-backup
2: MARIABACKUP-vm-aiqi-linux-test-backup
Select FileSet resource (1-2): 1
+-------+-------+----------+----------------+---------------------+------------------------------------------------+
| jobid | level | jobfiles | jobbytes       | starttime           | volumename                                     |
+-------+-------+----------+----------------+---------------------+------------------------------------------------+
|    24 | F     |  108,199 | 13,145,763,765 | 2021-12-25 08:06:41 | AI-Consolidated-vm-aiqi-linux-test-backup-0006 |
|    27 | I     |       95 |         68,530 | 2021-12-25 20:00:04 | AI-Incremental-vm-aiqi-linux-test-backup0008   |
|    32 | I     |       40 |      1,322,314 | 2021-12-26 20:00:09 | AI-Incremental-vm-aiqi-linux-test-backup0012   |
|    37 | I     |       53 |      1,429,611 | 2021-12-27 20:00:12 | AI-Incremental-vm-aiqi-linux-test-backup0015   |
|    41 | I     |      140 |        941,981 | 2021-12-27 21:35:21 | AI-Incremental-vm-aiqi-linux-test-backup0015   |
+-------+-------+----------+----------------+---------------------+------------------------------------------------+
You have selected the following JobIds: 24,27,32,37,41

Building directory tree for JobId(s) 24,27,32,37,41 ...  ++++++++++++++++++++++++++++++++++++++++++++
96,918 files inserted into the tree.

You are now entering file selection mode where you add (mark) and
remove (unmark) files to be restored. No files are initially added, unless
you used the "all" keyword on the command line.
Enter "done" to leave this mode.

cwd is: /
$ cd /var/lib/mysql.dumps/wordpressdb/
cwd is: /var/lib/mysql.dumps/wordpressdb/

$ dir
-rw-r--r--   1 0 (root) 112 (bareos)          1830  2021-12-27 21:35:21   /var/lib/mysql.dumps/wordpressdb/%create.gz
-rw-r--r--   1 0 (root) 112 (bareos)           149  2021-12-25 07:55:02   /var/lib/mysql.dumps/wordpressdb/%tables
-rw-r--r--   1 0 (root) 112 (bareos)           783  2021-12-25 07:55:02   /var/lib/mysql.dumps/wordpressdb/wp_commentmeta.sql.gz
-rw-r--r--   1 0 (root) 112 (bareos)          1161  2021-12-25 07:55:02   /var/lib/mysql.dumps/wordpressdb/wp_comments.sql.gz
-rw-r--r--   1 0 (root) 112 (bareos)           869  2021-12-25 07:55:02   /var/lib/mysql.dumps/wordpressdb/wp_links.sql.gz
-rw-r--r--   1 0 (root) 112 (bareos)        236139  2021-12-27 21:35:21   /var/lib/mysql.dumps/wordpressdb/wp_options.sql.gz
-rw-r--r--   1 0 (root) 112 (bareos)          1650  2021-12-27 21:35:21   /var/lib/mysql.dumps/wordpressdb/wp_postmeta.sql.gz
-rw-r--r--   1 0 (root) 112 (bareos)          3951  2021-12-27 21:35:21   /var/lib/mysql.dumps/wordpressdb/wp_posts.sql.gz
-rw-r--r--   1 0 (root) 112 (bareos)           770  2021-12-25 07:55:02   /var/lib/mysql.dumps/wordpressdb/wp_term_relationships.sql.gz
-rw-r--r--   1 0 (root) 112 (bareos)           838  2021-12-25 07:55:02   /var/lib/mysql.dumps/wordpressdb/wp_term_taxonomy.sql.gz
-rw-r--r--   1 0 (root) 112 (bareos)           780  2021-12-25 07:55:02   /var/lib/mysql.dumps/wordpressdb/wp_termmeta.sql.gz
-rw-r--r--   1 0 (root) 112 (bareos)           814  2021-12-25 07:55:02   /var/lib/mysql.dumps/wordpressdb/wp_terms.sql.gz
-rw-r--r--   1 0 (root) 112 (bareos)          1405  2021-12-27 21:35:21   /var/lib/mysql.dumps/wordpressdb/wp_usermeta.sql.gz
-rw-r--r--   1 0 (root) 112 (bareos)           983  2021-12-25 07:55:02   /var/lib/mysql.dumps/wordpressdb/wp_users.sql.gz
mark /var/lib/mysql.dumps/wordpressdb/%create.gz /var/lib/mysql.dumps/wordpressdb/%tables /var/lib/mysql.dumps/wordpressdb/wp_commentmeta.sql.gz /var/lib/mysql.dumps/wordpressdb/wp_comments.sql.gz /var/lib/mysql.dumps/wordpressdb/wp_links.sql.gz /var/lib/mysql.dumps/wordpressdb/wp_options.sql.gz /var/lib/mysql.dumps/wordpressdb/wp_postmeta.sql.gz /var/lib/mysql.dumps/wordpressdb/wp_posts.sql.gz /var/lib/mysql.dumps/wordpressdb/wp_term_relationships.sql.gz /var/lib/mysql.dumps/wordpressdb/wp_term_taxonomy.sql.gz /var/lib/mysql.dumps/wordpressdb/wp_termmeta.sql.gz /var/lib/mysql.dumps/wordpressdb/wp_terms.sql.gz /var/lib/mysql.dumps/wordpressdb/wp_usermeta.sql.gz /var/lib/mysql.dumps/wordpressdb/wp_users.sql.gz
14 files marked.

$ done
Bootstrap records written to /var/lib/bareos/bareos-dir.restore.4.bsr

The job will require the following
   Volume(s)                 Storage(s)                SD Device(s)
===========================================================================

    AI-Consolidated-vm-aiqi-linux-test-backup-0006 FileStorage-AI-Consolidated FileStorage-AI-Consolidated
    AI-Incremental-vm-aiqi-linux-test-backup0015 FileStorage-AI-Incremental FileStorage-AI-Incremental

Volumes marked with "*" are online.


14 files selected to be restored.

Run Restore job
JobName:         RestoreFiles
Bootstrap:       /var/lib/bareos/bareos-dir.restore.4.bsr
Where:           /tmp/bareos-restores
Replace:         Always
FileSet:         LinuxAll
Backup Client:   vm-aiqi-linux-test-backup-fd
Restore Client:  vm-aiqi-linux-test-backup-fd
Format:          Native
Storage:         FileStorage-AI-Consolidated
When:            2021-12-27 22:49:35
Catalog:         MyCatalog
Priority:        10
Plugin Options:  *None*
OK to run? (yes/mod/no): yes
Job queued. JobId=45
You have messages.
*

Messages:
12	2021-12-27 22:49:43	bareos-sd JobId 45: Please mount read Volume "AI-Incremental-vm-aiqi-linux-test-backup0015" for:
Job: RestoreFiles.2021-12-27_22.49.39_00
Storage: "FileStorage-AI-Consolidated" (/var/lib/bareos/storage-AI-Consolidated)
Pool: Incremental-BareOS
Media type: File

11	2021-12-27 22:49:43	bareos-sd JobId 45: Warning: stored/acquire.cc:286 Read open device "FileStorage-AI-Consolidated" (/var/lib/bareos/storage-AI-Consolidated) Volume "AI-Incremental-vm-aiqi-linux-test-backup0015" failed: ERR=stored/dev.cc:716 Could not open: /var/lib/bareos/storage-AI-Consolidated/AI-Incremental-vm-aiqi-linux-test-backup0015, ERR=No such file or directory
Bareos-always-incremental-restore-fail.txt (7,259 bytes)   
jcottin

jcottin

2021-12-27 15:53

reporter   ~0004421

Output with TXT might be easier to read.
jcottin

jcottin

2021-12-27 16:32

reporter   ~0004422

Device {
  Name = FileStorage-AI-Consolidated
  Media Type = File
  Archive Device = /var/lib/bareos/storage-AI-Consolidated
  LabelMedia = yes; # lets Bareos label unlabeled media
  Random Access = yes;
  AutomaticMount = yes; # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Description = "File device. A connecting Director must have the same Name and MediaType."
}

Device {
  Name = FileStorage-AI-Incremental
  Media Type = File
  Archive Device = /var/lib/bareos/storage-AI-Incremental
  LabelMedia = yes; # lets Bareos label unlabeled media
  Random Access = yes;
  AutomaticMount = yes; # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Description = "File device. A connecting Director must have the same Name and MediaType."
}

Storage {
  Name = FileStorage-AI-Consolidated
  Address = bareos-server # N.B. Use a fully qualified name here (do not use "localhost" here).
  Password = "22222222222222222222222222222222222222222222"
  Device = FileStorage-AI-Consolidated
  Media Type = File
}

Storage {
  Name = FileStorage-AI-Incremental
  Address = bareos-server # N.B. Use a fully qualified name here (do not use "localhost" here).
  Password = "22222222222222222222222222222222222222222222"
  Device = FileStorage-AI-Incremental
  Media Type = File
}
jcottin

jcottin

2021-12-27 16:43

reporter   ~0004423

The documentation said 2 storage.
But I created 2 device.

1 storage => 1 device.

I moved the data from one device (FILE: Directory) to another.
2 storage => 1 device.

Problem solved.
bruno-at-bareos

bruno-at-bareos

2021-12-28 09:37

manager   ~0004425

Thanks for sharing. Yes when the documentation talk about 2 storages it's in the director view, and not bareos-storage having 2 devices.
I close the issue.
bruno-at-bareos

bruno-at-bareos

2021-12-28 09:38

manager   ~0004426

AI need 2 storages on the director but One device able to read/write both Incremental and Full

Issue History

Date Modified Username Field Change
2021-12-27 15:29 jcottin New Issue
2021-12-27 15:29 jcottin Tag Attached: always incremental
2021-12-27 15:29 jcottin Tag Attached: bconsole
2021-12-27 15:29 jcottin Tag Attached: storage
2021-12-27 15:53 jcottin File Added: Bareos-always-incremental-restore-fail.txt
2021-12-27 15:53 jcottin Note Added: 0004421
2021-12-27 16:32 jcottin Note Added: 0004422
2021-12-27 16:43 jcottin Note Added: 0004423
2021-12-28 09:37 bruno-at-bareos Note Added: 0004425
2021-12-28 09:37 bruno-at-bareos Tag Detached: bconsole
2021-12-28 09:38 bruno-at-bareos Assigned To => bruno-at-bareos
2021-12-28 09:38 bruno-at-bareos Status new => resolved
2021-12-28 09:38 bruno-at-bareos Resolution open => no change required
2021-12-28 09:38 bruno-at-bareos Note Added: 0004426