View Issue Details

IDProjectCategoryView StatusLast Update
0000950bareos-corefile daemonpublic2023-03-23 16:41
Reporterbeckzg Assigned Tobruno-at-bareos  
PriorityhighSeveritymajorReproducibilityalways
Status closedResolutionwon't fix 
PlatformLinuxOSDebianOS Version9
Product Version17.2.4 
Summary0000950: MariaDB backup with percona plugin
DescriptionHi,
  I've a strange problem and get find out yet whats wrong, I use the percona plugin to backup up MariaDB 10.2.4 database.

The fileset configuration looks like this:

FileSet {
  Name = "MariaDB"
  Description = "Backup MariaDB"
  Include {
    Options {
      Signature = MD5 # calculate md5 checksum per file
    }
    Plugin = "python:module_path=/usr/lib/bareos/plugins:module_name=bareos-fd-percona:dumpbinary=mariabackup:dumpoptions=--backup --compress --stream=xbstream --parallel=4 --compress-threads=4:mycnf=/etc/mysql/backup.cnf"
  }
}

----------------------------------------------------------------------------------------------------
The first run (Full backup) went well:

bareos-dir JobId 25: Bareos bareos-dir 17.2.4 (21Sep17):
Build OS: x86_64-pc-linux-gnu debian Debian GNU/Linux 9.3 (stretch)
JobId: 25
Job: sql01.local-MariaDB.2018-05-18_12.41.14_32
Backup Level: Full (upgraded from Incremental)
Client: "sql01.local" 17.2.4 (21Sep17) x86_64-pc-linux-gnu,debian,Debian GNU/Linux 9.3 (stretch),Debian_9.0,x86_64
FileSet: "MariaDB" 2018-05-18 11:02:32
Pool: "Full" (From Job FullPool override)
Catalog: "MyCatalog" (From Client resource)
Storage: "File" (From Job resource)
Scheduled time: 18-May-2018 12:41:13
Start time: 18-May-2018 12:41:16
End time: 18-May-2018 12:54:47
Elapsed time: 13 mins 31 secs
Priority: 10
FD Files Written: 2
SD Files Written: 2
FD Bytes Written: 50,554,175,708 (50.55 GB)
SD Bytes Written: 50,554,176,649 (50.55 GB)
Rate: 62335.6 KB/s
Software Compression: None
VSS: no
Encryption: no
Accurate: no
Volume name(s): Full-0001|Full-0003|Full-0004|Full-0005|Full-0006|Full-0007|Full-0008|Full-0009|Full-0010|Full-0011|Full-0012|Full-0013|Full-0014|Full-0015|Full-0016|Full-0017|Full-0018|Full-0019|Full-0020|Full-0021|Full-0022|Full-0023|Full-0024|Full-0025|Full-0026|Full-0027|Full-0028|Full-0029|Full-0030|Full-0031|Full-0032|Full-0033|Full-0034|Full-0035|Full-0036|Full-0037|Full-0038|Full-0039|Full-0040|Full-0041|Full-0042|Full-0043|Full-0044|Full-0045|Full-0046|Full-0047|Full-0048|Full-0049
Volume Session Id: 1
Volume Session Time: 1526639588
Last Volume Bytes: 125,991,399 (125.9 MB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK

----------------------------------------------------------------------------------------------------
But when I try to run again (Incremental) I get the following error:

8-May 13:11 bareos-dir JobId 30: Start Backup JobId 30, Job=sql01.local-MariaDB.2018-05-18_13.11.14_59
18-May 13:11 bareos-dir JobId 30: Using Device "FileStorage" to write.
18-May 13:11 sql01.local JobId 30: Fatal error: python-fd: Traceback (most recent call last):
  File "/usr/lib/bareos/plugins/BareosFdWrapper.py", line 58, in restore_object_data
    return bareos_fd_plugin_object.restore_object_data(context, ROP)
  File "/usr/lib/bareos/plugins/BareosFdPercona.py", line 423, in restore_object_data
    self.rop_data[ROP.jobid] = json.loads(str(self.row_rop_raw))
  File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 380, in raw_decode
    obj, end = self.scan_once(s, idx)
  File "/usr/lib/python2.7/json/decoder.py", line 36, in errmsg
    lineno, colno = linecol(doc, pos)
TypeError: 'NoneType' object is not callable

18-May 13:11 sql01.local JobId 30: Fatal error: Failed to authenticate Storage daemon.
18-May 13:11 bareos-dir JobId 30: Fatal error: Bad response to Storage command: wanted 2000 OK storage
, got 2902 Bad storage

18-May 13:11 bareos-dir JobId 30: Error: Bareos bareos-dir 17.2.4 (21Sep17):
  Build OS: x86_64-pc-linux-gnu debian Debian GNU/Linux 9.3 (stretch)
  JobId: 30
  Job: sql01.local-MariaDB.2018-05-18_13.11.14_59
  Backup Level: Incremental, since=2018-05-18 12:41:16
  Client: "sql01.local" 17.2.4 (21Sep17) x86_64-pc-linux-gnu,debian,Debian GNU/Linux 9.3 (stretch),Debian_9.0,x86_64
  FileSet: "MariaDB" 2018-05-18 11:02:32
  Pool: "Incremental" (From Job IncPool override)
  Catalog: "MyCatalog" (From Client resource)
  Storage: "File" (From Job resource)
  Scheduled time: 18-May-2018 13:11:13
  Start time: 18-May-2018 13:11:16
  End time: 18-May-2018 13:11:16
  Elapsed time: 0 secs
  Priority: 10
  FD Files Written: 0
  SD Files Written: 0
  FD Bytes Written: 0 (0 B)
  SD Bytes Written: 0 (0 B)
  Rate: 0.0 KB/s
  Software Compression: None
  VSS: no
  Encryption: no
  Accurate: no
  Volume name(s):
  Volume Session Id: 5
  Volume Session Time: 1526640985
  Last Volume Bytes: 0 (0 B)
  Non-fatal FD errors: 2
  SD Errors: 0
  FD termination status: Fatal Error
  SD termination status: Waiting on FD
  Termination: *** Backup Error ***

  Any ideas what should be my problem?

  Kind Regards,
Zoltan
Steps To ReproduceFirst run a Full backup and after an Incremental backup.
TagsNo tags attached.

Activities

beckzg

beckzg

2018-05-22 13:44

reporter  

sql01.local.trace (15,581 bytes)
beckzg

beckzg

2018-05-22 16:26

reporter   ~0003011

The same error occurs with percona-xtrabackup and percona-xtrabackup-24 packages:

root@sql01:/etc/mysql# xtrabackup --version
xtrabackup version 2.3.10 based on MySQL server 5.6.24 Linux (x86_64) (revision id: bd0d4403f36)

root@sql01:/etc/mysql# xtrabackup --version
xtrabackup version 2.4.11 based on MySQL server 5.7.19 Linux (x86_64) (revision id: b4e0db5)


22-May 16:17 bareos-dir JobId 57: Start Backup JobId 57, Job=sql01.local-MariaDB.2018-05-22_16.17.07_19
22-May 16:17 bareos-dir JobId 57: Using Device "FileStorage" to write.
22-May 16:17 sql01.local JobId 57: Fatal error: python-fd: Traceback (most recent call last):
  File "/usr/lib/bareos/plugins/BareosFdWrapper.py", line 58, in restore_object_data
    return bareos_fd_plugin_object.restore_object_data(context, ROP)
  File "/usr/lib/bareos/plugins/BareosFdPercona.py", line 423, in restore_object_data
    self.rop_data[ROP.jobid] = json.loads(str(self.row_rop_raw))
  File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 380, in raw_decode
    obj, end = self.scan_once(s, idx)
  File "/usr/lib/python2.7/json/decoder.py", line 36, in errmsg
    lineno, colno = linecol(doc, pos)
TypeError: 'NoneType' object is not callable

22-May 16:17 sql01.local JobId 57: Fatal error: Failed to authenticate Storage daemon.
22-May 16:17 bareos-dir JobId 57: Fatal error: Bad response to Storage command: wanted 2000 OK storage
, got 2902 Bad storage
beckzg

beckzg

2018-05-22 16:29

reporter   ~0003012

root@sql01:/etc/mysql# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 9.4 (stretch)
Release: 9.4
Codename: stretch


root@sql01:/etc/mysql# mysql --version
mysql Ver 15.1 Distrib 10.2.15-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2
beckzg

beckzg

2018-05-22 16:32

reporter   ~0003013

root@sql01:/etc/mysql# mariabackup --version
mariabackup based on MariaDB server 10.2.15-MariaDB debian-linux-gnu (x86_64)
beckzg

beckzg

2018-06-26 10:07

reporter   ~0003052

I installed a completely new system with CentOS (CentOS Linux release 7.5.1804 (Core)) and Percona Server (Percona-Server-server-57-5.7.22-22.1.el7.x86_64).

The symptom is absolutely the same, the full backup runs but the incremental backup fails:

26-Jun 10:02 bareos-dir JobId 286: Using Device "FileStorage" to write.
26-Jun 10:02 sql03.local JobId 286: Fatal error: python-fd: Traceback (most recent call last):
  File "/usr/lib64/bareos/plugins/BareosFdWrapper.py", line 58, in restore_object_data
    return bareos_fd_plugin_object.restore_object_data(context, ROP)
  File "/usr/lib64/bareos/plugins/BareosFdPercona.py", line 423, in restore_object_data
    self.rop_data[ROP.jobid] = json.loads(str(self.row_rop_raw))
  File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
  File "/usr/lib64/python2.7/json/decoder.py", line 38, in errmsg
    lineno, colno = linecol(doc, pos)
TypeError: 'NoneType' object is not callable

26-Jun 10:02 sql03.local JobId 286: Fatal error: Failed to authenticate Storage daemon.
26-Jun 10:02 bareos-dir JobId 286: Fatal error: Bad response to Storage command: wanted 2000 OK storage
, got 2902 Bad storage
beckzg

beckzg

2018-07-05 23:32

reporter   ~0003063

I found the problem, this error only occurs if sqlite3 is used for catalog, now I changed to mysql and the jobs are running.

So it looks like it is a bug somewhere in the code related to sqlite3.
bruno-at-bareos

bruno-at-bareos

2023-03-23 16:41

manager   ~0004948

sqlite3 database catalog is no more possible to used.

Issue History

Date Modified Username Field Change
2018-05-22 10:42 beckzg New Issue
2018-05-22 13:44 beckzg File Added: sql01.local.trace
2018-05-22 16:26 beckzg Note Added: 0003011
2018-05-22 16:29 beckzg Note Added: 0003012
2018-05-22 16:32 beckzg Note Added: 0003013
2018-06-26 10:07 beckzg Note Added: 0003052
2018-07-05 23:32 beckzg Note Added: 0003063
2023-03-23 16:41 bruno-at-bareos Assigned To => bruno-at-bareos
2023-03-23 16:41 bruno-at-bareos Status new => closed
2023-03-23 16:41 bruno-at-bareos Resolution open => won't fix
2023-03-23 16:41 bruno-at-bareos Note Added: 0004948