Bareos Bug Tracker
Bareos Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0001007bareos-core[All Projects] apipublic2018-09-12 15:232018-10-17 13:11
ReporterIvanBayan 
Assigned Tojoergs 
PrioritynormalSeverityminorReproducibilityalways
StatusassignedResolutionopen 
PlatformLinuxOSUbuntuOS Version16.04
Product Version17.2.4 
Target VersionFixed in Version 
Summary0001007: JSON API: 'list files' returns malformed json
DescriptionFile listing of some jobs returns malformed json:

Steps To Reproduce$echo -e '.api 2\nlist files jobid=799'|sudo bconsole
Connecting to Director localhost:9101
1000 OK: mia-backup03-dir Version: 17.2.4 (21 Sep 2017)
Enter a period to cancel a command.
.api 2
{
  "jsonrpc": "2.0",
  "id": null,
  "result": {
    "api": 2
  }
}list files jobid=799
{
  "jsonrpc": "2.0",
  "id": null,
  "result": {
    "filenames": [
      {
        "filename": "/usr/sbin/pg_updatedicts"
      },
      {
        "filename": "/usr/sbin/"
      },
      {
        "filename": "/usr/sbin/tzconfig"
.....
      {
        "filename": "/var/lib/systemd/deb-systemd-helper-enabled/postgresql.service.dsh-also"
      },
      {},
      {},
      {},
      {},
      {},
.....
      {},
      {
        "nfo/right/america/antigua": "/usr/share/zoneinfo/right/America/Grand_Turk"
      },
      {
        "nfo/right/america/antigua": "/usr/share/zoneinfo/right/America/Cordoba"
      },
      {
        "nfo/right/america/antigua": "/usr/share/zoneinfo/right/America/Thule"
.......
      {
        "rade-scripts/specification": "/usr/share/doc/postgresql-common/changelog.gz"
      },
      {
        "rade-scripts/specification": "/usr/share/doc/postgresql-common/NEWS.Debian.gz"
      },
      {
        "rade-scripts/specification": "/usr/share/doc/postgresql-common/README.Devel"
      },
      {
        "rade-scripts/specification": "/usr/share/doc/postgresql-common/architecture.html"
      },
      {
        "rade-scripts/specification": "/usr/share/doc/postgresql-common/README.systemd"
      },
      {
        "rade-scripts/specification": "/usr/share/doc/postgresql-common/copyright"
      },
      {
        "rade-scripts/specification": "/usr/share/doc/postgresql-common/dependencies.png"
.....
TagsNo tags attached.
bareos-master: impact
bareos-master: action
bareos-18.2: impact
bareos-18.2: action
bareos-17.2: impact
bareos-17.2: action
bareos-16.2: impact
bareos-16.2: action
bareos-15.2: impact
bareos-15.2: action
bareos-14.2: impact
bareos-14.2: action
bareos-13.2: impact
bareos-13.2: action
bareos-12.4: impact
bareos-12.4: action
Attached Files? file icon 799_list [^] (93,437 bytes) 2018-09-13 13:57 [Show Content]
bz2 file icon job54.tar.bz2 [^] (13,114 bytes) 2018-09-19 09:02

- Relationships

-  Notes
(0003105)
joergs (administrator)
2018-09-13 13:51

This looks strange. Can you provide the output of the file list in API mode 0?

$echo -e 'list files jobid=799'|sudo bconsole
(0003106)
IvanBayan (reporter)
2018-09-13 13:57

I've just attached bconsole output.
(0003118)
Gordon Klimm (reporter)
2018-09-19 09:01

I can confirm this behaviour.
Additionally: by repeatedly executing the "api2 - call", i get (three) different malformed results in varying order.
(0003135)
joergs (administrator)
2018-10-09 17:08

That is very strange. I've tested it on mutliple systems, but could not reproduce this bug.

As one list contains >= 2300 files, I also checked a larger backup job (> 6000) files, but still no error.

I've tested newer version of Bareos, but I've not seen relevant changes after 17.2.4.

Can you give my another hint, how to reproduce the problem?

BTW: for what purpose you need this list in json format?
(0003137)
IvanBayan (reporter)
2018-10-11 15:28

First I decided that error was caused by special characters in path or path length, but after quick check I haven't found anything suspicious. How can I help you? I can check other job listings from same job, if it helps.

I was developing python module to provide api for backup restoration, files list was needed to select which of them to restore. Now I use .bvfs calls for same thing.
Gordon Klimm attached full listing with same issue, did you check it? May be if we found something similar between two cases, we will find what cause that problem.
(0003141)
Gordon Klimm (reporter)
2018-10-17 11:37

Here is another observation: it appears as if the error occours only after "n*100 lines" of output (api 0-lines)
(0003142)
IvanBayan (reporter)
2018-10-17 13:11

Huh, you are right. But looks like it occurs not on n*100 line, but on n*100 file entry, json output below:
   908        {
   909          "filename": "/usr/share/zoneinfo/posix/America/Scoresbysund"
   910        },
   911        {
   912          "filename": "/etc/sysctl.d/"
   913        },
   914        {
   915          "filename": "/var/lib/systemd/deb-systemd-helper-enabled/postgresql.service.dsh-also"
   
916        },
   917        {
   918          "/porto-novo": "/var/lib/systemd/deb-systemd-helper-enabled/ureadahead.service.dsh-also"
   
919        },
   920        {


If you grep for 'filename' entries, you will found something like:
~$ echo -e '.api 2\nlist files jobid=799'|sudo bconsole|grep \"/|nl|head -n 203|tail -n 4
   
200	        "filename": "/usr/share/zoneinfo/posix/America/Halifax"
   201	        "": "/usr/share/zoneinfo/posix/America/Port-au-Prince"
   202	        "": "/usr/share/zoneinfo/posix/America/Panama"
   203	        "": "/usr/share/zoneinfo/posix/America/Cancun"
~$ echo -e '.api 2\nlist files jobid=799'|sudo bconsole|grep \"/|nl|head -n 203|tail -n 4
   200	        "filename": "/usr/share/zoneinfo/posix/America/Halifax"
   201	        "filename": "/usr/share/zoneinfo/posix/America/Port-au-Prince"
   202	        "filename": "/usr/share/zoneinfo/posix/America/Panama"
   203	        "filename": "/usr/share/zoneinfo/posix/America/Cancun"
~$ echo -e '.api 2\nlist files jobid=799'|sudo bconsole|grep \"/|nl|head -n 103|tail -n 4
   100	        "filename": "/usr/share/zoneinfo/Europe/Paris"
   101	        "filename": "/usr/share/zoneinfo/Europe/Zaporozhye"
   102	        "filename": "/usr/share/zoneinfo/Europe/Warsaw"
   103	        "filename": "/usr/share/zoneinfo/Europe/Belgrade"
~$ echo -e '.api 2\nlist files jobid=799'|sudo bconsole|grep \"/|nl|head -n 103|tail -n 4
   100	        "filename": "/usr/share/zoneinfo/Europe/Paris"
   101	        "nfo/europe/podgorica": "/usr/share/zoneinfo/Europe/Zaporozhye"
   102	        "nfo/europe/podgorica": "/usr/share/zoneinfo/Europe/Warsaw"
   103	        "nfo/europe/podgorica": "/usr/share/zoneinfo/Europe/Belgrade"

- Issue History
Date Modified Username Field Change
2018-09-12 15:23 IvanBayan New Issue
2018-09-13 13:48 joergs Assigned To => joergs
2018-09-13 13:48 joergs Status new => assigned
2018-09-13 13:51 joergs Note Added: 0003105
2018-09-13 13:51 joergs Status assigned => feedback
2018-09-13 13:57 IvanBayan File Added: 799_list
2018-09-13 13:57 IvanBayan Note Added: 0003106
2018-09-13 13:57 IvanBayan Status feedback => assigned
2018-09-19 09:01 Gordon Klimm Note Added: 0003118
2018-09-19 09:02 Gordon Klimm File Added: job54.tar.bz2
2018-10-09 17:08 joergs Note Added: 0003135
2018-10-09 17:08 joergs Status assigned => feedback
2018-10-11 15:28 IvanBayan Note Added: 0003137
2018-10-11 15:28 IvanBayan Status feedback => assigned
2018-10-17 11:37 Gordon Klimm Note Added: 0003141
2018-10-17 13:11 IvanBayan Note Added: 0003142


Copyright © 2000 - 2018 MantisBT Team
Powered by Mantis Bugtracker