Bareos Bug Tracker
Bareos Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0000603bareos-core[All Projects] webuipublic2016-01-19 22:592017-06-08 16:31
ReporterShodan 
Assigned To 
PrioritynormalSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformLinuxOSDebianOS Version8
Product Version15.2.2 
Target VersionFixed in Version16.2.4 
Summary0000603: no files in bareos-webui restore menu
DescriptionWhen open Restore menu and select any backup jobs, no files shown in "Select files to be restored" window.
Restore job via bconsole menu shows files correctly.
Steps To ReproduceOpen Restore menu, select any Backup jobs
Additional InformationNo errors in apache logs, no php errors on a web page.
Bareos-webui has access to bareos-dir via restricted console.
Restore job works fine via bconsole.

php5-zendframework2 2.3.5-2.1
bareos-webui-15.2.2-37.1

/etc/bareos/conf.d/webui.conf
# Restricted console used by bareos-webui
Console {
  Name = bareos-admin
  Password = "*****"
  Profile = webui
}

# bareos-webui default profile resource
Profile {
  Name = webui
  CommandACL = status, messages, show, version, run, rerun, cancel, .api, .bvfs_*, list, llist, use, restore, .jobs, .filesets, .clients
  Job ACL = *all*
  Schedule ACL = *all*
  Catalog ACL = *all*
  Pool ACL = *all*
  Storage ACL = *all*
  Client ACL = *all*
  FileSet ACL = *all*
  Where ACL = *all*
}
TagsNo tags attached.
bareos-master: impact
bareos-master: action
bareos-18.2: impact
bareos-18.2: action
bareos-17.2: impact
bareos-17.2: action
bareos-16.2: impact
bareos-16.2: action
bareos-15.2: impact
bareos-15.2: action
bareos-14.2: impact
bareos-14.2: action
bareos-13.2: impact
bareos-13.2: action
bareos-12.4: impact
bareos-12.4: action
Attached Filespng file icon restore.png [^] (89,086 bytes) 2016-01-19 22:59

- Relationships
related to 0000570closed "Files to be restored" panel empty on non-widows filesets 

-  Notes
(0002137)
frank (manager)
2016-01-21 12:46

Please have a look at http://bugs.bareos.org/view.php?id=570 [^] and see if that helps you out.
(0002143)
Shodan (reporter)
2016-01-22 10:26

Hi frank,

Unfortunately, running bareos-dbcheck -f -c /etc/bareos/bareos-dir.conf does not fix my problem. Webui still shows no files in "Select files to be restored"
(0002147)
frank (manager)
2016-01-22 12:01
edited on: 2016-01-22 12:01

Did you also tried the following?

1. Find a jobid of a *nix/win client last full backup where webui does not show any files in restore panel/tree to choose from.

2. In bconsole execute the following command, to update the cache.

.bvfs_update jobid=<jobid>

3. In bconsole execute the following command with empty path argument.

.bvfs_lsdirs jobid=<jobid> path=


You should the see something like this:

*.bvfs_lsdirs jobid=654 path=
198 0 0 0 A A A A A A A A A A A A A A .
197 0 0 0 A A A A A A A A A A A A A A /
*

If you only get a one-liner like following, your catalog is maybe corrupt.

*.bvfs_lsdirs jobid=169964 path=
939173 0 0 0 A A A A A A A A A A A A A A .
*

(0002148)
Shodan (reporter)
2016-01-22 13:15
edited on: 2016-01-22 13:18

Sure, yes

*.bvfs_update jobid=28695
Using Catalog "MyCatalog"

*.bvfs_lsdirs jobid=28695 path=
Using Catalog "MyCatalog"
0 0 0 0 A A A A A A A A A A A A A A .

> bareos-dbcheck -f -c /etc/bareos/bareos-dir.conf

Hello, this is the database check/correct program.
Modify database is on. Verbose is off.
Please select the function you want to perform.
--//---
Select function number: 16
Checking for Filenames with a trailing slash
Found 0 bad Filename records.
Checking for Paths without a trailing slash
Found 0 bad Path records.
Checking for duplicate Filename entries.
Found 0 duplicate Filename records.
Checking for duplicate Path entries.
Found 1 duplicate Path records.
Checking for orphaned JobMedia entries.
Checking for orphaned File entries. This may take some time!
Pruning orphaned Path entries isn't possible when using BVFS.
Note. Index over the FilenameId column not found, that can greatly slow down dbcheck.
Create temporary index? (yes/no): yes
Create temporary index... This may take some time!
Checking for orphaned Filename entries. This may take some time!
Drop temporary index.
Checking for orphaned FileSet entries. This takes some time!
Found 0 orphaned FileSet records.
Checking for orphaned Client entries.
Found 12 orphaned Client records.
Deleting 12 orphaned Client records.
Checking for orphaned Job entries.
Found 0 orphaned Job records.
Checking for Admin Job entries.
Found 0 Admin Job records.
Checking for Restore Job entries.
Found 0 Restore Job records.

(0002150)
frank (manager)
2016-01-22 15:06

Please perform a ".bvfs_clear_cache yes" in bconsole if not already done and see if the problem still exists.
(0002155)
Shodan (reporter)
2016-01-22 17:09
edited on: 2016-01-22 17:18

Sorry, forgot about .bvfs_clear_cache yes
Seems it works now (trembling hands)

(0002156)
frank (manager)
2016-01-22 17:28

Great
(0002158)
Shodan (reporter)
2016-01-25 09:48

Sorry to ping you again , the problem is client has a 3 defined jobs, but webui shows fileset for the first job only, with "Merge all client filesets = yes"
If "Merge all client filesets = no", webui shows nothing.
(0002161)
frank (manager)
2016-01-26 14:34

Ok, I created the same scenario on a test machine and wasn't able to reproduce, merging filesets works as expected.

Just to make things clear. Only jobs of a client which had run before the actual one selected are merged.

You can try to figure out what's going on under the hood by using the bvfs api.

...

Get dependent jobs from a given JobId

Bvfs allows you to query the catalog against any combination of jobs. You can combine all Jobs and all FileSet for a Client in a single session.

To get all JobId needed to restore a particular job, you can use the .bvfs_get_jobids command.

.bvfs_get_jobids jobid=num [all]

Example:

*.bvfs_get_jobids jobid=10
1,2,5,10
*.bvfs_get_jobids jobid=10 all
1,2,3,5,10

In this example, a normal restore will need to use JobIds 1,2,5,10 to compute a complete restore of the system.

With the all option, the Director will use all defined FileSet for this client.

...

How to get files and directories of your jobid list is stated above in this comment section.

Further information on bvfs api can be found here: http://doc.bareos.org/master/html/bareos-developer-guide.html#sec:bvfs [^]
(0002162)
Shodan (reporter)
2016-01-26 15:35

Hi Frank,

The problem is fileset has a dynamically generated file names via bat script.

FileSet {
  Name = "LD4-DBM-TR2-01-DB"
  Include {
    Options {
      signature = MD5
    }
    File = "\\|backup.bat"
  }
}

And bat script itself
@echo off
powershell -command %programdata%\bareos\tr2_backup2.ps1 -folder E:\BACKUPS\DATA

where tr2_backup2.ps1 generates files names, that are created in the last 2 hours.

param(
[string]$folder = "e:\backups\data"
)

Get-ChildItem -path $folder | where { $_.CreationTime -gt ((get-date).AddHours(-2)) } | % { $_.FullName }
(0002177)
olga (reporter)
2016-02-02 15:01

Have the same problem with restore via webui but it is still broken:
root@bcp2m:~# bconsole
Connecting to Director bcp2m:9101
1000 OK: bcp2m-dir Version: 15.2.2 (16 November 2015)
Enter a period to cancel a command.
*.bvfs_update jobid=5418
Using Catalog "MyCatalog"
You have messages.
*.bvfs_lsdirs jobid=5418 path=
97 0 0 0 A A A A A A A A A A A A A A .
74 0 0 0 A A A A A A A A A A A A A A /
*.bvfs_clear_cache
Can't find 'yes' argument
*.bvfs_clear_cache yes
OK

root@bcp2m:~# bareos-dbcheck -f -c /etc/bareos/bareos-dir.conf
Hello, this is the database check/correct program.
Modify database is on. Verbose is off.
Please select the function you want to perform.

     1) Toggle modify database flag
     2) Toggle verbose flag
     3) Repair bad Filename records
     4) Repair bad Path records
     5) Eliminate duplicate Filename records
     6) Eliminate duplicate Path records
     7) Eliminate orphaned Jobmedia records
     8) Eliminate orphaned File records
     9) Eliminate orphaned Path records
    10) Eliminate orphaned Filename records
    11) Eliminate orphaned FileSet records
    12) Eliminate orphaned Client records
    13) Eliminate orphaned Job records
    14) Eliminate all Admin records
    15) Eliminate all Restore records
    16) All (3-15)
    17) Quit
Select function number: 16
Checking for Filenames with a trailing slash
Found 0 bad Filename records.
Checking for Paths without a trailing slash
Found 1 bad Path records.
Reparing 1 bad Filename records.
Checking for duplicate Filename entries.
Found 0 duplicate Filename records.
Checking for duplicate Path entries.
Found 1 duplicate Path records.
Checking for orphaned JobMedia entries.
Checking for orphaned File entries. This may take some time!
Note. Index over the PathId column not found, that can greatly slow down dbcheck.
Create temporary index? (yes/no): yes
Create temporary index... This may take some time!
Checking for orphaned Path entries. This may take some time!
Found 7 orphaned Path records.
Deleting 7 orphaned Path records.
Drop temporary index.
Note. Index over the FilenameId column not found, that can greatly slow down dbcheck.
Create temporary index? (yes/no): yes
Create temporary index... This may take some time!
Checking for orphaned Filename entries. This may take some time!
Drop temporary index.
Checking for orphaned FileSet entries. This takes some time!
Found 0 orphaned FileSet records.
Checking for orphaned Client entries.
Found 1 orphaned Client records.
Deleting 1 orphaned Client records.
Checking for orphaned Job entries.
Found 0 orphaned Job records.
Checking for Admin Job entries.
Found 0 Admin Job records.
Checking for Restore Job entries.
Found 0 Restore Job records.
(0002180)
Shodan (reporter)
2016-02-02 15:44

Olga, BTW, if you have a fileset with dynamically generated files on Bareos client's side, like "File = "\\|script", the DB file record will be empty and restored files will not be shown in webui menu.
(0002181)
olga (reporter)
2016-02-02 19:13

I have separate fileset for mysql backup:
FileSet {
  Name = "mysql"
    Include {
      Options {
        compression=GZIP
        signature = MD5

              }
  Plugin = "python:module_path=/usr/lib/bareos/plugins:module_name=bareos-fd-mysql:defaultsfile=/etc/mysql/.login.conf:dumpoptions=--single-transaction --triggers --routines --set-gtid-purged=OFF"
        }
}
(0002183)
Shodan (reporter)
2016-02-03 20:06

select files by jobid in bareos mysql table, if 0 files found - the game is over.
the files, shown in webui, should be defined in fileset on director's side, bug or feature perhaps.
(0002184)
olga (reporter)
2016-02-04 09:04

I have over that 3mln rows in File table and the game is not over :) I can perfectly restore via bconsole but no luck in webui.
(0002185)
Shodan (reporter)
2016-02-04 09:13

Try to catch sql statement from webui app and looking into it
(0002186)
olga (reporter)
2016-02-04 21:39

380 Connect bareos@XXX.XXX.XXX.XXX on bareos
          380 Query SELECT VersionId FROM Version
          380 Query SET wait_timeout=691200
          380 Query SET interactive_timeout=691200
          380 Query SELECT VolSessionId,VolSessionTime,PoolId,StartTime,EndTime,JobFiles,JobBytes,JobTDate,Job,JobStatus,Type,Level,ClientId,Name,PriorJobId,RealEndTime,JobId,FileSetId,SchedTime,RealEndTime,ReadBytes,HasBase,PurgedFiles FROM Job WHERE JobId=5568
          380 Query SELECT MAX(FileSet.FileSetId) FROM FileSet JOIN Job USING (FileSetId) WHERE Job.ClientId=1 GROUP BY FileSet
          380 Query CREATE TABLE btemp30 AS SELECT JobId, StartTime, EndTime, JobTDate, PurgedFiles FROM Job JOIN FileSet USING (FileSetId) WHERE ClientId = 1 AND Level='F' AND JobStatus IN ('T','W') AND Type='B' AND StartTime<'2016-02-03 21:00:02' AND FileSet.FileSet=(SELECT FileSet FROM FileSet WHERE FileSetId = 1) ORDER BY Job.JobTDate DESC LIMIT 1
          380 Query DROP TABLE btemp30
          380 Quit

temporary table (btemp30) is created and dropped but no events are found between.
(0002293)
frank (manager)
2016-06-10 17:36

Is this still an issue?
(0002295)
master_volkov (reporter)
2016-06-12 21:01
edited on: 2016-06-12 21:02

Olga, in case of automatic generation file list on windows client trougth File = "\\|backup.bat" you can try replace all "\" to "/".

In my case problem be in that and with "/" i can restore trougth webui without problem.

After change in fileset, do not forget do ".bvfs_clear_cache yes" in bconsole and run "/usr/sbin/bareos-dbcheck -f"



in case of success, that problem is described in https://bugs.bareos.org/view.php?id=666 [^]

(0002305)
Shodan (reporter)
2016-07-07 15:18
edited on: 2016-07-07 15:34

Sure, same issue for me with python plugin, reported by Olga.

If Fileset contains python plugin, like Plugin = "python:module_path=/usr/lib/bareos/plugins:module_name=bareos-fd-mysql", bareos-webui does not show restored files in tree, only empty root directory.
bconsole shows restored files correctly.


- Issue History
Date Modified Username Field Change
2016-01-19 22:59 Shodan New Issue
2016-01-19 22:59 Shodan File Added: restore.png
2016-01-21 12:46 frank Note Added: 0002137
2016-01-21 12:46 frank Assigned To => frank
2016-01-21 12:46 frank Status new => feedback
2016-01-22 10:26 Shodan Note Added: 0002143
2016-01-22 10:26 Shodan Status feedback => assigned
2016-01-22 12:01 frank Note Added: 0002147
2016-01-22 12:01 frank Note Edited: 0002147 View Revisions
2016-01-22 12:02 frank Status assigned => feedback
2016-01-22 13:15 Shodan Note Added: 0002148
2016-01-22 13:15 Shodan Status feedback => assigned
2016-01-22 13:18 Shodan Note Edited: 0002148 View Revisions
2016-01-22 15:06 frank Note Added: 0002150
2016-01-22 15:06 frank Status assigned => feedback
2016-01-22 15:11 frank Relationship added related to 0000570
2016-01-22 17:09 Shodan Note Added: 0002155
2016-01-22 17:09 Shodan Status feedback => assigned
2016-01-22 17:18 Shodan Note Edited: 0002155 View Revisions
2016-01-22 17:28 frank Note Added: 0002156
2016-01-22 17:28 frank Status assigned => feedback
2016-01-25 09:48 Shodan Note Added: 0002158
2016-01-25 09:48 Shodan Status feedback => assigned
2016-01-26 14:34 frank Note Added: 0002161
2016-01-26 14:36 frank Status assigned => feedback
2016-01-26 15:35 Shodan Note Added: 0002162
2016-01-26 15:35 Shodan Status feedback => assigned
2016-02-02 15:01 olga Note Added: 0002177
2016-02-02 15:44 Shodan Note Added: 0002180
2016-02-02 19:13 olga Note Added: 0002181
2016-02-03 20:06 Shodan Note Added: 0002183
2016-02-04 09:04 olga Note Added: 0002184
2016-02-04 09:13 Shodan Note Added: 0002185
2016-02-04 21:39 olga Note Added: 0002186
2016-06-10 17:36 frank Note Added: 0002293
2016-06-10 17:36 frank Status assigned => feedback
2016-06-12 21:01 master_volkov Note Added: 0002295
2016-06-12 21:02 master_volkov Note Edited: 0002295 View Revisions
2016-07-07 15:18 Shodan Note Added: 0002305
2016-07-07 15:18 Shodan Status feedback => assigned
2016-07-07 15:21 Shodan Note Edited: 0002305 View Revisions
2016-07-07 15:21 Shodan Note Edited: 0002305 View Revisions
2016-07-07 15:34 Shodan Note Edited: 0002305 View Revisions
2016-11-03 17:51 frank Status assigned => resolved
2016-11-03 17:51 frank Fixed in Version => 16.2.4
2016-11-03 17:51 frank Resolution open => fixed
2017-06-08 16:31 frank Status resolved => closed
2017-06-08 16:31 frank Assigned To frank =>


Copyright © 2000 - 2018 MantisBT Team
Powered by Mantis Bugtracker