View Issue Details

IDProjectCategoryView StatusLast Update
0000603bareos-corewebuipublic2017-06-08 16:31
ReporterShodan Assigned To 
PrioritynormalSeveritymajorReproducibilityalways
Status closedResolutionfixed 
PlatformLinuxOSDebianOS Version8
Product Version15.2.2 
Fixed in Version16.2.4 
Summary0000603: no files in bareos-webui restore menu
DescriptionWhen open Restore menu and select any backup jobs, no files shown in "Select files to be restored" window.
Restore job via bconsole menu shows files correctly.
Steps To ReproduceOpen Restore menu, select any Backup jobs
Additional InformationNo errors in apache logs, no php errors on a web page.
Bareos-webui has access to bareos-dir via restricted console.
Restore job works fine via bconsole.

php5-zendframework2 2.3.5-2.1
bareos-webui-15.2.2-37.1

/etc/bareos/conf.d/webui.conf
# Restricted console used by bareos-webui
Console {
  Name = bareos-admin
  Password = "*****"
  Profile = webui
}

# bareos-webui default profile resource
Profile {
  Name = webui
  CommandACL = status, messages, show, version, run, rerun, cancel, .api, .bvfs_*, list, llist, use, restore, .jobs, .filesets, .clients
  Job ACL = *all*
  Schedule ACL = *all*
  Catalog ACL = *all*
  Pool ACL = *all*
  Storage ACL = *all*
  Client ACL = *all*
  FileSet ACL = *all*
  Where ACL = *all*
}
TagsNo tags attached.

Relationships

related to 0000570 closed "Files to be restored" panel empty on non-widows filesets 

Activities

Shodan

Shodan

2016-01-19 22:59

reporter  

restore.png (89,086 bytes)   
restore.png (89,086 bytes)   
frank

frank

2016-01-21 12:46

developer   ~0002137

Please have a look at http://bugs.bareos.org/view.php?id=570 and see if that helps you out.
Shodan

Shodan

2016-01-22 10:26

reporter   ~0002143

Hi frank,

Unfortunately, running bareos-dbcheck -f -c /etc/bareos/bareos-dir.conf does not fix my problem. Webui still shows no files in "Select files to be restored"
frank

frank

2016-01-22 12:01

developer   ~0002147

Last edited: 2016-01-22 12:01

Did you also tried the following?

1. Find a jobid of a *nix/win client last full backup where webui does not show any files in restore panel/tree to choose from.

2. In bconsole execute the following command, to update the cache.

.bvfs_update jobid=<jobid>

3. In bconsole execute the following command with empty path argument.

.bvfs_lsdirs jobid=<jobid> path=


You should the see something like this:

*.bvfs_lsdirs jobid=654 path=
198 0 0 0 A A A A A A A A A A A A A A .
197 0 0 0 A A A A A A A A A A A A A A /
*

If you only get a one-liner like following, your catalog is maybe corrupt.

*.bvfs_lsdirs jobid=169964 path=
939173 0 0 0 A A A A A A A A A A A A A A .
*

Shodan

Shodan

2016-01-22 13:15

reporter   ~0002148

Last edited: 2016-01-22 13:18

Sure, yes

*.bvfs_update jobid=28695
Using Catalog "MyCatalog"

*.bvfs_lsdirs jobid=28695 path=
Using Catalog "MyCatalog"
0 0 0 0 A A A A A A A A A A A A A A .

> bareos-dbcheck -f -c /etc/bareos/bareos-dir.conf

Hello, this is the database check/correct program.
Modify database is on. Verbose is off.
Please select the function you want to perform.
--//---
Select function number: 16
Checking for Filenames with a trailing slash
Found 0 bad Filename records.
Checking for Paths without a trailing slash
Found 0 bad Path records.
Checking for duplicate Filename entries.
Found 0 duplicate Filename records.
Checking for duplicate Path entries.
Found 1 duplicate Path records.
Checking for orphaned JobMedia entries.
Checking for orphaned File entries. This may take some time!
Pruning orphaned Path entries isn't possible when using BVFS.
Note. Index over the FilenameId column not found, that can greatly slow down dbcheck.
Create temporary index? (yes/no): yes
Create temporary index... This may take some time!
Checking for orphaned Filename entries. This may take some time!
Drop temporary index.
Checking for orphaned FileSet entries. This takes some time!
Found 0 orphaned FileSet records.
Checking for orphaned Client entries.
Found 12 orphaned Client records.
Deleting 12 orphaned Client records.
Checking for orphaned Job entries.
Found 0 orphaned Job records.
Checking for Admin Job entries.
Found 0 Admin Job records.
Checking for Restore Job entries.
Found 0 Restore Job records.

frank

frank

2016-01-22 15:06

developer   ~0002150

Please perform a ".bvfs_clear_cache yes" in bconsole if not already done and see if the problem still exists.
Shodan

Shodan

2016-01-22 17:09

reporter   ~0002155

Last edited: 2016-01-22 17:18

Sorry, forgot about .bvfs_clear_cache yes
Seems it works now (trembling hands)

frank

frank

2016-01-22 17:28

developer   ~0002156

Great
Shodan

Shodan

2016-01-25 09:48

reporter   ~0002158

Sorry to ping you again , the problem is client has a 3 defined jobs, but webui shows fileset for the first job only, with "Merge all client filesets = yes"
If "Merge all client filesets = no", webui shows nothing.
frank

frank

2016-01-26 14:34

developer   ~0002161

Ok, I created the same scenario on a test machine and wasn't able to reproduce, merging filesets works as expected.

Just to make things clear. Only jobs of a client which had run before the actual one selected are merged.

You can try to figure out what's going on under the hood by using the bvfs api.

...

Get dependent jobs from a given JobId

Bvfs allows you to query the catalog against any combination of jobs. You can combine all Jobs and all FileSet for a Client in a single session.

To get all JobId needed to restore a particular job, you can use the .bvfs_get_jobids command.

.bvfs_get_jobids jobid=num [all]

Example:

*.bvfs_get_jobids jobid=10
1,2,5,10
*.bvfs_get_jobids jobid=10 all
1,2,3,5,10

In this example, a normal restore will need to use JobIds 1,2,5,10 to compute a complete restore of the system.

With the all option, the Director will use all defined FileSet for this client.

...

How to get files and directories of your jobid list is stated above in this comment section.

Further information on bvfs api can be found here: http://doc.bareos.org/master/html/bareos-developer-guide.html#sec:bvfs
Shodan

Shodan

2016-01-26 15:35

reporter   ~0002162

Hi Frank,

The problem is fileset has a dynamically generated file names via bat script.

FileSet {
  Name = "LD4-DBM-TR2-01-DB"
  Include {
    Options {
      signature = MD5
    }
    File = "\\|backup.bat"
  }
}

And bat script itself
@echo off
powershell -command %programdata%\bareos\tr2_backup2.ps1 -folder E:\BACKUPS\DATA

where tr2_backup2.ps1 generates files names, that are created in the last 2 hours.

param(
[string]$folder = "e:\backups\data"
)

Get-ChildItem -path $folder | where { $_.CreationTime -gt ((get-date).AddHours(-2)) } | % { $_.FullName }
olga

olga

2016-02-02 15:01

reporter   ~0002177

Have the same problem with restore via webui but it is still broken:
root@bcp2m:~# bconsole
Connecting to Director bcp2m:9101
1000 OK: bcp2m-dir Version: 15.2.2 (16 November 2015)
Enter a period to cancel a command.
*.bvfs_update jobid=5418
Using Catalog "MyCatalog"
You have messages.
*.bvfs_lsdirs jobid=5418 path=
97 0 0 0 A A A A A A A A A A A A A A .
74 0 0 0 A A A A A A A A A A A A A A /
*.bvfs_clear_cache
Can't find 'yes' argument
*.bvfs_clear_cache yes
OK

root@bcp2m:~# bareos-dbcheck -f -c /etc/bareos/bareos-dir.conf
Hello, this is the database check/correct program.
Modify database is on. Verbose is off.
Please select the function you want to perform.

     1) Toggle modify database flag
     2) Toggle verbose flag
     3) Repair bad Filename records
     4) Repair bad Path records
     5) Eliminate duplicate Filename records
     6) Eliminate duplicate Path records
     7) Eliminate orphaned Jobmedia records
     8) Eliminate orphaned File records
     9) Eliminate orphaned Path records
    10) Eliminate orphaned Filename records
    11) Eliminate orphaned FileSet records
    12) Eliminate orphaned Client records
    13) Eliminate orphaned Job records
    14) Eliminate all Admin records
    15) Eliminate all Restore records
    16) All (3-15)
    17) Quit
Select function number: 16
Checking for Filenames with a trailing slash
Found 0 bad Filename records.
Checking for Paths without a trailing slash
Found 1 bad Path records.
Reparing 1 bad Filename records.
Checking for duplicate Filename entries.
Found 0 duplicate Filename records.
Checking for duplicate Path entries.
Found 1 duplicate Path records.
Checking for orphaned JobMedia entries.
Checking for orphaned File entries. This may take some time!
Note. Index over the PathId column not found, that can greatly slow down dbcheck.
Create temporary index? (yes/no): yes
Create temporary index... This may take some time!
Checking for orphaned Path entries. This may take some time!
Found 7 orphaned Path records.
Deleting 7 orphaned Path records.
Drop temporary index.
Note. Index over the FilenameId column not found, that can greatly slow down dbcheck.
Create temporary index? (yes/no): yes
Create temporary index... This may take some time!
Checking for orphaned Filename entries. This may take some time!
Drop temporary index.
Checking for orphaned FileSet entries. This takes some time!
Found 0 orphaned FileSet records.
Checking for orphaned Client entries.
Found 1 orphaned Client records.
Deleting 1 orphaned Client records.
Checking for orphaned Job entries.
Found 0 orphaned Job records.
Checking for Admin Job entries.
Found 0 Admin Job records.
Checking for Restore Job entries.
Found 0 Restore Job records.
Shodan

Shodan

2016-02-02 15:44

reporter   ~0002180

Olga, BTW, if you have a fileset with dynamically generated files on Bareos client's side, like "File = "\\|script", the DB file record will be empty and restored files will not be shown in webui menu.
olga

olga

2016-02-02 19:13

reporter   ~0002181

I have separate fileset for mysql backup:
FileSet {
  Name = "mysql"
    Include {
      Options {
        compression=GZIP
        signature = MD5

              }
  Plugin = "python:module_path=/usr/lib/bareos/plugins:module_name=bareos-fd-mysql:defaultsfile=/etc/mysql/.login.conf:dumpoptions=--single-transaction --triggers --routines --set-gtid-purged=OFF"
        }
}
Shodan

Shodan

2016-02-03 20:06

reporter   ~0002183

select files by jobid in bareos mysql table, if 0 files found - the game is over.
the files, shown in webui, should be defined in fileset on director's side, bug or feature perhaps.
olga

olga

2016-02-04 09:04

reporter   ~0002184

I have over that 3mln rows in File table and the game is not over :) I can perfectly restore via bconsole but no luck in webui.
Shodan

Shodan

2016-02-04 09:13

reporter   ~0002185

Try to catch sql statement from webui app and looking into it
olga

olga

2016-02-04 21:39

reporter   ~0002186

380 Connect bareos@XXX.XXX.XXX.XXX on bareos
          380 Query SELECT VersionId FROM Version
          380 Query SET wait_timeout=691200
          380 Query SET interactive_timeout=691200
          380 Query SELECT VolSessionId,VolSessionTime,PoolId,StartTime,EndTime,JobFiles,JobBytes,JobTDate,Job,JobStatus,Type,Level,ClientId,Name,PriorJobId,RealEndTime,JobId,FileSetId,SchedTime,RealEndTime,ReadBytes,HasBase,PurgedFiles FROM Job WHERE JobId=5568
          380 Query SELECT MAX(FileSet.FileSetId) FROM FileSet JOIN Job USING (FileSetId) WHERE Job.ClientId=1 GROUP BY FileSet
          380 Query CREATE TABLE btemp30 AS SELECT JobId, StartTime, EndTime, JobTDate, PurgedFiles FROM Job JOIN FileSet USING (FileSetId) WHERE ClientId = 1 AND Level='F' AND JobStatus IN ('T','W') AND Type='B' AND StartTime<'2016-02-03 21:00:02' AND FileSet.FileSet=(SELECT FileSet FROM FileSet WHERE FileSetId = 1) ORDER BY Job.JobTDate DESC LIMIT 1
          380 Query DROP TABLE btemp30
          380 Quit

temporary table (btemp30) is created and dropped but no events are found between.
frank

frank

2016-06-10 17:36

developer   ~0002293

Is this still an issue?
master_volkov

master_volkov

2016-06-12 21:01

reporter   ~0002295

Last edited: 2016-06-12 21:02

Olga, in case of automatic generation file list on windows client trougth File = "\\|backup.bat" you can try replace all "\" to "/".

In my case problem be in that and with "/" i can restore trougth webui without problem.

After change in fileset, do not forget do ".bvfs_clear_cache yes" in bconsole and run "/usr/sbin/bareos-dbcheck -f"



in case of success, that problem is described in https://bugs.bareos.org/view.php?id=666

Shodan

Shodan

2016-07-07 15:18

reporter   ~0002305

Last edited: 2016-07-07 15:34

Sure, same issue for me with python plugin, reported by Olga.

If Fileset contains python plugin, like Plugin = "python:module_path=/usr/lib/bareos/plugins:module_name=bareos-fd-mysql", bareos-webui does not show restored files in tree, only empty root directory.
bconsole shows restored files correctly.

Issue History

Date Modified Username Field Change
2016-01-19 22:59 Shodan New Issue
2016-01-19 22:59 Shodan File Added: restore.png
2016-01-21 12:46 frank Note Added: 0002137
2016-01-21 12:46 frank Assigned To => frank
2016-01-21 12:46 frank Status new => feedback
2016-01-22 10:26 Shodan Note Added: 0002143
2016-01-22 10:26 Shodan Status feedback => assigned
2016-01-22 12:01 frank Note Added: 0002147
2016-01-22 12:01 frank Note Edited: 0002147
2016-01-22 12:02 frank Status assigned => feedback
2016-01-22 13:15 Shodan Note Added: 0002148
2016-01-22 13:15 Shodan Status feedback => assigned
2016-01-22 13:18 Shodan Note Edited: 0002148
2016-01-22 15:06 frank Note Added: 0002150
2016-01-22 15:06 frank Status assigned => feedback
2016-01-22 15:11 frank Relationship added related to 0000570
2016-01-22 17:09 Shodan Note Added: 0002155
2016-01-22 17:09 Shodan Status feedback => assigned
2016-01-22 17:18 Shodan Note Edited: 0002155
2016-01-22 17:28 frank Note Added: 0002156
2016-01-22 17:28 frank Status assigned => feedback
2016-01-25 09:48 Shodan Note Added: 0002158
2016-01-25 09:48 Shodan Status feedback => assigned
2016-01-26 14:34 frank Note Added: 0002161
2016-01-26 14:36 frank Status assigned => feedback
2016-01-26 15:35 Shodan Note Added: 0002162
2016-01-26 15:35 Shodan Status feedback => assigned
2016-02-02 15:01 olga Note Added: 0002177
2016-02-02 15:44 Shodan Note Added: 0002180
2016-02-02 19:13 olga Note Added: 0002181
2016-02-03 20:06 Shodan Note Added: 0002183
2016-02-04 09:04 olga Note Added: 0002184
2016-02-04 09:13 Shodan Note Added: 0002185
2016-02-04 21:39 olga Note Added: 0002186
2016-06-10 17:36 frank Note Added: 0002293
2016-06-10 17:36 frank Status assigned => feedback
2016-06-12 21:01 master_volkov Note Added: 0002295
2016-06-12 21:02 master_volkov Note Edited: 0002295
2016-07-07 15:18 Shodan Note Added: 0002305
2016-07-07 15:18 Shodan Status feedback => assigned
2016-07-07 15:21 Shodan Note Edited: 0002305
2016-07-07 15:21 Shodan Note Edited: 0002305
2016-07-07 15:34 Shodan Note Edited: 0002305
2016-11-03 17:51 frank Status assigned => resolved
2016-11-03 17:51 frank Fixed in Version => 16.2.4
2016-11-03 17:51 frank Resolution open => fixed
2017-06-08 16:31 frank Status resolved => closed
2017-06-08 16:31 frank Assigned To frank =>