View Issue Details

IDProjectCategoryView StatusLast Update
0000891bareos-corewebuipublic2019-12-18 15:25
ReporterHiFlyer Assigned Tofrank  
PrioritynormalSeveritymajorReproducibilityalways
Status closedResolutionfixed 
PlatformDell 9550OSUbuntuOS Version16.04
Product Version17.2.4 
Summary0000891: Webui will not open some windows clients directories
DescriptionJust upgraded from 16.2.4 to 17.2.4
Linux clients seem to work correctly.
Windows client with large directory will not open. Based on Frank solution for Oliver Fink, modified the RestoreModel.php file on Lines 67 & 81 changing == 2 to <= 2. and rebooted server.
Windows client with large directory still will not open. smaller directories will open.
then tried
.bvfs_clear_cache yes
systemclt restart bareos-dir
.bvfs_update
still does not open large windows directory.
Steps To Reproduceopen firefox window to server
go to restore tab
select windows machine.
open directories till I get to Documents/ (which gives the window that there are to many files)
open a different directory, it opens.
TagsNo tags attached.

Relationships

related to 0000893 closedfrank webui cannot list dirs with only files in them 

Activities

frank

frank

2018-01-19 12:23

developer   ~0002874

Ok, I've done some testing in my lab again. Webui had the mentioned patch as yours has and was running on a Fedora 27 VM.

DIRD, SD and FD running on a CentOS 7 VM.

Version : 17.2.4
Release : 9.1.el7

C:/Users/Administrator/Documents on a Windows Server 2012r2 contained 2000 folders and 2000 files.

- Backed C:/Users/Administrator/Documents
- Moved over to webui restore form for browsing the file tree

No issues, works like expected.

Questions:

a) How many folders and/or files does your Documents folder exactly contain?

b) Does it throw the error message "... too many files.", or keeps the wheel just spinning?

c) Are you able to browse through the file tree in bconsole via BVFS API and what does the output look like?

If you don't know how to use the BVFS API, just ask and I give you some assistance.
HiFlyer

HiFlyer

2018-01-19 18:05

reporter   ~0002877

Hopefully, the answers to your questions follow.

a small sample for the directory "Documents" from bconsole list jobid=2002 (which is my last full backup)

 C:/Users/xxxx/Documents/Amazon Kindle/Finding the Way and Other Tales of Valdemar_B00466ISG2.sdr/Finding the Way and Other Tales of Valdemar_B00466ISG2.mbs
 C:/Users/xxxx/Documents/Amazon Kindle/Fractured-asin_B0011UJMMK-type_EBOK-v_0.mbp
 C:/Users/xxxx/Documents/Amazon Kindle/Scarecrow Returns_B004T4KWYM.sdr/XRAY.entities.B004T4KWYM.asc
 C:/Users/xxxx/Documents/Amazon Kindle/The Real Enemy (Sophie Trace Trilogy, Book 1)_B002G1ZQ5A.azw1

I replaced the user name with xxxx

I imported the bconsole listing into Libreofficecalc and using / as a divider to gain the counts below:

In counts below, entries may be either files or directories in the counts. This info comes from the last full backup (no incrementals in webui or bconsole) using the bconsole list command sorted in spreadsheet

"Desktop" directory opens and displays correctly in Webui

In the directory "Desktop" there are a total of of 11,175 entries in all (including all files in subdirectories) of which there are 17 directories entries in "Desktop" directory

2 subdirectories down in "Desktop" one of the directories (ie "Newsletter") also says "Oops, something went wrong, probably too many files." in another window it opens

"Newsletter" has 25 entries and no subdirectories

"Documents" on attempt to display shows "Oops, something went wrong, probably too many files." in another window it opens

In the directory "Documents" there are a total of 15,012 entries in all (including all files in subdirectories) of which there are 55 directories entries in "Documents" directory

"Favorites" on attempt to display shows "Oops, something went wrong, probably too many files." in another window it opens

"Favorites" only has the following has the following files/directory entries:

Favorites/Bing.url
Favorites/Dell/Dell Auction.url
Favorites/Dell/Dell.url
Favorites/Dell/Support.Dell.Com.url
Favorites/Dell
Favorites/desktop.ini
Favorites/Links/desktop.ini
Favorites/Links
Favorites

"Music" on attempt to display shows "Oops, something went wrong, probably too many files." in another window it opens
In the directory "Music" there are a total of 85 files in all (including all files in subdirectories) of which there are 2 directories entries in "Music" directory

"Videos" on attempt to display shows "Oops, something went wrong, probably too many files." in another window it opens
"Videos" only has the following has the following files/directory entries:

Videos/Captures/desktop.ini
Videos/Captures
Videos/desktop.ini
Videos/Hidden by me - Shortcut.lnk
Videos

I have not attempted to open every possible subdirectory in the backup on Webui.

another interesting point, there is a second user on this machine and the following directories also display the "Oops, something went wrong, probably too many files."
"Favorites", "Pictures"
as this user is really not used, I have included all entries below from bconsole. It has also been a long time since I have logged into this user (i.e. me)

Application Data
Contacts/desktop.ini
Contacts
Cookies
Desktop/desktop.ini
Desktop
Documents/desktop.ini
Documents/My Music
Documents/My Pictures
Documents/My Videos
Documents
Downloads/desktop.ini
Downloads
Favorites/Bing.url
Favorites/Dell/Dell Auction.url
Favorites/Dell/Dell.url
Favorites/Dell/Support.Dell.Com.url
Favorites/Dell
Favorites/desktop.ini
Favorites/Links/desktop.ini
Favorites/Links
Favorites
IntelGraphicsProfiles/Brighten Video.man.igpi
IntelGraphicsProfiles/Darken Video.man.igpi
IntelGraphicsProfiles/Enhance Video Colors.man.igpi
IntelGraphicsProfiles
Links/desktop.ini
Links/Desktop.lnk
Links/Downloads.lnk
Links
Local Settings
Music/desktop.ini
Music
My Documents
NetHood
NTUSER.DAT
ntuser.dat.LOG1
ntuser.dat.LOG2
NTUSER.DAT{f89f0369-cc29-11e7-98b2-dfe6da7b2e58}.TM.blf
NTUSER.DAT{f89f0369-cc29-11e7-98b2-dfe6da7b2e58}.TMContainer00000000000000000001.regtrans-ms
NTUSER.DAT{f89f0369-cc29-11e7-98b2-dfe6da7b2e58}.TMContainer00000000000000000002.regtrans-ms
OneDrive/desktop.ini
OneDrive
Pictures/Camera Roll/desktop.ini
Pictures/Camera Roll
Pictures/desktop.ini
Pictures/Saved Pictures/desktop.ini
Pictures/Saved Pictures
Pictures
PrintHood
Recent
Saved Games/desktop.ini
Saved Games
Searches/desktop.ini
Searches/Everywhere.search-ms
Searches/Indexed Locations.search-ms
Searches/winrt--{S-1-5-21-519149844-2766317099-1904787894-1002}-.searchconnector-ms
Searches
SendTo
Start Menu
Templates
Videos/desktop.ini
Videos

If you wish me to use different command, please identify. Not sure how to browse in bconsole and as you asked for counts this method seemed to answer your questions.
also I initiated a restore of the offending "documents" directory that would not display in webui to my Linux computer (different physical computer) and while I canceled after awhile, it seemed to be working correctly.
In case it makes any difference, these backups were made when both the server (on Ubuntu 16.04) and the windows 10 machine were using bareos 16.2.4
and again thanks for the help.

also am I supposed to enclose things in the /code thing. I have forgotten exactly how to do.

bee
HiFlyer

HiFlyer

2018-01-24 14:08

reporter   ~0002881

Correction on my original issue. The problem is not restricted to Windows 10 computer. I started checking sub directories on my linux computer. some of them do not open and give me the error message "Oops, something went wrong, probably too many files" there is also a option on this page to "prevent this page from creating additional logs". This option is NOT presented on all error message windows even though it is a repeat request to open the same directory.

In addition I checked marked one of the small offending directories in webui for restore and it restored all 4 of the files and a subdirectory with 1 file correctly to my designated directory (/tmp/bareos-restores/home/...).

Summary, the issue described is against both windows 10 and Linux (Ubuntu 17.04) clients for display of the directories to be restored but it seems on a very limited test, the webui will restore the files even though the webui will not display those directories.

bee
HiFlyer

HiFlyer

2018-02-10 02:12

reporter   ~0002904

Frank, is there anything you can suggest I try to help define the issue better as you cannot seem to replicate the issue
HiFlyer

HiFlyer

2018-03-10 20:04

reporter   ~0002940

update
I have replaced 2 lines in the file /usr/bareos-webui/module/Restore/src/Restore/Controller/RestoreController.php
I changed line 349 from
               $items .= ',"text":"' . preg_replace('/[\x00-\x1F\x7F]/', '', $dir["name"]) . '"';
to
               $items .= ',"text":"' . preg_replace('/[\x00-\x1F\x7F]/', '', str_replace('"', '\"', $dir["name"])) . '"';

and line 372 from
            $items .= ',"text":"' . preg_replace('/[\x00-\x1F\x7F]/', '', addslashes($file["name"])) . '"';
to
            $items .= ',"text":"' . preg_replace('/[\x00-\x1F\x7F]/', '', str_replace('"', '\"', $file["name"])) . '"';

making them the same as git has for this file.
This has improved my ability to display the directories that previously would not display. It has not completely resolved the display issue.

Further based on the Linux
find . -iname "*\"*"
command, I have no files or directories with a " mark in them, and therefore should not have affected my situation to my best understanding.
HiFlyer

HiFlyer

2018-03-10 21:14

reporter   ~0002941

The issue appears to still be missing comma's after some types of directories. See from chrome diagonistic that there is a missing comma between the Directory name command and the file name statement below.

[{"id":"-7230","text":"Weil Media/","icon":"glyphicon glyphicon-folder-close","state":"","data":{"type":"D","lstat":"QME lATO EHt C Po Po A DAA BAA Y Bam1ms BSmiUg BXJPux A A d","jobid":2364,"pathid":7230,"fileid":5865435,"linkfileindex":0,"name":"Weil Media\/","fullpath":"\/home\/beckstein\/Documents\/My_Genealogy\/pictures\/Weil Media\/","stat":{"dev":66308,"user":"beckstein","ino":9700558,"nlink":2,"mode":16877,"rdev":0,"group":"beckstein","size":12288,"ctime":1462041521,"atime":1520130476,"mtime":1385833760}},"children":true}{"id":"5962133","text":"Christian Eckstein Grave.jpg","icon":"glyphicon glyphicon-file","state":"","data":{"lstat":"QME lAV6 IGg B Po Po A O85y BAA B3o Bad9+8 BUOo0+ BXJPux A A d","type":"F","jobid":2364,"pathid":7231,"fileid":5962133,"linkfileindex":0,"name":"Christian Eckstein Grave.jpg","stat":{"dev":66308,"user":"beckstein","ino":9700730,"nlink":1,"mode":33184,"rdev":0,"group":"beckstein","size":3919474,"ctime":1462041521,"atime":1517805500,"mtime":1413123390}}}]

This particular directory has one sub directory and one file in it. It seems there needs to be a comma after :true},{"id":
I am not sure where/how the fix needs to be in the file RestoreController.php
also thank you tonyalbers for identifying how to capure this info - in bug 0000893
HiFlyer

HiFlyer

2018-03-11 00:25

reporter   ~0002942

I have a solution to my problem.
In the file /usr/bareos-webui/module/Restore/src/Restore/Controller/RestoreController.php
I have modified line 355 to add an OR $fnum > 0 to the test for adding the comma and deleted the lines 363, 364 and 365 which appear to be an inappropriate patch (at least in my case) to fix the comma problem.
Please consider the modification for the base code.
thank you

    if($dnum > 0) {

         foreach($this->directories as $dir) {
            if($dir['name'] == ".") {
               --$dnum;
               next($this->directories);
            }
            elseif($dir['name'] == "..") {
               --$dnum;
               next($this->directories);
            }
            else {
               --$dnum;
               $items .= '{';
               $items .= '"id":"-' . $dir['pathid'] . '"';
               $items .= ',"text":"' . preg_replace('/[\x00-\x1F\x7F]/', '', str_replace('"', '\"', $dir["name"])) . '"';
               $items .= ',"icon":"glyphicon glyphicon-folder-close"';
               $items .= ',"state":""';
               $items .= ',"data":' . \Zend\Json\Json::encode($dir, \Zend\Json\Json::TYPE_OBJECT);
               $items .= ',"children":true';
               $items .= '}';
               if($dnum > 0 || $fnum > 0 ) {
                  $items .= ",";
               }
            }
         }

      }

/** if( $tmp > 2 && $fnum > 0 ) {
         $items .= ",";
      }
*/
      if($fnum > 0) {
frank

frank

2018-03-12 11:37

developer   ~0002943

This is related to https://bugs.bareos.org/view.php?id=893 which has already been fixed, thanks anyways.

Cheers
Frank

Issue History

Date Modified Username Field Change
2018-01-16 23:38 HiFlyer New Issue
2018-01-17 17:32 frank Assigned To => frank
2018-01-17 17:32 frank Status new => assigned
2018-01-19 12:23 frank Note Added: 0002874
2018-01-19 12:29 frank Status assigned => acknowledged
2018-01-19 18:05 HiFlyer Note Added: 0002877
2018-01-24 14:08 HiFlyer Note Added: 0002881
2018-02-10 02:12 HiFlyer Note Added: 0002904
2018-03-10 20:04 HiFlyer Note Added: 0002940
2018-03-10 21:14 HiFlyer Note Added: 0002941
2018-03-11 00:25 HiFlyer Note Added: 0002942
2018-03-12 11:33 frank Relationship added related to 0000893
2018-03-12 11:37 frank Note Added: 0002943
2018-03-12 11:38 frank Status acknowledged => resolved
2018-03-12 11:38 frank Resolution open => fixed
2019-12-18 15:25 arogge Status resolved => closed