View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0000719 | bareos-core | director | public | 2016-11-01 19:49 | 2017-12-18 12:12 |
Reporter | hop | Assigned To | |||
Priority | high | Severity | major | Reproducibility | always |
Status | closed | Resolution | fixed | ||
Platform | Linux | OS | Ubuntu | OS Version | 14.04 |
Product Version | 16.2.4 | ||||
Fixed in Version | 17.2.4 | ||||
Summary | 0000719: ERROR in output_formatter.c:822 Failed to send result as json. | ||||
Description | Webui fails. 01-Nov 19:05 qdehd4-dir: ERROR in output_formatter.c:822 Failed to send result as json. Maybe result message to long? Message length = 3318667 I did already read those other simmilar issues, and did learn, it was a "design decision". BUT I WANT TO DOUBT THAT DECISION HERE. HOW can it be, that the GUI defines how many volumes I have to have? (Or, as others wrote in the other reports, the number of backuped files?) So either there has to be a parameter to configure the message-size, or it has to be set to a higher value. | ||||
Steps To Reproduce | use "too" much volumes | ||||
Additional Information | And maybe, as a goodie, there could be a way to choose what the gui shows (and therefor loads/sends) when opening the "volumes" page. | ||||
Tags | No tags attached. | ||||
The maximum message length for TCP connections is 1000000 bytes, see https://github.com/bareos/bareos/blob/master/src/lib/bsock_tcp.h#L40 There have been a change that with 16.2 arbitrary message length are possible, however, other issues did arise from this, so it have been removed again. This is not a limitation of the webui, but of the Director->Console->JSON-Backend. How many volumes do you use? Have you updated from earlier versions? If yes, this the problem did not occur there? |
|
now we see none at all, which is worse than with the version 15.2, where the error Message popped up, but the gui still showed some (all? I didn't count them :-) ) But if it is a problem in the backend, why do I never have those issues with bconsole ?? |
|
You will have the same error, when you set the bconsole in JSON mode (command: ".api json") und sent the same command (llist volumes?). | |
Which shows, either the (json) platform or the way it is used is not suitable for this backup tool. Backups and especially their reports of the backuped files etc. can be VERY, VERY long. A backup-tool has to be able to cope with that. (BTW: Bareos fails in sending the Backup report, too. As this mail intends to have the number of volumes purged/used inside) It is "not the best thing", that a backup tool gets irritated by the number of files, volumes or whatever makes status or index reports "too long" for the tool. How can i be sure, i restored everything... etc etc (Note edited by author to be less offensive) |
|
I have the same issue , so i still have the same error when make go to restore tab and browser incremental backup , so all full backup are listed properly you see below 21-Nov 16:29 bdr01-dir: ERROR in output_formatter.c:822 Failed to send result as json. Maybe result message to long? Message length = 582 21-Nov 16:33 bdr01-dir: ERROR in output_formatter.c:822 Failed to send result as json. Maybe result message to long? Message length = 582 21-Nov 17:00 bdr01-dir: ERROR in output_formatter.c:822 Failed to send result as json. Maybe result message to long? Message length = 582 21-Nov 17:05 bdr01-dir: ERROR in output_formatter.c:822 Failed to send result as json. Maybe result message to long? Message length = 582 21-Nov 17:45 bdr01-dir: ERROR in output_formatter.c:822 Failed to send result as json. Maybe result message to long? Message length = 317 |
|
Also that happend when update to version 16.2 # @/etc/bareos/bareos-dir.d/webui-profiles.conf # # # bareos-webui default profile resource # Profile { Name = webui CommandACL = !.bvfs_clear_cache, !.exit, !.sql, !configure, !create, !delete, !purge, !sqlquery, !umount, !unmount, *all* |
|
@magedshaher: also the same code issues the error message, your problem is not related to the original problem of this ticket. The original problem is about too long messages (Message length = 3318667). You logs to only show short messages, that can not be sent for unknown reasons. Maybe you can write a message to the bareos-users mailing list and explain, * when the problem started * what exactly happend |
|
I'm also seeing the same error on two different installations. Installation one has 1680 volumes and installation two has 1336 volumes. Backing up to multiple 8TB drives with 50GB volumes using vchanger. I'm seeing this error when trying to view the pools or volumes under the storages tab. Running version: 16.4.3.1482412748 on both installations. BTW, as time goes on, we will be adding many more 8TB drives. So we could easily see 10,000 volumes as time goes on. |
|
I have the same Error when exec command "llist jobs". My bareos has 100 vms, and backup job will trigger 1hour every day |
|
Same here. "llist jobs days=3" is often sufficient to trigger the error. It also happens when I want to see the contents of our tape library. I just tried the experimental bareos-webui-16.4.3.1486047414.2712657 version and it still has the problem. In my opinion this is priority "block" or at least "high" because it make the UI barely usable on non-trivial sites. |
|
I'm going to change the category of the report to "director", as it is the source of the limitation and we need to fix it there first. | |
Not sure if I hit the same error ( ERROR in output_formatter.c:822... Message length = 5312), but maybe found a way to fix. 1. I found out when I did not merge filesets it worked. This made me believe it might have to do something with the processing size/time. 2.a. I changed the max_input_values in php.ini ( is 1000 default ) to a high number. 2.b. Changed max_execution_time in php.ini 2.c. Ran .bvfs_update command in bconsole. 3. I am not sure what action ins step 2 triggered it, but error is gone Maybe it helps someone. |
|
Turns out issue was not solved with my above comments. 13-Apr 10:56 bareos-dir: ERROR in output_formatter.c:822 Failed to send result as json. Maybe result message to long? Message length = 155 |
|
Probably related... this occurs, I think, when there are a lot of volumes in the Storages->Volumes tab. An error occurred An error occurred during execution; please try again later. Additional information: Zend\Json\Exception\RuntimeException File: /usr/share/bareos-webui/vendor/zendframework/zend-json/src/Json.php:66 Message: Decoding failed: Unexpected control character found Stack trace: #0 /usr/share/bareos-webui/module/Media/src/Media/Model/MediaModel.php(36): Zend\Json\Json::decode('{ "jsonrpc": "2...', 1) 0000001 /usr/share/bareos-webui/module/Media/src/Media/Controller/MediaController.php(63): Media\Model\MediaModel->getVolumes(Object(Bareos\BSock\BareosBSock)) 0000002 /usr/share/bareos-webui/vendor/zendframework/zend-mvc/src/Controller/AbstractActionController.php(82): Media\Controller\MediaController->indexAction() 0000003 [internal function]: Zend\Mvc\Controller\AbstractActionController->onDispatch(Object(Zend\Mvc\MvcEvent)) 0000004 /usr/share/bareos-webui/vendor/zendframework/zend-eventmanager/src/EventManager.php(444): call_user_func(Array, Object(Zend\Mvc\MvcEvent)) 0000005 /usr/share/bareos-webui/vendor/zendframework/zend-eventmanager/src/EventManager.php(205): Zend\EventManager\EventManager->triggerListeners('dispatch', Object(Zend\Mvc\MvcEvent), Object(Closure)) 0000006 /usr/share/bareos-webui/vendor/zendframework/zend-mvc/src/Controller/AbstractController.php(118): Zend\EventManager\EventManager->trigger('dispatch', Object(Zend\Mvc\MvcEvent), Object(Closure)) 0000007 /usr/share/bareos-webui/vendor/zendframework/zend-mvc/src/DispatchListener.php(93): Zend\Mvc\Controller\AbstractController->dispatch(Object(Zend\Http\PhpEnvironment\Request), Object(Zend\Http\PhpEnvironment\Response)) 0000008 [internal function]: Zend\Mvc\DispatchListener->onDispatch(Object(Zend\Mvc\MvcEvent)) 0000009 /usr/share/bareos-webui/vendor/zendframework/zend-eventmanager/src/EventManager.php(444): call_user_func(Array, Object(Zend\Mvc\MvcEvent)) 0000010 /usr/share/bareos-webui/vendor/zendframework/zend-eventmanager/src/EventManager.php(205): Zend\EventManager\EventManager->triggerListeners('dispatch', Object(Zend\Mvc\MvcEvent), Object(Closure)) 0000011 /usr/share/bareos-webui/vendor/zendframework/zend-mvc/src/Application.php(314): Zend\EventManager\EventManager->trigger('dispatch', Object(Zend\Mvc\MvcEvent), Object(Closure)) 0000012 /usr/share/bareos-webui/public/index.php(24): Zend\Mvc\Application->run() 0000013 {main} I also get an email with this message: 25-Apr 22:54 bareos-dir: ERROR in output_formatter.c:822 Failed to send result as json. Maybe result message to long? Message length = 4994996 |
|
We use one volume for every backup/job and according to it we have the same error. Is there any ETA to get this issue fixed or any recommendations/workaround how to get it working now ? |
|
Fix committed to bareos master branch with changesetid 7176. | |
bareos: master 42d7e719 2017-06-30 20:18 Ported: N/A Details Diff |
bsock: allow sending long messages While API modes 0 and 1 normally send data as small chunks (often line based), API mode 2 (JSON) collects the result of a bcommand, forms a JSON object and sends it. This can result in large messages. The maximum valid size of the result of a bcommand used to be 4 MB. The size of "llist jobs limit=1" is already close to 1 KB. So before this patch, "llist jobs" silently fails if the Bareos Director returns more than 5000 jobs. With this patch, the maximum length of a bcommand result is int32 (2GB). This is a modified version of commit 84ba7730bb4799f0793b2c0ee8572c5c75970ffe, which had to be reverted because of problems. Fixes 0000719: ERROR in output_formatter.c:822 Failed to send result as json. |
Affected Issues 0000719 |
|
mod - src/lib/bsock_tcp.c | Diff File |
Date Modified | Username | Field | Change |
---|---|---|---|
2016-11-01 19:49 | hop | New Issue | |
2016-11-02 10:21 | joergs | Note Added: 0002424 | |
2016-11-02 10:21 | joergs | Priority | high => normal |
2016-11-02 10:21 | joergs | Severity | block => minor |
2016-11-02 10:21 | joergs | Status | new => feedback |
2016-11-02 11:11 | hop | Note Added: 0002425 | |
2016-11-02 11:11 | hop | Status | feedback => new |
2016-11-02 11:14 | joergs | Note Added: 0002426 | |
2016-11-03 11:04 | hop | Note Added: 0002428 | |
2016-11-03 13:39 | hop | Note Edited: 0002428 | |
2016-11-21 17:09 | magedshaker | Note Added: 0002450 | |
2016-11-21 17:11 | magedshaker | Note Added: 0002451 | |
2016-11-22 11:06 | joergs | Note Added: 0002452 | |
2017-01-26 15:49 | Ryushin | Note Added: 0002532 | |
2017-01-26 15:51 | Ryushin | Note Edited: 0002532 | |
2017-02-09 07:59 | penn | Note Added: 0002553 | |
2017-03-02 13:22 | frank | Assigned To | => frank |
2017-03-02 13:22 | frank | Status | new => assigned |
2017-03-06 16:53 | Signum | Note Added: 0002602 | |
2017-03-06 17:21 | frank | Priority | normal => high |
2017-03-06 17:21 | frank | Severity | minor => major |
2017-03-17 11:11 | frank | Note Added: 0002612 | |
2017-03-17 11:11 | frank | Assigned To | frank => joergs |
2017-03-17 11:11 | frank | Category | webui => director |
2017-03-30 23:15 | pazion | Note Added: 0002619 | |
2017-03-30 23:17 | pazion | Note Edited: 0002619 | |
2017-04-13 11:13 | pazion | Note Added: 0002626 | |
2017-04-26 01:16 | asalvo | Note Added: 0002631 | |
2017-05-19 08:47 | romans | Note Added: 0002642 | |
2017-06-16 15:08 | stephand | Relationship added | related to 0000794 |
2017-09-02 00:27 | joergs | Changeset attached | => bareos master 42d7e719 |
2017-09-02 00:27 | joergs | Note Added: 0002727 | |
2017-09-02 00:27 | joergs | Status | assigned => resolved |
2017-09-02 00:27 | joergs | Resolution | open => fixed |
2017-12-18 12:06 | joergs | Relationship added | child of 0000861 |
2017-12-18 12:12 | joergs | Status | resolved => closed |
2017-12-18 12:12 | joergs | Assigned To | joergs => |
2017-12-18 12:12 | joergs | Fixed in Version | => 17.2.4 |