View Issue Details

IDProjectCategoryView StatusLast Update
0000786bareos-core[All Projects] storage daemonpublic2017-04-10 11:39
ReporterRobert SmitAssigned To 
PrioritynormalSeveritycrashReproducibilityrandom
Status newResolutionopen 
PlatformLinuxOSUbuntuOS Version14.04
Product Version16.2.4 
Fixed in Version 
Summary0000786: Storage daemon crashes at random
DescriptionWe have several Bareos-dir's operating on one Bareos-sd. This Bareos-sd seems to crash (bareos-sd service not running) at some point without prior notice, causing all the scheduled backups to fail!
The Bareos-sd backups to local storage (disks).
SD Version: 16.2.4 (01 July 2016) x86_64-pc-linux-gnu ubuntu Ubuntu 14.04 LTS

We have already tried the storage Daemon on two different machines (one VM and now on dedicated hardware).
Steps To ReproduceThis problem seems hard to reproduce, as it happens only once or twice (in our case) per month.
Additional InformationWe are running the storage daemon with debug level 20 now... This is from the log file at moment the crash occurred:
21-Feb 02:45 filestorage01 JobId 105657: Elapsed time=00:00:36, Transfer rate=22 Bytes/second
21-Feb 02:45 filestorage01 JobId 320669: Sending spooled attrs to the Director. Despooling 24,217 bytes ...
21-Feb 02:45 filestorage01 JobId 320679: Sending spooled attrs to the Director. Despooling 613 bytes ...
21-Feb 02:45 filestorage01: ABORTING due to ERROR in lockmgr.c:93
Mutex lock failure. ERR=Invalid argument
TagsNo tags attached.
bareos-master: impact
bareos-master: action
bareos-19.2: impact
bareos-19.2: action
bareos-18.2: impact
bareos-18.2: action
bareos-17.2: impact
bareos-17.2: action
bareos-16.2: impact
bareos-16.2: action
bareos-15.2: impact
bareos-15.2: action
bareos-14.2: impact
bareos-14.2: action
bareos-13.2: impact
bareos-13.2: action
bareos-12.4: impact
bareos-12.4: action

Activities

joergs

joergs

2017-02-22 10:01

administrator   ~0002579

I assume, bareos has created some traceback files in /var/lib/bareos/ ?

However, without installed debug packages they are not very useful.
Please install bareos-dbg and attach the traceback file on the next crash.
Robert Smit

Robert Smit

2017-04-10 11:39

reporter   ~0002623

This issue has not happened anymore.
A similar issue, where the SD also crashes happened more frequently and we (mistakenly) assumed it was the same issue.
The other issue with the crashing SD was solved by ourselves.

We had found in the log files that maximum open files in Linux was not high enough. After increasing the maximum open files in Linux we did not have any more crashes.

Therefore this issue can be closed.

Issue History

Date Modified Username Field Change
2017-02-21 11:28 Robert Smit New Issue
2017-02-22 10:01 joergs Note Added: 0002579
2017-02-22 10:01 joergs Status new => feedback
2017-04-10 11:39 Robert Smit Note Added: 0002623
2017-04-10 11:39 Robert Smit Status feedback => new