Bareos Bug Tracker - bareos-core
View Issue Details
0001006bareos-core[All Projects] storage daemonpublic2018-09-12 12:452018-09-14 15:14
Martin Svec 
joergs 
normalcrashalways
resolvedduplicate 
LinuxDebian9
17.2.4 
 
0001006: Storage daemon segfaults in update_job_statistics when starting scheduled jobs
If job statistics are turned on and multiple backup jobs are scheduled and started at the same time, bareos-sd 17.2.4 almost always segfaults. When started in gdb, I got the following bareos-sd backtrace:

(gdb) bt
#0 0x000055555556c991 in update_job_statistics (jcr=0x7fffbc001078, now=1536531334) at sd_stats.c:296
0000001 0x000055555556cc93 in statistics_thread_runner (arg=arg@entry=0x0) at sd_stats.c:386
0000002 0x00007ffff74add9f in lmgr_thread_launcher (x=0x5555557b7328) at lockmgr.c:928
0000003 0x00007ffff6309494 in start_thread (arg=0x7fffeeffd700) at pthread_create.c:333
0000004 0x00007ffff519aacf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

As a workaround, it suffices to set "Collect Job Statistics = no".
Turn on job statistics, schedule multiple backup-to-disk jobs using one common Schedule and let the director to start all the jobs according to the Schedule. Note that the concurrency is important. If the jobs are started manually one by one, the bug doesn't occur.
Our setup: Bareos 17.2.4 installed from bareos.org Debian packages (http://download.bareos.org/bareos/release/latest/Debian_9.0/ [^]). Director, storage daemon and webui are all on one physical server, catalog is in MySQL database. Backups are stored on a local disk device. Storage daemon has twenty identical disk Device resources, pointing to the same location:

Device {
  Name = Disk-Bareos-01-000
  Media Type = Disk
  Device Type = File
  Archive Device = /backup
  LabelMedia = yes
  Random Access = yes
  AutomaticMount = yes
  RemovableMedia = no
  AlwaysOpen = no
  Maximum Concurrent Jobs = 1
}

...

Device {
  Name = Disk-Bareos-01-019
  Media Type = Disk
  Device Type = File
  Archive Device = /backup
  LabelMedia = yes
  Random Access = yes
  AutomaticMount = yes
  RemovableMedia = no
  AlwaysOpen = no
  Maximum Concurrent Jobs = 1
}


We've 88 jobs and four different nightly Schedules, Maximum Concurrent Jobs is set to 20 both in director and storage daemon. That is, director typically starts up to 20 jobs at the same time.
No tags attached.
duplicate of 0000983closed franku SD crashes sometimes with statistics collection on 
Issue History
2018-09-12 12:45Martin SvecNew Issue
2018-09-14 15:13joergsRelationship addedduplicate of 0000983
2018-09-14 15:14joergsStatusnew => resolved
2018-09-14 15:14joergsResolutionopen => duplicate
2018-09-14 15:14joergsAssigned To => joergs

There are no notes attached to this issue.