View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0000996 | bareos-core | director | public | 2018-08-06 08:51 | 2023-07-17 16:35 |
Reporter | gnif | Assigned To | bruno-at-bareos | ||
Priority | high | Severity | crash | Reproducibility | always |
Status | closed | Resolution | fixed | ||
Platform | Linux | OS | Debian | OS Version | 9 |
Product Version | 18.2.3 | ||||
Summary | 0000996: Reloading director configuration crashes with a double free | ||||
Description | When reloading the director's configuration it crashes with a double free, back trace provided. | ||||
Steps To Reproduce | simply execute: systemctl reload bareos-dir | ||||
Additional Information | Created /tmp//bareos-dir.core.2957 for doing postmortem debugging [New LWP 2957] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/sbin/bareos-dir'. #0 0x00007f6ae7f8daea in __waitpid (pid=2958, stat_loc=0x7ffff1ce980c, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29 29 ../sysdeps/unix/sysv/linux/waitpid.c: No such file or directory. $1 = 1701994850 $2 = -851886296 $3 = -851886232 $4 = 0 $5 = -393116296 $6 = -393116222 $7 = -393116196 $8 = -393116184 $9 = 1701994850 $10 = -393116272 Environment variable "TestName" not defined. #0 0x00007f6ae7f8daea in __waitpid (pid=2958, stat_loc=0x7ffff1ce980c, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29 0000001 0x00007f6ae89009e8 in SignalHandler () from /usr/lib/bareos/libbareos.so.18 0000002 <signal handler called> 0000003 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 0000004 0x00007f6ae670f42a in __GI_abort () at abort.c:89 0000005 0x00007f6ae674bc00 in __libc_message (do_abort=do_abort@entry=2, fmt=fmt@entry=0x7f6ae6840d98 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:175 0000006 0x00007f6ae6751fc6 in malloc_printerr (action=3, str=0x7f6ae6840e60 "double free or corruption (out)", ptr=<optimized out>, ar_ptr=<optimized out>) at malloc.c:5049 0000007 0x00007f6ae675280e in _int_free (av=0x7f6ae6a74b00 <main_arena>, p=0x5641cd3ac5b0, have_lock=0) at malloc.c:3905 0000008 0x00005641cc62b4ea in FreeResource(CommonResourceHeader*, int) () 0000009 0x00007f6ae96450bf in ConfigurationParser::~ConfigurationParser() () from /usr/lib/bareos/libbareoscfg.so.18 0000010 0x00005641cc633bd1 in TerminateDird(int) () 0000011 0x00005641cc63387e in main () Thread 1 (Thread 0x7f6ae9a66b80 (LWP 2957)): #0 0x00007f6ae7f8daea in __waitpid (pid=2958, stat_loc=0x7ffff1ce980c, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29 0000001 0x00007f6ae89009e8 in SignalHandler () from /usr/lib/bareos/libbareos.so.18 0000002 <signal handler called> 0000003 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 0000004 0x00007f6ae670f42a in __GI_abort () at abort.c:89 0000005 0x00007f6ae674bc00 in __libc_message (do_abort=do_abort@entry=2, fmt=fmt@entry=0x7f6ae6840d98 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:175 0000006 0x00007f6ae6751fc6 in malloc_printerr (action=3, str=0x7f6ae6840e60 "double free or corruption (out)", ptr=<optimized out>, ar_ptr=<optimized out>) at malloc.c:5049 0000007 0x00007f6ae675280e in _int_free (av=0x7f6ae6a74b00 <main_arena>, p=0x5641cd3ac5b0, have_lock=0) at malloc.c:3905 0000008 0x00005641cc62b4ea in FreeResource(CommonResourceHeader*, int) () 0000009 0x00007f6ae96450bf in ConfigurationParser::~ConfigurationParser() () from /usr/lib/bareos/libbareoscfg.so.18 0000010 0x00005641cc633bd1 in TerminateDird(int) () 0000011 0x00005641cc63387e in main () #0 0x00007f6ae7f8daea in __waitpid (pid=2958, stat_loc=0x7ffff1ce980c, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29 29 in ../sysdeps/unix/sysv/linux/waitpid.c resultvar = 18446744073709551104 0000001 0x00007f6ae89009e8 in SignalHandler () from /usr/lib/bareos/libbareos.so.18 No symbol table info available. 0000002 <signal handler called> No locals. 0000003 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. set = {__val = {0, 3615882721364815733, 3919881043631615540, 3472384402208154977, 4135818719080688944, 8223625903106963297, 3472328295963438455, 4192904167887482928, 7365366848950382640, 3486176864243181158, 7378697628691542064, 2319406792463180593, 3472328228581767026, 3472310704041635888, 2314885599538982970, 2314885530818453536}} pid = <optimized out> tid = <optimized out> 0000004 0x00007f6ae670f42a in __GI_abort () at abort.c:89 89 abort.c: No such file or directory. save_stage = 2 act = {__sigaction_handler = {sa_handler = 0x2020202020373133, sa_sigaction = 0x2020202020373133}, sa_mask = {__val = {2314885530818453536, 2314885530818453536, 3907004821653777455, 8461816663211521631, 7236210882687479160, 8030813362246791725, 4051380807995373322, 7365405400577882211, 3474917680625312054, 2337418197644357680, 3472328296227680304, 3467824696768081952, 4135818719080679968, 3975887029563646776, 3762301437856527974, 140737250238960}}, sa_flags = 93, sa_restorer = 0x7ffff1cea1f0} sigs = {__val = {32, 0 <repeats 15 times>}} 0000005 0x00007f6ae674bc00 in __libc_message (do_abort=do_abort@entry=2, fmt=fmt@entry=0x7f6ae6840d98 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:175 175 ../sysdeps/posix/libc_fatal.c: No such file or directory. ap = {{gp_offset = 40, fp_offset = 0, overflow_arg_area = 0x7ffff1cea200, reg_save_area = 0x7ffff1cea190}} fd = 2 on_2 = <optimized out> list = <optimized out> nlist = <optimized out> cp = <optimized out> written = <optimized out> 0000006 0x00007f6ae6751fc6 in malloc_printerr (action=3, str=0x7f6ae6840e60 "double free or corruption (out)", ptr=<optimized out>, ar_ptr=<optimized out>) at malloc.c:5049 5049 malloc.c: No such file or directory. buf = "00005641cd3ac5c0" cp = <optimized out> ar_ptr = <optimized out> ptr = <optimized out> str = 0x7f6ae6840e60 "double free or corruption (out)" action = 3 0000007 0x00007f6ae675280e in _int_free (av=0x7f6ae6a74b00 <main_arena>, p=0x5641cd3ac5b0, have_lock=0) at malloc.c:3905 3905 in malloc.c size = <optimized out> fb = <optimized out> nextchunk = <optimized out> nextsize = <optimized out> nextinuse = <optimized out> prevsize = <optimized out> bck = <optimized out> fwd = <optimized out> errstr = <optimized out> locked = <optimized out> __func__ = "_int_free" | ||||
Tags | No tags attached. | ||||
It should be noted that this is occurring with the latest nightly, 18.2.3-1388.1 | |
Just realized I attached the wrong back trace. Even still it looks like the same problem. Here is the correct back trace. Created //var/lib/bareos/bareos-dir.core.2779 for doing postmortem debugging [New LWP 2779] [New LWP 2781] [New LWP 2782] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/sbin/bareos-dir'. #0 0x00007f7760d1cb3a in __waitpid (pid=2915, stat_loc=0x7ffd20f76dcc, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29 29 ../sysdeps/unix/sysv/linux/waitpid.c: No such file or directory. [Current thread is 1 (Thread 0x7f77627f5b80 (LWP 2779))] $1 = 1701994850 $2 = 1300803368 $3 = 1300803432 $4 = 1301118776 $5 = 1634366840 $6 = 1634366914 $7 = 1634366940 $8 = 1634366952 $9 = 1701994850 $10 = 1634366864 Environment variable "TestName" not defined. #0 0x00007f7760d1cb3a in __waitpid (pid=2915, stat_loc=0x7ffd20f76dcc, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29 0000001 0x00007f776168f9e8 in SignalHandler () from /usr/lib/bareos/libbareos.so.18 0000002 <signal handler called> 0000003 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 0000004 0x00007f775f49e42a in __GI_abort () at abort.c:89 0000005 0x00007f775f4dac00 in __libc_message (do_abort=do_abort@entry=2, fmt=fmt@entry=0x7f775f5cfd98 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:175 0000006 0x00007f775f4e0fc6 in malloc_printerr (action=3, str=0x7f775f5cfe60 "double free or corruption (out)", ptr=<optimized out>, ar_ptr=<optimized out>) at malloc.c:5049 0000007 0x00007f775f4e180e in _int_free (av=0x7f775f803b00 <main_arena>, p=0x55634d89f970, have_lock=0) at malloc.c:3905 0000008 0x000055634c6fe4ea in FreeResource(CommonResourceHeader*, int) () 0000009 0x000055634c705d8b in FreeSavedResources(resource_table_reference*) () 0000010 0x000055634c707140 in DoReloadConfig() () 0000011 0x000055634c706c65 in SighandlerReloadConfig () 0000012 <signal handler called> 0000013 0x00007f7760d1c7dd in nanosleep () at ../sysdeps/unix/syscall-template.S:84 0000014 0x00007f776166733b in Bmicrosleep(int, int) () from /usr/lib/bareos/libbareos.so.18 0000015 0x000055634c73b315 in wait_for_next_job(char*) () 0000016 0x000055634c7069f1 in main () Thread 3 (Thread 0x7f775de3c700 (LWP 2782)): #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225 0000001 0x00007f77616a00e5 in watchdog_thread () from /usr/lib/bareos/libbareos.so.18 0000002 0x00007f7760d13494 in start_thread (arg=0x7f775de3c700) at pthread_create.c:333 0000003 0x00007f775f552acf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97 Thread 2 (Thread 0x7f775e63d700 (LWP 2781)): #0 0x00007f775f54967d in poll () at ../sysdeps/unix/syscall-template.S:84 0000001 0x00007f77616572ed in BnetThreadServerTcp(dlist*, int, alist*, workq_tag*, bool, void* (*)(void*)) () from /usr/lib/bareos/libbareos.so.18 0000002 0x000055634c733f12 in connect_thread () 0000003 0x00007f7760d13494 in start_thread (arg=0x7f775e63d700) at pthread_create.c:333 0000004 0x00007f775f552acf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97 Thread 1 (Thread 0x7f77627f5b80 (LWP 2779)): #0 0x00007f7760d1cb3a in __waitpid (pid=2915, stat_loc=0x7ffd20f76dcc, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29 0000001 0x00007f776168f9e8 in SignalHandler () from /usr/lib/bareos/libbareos.so.18 0000002 <signal handler called> 0000003 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 0000004 0x00007f775f49e42a in __GI_abort () at abort.c:89 0000005 0x00007f775f4dac00 in __libc_message (do_abort=do_abort@entry=2, fmt=fmt@entry=0x7f775f5cfd98 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:175 0000006 0x00007f775f4e0fc6 in malloc_printerr (action=3, str=0x7f775f5cfe60 "double free or corruption (out)", ptr=<optimized out>, ar_ptr=<optimized out>) at malloc.c:5049 0000007 0x00007f775f4e180e in _int_free (av=0x7f775f803b00 <main_arena>, p=0x55634d89f970, have_lock=0) at malloc.c:3905 0000008 0x000055634c6fe4ea in FreeResource(CommonResourceHeader*, int) () 0000009 0x000055634c705d8b in FreeSavedResources(resource_table_reference*) () 0000010 0x000055634c707140 in DoReloadConfig() () 0000011 0x000055634c706c65 in SighandlerReloadConfig () 0000012 <signal handler called> 0000013 0x00007f7760d1c7dd in nanosleep () at ../sysdeps/unix/syscall-template.S:84 0000014 0x00007f776166733b in Bmicrosleep(int, int) () from /usr/lib/bareos/libbareos.so.18 0000015 0x000055634c73b315 in wait_for_next_job(char*) () 0000016 0x000055634c7069f1 in main () #0 0x00007f7760d1cb3a in __waitpid (pid=2915, stat_loc=0x7ffd20f76dcc, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29 29 in ../sysdeps/unix/sysv/linux/waitpid.c resultvar = 18446744073709551104 sc_cancel_oldtype = 2 0000001 0x00007f776168f9e8 in SignalHandler () from /usr/lib/bareos/libbareos.so.18 No symbol table info available. 0000002 <signal handler called> No locals. 0000003 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. set = {__val = {1, 0, 93884991526184, 140150730437832, 140725156541568, 140150730355432, 5, 0, 140725156541608, 140150728229057, 17, 140150730355432, 5, 0, 1, 140150730436976}} pid = <optimized out> tid = <optimized out> 0000004 0x00007f775f49e42a in __GI_abort () at abort.c:89 89 abort.c: No such file or directory. save_stage = 2 act = {__sigaction_handler = {sa_handler = 0x55634d91d528, sa_sigaction = 0x55634d91d528}, sa_mask = {__val = {140150685565744, 140725156542560, 140150728278874, 93884990994408, 93884990969280, 4294967295, 0, 93884991004336, 93884990998160, 0, 93884991245504, 140725156541728, 140150663855075, 0, 93, 140725156542384}}, sa_flags = -9, sa_restorer = 0x7ffd20f77550} sigs = {__val = {32, 0 <repeats 15 times>}} 0000005 0x00007f775f4dac00 in __libc_message (do_abort=do_abort@entry=2, fmt=fmt@entry=0x7f775f5cfd98 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:175 175 ../sysdeps/posix/libc_fatal.c: No such file or directory. ap = {{gp_offset = 40, fp_offset = 0, overflow_arg_area = 0x7ffd20f777c0, reg_save_area = 0x7ffd20f77750}} fd = 2 on_2 = <optimized out> list = <optimized out> nlist = <optimized out> cp = <optimized out> written = <optimized out> 0000006 0x00007f775f4e0fc6 in malloc_printerr (action=3, str=0x7f775f5cfe60 "double free or corruption (out)", ptr=<optimized out>, ar_ptr=<optimized out>) at malloc.c:5049 5049 malloc.c: No such file or directory. buf = "000055634d89f980" cp = <optimized out> ar_ptr = <optimized out> ptr = <optimized out> str = 0x7f775f5cfe60 "double free or corruption (out)" action = 3 0000007 0x00007f775f4e180e in _int_free (av=0x7f775f803b00 <main_arena>, p=0x55634d89f970, have_lock=0) at malloc.c:3905 3905 in malloc.c size = <optimized out> fb = <optimized out> nextchunk = <optimized out> nextsize = <optimized out> nextinuse = <optimized out> prevsize = <optimized out> bck = <optimized out> fwd = <optimized out> errstr = <optimized out> locked = <optimized out> __func__ = "_int_free" |
|
I found the error, FreeResource is attempting to free dhfile three times in a row, looks like a C&P bug. Please see https://github.com/bareos/bareos/pull/104 |
|
0000104 is not complete, please see: https://github.com/bareos/bareos/pull/105 |
|
Fixed in master | |
Date Modified | Username | Field | Change |
---|---|---|---|
2018-08-06 08:51 | gnif | New Issue | |
2018-08-06 08:52 | gnif | Note Added: 0003087 | |
2018-08-06 09:12 | gnif | Note Added: 0003088 | |
2018-08-06 12:05 | gnif | Note Added: 0003092 | |
2018-08-06 12:11 | gnif | Note Added: 0003093 | |
2023-07-17 16:35 | bruno-at-bareos | Assigned To | => bruno-at-bareos |
2023-07-17 16:35 | bruno-at-bareos | Status | new => closed |
2023-07-17 16:35 | bruno-at-bareos | Resolution | open => fixed |
2023-07-17 16:35 | bruno-at-bareos | Note Added: 0005199 |