View Issue Details

IDProjectCategoryView StatusLast Update
0001328bareos-core[All Projects] file daemonpublic2021-03-11 09:19
ReporterlluuaappAssigned To 
PrioritynormalSeveritycrashReproducibilityalways
Status newResolutionopen 
PlatformLinuxOSDebianOS Version10
Product Version20.0.1 
Fixed in Version 
Summary0001328: File Daemon Crash when Using Python bareos-fd-postgres Plugin
DescriptionDear Bareos Delevelopers,

the Postgres WAL backup, running twice every hours, crashes the file daemon several time a day on both of our servers where we are using it. Here is a debug log and a traceback attached to this ticket. I can (privately) send you a core dump if you wish.

The crash happens after the backup is finished successfully (the director does not complain about anything) and the plugin is released. This happens not everytime, but it crashes almost every 6 hours (so maybe every 10-12 backups).

Please tell me is you need any additional information.

Thanks,
Paul
Steps To Reproduce1. Setup the PostgreSQL Plugin (bareos-fd-postgres)
2. Run it more often than nightly
3. Wait until the client/fd crashes
TagsNo tags attached.
bareos-master: impact
bareos-master: action
bareos-19.2: impact
bareos-19.2: action
bareos-18.2: impact
bareos-18.2: action
bareos-17.2: impact
bareos-17.2: action
bareos-16.2: impact
bareos-16.2: action
bareos-15.2: impact
bareos-15.2: action
bareos-14.2: impact
bareos-14.2: action
bareos-13.2: impact
bareos-13.2: action
bareos-12.4: impact
bareos-12.4: action

Activities

lluuaapp

lluuaapp

2021-03-11 09:19

reporter  

bareos.8325.traceback (10,761 bytes)
logs.txt (10,046 bytes)
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (100): filed/backup.cc:557-845 FT_RESTORE_FIRST saving: python:module_path=/usr/lib/bareos/plugins:module_name=bareos-fd-postgres:postgresDataDir=/var/lib/postgresql/11/main:walArchive=/usr/local/var/lib/postgresql/11/archive/
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (130): filed/backup.cc:660-845 filed: sending python:module_path=/usr/lib/bareos/plugins:module_name=bareos-fd-postgres:postgresDataDir=/var/lib/postgresql/11/main:walArchive=/usr/local/var/lib/postgresql/11/archive/ to stored
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (150): filed/fd_plugins.cc:1111-845 SendPluginName=python:module_path=/usr/lib/bareos/plugins:module_name=bareos-fd-postgres:postgresDataDir=/var/lib/postgresql/11/main:walArchive=/usr/local/var/lib/postgresql/11/archive/
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (150): filed/fd_plugins.cc:1121-845 send plugin name hdr: 2 26 0
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (150): filed/fd_plugins.cc:1139-845 send plugin start/end: 2 1 1 python:module_path=/usr/lib/bareos/plugins:module_name=bareos-fd-postgres:postgresDataDir=/var/lib/postgresql/11/main:walArchive=/usr/local/var/lib/postgresql/11/archive/
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (300): filed/backup.cc:1294-845 encode_and_send_attrs fname=python:module_path=/usr/lib/bareos/plugins:module_name=bareos-fd-postgres:postgresDataDir=/var/lib/postgresql/11/main:walArchive=/usr/local/var/lib/postgresql/11/archive/
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (300): filed/backup.cc:1314-845 File python:module_path=/usr/lib/bareos/plugins:module_name=bareos-fd-postgres:postgresDataDir=/var/lib/postgresql/11/main:walArchive=/usr/local/var/lib/postgresql/11/archive/
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  attribs=A A IHA A A A A -B BAA B BgSaGF BgSaGF BgSaGF A A C
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  attribsEx=<NULL>
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (300): filed/backup.cc:1343-845 >stored: attrhdr 2 28 0iwan-fd (200): lib/binflate.cc:58-845 In: 20220 bytes
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (200): lib/binflate.cc:63-845 compressed=2454
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (100): filed/backup.cc:1415-845 Object compressed from 20220 to 2454 bytes
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (300): filed/backup.cc:1448-845 >stored: attr len=2689: 2 25 1615438213 2454 20220 1 python:module_path=/usr/lib/bareos/plugins:module_name=bareos-fd-postgres:postgresDataDir=/var/lib/postgresql/11/main:walArchive=/usr/local/var/lib/postgresql/11/archive/
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (150): filed/fd_plugins.cc:1111-845 SendPluginName=python:module_path=/usr/lib/bareos/plugins:module_name=bareos-fd-postgres:postgresDataDir=/var/lib/postgresql/11/main:walArchive=/usr/local/var/lib/postgresql/11/archive/
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (150): filed/fd_plugins.cc:1121-845 send plugin name hdr: 2 26 0
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (150): filed/fd_plugins.cc:1139-845 send plugin start/end: 2 0
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (100): module/bareosfd.cc:1442-845 python3-fd-mod: end_backup_file() entry point in Python called
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (100): filed/heartbeat.cc:157-845 Send kill to heartbeat id
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (100): lib/bsock.cc:136-0 Destruct BareosSocket
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (100): lib/bsock.cc:136-0 Destruct BareosSocket
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (100): filed/backup.cc:208-845 end blast_data ok=1
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (110): filed/dir_cmd.cc:2685-845 3000 OK append data
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (110): filed/dir_cmd.cc:2685-845 3000 OK end
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (200): filed/dir_cmd.cc:2182-845 SDJobStatus = 84 T
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (500): lib/bget_msg.cc:67-845 Got BNET_EOD
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (150): filed/fd_plugins.cc:441-845 plugin_ctx=7fb6281330e0 JobId=845
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (150): filed/fd_plugins.cc:229-845 IsEventForThisPlugin? name=<NULL> len=0 plugin=python3-fd.so plen=7
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (100): module/bareosfd.cc:1442-845 python3-fd-mod: handle_plugin_event called with bEventEndBackupJob
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (100): filed/dir_cmd.cc:476-845 Quit command loop. Canceled=0
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (200): lib/runscript.cc:118-845 runscript: running all RunScript object (ClientAfterJob) JobStatus=T
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (110): filed/dir_cmd.cc:512-845 End FD msg: 2800 End Job TermCode=84 JobFiles=2 ReadBytes=16777216 JobBytes=79904 Errors=0 VSS=0 Encrypt=1
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (150): filed/fd_plugins.cc:441-845 plugin_ctx=7fb6281330e0 JobId=845
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (150): filed/fd_plugins.cc:229-845 IsEventForThisPlugin? name=<NULL> len=0 plugin=python3-fd.so plen=7
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (100): module/bareosfd.cc:1442-845 python3-fd-mod: handle_plugin_event called with bEventJobEnd
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (150): filed/fd_plugins.cc:1909-845 Free instance fd-plugin_ctx_list=7fb6281330e0 JobId=845
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  BAREOS interrupted by signal 11: Segmentation violation
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  bareos-fd, iwan-fd got signal 11 - Segmentation violation. Attempting traceback.
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  exepath=/usr/sbin/
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (300): lib/signal.cc:183-845 Working=/var/lib/bareos
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (300): lib/signal.cc:184-845 btpath=/usr/sbin/btraceback
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (300): lib/signal.cc:185-845 exepath=/usr/sbin/bareos-fd
2021-03-11 05:50:13 iwan bareos-fd: err  BAREOS interrupted by signal 11: Segmentation violation
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  iwan-fd (500): lib/signal.cc:215-845 Doing waitpid
2021-03-11 05:50:13 iwan bareos-fd[1846]: info  Calling: /usr/sbin/btraceback /usr/sbin/bareos-fd 1847 /var/lib/bareos
2021-03-11 05:50:15 iwan bareos-fd[1846]: info  bsmtp: tools/bsmtp.cc:130-0 Fatal malformed reply from localhost: 501 <root>: sender address must contain a domain
2021-03-11 05:50:15 iwan bareos-fd[1846]: info  iwan-fd (500): lib/signal.cc:217-845 Done waitpid
2021-03-11 05:50:15 iwan bareos-fd[1846]: info  The btraceback call returned 1
2021-03-11 05:50:15 iwan bareos-fd[1846]: info  Dumping: /var/lib/bareos/iwan-fd.1847.bactrace
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  bareos-fd (100): lib/parse_conf.cc:210-0 config file = /etc/bareos/bareos-fd.d/*/*.conf
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  bareos-fd (100): lib/lex.cc:330-0 glob /etc/bareos/bareos-fd.d/*/*.conf: 5 files
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  bareos-fd (100): lib/lex.cc:223-0 open config file: /etc/bareos/bareos-fd.d/client/myself.conf
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  bareos-fd (100): lib/lex.cc:223-0 open config file: /etc/bareos/bareos-fd.d/director/backup-dir.conf
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  bareos-fd (100): lib/lex.cc:223-0 open config file: /etc/bareos/bareos-fd.d/director/bareos-dir.conf
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  bareos-fd (100): lib/lex.cc:223-0 open config file: /etc/bareos/bareos-fd.d/director/bareos-mon.conf
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  bareos-fd (100): lib/lex.cc:223-0 open config file: /etc/bareos/bareos-fd.d/messages/Standard.conf
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  bareos-fd (100): lib/lex.cc:330-0 glob /etc/bareos/bareos-fd.d/*/*.conf: 5 files
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  bareos-fd (100): lib/lex.cc:223-0 open config file: /etc/bareos/bareos-fd.d/client/myself.conf
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  bareos-fd (100): lib/lex.cc:223-0 open config file: /etc/bareos/bareos-fd.d/director/backup-dir.conf
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  bareos-fd (100): lib/lex.cc:223-0 open config file: /etc/bareos/bareos-fd.d/director/bareos-dir.conf
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  bareos-fd (100): lib/lex.cc:223-0 open config file: /etc/bareos/bareos-fd.d/director/bareos-mon.conf
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  bareos-fd (100): lib/lex.cc:223-0 open config file: /etc/bareos/bareos-fd.d/messages/Standard.conf
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  iwan-fd (250): lib/message.cc:246-0 Copied message resource 560351d77050
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  iwan-fd (100): lib/bsys.cc:576-0 ReadStateFile seek to 192
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  iwan-fd (100): lib/recent_job_results_list.cc:57-0 Read num_items=10
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  iwan-fd (50): lib/plugins.cc:212-0 LoadPlugins
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  iwan-fd (150): filed/fd_plugins.cc:1792-0 IsPluginCompatible called
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  iwan-fd (150): filed/fd_plugins.cc:1769-0 Loaded plugin: python3-fd.so
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  iwan-fd (10): filed/socket_server.cc:128-0 filed: listening on port 9102
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  iwan-fd (10): filed/socket_server.cc:128-0 filed: listening on port 9102
2021-03-11 05:50:15 iwan bareos-fd[2761]: info  iwan-fd (100): lib/bnet_server_tcp.cc:160-0 Addresses host[ipv6;2a01:4f8:171:281a::2;9102] host[ipv4;138.201.17.27;9102]
logs.txt (10,046 bytes)

Issue History

Date Modified Username Field Change
2021-03-11 09:19 lluuaapp New Issue
2021-03-11 09:19 lluuaapp File Added: bareos.8325.traceback
2021-03-11 09:19 lluuaapp File Added: logs.txt