View Issue Details

IDProjectCategoryView StatusLast Update
0001167bareos-corefile daemonpublic2023-07-17 10:11
Reporteroleg.cherkasov Assigned Tobruno-at-bareos  
PriorityhighSeverityminorReproducibilityalways
Status closedResolutionunable to reproduce 
Platformx86_64OSFreeBSDOS Version11.3-RELEASE-p5
Product Version17.2.8 
Summary0001167: bareos-fd daemon/fork mode does not as expected, after ESTIMATE or run a job the bareos-fd freezes with no log messages
DescriptionHi,

I have rather strange issue with 4 FreeBSD hosts recently upgraded to 11.3 from 11.2.

If bareos-fd running as a service/daemon (the way it suppose to) so it serves just few ESTIMATE on RUN JOB request. Then stops responding completely. DIR times out if I try to reach bareos-fd on those 4 FreeBSD hosts. Tried running with debugging options however it just stops logging, looks like it hangs somewhere waiting for I/O event from network or disk.

Running bareos-fd manually with -f option makes it work as normal:

/usr/local/sbin/bareos-fd -u root -g wheel -v -c /usr/local/etc/bareos/ -f

rc.conf/sysrc configuration:

bareos_fd_enable: YES

No custom options, no ports built rather all stack packages from PKG.

Upgrading to the latest patch level 5 does not help. Updating bareos17-client to the latest 17.2.8 does not solve the problem either.

It is worth to note that FreeBSD 12 and 12.1 run bareos-fd as a daemon with no issues.

Wonder if it is FreeBSD 11.3 issue only. Would make sense to check changelog related to fork()/vfork() updates however it is just a hypothesis.

Thanks,
Oleg
Steps To ReproduceRun ESTIMATE or RUN JOB more then once against the same bareos-fd on FreeBSD 11.3.
Additional Informationbareos17-client-17.2.8 package is used.
Tagsdebian 10, fd freebsd

Activities

mboscovich

mboscovich

2020-05-05 23:48

reporter   ~0003983

I have the same problem
Debian 10, bareos-filedaemon version 17.2.4-9.1.

I put bareos-fd on debug mode and this is the log when I run a estimate from director

root@pluton:~# bareos-fd -d 200 -c /etc/bareos/ -f
bareos-fd (100): parse_conf.c:151-0 config file = /etc/bareos/bareos-fd.d/*/*.conf
bareos-fd (100): lex.c:333-0 glob /etc/bareos/bareos-fd.d/*/*.conf: 4 files
bareos-fd (100): lex.c:227-0 open config file: /etc/bareos/bareos-fd.d/client/myself.conf
bareos-fd (100): lex.c:227-0 open config file: /etc/bareos/bareos-fd.d/director/bareos-dir.conf
bareos-fd (100): lex.c:227-0 open config file: /etc/bareos/bareos-fd.d/director/bareos-mon.conf
bareos-fd (100): lex.c:227-0 open config file: /etc/bareos/bareos-fd.d/messages/Standard.conf
bareos-fd (90): filed_conf.c:453-0 Inserting Director res: backup-dir
bareos-fd (100): lex.c:333-0 glob /etc/bareos/bareos-fd.d/*/*.conf: 4 files
bareos-fd (100): lex.c:227-0 open config file: /etc/bareos/bareos-fd.d/client/myself.conf
bareos-fd (100): lex.c:227-0 open config file: /etc/bareos/bareos-fd.d/director/bareos-dir.conf
bareos-fd (100): lex.c:227-0 open config file: /etc/bareos/bareos-fd.d/director/bareos-mon.conf
bareos-fd (100): lex.c:227-0 open config file: /etc/bareos/bareos-fd.d/messages/Standard.conf
pluton-fd (100): jcr.c:141-0 read_last_jobs seek to 192
pluton-fd (100): jcr.c:148-0 Read num_items=0
pluton-fd (150): fd_plugins.c:1664-0 plugin dir is NULL
pluton-fd (10): socket_server.c:97-0 filed: listening on port 9102
pluton-fd (100): bnet_server_tcp.c:174-0 Addresses host[ipv4;0.0.0.0;9102]
...
...
pluton-fd (100): accurate_htable.c:100-0 add fname=</etc/rc5.d/S01rsyslog> lstat=P4A NQ KH/ B A A A R BAA A Beh4g1 BdzAqT BeY84T A A d delta_seq=0 chksum=
pluton-fd (100): accurate_htable.c:100-0 add fname=</etc/rc5.d/S01console-setup.sh> lstat=P4A NY KH/ B A A A a BAA A Beh4g1 BdzAqh BeY84T A A d delta_seq=0 chksum=
pluton-fd (100): accurate_htable.c:100-0 add fname=</etc/rc5.d/> lstat=P4A NO EHt C A A A BAA BAA I Beh4g1 BeY9Wx BeY9Wx A A d delta_seq=0 chksum=
pluton-fd (100): accurate_htable.c:100-0 add fname=</etc/> lstat=P4A CAB EHt BG A A A BAA BAA I BeoLDn BeY9Wy BeY9Wy A A d delta_seq=0 chksum=
pluton-fd (100): accurate_htable.c:100-0 add fname=</> lstat=P4A C EHt S A A A BAA BAA I Beh4ez BeY84T BeY84T A A d delta_seq=0 chksum=
pluton-fd (100): dir_cmd.c:497-0 <dird: estimate listing=0

pluton-fd (100): dir_cmd.c:508-0 Executing estimate command.
pluton-fd (50): find.c:164-0 Verify=<V> Accurate=<Cmcs> BaseJob=<Jspug5> flags=<603987664>

And then its freeze there.
On director I showed this

Select Job resource (1-198): 165
Using Catalog "mycatalog"
Connecting to Client pluton-fd at pluton.intranet:9102

And never ends
arogge

arogge

2020-05-06 12:54

manager   ~0003984

You're running with debuglevel 200, findlib debugging will happen with 450 and above. I don't think it hangs, it probably just takes a very very long time.
Maybe you could retry with a higher debuglevel?
bruno-at-bareos

bruno-at-bareos

2023-07-06 16:23

manager   ~0005171

Is this still reproducible with current supported version of OS and Bareos 22x?
By the way, the time the estimate is running you can check if there activity on the concerning bareo-fd with command like lsof etc ....
It could look stale during hours on big FS search.
bruno-at-bareos

bruno-at-bareos

2023-07-17 10:11

manager   ~0005186

Closing no feedback, nor reproducible. Please test with supported OS and Bareos version.

Issue History

Date Modified Username Field Change
2020-01-10 14:08 oleg.cherkasov New Issue
2020-01-10 14:08 oleg.cherkasov Tag Attached: fd freebsd
2020-05-05 23:12 mboscovich Tag Attached: debian 10
2020-05-05 23:48 mboscovich Note Added: 0003983
2020-05-06 12:54 arogge Note Added: 0003984
2023-07-06 16:23 bruno-at-bareos Assigned To => bruno-at-bareos
2023-07-06 16:23 bruno-at-bareos Status new => feedback
2023-07-06 16:23 bruno-at-bareos Note Added: 0005171
2023-07-17 10:11 bruno-at-bareos Status feedback => closed
2023-07-17 10:11 bruno-at-bareos Resolution open => unable to reproduce
2023-07-17 10:11 bruno-at-bareos Note Added: 0005186