0001004: Very high cpu usage on Debian stretch

ID	Project	Category	View Status	Date Submitted	Last Update

0001004	bareos-core	storage daemon	public	2018-09-03 09:22	2019-12-18 15:24

Reporter	hostedpower	Assigned To	arogge
Priority	normal	Severity	major	Reproducibility	always
Status	closed	Resolution	fixed
Platform	Linux	OS	Debian	OS Version	9
Product Version	17.2.7

Summary	0001004: Very high cpu usage on Debian stretch
Description	Each time on consolidation we se a load of up to 18-20 (on a 6 cores machines). There are about 20 backup clients on this machine in total. This is a backup server with sd only, the director is somewhere else. This seems to have started pretty recently, there are 3 possible causes I can think of: - Bug in newer bareos version - Bug because of kernel update (less likely I would think) - Extra backup clients, but it has grown pretty slowly and I don't think we saw such high usage before
Steps To Reproduce	Use consolidation for about 20 clients on single SD, please see screenshot with perf top. htop shows load of about 18-20 and high cpu usage on a couple of bareos-sd processes.
Tags	No tags attached.

hostedpower 2018-09-03 09:22 reporter	high-cpu-bareos.png (65,852 bytes) high-cpu-bareos.png (65,852 bytes)

arogge 2019-07-31 15:55 manager ~0003543	I just had a look: it is in fact CRC32 that is that slow. There are faster implementations available and we will consider integrating one of them.

hostedpower 2019-07-31 16:00 reporter ~0003545	Wow that would be terrific, we really suffer on that server, even more than before since we've grown a bit in the meanwhile :)

arogge 2019-10-28 13:04 manager ~0003613	We have replaced the crc implementation with a faster one. This will be released in 19.2

bareos: master 838aef14 2019-08-19 18:02 arogge Ported: N/A Details Diff	core: replace crc32 implementation The previous crc32 implementation was not optimal from a performance point-of-view. This patch introduces Stephan Brumme's Fast CRC32 which is a lot faster than the old algorithm.	Affected Issues 0001004
	mod - core/src/stored/CMakeLists.txt	Diff File
	mod - core/src/stored/block.cc	Diff File
	rm - core/src/stored/crc32.cc	Diff File
	add - core/src/stored/crc32/Crc32.cpp	Diff File
	add - core/src/stored/crc32/Crc32.h	Diff File
	add - core/src/stored/crc32/Crc32.ino	Diff File
	add - core/src/stored/crc32/Crc32Best.ino	Diff File
	add - core/src/stored/crc32/Crc32Test.cpp	Diff File
	add - core/src/stored/crc32/LICENSE	Diff File
	add - core/src/stored/crc32/Makefile	Diff File
	add - core/src/stored/crc32/changelog.md	Diff File
	add - core/src/stored/crc32/crc32.js	Diff File
	add - core/src/stored/crc32/readme.md	Diff File
	mod - core/src/tests/CMakeLists.txt	Diff File

Date Modified	Username	Field	Change
2018-09-03 09:22	hostedpower	New Issue
2018-09-03 09:22	hostedpower	File Added: high-cpu-bareos.png
2019-07-31 15:55	arogge	Status	new => acknowledged
2019-07-31 15:55	arogge	Note Added: 0003543
2019-07-31 16:00	hostedpower	Note Added: 0003545
2019-10-28 13:01	arogge	Assigned To	=> arogge
2019-10-28 13:01	arogge	Status	acknowledged => assigned
2019-10-28 13:03	arogge	Changeset attached	=> bareos master 838aef14
2019-10-28 13:04	arogge	Status	assigned => resolved
2019-10-28 13:04	arogge	Resolution	open => fixed
2019-10-28 13:04	arogge	Note Added: 0003613
2019-12-18 15:24	arogge	Status	resolved => closed