View Issue Details

IDProjectCategoryView StatusLast Update
0001004bareos-core[All Projects] storage daemonpublic2019-10-28 13:04
ReporterhostedpowerAssigned Toarogge 
PrioritynormalSeveritymajorReproducibilityalways
Status resolvedResolutionfixed 
PlatformLinuxOSDebianOS Version9
Product Version17.2.7 
Fixed in Version 
Summary0001004: Very high cpu usage on Debian stretch
DescriptionEach time on consolidation we se a load of up to 18-20 (on a 6 cores machines).

There are about 20 backup clients on this machine in total. This is a backup server with sd only, the director is somewhere else.

This seems to have started pretty recently, there are 3 possible causes I can think of:

- Bug in newer bareos version
- Bug because of kernel update (less likely I would think)
- Extra backup clients, but it has grown pretty slowly and I don't think we saw such high usage before


Steps To ReproduceUse consolidation for about 20 clients on single SD, please see screenshot with perf top.

htop shows load of about 18-20 and high cpu usage on a couple of bareos-sd processes.
TagsNo tags attached.
bareos-master: impactyes
bareos-master: actionfixed
bareos-19.2: impact
bareos-19.2: action
bareos-18.2: impactyes
bareos-18.2: actionnone
bareos-17.2: impactyes
bareos-17.2: actionnone
bareos-16.2: impactyes
bareos-16.2: actionnone
bareos-15.2: impact
bareos-15.2: action
bareos-14.2: impact
bareos-14.2: action
bareos-13.2: impact
bareos-13.2: action
bareos-12.4: impact
bareos-12.4: action

Activities

hostedpower

hostedpower

2018-09-03 09:22

reporter  

high-cpu-bareos.png (65,852 bytes)
high-cpu-bareos.png (65,852 bytes)
arogge

arogge

2019-07-31 15:55

developer   ~0003543

I just had a look: it is in fact CRC32 that is that slow.
There are faster implementations available and we will consider integrating one of them.
hostedpower

hostedpower

2019-07-31 16:00

reporter   ~0003545

Wow that would be terrific, we really suffer on that server, even more than before since we've grown a bit in the meanwhile :)
arogge

arogge

2019-10-28 13:04

developer   ~0003613

We have replaced the crc implementation with a faster one. This will be released in 19.2

Related Changesets

bareos: master 838aef14

2019-08-19 16:02:25

arogge

Ported: N/A

Details Diff
core: replace crc32 implementation

The previous crc32 implementation was not optimal from a performance
point-of-view.
This patch introduces Stephan Brumme's Fast CRC32 which is a lot faster
than the old algorithm.
Affected Issues
0001004
mod - core/src/stored/CMakeLists.txt Diff File
mod - core/src/stored/block.cc Diff File
rm - core/src/stored/crc32.cc Diff File
add - core/src/stored/crc32/Crc32.cpp Diff File
add - core/src/stored/crc32/Crc32.h Diff File
add - core/src/stored/crc32/Crc32.ino Diff File
add - core/src/stored/crc32/Crc32Best.ino Diff File
add - core/src/stored/crc32/Crc32Test.cpp Diff File
add - core/src/stored/crc32/LICENSE Diff File
add - core/src/stored/crc32/Makefile Diff File
add - core/src/stored/crc32/changelog.md Diff File
add - core/src/stored/crc32/crc32.js Diff File
add - core/src/stored/crc32/readme.md Diff File
mod - core/src/tests/CMakeLists.txt Diff File

Issue History

Date Modified Username Field Change
2018-09-03 09:22 hostedpower New Issue
2018-09-03 09:22 hostedpower File Added: high-cpu-bareos.png
2019-07-31 15:55 arogge Status new => acknowledged
2019-07-31 15:55 arogge Note Added: 0003543
2019-07-31 16:00 hostedpower Note Added: 0003545
2019-10-28 13:01 arogge Assigned To => arogge
2019-10-28 13:01 arogge Status acknowledged => assigned
2019-10-28 13:03 arogge Changeset attached => bareos master 838aef14
2019-10-28 13:04 arogge Status assigned => resolved
2019-10-28 13:04 arogge Resolution open => fixed
2019-10-28 13:04 arogge bareos-master: impact => yes
2019-10-28 13:04 arogge bareos-master: action => fixed
2019-10-28 13:04 arogge bareos-18.2: impact => yes
2019-10-28 13:04 arogge bareos-18.2: action => none
2019-10-28 13:04 arogge bareos-17.2: impact => yes
2019-10-28 13:04 arogge bareos-17.2: action => none
2019-10-28 13:04 arogge bareos-16.2: impact => yes
2019-10-28 13:04 arogge bareos-16.2: action => none
2019-10-28 13:04 arogge Note Added: 0003613