View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0001092bareos-corefile daemonpublic2019-06-18 21:572023-08-23 13:55
ReporterCookiefamily Assigned Tobruno-at-bareos  
PrioritynormalSeveritycrashReproducibilityalways
Status closedResolutionno change required 
OSUbuntuOS Version18.04.2 LTS 
Product Version18.2.5 
Summary0001092: Out of memory when recovering with ReaR
DescriptionWhen recovering a Test-VM which I intentionally broke to recover it with Bareos and ReaR (first time testing the recovery process), the OOM-Killer kills the bareos-fd as well as other services and the restore aborts.
See the attachements for a screenshot of the KVM when the OOM-Killer went through.
Also attached is the output of the bareos-dir which obviously errors because the connection dies.
What might be interesting is the line "Error: lib/bsock_tcp.cc:417 Wrote 35 bytes to File Daemon:IP:9102, but only 0 accepted.", It seems like it doesn't even start copying files and locks up before that.

I can restore files just fine, when reinstalling the machine from scratch and doing a restore that way.
Steps To ReproduceClient:
1. Boot from ReaR-ISO, choose "Recover test01"
2. login as root
3. execute rear recover
4. enter yes for automatic disk layout configuration

When "waiting for job to start" appears do the following on the bareos-director:
1. restore client=test01.FQDN.com-fd
2. select most recent backup
3. mark *

Verify the settings:
JobName: test01.FQDN.com-restore
Bootstrap: /var/lib/bareos/bareos-dir.restore.5.bsr
Where: /
Replace: Always
FileSet: UnixDefault
Backup Client: test01.FQDN.com-fd
Restore Client: test01.FQDN.com-fd
Format: Native
Storage: File
When: 2019-06-09 19:12:24
Catalog: MyCatalog
Priority: 1
Plugin Options: *None*

Wait 0000001:0000020 seconds and the OOM Killer does its job. This works 100% of the time.
Additional InformationThe VM is a KVM running on a Xeon E3-1270v6 with 1 core, 2gb of Ram (upped it to 4gb now because I ran into the OOM, issue persists) and a 20gb virtual disk managed via Proxmox.
The Backup size is 2.8gb compressed.
The ReaR team sees the error in the bareos file deamon https://github.com/rear/rear/issues/2158 .
Tagscrash, fd

Activities

Cookiefamily

Cookiefamily

2019-06-18 21:57

reporter  

bareos-dir-log.txt (3,831 bytes)   
bareos-fd-oom.png (51,669 bytes)   
bruno-at-bareos

bruno-at-bareos

2023-08-23 13:55

manager   ~0005347

backporting upstream comment: The where parameter is set to a ram mount, so the restore simply fill it up quickly.

In case someone tumble upon this. I tried this today and got the same error. It turned out the default restore location Where is /tmp/bareos-restores, which on the rear recover console is in ram, thus you will get OOM pretty quick. You will need to specify the Where parameter and set it to /mnt/local which is the restoration destination disk. Hope it helps someone.

not a bug.

Issue History

Date Modified Username Field Change
2019-06-18 21:57 Cookiefamily New Issue
2019-06-18 21:57 Cookiefamily Tag Attached: crash
2019-06-18 21:57 Cookiefamily Tag Attached: fd
2019-06-18 21:57 Cookiefamily File Added: bareos-dir-log.txt
2019-06-18 21:57 Cookiefamily File Added: bareos-fd-oom.png
2023-08-23 13:55 bruno-at-bareos Assigned To => bruno-at-bareos
2023-08-23 13:55 bruno-at-bareos Status new => closed
2023-08-23 13:55 bruno-at-bareos Resolution open => no change required
2023-08-23 13:55 bruno-at-bareos Note Added: 0005347