View Issue Details

IDProjectCategoryView StatusLast Update
0000512bareos-core[All Projects] Generalpublic2015-11-06 17:39
ReportergchongongAssigned To 
PrioritylowSeverityminorReproducibilitysometimes
Status acknowledgedResolutionopen 
PlatformLinuxOSCentOSOS Version7
Product Version14.2.5 
Fixed in Version 
Summary0000512: Abnormal backup
DescriptionI have a Centos 7 system with the following disk layout

Disk /dev/sda: 64.4 GB, 64424509440 bytes, 125829120 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000a83e8

   Device Boot Start End Blocks Id System
/dev/sda1 * 2048 4196351 2097152 83 Linux
/dev/sda2 4196352 104857599 50330624 8e Linux LVM
/dev/sda3 104857600 125829119 10485760 8e Linux LVM
[root@freeipa01 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/centos-root 20G 2.1G 18G 11% /
devtmpfs 1.9G 0 1.9G 0% /dev
tmpfs 1.9G 0 1.9G 0% /dev/shm
tmpfs 1.9G 185M 1.7G 10% /run
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/mapper/centos-home 10G 33M 10G 1% /home
/dev/sda1 2.0G 117M 1.9G 6% /boot
/dev/mapper/centos-tmp 10G 33M 10G 1% /tmp
/dev/mapper/centos-var 10G 809M 9.3G 8% /var

When I do a full backup of this system, the backup take took more than 5 hours, wrote 763GB on 9 volumes as you can see in the report below.

03-Sep 03:13 bareos-dir JobId 295: Bareos bareos-dir 14.2.2 (12Dec14):
  Build OS: x86_64-redhat-linux-gnu redhat CentOS Linux release 7.0.1406 (Core)
  JobId: 295
  Job: freeipa01.2015-09-02_21.47.31_13
  Backup Level: Full
  Client: "freeipa01-fd" 14.2.2 (12Dec14) x86_64-redhat-linux-gnu,redhat,CentOS Linux release 7.0.1406 (Core)
  FileSet: "Linux All" 2015-03-20 15:18:24
  Pool: "Full" (From user input)
  Catalog: "MyCatalog" (From Client resource)
  Storage: "File" (From Job resource)
  Scheduled time: 02-Sep-2015 21:47:19
  Start time: 02-Sep-2015 21:47:33
  End time: 03-Sep-2015 03:13:26
  Elapsed time: 5 hours 25 mins 53 secs
  Priority: 10
  FD Files Written: 87,617
  SD Files Written: 87,617
  FD Bytes Written: 763,405,009,097 (763.4 GB)
  SD Bytes Written: 763,418,862,188 (763.4 GB)
  Rate: 39042.9 KB/s
  Software Compression: None
  VSS: no
  Encryption: no
  Accurate: no
  Volume name(s): Full-0349|Full-0343|Full-0350|Full-0351|Full-0352|Full-0353|Full-0354|Full-0355|Full-0356
  Volume Session Id: 2
  Volume Session Time: 1441244045
  Last Volume Bytes: 8,987,859,692 (8.987 GB)
  Non-fatal FD errors: 0
  SD Errors: 0
  FD termination status: OK
  SD termination status: OK
  Termination: Backup OK


I am having this behaviour with my other centos 7 systems and it is filling up the disk storage, Is this normal behaviour?

Steps To ReproduceI have tried on other centos 7 systems and get the same results
TagsNo tags attached.
bareos-master: impactyes
bareos-master: action
bareos-19.2: impact
bareos-19.2: action
bareos-18.2: impact
bareos-18.2: action
bareos-17.2: impact
bareos-17.2: action
bareos-16.2: impact
bareos-16.2: action
bareos-15.2: impactyes
bareos-15.2: action
bareos-14.2: impactyes
bareos-14.2: action
bareos-13.2: impact
bareos-13.2: action
bareos-12.4: impact
bareos-12.4: action

Activities

mvwieringen

mvwieringen

2015-09-03 16:32

developer   ~0001818

Sparse files ? without sparse file setting in fileset
Backing up the data that you backup e.g. forgot to exclude the backup area ?
Not backing up any device files etc. e.g. the normal fileset work fine for
others.

If its backing up that data amount its there we do not make up anything just
walk the filesystem open the files and backup the data.

Try the estimate command on the client with the same fileset and check what
included is what you expect to be part of the backup.

Thousands of backups are made every day so this must be a local config problem.
gchongong

gchongong

2015-09-03 16:52

reporter   ~0001819

Here is the estimate. Seems to match the number of bytes copied. I cant understand how a disk of 65GB would hold 763.4 GB of data...thats puzzling


Using Catalog "MyCatalog"
Connecting to Client freeipa01-fd at freeipa01:9102
2000 OK estimate files=87,679 bytes=763,406,448,298

 FD Bytes Written: 763,405,009,097 (763.4 GB)
 SD Bytes Written: 763,418,862,188 (763.4 GB)


Here is the filesetFileSet {
  Name = "Linux All"
  Include {
    Options {
      Signature = MD5 # calculate md5 checksum per file
      One FS = No # change into other filessytems
      FS Type = ext2 # filesystems of given types will be backed up
      FS Type = ext3 # others will be ignored
      FS Type = ext4
      FS Type = xfs
      FS Type = reiserfs
      FS Type = jfs
      FS Type = btrfs
    }
    File = /
    File = /boot
    File = /var
    File = /home
    File = /opt

  }

Exclude block
  Exclude {
    File = /var/lib/bareos
    File = /var/lib/bareos/storage
    File = /proc
    File = /tmp
    File = /.journal
    File = /.fsck
  }
gchongong

gchongong

2015-09-03 18:40

reporter   ~0001820

Found the problem, sparse files... but setting sparse to yes did not fix it either, had to exclude /var/log/lastlog

[root@freeipa01 log]# du -h /var/log/lastlog
60K /var/log/lastlog

[root@freeipa01 log]# du -h --apparent-size lastlog
354G lastlog
[root@freeipa01 log]#

Issue History

Date Modified Username Field Change
2015-09-03 15:11 gchongong New Issue
2015-09-03 16:28 mvwieringen Priority high => low
2015-09-03 16:28 mvwieringen Severity major => minor
2015-09-03 16:32 mvwieringen Note Added: 0001818
2015-09-03 16:52 gchongong Note Added: 0001819
2015-09-03 18:40 gchongong Note Added: 0001820
2015-11-06 17:39 maik bareos-master: impact => yes
2015-11-06 17:39 maik bareos-15.2: impact => yes
2015-11-06 17:39 maik bareos-14.2: impact => yes
2015-11-06 17:39 maik Status new => acknowledged