 |
Note: I use bareos 16.2.4-12.1 |
|
|
We were not able to reproduce the reported behaviour. The quick test looked like following: ------------------------------------- /dev/xvdb1 /data btrfs rw,relatime,ssd,space_cache,subvolid=5,subvol=/ 0 0 /dev/xvdb1 /data/cloud btrfs rw,relatime,ssd,space_cache,subvolid=257,subvol=/cloud 0 0 FileSet { Name = "fileset-bareos-fd" Include { Options { Signature = MD5 OneFS = No Fs Type = "btrfs" Fs Type = "ext2" Fs Type = "ext3" Fs Type = "ext4" Fs Type = "reiserfs" Fs Type = "jfs" Fs Type = "xfs" Fs Type = "zfs" } File = "/data" } Exclude { File = "/data/.snapshots" } } /data ├── cloud │ ├── d1 │ │ ├── f1 │ │ ├── f2 │ │ ├── f3 │ │ ├── f4 │ │ ├── f5 │ │ └── f6 │ ├── d2 │ └── d3 └── test ├── file1 ├── file2 └── file3 5 directories, 9 files 23-Aug 14:31 bareos-sd JobId 438: Volume "Full-0001" previously written, moving to end of data. 23-Aug 14:31 bareos-sd JobId 438: Ready to append to end of Volume "Full-0001" size=275182644 23-Aug 14:31 bareos-sd JobId 438: Elapsed time=00:00:20, Transfer rate=64 Bytes/second 23-Aug 14:31 bareos-dir JobId 438: Bareos bareos-dir 17.2.2 (01May17): Build OS: x86_64-redhat-linux-gnu redhat Fedora release 26 (Twenty Six) JobId: 438 Job: backup-bareos-fd.2017-08-23_14.30.40_51 Backup Level: Full Client: "bareos-fd" 17.2.2 (01May17) x86_64-redhat-linux-gnu,redhat,Fedora release 26 (Twenty Six),Fedora_26,x86_64 FileSet: "fileset-bareos-fd" 2017-08-23 12:19:30 Pool: "Full" (From Job FullPool override) Catalog: "MyCatalog" (From Client resource) Storage: "File" (From Pool resource) Scheduled time: 23-Aug-2017 14:30:36 Start time: 23-Aug-2017 14:31:03 End time: 23-Aug-2017 14:31:23 Elapsed time: 20 secs Priority: 10 FD Files Written: 15 SD Files Written: 15 FD Bytes Written: 0 (0 B) SD Bytes Written: 1,289 (1.289 KB) Rate: 0.0 KB/s Software Compression: None VSS: no Encryption: no Accurate: no Volume name(s): Full-0001 Volume Session Id: 9 Volume Session Time: 1503482479 Last Volume Bytes: 275,184,651 (275.1 MB) Non-fatal FD errors: 0 SD Errors: 0 FD termination status: OK SD termination status: OK Termination: Backup OK *list files jobid=438 /data/ /data/test/file1 /data/test/file2 /data/test/file3 /data/test/ /data/cloud/ /data/cloud/d3/ /data/cloud/d2/ /data/cloud/d1/f1 /data/cloud/d1/f2 /data/cloud/d1/f3 /data/cloud/d1/f4 /data/cloud/d1/f5 /data/cloud/d1/f6 /data/cloud/d1/ FD Trace: --------- 23-Aug-2017 14:31:07.413261 sheldon-fd (300): find_one.c:980-438 File ----: /data/cloud 23-Aug-2017 14:31:07.413272 sheldon-fd (450): find.c:247-438 enter accept_file: fname=/data/cloud 23-Aug-2017 14:31:07.413282 sheldon-fd (300): find_one.c:660-438 Create temp ff packet for dir: /data/cloud 23-Aug-2017 14:31:19.838969 sheldon-fd (200): heartbeat.c:100-0 wait_intr=0 stop=0 23-Aug-2017 14:31:19.839500 sheldon-fd (100): find_one.c:115-438 Accepting fstype btrfs for "/data/cloud" Please do a 'setdebug level=100 trace=1 timestamp=1 client=<client-with-btrfs-mounts>' in bconsole, run the job and attach the result .trace file to the report. Thanks & cheers, Frank |
|
|
Aha, it cannot determine the filesystem of the subvolumes in my setup. The btrfs subvolumes are not shown up in /proc/mounts and are indirectly mounted by btrfs itself: btrfs subvol list -t /data/ ID gen top level path -- --- --------- ---- 5319 643224 5 backups 5321 643287 5 intelliad 5322 643316 5321 intelliad/Groups 5323 643314 5321 intelliad/homes 5324 643305 5321 intelliad/Public 5326 643028 5 owncloud-old trace file: 24-Aug-2017 11:24:01.797135 hamburg.muc.intelliad.de (50): cram-md5.c:68-0 send: auth cram-md5 <1541027922.1503566641@hamburg.muc.intelliad.de> ssl=0 24-Aug-2017 11:24:01.798124 hamburg.muc.intelliad.de (100): cram-md5.c:123-0 cram-get received: auth cram-md5 <537810211.1503566641@bareos-dir> ssl=0 24-Aug-2017 11:24:01.798170 hamburg.muc.intelliad.de (99): cram-md5.c:143-0 sending resp to challenge: m6pQJ7+cGwMg0/UIa+wo8D 24-Aug-2017 11:24:01.799191 hamburg.muc.intelliad.de (100): dir_cmd.c:495-0 <dird: setdebug=100 trace=1 hangup=-1 timestamp=1 24-Aug-2017 11:24:01.799221 hamburg.muc.intelliad.de (100): dir_cmd.c:506-0 Executing setdebug= command. 24-Aug-2017 11:24:01.799235 hamburg.muc.intelliad.de (50): dir_cmd.c:923-0 setdebug_cmd: setdebug=100 trace=1 hangup=-1 timestamp=1 24-Aug-2017 11:24:01.799249 hamburg.muc.intelliad.de (50): dir_cmd.c:950-0 level=100 trace=1 hangup=0 timestamp=1 tracefilename=/var/lib/bareos/hamburg.muc.intelliad.de.trace 24-Aug-2017 11:24:01.799723 hamburg.muc.intelliad.de (100): dir_cmd.c:568-0 Done with free_jcr 24-Aug-2017 11:24:24.048839 hamburg.muc.intelliad.de (50): cram-md5.c:68-0 send: auth cram-md5 <1088453218.1503566664@hamburg.muc.intelliad.de> ssl=0 24-Aug-2017 11:24:24.050032 hamburg.muc.intelliad.de (100): cram-md5.c:123-0 cram-get received: auth cram-md5 <1606416222.1503566664@bareos-dir> ssl=0 24-Aug-2017 11:24:24.050105 hamburg.muc.intelliad.de (99): cram-md5.c:143-0 sending resp to challenge: Bj1Jj7/h+4kiDy/WIX+dGB 24-Aug-2017 11:24:24.051493 hamburg.muc.intelliad.de (100): dir_cmd.c:495-0 <dird: JobId=703 Job=Backup_hamburg.muc.intelliad.de.2017-08-24_11.24.21_06 SDid=684 SDtime=1503318714 Authorization=PIAC-GIGP-FEKL-AHOP-HPDM-IJLC-MHLH-AIAB 24-Aug-2017 11:24:24.051527 hamburg.muc.intelliad.de (100): dir_cmd.c:506-0 Executing JobId= command. 24-Aug-2017 11:24:24.051546 hamburg.muc.intelliad.de (5): dir_cmd.c:1647-703 set sd auth key 24-Aug-2017 11:24:24.053200 hamburg.muc.intelliad.de (100): dir_cmd.c:495-703 <dird: level = full mtime_only=0 24-Aug-2017 11:24:24.053233 hamburg.muc.intelliad.de (100): dir_cmd.c:506-703 Executing level = command. 24-Aug-2017 11:24:24.053248 hamburg.muc.intelliad.de (10): dir_cmd.c:1469-703 level_cmd: level = full mtime_only=0 24-Aug-2017 11:24:24.053977 hamburg.muc.intelliad.de (100): dir_cmd.c:495-703 <dird: fileset vss=1 24-Aug-2017 11:24:24.054001 hamburg.muc.intelliad.de (100): dir_cmd.c:506-703 Executing fileset command. 24-Aug-2017 11:24:24.094531 hamburg.muc.intelliad.de (100): fileset.c:263-703 I 24-Aug-2017 11:24:24.094571 hamburg.muc.intelliad.de (100): fileset.c:263-703 O Mf 24-Aug-2017 11:24:24.094590 hamburg.muc.intelliad.de (100): fileset.c:263-703 X btrfs 24-Aug-2017 11:24:24.094609 hamburg.muc.intelliad.de (100): fileset.c:263-703 X ext2 24-Aug-2017 11:24:24.094625 hamburg.muc.intelliad.de (100): fileset.c:263-703 X ext3 24-Aug-2017 11:24:24.094641 hamburg.muc.intelliad.de (100): fileset.c:263-703 X ext4 24-Aug-2017 11:24:24.094664 hamburg.muc.intelliad.de (100): fileset.c:263-703 X reiserfs 24-Aug-2017 11:24:24.094692 hamburg.muc.intelliad.de (100): fileset.c:263-703 X jfs 24-Aug-2017 11:24:24.094720 hamburg.muc.intelliad.de (100): fileset.c:263-703 X xfs 24-Aug-2017 11:24:24.094746 hamburg.muc.intelliad.de (100): fileset.c:263-703 X zfs 24-Aug-2017 11:24:24.094763 hamburg.muc.intelliad.de (100): fileset.c:263-703 N 24-Aug-2017 11:24:24.094777 hamburg.muc.intelliad.de (100): fileset.c:263-703 F /data 24-Aug-2017 11:24:24.094794 hamburg.muc.intelliad.de (100): fileset.c:263-703 N 24-Aug-2017 11:24:24.094818 hamburg.muc.intelliad.de (100): fileset.c:263-703 N 24-Aug-2017 11:24:24.094835 hamburg.muc.intelliad.de (100): fileset.c:263-703 E 24-Aug-2017 11:24:24.094856 hamburg.muc.intelliad.de (100): fileset.c:263-703 F /data/.snapshots 24-Aug-2017 11:24:24.094877 hamburg.muc.intelliad.de (100): fileset.c:263-703 N 24-Aug-2017 11:24:24.094903 hamburg.muc.intelliad.de (100): fileset.c:263-703 N 24-Aug-2017 11:24:24.095505 hamburg.muc.intelliad.de (100): dir_cmd.c:495-703 <dird: getSecureEraseCmd 24-Aug-2017 11:24:24.095529 hamburg.muc.intelliad.de (100): dir_cmd.c:506-703 Executing getSecureEraseCmd command. 24-Aug-2017 11:24:24.096229 hamburg.muc.intelliad.de (100): dir_cmd.c:495-703 <dird: storage address=silicon.b3.rz1.intelliad.com port=9103 ssl=0 24-Aug-2017 11:24:24.096263 hamburg.muc.intelliad.de (100): dir_cmd.c:506-703 Executing storage command. 24-Aug-2017 11:24:24.096279 hamburg.muc.intelliad.de (100): dir_cmd.c:1666-703 StorageCmd: storage address=silicon.b3.rz1.intelliad.com port=9103 ssl=0 24-Aug-2017 11:24:24.096295 hamburg.muc.intelliad.de (100): bsock.c:70-703 All source addresses 24-Aug-2017 11:24:24.100064 hamburg.muc.intelliad.de (100): bsock_tcp.c:230-703 Current host[ipv4;10.113.48.168;9103] All host[ipv4;10.113.48.168;9103] 24-Aug-2017 11:24:24.100609 hamburg.muc.intelliad.de (100): bsock_tcp.c:153-703 who=Storage daemon host=silicon.b3.rz1.intelliad.com port=9103 24-Aug-2017 11:24:24.101490 hamburg.muc.intelliad.de (100): cram-md5.c:123-703 cram-get received: auth cram-md5 <1582882299.1503566664@silicon.b3.rz1.intelliad.com> ssl=0 24-Aug-2017 11:24:24.101541 hamburg.muc.intelliad.de (99): cram-md5.c:143-703 sending resp to challenge: R+gZp20rg5+Nm3+327++dB 24-Aug-2017 11:24:24.101974 hamburg.muc.intelliad.de (50): cram-md5.c:75-703 send: auth cram-md5 <1880318342.1503566664@hamburg.muc.intelliad.de> ssl=0 24-Aug-2017 11:24:24.102409 hamburg.muc.intelliad.de (50): cram-md5.c:94-703 Authenticate OK a4tSr4AoY4+Ty+/2hg+TgD 24-Aug-2017 11:24:24.103886 hamburg.muc.intelliad.de (100): dir_cmd.c:495-703 <dird: backup FileIndex=0 24-Aug-2017 11:24:24.103922 hamburg.muc.intelliad.de (100): dir_cmd.c:506-703 Executing backup command. 24-Aug-2017 11:24:24.103936 hamburg.muc.intelliad.de (100): dir_cmd.c:1944-703 JobFiles=0 24-Aug-2017 11:24:24.103949 hamburg.muc.intelliad.de (100): dir_cmd.c:1974-703 begin backup ff=7f1dc80017e8 24-Aug-2017 11:24:24.228264 hamburg.muc.intelliad.de (50): find.c:162-703 Verify=<V> Accurate=<Cmcs> BaseJob=<Jspug5> flags=<-939517488> 24-Aug-2017 11:24:24.228680 hamburg.muc.intelliad.de (100): find_one.c:113-703 Accepting fstype btrfs for "/data" 24-Aug-2017 11:24:24.229034 hamburg.muc.intelliad.de (50): find_one.c:109-703 Cannot determine file system type for "/data/backups" 24-Aug-2017 11:24:24.229346 hamburg.muc.intelliad.de (50): find_one.c:109-703 Cannot determine file system type for "/data/intelliad" 24-Aug-2017 11:24:24.229643 hamburg.muc.intelliad.de (50): find_one.c:109-703 Cannot determine file system type for "/data/owncloud-old" 24-Aug-2017 11:24:24.229794 hamburg.muc.intelliad.de (100): heartbeat.c:151-703 Send kill to heartbeat id 24-Aug-2017 11:24:24.280008 hamburg.muc.intelliad.de (100): backup.c:183-703 end blast_data ok=1 24-Aug-2017 11:24:24.318303 hamburg.muc.intelliad.de (100): dir_cmd.c:509-703 Quit command loop. Canceled=0 24-Aug-2017 11:24:24.318520 hamburg.muc.intelliad.de (100): jcr.c:184-703 write_last_jobs seek to 192 24-Aug-2017 11:24:24.318575 hamburg.muc.intelliad.de (100): dir_cmd.c:568-0 Done with free_jcr |
|
|
btw. the client to backup is running Ubuntu xenial 16.04. |
|
|
24-Aug-2017 11:24:24.229034 hamburg.muc.intelliad.de (50): find_one.c:109-703 Cannot determine file system type for "/data/backups" 24-Aug-2017 11:24:24.229346 hamburg.muc.intelliad.de (50): find_one.c:109-703 Cannot determine file system type for "/data/intelliad" 24-Aug-2017 11:24:24.229643 hamburg.muc.intelliad.de (50): find_one.c:109-703 Cannot determine file system type for "/data/owncloud-old" FS Type can not be determined because of a not mounted subvolume. Solution: Drop all FS Types from your Fileset and leave OneFS value to No. Is there any reason why your subvolumes are not explictly mounted? |
|
|
Because this is how btrfs works? https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-subvolume -- A subvolume in btrfs can be accessed in two ways: * like any other directory that is accessible to the user * like a separately mounted filesystem (options subvol or subvolid) -- As a workaround im going to use onefs=no. Anyway, how does bareos tries to resolve this directories to an file system? Should this be fixed? If not, it would be helpful to mention this in the documentation, because this is not an obvious behavior. |
|
|
> Anyway, how does bareos tries to resolve this directories to an file system? Should this be fixed? We will discuss that. > If not, it would be helpful to mention this in the documentation, because this is not an obvious behavior. Of course. |
|
|
This way of mounting subvolumes in btrfs is also the default on openSUSE. None of the system subvolume will be picked if oneFS = Yes Some like .snapshots need to be excluded of course, but boot efi related are important. btrfs subvol list -t / ID gen top level path -- --- --------- ---- 257 30366 5 @ 258 30366 257 @/.snapshots 259 32111 258 @/.snapshots/1/snapshot 260 30366 257 @/boot/grub2/i386-pc 261 30366 257 @/boot/grub2/x86_64-efi 262 30366 257 @/opt 263 30366 257 @/srv 264 30366 257 @/usr/local 265 32111 257 @/var/cache 266 30366 257 @/var/crash 267 30366 257 @/var/lib/libvirt/images 268 30366 257 @/var/lib/machines 269 30366 257 @/var/lib/mailman 270 30366 257 @/var/lib/mariadb 271 30366 257 @/var/lib/mysql 272 30366 257 @/var/lib/named 273 32116 257 @/var/lib/pgsql 274 32111 257 @/var/log 275 30366 257 @/var/opt 276 32116 257 @/var/spool 277 32110 257 @/var/tmp 281 30366 258 @/.snapshots/2/snapshot 282 30366 258 @/.snapshots/3/snapshot 283 30366 258 @/.snapshots/4/snapshot 287 30366 258 @/.snapshots/5/snapshot 289 30366 258 @/.snapshots/6/snapshot 446 30366 258 @/.snapshots/34/snapshot 447 30366 258 @/.snapshots/35/snapshot 480 30366 258 @/.snapshots/36/snapshot 481 30366 258 @/.snapshots/37/snapshot 482 30366 258 @/.snapshots/38/snapshot 483 30366 258 @/.snapshots/39/snapshot 497 30366 258 @/.snapshots/40/snapshot 498 30366 258 @/.snapshots/41/snapshot 499 30366 258 @/.snapshots/42/snapshot 500 30366 258 @/.snapshots/43/snapshot |
|
|
With actual fileset FileSet { Name = "openSUSE-Full-btrfs" Include { Options { Signature = "XXH128" Compression = "LZ4" OneFS = No AclSupport = Yes XattrSupport = Yes Verify = "sm1" Accurate = "mcspug" Exclude = Yes WildDir = "/tmp" WildDir = "/var/tmp" WildDir = "/var/cache/man/*" WildDir = "/var/lib/mysql/*" WildDir = "/var/lib/pgsql/*" WildDir = "/var/lib/bareos*" WildDir = "/var/spool/*" WildDir = "/lost+found" WildDir = "/var/lib/libvirt/images/*" WildDir = ".Trash" WildDir = "Trash" WildDir = "/.snapshots" WildDir = "/media/*" WildDir = "/mnt/*" WildDir = "/ioda/*" WildFile = "/var/lib/bareos/*" WildBase = "core" WildBase = "*.tmp" FsType = "vfat" FsType = "btrfs" } File = "/" } } and a list of subvolumes btrfs subvol list -t / ID gen top level path -- --- --------- ---- 257 5563123 5 @ 258 5558373 257 boot/grub2/i386-pc 259 5561107 257 boot/grub2/x86_64-efi 261 5561116 257 opt 262 5561116 257 srv 264 5550782 257 usr/local 285 5563126 257 ioda 287 5563120 257 .snapshots 513 5561116 287 .snapshots/15/snapshot 5064 5563126 257 var 83532 5561116 287 .snapshots/30187/snapshot 88714 5561116 287 .snapshots/32490/snapshot 90392 5561116 287 .snapshots/332 The whole is took into account in backup and available for restore. |
|
|
Is this still reproducible with supported OS and Bareos version 22x? |
|
|
Please report with supported OS and Bareos version if still reproducible. |
- Anonymous
