bareos: master fa9aabd5

Diff Back to Repository
Author Committer Branch Timestamp Parent
mvwieringen mvwieringen master 2013-12-09 15:12 master 3901288d Pending
Affected Issues  0000008: Handle Windows Encrypted Files using Win raw encryption
 0000228: Some windows file attributes are not handled.
Changeset Bring windows support up to Windows 2012 server.

This commit bring the windows code to support all new
features that have been added to windows up till windows
2012 server. We also reindented a lot of code along the
way to be more readable and fixes most comments.

The actual added features:
 - Volume Mountpoints support (VMP).

   Windows for some version now supports so called Volume
   MountPoint which allow you to mount a volume somewhere
   in the filesystem layout analog to what UNIX has been
   doing since it inception without assigning a drive letter.
   We need to make sure we also add the VMPs to the VSS snapshot
   list otherwise the backup will fail.

 - Use GetFileInformationByHandleEx to get real changetime.

   As it seems ctime in the stat struct on UNIX means for
   some time now the last change time of the inode and not
   the creation time. On Windows we need a special API to
   get the actual change time named GetFileInformationByHandleEx().
   While implementing this new feature we reformated some of
   the layout and dropped some unused code.

 - Windows dedup support

   Windows 2012 has dedup support which needs handling.

   To get rehydrated data when reading the file you should open
   the file without the FILE_FLAG_OPEN_REPARSE_POINT flag set.
   If you want to read the deduped data effeciently you need to
   write new code which interfaces to a new API which is currently
   out of scope. The original file on the filesystem is replaced
   with a so called reparse point so if we open the reparse point
   instead of where it points to we get data which has a pointer
   to a map of all the data streams and chunks required to
   â€œrehydrate” the file and not its actual data.

   We now changed the stat emulation functions to detect the special
   reparse point with type IO_REPARSE_TAG_DEDUP which indicates
   a deduped file.

   When that is set we set a special bit on the file so we know we
   need to open the file with the O_NOFOLLOW flag which will open
   the file without the FILE_FLAG_OPEN_REPARSE_POINT flag set.

   More info at:

   http://blogs.technet.com/b/filecab/archive/2012/05/21/introduction-to-data-deduplication-in-windows-server-2012.aspx

- Store all file attributes

   Windows has gathered quite some special specific file
   flags over the years but not all are saved during backup so
   some are never restored by the restore process. The most
   important ones are the ARCHIVE flag which is "misused" by
   some programs for storing some special information. Others
   that are known not to be stored are the COMPRESSED flag which
   means that a restored file looses it and will be restored as
   an uncompressed file.

   As we are running out of opportunities to store the special
   flags as some mode in the st_mode member of the stat struct
   a redesign was needed to store the unique file flag somewhere
   else. We are reusing the st_rdev field. That is already used for
   reparse points in older versions but as those are never restored
   anyway we can reuse them without breaking backwards compatability.

- Support for Windows EFS filesystems

  Windows has for quite some time now support for a so called EFS
  filesystem. This is an encrypted filesystem, to be able to backup the
  data and to restore it we need to use a special API. With this API you
  in essence export the data on backup and import it on restore. This
  way you never have access to the unencrypted data but just import and
  export the encrypted data. This is the cleanest way of handling
  encryption by just seeing the data as some opaque data and not try to
  do anything special with it. The problem with the restore API is
  however that it kind of works in a completely different way then a
  normal read/write API. You call a import function and that call the
  read callback multiple times until all data is imported/restored. As
  we don't want to rewrite the whole internal works of the filed for
  this we now create a seperate thread that runs during the whole
  restore of a filesystem and behaves as a consumer/producer problem
  (well known in the pthreads world). In this solution the producer is
  the filed that produces encrypted data out of the incoming data
  streams from the storage daemon and the seperate thread consumes that
  data by returning it as encrypted data in the callback function issues
  by the special EFS import API. There is some synchronization between
  the different threads so it knows when one file is done and it should
  end the import of that file. We also keep some of the buffers
  lingering around so we don't need to do malloc/free/malloc loops all
  the time, we cleanup the whole thread and the buffers used at the end
  of the whole restore process.

Fixes 0000228: Some windows file attributes are not handled.
Fixes 0000008: Handle Windows Encrypted Files using Win raw encryption

Signed-off-by: Marco van Wieringen <marco.van.wieringen@bareos.com>
mod - src/filed/backup.c Diff File
mod - src/filed/crypto.c Diff File
mod - src/filed/dir_cmd.c Diff File
mod - src/filed/fd_plugins.c Diff File
mod - src/filed/filed.c Diff File
mod - src/filed/fileset.c Diff File
mod - src/filed/protos.h Diff File
mod - src/filed/restore.c Diff File
mod - src/filed/verify.c Diff File
mod - src/findlib/attribs.c Diff File
mod - src/findlib/bfile.c Diff File
mod - src/findlib/bfile.h Diff File
mod - src/findlib/create_file.c Diff File
mod - src/findlib/find.c Diff File
mod - src/findlib/find.h Diff File
mod - src/findlib/find_one.c Diff File
mod - src/findlib/mkpath.c Diff File
mod - src/findlib/protos.h Diff File
mod - src/include/baconfig.h Diff File
mod - src/include/bareos.h Diff File
mod - src/include/bc_types.h Diff File
mod - src/include/jcr.h Diff File
mod - src/lib/Makefile.in Diff File
add - src/lib/cbuf.c Diff File
add - src/lib/cbuf.h Diff File
mod - src/plugins/filed/fd_common.h Diff File
mod - src/win32/Makefile Diff File
mod - src/win32/Makefile.inc Diff File
mod - src/win32/compat/compat.c Diff File
mod - src/win32/compat/include/compat.h Diff File
mod - src/win32/compat/include/mingwconfig.h Diff File
mod - src/win32/compat/print.c Diff File
mod - src/win32/compat/winapi.c Diff File
mod - src/win32/console/Makefile Diff File
mod - src/win32/filed/Makefile Diff File
mod - src/win32/filed/vss.c Diff File
mod - src/win32/filed/vss_generic.c Diff File
mod - src/win32/findlib/Makefile Diff File
add - src/win32/findlib/win32.c Diff File
mod - src/win32/include/vss.h Diff File
mod - src/win32/include/winapi.h Diff File
mod - src/win32/lib/Makefile Diff File
mod - src/win32/plugins/filed/Makefile Diff File