View Issue Details

IDProjectCategoryView StatusLast Update
0000555bareos-corestorage daemonpublic2019-12-18 15:25
Reporterogagnon Assigned To 
PrioritynormalSeverityminorReproducibilityalways
Status closedResolutionfixed 
PlatformLinuxOSDebianOS Version7
Product Version15.4.0 
Summary0000555: Copy jobs don't respect "Spool Data = no"
DescriptionHi,
A "bug" with the Storage Daemon is upsetting me.
When a Job is done with Spooling Data enabled, whenever that Job is Copied to another SD, the Spooling option stays even if the Copy Job is set with the "Spool Data = no" parameter.

Spooling the data while doing on-File backup is normal. But after the Job is done and needs to be Copied to another SD for offsite backup, the Spooling is totally unnecessary, slow and unproductive since the Job data won't change.

What happens is that the Job data is transferred once at the offsite SD in the Spool Directory and when the transfer is over, it is copied back in the Volume, wasting a lot of time and hard drive IOPS.

Steps To Reproduce* Create a Backup Job with Spool Data enabled to a Pool with a NextPool Storage on another SD.
* Create a Copy Job for the Backup Job with the "Spool Data = no" parameter.
* Execute the Backup Job: data is spooled. When over, execute the Copy Job. The data is still spooled by the other SD.

Additional InformationCopy Job:
Job {
  Name = "CopyJob"
  Type = Copy
  Client=Primary-fd
  Schedule = "CopySchedule"
  FileSet = "None"
  Pool = File
  Storage = RAID
  Priority = 12
  Messages = Standard
  Allow Duplicate Jobs = yes
  Cancel Queued Duplicates = no
  Spool Data = no
  Spool Attributes = no
  Selection Type = SQLQuery
  Selection Pattern = "SELECT DISTINCT J.JobId, J.StartTime
         FROM Job J, Pool P
         WHERE P.Name = 'File'
           AND P.PoolId = J.PoolId
           AND J.Type = 'B'
           AND J.Level IN ('I', 'D')
           AND J.JobStatus IN ('T','W')
           AND J.jobBytes > 0
           AND J.StartTime > NOW( ) - INTERVAL 2 DAY
           AND J.JobId NOT IN
           (SELECT PriorJobId FROM Job WHERE Type IN ('B','C') AND Job.JobStatus IN ('T','W') AND PriorJobId != 0)
      ORDER BY J.StartTime;"
}


TagsNo tags attached.

Activities

mvwieringen

mvwieringen

2015-11-07 17:56

developer   ~0001952

I don't think this is really major so changed it to minor as its more
annoying then an enormous problem. How things currently work the Migrate/Copy
Job e.g. not the selection one more or less copies all info from the original
Job of which its a going to be a Copy or Migration. So in that sense its works
as designed. The set_jcr_defaults() function clones the original Job into a so
called mig_jcr which performs the Copy or Migrate and is controlled by an other
JCR which is called the cjcr (e.g. Controlling JCR).

You could try the following patch and see if that reliefs the problem.

diff --git a/src/dird/migrate.c b/src/dird/migrate.c
index a421d5c..f90fb87 100644
--- a/src/dird/migrate.c
+++ b/src/dird/migrate.c
@@ -1171,6 +1171,12 @@ bool do_migration_init(JCR *jcr)
        */
       mig_jcr->IgnoreDuplicateJobChecking = true;
 
+ /*
+ * Copy some overwrites back from the Control Job to the migration and copy job.
+ */
+ mig_jcr->spool_data = jcr->spool_data;
+ mig_jcr->spool_size = jcr->spool_size;
+
       if (!setup_job(mig_jcr, true)) {
          Jmsg(jcr, M_FATAL, 0, _("setup job failed.\n"));
          return false;
 
ogagnon

ogagnon

2015-11-07 20:16

reporter   ~0001953

Thanks Marco, I've patched the sources. I'm currently doing a Full Backup of the whole company data so I will get back with the results when it is over.

O.
mvwieringen

mvwieringen

2015-11-19 14:49

developer   ~0002005

Fix committed to bareos bareos-15.2 branch with changesetid 5894.

Related Changesets

bareos: bareos-15.2 3dff47a2

2015-11-17 16:33

mvwieringen

Ported: N/A

Details Diff
We shouldn't do spooling on copy/migrate jobs.

We clone the Migrate/Copy Job to be an clone of the original Job and as
such we also clone things like Spool Data and Spool Size. This patch
resets the cloned Job to use the Spool Data and Spool Size setting of
the actual Migrate/Copy control Job.

Fixes 0000555: Copy jobs don't respect "Spool Data = no"
Affected Issues
0000555
mod - src/dird/migrate.c Diff File

Issue History

Date Modified Username Field Change
2015-11-07 14:10 ogagnon New Issue
2015-11-07 17:56 mvwieringen Note Added: 0001952
2015-11-07 17:56 mvwieringen Severity major => minor
2015-11-07 17:56 mvwieringen Status new => feedback
2015-11-07 20:16 ogagnon Note Added: 0001953
2015-11-07 20:16 ogagnon Status feedback => new
2015-11-19 14:49 mvwieringen Changeset attached => bareos bareos-15.2 3dff47a2
2015-11-19 14:49 mvwieringen Note Added: 0002005
2015-11-19 14:49 mvwieringen Status new => resolved
2015-11-19 14:49 mvwieringen Resolution open => fixed
2019-12-18 15:25 arogge Status resolved => closed