View Issue Details

IDProjectCategoryView StatusLast Update
0001088bareos-core[All Projects] directorpublic2019-07-04 16:34
ReporterstephandAssigned To 
PrioritynormalSeveritymajorReproducibilityalways
Status acknowledgedResolutionopen 
Product Version 
Target VersionFixed in Version 
Summary0001088: Integer out of range when using large amounts of files with Base Jobs
DescriptionWhen using Base Jobs (for details see https://docs.bareos.org/TasksAndConcepts/FileDeduplicationUsingBaseJobs.html) with large
amounts of files, after a longer of operation without problems, the jobs which are using a Base Jobs start to fail.
The job error message contains "Fatal error:" but does not contain the underlying database error message, but in the PostgreSQL
database logs, an "integer out of range" error is logged for the following kind of SQL statement:
INSERT INTO BaseFiles (BaseJobId, JobId, FileId, FileIndex) SELECT B.JobId AS BaseJobId, 89793 AS JobId, B.FileId, B.FileIndex FROM basefile89793 AS A, new_basefile86703 AS B WHERE A.Path = B.Path AND A.Name = B.Name ORDER BY B.FileId
Steps To ReproduceTo reproduce this issue, a lot of jobs based on Base Jobs must be run with large amounts of files.
Additional InformationCREATE TABLE basefiles
(
    BaseId SERIAL NOT NULL,
    JobId INTEGER NOT NULL,
    FileId BIGINT NOT NULL,
    FileIndex INTEGER,
    BaseJobId INTEGER,
    PRIMARY KEY (BaseId)
);

and for MySQL/MariaDB it is:

CREATE TABLE BaseFiles (
   BaseId INTEGER UNSIGNED AUTO_INCREMENT,
   BaseJobId INTEGER UNSIGNED NOT NULL REFERENCES Job,
   JobId INTEGER UNSIGNED NOT NULL REFERENCES Job,
   FileId BIGINT UNSIGNED NOT NULL REFERENCES File,
   FileIndex INTEGER UNSIGNED,
   PRIMARY KEY(BaseId)
);

In both cases, SERIAL and INTEGER is a 4 byte integer.

To fix this issue, the data type must be changed to 8 byte integer (BIGSERIAL/BIGINT).

For PostgreSQL this can be done manually by:
ALTER TABLE basefiles ALTER COLUMN baseid SET DATA TYPE bigint;

However, this should be fixed properly in the corresponding DDL files.
TagsNo tags attached.
bareos-master: impact
bareos-master: action
bareos-18.2: impact
bareos-18.2: action
bareos-17.2: impact
bareos-17.2: action
bareos-16.2: impact
bareos-16.2: action
bareos-15.2: impact
bareos-15.2: action
bareos-14.2: impact
bareos-14.2: action
bareos-13.2: impact
bareos-13.2: action
bareos-12.4: impact
bareos-12.4: action

Activities

There are no notes attached to this issue.

Issue History

Date Modified Username Field Change
2019-06-05 16:14 stephand New Issue
2019-07-04 16:34 arogge Status new => acknowledged