View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0000865bareos-coredirectorpublic2017-10-17 13:362023-09-27 10:37
Reporterchaos_prevails Assigned Tobruno-at-bareos  
PrioritynormalSeverityminorReproducibilityalways
Status closedResolutionno change required 
PlatformLinux UbuntuOSServer x64OS Version16.04 latest
Product Version16.2.4 
Summary0000865: mailcommand not working with big job and too many files (timeout?) sendmail script works with short jobs
DescriptionA full backup job contains a couple of thousand files. I want to have a list of all files (to know if all directories are fine, verify some important files in some directories, ... ) sent to me by email after job completion.

This works if the number of files is relatively low, but stops working with a couple of thousand files.

There is nothing in /var/log/syslog, or /var/log/bareos.log (beside the successful job).

/var/log/bareos-audit.log shows the bconsole queries of the sendmail script (same queries as successful sent emails):
...
13-Oct 08:37 bareos-dir: Console [default] from [127.0.0.1] cmdline list joblog jobid=73 << job for which I don't receive email
13-Oct 08:37 bareos-dir: Console [default] from [127.0.0.1] cmdline list files jobid=73 << job for which I don't receive email
13-Oct 08:38 bareos-dir: Console [default] from [127.0.0.1] cmdline list joblog jobid=74 << job for which I receive email
13-Oct 08:38 bareos-dir: Console [default] from [127.0.0.1] cmdline list files jobid=74 << job for which I receive email
...

/var/log/mail does NOT show that the email is sent for the job in question (while it shows sent emails of other jobs, which I also receive in my mailbox)

Strangely enough, if I start a different small job (because testing receiving emails with a job taking 2 days is not so efficient), with the jobid of the big job manually added to the sendmail script, it works.

So it seems not to be the sendmail script, but rather a problem with a job which takes very long + a long list of files to be added to an email.

BTW: it works neither with sendemail nor with ssmtp (my mailserver needs smtp authentification)

Please find my configuration in the additional information
Steps To Reproduce1) have the list of files added to your email (e.g. as attachment) after job completion
2) start a job with a lot of files to backup
3) the email is not sent (no entry in /var/log/mail.log)
Additional Information/etc/bareos/bareos-dir.d/messages/Standard.conf

Messages {
  Name = Standard
  Description = "Reasonable message delivery -- send most everything to email address and to the console."
  operatorcommand = "/usr/local/bin/bareos-messages.sh \"[Operator] %d: %j Messages (jobid %i)\" %i \"%r\""
  mailcommand = "/usr/local/bin/bareos-messages.sh \"[SERVICE %e] %d: %t %n %l on %c (jobid %i)\" %i \"%r\""
  operator = it@XXXXX = mount # (0000003)
  mail = it@XXXXXX = all, !skipped, !saved, !audit # (0000002)
  console = all, !skipped, !saved, !audit
  append = "/var/log/bareos/bareos.log" = all, !skipped, !saved, !audit
  catalog = all, !skipped, !saved, !audit
}


/usr/local/bin/bareos-messages.sh

#!/bin/bash

SUBJECT=$1
#JOBID=$2
JOBID=73
RECIPIENTS=$3
ATTACHMENT_PATH=/tmp/
ATTACHMENT_FILE=bareosjobfilelist.gz

#if Operator or Daemon message
if [[ $SUBJECT == *"[Operator]"* || $SUBJECT == *"[Daemon]"* ]]; then

BODY=$(cat <<EOF
*Messages*
`echo messages | bconsole`
EOF
)

#send operator or daemon email
echo "$BODY" | mail -s "$SUBJECT" "$RECIPIENTS"

else #Service start

BODY=$(cat <<EOF
*Joblog*
`echo list joblog jobid=$JOBID | bconsole`
EOF
)
FILES=`echo list files jobid=$JOBID | bconsole`
FILESNO=`echo "$FILES" | wc -l`

if (( "$FILESNO" > 2000 )); then #start create attachment

echo "$FILES" | gzip > $ATTACHMENT_PATH$ATTACHMENT_FILE

#send email with attachment
(echo "$BODY"; uuencode $ATTACHMENT_PATH$ATTACHMENT_FILE $ATTACHMENT_FILE) | mail -s "$SUBJECT" "$RECIPIENTS"
rm `echo $ATTACHMENT_PATH$ATTACHMENT_FILE`

else # start add files to body

BODY+=$(cat <<EOF


*Files backed up*
`echo "$FILES"`
EOF
)

#send email without attachment
echo "$BODY" | mail -s "$SUBJECT" "$RECIPIENTS"

fi #end create attachment
fi #end Service

TagsNo tags attached.

Activities

Shodan

Shodan

2017-11-16 12:46

reporter   ~0002816

Similar problem for me, I'm using zabbix.sh script for zabbix monitoring

Messages {
  Name = Standard
  Mail Command = "/etc/bareos/zabbix/zabbix.sh %i %t %e %c %l"
  Mail = admin@local.com = all, !skipped
}

The code fragment below, verify job status code

sql="/usr/bin/mysql --defaults-file=$bareosDbPass -h$bareosDbAddr -P$bareosDbPort -u$bareosDbUser -D$bareosDbName -NB -e"
bareosJobStatus=$($sql "select JobStatus from Job where JobId=$bareosJobId;" 2>/dev/null)
if [ -z $bareosJobStatus ] ; then exit 13 ; fi
case $bareosJobStatus in
        "T") status=0 ;; (success)
        "W") status=1 ;; (warning)
        *) status=2 ;; (any other status)
esac

Bareos director and script are using the same remote mysql database

Catalog {
  Name = MyCatalog
  Reconnect = yes
  dbdriver = mysql
  dbname = bareos
  dbaddress = 10.1.1.1
  dbport = 3306
  dbuser = bareos
  dbpassword = "bareos"
}

All works perfectly for version 16, but after upgrade bareos to version 17RC, I've got a lot of zabbix alerts about failed backup jobs. The failed jobs has random order (aka race conditional)
After investigation, I've found that jobs have status "T" (success) and zabbix alerts are false positive, also zabbix.sh script got status "R" (running) for the same backup jobs.

My test
1) Run some lightweight backup job, job is is 1111 for example, bareos director and mysql database have no load
2) Job finished successfully with status OK, message appears in bconsole.
3) job with ID 1111 still has status "R" (running) in mysql database.
4) zabbix.sh script got status "R" for job id 1111 and exit with code 2 (failed)
5) After some time (1-2 seconds), job id 1111 changed status to "T" (success) in mysql database.
bruno-at-bareos

bruno-at-bareos

2023-07-27 15:16

manager   ~0005255

First of all to be sure, is this still the case with recent code like Bareos 22.1.0?
It look like your mailer in limiting in somewhat the number of line the message can contain.
bsmtp has also a -l parameter to help limiting the number of line.

Maybe if you need to check something, will it be more convenient to do the check on the bareos host (a runscript after with list file jobid=id output and only send the result.
chaos_prevails

chaos_prevails

2023-08-01 13:44

reporter   ~0005295

Hello Bruno,

thanks for following up, this was still the case for v21, we haven't upgraded yet to the latest version. I will post here again when we did the upgrade to see if we still experience the same problem.

kind regards,
bruno-at-bareos

bruno-at-bareos

2023-09-27 10:37

manager   ~0005449

as bsmtp has a parameter to setup the limit, there's nothing we can change here.
you can still use a post job script to send the joblog as attachment instead relying on simple tool as bsmtp.

Issue History

Date Modified Username Field Change
2017-10-17 13:36 chaos_prevails New Issue
2017-11-16 12:46 Shodan Note Added: 0002816
2023-07-27 15:16 bruno-at-bareos Assigned To => bruno-at-bareos
2023-07-27 15:16 bruno-at-bareos Status new => feedback
2023-07-27 15:16 bruno-at-bareos Note Added: 0005255
2023-08-01 13:44 chaos_prevails Note Added: 0005295
2023-08-01 13:44 chaos_prevails Status feedback => assigned
2023-09-27 10:37 bruno-at-bareos Status assigned => closed
2023-09-27 10:37 bruno-at-bareos Resolution open => no change required
2023-09-27 10:37 bruno-at-bareos Note Added: 0005449