View Issue Details

IDProjectCategoryView StatusLast Update
0001584bareos-corefile daemonpublic2024-01-16 15:11
Reportermdc Assigned Tobruno-at-bareos  
PrioritynormalSeverityminorReproducibilityalways
Status resolvedResolutionfixed 
Platformx86_64OSCentOSOS Versionstream 8
Summary0001584: Incremental PostgreSQL backup will log an warning on the database server
DescriptionUsing the new PostgreSQL plug-in of Bareos 23 will log an error on the database server when running an incremental backup.
Seen on RHEL7-9 on PostgreSQL 12 and 14. (Possible also on other)

warning on the data base server after the incremental one:
2023-12-19T10:32:44+0100 <HOSTNAME> postgres[63014]: [8-1] < 2023-12-19 10:32:44.100 CET >WARNING: aborting backup due to backend exiting before pg_stop_backup was called
Steps To ReproduceLet an full backup run following by an incremental one.
Tagsplugin, postgresql

Activities

bruno-at-bareos

bruno-at-bareos

2023-12-19 11:12

manager   ~0005648

The connection by pg8000 between the start and the end of the job need to be stable.

maybe you have something that kill the connection. none of our tests shows a trouble.
you may have to track the connection/disconnection in postgresql logs.
mdc

mdc

2023-12-19 11:26

reporter   ~0005649

On all servers, the file daemon and the database are on the same. And the connection are done by an socket.
Here the log of the data base with enabled connection logging:
2023-12-19T11:24:15+0100 postgres[2245125]: [13-1] < 2023-12-19 11:24:15.967 CET >LOG: connection received: host=[local]
2023-12-19T11:24:15+0100 postgres[2245125]: [14-1] < 2023-12-19 11:24:15.969 CET >LOG: connection authenticated: identity="root" method=peer (/var/lib/pgsql/14/data/pg_hba.conf:83)
2023-12-19T11:24:15+0100 postgres[2245125]: [15-1] < 2023-12-19 11:24:15.969 CET >LOG: connection authorized: user=root database=postgres
2023-12-19T11:24:17+0100 postgres[2245125]: [16-1] < 2023-12-19 11:24:17.309 CET >WARNING: aborting backup due to backend exiting before pg_stop_backup was called
2023-12-19T11:24:17+0100 postgres[2245125]: [17-1] < 2023-12-19 11:24:17.309 CET >LOG: disconnection: session time: 0:00:01.342 user=root database=postgres host=[local]
bruno-at-bareos

bruno-at-bareos

2023-12-19 15:52

manager   ~0005650

Would you mind to help us, by running at least one job with setdebug level=150 on the client.
A description how the PG cluster is setup and run, to maybe have a chance to reproduce this issue.

None of the systemtest in use actually has shown this error, on any platform tested.
mdc

mdc

2023-12-20 08:43

reporter   ~0005651

Yes of curse. Here are the debug output, I hope it will help.
But don't wonder, about the delay of the next answer, because today is my last day in the office until the "next year".
debug.log (65,056 bytes)
mdc

mdc

2023-12-20 08:44

reporter   ~0005652

The client was running via:
bareos-fd -d 150 -f 2>&1 >/tmp/debug.log
for this test.
bruno-at-bareos

bruno-at-bareos

2023-12-21 09:56

manager   ~0005659

We will need the fileset, and the corresponding log in the pg cluster to have a chance to understand what's happening there.
bruno-at-bareos

bruno-at-bareos

2024-01-03 14:45

manager   ~0005663

ok found the information in postgresql database log
bruno-at-bareos

bruno-at-bareos

2024-01-09 16:13

manager   ~0005678

PR created https://github.com/bareos/bareos/pull/1655
Packages to test will appear after build & test in https://download.bareos.org/experimental/PR-1655/
mdc

mdc

2024-01-10 07:12

reporter   ~0005680

Hi Bruno,
I have build an new 23 version with this patch and the warning are now gone.
Thanks
bruno-at-bareos

bruno-at-bareos

2024-01-10 10:44

manager   ~0005681

Thanks for the report, I'm still investigating some use case (especially incrementals without changes which may raise the job in W state).
Will fix that a bit later.
bruno-at-bareos

bruno-at-bareos

2024-01-16 15:11

manager   ~0005686

Fix committed to bareos master branch with changesetid 18537.

Related Changesets

bareos: master b5f2eb18

2024-01-09 14:12

bruno-at-bareos

Ported: N/A

Details Diff
plugins: postgresql fix missing pg_backup_stop() call

- pg_backup_stop() is called in all job level depending of the
`backup_running` status. This allow a better support of
incremental with no changes.
- add in testrunner-default checks of WARNING or ERROR in cluster
logs
- is_backup_running boolean true if pg_backup_start has been run
- is_backup_full boolean true if backup level is Full
- apply black formatting suggestion

Fixes 0001584: Incremental PostgreSQL backup will log an warning on
the database server

Signed-off-by: Bruno Friedmann <bruno.friedmann@bareos.com>
Affected Issues
0001584
mod - core/src/plugins/filed/python/postgresql/bareos-fd-postgresql.py Diff File
mod - systemtests/tests/py3plug-fd-postgresql/testrunner-default Diff File

Issue History

Date Modified Username Field Change
2023-12-19 10:44 mdc New Issue
2023-12-19 11:12 bruno-at-bareos Assigned To => bruno-at-bareos
2023-12-19 11:12 bruno-at-bareos Status new => feedback
2023-12-19 11:12 bruno-at-bareos Note Added: 0005648
2023-12-19 11:26 mdc Note Added: 0005649
2023-12-19 11:26 mdc Status feedback => assigned
2023-12-19 15:52 bruno-at-bareos Note Added: 0005650
2023-12-19 15:53 bruno-at-bareos Status assigned => feedback
2023-12-19 16:39 bruno-at-bareos Tag Attached: plugin
2023-12-19 16:39 bruno-at-bareos Tag Attached: postgresql
2023-12-20 08:43 mdc Note Added: 0005651
2023-12-20 08:43 mdc File Added: debug.log
2023-12-20 08:43 mdc Status feedback => assigned
2023-12-20 08:44 mdc Note Added: 0005652
2023-12-21 09:56 bruno-at-bareos Note Added: 0005659
2024-01-03 14:45 bruno-at-bareos Status assigned => confirmed
2024-01-03 14:45 bruno-at-bareos Note Added: 0005663
2024-01-09 16:13 bruno-at-bareos Status confirmed => assigned
2024-01-09 16:13 bruno-at-bareos Note Added: 0005678
2024-01-09 16:13 bruno-at-bareos Assigned To bruno-at-bareos =>
2024-01-09 16:14 bruno-at-bareos Assigned To => bruno-at-bareos
2024-01-10 07:12 mdc Note Added: 0005680
2024-01-10 10:44 bruno-at-bareos Note Added: 0005681
2024-01-16 15:11 bruno-at-bareos Changeset attached => bareos master b5f2eb18
2024-01-16 15:11 bruno-at-bareos Note Added: 0005686
2024-01-16 15:11 bruno-at-bareos Status assigned => resolved
2024-01-16 15:11 bruno-at-bareos Resolution open => fixed