View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0001155bareos-coreGeneralpublic2019-12-13 09:312024-03-20 13:54
Reporterbigz Assigned Tojoergs  
PriorityhighSeverityfeatureReproducibilityalways
Status resolvedResolutionfixed 
PlatformLinuxOSanyOS Version3
Product Version19.2.4~pre 
Summary0001155: Impossible to connect to a TLS with no PSK configured director
DescriptionI configured my director with TLS in a centos7 docker image.

I want to connect with python-bareos pip module on order to send a command. The python client does not support TLS configuration without PSK.

I think python client does not support this configuration. I do enhancement in my fork github repo (https://github.com/bigzbigz/bareos/tree/dev/bigz/master/python-support-tls-without-psk)

I plan to push a pull request on the officiel repo in order to fix the problem. I need your opinion before.


Steps To ReproduceI work in a venv

-> % pip install sslpsk python-bareos
[...]
-> % pip list
Package Version Location
--------------- ------- --------------------------------------------
pip 19.3.1
pkg-resources 0.0.0
python-bareos 18.2.5
python-dateutil 2.8.1
setuptools 42.0.2
six 1.13.0
sslpsk 1.0.0
wheel 0.33.6

I try with TLS-PSK require

-> % python bconsole.py -d --name bareos-dir --port 9101 --address bareos-dir -p $PASS --tls-psk-require
DEBUG bconsole.<module>: options: {'name': 'bareos-dir', 'password': 'xxxxxxxx', 'port': '9101', 'address': 'bareos-dir', 'protocolversion': 2, 'tls_psk_require': True}
DEBUG lowlevel.__init__: init
DEBUG lowlevel.__connect_plain: connected to bareos-dir:9101
DEBUG lowlevel.__connect_tls_psk: identity = R_CONSOLEbareos-dir, password = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Traceback (most recent call last):
  File "bconsole.py", line 28, in <module>
    director = bareos.bsock.DirectorConsole(**bareos_args)
  File "/home/user/Downloads/bareos/python-bareos/bareos/bsock/directorconsole.py", line 99, in __init__
    self.connect(address, port, dirname, ConnectionType.DIRECTOR, name, password)
  File "/home/user/Downloads/bareos/python-bareos/bareos/bsock/lowlevel.py", line 104, in connect
    return self.__connect()
  File "/home/user/Downloads/bareos/python-bareos/bareos/bsock/lowlevel.py", line 119, in __connect
    self.__connect_tls_psk()
  File "/home/user/Downloads/bareos/python-bareos/bareos/bsock/lowlevel.py", line 191, in __connect_tls_psk
    server_side=False)
  File "/home/user/.virtualenvs/bareos/lib/python3.7/site-packages/sslpsk/sslpsk.py", line 106, in wrap_socket
    _ssl_set_psk_client_callback(sock, cb)
  File "/home/user/.virtualenvs/bareos/lib/python3.7/site-packages/sslpsk/sslpsk.py", line 73, in _ssl_set_psk_client_callback
    ssl_id = _sslpsk.sslpsk_set_psk_client_callback(_sslobj(sock))
  File "/home/user/.virtualenvs/bareos/lib/python3.7/site-packages/sslpsk/sslpsk.py", line 55, in _sslobj
    return sock._sslobj._sslobj
AttributeError: '_ssl._SSLSocket' object has no attribute '_sslobj'

I try with no TLS-PSK require (default configuration)

-> % python bconsole.py -d --name bareos-dir --port 9101 --address bareos-dir -p $PASS
/home/user/Downloads/bareos/python-bareos/bareos/bsock/lowlevel.py:38: UserWarning: Connection encryption via TLS-PSK is not available, as the module sslpsk is not installed.
  warnings.warn(u'Connection encryption via TLS-PSK is not available, as the module sslpsk is not installed.')
DEBUG bconsole.<module>: options: {'name': 'bareos-dir', 'password': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx', 'port': '9101', 'address': 'bareos-dir', 'protocolversion': 2, 'tls_psk_require': False}
DEBUG lowlevel.__init__: init
DEBUG lowlevel.__connect_plain: connected to bareos-dir:9101
DEBUG lowlevel.__connect: Encryption: None
DEBUG lowlevel.send: bytearray(b'Hello bareos-dir calling version 18.2.5')
DEBUG lowlevel.recv_bytes: expecting 4 bytes.
DEBUG lowlevel.recv: header: -4
WARNING lowlevel._handleSocketError: socket error: Conversation terminated (-4)
Received unexcepted signal: Conversation terminated (-4)
Additional InformationDirector configuration:
Director {
  Name = @@DIR_NAME@@-dir
  DIRport = 9101 # where we listen for UA connections
  QueryFile = "/usr/lib/bareos/scripts/query.sql"
  WorkingDirectory = "/var/spool/bareos"
  PidDirectory = "/var/run"
  Password = "@@DIR_PASSWORD@@" # Console password
  Messages = Daemon
  Auditing = yes
  TLS Enable = yes
  TLS Require = yes
  TLS DH File = /etc/ssl/dh1024.pem
  TLS CA Certificate File = /etc/ssl/certs/ca-bundle.crt
  TLS Key = /etc/ssl/private/client.key
  TLS Certificate = /etc/ssl/certs/client.pem
}
TagsNo tags attached.

Activities

arogge

arogge

2019-12-13 10:46

manager   ~0003663

Thanks for taking a look.
I'm not sure I already understand what happens in your environment. However, if you want to touch the code, you should probably checkout the master branch and use python-bareos from there.
bigz

bigz

2019-12-13 10:57

reporter   ~0003664

I verify the error with the code from the master branch in python-bareos folder.
joergs

joergs

2019-12-13 11:39

developer   ~0003665

it is correct, that python-bareos does not support TLS other then TLS-PSK.
My assumption has been, that most new installations will use TLS-PSK. However, a patch to also support normal TLS without PSK is welcome.
I took a first look at your code. It looks good so far.
However, if I read it correctly, you allow TLS, but don't verify against a custom CA? Have I missed something there or is it your intention to plainly accepting all TLS connections?

Have you seen the systemtest testing the python-bareos authentication?
See https://docs.bareos.org/master/DeveloperGuide/BuildAndTestBareos.html#building-the-test-environment

Instead of running all tests, you can also change to the build/systemtests/tests/python-bareos-test directory and run "./testrunner" from there.
This way you can verify, that your change to not change existing behavior and maybe you can add extra tests for your functionality.

With what version of Python have you tested? I experienced difficulties with the Python3 version of sslpsk. What OS/distribution did you use, as at least on new Fedora (>=30) systems there are also compile problems with sslpsk?

Currently, we use tls_psk_enable and tls_psk_require parameter. You added tls_enable and tls_require. I'm not sure, if this is the best way to configure it, especially, if more parameter as CA are required. I'll discuss about this in our next developer meeting.
bigz

bigz

2019-12-13 12:06

reporter   ~0003666

You can use a custom CA (this is my configuration). The use of ssl.wrap_socket automatically check the CA you installed in the operating system (normally in /etc/ssl/certs). It is possible to load an extra CA chain

I don't see the systemtest. I use my travis-ci account to check the existing CI from the official repo.
I will think about if a new test is possible to verify my enhancement.

I use Python 3.7.3 version. My OS is Ubuntu 19.04 and I use official python package. Modules are installed with a virtualenv with pip command.
bigz

bigz

2019-12-14 22:01

reporter   ~0003668

Last edited: 2019-12-14 22:30

 Hello joergs
I have difficulties to build bareos project with cmake like you explain in your note. I think I have dependencies missing but I don't find which one is missing. I installed libacl1-dev and zlib1g-dev on my ubuntu19.04. Do you have the list of dependencies packages is needed ?
When I use this command I have this error.
-> % cmake -Dsqlite3=yes -Dtraymonitor=yes ../bareos
[...]
-- Disabled test: system:bconsole-pam
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
Readline_INCLUDE_DIR (ADVANCED)
   used as include directory in directory /home/user/Perso/clion/bareos/core/src/console
   used as include directory in directory /home/user/Perso/clion/bareos/core/src/console
   used as include directory in directory /home/user/Perso/clion/bareos/core/src/console
   used as include directory in directory /home/user/Perso/clion/bareos/core/src/console
   used as include directory in directory /home/user/Perso/clion/bareos/core/src/console
   used as include directory in directory /home/user/Perso/clion/bareos/core/src/console
   used as include directory in directory /home/user/Perso/clion/bareos/core/src/console
   used as include directory in directory /home/user/Perso/clion/bareos/core/src/console
   used as include directory in directory /home/user/Perso/clion/bareos/core/src/console

I don't understand the error
Thanks
bigz

bigz

2019-12-14 23:40

reporter   ~0003669

I passed to build the project...I continue investigation but my previous errors are solved
joergs

joergs

2019-12-15 09:48

developer   ~0003670

Good that you passed the build process. You find the dependency packages in the files we use to create Debian packages: https://github.com/bareos/bareos/blob/master/core/platforms/packaging/bareos.dsc and/or https://github.com/bareos/bareos/blob/master/core/debian/control (or http://download.bareos.org/bareos/experimental/nightly/xUbuntu_18.04/bareos_19.2.4*.dsc). Make sure to have libjansson-dev installed, otherwise, Bareos will build but misses functionality required for the test.
bigz

bigz

2019-12-15 14:51

reporter   ~0003671

Hello,
I have a small error in ./testrunner

-> % /bin/zsh /home/user/Perso/clion/bareos/cmake-build-release/systemtests/tests/python-bareos-test/testrunner [devel|…]
creating database (sqlite3)
running /home/user/Perso/clion/bareos/cmake-build-release/systemtests/scripts/setup
 
 
=== python-bareos-test: starting at 14:46:34 ===
=
=
exit(0) is called. Set test to failure and end test.
end_test:7: no matches found: /home/user/Perso/clion/bareos/cmake-build-release/systemtests/tests/python-bareos-test/working/bareos.*.traceback
end_test:8: no matches found: /home/user/Perso/clion/bareos/cmake-build-release/systemtests/tests/python-bareos-test/working/bareos.*.traceback
end_test:9: no matches found: /home/user/Perso/clion/bareos/cmake-build-release/systemtests/tests/python-bareos-test/working/*.bactrace
end_test:10: no matches found: /home/user/Perso/clion/bareos/cmake-build-release/systemtests/tests/python-bareos-test/working/*.bactrace
 
  !!!!! python-bareos-test failed!!! 14:46:34 !!!!!
   Status: estat=998 zombie=0 backup=0 restore=0 diff=0
 
I think I don't understand the behavior of the start_test() function in functions. A trap is added at the beginning of the function and the trap is always taken at the end of this start_test() function as a consequence end_test() is called and no tests are done. Is it a desired bahavior ?
joergs

joergs

2019-12-15 19:26

developer   ~0003672

Interesting. However, this problem does only occur when using zsh. It seams, that you are the first who ever tried it with it. Normally, we use bash (dash) or ksh. With these, the test runs as expected.
bigz

bigz

2019-12-16 20:22

reporter   ~0003675

Last edited: 2019-12-16 22:30

 Problem is solved and it comes with my zsh interpreter. I just change it with bash.
I already have a problem because I use default python3.7 version of my ubuntu OS. It seems to have a problem with sslpsk module and python version 3.7 (https://github.com/drbild/sslpsk/issues/11). I will try with python3.6 and I'll give you the answer.
bigz

bigz

2019-12-17 22:44

reporter   ~0003679

I change my python version to 3.6.5 in order to avoid sslpsk problem
Sorry but I already have errors when I execute ./testrunner from master branch. I upload console.log file with stdout.

Please could you watch and tell me what do you think about ? In my opinion problem comes from "WARNING lowlevel._handleSocketError: socket error: [SSL: ATTEMPT_TO_REUSE_SESSION_IN_DIFFERENT_CONTEXT] attempt to reuse session in different context (_ssl.c:833)". In this situation, the connection falls back in plain and the test fails.
I have no problem when I use builded bconsole -c bconsole-admin-tls.conf or bconsole -c bconsole-admin-notls.conf command. All of 2 are encrypted with TLS_CHACHA20_POLY1305_SHA256
I try to google ATTEMPT_TO_REUSE_SESSION_IN_DIFFERENT_CONTEXT but I don't find an interesting answer.
Maybe you have an opinion on the problem ?
As you send in your email, I will create a draft pull request tomorrow.
console.log (32,684 bytes)   
bigz

bigz

2019-12-18 22:48

reporter   ~0003705

Last edited: 2019-12-18 23:10

 Hello,
I work today and I rebase to the bareos master branch. I do not have the problem anymore. You have been doing commits in last few days but I don't understand how do you solve my problem

I did small fixes in python-bareos with a pull request in bareos github repo.

I already have error when I execute python unittests .Do you manage to perfom the unittest ? Do you send me a log file of the execution ?

Thanks
joergs

joergs

2019-12-19 12:55

developer   ~0003707

Hi, I accepted https://github.com/bareos/bareos/pull/382.

Have I understood you correctly, that connecting to a Director console without TLS-PSK, but with TLS by certificate does work now? I've not changed the behavior intentionally.

The systemtest also fails on my system when using Python 3. With Python 2 it works without problems. I assumed a general problem with sslpsk on Python 3, but after you saying, it works somehow in your environment, I assumend a local problem.
After your hint, I checked the project https://github.com/drbild/sslpsk again and saw, that the example code works on Python 3. I hope to find the time to check about this in more detail soon.
joergs

joergs

2019-12-20 17:22

developer   ~0003712

I'm not sure, what have changed, but the example and test code from https://github.com/drbild/sslpsk does no longer work on my machine.
bigz

bigz

2019-12-21 14:42

reporter   ~0003713

Hello,
It seems to don't work for him as well => https://travis-ci.org/drbild/sslpsk
b.braunger@syseleven.de

b.braunger@syseleven.de

2021-03-18 16:18

reporter   ~0004100

I have to hosts using python-baroes with sslpsk. One Ubuntu 14.04 with python 3.4 (works well) and one Ubuntu 20.04 with python 3.6 which throw this error:

ssl.SSLError: [SSL: ATTEMPT_TO_REUSE_SESSION_IN_DIFFERENT_CONTEXT] attempt to reuse session in different context (_ssl.c:852)

Am I getting that right, that sslpsk version 1.0.0 seems not to work on modern systems at all?

One can circumvent this problem by setting tls_psk_require=False and let the connection fallback to unencrypted mode.
joergs

joergs

2021-03-18 16:25

developer   ~0004101

Yes, this is correct. Another way to circumvent this problem is described at https://docs.bareos.org/master/include/autogenerated/autosummary/python-bareos/bareos.bsock.html#transport-encryption-tls-psk

Basically is says, use the latest version from sslpsk master and set it ti TLSv1.2.
bigz

bigz

2021-03-19 22:41

reporter   ~0004102

I confirm joergs' words
bruno-at-bareos

bruno-at-bareos

2024-03-20 13:54

manager   ~0005846

Pr merged. tls-psk is native in python 3.13

Issue History

Date Modified Username Field Change
2019-12-13 09:31 bigz New Issue
2019-12-13 10:46 arogge Note Added: 0003663
2019-12-13 10:55 joergs Assigned To => joergs
2019-12-13 10:55 joergs Status new => assigned
2019-12-13 10:57 bigz Note Added: 0003664
2019-12-13 11:39 joergs Status assigned => feedback
2019-12-13 11:39 joergs Note Added: 0003665
2019-12-13 11:41 joergs Severity crash => feature
2019-12-13 11:41 joergs Product Version 18.2.5 => 19.2.4~pre
2019-12-13 12:06 bigz Note Added: 0003666
2019-12-13 12:06 bigz Status feedback => assigned
2019-12-14 22:01 bigz Note Added: 0003668
2019-12-14 22:30 bigz Note Edited: 0003668
2019-12-14 23:40 bigz Note Added: 0003669
2019-12-15 09:48 joergs Note Added: 0003670
2019-12-15 14:51 bigz Note Added: 0003671
2019-12-15 19:26 joergs Note Added: 0003672
2019-12-16 20:22 bigz Note Added: 0003675
2019-12-16 22:30 bigz Note Edited: 0003675
2019-12-17 22:44 bigz File Added: console.log
2019-12-17 22:44 bigz Note Added: 0003679
2019-12-18 22:48 bigz Note Added: 0003705
2019-12-18 23:10 bigz Note Edited: 0003705
2019-12-19 12:55 joergs Note Added: 0003707
2019-12-20 17:22 joergs Note Added: 0003712
2019-12-21 14:42 bigz Note Added: 0003713
2021-03-18 16:18 b.braunger@syseleven.de Note Added: 0004100
2021-03-18 16:25 joergs Note Added: 0004101
2021-03-19 22:41 bigz Note Added: 0004102
2024-03-20 13:54 bruno-at-bareos Status assigned => resolved
2024-03-20 13:54 bruno-at-bareos Resolution open => fixed
2024-03-20 13:54 bruno-at-bareos Note Added: 0005846