Failure accessing from different networks - PyO3 modules may only be initialized once per interpreter process

[EchedelleLR]

Top-level intent

Accessing privacyIDEA webui from different networks to which Apache HTTP Server has access to.

Steps to reproduce

1. Install privacyIDEA 3.9.2 on a server with either Python 3.9 or 3.11
2. Configure Apache HTTP Server with wsgi module against the wsgi app of privacyIDEA
3. Start server
4. Make first request on a network where Apache HTTP Server is listening to for privacyIDEA site
5. Make a second request from other network where Apache HTTP Server is listening to for privacyIDEA site

Expected outcome

Both requests should have been successful

Actual outcome

Second request failes with 500 server error on Apache HTTP Server side

Configuration

• privacyIDEA version: 3.9.2
• Installation method: PyPI + exact requirements for that version from GitHub
• Python version: 3.9 and 3.11
• Operating system: Debian 11 and Debian 12
• Webserver: Apache HTTP Server
• Token database: MariaDB

Log file

[Wed Feb 28 10:28:21.291571 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044] Traceback (most recent call last):
[Wed Feb 28 10:28:21.291639 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]   File "/etc/privacyidea/privacyideaapp.wsgi", line 3, in <module>
[Wed Feb 28 10:28:21.291650 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]     from privacyidea.app import create_app
[Wed Feb 28 10:28:21.291664 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]   File "/srv/privacyidea/lib/python3.9/site-packages/privacyidea/app.py", line 34, in <module>
[Wed Feb 28 10:28:21.291672 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]     import privacyidea.api.before_after
[Wed Feb 28 10:28:21.291686 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]   File "/srv/privacyidea/lib/python3.9/site-packages/privacyidea/api/before_after.py", line 30, in <module>
[Wed Feb 28 10:28:21.291694 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]     from .lib.utils import (send_error, get_all_params)
[Wed Feb 28 10:28:21.291706 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]   File "/srv/privacyidea/lib/python3.9/site-packages/privacyidea/api/lib/utils.py", line 36, in <module>
[Wed Feb 28 10:28:21.291714 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]     import jwt
[Wed Feb 28 10:28:21.291726 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]   File "/srv/privacyidea/lib/python3.9/site-packages/jwt/__init__.py", line 1, in <module>
[Wed Feb 28 10:28:21.291734 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]     from .api_jwk import PyJWK, PyJWKSet
[Wed Feb 28 10:28:21.291746 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]   File "/srv/privacyidea/lib/python3.9/site-packages/jwt/api_jwk.py", line 6, in <module>
[Wed Feb 28 10:28:21.291754 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]     from .algorithms import get_default_algorithms
[Wed Feb 28 10:28:21.291767 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]   File "/srv/privacyidea/lib/python3.9/site-packages/jwt/algorithms.py", line 6, in <module>
[Wed Feb 28 10:28:21.291775 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]     from .utils import (
[Wed Feb 28 10:28:21.291802 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]   File "/srv/privacyidea/lib/python3.9/site-packages/jwt/utils.py", line 7, in <module>
[Wed Feb 28 10:28:21.291811 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]     from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurve
[Wed Feb 28 10:28:21.291825 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]   File "/srv/privacyidea/lib/python3.9/site-packages/cryptography/hazmat/primitives/asymmetric/ec.py", line 11, in <module>
[Wed Feb 28 10:28:21.291833 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]     from cryptography.hazmat._oid import ObjectIdentifier
[Wed Feb 28 10:28:21.291846 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]   File "/srv/privacyidea/lib/python3.9/site-packages/cryptography/hazmat/_oid.py", line 9, in <module>
[Wed Feb 28 10:28:21.291854 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044]     from cryptography.hazmat.bindings._rust import (
[Wed Feb 28 10:28:21.291882 2024] [wsgi:error] [pid 159:tid 139807042434816] [remote 172.24.0.1:53044] ImportError: PyO3 modules may only be initialized once per interpreter process

Additional notes

This error does not exist in a previous test using privacyIDEA 3.8.x. Initially I thought that was the Python version as it was tested with 3.11 which is not yet supported but this also failed with 3.9 which should be supported properly so I created another environment using Debian 11 which ships Python 3.9 by default and could reproduce the error.

[github-actions]

Thank you for filing an issue and sharing your observations or ideas. Please be sure to provide as much information as possible to help us to work on this issue.

[EchedelleLR]

I discovered the issue, I think at https://github.com/privacyidea/privacyidea/blob/master/deploy/apache/sites-available/privacyidea.conf and comparing it to the https://privacyidea.readthedocs.io/en/latest/installation/system/wsgiscript.html#wsgi-configuration-for-the-apache-webserver minimal settings.

The WSGIApplicationGroup %{GLOBAL} is absent, making the server run spare servers without link to each other and resulting in that error.

[plettich]

Ah Yes, we should remove that "deploy" directory or at least consolidate it with the documentation and the package build scripts.

[EchedelleLR]

That deploy directory is good because it shows examples of the whole configuration and gives a global vision.

If you integrate it into the documentation, it should expand the current section (in Apache HTTP Server as example) by a lot.