Repository navigation
“lxml & xmlsec libxml2 library version mismatch” error under uWSGI #320
Description
If libxml2-dev was installed when uWSGI was built, then importing xmlsec within uWSGI leads to an incorrect error xmlsec.InternalError: (-1, 'lxml & xmlsec libxml2 library version mismatch'). There’s no such error outside of uWSGI.
(I can work around this error with --no-binary=lxml --no-binary=xmlsec, but that wastes a lot more CI time and shouldn’t be necessary.)
Reproduction in a fresh container:
$ docker run --rm -it ubuntu:22.04
root@076333186bac:/# apt update
…
root@076333186bac:/# apt install -y gcc libxml2-dev python3-dev python3-venv pkg-config
…
root@076333186bac:/# python3 -m venv venv
root@076333186bac:/# . venv/bin/activate
(venv) root@076333186bac:/# pip install uWSGI xmlsec
…
Successfully installed lxml-5.2.1 uWSGI-2.0.25.1 xmlsec-1.3.14
(venv) root@076333186bac:/# cat > app.py <<EOF
import xmlsec
def application(env, start_response):
start_response("200 OK", [("Content-Type", "text/plain")])
return [b"Hello, world!\n"]
EOF
(venv) root@076333186bac:/# python -c 'import xmlsec'
(venv) root@076333186bac:/# uwsgi --http :9090 --wsgi-file app.py
*** Starting uWSGI 2.0.25.1 (64bit) on [Mon May 6 22:18:16 2024] ***
compiled with version: 11.4.0 on 06 May 2024 22:17:53
os: Linux-6.8.7 #1-NixOS SMP PREEMPT_DYNAMIC Wed Apr 17 09:23:43 UTC 2024
nodename: 076333186bac
machine: x86_64
clock source: unix
detected number of CPU cores: 12
current working directory: /
detected binary path: /venv/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
*** WARNING: you are running uWSGI without its master process manager ***
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uWSGI http bound on :9090 fd 4
spawned uWSGI http 1 (pid: 4203)
uwsgi socket 0 bound to TCP address 127.0.0.1:46691 (port auto-assigned) fd 3
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
Python version: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x55a23eea0bc0
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 72904 bytes (71 KB) for 1 cores
*** Operational MODE: single process ***
Traceback (most recent call last):
File "app.py", line 1, in <module>
import xmlsec
xmlsec.InternalError: (-1, 'lxml & xmlsec libxml2 library version mismatch')
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 4202, cores: 1)
Activity
andersk commentedon May 7, 2024
Some debugging information:
xmlParserVersion,LIBXML_VERSION, andLIBXML_COMPILED_VERSIONare the same inside and outside uWSGI.requirements: Drop the --no-binary for lxml and xmlsec.
requirements: Drop the --no-binary for lxml and xmlsec.
eliasmazur commentedon May 20, 2024
I'm getting this same error on a container that I just re-compiled and it was working fine. I use python3-saml and that's breaking with this error when I run the container.
Here is the log:
Here is my Dockerfile:
And this is the requirements.txt:
I'm stuck. Any help is appreciated.
Thanks
andres-holvi commentedon May 20, 2024
It seems like
lxml@5.2.1useslibxml2@2.12.6. However,libxml2@2.12.7has been released, andpython-xmlsec@1.3.14breaks because of this libxml2 version mismatch (?)Not sure who should be in charge of fixing this, but I suppose using libxml2@2.12.6 could work.
gregbrant2 commentedon May 20, 2024
@eliasmazur I have the same use case as you with python3-saml. Based on what @andersk said, I've restricted the package version of LXML like so
My build is working again now.
eliasmazur commentedon May 20, 2024
Thanks for the help. I realized I was restricting all packages except xmlsec. I restricted to an earlier version and it now works. Looks like python3-saml was installing the latest xmlsec and that was causing the mismatch.
mstuttgart commentedon May 24, 2024
In my work, we use sigxml and xmlsec. Both install lxml with binaries, causing the same error.
xmlsec.InternalError: (-1, 'lxml & xmlsec libxml2 library version mismatch')To resolve this, I install using the
--no-binarypip flag. (work on Ubuntu 22.04, Python 3.10 and Pip 22.0.2)buddhi-vamsi13 commentedon May 31, 2024
Hi,
we have the similiar issue can you provide us the version which u have mentioned
eliasmazur commentedon May 31, 2024
For the libxml in Dockerfile:
RUN apt-get install -y libxml2-dev=2.9.4+dfsg1-6.1ubuntu1.9 libxmlsec1-dev=1.2.25-1build1
For the lxml and xmlsec in requirements.txt:
lxml==4.9.3
xmlsec==1.3.13
Hope this helps.
sergei-maertens commentedon Jun 19, 2024
Would this mean that a new release/wheel is needed also built with libxml2@2.12.7? I was very excited about being able to install binary wheels again with xmlsec 1.3.14, but it seems that a patch release of libxml broke that again?
sergei-maertens commentedon Jun 19, 2024
Trying this, the problem is not solved for us:
still produces the uwsgi startup crash, while a normal python interpreter is fine (just as reported in the initial description). I have tried uwsgi 2.0.23 and 2.0.26 so far.
Making sure that
libxml2-devis NOT present in the system packages when uwsgi is being installed/built resolves the issue.🐳 Do not install libxml2-dev in docker image build
Remove lxml dev packages from CI and Dockerfile
Remove lxml dev packages from CI and Dockerfile
15 remaining items
set lxml version to avoid library version mismatch
chore: restrict "lxml" version
chore: restrict "lxml" version
chore: restrict "lxml" version
chore: restrict "lxml" version
fix: build both lxml and xmlsec from source to resolve libxml2 versio…
ShaheedHaque commentedon Jun 16, 2025
FWIW, Ubuntu 25.04 comes with:
Unsurprisingly, I found I was able to get away with just setting my venv with one pin on
lxml<5.4. The venv has python3-saml at 1.16.0.ttrei commentedon Jul 4, 2025
Encountered the "lxml & xmlsec libxml2 library version mismatch" error with following version combination (both installed using linux binary wheels from pypi.org):
Downgrading lxml resolved the issue:
josephsw commentedon Jul 21, 2025
FYI: Looks like this was fixed in xmlsec 1.3.16 (when used with lxml 6.0.0).https://github.com/xmlsec/python-xmlsec/releases/tag/1.3.16
Edit: not fixed for uWSGI, see below
andersk commentedon Jul 21, 2025
This is not fixed. The reproduction recipe from my original report still gives the same error with the current versions (uWSGI 2.0.30, xmlsec 1.3.16, lxml 6.0.0).
josephsw commentedon Jul 22, 2025
Following your repro steps, it looks like it installs
libxml2-dev arm64 2.9.13+dfsg-1ubuntu0.7at theapt install -y gcc libxml2-dev python3-dev python3-venvstep (and I can repro the error when running uwsgi following those steps).I'm guessing the issue stems from uWSGI also using libxml2, but it depending on your system version instead of some compiled wheel version.
I tried manually compiling the
libxml22.14.4 source (mentioned in the python-xmlsec 1.3.16 release notes), recompiling thelxmlandxmlsecpython dependencies against that (pip install --no-binary :all: --force-reinstall lxml xmlsecafter installingmesonand a bunch of dependencies) and then running the uwsgi server again, and the error no longer appears (curl -v http://127.0.0.1:9090/returnedHello, world!). So depending on what your setup looks like, recompiling against source may be an option (it's also noted in the python-xmlsec 1.3.16 release notes):andersk commentedon Jul 22, 2025
My original report explained why I don’t want to do that. It’s expensive.
phette23 commentedon Jul 22, 2025
I think @ josephsw is right. We are working around this by installing uWSGI with
UWSGI_PROFILE_OVERRIDE="xml=no"with compatible versions of python-xmlsec and lxml. uWSGI seems to use a different libxml2 and coerces xmlsec to use it too, while lxml sticks with its builtin. Or that's my theory.mxamin commentedon Jul 23, 2025
uWSGI preloads libxml2.so into the global symbol table, which interferes with Python modules like xmlsec that are statically linked to their own libxml2 copy. This happens because the dynamic linker prefers already-loaded symbols, causing xmlsec to bind to the wrong libxml2 at runtime. The issue aligns with standard ELF/RTLD_GLOBAL behavior. It’s difficult to fully isolate without dynamic linking or symbol exporting control, so for now, I’m planning to properly address it in #356
fix: build both lxml and xmlsec from source to resolve libxml2 versio…