You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add serialized XML interface to avoid libxml2 ABI conflicts with lxml #356
python-xmlsec currently relies on passing raw xmlNodePtr objects between lxml (which builds on libxml2) and xmlsec1 (which also uses libxml2). This creates a fragile situation where different versions of libxml2 may be loaded into the same process, leading to:
Segfaults or memory corruption due to incompatible struct layouts
Invalid memory free errors (e.g., double-free or mismatched allocators)
Signature verification failures caused by inconsistent parser state
Undefined behavior from mismatched libxml2 global configuration
This occurs because:
lxmlbundles its own libxml2 and libxslt (especially in binary wheels) to ease installation for users on Windows, macOS, and some Linux platforms.
python-xmlsec binds to xmlsec1, which in turn links to the system's libxml2.
Pointers like xmlNodePtr created by lxml are then passed to python-xmlsec functions like tree.find_node() or SignatureContext.sign().
If the libxml2 versions are not ABI-compatible, this can easily lead to crashes, unpredictable behavior, or memory corruption.
Proposed Solution: Decoupling via Canonicalized XML
Instead of passing xmlNodePtr from lxml to python-xmlsec, we should support passing serialized XML (as bytes), ideally using Canonical XML (C14N) where appropriate. This isolates the XML parsing and memory management between the two libraries.
Example Usage
fromlxmlimportetreeimportxmlsecdoc=etree.fromstring("<Root><Signature/></Root>")
c14n_bytes=etree.tostring(doc, method="c14n", exclusive=True)
# Proposed new API:signed_bytes=xmlsec.sign_serialized(c14n_bytes, key_file="key.pem")
# Parse back with lxml if neededsigned_doc=etree.fromstring(signed_bytes)
You could provide convenience APIs in pyxmlsec that call (and configure) the C14N parsing and serialisation of lxml in the background, so that users don't have to get the byte streams right on their own. Whenever users pass you an object for which etree.iselement(obj) returns true, you can call etree.tostring(…) to serialise it as C14N and then handle it as that. File input is equally easy to handle, I guess.
If I understand the c14n.c file in libxmlsec correctly, it uses C14N 1.x as provided by libxml2 and not C14N 2.0 (which lxml provides as well). Then method="c14n" should do the right thing in lxml's serialiser. But the C file also seems to provide several different C14N configurations, you'd probably have to match the behaviour here.
Activity
scoder commentedon Jul 11, 2025
You could provide convenience APIs in pyxmlsec that call (and configure) the C14N parsing and serialisation of lxml in the background, so that users don't have to get the byte streams right on their own. Whenever users pass you an object for which
etree.iselement(obj)returns true, you can calletree.tostring(…)to serialise it as C14N and then handle it as that. File input is equally easy to handle, I guess.If I understand the
c14n.cfile in libxmlsec correctly, it uses C14N 1.x as provided by libxml2 and not C14N 2.0 (which lxml provides as well). Thenmethod="c14n"should do the right thing in lxml's serialiser. But the C file also seems to provide several different C14N configurations, you'd probably have to match the behaviour here.mxamin commentedon Jul 16, 2025
@scoder Thanks for the input, I'll look into that.
issue-356: detect libxml2 mismatch between extension and runtime
Extract reusable round-trip helper for the #356 decoupling
Unify the shadow reflection engine and prune its API (#356)
Guard the shadow invariants with a source-level test (#356)
Reflect root replacement by morphing the live root in place (#356)
Match id attributes by local name under the shadow (#356)
Never evict a live id registration (#356)
Read code, not comments, in the shadow audit (#356)
Drop the source-level shadow audit (#356)
Replay id registrations at the node they were registered for (#356)
Keep entity references and adopted nodes straight under the shadow (#356
Move an attached encrypt_xml template, fingerprint text (#356)
Refuse a duplicate id at register_id under the shadow (#356)
Bound the copy's walks, load an external DTD subset (#356)
Tell two same-valued attributes of one element apart (#356)
Skip the external-DTD tests where nothing may load one (#356)
Copy a subtree its document lost as an unlinked node (#356)
Snapshot add_ids' scope, retire dead registrations (#356)
See the entity references holding a subtree alive (#356)