Crash when running the test suite with Python 3.10 #210

Activity

1. mcepl commented on May 21, 2022

   

   mcepl

   on May 21, 2022

   Author

   More actions

   Or there is this backtrace ... not exactly the same: tests/test_xmlsec.py::TestModule::test_reinitialize_module passed and it failed only in tests/test_constants.py::test_transform_str: https://gist.github.com/mcepl/7c0cba8a3b41780dbb3e56ed31bbf299
2. dirkmueller commented on May 23, 2022

   

   dirkmueller

   on May 23, 2022

   Last edited by dirkmueller

   More actions

   I have similar one under valgrind:

   tests/test_constants.py ==15782== Invalid read of size 8
   ==15782==    at 0x969DFEF: PyXmlSec_Transform__str__ (constants.c:25)
   ==15782==    by 0x49AFEA3: PyObject_Str (in /usr/lib64/libpython3.10.so.1.0)
   ==15782==    by 0x49D1F1A: ??? (in /usr/lib64/libpython3.10.so.1.0)
   ==15782==    by 0x499C668: ??? (in /usr/lib64/libpython3.10.so.1.0)
   ==15782==    by 0x499C4E3: _PyObject_MakeTpCall (in /usr/lib64/libpython3.10.so.1.0)
   ==15782==    by 0x49990D1: _PyEval_EvalFrameDefault (in /usr/lib64/libpython3.10.so.1.0)
   ==15782==    by 0x4993672: ??? (in /usr/lib64/libpython3.10.so.1.0)
   ==15782==    by 0x499737E: _PyEval_EvalFrameDefault (in /usr/lib64/libpython3.10.so.1.0)
   ==15782==    by 0x4993672: ??? (in /usr/lib64/libpython3.10.so.1.0)
   ==15782==    by 0x49948B9: _PyEval_EvalFrameDefault (in /usr/lib64/libpython3.10.so.1.0)
   ==15782==    by 0x4993672: ??? (in /usr/lib64/libpython3.10.so.1.0)
   ==15782==    by 0x49A6A91: PyVectorcall_Call (in /usr/lib64/libpython3.10.so.1.0)
   ==15782==  Address 0x9963b50 is not stack'd, malloc'd or (recently) free'd
   ==15782== 
   Fatal Python error: Segmentation fault
   

   it appears this is due to

   https://github.com/mehcode/python-xmlsec/blob/2460f50a51ae3719cdab5dbeabc18b8ef8c71f52/src/constants.c#L25

   where id is suddenly pointing into the wild west. it looks like https://github.com/mehcode/python-xmlsec/blob/2460f50a51ae3719cdab5dbeabc18b8ef8c71f52/src/constants.c#L133 is the only place where id is assigned.

   I'm obviously not understanding things, but this is not doing a deep-copy, just copying the pointer. what exactly is supposed to keep the reference alive?

   shouldn't this be copied and freed then in __del__ ?
3. mcepl commented on Jun 2, 2022

   

   mcepl

   on Jun 2, 2022

   Author

   More actions

   Isn't it just that transform->id->href can be uninitialized? I have to admit that I got lost in the bizarre maze of macro hell in src/constants.c.
4. dirkmueller commented on Jun 8, 2022

   

   dirkmueller

   on Jun 8, 2022

   More actions

   I don't see how it can be uninitialized, it is unconditionally initialized in the constructing function?
5. mcepl commented on Jun 8, 2022

   

   mcepl

   on Jun 8, 2022

   Last edited by mcepl

   Author

   More actions

   s/uninitialized/has not been initialized/ I had lost my way in the endless bogs of src/constants.c, but I am just not persuaded still that there are no situations where transform->id->href has not been set at all.
6. danigm commented on Dec 19, 2022

   

   danigm

   on Dec 19, 2022

   More actions

   Debugging this problem I've found that the problem is when trying to access transform->id->href, but it's even not null, so it should be something deeper.

   25	    if (transform->id->href != NULL)
   (gdb) p transform
   $1 = (PyXmlSec_Transform *) 0x7ffff5704230
   (gdb) p transform->id
   $2 = (xmlSecTransformId) 0x7ffff5d13220
   (gdb) p transform->id->href
   Cannot access memory at address 0x7ffff5d13230
   

   I've also detected that this problem doesn't happen to me if I run just the test_constants but it happens when running the full test suite, so it should be some weird state.
7. danigm commented on Dec 19, 2022

   

   danigm

   on Dec 19, 2022

   More actions

   Looks like the problem is with test_reinitialize_module, the call to shutdown and a race condition or something like that. I've detected that running the tests with pytest -n 1 doesn't produce this problem.
8. bmwiedemann

   added a commit that references this issue on Dec 19, 2022

   Update python-xmlsec to version 1.3.13 / rev 8 via SR 1043746

   ...

   d837a4d