Skip to content

Crash for 3.13rc1 on Windows with free threading during initialize #122918

Closed
Closed
Crash for 3.13rc1 on Windows with free threading during initialize#122918
Labels
OS-windowstopic-free-threadingtype-crashA hard crash of the interpreter, possibly with a core dump

Description

@tonybaloney
tonybaloney
opened on Aug 11, 2024

Crash report

What happened?

I'm trying to use Python 3.13rc1 with free-threading and debugging enabled on Windows 11. On startup, Python crashes. I've got a stack trace of the issue, which is pointing to the ob_type being invalid for an object during the startup sequence.

When I run Python at the command line (for anything other than show version) it crashes immediately.

Exception thrown: read access violation.
v->**ob_type** was 0xFFFFFFFFFFFFFF8F.

Full stack trace

python313t_d.dll!binary_op1(_object * v, _object * w, const int op_slot, const char * op_name) Line 934
	at C:\Users\anthonyshaw\source\repos\cpython\Objects\abstract.c(934)
python313t_d.dll!PyNumber_Add(_object * v, _object * w) Line 1141
	at C:\Users\anthonyshaw\source\repos\cpython\Objects\abstract.c(1141)
python313t_d.dll!_PyEval_EvalFrameDefault(_ts * tstate, _PyInterpreterFrame * frame, int throwflag) Line 132
	at C:\Users\anthonyshaw\source\repos\cpython\Python\generated_cases.c.h(132)
python313t_d.dll!_PyEval_Vector(_ts * tstate, PyFunctionObject * func, _object * locals, _object * const * args, unsigned __int64 argcount, _object * kwnames) Line 1806
	at C:\Users\anthonyshaw\source\repos\cpython\Python\ceval.c(1806)
python313t_d.dll!PyEval_EvalCode(_object * co, _object * globals, _object * locals) Line 596
	at C:\Users\anthonyshaw\source\repos\cpython\Python\ceval.c(596)
python313t_d.dll!exec_code_in_module(_ts * tstate, _object * name, _object * module_dict, _object * code_object) Line 2617
	at C:\Users\anthonyshaw\source\repos\cpython\Python\import.c(2617)
python313t_d.dll!PyImport_ImportFrozenModuleObject(_object * name) Line 3048
	at C:\Users\anthonyshaw\source\repos\cpython\Python\import.c(3048)
python313t_d.dll!PyImport_ImportFrozenModule(const char * name) Line 3087
	at C:\Users\anthonyshaw\source\repos\cpython\Python\import.c(3087)
python313t_d.dll!init_importlib(_ts * tstate, _object * sysmod) Line 3167
	at C:\Users\anthonyshaw\source\repos\cpython\Python\import.c(3167)
python313t_d.dll!_PyImport_InitCore(_ts * tstate, _object * sysmod, int importlib) Line 4039
	at C:\Users\anthonyshaw\source\repos\cpython\Python\import.c(4039)
python313t_d.dll!pycore_interp_init(_ts * tstate) Line 904
	at C:\Users\anthonyshaw\source\repos\cpython\Python\pylifecycle.c(904)
python313t_d.dll!pyinit_config(pyruntimestate * runtime, _ts * * tstate_p, const PyConfig * config) Line 933
	at C:\Users\anthonyshaw\source\repos\cpython\Python\pylifecycle.c(933)
python313t_d.dll!pyinit_core(pyruntimestate * runtime, const PyConfig * src_config, _ts * * tstate_p) Line 1096
	at C:\Users\anthonyshaw\source\repos\cpython\Python\pylifecycle.c(1096)
python313t_d.dll!Py_InitializeFromConfig(const PyConfig * config) Line 1398
	at C:\Users\anthonyshaw\source\repos\cpython\Python\pylifecycle.c(1398)
python313t_d.dll!Py_InitializeEx(int install_sigs) Line 1436
	at C:\Users\anthonyshaw\source\repos\cpython\Python\pylifecycle.c(1436)
python313t_d.dll!Py_Initialize() Line 1446
	at C:\Users\anthonyshaw\source\repos\cpython\Python\pylifecycle.c(1446)
[Managed to Native Transition]

CPython versions tested on:

CPython main branch

Operating systems tested on:

Windows

Output from running 'python -VV' on the command line:

Python 3.13.0rc1 experimental free-threading build (tags/v3.13.0rc1:e4a3e786a5e, Aug 12 2024, 08:07:19) [MSC v.1940 64 bit (AMD64)]

Activity

  1. tonybaloney
    added
    type-crashA hard crash of the interpreter, possibly with a core dump
    on Aug 11, 2024

  2. tonybaloney commented on Aug 11, 2024

    @tonybaloney
    tonybaloney
    on Aug 11, 2024
    ContributorAuthor

    I looked at the build to see how it was reproducing workable binaries, but the build for Windows free-threading looks broken?
    @DinoV @zooba

  3. tonybaloney commented on Aug 11, 2024

    @tonybaloney
    tonybaloney
    on Aug 11, 2024
    ContributorAuthor

    This is compiled from source against the 3.13rc1 tag

  4. colesbury commented on Aug 12, 2024

    @colesbury
    colesbury
    on Aug 12, 2024
    Contributor

    We saw these sorts of crashes before when the free-threaded interpreter was loading shared libraries compiled for the default build.

    Does the crash go away if you do a "clean" build?

  5. Eclips4
    added
    OS-windows
    topic-free-threading
    on Aug 12, 2024

  6. tonybaloney commented on Aug 12, 2024

    @tonybaloney
    tonybaloney
    on Aug 12, 2024
    ContributorAuthor

    I tried PCBuild\clean.bat first, but that is nowhere near as comprehensive as something like make distclean or make clobber (which doesn't work on Windows).

    A fresh clone worked against main and the v3.13.0rc1 tags.

    We can either close this or improve clean.bat to remove the libraries that caused this

  7. colesbury commented on Aug 12, 2024

    @colesbury
    colesbury
    on Aug 12, 2024
    Contributor

    Improving the clean.bat script seems like a good idea

  8. zooba commented on Aug 13, 2024

    @zooba
    zooba
    on Aug 13, 2024
    Member

    clean.bat is going to clean the build configuration that you pass it (same syntax as build.bat), by design.

    Adding a distclean.bat that does git clean -xdf might be the best option here.

  9. ngoldbaum commented on Aug 14, 2024

    @ngoldbaum
    ngoldbaum
    on Aug 14, 2024
    Contributor

    I'm seeing similar crashes in my PyO3 dev environment on MacOS. Interestingly, it doesn't happen if I use a debug python 3.13 free-threaded build.

    May be completely unrelated to the Windows-specific issues described above.

    Here's the traceback from the crash:

    Details 
    running 681 tests
test buffer::tests::test_compatible_size ... ok
test buffer::tests::test_element_type_from_format ... ok
test conversions::std::array::tests::array_try_from_fn ... ok
Process 18512 stopped
* thread #3, name = 'buffer::tests::test_bytes_buffer', stop reason = EXC_BAD_ACCESS (code=2, address=0x1015edb08)
    frame #0: 0x000000010132d048 libpython3.13t.dylib`chacha_block(ctx=0x00000001015edac8) at random.c:67:20 [opt]
   64
   65  	  // add scrambled data to the initial state
   66  	  for (size_t i = 0; i < 16; i++) {
-> 67  	    ctx->output[i] = x[i] + ctx->input[i];
   68  	  }
   69  	  ctx->output_available = 16;
   70
Target 0: (pyo3-71e3a8c0dab6095a) stopped.
warning: libpython3.13t.dylib was compiled with optimization - stepping may behave oddly; variables may not be available.
(lldb) bt
* thread #3, name = 'buffer::tests::test_bytes_buffer', stop reason = EXC_BAD_ACCESS (code=2, address=0x1015edb08)
  * frame #0: 0x000000010132d048 libpython3.13t.dylib`chacha_block(ctx=0x00000001015edac8) at random.c:67:20 [opt]
    frame #1: 0x00000001013218d0 libpython3.13t.dylib`_mi_os_get_aligned_hint [inlined] chacha_next32(ctx=0x00000001015edac8) at random.c:83:5 [opt]
    frame #2: 0x00000001013218bc libpython3.13t.dylib`_mi_os_get_aligned_hint [inlined] _mi_random_next(ctx=0x00000001015edac8) at random.c:149:25 [opt]
    frame #3: 0x00000001013218bc libpython3.13t.dylib`_mi_os_get_aligned_hint [inlined] _mi_heap_random_next(heap=0x00000001015ecf80) at heap.c:258:10 [opt]
    frame #4: 0x00000001013218b8 libpython3.13t.dylib`_mi_os_get_aligned_hint(try_alignment=33554432, size=1073741824) at os.c:118:19 [opt]
    frame #5: 0x000000010132ee50 libpython3.13t.dylib`unix_mmap_prim(addr=0x0000000000000000, size=1073741824, try_alignment=33554432, protect_flags=3, flags=4162, fd=1677721600) at prim.c:190:18 [opt]
    frame #6: 0x0000000101327f68 libpython3.13t.dylib`_mi_prim_alloc [inlined] unix_mmap(addr=0x0000000000000000, size=<unavailable>, try_alignment=<unavailable>, protect_flags=3, large_only=false, allow_large=<unavailable>, is_large=<unavailable>) at prim.c:297:9 [opt]
    frame #7: 0x0000000101327ed4 libpython3.13t.dylib`_mi_prim_alloc(size=1073741824, try_alignment=33554432, commit=<unavailable>, allow_large=<unavailable>, is_large=0x0000000170211bdf, is_zero=<unavailable>, addr=0x0000000170211b78) at prim.c:334:11 [opt]
    frame #8: 0x0000000101321c18 libpython3.13t.dylib`mi_os_prim_alloc(size=1073741824, try_alignment=33554432, commit=true, allow_large=<unavailable>, is_large=<unavailable>, is_zero=<unavailable>, stats=<unavailable>) at os.c:201:13 [opt]
    frame #9: 0x000000010131a648 libpython3.13t.dylib`_mi_os_alloc_aligned [inlined] mi_os_prim_alloc_aligned(size=1073741824, alignment=33554432, commit=true, allow_large=<unavailable>, is_large=0x0000000170211bdf, is_zero=0x0000000170211bde, base=<unavailable>, stats=<unavailable>) at os.c:234:13 [opt]
    frame #10: 0x000000010131a60c libpython3.13t.dylib`_mi_os_alloc_aligned(size=<unavailable>, alignment=33554432, commit=true, allow_large=<unavailable>, memid=0x0000000170211c68, tld_stats=<unavailable>) at os.c:320:13 [opt]
    frame #11: 0x000000010131bb84 libpython3.13t.dylib`mi_reserve_os_memory_ex(size=1073741824, commit=true, allow_large=<unavailable>, exclusive=false, arena_id=0x0000000170211ccc) at arena.c:813:17 [opt]
    frame #12: 0x0000000101319fd0 libpython3.13t.dylib`_mi_arena_alloc_aligned [inlined] mi_arena_reserve(req_size=33554432, allow_large=true, req_arena_id=0, arena_id=0x0000000170211ccc) at arena.c:362:11 [opt]
    frame #13: 0x0000000101319f6c libpython3.13t.dylib`_mi_arena_alloc_aligned(size=33554432, alignment=33554432, align_offset=0, commit=true, allow_large=true, req_arena_id=0, memid=0x0000000170211da8, tld=0x00000001016cc828) at arena.c:383:11 [opt]
    frame #14: 0x000000010132dde8 libpython3.13t.dylib`mi_segment_alloc [inlined] mi_segment_os_alloc(required=0, page_alignment=<unavailable>, eager_delayed=<unavailable>, req_arena_id=0, psegment_slices=<unavailable>, ppre_size=<unavailable>, pinfo_slices=<unavailable>, commit=<unavailable>, tld=0x00000001016cc490, os_tld=<unavailable>) at segment.c:823:42 [opt]
    frame #15: 0x000000010132ddd0 libpython3.13t.dylib`mi_segment_alloc(required=0, page_alignment=<unavailable>, req_arena_id=0, tld=<unavailable>, os_tld=<unavailable>, huge_page=0x0000000000000000) at segment.c:882:27 [opt]
    frame #16: 0x0000000101326a38 libpython3.13t.dylib`mi_segments_page_alloc [inlined] mi_segment_reclaim_or_alloc(heap=0x00000001016cac80, needed_slices=1, block_size=64, tld=0x00000001016cc490, os_tld=0x00000001016cc828) at segment.c:1489:10 [opt]
    frame #17: 0x0000000101326788 libpython3.13t.dylib`mi_segments_page_alloc(heap=0x00000001016cac80, page_kind=<unavailable>, required=64, block_size=64, tld=0x00000001016cc490, os_tld=0x00000001016cc828) at segment.c:1508:9 [opt]
    frame #18: 0x000000010132cc7c libpython3.13t.dylib`mi_page_fresh_alloc(heap=0x00000001016cac80, pq=0x00000001016cb150, block_size=64, page_alignment=<unavailable>) at page.c:284:21 [opt]
    frame #19: 0x0000000101324044 libpython3.13t.dylib`mi_find_page [inlined] mi_page_fresh(heap=0x00000001016cac80, pq=0x00000001016cb150) at page.c:305:21 [opt]
    frame #20: 0x0000000101324030 libpython3.13t.dylib`mi_find_page [inlined] mi_page_queue_find_free_ex(heap=0x00000001016cac80, pq=0x00000001016cb150, first_try=true) at page.c:782:12 [opt]
    frame #21: 0x0000000101323ff0 libpython3.13t.dylib`mi_find_page [inlined] mi_find_free_page(heap=0x00000001016cac80, size=<unavailable>) at page.c:821:10 [opt]
    frame #22: 0x0000000101323e6c libpython3.13t.dylib`mi_find_page(heap=0x00000001016cac80, size=64, huge_alignment=0) at page.c:920:12 [opt]
    frame #23: 0x0000000101316158 libpython3.13t.dylib`_mi_malloc_generic(heap=0x00000001016cac80, size=<unavailable>, zero=<unavailable>, huge_alignment=0) at page.c:946:21 [opt]
    frame #24: 0x0000000101422014 libpython3.13t.dylib`gc_alloc [inlined] _PyObject_MallocWithType(tp=<unavailable>, size=<unavailable>) at pycore_object_alloc.h:46:17 [opt]
    frame #25: 0x0000000101421ff4 libpython3.13t.dylib`gc_alloc(tp=<unavailable>, basicsize=<unavailable>, presize=0) at gc_free_threading.c:1695:17 [opt]
    frame #26: 0x0000000101421ea4 libpython3.13t.dylib`_PyObject_GC_New(tp=0x0000000101657268) at gc_free_threading.c:1716:20 [opt]
    frame #27: 0x00000001012f266c libpython3.13t.dylib`PyDict_New [inlined] new_dict(interp=0x0000000101698e80, keys=<unavailable>, values=0x0000000000000000, used=0, free_values_on_failure=0) at dictobject.c:929:14 [opt]
    frame #28: 0x00000001012f263c libpython3.13t.dylib`PyDict_New at dictobject.c:1026:12 [opt]
    frame #29: 0x0000000101386050 libpython3.13t.dylib`_PyUnicode_InitGlobalObjects [inlined] init_interned_dict(interp=0x0000000101698e80) at unicodeobject.c:284:37 [opt]
    frame #30: 0x000000010138604c libpython3.13t.dylib`_PyUnicode_InitGlobalObjects(interp=0x0000000101698e80) at unicodeobject.c:15016:9 [opt]
    frame #31: 0x00000001014521f0 libpython3.13t.dylib`pycore_interp_init [inlined] pycore_init_global_objects(interp=0x0000000101698e80) at pylifecycle.c:702:14 [opt]
    frame #32: 0x00000001014521dc libpython3.13t.dylib`pycore_interp_init(tstate=0x00000001016c9330) at pylifecycle.c:852:14 [opt]
    frame #33: 0x000000010144f89c libpython3.13t.dylib`Py_InitializeFromConfig [inlined] pyinit_config(runtime=<unavailable>, tstate_p=<unavailable>, config=0x0000000170212130) at pylifecycle.c:933:14 [opt]
    frame #34: 0x000000010144f784 libpython3.13t.dylib`Py_InitializeFromConfig [inlined] pyinit_core(runtime=<unavailable>, src_config=<unavailable>, tstate_p=<unavailable>) at pylifecycle.c:1096:18 [opt]
    frame #35: 0x000000010144f784 libpython3.13t.dylib`Py_InitializeFromConfig(config=0x00000001702123a0) at pylifecycle.c:1398:14 [opt]
    frame #36: 0x000000010144f9b0 libpython3.13t.dylib`Py_InitializeEx(install_sigs=<unavailable>) at pylifecycle.c:1436:14 [opt]
    frame #37: 0x0000000100105b2c pyo3-71e3a8c0dab6095a`pyo3::gil::prepare_freethreaded_python::_$u7b$$u7b$closure$u7d$$u7d$::h2cd40f096e08bff0((null)={closure_env#0} @ 0x00000001702125c7, (null)=0x0000000170212640) at gil.rs:69:13
    frame #38: 0x00000001000654fc pyo3-71e3a8c0dab6095a`std::sync::once::Once::call_once_force::_$u7b$$u7b$closure$u7d$$u7d$::hdab74cb30759684d(p=0x0000000170212640) at once.rs:208:40
    frame #39: 0x0000000100314024 pyo3-71e3a8c0dab6095a`std::sys::sync::once::queue::Once::call::h9dcea807fd617ccf at queue.rs:183:21 [opt]
    frame #40: 0x00000001000652f0 pyo3-71e3a8c0dab6095a`std::sync::once::Once::call_once_force::h6081e1899cf15adf(self=0x00000001004d8ea0, f={closure_env#0} @ 0x000000017021271f) at once.rs:208:9
    frame #41: 0x0000000100000b0c pyo3-71e3a8c0dab6095a`pyo3::gil::prepare_freethreaded_python::hb6ac25f766ef5eb8 at gil.rs:66:5
    frame #42: 0x0000000100000b7c pyo3-71e3a8c0dab6095a`pyo3::gil::GILGuard::acquire::hcdf69afc9ea1299e at gil.rs:174:21
    frame #43: 0x00000001001a1708 pyo3-71e3a8c0dab6095a`pyo3::marker::Python::with_gil::h089ec9aca00e522e(f={closure_env#0} @ 0x000000017021279f) at marker.rs:403:21
    frame #44: 0x0000000100216c2c pyo3-71e3a8c0dab6095a`pyo3::buffer::tests::test_bytes_buffer::h458708c8fd91dd1e at buffer.rs:849:9
    frame #45: 0x0000000100035e40 pyo3-71e3a8c0dab6095a`pyo3::buffer::tests::test_bytes_buffer::_$u7b$$u7b$closure$u7d$$u7d$::h8009c230847e71d5((null)=0x00000001702127fe) at buffer.rs:848:27

    Both the debug python build and the optimized, crashing Python are built from source using pyenv.

  10. colesbury commented on Aug 14, 2024

    @colesbury
    colesbury
    on Aug 14, 2024
    Contributor

    That looks unrelated to the Windows issues. The traceback makes me thing there is a bug in CPython (not just PyO3). Opened a new issue: #123022

  11. vstinner commented on Aug 30, 2024

    @vstinner
    vstinner
    on Aug 30, 2024
    Member

    That looks unrelated to the Windows issues. The traceback makes me thing there is a bug in CPython (not just PyO3). Opened a new issue: #123022

    Since the issue gh-123022 has been fixed, can we close this issue as well?

  12. tonybaloney
    closed this as completedon Aug 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    OS-windowstopic-free-threadingtype-crashA hard crash of the interpreter, possibly with a core dump

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions