Repository navigation
[3.12] Crash of repeated Programs/_testembed.c on Windows DEBUG build #128833
Description
Bug report
Bug description:
This is an issue that only affects Windows DEBUG builds and Python 3.12. I discovered it while writing #127780 and @Yhg1s (#127780 (comment)) investigated it more in details.
The reproducer is as follows:
PS D:\Python\cpython> D:\\Python\\cpython\\PCbuild\\amd64\\_testembed_d.exe test_repeated_init_exec 'import dis
>> import importlib._bootstrap
>> import opcode
>> import test.test_dis
>>
>> def is_specialized(f):
>> for instruction in dis.get_instructions(f, adaptive=True):
>> opname = instruction.opname
>> if (
>> opname in opcode._specialized_instructions
>> # Exclude superinstructions:
>> and \"__\" not in opname
>> ):
>> return True
>> return False
>>
>> func = importlib._bootstrap._handle_fromlist
>>
>> # "copy" the code to un-specialize it:
>> func.__code__ = func.__code__.replace()
>>
>> assert not is_specialized(func), \"specialized instructions found\"
>>
>> for i in range(test.test_dis.ADAPTIVE_WARMUP_DELAY):
>> func(importlib._bootstrap, [\"x\"], lambda *args: None)
>>
>> assert is_specialized(func), \"no specialized instructions found\"
>>
>> print(\"Tests passed\")'
--- Loop #1 ---
Tests passed
--- Loop #2 ---
Assertion failed: PyUnicode_CheckExact(ep->me_key), file D:\Python\cpython\Objects\dictobject.c, line 939Paraphrasing Thomas, the assertion failure happens only the second time, namely after finalizing and reinitializing, and the ep the assertion fails on is a deallocated object. We suspect that something isn't refcounted correctly, isn't marked as immortal when it should be, or isn't recreated on interpreter starutp when it should be. However, we don't why the issue does not appear on Linux.
According to Thomas, this is likely an issue with repeated calls to Py_Finalize and Py_Initialize. However, since 3.12 and 3.13 manage objects lifetime quite differently, it might be easier to just patch the test currently failing as follows: 68a33f3.
CPython versions tested on:
3.12
Operating systems tested on:
Windows, Linux
Activity
[-][3.12] Crash of `Programs/_testembed.c` on Windows DEBUG build[/-][+][3.12] Crash of repeated `Programs/_testembed.c` on Windows DEBUG build[/+][3.12] gh-128833: Patch `test_embed.test_specialized_static_code_gets…
ZeroIntensity commentedon Jan 28, 2025
Triaging: is everything done here?
picnixz commentedon Jan 28, 2025
No, the root cause is still not explained perfectly and we still have no perfect fix. My patch seems to work but we don't really know why it works. I'm keeping it open unless we decide to just ignore that issue =/
vstinner commentedon Jan 29, 2025
I cannot reproduce the bug on the current 3.12 branch on Linux with Python built in debug mode.
picnixz commentedon Jan 29, 2025
On Linux there was no issue IIRC but we tested on both and observed that this failed on Windows 3.12 but maybe it works now?
picnixz commentedon Mar 24, 2025
Since 3.12 is nearing its last bug fix, and since we didn't have time to investigate, let's close this.