Repository navigation
TSan reported race condition in _PyPegen_is_memoized in test_importlib #122581
Description
Bug report
I think this is a race on the global _PyRuntime.parser.memo_statistics. That's only used in the debug build for stats, so the race isn't particularly dangerous, but we should fix it.
WARNING: ThreadSanitizer: data race (pid=8921)
Write of size 8 at 0x55b27b2a92f0 by thread T84:
#0 _PyPegen_is_memoized /home/runner/work/cpython/cpython/Parser/pegen.c:351:39 (python+0x166292) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#1 bitwise_or_rule /home/runner/work/cpython/cpython/Parser/parser.c:[12](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:13)893:9 (python+0x1842c2) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#2 bitwise_or_raw /home/runner/work/cpython/cpython/Parser/parser.c:12951:18 (python+0x184511) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#3 bitwise_or_rule /home/runner/work/cpython/cpython/Parser/parser.c:12906:22 (python+0x184511)
#4 comparison_rule /home/runner/work/cpython/cpython/Parser/parser.c:12146:18 (python+0x19bc7f) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#5 inversion_rule /home/runner/work/cpython/cpython/Parser/parser.c:12097:31 (python+0x19b869) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#6 conjunction_rule /home/runner/work/cpython/cpython/Parser/parser.c:11974:18 (python+0x19a9b8) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#7 disjunction_rule /home/runner/work/cpython/cpython/Parser/parser.c:11886:18 (python+0x1938f8) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#8 expression_rule /home/runner/work/cpython/cpython/Parser/parser.c:11174:18 (python+0x182e81) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#9 named_expression_rule /home/runner/work/cpython/cpython/Parser/parser.c:11832:31 (python+0x181b40) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#10 if_stmt_rule /home/runner/work/cpython/cpython/Parser/parser.c:5927:18 (python+0x1769b7) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#11 compound_stmt_rule /home/runner/work/cpython/cpython/Parser/parser.c:2114:28 (python+0x1769b7)
#12 statement_rule /home/runner/work/cpython/cpython/Parser/parser.c:1419:18 (python+0x17582f) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#[13](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:14) _loop1_3_rule /home/runner/work/cpython/cpython/Parser/parser.c:25738:30 (python+0x17582f)
#14 statements_rule /home/runner/work/cpython/cpython/Parser/parser.c:1376:18 (python+0x17582f)
#15 file_rule /home/runner/work/cpython/cpython/Parser/parser.c:1178:18 (python+0x170f5a) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#16 _PyPegen_parse /home/runner/work/cpython/cpython/Parser/parser.c:42410:18 (python+0x170f5a)
#17 _PyPegen_run_parser /home/runner/work/cpython/cpython/Parser/pegen.c:869:17 (python+0x167e01) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#18 _PyPegen_run_parser_from_string /home/runner/work/cpython/cpython/Parser/pegen.c:992:[14](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:15) (python+0x1684e7) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#19 _PyParser_ASTFromString /home/runner/work/cpython/cpython/Parser/peg_api.c:13:21 (python+0x2175dd) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#20 Py_CompileStringObject /home/runner/work/cpython/cpython/Python/pythonrun.c:1435:11 (python+0x5fa5ab) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
...
Previous write of size 8 at 0x55b27b2a92f0 by thread T83:
#0 _PyPegen_is_memoized /home/runner/work/cpython/cpython/Parser/pegen.c:351:39 (python+0x166292) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#1 bitwise_or_rule /home/runner/work/cpython/cpython/Parser/parser.c:12893:9 (python+0x1842c2) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#2 bitwise_or_raw /home/runner/work/cpython/cpython/Parser/parser.c:12951:18 (python+0x184511) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#3 bitwise_or_rule /home/runner/work/cpython/cpython/Parser/parser.c:12906:22 (python+0x184511)
#4 comparison_rule /home/runner/work/cpython/cpython/Parser/parser.c:12146:18 (python+0x19bc7f) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#5 inversion_rule /home/runner/work/cpython/cpython/Parser/parser.c:12097:31 (python+0x19b869) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#6 conjunction_rule /home/runner/work/cpython/cpython/Parser/parser.c:11974:18 (python+0x19a9b8) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#7 disjunction_rule /home/runner/work/cpython/cpython/Parser/parser.c:11886:18 (python+0x1938f8) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#8 expression_rule /home/runner/work/cpython/cpython/Parser/parser.c:11174:18 (python+0x182e81) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#9 star_expression_rule /home/runner/work/cpython/cpython/Parser/parser.c:11551:31 (python+0x1a0839) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#10 star_expressions_rule /home/runner/work/cpython/cpython/Parser/parser.c:11391:18 (python+0x19f602) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#11 simple_stmt_rule /home/runner/work/cpython/cpython/Parser/parser.c:17[63](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:64):18 (python+0x20241e) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#12 simple_stmts_rule /home/runner/work/cpython/cpython/Parser/parser.c:1618:18 (python+0x177d4a) (BuildId: 5a0193[64](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:65)4aa87641efb42b5a646a83190f2fd33b)
#13 statement_rule /home/runner/work/cpython/cpython/Parser/parser.c:1443:34 (python+0x175911) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#14 _loop1_3_rule /home/runner/work/cpython/cpython/Parser/parser.c:25738:30 (python+0x175911)
#15 statements_rule /home/runner/work/cpython/cpython/Parser/parser.c:1376:18 (python+0x175911)
#16 file_rule /home/runner/work/cpython/cpython/Parser/parser.c:1178:18 (python+0x170f5a) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#17 _PyPegen_parse /home/runner/work/cpython/cpython/Parser/parser.c:42410:18 (python+0x170f5a)
#18 _PyPegen_run_parser /home/runner/work/cpython/cpython/Parser/pegen.c:869:17 (python+0x1[67](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:68)e01) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#19 _PyPegen_run_parser_from_string /home/runner/work/cpython/cpython/Parser/pegen.c:992:14 (python+0x1[68](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:69)4e7) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#20 _PyParser_ASTFromString /home/runner/work/cpython/cpython/Parser/peg_api.c:13:21 (python+0x2175dd) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
#21 Py_CompileStringObject /home/runner/work/cpython/cpython/Python/pythonrun.c:1435:11 (python+0x5fa5ab) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
...
Job: https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577
Full log archive: https://gist.github.com/colesbury/bfd65049b17505fcae64f5afd3ed388e
Activity
colesbury commentedon Aug 1, 2024
cc @lysnikolaou, since you were looking at parser thread-safety recently
pythongh-122581: Avoid data races when collecting parser statistics
gh-122581: Avoid data races when collecting parser statistics (#122694)
pythongh-122581: Avoid data races when collecting parser statistics (p…
[3.13] gh-122581: Avoid data races when collecting parser statistics (G…
pythongh-122581: Avoid data races when collecting parser statistics (p…
pythongh-122581: Avoid data races when collecting parser statistics (p…
colesbury commentedon Dec 18, 2025
I'm seeing this in the default (GIL-enabled) build with things like:
pythongh-122581: Use parser mutex in default build for subinterpreters
pythongh-122581: Use parser mutex in default build for subinterpreters
colesbury commentedon Dec 18, 2025
I put up a PR: #142959
gh-122581: Use parser mutex in default build for subinterpreters (gh-…
pythongh-122581: Use parser mutex in default build for subinterpreters (
kumaraditya303 commentedon Jan 9, 2026
Fixed by #142959