Repository navigation
test_opcache fails when run with -Xuops #108311
Description
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Output from running 'python -VV' on the command line:
Python 3.13.0a0 (heads/fix-func-cache-dirty:531930f47f, Aug 22 2023, 07:17:15) [Clang 14.0.3 (clang-1403.0.22.14.1)]
A clear and concise description of the bug:
~/cpython$ ./python.exe -Xuops -m test test_opcache -m test_store_attr_with_hint
0:00:00 load avg: 27.04 Run tests sequentially
0:00:00 load avg: 27.04 [1/1] test_opcache
test test_opcache failed -- Traceback (most recent call last):
File "/Users/guido/cpython/Lib/test/test_opcache.py", line 955, in test_store_attr_with_hint
self.assert_races_do_not_crash(opname, get_items, read, write)
File "/Users/guido/cpython/Lib/test/test_opcache.py", line 524, in assert_races_do_not_crash
self.assert_specialized(read, opname)
File "/Users/guido/cpython/Lib/test/test_opcache.py", line 503, in assert_specialized
self.assertIn(opname, opnames)
AssertionError: 'STORE_ATTR_WITH_HINT' not found in {'RETURN_CONST', 'FOR_ITER_LIST', 'LOAD_CONST', 'RESUME', 'STORE_ATTR', 'STORE_FAST', 'ENTER_EXECUTOR', 'END_FOR', 'GET_ITER', 'LOAD_FAST'}
test_opcache failed (1 failure)
== Tests result: FAILURE ==
1 test failed:
test_opcache
Total duration: 268 ms
Tests result: FAILURE
Activity
gvanrossum commentedon Aug 22, 2023
@markshannon Do you have any idea what could be going on here? I'm afraid I don't completely understand what the failing test (
test_store_attr_with_hint) is doing.gvanrossum commentedon Aug 22, 2023
@markshannon I'm not at all sure I have the right fix (#108312 just disables the optimizer for the duration of the one test).
It looks like there's some non-determinism between the specialization of
LOAD_ATTRand the optimizer onJUMP_BACKWARD-- if the latter runs first,LOAD_ATTRwill never be specialized. But the question is, why would this happen sometimes but not all the time? When looking into it a few days ago in more detail, IIRC I found that 1 out of 1000 runs had this problem, where every other run had the correct specialization.) Maybe some counter does not get deterministically reset by.replace()?gvanrossum commentedon Aug 25, 2023
I think we do need to disable the executor-based optimizer for these tests. I observed that this loop in
assert_specializedgets translated into a superblock, tracing into the read() function:
before read() has had a chance to specialize its STORE_ATTR (which is currently a viable Tier 2 opcode). So then the subsequent assert_specialized() call fails.
Note that the tracing into read() stops when it encounters the JUMP_BACKWARD of the loop in read(); but it has already executed the unspecialized STORE_ATTR at that point, and it keeps doing so (during the warmup phase), until that JUMP_BACKWARD itself gets translated to Tier 2.
The solution is to just disable the Tier 2 optimizer in the entire assert_races_do_not_crash() function.
FWIW I also discovered that the JUMP_BACKWARD counter is initialized to 17 (the constant computed by adaptive_counter_warmup()), like all other counters, by
_PyCode_Quicken()in specialize.c. Arguably, since this counter is supposed to count upwards from zero (in steps of 16, i.e.1 << OPTIMIZER_BITS_IN_COUNTER), it ought to be initialized to zero (as I had expected it would be). But this ought to be a separate issue and PR.gh-108311: Fix test_store_attr_with_hint by disabling optimizer in de…