[JacobStephens2]

Summary

Two scripts import tomllib, which needs Python 3.11: the Bug origin regressions helper and the release generator. Before this change, a python3 of 3.9 or 3.10 caused two problems:

• scripts/affected-tests ran the full Rust suite anyway.
• settlement::historical_review_and_reclosed_origin_regressions then failed only when the suite ended, with an import traceback.

Now all three scripts refuse an older interpreter at once, using the same single line:

 import sys
+# Before tomllib: name the Python needed rather than an import traceback.
+if sys.version_info < (3, 11):
+    sys.exit("Python 3.11 or later is required; found Python "
+             f"{'.'.join(map(str, sys.version_info[:3]))} at {sys.executable}")
 import tomllib

scripts/affected-tests             # guard after stdlib imports, before any git call or selection
scripts/generate-release.py        # guard before import tomllib
tests/bug_origins/regressions.py   # guard before import tomllib
docs/testing.md                    # states the 3.11 minimum
scripts/test_python_minimum.py     # new: helpers reject 3.9/3.10; all three still parse as 3.9
scripts/test_affected_tests.py     # selector rejects 3.9/3.10 without running checks; 3.11 selects as before

The check is copied into each script, not shared. The issue asked for sharing only "if sharing stays simple". The scripts live in two directories, and the check has to run before import tomllib, so a shared module would mean editing sys.path before any import. Tests pin the exact message in every copy.

The tests fake older interpreters: they set sys.version_info and run each script through runpy. A separate ast.parse(..., feature_version=(3, 9)) check ensures the scripts still parse far enough on 3.9 to print the message.

Closes #1896

Evidence

Execution evidence

These runs used real interpreters installed with uv python install 3.9 3.10 3.11. Each run put a python3 shim at the front of PATH.

• Before (on main, with python3 = 3.9.24):

  • python3 scripts/affected-tests --help printed its usage and exited 0, so the selector would go on to select and run checks.
  • python3 tests/bug_origins/regressions.py printed:

      File ".../tests/bug_origins/regressions.py", line 10, in <module>
        import tomllib
    ModuleNotFoundError: No module named 'tomllib'
    

  • python3 scripts/generate-release.py --check failed with the same ModuleNotFoundError.

  After (same shim, 3.9 then 3.10):

  $ python3 scripts/affected-tests main; echo exit=$?
  Python 3.11 or later is required; found Python 3.9.24 at /tmp/tmp.0F9Mrkth7z/python3
  exit=1
  $ python3 tests/bug_origins/regressions.py x y
  Python 3.11 or later is required; found Python 3.9.24 at /tmp/tmp.0F9Mrkth7z/python3
  $ python3 scripts/generate-release.py --check
  Python 3.11 or later is required; found Python 3.9.24 at /tmp/tmp.0F9Mrkth7z/python3
  (3.10.19: the same three lines with "found Python 3.10.19")
  

  With 3.11.14, the selector went on to its normal selection and started the full Rust suite.

• Before: the regression tests were written before the fix, and they failed as follows:

  • python3 -m unittest scripts/test_python_minimum.py failed 4 subtests. regressions.py printed a Traceback ... and generate-release.py printed usage: ..., where the tests expected the one-line message.
  • test_affected_tests.AffectedTests.test_python_older_than_3_11_is_rejected_before_any_check failed for 3.9.18 and 3.10.14 with AssertionError: 0 == 0: the selector exited 0 and ran checks.

  After: python3 -m unittest discover -s scripts -p 'test_*.py' → Ran 53 tests ... OK with 3.12.3. Under the real 3.11.14 it also reports OK.

• Full Rust suite: scripts/affected-tests main ran on a09cfc7, and cargo test --no-fail-fast on cc97102 (host Python 3.12.3).

  • settlement::historical_review_and_reclosed_origin_regressions ... ok.
  • Each run had intermittent failures in tests that start no Python I changed, and every one passed when rerun alone:
    • affected-tests: process findings::closing_rechecks_each_merged_base_and_leaves_a_failing_finding_for_the_day_shift and findings::fixing_dispatches_each_reproduced_finding_and_keeps_unreproduced_findings_private.
    • --no-fail-fast: process findings::merged_fix_is_rechecked_after_a_later_fix_fails (3/3 alone) and update console::admin_update_answers_after_installation_then_console_serves_the_new_version (3/3 alone).
  • findings::closing_rechecks_... also fails intermittently on main (1 of 3 runs), so it predates this change.
  • Every other target passed.

Confirmed cause:

• Neither the selector nor the tomllib importers checked the interpreter version.
• docs/testing.md asked only for "Python 3".
• The selector itself runs fine on 3.9, so nothing failed until the Rust test started the helper, at the end of the suite.

Merge Danger

Door: two-way

Blast Radius: tooling

The change affects only hosts whose python3 is older than 3.11. On those hosts the selector, the release generator and the settlement regression test now stop at once with one line, where before they failed later. CI's ubuntu-24.04 runners have Python 3.12. Rust code is unchanged.

Review

Both reviewers read all six changed files. Reports: Standards 6 findings, Spec 2. I fixed these:

• Standards: the shared test wrapper (2), the comment on the placeholder arguments (3), the doc wording and the TypeScript row (4), the comments on the guards (6).
• Spec: the TypeScript row (1), and a test that the scripts still parse as 3.9 (2).

Unaddressed findings

Standards

• 1. Duplicated version guard in three scripts. Declined. The issue says "shared if sharing stays simple", and it is not simple here: the guard must run before import tomllib in scripts that live in scripts/ and tests/bug_origins/. The reviewer agreed. scripts/test_python_minimum.py and test_affected_tests.py pin the exact message in each copy.
• 4 (part). The comment at src/evals/experiment/execution/tests.rs:188 mentions macOS runners' older Python. Declined. It is outside this diff, and the fake script there avoids tomllib on purpose, which stays correct.
• 5. Local selection does not pick test_python_minimum.py. No change needed. The reviewer said this is not a violation: docs/testing.md says to run python3 -m unittest discover -s scripts -p 'test_*.py' for Python tooling changes, and CI runs it.

Spec

• 1 (part). docs/testing.md:122 still says the Console session-expiry test "needs Python 3". Declined. tests/console/session-expiry.py runs directly, not through the selector, and does not use tomllib, so "3.11" there would overstate what it needs. The opening paragraph states the single 3.11 minimum for the tests and the release generator.

No changed file was left unread by the review.

Built with claude 2.1.314 · opus · default effort

Security review outcome

No unaddressed introduced findings.