Skip to content

Skip unconsumed Markdown in affected test selection - #1900

Merged
JacobStephens2 merged 2 commits into
mainfrom
issue-1612
Nov 9, 2026
Merged

JacobStephens2 merged 2 commits into
mainfrom
issue-1612

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Nov 9, 2026 •

Copy link
Copy Markdown
Owner

Summary

changed Markdown
  known build/test consumer -> retain checks from DOC_CHECKS
  otherwise                -> explain docs-only skip; add no checks
other changed paths        -> existing selection

Adds a repository reference guard and documents the selection rules. Current consumers require exceptions for the manual, glossary, experiment format, Console API ADR, and Go benchmark report. site/tests/docs.test.cjs now reads the glossary, so it receives site checks rather than the issue's proposed skip.

Closes #1612

Evidence

Execution evidence

  • Before: python3 -m unittest discover -s scripts -p test_affected_tests.py -k unread_markdown failed for all six documentation paths: [['cargo', 'nextest', 'run']] != [].
    After: The same command passed; docs-only paths select no runner and print the reason.
  • Tests exercise these seams: scripts/affected-tests <base> through real Git and stand-in external runners; the repository's Markdown-reference audit through real source files and a deliberately added consumer.
  • Before: python3 -m unittest discover -s scripts -p test_affected_tests.py -k backtick failed with AssertionError: AssertionError not raised.
    After: The same command passes (Ran 1 test, OK). The reviewer’s exact reproducer, which first reads the doc with Node and then checks the guard, also exits 0. The regression test remains in the suite.
  • python3 -m unittest discover -s scripts -p 'test_*.py': 49 tests passed in 26.876s with Python 3.12 on PATH for the runner and its subprocesses. The host's default Python 3.9 lacks tomllib, so the release-tooling tests require this environment setup.
  • scripts/affected-tests main selected the full Rust suite for the selector edits (Cargo fallback because nextest is unavailable). First run stopped after 509.93s because a Python subprocess lacked tomllib; cargo test --test bug_origins settlement::historical_review_and_reclosed_origin_regressions passes with Python 3.12 on PATH. Compilation was included (3m45s).
  • Re-running scripts/affected-tests main with Python 3.12 on PATH stopped after 461.84s at the unchanged Console test idle::unchanged_requests_reuse_retained_records_but_replacements_are_visible: unchanged request reread 158035 bytes. Its isolated rerun, cargo test --test console idle::unchanged_requests_reuse_retained_records_but_replacements_are_visible, also fails (unchanged request reread 156691 bytes). No Rust sources or tests changed in this PR. Before that stop, all 1,661 Rust unit tests passed; remaining integration binaries were not run locally.
  • GitHub CI is running on head 8bf2109c; site, Console browser, and release checks have passed. The full local Rust suite is not green.

Unaddressed findings

  • Standards: None.
  • Spec: None; the backtick-literal guard finding was reproduced and fixed with a retained regression test.
  • Security: None.

Review coverage: both reviewers read all three changed files. Unread changed files: none. Generated-output exclusions: none.

Review totals: Standards 0 findings; Spec 1 finding, fixed. Reports and final files-read lists are retained beside the Session trace.

Security review outcome

No unaddressed introduced findings.

Merge Danger

Door: two-way

Blast Radius: local

Changes local test selection. CI retains its existing checks on every PR.

Built with codex codex-cli 0.160.0 · gpt-6.1-sol · high

@JacobStephens2
JacobStephens2 merged commit 2657566 into main Nov 9, 2026
16 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-1612 branch November 9, 2026 22:20
@JacobStephens2 JacobStephens2 mentioned this pull request Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

affected-tests runs the full Rust suite for docs-only changes

1 participant