Skip to content

README: consumers assertions, per-gate result.json keys, manifest fields - #25

Merged
dmccoystephenson merged 1 commit into
mainfrom
feature/readme-accuracy-sweep
Nov 7, 2026
Merged

dmccoystephenson merged 1 commit into
mainfrom
feature/readme-accuracy-sweep

Conversation

@dmccoystephenson

Copy link
Copy Markdown
Member

Summary

A Stage A documentation sweep: README.md was checked against the gates, the workflows, the scenario docstrings and the tags. No code changed, and every addition describes current behaviour.

  • Consumers gate: an assertion list was added, since the tests and save-compatibility gates had one and this gate did not. It records that control-<Name> always passes, that candidate-<Name> passes for a pre-existing consumer, and that control and harness are recorded only on failure. The harness's pytest and pytest-cov installs are also stated.
  • All-skipped consumers: it is now stated that a run in which every consumer is skipped still passes. This is today's behaviour, and Gates can pass having checked nothing: every consumer skipped, or saves with no files #21 is open on whether to change it.
  • result.json: the keys each gate adds were undocumented, although callers read them. A table was added under the shared shape.
  • Fixture manifest: what expected and primaryFile do was added.
  • "not in a tag yet" row: PR Read a scenario's view past stderr; match a pin URL only as a whole name #23 also made a non-object view fail, and the row did not say so. v5 takes any JSON on the last line as the view (confirmed with git show v5:gates/save_compat.py).

No tracking issue: this is a gap found during triage.

Checked with no drift found

  • Input tables and defaults, against both trigger blocks of tests.yml, consumers.yml and save-compat.yml; the two blocks agree in all three.
  • The tests and save-compatibility assertion lists, against the result.check(...) calls, in order.
  • The time-limits table, against run(), the suite timeouts, the manifest download, pip_install() and timeout-minutes.
  • The tag table, against git ls-tree and git grep on v1 to v5.
  • External names: the baseline tags in the dispatch examples, and the Cave, Night Ferry, ophidian and plugin-fixtures repositories, all exist. The tak/fishe-2026-07-04 tag exists.
  • Module docstrings in gates/*.py, workflow header comments, and the docstrings of tak_slots.py, tak_game_saves.py, cave_saves.py and fishe_saves.py. The remaining scenarios' docstrings were not read in this sweep, and the claims in the README's prose about them were not traced into their code.

Found but not changed

Test plan

  • git diff --stat origin/main: README.md only, 20 insertions and 2 deletions.
  • Each added claim was read against its source, cited in the self-review.
  • Check workflow green on the PR head.

Gate run: UNVERIFIED, not applicable. python3.12 needed an approval the dispatch could not receive, so no gate was run. This PR changes no gate, scenario or workflow, so none of the files a gate run would validate are touched.

Issues not taken this cycle

This PR description was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).


drafted by Claude on behalf of Daniel Stephenson

🤖 Generated with Claude Code

A documentation sweep against the gates, workflows and tags. Adds the
consumers gate's assertion list (the other two gates had one), the keys
each gate adds to result.json, what a manifest's expected and primaryFile
do, and the view-must-be-an-object change from #23 to the untagged row.
States that an all-skipped consumers run passes, as it does today (#21).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dmccoystephenson

Copy link
Copy Markdown
Member Author

Self-review rubric, scored against commit 0b2ab4d.

  • CI + local gate run: PASS for CI; gate run UNVERIFIED-not-applicable. The Check workflow (compile, which also runs the unit suite on 3.12) passed on the head. python3.12 needed an approval this dispatch could not receive, so no gate ran, and git diff --name-only origin/main...HEAD lists only README.md, so nothing a gate run would validate was touched.
  • Gate-run evidence: N/A — no script, scenario or workflow is touched.
  • Scope: PASS — one file, 20 insertions and 2 deletions, all from the sweep.
  • Tests-new: N/A — no code.
  • Tests-fix: N/A — no bug fix.
  • Tests-mutation: N/A — no characterization tests.
  • Sibling structure: PASS — the assertions paragraph takes the form of the two sibling sections, with a per-consumer qualifier because this gate repeats per consumer, as the tests gate's line does per interpreter.
  • Sibling renames: N/A — nothing was renamed.
  • Docs: PASS — Phase 7 table pass results:
    • Gate tables against inputs: of all three workflows (both blocks): no drift.
    • Assertion lists against result.check(...): consumers list was missing, added; the other two had no drift.
    • result.json shape against Result.doc: shared keys had no drift; per-gate keys were missing, added from the result.doc writes in each gate.
    • Fixture manifest against fixture_phase(): expected and primaryFile behaviour added. README.md:147 was read against gates/save_compat.py:325-335.
    • Module docstrings and workflow header comments: no drift.
    • CLAUDE.md: drift found and not corrected (see below).
  • Visuals: N/A — nothing a README image depicts changed.
  • Issue resolution: N/A — no Closes reference.
  • Input parity: N/A — no workflow touched; the blocks were compared anyway and agree.
  • Read-only: PASS — git grep -n "secrets\." .github/workflows is empty.
  • Stdlib only: N/A — no code.
  • No tag moved: PASS — git ls-remote --tags origin still shows v1 to v5, tak/fishe-2026-07-04 and roam/longplay-0.12.0; none was pushed.

Claims traced to source:

  • control-<Name> always passes: gates/consumers_gate.py:151, called with True.
  • candidate-<Name> passes when pre-existing: gates/consumers_gate.py:166-168.
  • control recorded only on failure: gates/consumers_gate.py:119,128,145, all called with False.
  • pytest, pytest-cov and extras: gates/consumers_gate.py:141.
  • All-skipped run passes: gates/consumers_gate.py:133-137, and the reproduction in Gates can pass having checked nothing: every consumer skipped, or saves with no files #21.
  • Non-object view in v5: v5:gates/save_compat.py:204 returns any parsed JSON.

Judgment calls for a reviewer:

Phases 5 to 7: no human reviewer can be awaited in a headless dispatch, and no review comment changed anything, so the table pass above stands. Merge is judged next: README.md matches no do-not-auto-merge path, and no assertion's name or meaning changes.

This comment was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).


drafted by Claude on behalf of Daniel Stephenson

@dmccoystephenson
dmccoystephenson merged commit 90e7983 into main Nov 7, 2026
1 check passed
@dmccoystephenson
dmccoystephenson deleted the feature/readme-accuracy-sweep branch November 7, 2026 07:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant