Skip to content

Save-compat: unit tests for compare, fidelity_failures, diff_files, counts - #27

Merged
dmccoystephenson merged 1 commit into
mainfrom
feature/save-compat-compare-tests
Nov 11, 2026
Merged

dmccoystephenson merged 1 commit into
mainfrom
feature/save-compat-compare-tests

Conversation

@dmccoystephenson

Copy link
Copy Markdown
Member

Summary

Stage B (unit-test expansion). The save-compatibility gate's view and file comparison helpers had no tests, although they decide candidate-read, candidate-append, fixture-read, fixture-unchanged and fixture-counts. New tests/test_save_compat_compare.py (15 tests) characterizes their current behaviour:

  • compare(): equal views pass. A differing top-level field is named. A nested difference is located by path and save name (saves[1 'b'].gold). A missing save fails. fidelity blocks are left out of the equality check. A candidate fidelity value that is false, or truthy but not exactly true, fails. A baseline fidelity failure is reported in the detail but does not fail the candidate.
  • fidelity_failures(): empty when every value is true. A non-object fidelity is reported. Top-level names carry no leading dot.
  • diff_files(): empty when nothing changed. Removed, changed and added files are listed in name order.
  • counts(): saves, unreadable and valid are counted along with next_slot. None, {} and a non-dict view count nothing.

No production code is changed. No test showed a bug.

No tracking issue: the gap was found during triage.

Verification

  • python3 -m unittest over every module that imports on the interpreter this dispatch can run (Python 3.8): 62 tests, OK (4 skipped, already skipped before this change). tests/test_tests_gate_probe.py does not import on 3.8. CI runs the full suite on 3.12.
  • Mutation checks, each made temporarily in gates/save_compat.py and then restored with git checkout:
    • strip_fidelity keeping fidelity keys, compare dropping the baseline-fidelity note, and counts counting any validation value as valid, applied together: 4 tests failed (test_fidelity_blocks_are_not_compared_for_equality, test_a_false_fidelity_value_in_the_candidate_fails, test_a_baseline_fidelity_failure_is_reported_but_passes, test_counts_saves_unreadable_and_valid).
    • fidelity_failures accepting any truthy value (not ok in place of ok is not True), applied alone: test_a_truthy_but_not_true_fidelity_value_fails failed.
    • After the restore, all 15 passed.
  • Local gate run: not applicable. No gate, scenario or workflow file is changed. python3.12 also needs an approval this headless dispatch cannot get, so no gate run was possible in any case.

Test plan

  • New tests pass locally. Mutations make them fail.
  • CI (Check / compile) green on the PR head (runs py_compile and unittest discover -s tests on 3.12)

Skipped issues

This PR description was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).

🤖 Generated with Claude Code


drafted by Claude on behalf of Daniel Stephenson

…ounts

These helpers decide candidate-read, candidate-append, fixture-read,
fixture-unchanged and fixture-counts, and had no tests. The new tests
characterize their current behaviour, including that a fidelity value
must be exactly true and that a baseline's own fidelity failures are
reported without failing the candidate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dmccoystephenson

Copy link
Copy Markdown
Member Author

Self-review rubric (performed inline; no independent reviewer was available to this dispatch):

  • Scope: PASS. One new file, tests/test_save_compat_compare.py. No production, workflow or doc file is touched.
  • Tests-new: N/A. No new public function is added.
  • Tests-fix: N/A. No bug fix.
  • Tests-mutation: PASS. strip_fidelity keeping fidelity keys, compare dropping the baseline note and counts counting any validation as valid together failed 4 tests. fidelity_failures with not ok in place of ok is not True, applied alone, failed test_a_truthy_but_not_true_fidelity_value_fails. After the file was restored with git checkout, all 15 passed.
  • Sibling structure: PASS. Same module docstring, sys.path insert, import save_compat and unittest.TestCase classes per helper as tests/test_save_compat_files.py and tests/test_save_compat_migrations.py.
  • Sibling renames: N/A.
  • Docs: PASS for this diff. README tables, assertion lists, module docstrings and workflow headers describe the gate, and none of them changes, so none was affected. One drift was found outside the diff: CLAUDE.md:19 describes tests/ as "the gate helpers (declared migrations)", but the directory also tests Result, find_pin, the import probe, parse_view/view, file_count and now compare. CLAUDE.md is agent-loaded config that needs separate authorization to edit, so it is left for the owner.
  • Visuals: N/A. Nothing visible changes.
  • Issue resolution: N/A. No Closes; the PR body records the gap as found during triage.
  • CI + local gate run: PASS (CI) / not applicable (gate run). Check / compile is green on the head: Ran 66 tests ... OK on CPython 3.12.16. No gate, scenario or workflow is changed, so no gate run is required.
  • Gate-run evidence: N/A. No gate script, scenario or workflow is touched.
  • Input parity: N/A. No workflow is touched.
  • Read-only: PASS. git grep -n "secrets\." .github/workflows is empty.
  • Stdlib only: PASS. The new file imports only sys, unittest, pathlib and save_compat.
  • No tag moved: PASS. No tag operation was made, and git ls-remote --tags origin lists the same 12 tags.

Do-not-auto-merge check: no modified path matches (tests/ only; no workflow, gates/common.py, assertion rename or tag).

This review was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).


drafted by Claude on behalf of Daniel Stephenson

@dmccoystephenson
dmccoystephenson merged commit 7db61e5 into main Nov 11, 2026
1 check passed
@dmccoystephenson
dmccoystephenson deleted the feature/save-compat-compare-tests branch November 11, 2026 04:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant