Repository navigation
Save-compat: unit tests for compare, fidelity_failures, diff_files, counts - #27
Merged
Merged
Conversation
…ounts These helpers decide candidate-read, candidate-append, fixture-read, fixture-unchanged and fixture-counts, and had no tests. The new tests characterize their current behaviour, including that a fidelity value must be exactly true and that a baseline's own fidelity failures are reported without failing the candidate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Member
Author
|
Self-review rubric (performed inline; no independent reviewer was available to this dispatch):
Do-not-auto-merge check: no modified path matches ( This review was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener). drafted by Claude on behalf of Daniel Stephenson |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stage B (unit-test expansion). The save-compatibility gate's view and file comparison helpers had no tests, although they decide
candidate-read,candidate-append,fixture-read,fixture-unchangedandfixture-counts. Newtests/test_save_compat_compare.py(15 tests) characterizes their current behaviour:compare(): equal views pass. A differing top-level field is named. A nested difference is located by path and save name (saves[1 'b'].gold). A missing save fails.fidelityblocks are left out of the equality check. A candidate fidelity value that isfalse, or truthy but not exactlytrue, fails. A baseline fidelity failure is reported in the detail but does not fail the candidate.fidelity_failures(): empty when every value is true. A non-objectfidelityis reported. Top-level names carry no leading dot.diff_files(): empty when nothing changed. Removed, changed and added files are listed in name order.counts(): saves, unreadable and valid are counted along withnext_slot.None,{}and a non-dict view count nothing.No production code is changed. No test showed a bug.
No tracking issue: the gap was found during triage.
Verification
python3 -m unittestover every module that imports on the interpreter this dispatch can run (Python 3.8): 62 tests, OK (4 skipped, already skipped before this change).tests/test_tests_gate_probe.pydoes not import on 3.8. CI runs the full suite on 3.12.gates/save_compat.pyand then restored withgit checkout:strip_fidelitykeepingfidelitykeys,comparedropping the baseline-fidelity note, andcountscounting anyvalidationvalue as valid, applied together: 4 tests failed (test_fidelity_blocks_are_not_compared_for_equality,test_a_false_fidelity_value_in_the_candidate_fails,test_a_baseline_fidelity_failure_is_reported_but_passes,test_counts_saves_unreadable_and_valid).fidelity_failuresaccepting any truthy value (not okin place ofok is not True), applied alone:test_a_truthy_but_not_true_fidelity_value_failsfailed.python3.12also needs an approval this headless dispatch cannot get, so no gate run was possible in any case.Test plan
Check/compile) green on the PR head (runspy_compileandunittest discover -s testson 3.12)Skipped issues
installed_commitreading stderr): the fix is ingates/common.py, which is on the do-not-auto-merge list, and it needs a real gate run that this dispatch cannot make.controlfailure.workflow_callcheckout): waits on an owner decision on whetherworkflow_callis supported. Verifying it needs a call from a second repository.This PR description was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).
🤖 Generated with Claude Code
drafted by Claude on behalf of Daniel Stephenson