Repository navigation
Pure closure classifier (issue #147) - #151
Merged
Merged
Conversation
Contract, input model and tests for the pure classifier. The classification
logic is deliberately absent — `classify` returns a placeholder so the suite
fails, and the GREEN commit has something to turn green. RED and GREEN are kept
as separate commits so the chronology stays legible.
Input boundary, decided before implementation. The Step 0B envelope
(pull_request/reviews/review_comments/issue_comments/check_runs) is a TEST
container, not the production schema — a file has to carry several surfaces at
once. The production model is instead:
Subject repository, pull_request, expected_sha, head_before, head_after
Policy id, required_observations (authoritative order), completeness_claimed
observations id -> Acquisition<CheckEvidence | ReviewEvidence>
falsifications established facts with provenance
known_debts
Acquisition status is stated by the caller, never inferred from an absent key,
an empty array, a null or a parse error:
Available(T) NotProduced RateLimited Failed{reason}
which is what keeps OWED (nobody produced it) apart from CANNOT_CHECK (an
attempt failed). A failed query that yielded no values is not an empty result.
No prose oracle. Nothing reads English. A defect claim arrives as an already
established FalsificationFact with provenance; extracting such facts from vendor
responses belongs to the acquisition layer.
The fixture loader in tests/frozen_fixtures.rs unwraps the envelope into typed
inputs and may not inject outcomes: no fixture name maps to an expected state.
It supplies observable facts only — which commit a review binds to, whether a
review comment references it by pull_request_review_id, which checks exist at
which SHA. Each fixture test asserts its hostile precondition from the JSON
before asserting any classifier behaviour.
Two harness mappings are labelled as such rather than buried: vendor check name
-> policy observation id, and reviewer login -> policy observation id. #147
enumerates the ids but does not state either mapping; resolving them is the
acquisition layer's job.
Cases with no historical fixture witness (OWED, CANNOT_CHECK) are marked
logic-level unit witnesses on the tests themselves. The frozen corpus is not
claimed to cover them.
Refs #147
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013K3sUCBMGGjf5v4AqsW7Qk
Minimal logic to satisfy the RED suite. No acquisition, no network, no GitHub
API, no local Git authority, no attestation.
State derivation, per required observation in authoritative policy order:
check bound to expected_sha + success -> PASS
bound to expected_sha + failure -> FINDING
bound elsewhere -> OWED (not evidence about this
subject; never combined
across commits)
review bound to expected_sha, no finding -> PASS
bound to expected_sha, finding -> FINDING
bound elsewhere -> OWED (no verdict here at all)
either NotProduced | RateLimited -> OWED
Failed{..} -> CANNOT_CHECK
absent from the input -> OWED
Positive reviewer evidence is strict about commit binding. Falsification is
deliberately wide: an established fact counts even with no commit binding at
all, because #147 makes falsification the wider surface and gating it on a
binding an issue comment cannot have would silently narrow it.
Headline is derived presentation over everything recorded — the observation
vector, plus falsification (which owns no required slot), plus subject
staleness — and never rewrites the vector. A head that moved sets subject_stale
and the STALE headline while each observation still reports what it said about
the frozen subject.
Each call is an independent snapshot: no memory of a previous headline, so a
finding that disappears leaves CANNOT_CHECK as the headline rather than being
inherited or decaying to PASS.
Output is deterministic: no timestamp, no generated id, observations in policy
order, same semantic input -> byte-identical JSON. The predicate records the
required set and completenessClaimed=false, so PASS reads as "everything asked
for passed", never as "the set is complete" or "this may be merged".
24 tests pass: 20 acceptance, 4 against the frozen Step 0B fixtures. The frozen
fixtures, their README and the integrity validator are unchanged.
Refs #147
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013K3sUCBMGGjf5v4AqsW7Qk
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: QUIET Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…s (RED-2) Review findings against the classifier. The GREEN suite was green while holding three wrong semantics and one avoidable hole; these tests state the corrected rules and fail against the current logic. Contract/type changes are here so the tests compile; the behaviour is not yet implemented. 1. A CLAIMED falsification was treated as a FINDING. #147 defines FINDING as an admissible VERIFIED defect, and the wide falsification surface means a counterexample may arrive from a surface that cannot carry a verdict — not that an unverified claim is a defect. V0 semantics: REPRODUCED -> FINDING CLAIMED -> OWED (verification still owed) VERIFICATION_FAILED -> CANNOT_CHECK (attempted, no trustworthy answer) FalsificationOut therefore gains its own `state`, so a headline of FINDING, OWED or CANNOT_CHECK driven by this channel is explainable from the predicate rather than only from this crate's source. 2. A falsification explicitly bound to ANOTHER commit still raised this subject's snapshot. A wide surface means more KINDS of surface, not defects of any commit in the universe, and #147 forbids combining evidence across SHAs. Now ClassifierError::FalsificationSubjectMismatch. `None` stays admissible: an issue comment has no binding to give. 3. A required observation missing from the input map became OWED. That contradicts the rule this design exists for — acquisition state is carried by the caller, never inferred. An absent entry could be not-produced, a broken adapter, or a lost element; naming that unknown OWED converts a construction failure into a domain fact. Now ClassifierError::MissingRequiredObservation, and classify returns Result<Predicate, ClassifierError>: malformed input gets no well-formed predicate. 4. Policy.completeness_claimed was a caller-settable bool documented as "always false in V0" — a false claim the library would have serialized faithfully. The field is removed from the input entirely and the output materializes completenessClaimed: false unconditionally, so the forbidden state cannot be constructed. Also: `falsifications[]` is no longer described as established defects. It is a falsification channel whose records carry their own state, since a CLAIMED record is precisely not an established defect. RED-2 state: 4 failing (the four corrected rules), 20 passing, 4 fixture tests passing. Frozen Step 0B fixtures, README and validator untouched. Refs #147 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013K3sUCBMGGjf5v4AqsW7Qk
…(GREEN-2)
Turns RED-2 green. Four rules, no more.
Falsification records now carry their own state, derived from how far
verification actually got:
REPRODUCED -> FINDING
CLAIMED -> OWED
VERIFICATION_FAILED -> CANNOT_CHECK
and the headline is the maximum over the observation vector CHAINED with the
falsification states, rather than a special case that could only ever raise
FINDING. A claimed counterexample therefore leaves an otherwise-green snapshot
at OWED, which is what it is: a defect nobody has established yet.
Input validity is checked before anything is derived, because a predicate built
from an input we misread looks exactly like one we did not:
required id absent from the map -> MissingRequiredObservation
falsification bound to another commit -> FalsificationSubjectMismatch
The unreachable `None` arm left after the presence check maps to CANNOT_CHECK
rather than OWED, so even the impossible path refuses to invent an authoritative
absence.
completenessClaimed is materialized as false unconditionally; the input has no
field through which true could be constructed.
24 acceptance + 4 fixture tests pass. Workspace fmt, clippy -D warnings, and the
worker-gate-equivalent test run are clean. Frozen Step 0B fixtures, README and
validator untouched.
Refs #147
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013K3sUCBMGGjf5v4AqsW7Qk
Third review finding, and one floor below the previous two. #147 requires the predicate to "preserve provenance for each observation sufficiently to re-fetch/re-evaluate it later". The classifier was discarding provenance it had already been handed: NotProduced -> OWED, source None RateLimited -> OWED, source None Failed{reason} -> CANNOT_CHECK, reason LOST After serialization nobody can tell "nobody produced it" from "the reviewer was rate-limited", though those are two different observed facts. And "it failed" without a reason cannot be re-evaluated at all — which is the one thing the predicate is supposed to make possible. This is the same destructive collapse the headline rule forbids, one level down: several outcomes mapped onto one state, with the outcome thrown away. The falsification channel inherited the defect the moment it was added in RED-2: VERIFICATION_FAILED recorded that verification failed but never why. Contract change, behaviour not yet implemented: ObservationOut.acquisition = { status, reason? } status: AVAILABLE | NOT_PRODUCED | RATE_LIMITED | FAILED reason: required for FAILED Verification::VerificationFailed -> Verification::Failed { reason } FalsificationOut.verification = { status, reason? } State and acquisition outcome are deliberately separate fields: NOT_PRODUCED and RATE_LIMITED both yield OWED, and the predicate now records which path got there. RED-3 state: 3 failing (the three provenance witnesses), 25 passing, 4 fixture tests passing. Frozen Step 0B fixtures, README and validator untouched. Refs #147 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013K3sUCBMGGjf5v4AqsW7Qk
Turns RED-3 green. The classifier no longer discards provenance it was handed.
Each observation now reports its acquisition outcome alongside its state,
because several outcomes map to one state and only the state was surviving:
AVAILABLE -> PASS | FINDING | OWED (wrong SHA)
NOT_PRODUCED -> OWED
RATE_LIMITED -> OWED
FAILED { reason } -> CANNOT_CHECK, reason carried
so "nobody produced it" and "the reviewer was rate-limited" are once again two
readable facts rather than one indistinguishable OWED, and a failed acquisition
says why — without which #147's re-fetch/re-evaluate requirement cannot be met.
The falsification channel gets the same treatment: Verification::Failed carries
a reason and FalsificationOut serializes status plus reason.
The unreachable `None` arm now reports FAILED with an explicit reason rather
than a bare CANNOT_CHECK, so even the impossible path explains itself.
Also corrects a doc comment that RED-2 made stale: FalsificationFact was still
described as "already-established", which stopped being true the moment
`Claimed` was introduced — a claimed record is exactly a counterexample whose
verification is still owed.
28 acceptance + 4 fixture tests pass. Workspace fmt, clippy -D warnings and the
worker-gate-equivalent run are clean. Frozen Step 0B fixtures, README and
validator untouched.
Refs #147
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013K3sUCBMGGjf5v4AqsW7Qk
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Observations in, deterministic predicate out. No in-toto, no
actions/attest, no workflow, no GitHub API, no network, no local Git authority, no scheduler, no PR mutation.Six commits, three RED/GREEN pairs, none squashed or amended:
Each RED failed on exactly its own new assertions and nothing else, so every correction is isolated rather than a rewrite in disguise.
Round 3 — provenance preservation
#147 requires the predicate to preserve provenance "sufficiently to re-fetch/re-evaluate later". The classifier was discarding provenance it had already been handed:
After serialization, "nobody produced it" was indistinguishable from "the reviewer was rate-limited" — two different observed facts flattened into one word. And
CANNOT_CHECKwith no reason cannot be re-evaluated at all, which is the single thing the predicate exists to make possible. Same destructive collapse the headline rule forbids, one level down. The falsification channel inherited it the moment RED-2 addedVERIFICATION_FAILED.State and acquisition outcome are separate fields on purpose: several outcomes map to one state, and the predicate now records which path got there.
Round 2 — semantic state corrections
CLAIMEDcounted asFINDING. #147 definesFINDINGas an admissible verified defect; the wide surface means a counterexample may arrive from a surface that cannot carry a verdict, not that an unverified claim is a defect.The headline became a plain max over observations chained with falsification states, replacing a special case that could only ever raise
FINDING— which is why the old shape could not expressOWEDfrom this channel at all.Falsification bound to another commit →
ClassifierError::FalsificationSubjectMismatch. A wide surface means more kinds of surface, not defects of any commit in the universe.Nonestays admissible.Missing required observation →
ClassifierError::MissingRequiredObservation, soclassifyreturnsResult. Naming that unknownOWEDconverted a construction failure into a domain fact — the exact move this design refuses.completeness_claimedremoved from the input; the output materializesfalseunconditionally, so the forbidden state is unconstructible rather than merely discouraged.Input boundary
The Step 0B envelope is a test container, not the production schema:
Positive evidence is strict about commit binding; evidence bound elsewhere yields
OWED, never combination across SHAs. The headline never rewrites the vector; a stale head setssubject_staleandSTALEwhile each observation still reports what it said. Each call is an independent snapshot.The three frozen fixtures
Each asserts its hostile precondition from the JSON before any classifier expectation.
stale-review-wrong-shareview/codexOWED, four CI PASS, headline OWED;subject_stalefalse — the review is stale, not the snapshotfalsification-in-commentreview/codexPASS,review/coderabbitOWED, falsification preserved, headline FINDINGconflicting-review-surfacesreview/codexFINDING; the "no issues" comment has no channel into positive evidence and is not deleted eitherThe loader may not inject outcomes: a review carries a finding iff a review comment references it by
pull_request_review_id— structural, no body text read. Two harness mappings (check name → id, reviewer login → id) are labelled as such since #147 states neither.No historical witness
OWED,CANNOT_CHECKand every RED-2/RED-3 rule are exercised by constructed inputs, marked on the tests as logic-level unit witnesses. Step 0B is not extended. Pagination and reaction semantics remain absent by design.Scope scan
actions/attest,in-toto,reqwest,Command::new,std::process,GIT_DIR,tokio→ 0 hits. Dependencies:serde,serde_json.git diff origin/main -- tests/fixtures/github tests/github_fixture_integrity.rsempty.Local verification
Residual, recorded and NOT addressed here
source.kind+stable_idmay be insufficient to re-audit a mutable comment after it is edited. Closing that needs a chosen snapshot-or-digest rule, and picking one inside the pure classifier would be inventing a norm. It should close before the attestation envelope. Recorded as owed.Refs #147
🤖 Generated with Claude Code
https://claude.ai/code/session_013K3sUCBMGGjf5v4AqsW7Qk