Repository navigation
Description
Problem
#145 exposed a separate engineering problem that must not be solved inside the resolver preregistration itself.
The resolver semantics stopped moving at f748f66, while subsequent review rounds increasingly hardened a Markdown-embedded Git-history checker. Category C ("a mechanical check advertised as PASS/coverage can false-PASS") turned out to be recursive: checker → bootstrap → repository selection → graph controls → traversal → neutralizers → helpers. Every finding could honestly remain category C, so the process had no fixed point.
At terminal report head ed7969c4636362bcfc248bcb9957e140ca899f8c, #145 explicitly demotes that checker:
The preregistration does not claim its evidence machinery is self-verifying.
Its PASS is not part of the RS-P0 closure basis. Reproduced defects O-1/O-2 remain OPEN rather than triggering another repair round.
The useful problem that remains is narrower:
For an explicitly frozen Git commit X and an explicitly enumerated closure policy P, produce an immutable, authenticated snapshot of the observations actually obtained about X, without claiming that P is complete or that X is globally correct.
This matters especially for agent-driven development, where review comments, bot walkthroughs, CI results, mutable comments, retries, and multiple model reviewers otherwise create a provenance problem large enough to become its own project.
Step 0 — preserve the regression corpus before the current session dies
The #145 review session built a regression corpus in scratch storage. The descriptions survive in the evidence record and closure comment, but the executable fixtures did not.
Preserve the material outside #145's branch. Do not modify #145 to do this.
Step 0 splits in two. The split is not bookkeeping — it marks the boundary of the tool: Git-level fixtures demonstrate why closure authority must not rest on local Git, while API-level fixtures test the classifier itself and must run with neither Git nor network.
Step 0A — historical Git-level failure corpus
Purpose:
preserve executable recipes demonstrating why local Git history
must not be the closure authority.
Status:
PRESERVED and MERGED (PR #148 -> 7ed9e49)
Contents:
build recipes (build-corpus.sh), not 170 MB of derived .git state
the checker used historically (check.py)
hostile PATH / repository selection / ref / config / history cases
merge shapes
shallow cases
O-1 and O-2 recipes
Scale:
13 Git-level scenarios / 14 assertions
Important:
O-1 is NOT a discriminating witness.
RECIPE PRESERVED / DISCRIMINATING WITNESS OWED.
Removing the docs/tasks tree makes the block exit 2 — the correct outcome reached for the wrong reason, because the history enumeration fails before blob() is consulted and the mechanism under test never runs. A real discriminator must break only a tree the merge comparison consults: reachable from a merge parent, and from no commit the contract-limited enumeration diff-trees.
Step 0B — GitHub/API classifier fixtures
Purpose:
preserve API-level cases against which the pure classifier is tested.
Status:
OWED
Contents:
stale reviewer artifact bound to the wrong SHA
non-verdict surface carrying a real falsification claim
same-vendor / same-SHA conflicting surfaces
These must not be forced into build-corpus.sh. They belong as serialized API responses:
tests/fixtures/github/
stale-review-wrong-sha.json
falsification-in-comment.json
conflicting-review-surfaces.json
Fixture rules (mandatory)
Both were derived from practice during #145 and #148, not from theory.
1. A negative fixture counts as verification only if it DISCRIMINATES.
control, defect absent -> expected result
defect active -> wrong result
defect repaired -> expected result
A fixture satisfying only the first and third points is preserved as a recipe, never scored as a witness. This is O-1's exact status.
2. Before evaluating the asserted outcome, an adversarial fixture must positively prove that the hostile condition it claims to construct actually exists.
Two construction failures while building the 0A corpus:
| Failure | Consequence |
|---|---|
remote-tracking refs are not copied by clone |
fixtures silently lacked the intended subject |
--depth is silently ignored for local-path clones |
the "shallow" fixture was a complete repository |
One class:
fixture construction fails -> test still runs -> hostile condition absent -> VACUOUS PASS
The --depth case is the sharper specimen: green not because the system withstood an attack, but because the attack was never constructed. Agentic development can produce such tests in industrial quantities, so the precondition (rev-parse --is-shallow-repository, presence of the replace ref, the marker file, …) must be asserted before the property is evaluated.
Editing note. Placeholders in this issue use [square brackets], not angle brackets. GitHub's sanitizer strips a letter-initial <word …> as an HTML tag even inside a fenced block, which silently deleted three lines from Expected implementation surface twice. <40-hex> survives only because a digit-initial token is not parsed as a tag. Same failure class as the rest of this issue: the artifact stored is not the artifact typed.
Non-goal: do NOT build a "proof that a PR is correct"
This effort MUST NOT claim any of the following:
- global correctness of a PR;
- completeness of the configured observation set;
- completeness of Codex/CodeRabbit review;
- absence of unknown defects;
- that a clean review proves the artifact correct;
- that the closure tool verifies its own trusted base;
- that an attestation authorizes merge.
The intended claim is only:
Every observation explicitly required by policy P has the recorded state for immutable subject X, under the declared trusted base and assumptions.
Architecture
GitHub API / GitHub Actions
|
| exact PR head, checks, submitted reviews
v
small read-only classifier
|
| closure predicate JSON
v
in-toto Statement / GitHub artifact attestation
|
v
immutable authenticated closure snapshot
Critical boundary
Local Git history is NOT the authority for closure observations.
If GitHub already owns an observation (PR head identity, workflow/check status, submitted review object, review commit binding), obtain it from the GitHub API / Actions metadata rather than reconstructing it through a local checkout.
This deliberately removes the attack surface that consumed #145:
GIT_DIR/GIT_WORK_TREE;- replace refs;
- grafts;
- alternates;
- repository-local config;
- fsmonitor;
- signature helpers;
- shallow-history interpretation;
- ambient cwd / wrong repository;
- shell pipelines turning acquisition failure into an empty set.
A local checkout may still be used by an individual CI job for the property that job tests, but it must not become the authority for the closure aggregator's GitHub-owned observations.
Declared trusted computing base (TCB) v0
The tool MUST name rather than recursively eliminate its trusted base.
Initial TCB:
- GitHub repository / PR / review / check object semantics;
- GitHub Actions runner and process boundary for the attestation workflow;
- GitHub OIDC / Sigstore identity used by GitHub artifact attestations;
- pinned
actions/attestaction; - the closure policy definition itself;
- the small classifier implementation;
- HTTPS/API transport and authentication supplied by GitHub Actions.
The tool does not earn the right to say "I trust only what I fully checked myself".
Changing the TCB is a policy/version change, not an invitation to recursively harden the old version forever.
State model
Each required observation has one independent state:
PASS
admissible positive evidence exists for frozen head X
FINDING
an admissible verified defect exists for X
OWED
required evidence/verdict does not yet exist; nobody has produced it
CANNOT_CHECK
an attempt to obtain a required observation did not yield a trustworthy answer
STALE
current PR head differs from expected X before or after evaluation
Top-level headline precedence is presentation only:
STALE > FINDING > CANNOT_CHECK > OWED > PASS
Never destructively collapse the per-observation vector into the headline.
Example:
frozen_head: X
observations:
ci/worktree-verifier: PASS
local/special-check: CANNOT_CHECK
reviewer/codex: FINDING
reviewer/coderabbit: OWED
headline: FINDINGIf the Codex finding disappears on a later snapshot but the local check remains unavailable, the next headline is CANNOT_CHECK, not PASS.
Required-set rule
Every snapshot MUST enumerate the observation universe it required.
policy:
id: 007/closure/v1
required_observations:
- ci/worker-gate
- ci/dependency-policy
- ci/dependency-advisories
- ci/worktree-verifier
- review/codex
- review/coderabbit
completeness_claimed: falsePASS means every observation in this explicitly enumerated set passed.
It does not mean the set itself is complete.
A digest of the canonical policy representation may be recorded for convenience, but the enumeration is authoritative; the digest must not become a new authority mechanism.
Reviewer evidence rules
For closure verdict accounting:
- a submitted review object must exist;
- its
commit_idmust equal frozen head X; - its current state/body must encode the required verdict;
- head must still be X when closure is emitted.
Comments, acknowledgements, walkthroughs, command replies, reactions and "processing" notices are not verdict evidence.
However, falsification has a wider surface than positive closure evidence:
A concrete defect claim from any surface must be checked if it is verifiable, even when that surface is inadmissible as a closure verdict.
This preserves the historical distinction:
- strict form for positive closure evidence;
- any checkable counterexample may falsify.
Rate-limited reviewer → OWED, never PASS/FAIL.
Observation acquisition failure → CANNOT_CHECK, never OWED and never "nothing found".
Wrong-SHA review object → inadmissible for X; never combine CI/reviews across SHAs.
Snapshot / predicate shape
V0 should use a small custom predicate rather than forcing the model into a standard predicate that does not preserve the five-state vector.
Conceptual payload:
{
"schemaVersion": 1,
"repository": "PhysShell/007",
"pullRequest": 123,
"gitCommit": "<40-hex>",
"policy": {
"id": "007/closure/v1",
"requiredObservations": ["..."],
"completenessClaimed": false
},
"observations": [
{
"id": "ci/worker-gate",
"state": "PASS",
"source": {
"kind": "github-actions-check",
"stable_id": "..."
}
}
],
"headline": "OWED",
"knownDebts": ["..."]
}The predicate MUST preserve provenance for each observation sufficiently to re-fetch/re-evaluate it later.
in-toto / GitHub attestation role
In-toto is not the classifier and not the merge policy.
Its role is the authenticated envelope:
immutable subject X
+ typed predicate Y
+ authenticated producer identity
= durable attestation
V0 should use GitHub's official artifact-attestation path (actions/attest) with a custom predicate type such as:
https://github.com/PhysShell/007/attestations/closure/v1
Do not implement custom cryptography.
Do not adopt in-toto layouts or a general supply-chain framework unless a concrete consumer later requires them.
Do not use a prototype verifier as production authority merely because it has an in-toto name attached to it.
Subject binding
Prefer direct immutable commit identity in the logical model.
If the chosen GitHub attestation action requires a SHA-256 artifact subject rather than a Git commit digest, V0 MAY create a deterministic descriptor such as:
{"repository":"PhysShell/007","gitCommit":"<40-hex>"}and attest the SHA-256 of those exact bytes.
If this indirection is used:
- canonical serialization must be specified;
- descriptor bytes must be preserved with the attestation/output;
- no semantic claim may depend on a mutable path or branch name;
- this is representation convenience only, not new authority.
Workflow shape
Add a separate manual workflow, conceptually:
.github/workflows/closure-attestation.yml
Inputs:
pr
expected_sha
policy_id
Execution:
1. read PR head from GitHub API
2. if head != expected_sha -> STALE, do not emit positive attestation
3. fetch required observations for expected_sha
4. classify every observation independently
5. re-read PR head from GitHub API
6. if head != expected_sha -> STALE
7. materialize deterministic subject + predicate
8. sign/store with actions/attest
9. print compact human report + attestation identifier
No internal scheduler. No polling loop. No automatic reviewer request. No merge action.
A later invocation creates a new snapshot; it never rewrites an old attestation.
Sequencing / provenance invariant
Do not retrofit this tool into #145's closure basis.
The review process must not author part of the evidence basis it is independently evaluating. Doing so would falsify the preregistration's central provenance claim that the observations and decision rule were frozen before the implementation existed.
#145's terminal report at ed7969c remains what it is: no clean reviewer verdict was obtained or requested at that head, the embedded checker is not a closure oracle, and O-1/O-2 plus the existing owed items remain open.
The new tool starts only as a follow-up effort.
Build order
1. preserve Git corpus recipes DONE (PR #148)
2. merge / durable preservation DONE (7ed9e49)
3. add the three API-level recorded fixtures step 0B
4. build the PURE CLASSIFIER against them NO in-toto yet
input JSON observations -> classifier -> closure-predicate.json
5. historical nasty cases behave as specified
6. ONLY THEN wrap the output: actions/attest / in-toto envelope
Step 4 must precede step 6. The dull thing must exist and survive the historical cases first; otherwise the attestation layer starts influencing classifier semantics, which is precisely what keeping in-toto as an envelope is meant to prevent.
Acceptance corpus / required negative cases
The first implementation MUST include regression tests derived from the #145 failures, not only happy-path unit tests.
At minimum:
- current head != expected head at start →
STALE; - head moves between initial and final read → whole snapshot
STALE; - submitted clean review exists but
commit_id != X→ not positive evidence for X; - clean sentence in mutable comment → not a verdict;
- concrete bug in command reply/non-verdict surface → still surfaced as falsification input;
- reviewer rate-limited / no verdict object →
OWED; - observation API/tool failure →
CANNOT_CHECK, not empty-set success; FINDINGplus anotherCANNOT_CHECKpreserves both states; headline only isFINDING;- next snapshot removes FINDING but keeps CANNOT_CHECK → headline becomes
CANNOT_CHECK, not PASS; - PASS snapshot explicitly enumerates required set and says
completenessClaimed=false; - CI and reviews from different SHAs cannot combine;
- same vendor/same SHA conflicting surfaces: submitted commit-bound review object controls verdict accounting, but a concrete defect claim from another surface is still checked;
- wrong local repository / hostile Git state is irrelevant to GitHub-owned closure observations because the aggregator does not use local Git as authority;
- known debt remains debt after clean reviewers; clean review does not magically discharge W8/W9/etc.;
- closure snapshot does not authorize merge.
Also preserve O-1/O-2 and the earlier hostile-Git fixtures as tests of the old approach's failure mode, so the new design can demonstrate that these inputs are outside its authority path rather than endlessly adding neutralizers.
Expected implementation surface
Keep V0 deliberately small:
.github/workflows/closure-attestation.yml
[small classifier module/tool]
[closure predicate/schema doc]
[regression fixtures/tests]
The classifier should be read-only and should not invoke local git for GitHub-owned observations.
If the implementation grows into a generalized evidence-governance framework, stop and split it. A useful target is a few hundred lines of classifier/workflow logic, not another subsystem hidden inside documentation.
Definition of done for V0
- Step 0A — Git-level regression recipes/checker preserved and discoverable. 13 scenarios / 14 assertions. O-1 recipe preserved; discriminating witness OWED. (PR Preserve #145 closure regression corpus recipes (issue #147 step 0A) #148, merged as
7ed9e49) - Step 0B — GitHub/API-level regression fixtures preserved: wrong-SHA reviewer artifact; non-verdict surface carrying falsification; same-vendor/same-SHA conflicting surfaces.
- Closure policy explicitly enumerates required observations and disclaims completeness.
- Per-observation five-state vector is preserved; headline is derived only.
- Initial and final exact-head checks are API-backed and fail stale.
- GitHub-owned observations come from GitHub API/Actions metadata, not local Git reconstruction.
- Acquisition failure cannot become an empty-set / negative observation.
- Every adversarial fixture positively proves its hostile condition exists before the asserted outcome is evaluated.
- Every negative fixture discriminates the mechanism it claims to test, or is recorded as a recipe with the witness OWED.
- Reviewer verdict accounting is commit-bound and distinguishes verdict surface from falsification surface.
- Known debt is represented independently of reviewer cleanliness.
- A deterministic closure predicate is emitted.
- Predicate is authenticated/stored as an in-toto-compatible GitHub attestation using pinned official tooling.
- Attestation verification is demonstrated independently of the producing step.
- No scheduler, merge action, reviewer-request action, or mutation of the PR is part of V0.
- Tool is not used retroactively as evidence for RS-P0 (attempt 2): resolver observation contract preregistration — re-derived, evidence split out #145.
- Regression corpus covers the historical false-PASS and stale-evidence failures listed above.
Success statement
V0 succeeds if, for a frozen commit X, a later reader can answer mechanically:
Which observations did policy P require, what state did each observation have, what known debt remained, which immutable subject was judged, and which authenticated workflow emitted this snapshot?
It does not need to answer:
Is X correct?
That stronger question belongs to the tests, reviewers, formal arguments and human/project policy whose results the snapshot records.
Activity
tools: record corpus status honestly and the two rules it earned
PhysShell commentedon Aug 16, 2026
Step 0 splits into 0A and 0B
Step 0 is done for the Git-level half only. The split is worth making explicit because it is not bookkeeping — it marks the boundary of the tool.
The two halves do different jobs:
So the three reviewer scenarios should not be forced into
build-corpus.sh. They belong as serialized API responses:Two rules to adopt as requirements
1. A negative regression fixture must discriminate the mechanism it claims to test.
O-1 currently has only the first and third points, so its honest status is
RECIPE PRESERVED / DISCRIMINATING WITNESS OWED— neither PASS nor FAIL. Removing thedocs/taskstree makes the block exit 2, which is the right outcome for the wrong reason: the history enumeration fails beforeblob()is consulted, so the mechanism under test never runs. Better to carry that as owed than to bank a green test that is green because execution died a floor earlier.2. Every adversarial fixture must carry a positive control proving the hostile condition actually exists before the asserted result is evaluated.
Two construction failures while writing
build-corpus.sh:clone--depthis silently ignored for local-path clonesOne class:
The second is the more dangerous: it produced a green result for a scenario that did not exist. Harnesses here must assert the precondition (
rev-parse --is-shallow-repository, presence of the replace ref, the marker file, …) before evaluating the property.Sequencing correction: classifier before in-toto
Step 4 must precede step 5. What should exist first is the dull thing:
evaluated against the historically nasty cases. Only then does
closure-predicate.jsonget handed toactions/attest. In-toto stays the envelope, as agreed, and never gets to influence classifier semantics — starting with the attestation layer is exactly how it would.Status correction
The corpus should not be reported as "15/15". Actual:
build-corpus.shexits 0 because every case matches its recorded expectation, and O-1's recorded expectation says the shape does not reach its mechanism.Generated by Claude Code
Merge pull request #148 from PhysShell/claude/closure-corpus-preserve
Merge pull request #149 from PhysShell/claude/closure-fixtures-0b
test: drop an integrity relation the frozen README does not state
Merge pull request #150 from PhysShell/claude/fixture-integrity-valid…
test: correct falsification, input-validity and completeness semantic…
feat: implement corrected falsification and input-validity semantics …
test: predicate must preserve acquisition provenance (RED-3)
feat: preserve acquisition provenance in the predicate (GREEN-3)
Merge pull request #151 from PhysShell/claude/closure-classifier
PhysShell commentedon Aug 20, 2026
Reviewer verdict delivery-surface witness
Status: field observation and prior art for a future slice.
This is not a normative amendment to this issue, not a change to any frozen contract, and not evidence affecting the closure of #153. Nothing here alters the acceptance surface of an open or merged PR. It is recorded so that the observation survives the conversation it happened in, and so the acquisition / classifier-provenance slice starts from it rather than rediscovering it.
The observation
Recorded live on #153 between 2026-08-18 and 2026-08-20, from an external automated reviewer. No part of it was constructed.
Why the asymmetry between B and C is the load-bearing part
Both are the same actor on the same surface, and to a human reading the PR both look like "the reviewer reviewed". Mechanically they are not two strengths of the same thing:
Claimedand can only become anything stronger through the verification-witness binding, which is OWED.closure-source-provenance-v1.md§8.5 gives the issue-comment projection no commit field at all, while §8.3 makes/commit_ida required field of a submitted review. A SHA written in prose inside a mutable body is a reference to a mutable object, which §3 explicitly refuses as provenance.So a positive verdict delivered by comment cannot satisfy the positive review observation, cannot produce a wrong-SHA
OWEDeither (there is nocommit_idto disagree with head — not a wrong one, no field), and must not be counted toward PASS accounting.The failure mode this guards against is specific and tempting: an adapter that resolves "is this a reviewer verdict?" by checking the author, because the author is the same bot in both cases.
Why E is stronger than "the object changed"
E is not another instance of D. Its value is the loop.
Two snapshots at the edges of an interval can be equal while the object was, in between, in a state that something acted on. So neither a stable id, nor "fetch now and compare later", nor even "compare the endpoints" recovers an observation history. What is required is the immutable snapshot taken at the moment the observation was used — which is
closure-source-provenance-v1.md§3 and §11 arriving from a direction the specimens did not cover: the existing corpus witnesses monotone edits, and this one is non-monotone.Where this came from
#153 froze a contract asserting that a stable GitHub object id is not an immutable evidence version. The independent review of that PR was delivered through issue comments, one of which was then edited in place to the opposite operational conclusion, while a second cycled through four states under a single id.
The corpus arrived on its own. Worth using it before the acquisition adapter makes the same assumption in code.
Sources
b0952b5e15e5e3ee239f7c6fbefafb67a4598c2ddocs/architecture/closure-source-provenance-v1.md§3, §8.3, §8.5, §11 — merged as4d60db64…docs/architecture/closure-redaction-policy-v1.md§19 — proposed on docs: closure redaction / secret-retention contract V1 (contract freeze + specimens) #153, not mergedtests/fixtures/closure-provenance/mutable-comment-v1.json— specimen A, the monotone caseGenerated by Claude Code