Repository navigation
Description
Classification
- Change type: enhancement / run evidence contract
- Priority: P1
- Expected value: high — provides the action/outcome data needed by OwnAudit triage and later scheduling research
- Implementation risk: medium-high — incomplete records create misleading training and audit data
- Repository owner: 007 execution/orchestration layer
- Consumes identity/event contracts from:
PhysShell/Own.NET#266andPhysShell/Own.NET#268 - Production status: recommended improvement after the identity contract is stable
Problem
The current 007 run record correctly captures run-level metadata, task text, agent stdout, diff and per-step gate verdicts. That answers:
Did this isolated run pass its gates?
It does not yet answer:
What action was attempted?
Which finding(s) motivated it?
What did it cost?
Which verifier established the outcome?
Was the result confirmed, rejected, inconclusive or a regression?
Without action-level records, future triage/scheduling work would have to infer intent and outcome from prose/stdout. That is slower than a sedated sloth and less trustworthy than it sounds.
Goal
Add a versioned, append-friendly ActionRecord contract under each run record.
007 remains the executor and evidence producer. OwnAudit remains the consumer that interprets evidence and ranks/triages findings.
Proposed record shape
Equivalent fields:
struct ActionRecord {
schema: u32,
action_id: String,
run_id: String,
parent_action_id: Option<String>,
target_repo: String,
base_commit: String,
target_pattern_ids: Vec<String>,
target_occurrence_ids: Vec<String>,
target_lineage_ids: Vec<String>,
policy_id: String,
action_kind: ActionKind,
input_digest: String,
started_at: String,
finished_at: String,
cost: ActionCost,
outcome: ActionOutcome,
verifier_results: Vec<VerifierResult>,
artifact_refs: Vec<ArtifactRef>,
}Exact Rust types are reviewable. The information must remain explicit.
Action kinds
Initial vocabulary:
HumanReviewRequested
JudgeRequested
DeepAnalyzerRun
RuntimeScenarioRun
TestGenerationRequested
FixProposed
FixApplied
FixRejected
FindingSuppressionProposed
FindingSuppressionApplied
NoAction
NoAction is important for unbiased evaluation: actions considered but skipped by policy must not disappear from the dataset.
Outcomes
Initial vocabulary:
Confirmed
Rejected
Fixed
NoChange
IntroducedRegression
TimedOut
Cancelled
PolicyDenied
OperationalError
Inconclusive
A successful process exit is not automatically Confirmed or Fixed. Outcome requires verifier evidence.
Verifier result
Each verifier result should record:
verifier_kind
verifier_name/version
command or policy step identity
input/artifact digest
verdict
exit code
started/finished timestamps
log/artifact refs
Examples:
- re-audit reports target occurrence absent with comparable config;
- regression tests pass/fail;
- runtime scenario confirms retention;
- diff policy rejects unrelated changes;
- human disposition is pending/recorded externally.
007 must record what the verifier said, not reinterpret domain truth beyond its execution contract.
Cost record
Use measured fields where available:
wall_time_ms
agent_cost_usd
input_tokens
output_tokens
cpu_time_ms
peak_rss_bytes
human_minutes # only when supplied, never invented
Unknown values stay null/unknown. Do not convert missing cost into zero.
On-disk layout
Suggested additive layout:
runs/<target>/<run-id>/
meta.json
actions/
<action-id>/
action.json
verifier/
<name>.json
<name>.log
artifacts/
...
meta.json may carry only an action summary/index. The canonical detailed records live under actions/ to avoid turning one JSON file into a junk drawer.
Identity and provenance
action_idis stable within the stored record and collision-resistant.- Every action pins
run_id, base commit, policy/config identity and input digest. - Finding links use the final pattern/occurrence/lineage contract; legacy runs may omit them explicitly.
- Artifact references include digests and paths relative to the run record.
- Unknown schema or digest mismatch fails loudly during read/export.
- Sensitive agent output/secrets remain subject to the existing redaction/output-filter policy; action records must not become a new secret sink.
Export seam
Add a deterministic export suitable for OwnAudit ingestion, for example:
o7 evidence export --run <id> --format json
The export should contain normalized action/outcome/provenance data, not raw secrets or unbounded stdout.
It must be possible to export all eligible runs incrementally without duplicating already-ingested actions.
Backward compatibility
- Existing run records remain readable.
- Missing
actions/means legacy run-level evidence, not a parse failure. RunMeta.schemais bumped only if required by a breaking change; additive action references should prefer optional/versioned fields.- Existing
o7 runPASS/FAIL exit behavior is unchanged.
Required controls
- One run with two distinct actions records both independently.
- Two actions target the same pattern but different occurrences.
- Fix process exits zero but re-audit still finds the issue →
NoChange, notFixed. - Tests pass but unrelated diff violates policy → verifier conflict remains visible.
- Action times out → timeout outcome plus partial verifier artifacts retained.
- Missing cost fields remain unknown.
- Duplicate export/ingest uses stable IDs and does not duplicate actions.
- Artifact digest mismatch is rejected.
- Legacy run without actions remains readable/exportable with explicit limitations.
- Redacted/filtered fields never leak through the normalized export.
Acceptance
- Versioned ActionRecord/VerifierResult/ActionCost contracts are documented and implemented.
-
o7 runcan emit one or more action records without changing current gate semantics. - Actions link to finding identities when supplied.
- Outcomes are verifier-backed and distinguish process success from domain success.
- Measured costs are recorded; missing costs are never treated as zero.
- Normalized deterministic export exists for OwnAudit.
- Existing run records and CLI exit behavior remain compatible.
- Digest/provenance and secret-redaction controls are tested.
Non-goals
- No ranking or domain triage inside 007.
- No automatic disposition of findings.
- No contextual bandit, CFR or RL scheduler in this issue.
- No new sandbox/enforcement policy.
- No migration of OwnAudit's ledger into the private 007 repository.
- No inference of actions/outcomes from unstructured stdout when explicit records are unavailable.
Activity
PhysShell commentedon Jul 13, 2026
Primary consumers of this export are
PhysShell/Own.NET#268(canonical Evidence Ledger) andPhysShell/OwnAudit#41/#42(triage and later calibration evaluation). 007 should emit measured facts and verifier outcomes; it should not absorb their domain disposition policy.PhysShell commentedon Jul 13, 2026
Companion correctness issue: #43 defines replayable run semantics, fail-closed required-gate reduction, event/artifact digest verification, and
o7 replay.Scope boundary:
When both exist,
ActionRecord/VerifierResultshould reference the run replay report/event-chain root rather than implementing a second integrity or verdict reducer.PhysShell commentedon Nov 20, 2026
Restate v1.8.1: effect receipt tiers and canonical invocation identity (20.11.2026)
Research page · release notes.
For action-level evidence, distinguish (1) logical invocation/idempotency key, (2) one specific invocation incarnation / log position (
canonical_idin Restate 1.8.1), (3) durable completed step result, (4) remote side effect actually committed and independently acknowledged. These are not interchangeable. Especially a request with lost response may have applied at remote even when durable step receipt is absent; record UNKNOWN pending reconciliation.Do not copy a Restate server schema or adopt a second event store. Only compare this tiered receipt model against existing exact-SHA event IDs and run-generation fences. If current IDs already disambiguate and trace external effects, STOP. Release-note identity is version-specific; older/restored invocations may carry null/placeholder canonical IDs; don't assume a non-null new-format ID on every legacy record.
PhysShell commentedon Nov 20, 2026
TESTed donor: TESTED-03 / structured judge receipts (20.11.2026)
Research page · evaluation · output golden tests.
TESTed emits JSON Lines with nested start/close events for judgement, tab, context, testcase and test, and stores expected/generated result with status. It uses distinct oracles for return value, text, stderr/stdout, exception, exit code and file. A failed oracle is reported rather than silently counted as success.
TESTED-03 / STRUCTURED-JUDGE-RECEIPTS / KILL-FIRST: map a 6-case synthetic run into current 007 action/evidence schema. Preserve
run_id, attempt ID, source SHA, suite SHA, language capabilities, oracle identity/version, observable result vs expected, explicit terminal states: compile fail, runtime exception, wrong value, unsupported suite feature, timeout, oracle error, environment failure, NOT_TESTED/UNKNOWN, pass. Inject truncated JSONL and retry/replay to check idempotent ingestion and no phantom PASS; expected semantics and full per-test evidence should round trip without info loss.GO only if existing receipts misclassify a demonstrated case or omit causally useful evidence. STOP if current exact-SHA event log already represents all states. No new CI gate or duplicate event store. A passing subset is bounded test evidence, not proof of general correctness.