Deepen Harness stream interpretation behind one evidence interface #640

What to build

Deepen Harness stream interpretation so a change to one Harness's event format has one place to land. Its stream decoder must supply progress, Model evidence and safeguard evidence together; callers must no longer interpret that Harness's raw fields again. Preserve the behavior seen by Sessions, Session log recording, Model and Effort checks, release summaries, and Run notifications.

This is one standalone Ticket against the Architecture review's base, main at 86c3efc. It is a refactor of existing behavior, with no new command or setting.

Acceptance criteria

• Every existing Harness stream decoder supplies progress and semantic evidence through the same internal interface. Live Model and safeguard field interpretation lives with the dialect implementation; shared Model and Security policy contains neither JSON parsing nor Harness-name selection.
• Interpretation's public interface and ordinary progress, completion reports, final messages, diagnostics, session identities, usage and killed-work behavior remain compatible for all six Harnesses.
• Observed Models remain ordered and deduplicated, include delegated Claude/Grok assistant evidence and Claude result usage evidence, and exclude empty names, synthetic answers and API-error assistant messages. Requested Models never become observations.
• Security Model progress remains main-loop-only, suppresses consecutive repetitions, and preserves Codex's requested/default wording. Evidence from repeated initialization events and delegated results survives the decoder's existing progress/completion guards.
• Existing Claude and Codex safeguard classifications remain sticky only while Security policy is enabled. Ordinary sessions, checks and summaries retain their existing outcomes. Enabling or resetting Security policy does not reset observed Models or retroactively enforce earlier refusal evidence.
• Session recording can still checkpoint observed stream Models before completion. Interruption still suppresses completion reports and retained acquisition; completed retained evidence remains authoritative even when its Model list is empty.
• Behavioral fixtures through the existing Interpretation interface cover the Model, refusal and initialization distinctions below. Existing retained-record, interruption, Session recording, Run notification and Security-run regressions continue to pass. Remove obsolete raw-observer code and any tests superseded by equivalent interface coverage.
• The completed change passes formatting, linting and the Rust test suite. No new dependency, Harness launch policy, permission rule, Model source, retry mechanism, or public output format is introduced.

Blocked by

None (can start immediately).

Decisions

Design tree and settlement

The tree is: scope → decoder seam → per-line interface → common policy and dialect extraction → completion/recording compatibility → verification and delivery size. The rounds below settle that entire tree; no human decision remains.

Round 1: scope and ownership

1. Deepen the existing decoder seam. Six real Harness adapters already select five stream decoder implementations, with Claude and Grok sharing private dialect rules. Keep that arrangement: the dependency here is in-process, and additional adapter abstractions would add no variation. The gain is locality of protocol knowledge, not a claim about parsing speed.
2. Keep Interpretation as the caller and test interface. Sessions, checks and summaries already use it, while Session log recording reads its live Models. Keeping this seam gives the change leverage across those callers without making them learn a new interface.
3. Keep retained acquisition and process ownership outside stream decoding. Existing retained-record reconciliation, interruption checks, process stopping and Session recording have distinct lifecycle responsibilities. Their current contracts settle this choice; moving them would widen a small refactor unnecessarily. ADRs 0011, 0013 and 0015 remain intact.

Round 2: interface and policy

4. Keep two decoder operations: condense and consuming completion. Condense still accepts one raw line, but returns an owned StreamUpdate containing the existing progress lines and an ordered list of semantic evidence. Completion still returns the existing Facts. This hides the entire live protocol behind one operation while avoiding new state inspection methods.
5. Use only three evidence variants. An observed Model carries its name and either main-loop or attribution-only scope; a requested-Model fallback carries no observed name; a safeguard refusal carries the existing typed refusal category. Main-loop evidence contributes both attribution and eligible Security progress. Attribution-only evidence contributes attribution alone. These facts cover current uses without a general event system.
6. Shared policy owns accumulation and enforcement. Interpretation applies evidence immediately, retains first-observed Model order, deduplicates names, excludes empty and synthetic names, and makes Models available before completion. Optional Security policy owns last-progress-Model suppression, requested/default wording and sticky refusal enforcement. It consumes semantic facts rather than raw fields. Decoders extract facts even when Security policy is off; only active Security policy enforces them. Calling for_security resets Security state, retains observed Models, and does not replay past evidence, preserving current behavior.
7. Preserve ordering and tolerant input. Unknown or malformed lines return an empty update; all JSON shapes currently tolerated remain tolerated. Evidence extraction happens before dialect-specific guards and early returns, and shared Security progress is appended after ordinary progress. This preserves facts that the current independent observers see even when progress/completion ignores an event. No new error mode is added.
8. Choose an owned return over the other designs. A callback observer would add another internal interface and callback-order knowledge for no present additional consumer. Passing a parsed value plus mutable collector would couple adapters to shared policy state and its drain order. The owned return makes protocol facts explicit and keeps policy out of adapters; its small allocation cost is acceptable for the existing line-at-a-time workload. No performance requirement depends on eliminating that cost.

Round 3: dialect and lifecycle contracts

9. Preserve each dialect's evidence predicates exactly. A protocol change must not silently become a behavior change:

10. Preserve the completion contract. Populate the report's stream Models from shared accumulation before existing retained reconciliation. A present retained record replaces that evidence, including with an empty set; an absent record preserves it. Existing interruption checks still yield no completion report or retained read, leaving the recording checkpoint intact. Execution failure, terminal turn failure and typed Security refusal retain their existing outcome/report separation and diagnostics. This protects the ownership work recently landed for Session logs.
11. Keep protocol-free policy implementation private. Model accumulation and Security policy may remain small private modules, but remove their raw observers and Harness switches. The existing exported refusal type remains available to callers that classify failures. No caller gains access to internal evidence variants, and no new domain term or changed definition requires a CONTEXT update. This reversible internal design needs no new ADR.

Round 4: verification and delivery

12. Test through Interpretation, replacing rather than layering. Use real dialect adapters and recorded event fixtures to assert progress, observed_models, reports, outcomes and typed refusals. Cover ordinary and Security interpretations of the same fixtures; root and delegated assistants/results; synthetic/API-error answers; repeated Agy init and Codex starts; Codex requested versus default fallback with no observations; sticky refusals versus harmless marker discussion; and malformed input. Assertions should survive changing private evidence representation. Existing protocol progress tests already cross this seam and should survive.
13. Retain higher-seam regressions where they prove lifecycle behavior. Keep Session recording's interruption and empty-authoritative-retained tests, Run notification's delegated attribution test, and Security-run tests for requested Model progress across Resume and refused reproductions. They prove integration beyond per-line interpretation rather than duplicating internal type checks. No real external Harness calls are needed: existing local fixtures and retained-record stand-ins suffice.
14. Publish one Ticket with no blockers. The work changes one internal return shape, migrates five decoder implementations and their common policy, and verifies existing callers. It fits a fresh context window; splitting it would leave two evidence paths alive or publish a mechanical type change without its behavioral result. The top issue is labelled only needs-triage; thirdshift determines readiness afterward.

Review validation

Baseline checks on the reviewed head, before implementation:

• cargo test --bin thirdshift harness::: 127 passed.
• cargo test --bin thirdshift session::recording::tests: 11 passed.
• cargo test --test run_notification a_notification_includes_delegated_models_and_result_usage_without_synthetic_answers -- --exact: 1 passed.
• cargo test --test security_run codex_logs_the_requested_model_for_each_security_session_and_resume -- --exact: 1 passed.
• cargo test --test security_run refused_security_reproductions_keep_the_record_and_stop_before_the_next_finding -- --exact: 1 passed.

These establish existing contracts, not validation of an implemented refactor. The implementation must run the acceptance checks above.

Deletion test and priority: Removing the separate raw observers concentrates their protocol knowledge in the existing dialect implementations; removing the deepened decoder would distribute it back across caller policies. This is the top uncovered Strong recommendation because recently changed Model and refusal behavior already requires reading the same events in three places, and existing contracts settle its complete design.

Coverage: Open issues were checked by title and body without label filtering, then refreshed before publication. #584 concerns retry policy, #428 additional Harness/Model pairs, #429 reviewer selection and #588 stopping-stage reporting; none proposes this deepening of live stream evidence. No ADR conflicts.

Architecture review

Strength: Strong · Dependencies: in-process

Files: src/harness/interpretation.rs, src/harness/interpretation/models.rs, src/harness/interpretation/security.rs, src/harness/interpretation/stream.rs, src/harness/codex/stream.rs, src/harness/agy/stream.rs, src/harness/muse/stream.rs, src/harness/opencode/stream.rs, src/harness/interpretation_tests.rs, src/session/recording/tests.rs, tests/run_notification.rs, tests/security_run.rs

Before

flowchart LR
  Recording["Session log recording"] --> Interpretation["Harness interpretation"]
  Interpretation --> Decoder["Harness stream decoders"]
  Interpretation -. "raw dialect fields" .-> Models["Model attribution"]
  Interpretation -. "raw dialect fields" .-> Security["Security session policy"]
  classDef leak stroke:#dc2626,stroke-width:2px;
  class Models,Security leak

After

flowchart LR
  Recording["Session log recording"] --> Interpretation["Harness interpretation"]
  Interpretation --> Decoder["Harness stream decoders"]
  Decoder -- "progress and semantic evidence" --> Interpretation
  Interpretation --> Models["Model attribution"]
  Interpretation --> Security["Security session policy"]
  classDef deep stroke-width:3px;
  class Decoder deep

Problem: The Harness stream decoder module is shallow for evidence: its caller rereads raw dialect fields for Model attribution and Security session policy.

Solution: Deepen its interface to return semantic evidence beside progress, placing protocol implementation behind the existing decoder seam and keeping common policy outside each adapter.

Wins

• locality: protocol changes in one adapter
• leverage: one interface supplies all evidence
• depth: fewer caller protocol assumptions
• Tests keep the existing interface