Repository navigation
Spec: Run metrics, the signals each Run leaves, kept apart - #1141
Merged
Merged
Conversation
Keep Run metrics in completed Command records
Record review finding counts in Run metrics
Count Unaddressed findings by kind in Run metrics
Show Run metrics and missing signals in Console details
Show Run metrics and weekly trends
Record Session-trace signals and Harness-version trends
Flag destructive commands in Run metrics
Count confirmed Bug origins against their Runs
JacobStephens2
marked this pull request as ready for review
October 25, 2026 07:36
Preserve Run metrics with the captured log inventory and shared Session call lifecycle. Keep details on the common read-time metrics projection, regenerate prompts, and retain the current Ticket run terminology. Keep help within its line budget and bound test scope launchers during version probes. Rust unit and integration targets, site and Console tests, release-generator checks, clippy, and formatting passed across the affected-test run and continuation.
Preserve the shared read-time Run metrics projection while adopting retained Scoring sample handles. Keep settlement decisions attached to their exact source files and withdraw superseded Eval proposals only after a successful save. Validation: cargo check --tests; cargo fmt --check; cargo clippy --all-targets -- -D warnings; scripts/affected-tests main (passed in 22m03s with the installed Python 3.11 and test-only Git signing/systemd isolation). The default Python 3.9 lacked tomllib; the affected Bug-origin regression passed with Python 3.11 before rerunning the full selector.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Run metrics keep each Run's friction and Session-trace signals separate, in its Command record and local read-only views.
Completion accounting keeps Unaddressed findings by Standards, Spec and Security from Delivery's final PR body, and review counts from machine-readable summaries, counted once across Resume. Stage durations are compared with repository Usual ranges. Every Ticket Run and Base fix keeps its own metrics.
Weekly trends keep each signal independent, split by repository, Harness, Harness version, requested Model, Effort, Run kind and session kind, with week/repository filters. Tokens retain the observed Model and cache split; cost is used only when reported. Historical reads reconstruct available signals, preserve missing values and never rewrite old records. Confirmed Bug origins are joined from local Scoring data when read.
The whole-Spec review fixed two integration gaps: historical metrics and late Bug-origin confirmations now appear in
show --jsonand Console details through the same projection asmetrics. Regression tests cover both views and a confirmation changing to a rejection between Console requests. Two misleading test names were corrected. No price table, combined score or remote metrics fetch is added.Closes #981
Evidence
show --jsondespite appearing inmetrics.After: both exact commands pass; historical minutes are
30.0and the late Bug-origin count is1.npm test --prefix console: 60 passed, including historical/unfinished Runs and phone-width details.cargo fmt --checkandcargo clippy --all-targets -- -D warnings: passed.scripts/affected-tests main: the site tests pass; the Rust run encounters an unrelated host warning from the installedsnapd.servicein the strict deployment-unit check.SYSTEMD_UNIT_PATH=/usr/lib/systemd/system cargo test --test site_deploypasses all 7 deployment tests. All 7 targets after deployment (spec_push_race,spec_run,status,stop,toolchain,update,user_config) pass in a separate continuation with that same unit-path isolation.GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=tag.gpgSign GIT_CONFIG_VALUE_0=falseto isolate tag fixtures from this host's global signing setting. User configuration is unchanged.Unaddressed findings
Standards
None.
Spec
None.
Review coverage
Standards: 1 P3 naming finding, addressed. Spec: 2 P2 details-view integration findings, reproduced and addressed. Each reviewer read all 57 changed files, including generated output, plus supporting standards, domain definitions and ADRs. Unread changed files: none. Generated-output exclusions: none. Final reports and combined files-read lists are retained as
standards.mdandspec.mdwith the Spec-review Session evidence.Merge Danger
Door: two-way
Readers accept Command records with or without metrics; historical reads do not rewrite them. Older binaries may omit version-2 records from their views, while the records remain on disk. A release carrying the changed Factory code-review skill requires an Experiment once #977 is available, as specified by #981.
Blast Radius: metrics
Completion recording, Session-trace accounting, CLI metrics, Console details and their documentation. Destructive-command flags remain advisory.
Built with codex · gpt-6.1-sol · high
Tickets
thirdshift metrics: one Run, and weekly trends #1036 done