Repository navigation
Measure skill catalogs and clarify Factory resolution - #764
Merged
Merged
Conversation
JacobStephens2
marked this pull request as ready for review
October 17, 2026 17:53
This was referenced Oct 17, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #735.
Record evidence-backed skill catalogs and recommend keeping Run startup unchanged. Preserve two concurrent investigations as separate observations; their different roots and counts have no established causal explanation. No harmful Factory skill selection was demonstrated in either bounded sample.
The notes explain namespaced Factory resolution, supported Codex/Claude configuration, intentionally available tools, and Continuation/Resume limits. Separate JSON baselines retain each investigation's provenance. The concurrent Continuation is an older 0.162.1 observation; these counts are not controlled comparisons of session types or CLI versions.
Evidence
After: Retain seven actual session observations across the two investigations. Both published recomputation commands pass, matching counts, bytes, proxy tokens, duplicates and SHA-256 hashes. Different session IDs/catalog roots remain explicitly attributed rather than averaged or interpreted as a change rate.
After: An invocation-only, no-turn diagnostic suppressed 15 duplicate Host entries: 32 entries / 9,506 bytes / 2,251 proxy tokens, retaining all ten Factory names and accountability instructions. Personal configuration stayed byte-identical. This measures catalog savings; it does not establish billed savings, runtime selection, hook execution or Resume compatibility. No startup suppression is shipped.
cargo check,cargo fmt --check,cargo clippy --all-targets -- -D warnings,cargo test --test continuation, andcargo test skills::tests::links_factory_skills_in_existing_and_missing_directories_for_every_adapterpassed. Fullcargo test: 1,872 passed, 2 ignored, 0 failed across 48 test binaries. The subsequent reconciliation changed only documentation/evidence; Rust source, skills and tests are unchanged. Typechecking, formatting, both catalog recomputations and the retained UTC check pass after reconciliation.Validation seams: model-visible Codex prompt input and recorded catalogs; existing Factory-skill worktree linking and Continuation CLI flow; displayed UTC timestamps against captured metadata. No new production behavior was introduced for this investigation.
Unaddressed findings
Both reviewers read all five changed files on the combined branch. Unread changed files: none. Initial and combined reports, including final files-read lists, are saved in the requested
.thirdshift-review-df7VoO/standards.mdandspec.mdfiles in the worktree.The earlier guidance-mode Security review reported no introduced findings at concurrent commit
6006f33; that review predates the documentation reconciliation and is not claimed as a Security audit of the combined head.Security review outcome
Merge Danger
Door: two-way
Documentation and a captured baseline only; no Harness settings, embedded skills, instruction files, hooks or personal/global configuration changes.
Blast Radius: documentation
Catalog measurements are host/version/context snapshots. Proxy token counts do not prove end-to-end cost or latency improvements.
Built with codex · gpt-6.1-sol · high
CI notes
RESULT: cancelled; this was caused by the superseded run, not a branch regression, so no code or check changes were made.970689f:cargo nextest run --partition hash:2/4passed 463/463 tests;cargo nextest run --partition hash:4/4passed 484/484 tests. Tests outside each hash partition were excluded by the unchanged partition selection, as in CI; no checks or assertions were changed. macOS validation belongs to replacement run 39056268502.