Repository navigation
Freeze Custom process Arms and home skills for Experiment Trials - #1928
Merged
Merged
Conversation
A Process Trial keeps one home for every Step and prepares a new sandbox there when a Step uses another Harness. Copying a login through a symlink in that home wrote outside the sandbox. Host writes and read-only binds into the fresh home now refuse a symlink along the destination.
Preserve Custom process and frozen home-skill documentation alongside the process Eval task proposal guidance. Regenerate the manual from the merged source.
Preserve frozen Custom process Arms and home skills alongside final-commit process grading with the reference tests and shared fakes overlaid. Regenerate the manual and reconcile the three hand-written site pages. Validation: cargo check --bin thirdshift, cargo fmt --check, all-target clippy, and all 68 site checks pass.
This was referenced Nov 10, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Freeze a Custom process and its referenced home skills for either Arm, retaining each file's SHA-256 and every Step's Harness, Model and Effort. Baseline defaults to the Built-in Ticket run.
Closes #1877
Evidence
Execution evidence
cargo test --test eval_arms custom_process_and_home_skill_are_frozen_with_step_choicesfailed with theprepare-armsusage error.After: Same command passes (1 test); frozen Process, instructions, helper and Step choices recorded.
cargo test --test experiments custom_process_trials_use_frozen_home_skills_beside_the_builtin_baselinefailed when a later Trial reread the changed original Process (TOML parse error).After: Same command passes: 6 delivered Trials, Custom Candidate Steps and Built-in Baseline Steps.
python3 .thirdshift-review-FzmsRH/repro_spec.py issueandpython3 .thirdshift-review-FzmsRH/repro_spec.py sandboxboth failed withcodex exited 55; the latter reportedgitfile does not point to a valid repository.After: Both exact commands exit 0 with
outcome: delivered. Retained regressioncargo test --test trial custom_throwaway_steps_receive_the_issue_and_keep_git_access_on_another_harness -- --nocapturepasses both pass-through and bwrap modes.cargo clippy --bin thirdshift -- -D warningspasses. Frozen-file tampering, mixed pinned Harnesses and older Built-in-only Baseline protocol regressions pass.524206a9: 3,947 Rust tests passed, 91 skipped, plus Site, Console, formatting, all-target clippy and Python tooling.cargo nextest runtook 19m45s including 1m03s compilation (1,121.179s test execution). Both Linux release builds also passed.scripts/affected-tests origin/issue-1484selected site checks and the full Rust suite: 68 site checks and 1,663 Rust unit tests passed. The first full run stopped because the host selected Python 3.9 withouttomllib;cargo test --test bug_origins settlement::historical_review_and_reclosed_origin_regressions -- --nocapturepassed with Python 3.11. The Python 3.11 retry was stopped after successful CI covered the exact head, reusing CI asdocs/testing.mdinstructs.Test seams: the
evals prepare-arms,experimentand hidden--trialCLI interfaces, plus the rendered site documentation's format contract.Visual evidence
Documentation captures use the same expanded sections and viewport states. Before is the pinned
issue-1484base; After is the implementation. Public documentation only.Home, Experiment section
1440×900 desktop
390×844 mobile
How it works, Process Trials
1440×900 desktop
390×844 mobile
Manual, Process Trials
1440×900 desktop
390×844 mobile
Architecture, Trial execution
1440×900 desktop
390×844 mobile
Review coverage
Reviewed against
issue-1484at8ad2733cwiththirdshift-code-review. Standards: 1 finding, addressed. Spec: 2 findings, addressed. Every behavioral reproduction was run as written before deciding. Both axes read all 26 changed files; no unread changed files and no generated-output exclusions. Reports and finding counts are retained in.thirdshift-review-FzmsRH/{standards.md,spec.md,summary.json}beside the Session evidence.Unaddressed findings
Security review outcome
No unaddressed introduced findings.
Merge Danger
Door: two-way
Blast Radius: Evals
Experiment JSON advances to format 5 to describe frozen Process inputs; older saved Arm records remain readable. Process files needing GitHub before their first publishing Step remain refused.
Built with codex codex-cli 0.166.0 · gpt-6.1-sol · xhigh