[JacobStephens2]

Summary

Process Eval tasks can now be drafted from merged Tickets and screened before preparation.

evals propose-process <repository>
  → proposed task: base + merge + Ticket + tests + fakes + selectors
evals approve process <task>
evals prepare process <task> <destination>
  → fail at base + pass at merge; refuse changed inline Rust tests
  → publish base-only Trial repository

The process suite declares three Trials and dependencies on its five Factory skills, Built-in process sources and shared prompt, Harness, Model and Effort defaults.
An admin-scoped API twin proposes tasks only from known Launch directories.

Closes #1876

Evidence

Execution evidence

• Before: cargo test --test evals process_tasks::preparation -- --nocapture failed both new cases with unknown field process_tests.
  After: the same command passed all three preparation cases, including inline-test rejection.
• Before: cargo test --test evals process_tasks::preparation_refuses_changed_inline_tests -- --nocapture failed with assertion failed: !output.status.success().
  After: the same case passes and names the changed source file.
• Before: cargo test --test evals process_tasks::proposals -- --nocapture failed with the existing evals usage message.
  After: cargo test --test evals process_tasks:: -- --nocapture passed all four fixture cases, including proposal approval and preparation.
• Exact reviewer reproductions were run before deciding and retained as regressions:
  • Executable overlay: python3 .thirdshift-review-2iXnN6/spec_executable_overlay.py failed with Permission denied; now exits 0.
  • Rebase merge: python3 .thirdshift-review-2iXnN6/spec_rebase_merge.py failed with pull request changes no source; now exits 0.
  • Squash merge after updating main: python3 .thirdshift-review-2iXnN6/spec_squash_updated_base.py reported four source lines instead of two; now exits 0 and checks the correct base.
• cargo test --bin thirdshift console::parity::every_cli_command_has_an_api_twin_or_reasoned_exclusion initially failed because ProposeProcess had no API twin; it now passes. The new admin endpoint accepts known Launch directories.
• cargo test --test evals: 22 passed on the final commit. The Console admin cases and exact CLI/API parity test passed. cargo clippy --all-targets -- -D warnings and npm run docs:check --prefix site passed.
• CI on 18641d83 passed the full Rust suite, site tests and Console browser tests. Release build and smoke checks passed too.
• Local scripts/affected-tests origin/issue-1484 completed 67 site and 169 Console browser tests successfully before its Rust phase found Python 3.9's missing tomllib. Selecting the installed Python 3.11 fixed the exact bug-origin regression and its full 18-test integration file.
• Local validation limitation: the subsequent warm cargo test run stopped after 682.07 s (no substantive compilation) at idle::unchanged_requests_reuse_retained_records_but_replacements_are_visible: unchanged request reread 107775 bytes versus a 48 KiB threshold. The exact isolated rerun fails locally too; CI passes this test on the same commit. A repeated Fleet browser case also timed out, then passed in isolation without concurrent test load. These are reported as validation discrepancies; the successful CI suite supplies full-suite validation.

Tests cross the thirdshift evals CLI seam, using real temporary Git repositories and a fake GitHub executable at the external GitHub seam.
The admin scope and known-directory guard are checked through HTTP.

Visual evidence

Documentation captures use matching expanded-page states at 1440×900 desktop and 390×844 mobile.

/ — desktop

/ — mobile

/how-it-works/ — desktop

/how-it-works/ — mobile

/architecture/ — desktop

/architecture/ — mobile

/docs/ — desktop

/docs/ — mobile

Unaddressed findings

• Standards: none. The GraphQL Message Chains finding was addressed by keeping transport structures private to the GitHub adapter.
• Spec: none. Both findings were fixed, including the squash-merge variant of the range finding; every reproduction now passes and its regression is retained.
• Security: none.

Security review outcome

No unaddressed introduced findings.

Review coverage

Independent Standards and Spec reviews compare against issue-1484, pinned at 8ad2733cfb7f8d3b9869a2c1b8f00b2eb224b447 (the fetched origin branch). Each axis read all 30 changed files after one coverage follow-up. Unread changed files: none under either axis. Generated-output exclusions: none. Standards raised 1 finding; Spec raised 2; all were addressed. Session artifacts retain each report's final files-read list and summary.json with {"standards":1,"spec":2} in .thirdshift-review-2iXnN6/.

Merge Danger

Door: two-way

Adds an optional manifest table and proposal command; existing review tasks retain their format. Process preparation executes explicitly selected test commands on the host with network access and a ten-minute deadline per command.

Blast Radius: Evals

Selector inference covers Rust integration tests and Python scripts. Drafts need Day-shift review and approval before preparation; unfamiliar layouts are reported as skipped.

Built with codex codex-cli 0.166.0 · gpt-6.1-sol · xhigh