Choose the Harness, Model and Effort a Command runs on (Claude Code or Codex) #136

Problem Statement

Every agent session thirdshift starts runs on Claude Code. Its model and effort are whatever Claude's own settings say. A user who wants a Command to run on Codex can't do it. Nor can a user who wants a particular model, such as Opus 5.5 or GPT-6.1-Sol, or a particular effort, such as max. There's no flag for it, no User config setting, and no Setup question.

The Factory skills are also tied to Claude's plugin system (claude --plugin-dir, skills named /thirdshift:<skill>). Codex has no equivalent of that, so the skills can't reach a Codex session as they are loaded today.

Solution

A Command's agent sessions run on a Harness, claude or codex, with a Model and an Effort. A command line chooses them with words like its other flags:

thirdshift <issue-url> harness codex model gpt-6.1-sol effort max

Any of the three can be left out. A missing one falls back to the User config, then to the Harness's own default. thirdshift ships no model names of its own. The User config holds a default Harness, and a Model and Effort for each Harness, so harness claude on the command line picks up Claude's configured model rather than Codex's. Setup asks for the Harness, then the Model and Effort for it.

One choice covers the whole Command: the implement session, every Repair and Resume, the Spec review, the Architecture review, each Ticket's Run in a Spec run, and a Base fix.

Before a Run starts work (before the Claim, the worktree or any log), thirdshift checks three things:

• the chosen Harness is installed
• the Model exists
• the Effort is one the Model takes

A typo fails at that point, as an Origin match failure does, never after an issue has been claimed.

The Factory skills stop being a Claude plugin. Each is named thirdshift-<skill> and linked into the Run's worktree where its Harness finds project skills, hidden from git. So both Harnesses load the same skills under the same names.

The Harness, Model and Effort a Command ran with are recorded in its Command log, its Activity log line, the pull request body and the Run notification.

User Stories

1. As a thirdshift user, I want to write harness codex on a Run's command, so that its sessions run on Codex instead of Claude Code.
2. As a thirdshift user, I want to write harness claude on a Run's command, so that I can use Claude Code for one Run when my User config defaults to Codex.
3. As a thirdshift user, I want to write model <name> on a command, so that one Command runs on a model other than my default.
4. As a thirdshift user, I want to write effort <level> on a command, so that I can make one Command reason harder or more cheaply than my default.
5. As a thirdshift user, I want to combine harness, model and effort in any order among a Run's other flags, so that the command line reads the way the existing flags do.
6. As a thirdshift user, I want any of harness, model and effort that I leave out to come from my User config, so that I don't repeat my defaults on every command.
7. As a thirdshift user, I want a Model or Effort that neither the command nor the User config names to be left to the Harness's own default, so that thirdshift never forces a model name that may be out of date.
8. As a thirdshift user with no Harness configured anywhere, I want Commands to run on Claude Code, so that nothing changes for me until I choose otherwise.
9. As a thirdshift user, I want my User config to hold a Model and Effort for each Harness, so that switching Harness on the command line uses that Harness's own configured Model and Effort, not the other Harness's.
10. As a thirdshift user, I want model and effort given on a command to apply to whichever Harness that command ends up using, so that model gpt-6.1-sol with my Codex default does what I mean.
11. As a thirdshift user, I want the same Harness, Model and Effort on every session in a Command, including Repairs, Resumes, the Spec review, the Architecture review, each Ticket's Run and a Base fix, so that one Command's work comes from one model.
12. As a thirdshift user running an Architect run or a Pickup run, I want harness, model and effort to reach the Spec run or Run it dispatches, so that scheduled work runs on the model I chose.
13. As a thirdshift user, I want a Harness that isn't installed to fail the Run before it does any work, naming the missing CLI and the setting that chose it, so that I know exactly what to fix.
14. As a thirdshift user, I want a Model or Effort the Harness rejects to fail the Run before the Claim, the worktree or any log, so that a typo never leaves a claimed issue with nothing done.
15. As a thirdshift user who schedules Passes, I want those checks made only once a Pass has decided not to skip, so that a skipped Pass never fails on configuration it didn't need.
16. As a thirdshift user, I want a Spec run to check its Model once, rather than once per Ticket, so that the check costs one call.
17. As a Codex user, I want model GPT-6.1-Sol and effort Max to be accepted however I capitalise them, and to be passed to Codex as gpt-6.1-sol and max, so that the names I see in Codex's model list just work.
18. As a Codex user, I want thirdshift to check the Model and Effort against Codex's own model list, which costs nothing, so that the check is instant and never spends tokens.
19. As a Claude user, I want a named Model checked with a tiny test call to Claude, so that Opus 5.5 (which Claude rejects) fails before the Run starts work, and Claude's error points me to opus or claude-opus-5-5.
20. As a Claude user who names no Model, I want no test call made, so that a default Run costs nothing extra.
21. As a new thirdshift user running Setup, I want to be asked which Harness to use, with both listed and any that isn't installed marked so, so that I learn what thirdshift can run on.
22. As a thirdshift user running Setup, I want Setup to refuse a Harness that isn't installed, so that I can't save a default that can't run.
23. As a thirdshift user running Setup, I want the default Harness answer to be the one I have installed, or claude when I have both, so that pressing Enter gives a working answer.
24. As a thirdshift user running Setup, I want to be asked the Model and Effort only for the Harness I chose, with blank meaning the Harness's own default, so that Setup stays short.
25. As a Codex user running Setup, I want the Model question to list the models Codex offers and the Effort question to list the levels allowed, so that I don't have to look them up.
26. As a thirdshift user running Setup, I want my Model and Effort answers checked the way a Run checks them, so that Setup never saves a value a Run would reject.
27. As a thirdshift user, I want the other Harness's settings written into my User config with blank values, so that the file shows everything I can change.
28. As a thirdshift user with an existing User config, I want re-running Setup to add the Harness settings while keeping my other values and comments, so that upgrading thirdshift doesn't lose my settings.
29. As a thirdshift user, I want an unknown key under the Harness settings to be an error, as for every other User config section, so that a typo can't silently leave a setting unset.
30. As a Day shift reviewer, I want the pull request body to say which Harness, Model and Effort built it, so that I can weigh the work and compare models.
31. As a thirdshift user, I want the Command log and the Activity log's start line to name the Harness, Model and Effort, so that I can tell afterwards what ran.
32. As a thirdshift user, I want the Run notification to name the Harness, Model and Effort, so that the email says what produced the outcome.
33. As a thirdshift user, I want the Factory skills named thirdshift-<skill>, so that they never collide with a repository's own skills of the same name, such as this repository's own code-review.
34. As a thirdshift user, I want the Factory skills linked into each Run's worktree and kept out of git, so that no session commits them and nothing global on my machine changes.
35. As a thirdshift user, I want Claude sessions to load the Factory skills the same way Codex sessions do, so that both Harnesses behave alike and there is one mechanism to maintain.
36. As a thirdshift user, I want every Session prompt to load its skill for certain on its first line, using the Harness's own syntax, so that an unattended session never misses its skill.
37. As a thirdshift user, I want the Factory skills to refer to each other by their thirdshift- names in plain prose, so that they read correctly under either Harness.
38. As a Codex user, I want Codex sessions to run unattended with the network available, so that they can commit, push and call gh without a human approving each command.
39. As a Codex user, I want each Resume of a Codex session to keep the Model, Effort and permissions the session started with, so that a resumed session is the same kind of session.
40. As a Codex user, I want a session that ends its turn while a command it started is still running to be resumed once, as a Claude session is, so that killed background work is handled the same way under both Harnesses.
41. As a Codex user, I want progress lines and a session-ended summary on stderr from Codex sessions, as from Claude sessions, so that I can follow a Run whichever Harness it uses.
42. As a Codex user, I want each Codex session's full event stream kept as its Session log, so that I can read back what it did.
43. As a Codex user, I want an interrupt to stop a Codex session cleanly before it is killed, so that interrupting a Run behaves as it does on Claude.
44. As a Codex user working in a repository that has only a CLAUDE.md, I want Codex sessions to read it, so that they get the project's instructions. A repository's AGENTS.md still comes first.
45. As a Codex user, I want a Codex session that fails, for example on a rejected request, to fail the Run with Codex's error in its cause, so that I can see why.
46. As a reader of the Prompts and skills page, I want it to keep showing each prompt once, with a note on how Codex sessions name skills, so that the page stays readable.

Implementation Decisions

• New domain terms (already in CONTEXT.md): Harness, Model, Effort, one of each per Command. The Factory skills and User config entries are updated to match.

• ADR 0012 (Factory skills are linked into the worktree under a thirdshift- prefix, and Codex sessions run unsandboxed) records the two decisions that would surprise a reader. It replaces ADR 0001's plugin consequence. Research with sources: docs/research/codex-headless-harness.md.

• Command line. harness <name>, model <name> and effort <level> are word-and-value flags like base <branch>. They're accepted wherever a Run's other flags are: on a Run, an Architect run (unless it's --plan-only, since a plan-only run dispatches nothing) and a Pickup run. Each is passed down to child Runs (a Ticket's Run, a Base fix, a dispatched Run) through the existing child-Run arguments, the same way other asks are.

• How a value is chosen. For each of Harness, Model and Effort: the command wins, then the User config, then the default. For the Harness the default is claude. For Model and Effort it is nothing, which leaves the choice to the Harness.

• User config shape.

  [harness]
  default = "claude"      # or "codex"
  
  [harness.claude]
  model = ""              # blank: Claude Code's own default
  effort = ""
  
  [harness.codex]
  model = "gpt-6.1-sol"
  effort = "max"

  Unknown keys are errors, as elsewhere in the User config.

• Checks before work. These run where an Origin match failure happens: before the Claim, the worktree and any Command log.

  • The chosen Harness's CLI must be on PATH.
  • Codex: the Model and Effort are checked against codex debug models, matching names regardless of case and turning display names into slugs. Effort must be one of the levels that Model supports.
  • Claude: only when a Model is named, a minimal non-interactive test call checks it. Model names pass to Claude unchanged, with no translation. Effort is passed as given, since Claude accepts any capitalisation.
  • A Pass makes these checks only once it has decided not to skip. A Spec run makes them once for all its Tickets.

• Setup. It always asks for the Harness. Both are listed, any that isn't installed is marked so and refused, and the default answer is whichever is installed, or claude when both are. Model and Effort are asked only for the chosen Harness, blank meaning the Harness's default. For Codex, the questions list the models and levels from Codex's model list. Answers are checked as a Run checks them. The other Harness's section is written with blank values. Setup over an existing User config adds the new keys and keeps the rest, as it does today.

• Factory skills, both Harnesses.

  • The skills in skills/ are renamed in source to thirdshift-<skill>: the directory, the frontmatter name:, and every reference one skill makes to another, written in plain prose without a sigil.
  • thirdshift writes the embedded skills out once per Command and links each into the Run's worktree: under .claude/skills/ for Claude, .agents/skills/ for Codex.
  • It adds thirdshift-* patterns for those two directories to the repository's .git/info/exclude, once and idempotently, and leaves them there.
  • The generated plugin manifest and --plugin-dir are removed.

• Session prompts. The prompt module renders each prompt for a Harness. The first line invokes the skill with that Harness's syntax: /thirdshift-<skill> for Claude, $thirdshift-<skill> for Codex. Other skills are named in plain prose ("the thirdshift-code-review skill"). The background-work line, which today names claude -p, is reworded for either Harness. The generated prompts/ page shows the Claude rendering, with a one-line note about Codex.

• The session seam. Agent sessions already go through one module that starts a session, follows its output stream into the Session log and progress lines, captures its session id, reports killed background work, and supports a Resume. That module takes the Harness, Model and Effort. Each Harness supplies how to start a session and a Resume, and how to read its stream.

  • Claude: as today, plus --model and --effort when set, and the skills found in the worktree instead of from a plugin.
  • Codex: codex exec --json --dangerously-bypass-approvals-and-sandbox, with -m and the reasoning-effort setting when set. It also passes the setting that reads CLAUDE.md when there's no AGENTS.md, and starts with stdin set to null.
    • A Resume is codex exec resume <thread id>, passing the Model, Effort and bypass flag again, since a resume doesn't keep them.
    • The session id is the thread.started thread id. Progress comes from item.* events and the final message from the last agent_message. A failed session shows as turn.failed or a non-zero exit, with Codex's error as the cause.
    • Killed background work is a command_execution (or sub-agent call) still in_progress when the turn ends. That triggers the same single Resume as on Claude.
    • On interrupt, a Codex session gets SIGINT first, then the existing SIGTERM/SIGKILL stop.

• Recording. The Command log's opening lines and the Activity log's start line name the Harness, Model and Effort. A blank Model or Effort is shown as the Harness's default. The pull request body carries one line such as Built with codex · gpt-6.1-sol · max, which thirdshift writes, not the agent. The Run notification carries the same line.

Testing Decisions

• Good tests: test behaviour through thirdshift's command line, as a user or a scheduler sees it: what it runs, with which arguments, what it prints, writes and records, and how the Run ends. Don't test internal functions or data shapes.
• One test seam: the existing end-to-end Scenario harness. Its fake executable, put first on PATH, already acts as gh or claude depending on the name it's called by, recording each call's arguments, prompt, working directory and skills.
  • It gains a codex personality. That covers codex exec --json, codex exec resume, emitting Codex's event stream (thread.started, item.*, turn.completed, turn.failed), and answering codex debug models from a fixed catalog. It runs per-test scripts the way the fake claude does.
  • The skills snapshot moves from the plugin directory to the thirdshift-* skills linked into the worktree. Tests also check they're excluded from git.
  • The fake's existing plain-text claude -p mode answers the Claude model check. A test script decides whether a model is rejected.
  • Whether a Harness counts as installed is set by which fakes are on PATH.
• What's covered:
  • the flags, and the command, then User config, then default fallback
  • Harness, Model and Effort reaching every session kind and child Run
  • each pre-work check failing before the Claim, the worktree and any log, and not running on a skipped Pass
  • the Spec run checking once
  • Codex name translation
  • skill names in prompts and in the worktree
  • recording in the Command log, the Activity log, the PR body and the Run notification
  • the Codex session's progress lines, Session log, Resume after killed work, failure causes and interrupt
• Setup is tested through the existing terminal Keystrokes scenarios: the Harness list with uninstalled ones marked, refusal of an uninstalled one, the default answer, the follow-up questions for the chosen Harness, checked answers, and keeping an existing config.
• Prior art:
  • tests/background_work.rs for killed background work and Resume
  • tests/setup.rs and tests/setup_offer.rs for Setup
  • tests/user_config.rs for config parsing errors
  • tests/preflight.rs for failures before work
  • tests/command_logs.rs and tests/activity_log.rs for recording
  • tests/run_notification.rs for the email
  • tests/fakes/claude.rs for the shape of the new fake
• Prompts page: the generated prompts/ page is regenerated with UPDATE_PROMPTS=1 cargo test prompts_page, and its test keeps it in sync.

Out of Scope

• A sandboxed Codex mode (workspace-write with network on and the repository's shared .git added as writable) for machines where bubblewrap can run. It's left out because it can't be verified on a stock Ubuntu machine. It will be filed as its own needs-triage issue.
• Choosing a Harness per session kind, for example Claude for the Architecture review and Codex for implementing.
• Translating Claude display names such as Opus 5.5 into model ids. Claude publishes no model list, so the user writes the id or alias Claude accepts.
• Carrying the user's Claude-only instructions and hooks (~/.claude/) into Codex sessions. That belongs to the user's machine setup, not to thirdshift.
• Harnesses other than Claude Code and Codex.

Further Notes

• Live checks on codex-cli 0.160.0 (see section 8 of the research note):
  • Codex rejects display names and capitalised efforts only on its first API request, after a session has started. That's why thirdshift checks names against codex debug models beforehand.
  • Codex waits for stdin to close even when it's given a prompt.
  • Codex adds a trust_level = "trusted" entry for each target repository's root to the user's ~/.codex/config.toml.
• Codex reads AGENTS.md; Claude Code reads it too. The CLAUDE.md fallback only matters for repositories without one.
• Planned Tickets:
  1. The Harness, Model and Effort settings with Claude working end to end: flags, User config, Setup, the checks before work, the move from the plugin to skills linked into the worktree, rendering prompts for a Harness, and recording.
  2. The Codex Harness, blocked by Ticket 1.