Skip to content

Setup asks for the Harness, Model and Effort (#393) #403

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JacobStephens2 merged 2 commits into from
Oct 5, 2026
Merged

Setup asks for the Harness, Model and Effort (#393) #403

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ee85338
Setup asks for the Harness, Model and Effort (#393)
JacobStephens2 Oct 5, 2026
57e6904
Review clean-ups for Setup's Harness questions (#393)
JacobStephens2 Oct 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 17 additions & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -242,15 +242,27 @@ parallel = 3 # how many Tickets a Spec run runs at once; default 3

[pickup]
limit = 3 # how many open issues labelled in-progress stop a Pickup run taking another; default 3

[harness]
default = "claude" # the Harness every Run's sessions run on, claude or codex; default claude

[harness.claude]
model = "" # the Model Claude Code's sessions run on; default blank, for Claude Code's own
effort = "" # how hard that Model reasons; default blank, for Claude Code's own

[harness.codex]
model = "" # the Model Codex's sessions run on; default blank, for Codex's own
effort = "" # how hard that Model reasons; default blank, for Codex's own
```

Every key holds its real value, so a Run reading it does exactly what it does with no file. `email.to` has no default, so `setup` suggests one: the public email of your GitHub profile (from `gh api user`), else your global git `user.email`, unless that is a `@users.noreply.github.com` address, which can't receive mail. With neither, `email.to` is the only line written commented out, as above. `setup` never asks `gh` for more scopes, so a private GitHub email is not read, and a Run never looks the suggestion up: `--email` with no address and no `email.to` still stops the Run. From a terminal (stdin and stderr both terminals), `setup` first asks, on stderr:

1. Every Run a Merge run? (`merge.always`)
2. With that on, every Run may start a Base fix when the Base branch's CI is red? (`base.fix`). With it off, this is not asked, and `base.fix` is written at its default, `false`.
3. Every Run first fast-forwards your checkout of the Base branch? (`launch.pull`)
4. Run notifications? (`email.always`). If yes, the address (`email.to`), asked again until it has an `@`, then the sender (`email.from`).
5. With notifications on, the Resend API key, with input hidden. With none saved it asks `Resend API key (input hidden, Enter to skip):`; with one in the [Credentials](#email), `Resend API key (input hidden, Enter keeps the saved one):`, and a new one replaces it. Surrounding spaces are trimmed, and anything that doesn't start with `re_` is asked again. A key you give is saved in the Credentials, `~/.thirdshift/credentials.toml`, created with mode 600 (and `~/.thirdshift` with it) or edited in place, keeping its comments and anything else in it and changing only `resend.key`; `setup` then prints `wrote the Credentials <path>`. The key is never printed, nor written to the User config. Skipping writes no Credentials and says how to add a key later: rerun `thirdshift setup`, or set `RESEND_API_KEY`. With `RESEND_API_KEY` set and not empty, which wins over the Credentials, nothing is asked, and it says the key comes from `RESEND_API_KEY`. With a key found or given, it offers to send a test email (default No), as `thirdshift email-test` does, once the files are written.
1. The Harness for every Run's sessions (`harness.default`), listing both, each marked where sessions can't run on it here: `claude (not installed)` when `claude` isn't on `PATH`, and `codex (not supported yet)` until Codex sessions are. A marked Harness, or a name that isn't one, is refused and asked again. The default is the one sessions can run on, `claude` when both can. Then the Model (`harness.<name>.model`) and the Effort (`harness.<name>.effort`) for the Harness chosen only, each defaulting to the User config's, where Enter on none keeps the Harness's own default and `-` clears one. A named Model gets the test call a Run makes (see [Harness, Model and Effort](#harness-model-and-effort)); if Claude refuses it, Setup shows what Claude said and asks the Model and Effort again. The other Harness's section is left as it is. With no Harness sessions can run on here, Setup says so, asks none of this, and leaves the harness settings as they are.
2. Every Run a Merge run? (`merge.always`)
3. With that on, every Run may start a Base fix when the Base branch's CI is red? (`base.fix`). With it off, this is not asked, and `base.fix` is written at its default, `false`.
4. Every Run first fast-forwards your checkout of the Base branch? (`launch.pull`)
5. Run notifications? (`email.always`). If yes, the address (`email.to`), asked again until it has an `@`, then the sender (`email.from`).
6. With notifications on, the Resend API key, with input hidden. With none saved it asks `Resend API key (input hidden, Enter to skip):`; with one in the [Credentials](#email), `Resend API key (input hidden, Enter keeps the saved one):`, and a new one replaces it. Surrounding spaces are trimmed, and anything that doesn't start with `re_` is asked again. A key you give is saved in the Credentials, `~/.thirdshift/credentials.toml`, created with mode 600 (and `~/.thirdshift` with it) or edited in place, keeping its comments and anything else in it and changing only `resend.key`; `setup` then prints `wrote the Credentials <path>`. The key is never printed, nor written to the User config. Skipping writes no Credentials and says how to add a key later: rerun `thirdshift setup`, or set `RESEND_API_KEY`. With `RESEND_API_KEY` set and not empty, which wins over the Credentials, nothing is asked, and it says the key comes from `RESEND_API_KEY`. With a key found or given, it offers to send a test email (default No), as `thirdshift email-test` does, once the files are written.

Pressing Enter takes the default shown, which is the file's current value, or else the setting's default, and for the address the suggested email above. `logs.dir`, `activity.quiet_skips`, `spec.parallel` and `pickup.limit` are not asked about. The answers are written like everything else below: in place, keeping your comments. Ctrl-C during the questions, the key included, writes nothing: neither the User config nor the Credentials. With notifications off, nothing about a key is asked, and saved Credentials stay as they were, so `--email` on a single Run still works. Credentials a Run would refuse (see [Email](#email)) are refused before any question, exit `1`, and not touched. With no terminal, as from cron or `thirdshift setup </dev/null`, `setup` asks nothing and never writes the Credentials. Either way it prints the file's path on stderr and exits `0` with stdout empty. Over a User config that is already there, `setup` edits it in place: its comments and key order stay, as do the values it didn't ask about, and each key it lacks is added at its default with its comment, so afterwards the file lists every setting this version knows. One that already does, down to the commented-out `email.to` line, is left byte for byte as it was. A key added to an inline table, such as `launch = { pull = true }`, gets no comment, since TOML has no place for one there. One a Run would refuse is refused the same way, exit `1`, and not touched. Any argument after `setup` is an argument error (exit `2`).

Expand Down
189 changes: 150 additions & 39 deletions src/config.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -367,17 +367,28 @@ fn with_answers(text: &str, answers: &Answers) -> Result<String> {
set(&mut document, "email", "from", notifications.from.as_str());
set_email_to(&mut document, &notifications.to);
}
if let Some((harness, chosen)) = &answers.harness {
set(&mut document, "harness", "default", harness.name());
let section = format!("harness.{}", harness.name());
for (key, value) in [("model", &chosen.model), ("effort", &chosen.effort)] {
set(&mut document, &section, key, value.as_deref().unwrap_or(""));
}
}
Ok(document.to_string())
}

/// Set `section.key`, which `document` holds, to `value`, keeping the
/// spacing and comment around the old value, and the comment in the same
/// column where the spaces before it allow. An equal value is left as it was
/// written.
/// written. `section` may be a subsection, as `harness.claude`.
fn set(document: &mut DocumentMut, section: &str, key: &str, value: impl Into<toml_edit::Value>) {
let value = value.into();
let old = document[section]
.as_table_like_mut()
let old = section
.split('.')
.try_fold(document.as_item_mut(), |item, name| {
item.as_table_like_mut()?.get_mut(name)
})
.and_then(Item::as_table_like_mut)
.and_then(|settings| settings.get_mut(key))
.and_then(Item::as_value_mut)
.unwrap_or_else(|| panic!("a completed User config has {section}.{key}"));
Expand Down Expand Up @@ -475,46 +486,23 @@ pub fn replace(path: &Path, text: &str) -> std::io::Result<()> {
/// `text`, a User config a Run accepts, with each key it lacks added at its
/// default, as `DEFAULTS` writes it. Everything already in `text` stays as
/// it was: a missing key goes at the end of its section, and a missing
/// section after the last line of `text`. A key added to an inline table
/// section, or a key of a section `text` names only in its subsections'
/// headers, after the last line of `text`. A key added to an inline table
/// gets no comment, as TOML has no place for one there.
fn complete(text: &str) -> Result<String> {
let mut document: DocumentMut = text.parse().context("can't parse the User config")?;
let defaults: DocumentMut = DEFAULTS.parse().expect("DEFAULTS is valid TOML");
let mut missing = DocumentMut::new();
for (section, default_settings) in defaults.iter() {
let default_settings = default_settings
.as_table()
.expect("every key in DEFAULTS is in a section");
match document.get_mut(section) {
None => {
missing.insert(section, Item::Table(default_settings.clone()));
}
Some(Item::Table(settings)) => {
for (key, item) in default_settings.iter() {
if !settings.contains_key(key) {
let mut key = default_settings
.key(key)
.expect("the key is in DEFAULTS")
.clone();
key.leaf_decor_mut().set_prefix("");
settings.insert_formatted(&key, item.clone());
}
}
if section == "email" && !settings.is_dotted() {
note_email_to(settings, default_settings, text);
}
}
Some(Item::Value(toml_edit::Value::InlineTable(settings))) => {
for (key, item) in default_settings.iter() {
if !settings.contains_key(key) {
let mut value = item.as_value().expect("DEFAULTS has only values").clone();
value.decor_mut().clear();
settings.insert(key, value);
}
}
}
Some(_) => unreachable!("a Run refuses {section} that isn't a section"),
}
complete_table(
document.as_table_mut(),
defaults.as_table(),
missing.as_table_mut(),
);
if let Some(Item::Table(email)) = document.get_mut("email")
&& !email.is_dotted()
{
let defaults = defaults["email"].as_table().expect("DEFAULTS has [email]");
note_email_to(email, defaults, text);
}
let mut completed = document.to_string();
let missing = missing.to_string();
Expand All @@ -529,6 +517,78 @@ fn complete(text: &str) -> Result<String> {
Ok(completed)
}

/// Add to `settings`, a table of a User config, each key of `defaults`, the
/// same table in `DEFAULTS`, that it lacks, and the same for each of its
/// subsections. A missing key goes at the end of `settings`, unless
/// `settings` has no header of its own, as `[harness]` with only
/// `[harness.claude]` written; then it goes, as does a missing subsection, in
/// `missing`, the same table of what is written after the file.
fn complete_table(
settings: &mut toml_edit::Table,
defaults: &toml_edit::Table,
missing: &mut toml_edit::Table,
) {
let implicit = settings.is_implicit() && !settings.is_dotted();
for (key, item) in defaults.iter() {
let default_key = defaults.key(key).expect("the key is in DEFAULTS");
match (item, settings.get_mut(key)) {
(Item::Table(default_settings), Some(Item::Table(settings))) => {
let missing = missing.entry(key).or_insert_with(|| {
let mut table = toml_edit::Table::new();
table.set_implicit(true);
table.set_position(default_settings.position());
Item::Table(table)
});
let missing = missing
.as_table_mut()
.expect("a missing section is a table");
complete_table(settings, default_settings, missing);
}
(
Item::Table(default_settings),
Some(Item::Value(toml_edit::Value::InlineTable(settings))),
) => complete_inline(settings, default_settings),
(Item::Table(_), Some(_)) => unreachable!("a Run refuses {key} that isn't a section"),
(Item::Table(_), None) => {
missing.insert_formatted(default_key, item.clone());
}
(_, Some(_)) => {}
(_, None) if implicit => {
missing.set_implicit(false);
missing.insert_formatted(default_key, item.clone());
}
(_, None) => {
let mut key = default_key.clone();
key.leaf_decor_mut().set_prefix("");
settings.insert_formatted(&key, item.clone());
}
}
}
}

/// Add to `settings`, an inline table of a User config, each key of
/// `defaults`, the same table in `DEFAULTS`, that it lacks, with no comment.
fn complete_inline(settings: &mut toml_edit::InlineTable, defaults: &toml_edit::Table) {
for (key, item) in defaults.iter() {
match (item, settings.get_mut(key)) {
(Item::Table(defaults), Some(toml_edit::Value::InlineTable(settings))) => {
complete_inline(settings, defaults)
}
(_, Some(_)) => {}
(Item::Table(defaults), None) => {
let mut table = toml_edit::InlineTable::new();
complete_inline(&mut table, defaults);
settings.insert(key, toml_edit::Value::InlineTable(table));
}
(_, None) => {
let mut value = item.as_value().expect("DEFAULTS has only values").clone();
value.decor_mut().clear();
settings.insert(key, value);
}
}
}
}

/// With no `email.to` in `email`, the `[email]` section of `text`, and no
/// commented-out one either, add the commented-out example line from
/// `defaults`, the `[email]` section of `DEFAULTS`, just before `email.from`:
Expand Down Expand Up @@ -620,6 +680,17 @@ parallel = 3 # how many Tickets a Spec run runs at once; default 3

[pickup]
limit = 3 # how many open issues labelled in-progress stop a Pickup run taking another; default 3

[harness]
default = "claude" # the Harness every Run's sessions run on, claude or codex; default claude

[harness.claude]
model = "" # the Model Claude Code's sessions run on; default blank, for Claude Code's own
effort = "" # how hard that Model reasons; default blank, for Claude Code's own

[harness.codex]
model = "" # the Model Codex's sessions run on; default blank, for Codex's own
effort = "" # how hard that Model reasons; default blank, for Codex's own
"#;

/// The line `DEFAULTS` holds for `email.to`, which has no default.
Expand Down Expand Up @@ -730,6 +801,8 @@ mod tests {
Some(crate::email::DEFAULT_FROM)
);
config.email.from = None;
assert_eq!(config.harness.default, Some(Harness::Claude));
config.harness.default = None;
assert_eq!(config, UserConfig::defaults(Path::new("/home/me")));
}

Expand Down Expand Up @@ -994,7 +1067,8 @@ mod tests {
}

/// `text` completed, after checking a Run reads it with the settings it
/// had, plus the defaults for the keys it lacked.
/// had, plus the defaults for the keys it lacked, which change nothing a
/// Run does.
fn completed(text: &str) -> String {
let completed = complete(text).unwrap();
let before = parse(text).unwrap();
Expand All @@ -1006,6 +1080,10 @@ mod tests {
);
after.email.from = None;
}
if before.harness.default.is_none() {
assert_eq!(after.harness.default, Some(Harness::Claude));
after.harness.default = None;
}
assert_eq!(after, before, "{completed}");
let again = complete(&completed).unwrap();
assert_eq!(again, completed, "completing twice changed it");
Expand Down Expand Up @@ -1044,6 +1122,38 @@ mod tests {
}
}

#[test]
fn completing_adds_each_harness_key_a_user_config_lacks() {
for text in [
"[harness]\ndefault = \"claude\"\n",
"[harness.claude]\nmodel = \"opus\" # mine\n",
"[harness.codex]\neffort = \"max\"\n\n[merge]\nalways = true\n",
"[harness]\n\n[harness.claude]\neffort = \"high\"\n",
"harness.default = \"claude\"\nharness.claude.model = \"opus\"\n",
"harness = { claude = { model = \"opus\" } }\n",
"[harness]\nclaude = { effort = \"low\" }\n",
] {
let completed = completed(text);
let config: toml::Table = completed.parse().unwrap();
let harness = config["harness"].as_table().unwrap();
assert!(harness.contains_key("default"), "{text:?}:\n{completed}");
for name in ["claude", "codex"] {
let settings = harness[name].as_table().unwrap();
for key in ["model", "effort"] {
assert!(settings.contains_key(key), "{text:?}:\n{completed}");
}
}
// An inline table gains its missing keys on its own line.
let mut lines = completed.lines();
for line in text.lines().filter(|line| !line.contains('{')) {
assert!(
lines.any(|kept| kept == line),
"{line:?} of {text:?} is lost or moved:\n{completed}"
);
}
}
}

#[test]
fn completing_adds_email_to_commented_out_once() {
for text in [
Expand All @@ -1066,6 +1176,7 @@ mod tests {

fn notifications_to(to: &str) -> Answers {
Answers {
harness: None,
merge_always: false,
base_fix: false,
launch_pull: false,
Expand Down
Loading
Loading