Skip to content

Parallel Tickets: spec.parallel and parallel <n> - #163

Merged
JacobStephens2 merged 11 commits into
mainfrom
issue-143
Sep 30, 2026
Merged

JacobStephens2 merged 11 commits into
mainfrom
issue-143

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Sep 30, 2026 •

Copy link
Copy Markdown
Owner

Summary

A Spec run now runs several ready Tickets at once. The limit comes from parallel <n>, else spec.parallel in the User config, else 3. main's rule still holds: a failed Ticket stops only the Tickets it blocks.

 land_tickets(spec, tickets, worktree, parallel)
   push the Spec branch
-  while a Ticket is ready
-    run it, wait for it to end, record its outcome
-    read the graph again
+  loop
+    while no error, no interrupt, and running < parallel
+      start the lowest-numbered ready Ticket not running and with no outcome
+        (child thirdshift + a thread that sends (number, outcome) when it ends)
+    if nothing is running: stop
+    wait for any Ticket's Run to end, record its outcome (landed / failed)
+    read the graph again
+    (an interrupt, a start error or a failed read stops new starts; running ones are waited for)
   any Ticket not done → report each one → Failed spec run
   else Spec PR as a draft → Spec review → ready

The Tickets' Runs share one Launch directory. Two things keep git's lock files from failing them:

 Worktree::create_fresh / continue_existing / drop
+  hold <git-common-dir>/thirdshift-worktrees.lock   # one worktree add/remove at a time
 Git::run
+  while git fails on "<x>.lock': File exists" / "could not lock config file"
+    wait 100ms and run it again, for up to 10s  # concurrent fetches of the Spec branch

Command surface and config:

 thirdshift [merge] [email] <Issue URL>
+  parallel <n> | --parallel <n>   any position, once; 0, -1, x → exit 2 with help
+                                  on an issue with no sub-issues → stops before any work
 ~/.thirdshift/config.toml
+[spec]
+parallel = 3   # how many Tickets a Spec run runs at once; default 3

Harness: the fake gh holds a file lock for each call and lets go while hook scripts run, since those call gh themselves. The fake claude locks its record file.

Help and the README's "Spec runs" section document parallel and spec.parallel.

Evidence

  • Before: thirdshift parallel 2 <Spec URL> → unexpected argument. [spec] in the User config → unknown section [spec]. Tickets ran one after another.
    After: new tests in tests/spec_run.rs, all passing:
  • Unit tests:
    • git::a_lock_another_git_holds_is_waited_for: a held config.lock or ref lock is released after 300ms. Before the retry it failed with could not lock config file .git/config: File exists.
    • worktree::a_worktree_is_created_only_once_another_run_lets_go_of_the_launch_directory: fails with the lock removed.
    • config parse and error tests.
  • Other tests: command_surface.rs covers the argument errors and the help. user_config.rs covers spec.parallel = 0 / -2 / 1.5 / "3" and spec.paralel. setup.rs covers spec.parallel = 3 with a comment.
  • Full cargo test --no-fail-fast passes, with clippy and rustfmt clean. In one earlier full run, merge_run::ctrl_c_after_the_merge_finishes_the_post_merge_steps_and_exits_as_merged hit its 10s "agent never started" deadline under load. That test is untouched by this branch. It passed 3/3 alone and in the next full run.

Merge Danger

Door: two-way

Blast Radius: Spec runs

Single-issue Runs are unchanged except in two places. Git::run now retries on a held lock file, so a stale lock left by a crashed git makes a Run wait up to 10s before failing. Worktree creation and cleanup also take a file lock in .git/.

Unaddressed findings

Standards

  • launch_directory() in the worktree.rs tests nearly copies repo_with_origin() in the git.rs tests. They are private test helpers in two modules and differ in setup (clone vs remote config). A shared test-support module for two callers isn't worth it.
  • The "at most once" guard appears in the parallel arm, the SPEC_BRANCH arm and ask_once. Each is two lines with a context-specific message. Sharing it would add indirection for little gain.
  • Parallel { tickets, asked } lives in spec_run but is built in main.rs and read in run.rs. It keeps the limit and whether the command asked for it together. run.rs is where the "not a Spec" check has to happen, once the Tickets are read.
  • land_tickets schedules, reads the graph again and reports. The report is already its own function (report), and the loop is the scheduler. Splitting the loop out would only move its locals into a return type.
  • ended / endings for the channel ends. They sit a few lines apart, and each one's type makes its role clear. (The vaguer read was renamed to reread.)
  • The default of 3 is written in code, DEFAULTS and the help. the_defaults_setup_writes_are_what_a_run_does_with_no_user_config already ties the code default to DEFAULTS. The help is prose.
  • held_lock matches git's English stderr. git has no exit code for a held lock. Forcing LC_ALL=C on every git call would also change the language of hook output that the Run relays.

Spec

  • The lock retry and the worktree lock go beyond the spec's "serialise worktree creation, or retry on a lock". They fix different failures. The retry covers idempotent commands such as fetches from any worktree. The flock covers worktree add -b / branch -D, which can fail partway, where a retry would be wrong. The lock also covers cleanup, the other half of creation.
  • branch::select and pull_base_branch run outside the worktree lock and rely only on the retry. They are reads or idempotent fetches, which is the case the retry exists for.
  • The retry recognises only two lock messages. They are the messages git prints for ref, index and config locks ('<path>.lock': File exists and could not lock config file). Nothing else has been seen in tests.
  • The fake claude also takes a lock, which wasn't asked for. It was needed: parallel sessions write its record file at once.
  • An error that isn't a Ticket failing, or an interrupt, waits for the running Tickets before the Spec run fails. Each child Run goes through its own Failed run path. Abandoning them would orphan them mid-cleanup. Interrupting a Spec run #145 owns interrupt behaviour.
  • Commit 8150374 is an empty automated "failed run" marker in the history. The branch is already pushed, so it stays rather than rewriting shared history. It will vanish in a squash merge.
  • The Launch-directory integration test doesn't force a collision. The deterministic coverage comes from the two unit tests above.

Closes #143

🤖 Generated with Claude Code

JacobStephens2 and others added 4 commits September 30, 2026 00:07
…c.parallel

Whenever a Ticket's Run ends, the Spec run reads the graph again and starts
ready Tickets, lowest number first, until the limit is reached: the
parallel <n> flag, else spec.parallel in the User config, else 3. Setup
writes spec.parallel at its default. parallel on an issue with no
sub-issues stops before any work.

Tickets' Runs share the Launch directory, so worktree creation and cleanup
there take a file lock, and a git command that fails on a lock file another
git holds is tried again. The fake gh and claude lock their state files,
as parallel child Runs call them at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#144

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each running Ticket's thread now runs main's interrupt-aware wait: it
passes an interrupt on to its Run and reports the Ticket as interrupted,
so an interrupt during a parallel Spec run ends every running Ticket
through its Failed run path before the Spec run ends as interrupted.
Main's terminal setup tests now expect the spec.parallel key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30T00:43:09Z, host tracewake. Uncommitted work at the time of failure is included in this commit.
@JacobStephens2
JacobStephens2 marked this pull request as draft September 30, 2026 00:43
JacobStephens2 and others added 2 commits September 30, 2026 02:14
Parallel scheduling now keeps main's rule that a failed Ticket stops only
the Tickets it blocks, and passes the Spec review's Logs through.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd a clearer name for the Tickets read again

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JacobStephens2
JacobStephens2 marked this pull request as ready for review September 30, 2026 02:45
JacobStephens2 and others added 5 commits September 30, 2026 02:45
A blocker's Run closes its issue before it ends, so re-reading the
Tickets after another Ticket ended could find the blocker closed and
start the blocked Ticket before the blocker's Run had landed. This made
independent_tickets_run_at_once_and_the_ticket_they_block_waits_for_both
flaky on macOS CI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolve spec_run::run taking both main's Goal (Repair loop and
Self-merge for the Spec PR) and this branch's parallel Ticket count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two Tickets of a Spec run fetching the same Base branch from one Launch
directory can race on refs/remotes/origin/<base>: one fetch fails with
"cannot lock ref ... is at X but expected Y". Treat that like a held lock
file and run the command again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JacobStephens2
JacobStephens2 merged commit 8e3706b into main Sep 30, 2026
13 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-143 branch September 30, 2026 03:53
@JacobStephens2 JacobStephens2 mentioned this pull request Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parallel Tickets: spec.parallel and parallel <n>

1 participant