Skip to content

A Spec run can't start a Ticket's Run after the thirdshift binary is replaced mid-run #261

Closed

Description

@JacobStephens2

This was generated by AI during triage.

What happened

The Spec run for #237 on tracewake failed with:

could not start the Run for #248: No such file or directory (os error 2)

#246 and #247 landed; #248 was never started and #249 stayed blocked behind it (Spec PR #257).

Cause

The installed thirdshift was replaced while the Spec run was in flight, and a Spec run starts each Ticket's Run by re-launching its own executable by path.

Time (UTC, 2026-10-01) Event
~08:53 Spec run for #237 starts on 0.5.0 and starts the Runs for #246 and #247
09:06:21 v0.6.0 released
09:08:43 the installed binary is replaced with 0.6.0 (binary and install receipt share this timestamp)
09:26:41 #247 lands, unblocking #248; starting its Run fails with ENOENT
09:43:21 #246, already running, lands; the Spec run ends as a Failed spec run

A child Run is started with the path std::env::current_exe() returns. On Linux that is the target of /proc/self/exe, and once the installer has renamed a new binary over the old one that target reads <install path>/thirdshift (deleted), a path that doesn't exist. The running process is fine, as the README and the update module say, but every child it starts afterwards fails. A Run's Base fix is started the same way, so it is exposed too.

Reproduction

Confirmed outside thirdshift on tracewake (Linux 6.8):

  1. Copy a binary to a, start it, then mv another file over a.
  2. readlink /proc/<pid>/exe prints …/a (deleted); executing that path fails with No such file or directory.
  3. Executing /proc/self/exe from inside the process still runs the original binary.

Any way of replacing the binary triggers it: thirdshift update, the curl installer, or cargo install.

Recovery for #237

Nothing was damaged. Running the Spec again continues the Spec branch and takes #248, then #249.

Activity

  1. JacobStephens2 commented on Oct 1, 2026

    @JacobStephens2
    OwnerAuthor

    This was generated by AI during triage.

    Agent Brief

    Category: bug
    Summary: A Spec run or a Run can no longer start a child Run once the thirdshift binary it was started from has been replaced

    Current behavior:
    A child Run (a Ticket's Run in a Spec run, or a Base fix) is started by spawning the path std::env::current_exe() returns. On Linux, after the installed binary is replaced by a rename, as the release installer, thirdshift update and cargo install all do, that path ends in (deleted) and doesn't exist. The spawn fails with could not start the Run for #<n>: No such file or directory (os error 2), the Ticket is never started, the Tickets it blocks stay blocked, and the Spec run ends as a Failed spec run. The README's claim that updating is safe while a Run is using the old binary is true only for a Run that starts no child afterwards.

    Desired behavior:
    A parent that is already running keeps being able to start child Runs after the binary at its install path has been replaced or removed, and the children run the same binary as the parent, so the hidden arguments a parent passes its child can never meet a different version. On Linux this means starting the child through the kernel's own link to the running executable (/proc/self/exe) rather than through the path that link resolves to. On platforms with no such link (macOS), starting the child by the executable's path, as now, is fine. A child started this way must itself be able to start a child (a Ticket's Run starting a Base fix).

    When a child Run can't be started for any reason, the cause names the executable thirdshift tried to start, so the Failed run's cause line and Run notification explain themselves.

    Key interfaces:

    • The child Run module's start(): the one place a child thirdshift is spawned; what it passes to Command::new is what changes. Its Kind, arguments and stdio stay as they are.
    • The could not start the Run for #<n> error context: gains the executable that was tried.
    • The update module's doc comment and the README's Updating section: both say a Run still using the old binary keeps running; they should stay true and say that this covers the child Runs it starts later.

    Acceptance criteria:

    • On Linux, a Spec run whose binary is replaced (renamed over) or deleted after it started still starts its remaining Tickets' Runs, and they reach their goal.
    • The same holds for a Base fix started by a Run whose binary was replaced after it started.
    • A test covers it end to end: it runs a copy of the built binary, replaces or removes that copy while the parent is running and before a later child is started, and fails without the fix. It may be Linux-only.
    • Children started after the replacement run the parent's binary, not the replacement.
    • Behavior on macOS is unchanged, and the crate still builds there.
    • A failure to start a child Run names the executable that was tried in its cause.
    • The README's Updating section and the update module's doc comment describe the behavior as it now is.
    • cargo test and cargo clippy pass.

    Out of scope:

    • Making thirdshift update refuse, wait or warn while Runs are in flight.
    • Detecting a version change and restarting or upgrading a Run mid-flight.
    • Changing how a Spec run treats a Ticket that fails to start (it ends the Spec run; that stays).
    • The Architect run's dispatch of its Spec run or Run, unless it turns out to start a child thirdshift the same way, in which case it gets the same treatment.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingready-for-agentFully specified, ready for an AFK agent

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions