Skip to content

Spec: thirdshift release Command and Pass for scheduled and on-demand releases - #2162

Merged
JacobStephens2 merged 31 commits into
mainfrom
issue-2054
Nov 15, 2026
Merged

JacobStephens2 merged 31 commits into
mainfrom
issue-2054

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Nov 13, 2026 •

Copy link
Copy Markdown
Owner

Summary

This Spec adds thirdshift release as a native Release run Command, scoped to JacobStephens2/thirdshift. It replaces scripts/release.sh, scripts/release-experiment.py and the hidden --release-summary shim; scripts/release-notes.sh stays for release-notes.yml.

  • Without <version> (bare, or with --patch, --minor or --major) it runs as a Pass.
  • With thirdshift release <version> it runs as a direct Command that cuts or resumes that version.
thirdshift release [<version> | --major | --minor | --patch] [--with-experiment] [--review]
  fences: Linux · not inside a factory Session · origin is JacobStephens2/thirdshift
  Pass admission: live Experiment / Quota wait → quiet skip (Activity log, dedup)
  ~/.thirdshift/release.lock + origin release reservation (outside the Command limit)
  recovery::discover                       # unfinished release-X.Y.Z / PR / unpublished vX.Y.Z
    machine-recoverable → resume or recut   # pushed branch, open non-draft PR, failed unpublished workflow
    human-blocked       → Pass skips / direct fails
  selection::candidate                     # newest green first-parent main commit with shipped paths
    pending main CI → waits; record live in Working, reachable by Stop
    all red → Pass skips / direct fails
  summary::select                          # one tool-less bwrap Session: minor|patch + notes (never major)
    failure → patch + generated notes outside the empty summary markers
  [--with-experiment] in-process Experiment against the last release tag
  bump commit (Cargo.toml + Cargo.lock only) → Release X.Y.Z PR (in-progress while live)
  wait for checks and mergeability (5-minute grace) · ≤1 targeted Fix run (--no-merge --no-base-fix)
  push vX.Y.Z → wait for release.yml → merge the bump PR → Done (or Needs you)

Around the command:

 Review run / Fix run Passes
+  skip release-* PRs into main (targeted `thirdshift fix <pr> --no-merge` still works)
 POST /api/v1/start, Console Start controls
+  release (version | bump, with_experiment, harness/email fields) behind admin scope
+  release --review stays CLI-only (ADR 0026 parity exclusion)
 Destructive-command signals
+  thirdshift release, git push --tags / refs/tags/
 .github/workflows/release.yml (generated)
+  release-tag-guard: github.actor == JacobStephens2 and the tag tree == main ancestor + exact version bump
+  processes/** in RELEASE_PATHS
 scripts/
+  setup-release-tags.sh   # Day shift wizard for the Admin-only v* tag ruleset
+  release-tag-guard.py
-  release.sh, release-experiment.py, release-summary.md

Whole-Spec review fixes (c950a6f):

  • A resumed bump PR whose body still names its content base keeps its summary, including one edited with --review. Before this fix, every resume ran a new summary Session and overwrote the body.
  • The --with-experiment pin is fetched from the run's own isolated repository by path, so the Launch directory's origin settings (uploadpack, sshCommand, credential helpers) never run.
  • The pin is removed once release-X.Y.Z is on origin. An abandoned version no longer blocks every later Pass with "awaits its Experiment".
  • Hints printed during the bump-PR wait (… then thirdshift release X.Y.Z) keep --with-experiment when the run used it.
  • The manual now lists the Plan usage skip and says when a resumed summary is kept. Test changes: Console start requests use the 45-second helper, pending-CI waits use WAIT_BOUND, and the gh fake's unreachable --jq branch is removed.

Earlier review fixes (c98a51f): a Pass without --with-experiment never publishes a pinned version; a Pass waiting on main CI keeps a live record; release waits have their 5-minute grace; a failed summary leaves the markers empty; a refused Experiment prints per-suite results.

Tests sit at one seam, the thirdshift release binary run against a bare origin with fake gh and Harnesses (tests/release/, tests/experiments/release.rs), plus the Console API (tests/console/start.rs) and the workflow generator unit tests.

Closes #2054. This also resolves #336, and #653 for local-file reads. The remaining toolchain binds are tracked in #2230.

Evidence

Execution evidence

This host now has a C toolchain, so every run below was made here, directly.

  • Before (HEAD df76d0f): the reviewers' tests, added to tests/release/native.rs and run with cargo test --locked --test release -- resuming_an_open_bump_pr_keeps release_experiment_pin_does_not_run an_abandoned_experiment_cut_leaves_no_pin (plus the two declined tests listed under Unaddressed findings):
    native::resuming_an_open_bump_pr_keeps_its_reviewed_summary_when_the_base_is_unchanged
      panicked: resume overwrote the reviewed summary: … <!-- release-summary:start -->\nRegenerated summary.
    native::release_experiment_pin_does_not_run_launch_repository_git_programs
      panicked: Release run executed the Launch directory's remote.origin.uploadpack
    native::an_abandoned_experiment_cut_leaves_no_pin_that_blocks_later_passes
      panicked: … Release run skipped: unfinished local release 0.2.0 awaits its Experiment; resume it with thirdshift release 0.2.0 --with-experiment
    
    cargo test --locked --test experiments -- native_release_resumes_its_experiment_and_refuses_an_unattended_inconclusive_verdict, with the new --with-experiment hint assertion:
    panicked at tests/experiments/release.rs:121:5:
    thirdshift: no release bump checks appeared at 89b23c78…; inspect GitHub Actions, then thirdshift release 0.2.0
    
    After (c950a6f): each of those commands gives test result: ok, with 1 or 3 passed and 0 failed.
  • Release suite: cargo test --locked --test release gave 141 passed; 0 failed; 3 ignored at df76d0f and 144 passed; 0 failed; 3 ignored after the fixes.
  • scripts/affected-tests refs/remotes/origin/main chose the full suite:
    • npm test --prefix site: PASS.
    • cargo build --locked: PASS.
    • npm test --prefix console: one failure, Fleet Inbox polls with page subscriptions, retains rows, and fits a phone, a 5-second locator timeout while the machine was loaded. On its own, node --test tests/fleet.test.cjs gives ℹ pass 2 ℹ fail 0.
    • The script stops at its first failure, so I ran its remaining steps by hand. python3 -m unittest discover -s scripts -p test_generate_release.py ran 10 tests, OK, and test_affected_tests.py ran 46 tests, OK.
    • Full Rust suite, cargo test --locked --no-fail-fast: 4648 passed, 2 failed and 207 ignored.
      • release experiments::repaired_content_waits_for_its_own_experiment_before_tagging timed out under full-suite load: Experiment at 520067a5… never reached waiting_on_rechecks. Run alone, it gives 1 passed, and the release suite on its own gave 144 passed.
      • plan_usage plan_usage_shows_configured_routes_in_order_and_omits_api_key_and_unused_harnesses asserts that agy is not installed, but this host has ~/.local/bin/agy. With that directory off PATH it gives 1 passed. This diff doesn't touch it.
  • Other checks:
    • cargo clippy --locked --all-targets -- -D warnings: clean.
    • cargo fmt --all: applied.
    • cargo test --locked --bin thirdshift prompts_page: 14 passed.
    • npm run docs:check --prefix site: exits 0 after regenerating the manual.
    • python3 scripts/generate-release.py --check: exits 0 with a clean tree.

Visual evidence

The whole-Spec review fixes (c950a6f) change no rendered UI, so these captures, from the earlier round, still show the Console change.

Console Start work, signed in with full scopes, for a JacobStephens2/thirdshift Launch directory, with the machine name redacted to host.example. Before is the Base branch main at 864c1b8; After is this branch, with Start Release run opened.

Desktop 1440×900:

Before
After

Mobile 390×844:

Before
After

The hand-written site page changes are text only: release wording, the five-minute grace and the start-options scopes.

Review coverage

  • This review covered the whole Spec branch, refs/remotes/origin/main...df76d0f1 (104 files). Each axis had one follow-up pass for files it had not read.
  • Standards: read every changed file except the three generated paths below; that includes the removed scripts/release.sh in full.
  • Spec: opened every changed file except the three generated paths below. It read some only in part:
    • removed scripts/release.sh lines 230-405, 470-590 and 720-1049;
    • tests/release.rs lines 1050-1799 and 1885-2199, read only as a test-name list;
    • tests/release/recovery.rs lines 60-150 and the tests/release/experiments.rs helpers, skimmed by name.
  • Both axes excluded these generated paths and did not read them:
    • site/docs/index.html, generated from docs/guide.md; npm run docs:check --prefix site exits 0.
    • .github/workflows/release.yml, generated by scripts/generate-release.py with scripts/release-tag-guard.py; python3 scripts/generate-release.py --check exits 0. Both reviewers checked that its guard job gates plan, host and custom-publish-crates.
    • prompts/README.md and site/prompts/index.html, generated from src/prompt.rs and src/prompts_page.rs; cargo test --bin thirdshift prompts_page passes 14 tests. prompts/release-*.md were read directly.
  • Changed files left unread: none.

Unaddressed findings

Standards

All of these are judgement calls under the smell baseline, with no documented standard behind them and no defect shown.

  • Test #1, Spec: thirdshift, turn a GitHub issue into a ready-for-review PR unattended #2, Walking skeleton: a fresh run on the happy path #3, Pre-flight checks #16: Primitive Obsession and Duplicated Code (String versions, string workflow conclusions, merge-base ancestry checks, repeated Cut literals). These are refactors with no behaviour change, and the release suite pins the behaviour. They are left for an Architecture review rather than churned into this Spec's last pass.
  • Pre-flight checks #4, LiveLabel vs review::reservation::Reservation. Reservation also owns the Review run's acquisition and stale-disposal rules (ADR 0014), which a Release run must not inherit. LiveLabel only adds and removes in-progress for ADR 0029.
  • The Failed run path #5, the hard-coded repository in src/console/start_controls.rs:111. crate::release exists only on Linux (src/main.rs, #[cfg(target_os = "linux")] mod release;), while the Console builds on every platform.
  • Continuation of an existing Issue branch #6, start scopes in three encodings. tests/console/start.rs (release_start_schema_exposes_pass_bumps_and_standard_options) checks the published OpenAPI x-command-scopes, x-scope and x-any-scopes. Unifying PageRoute scopes is a Console-wide refactor beyond this Spec.
  • Numbered Issue branches after a merged or closed PR #7, Progress lines on stderr #10, flag arguments and the 7-parameter fix::targeted_at. These are judgement calls on existing interfaces this Spec extended.
  • Keeping the PR mergeable: merge the Base branch and conflict Repair #8, part of it. The doc comment now sits on run, and Ok(None) is documented. The repeated --review and session checks are deliberate layers: main.rs refuses before Setup, as the Spec requires ("validated up front before expensive work"), and release::run guards the in-process entry used by the Console.
  • #12a, THIRDSHIFT_POLL_MS=1. Raising it would slow every polling test in the suite. The waits are bounded by pending_polls and are not deadlines, so the Deadlines rule does not apply.
  • #12c and #12e. The five live-wait loops differ in what they watch for, and the RELEASE_GRACE test is a one-line guard. Both are judgement calls.
  • Make a command to read the thirdshift activity logs #14, two shipped-path lists. RELEASE_PATHS (scripts/generate-release.py:22) decides which PRs exercise release.yml. The Pass filter is the Spec's list of shipped paths, plus .cargo/ and askama.toml, which change the compiled binary. The two answer different questions, and docs/ci-performance.md names both.
  • README, credit and install docs #15, rewritten historical docs, except the typo. The lower-case "the former…" in docs/research/eval-build-vs-buy.md is fixed. Restoring scripts/release.sh:NNN references would point at a file this Spec deletes.
  • Print progress lines on stderr #18, the bump rule in Rust and in release-tag-guard.py. native_release_publishes_green_content_without_reading_the_launch_tree (tests/release/native.rs) runs scripts/release-tag-guard.py against the tag the Release run actually pushed, so the two implementations are checked against each other end to end, though on fixture manifests. A check against this repository's own manifests is a reasonable follow-up, but no drift exists today.

Spec

Security

  • None. The one finding introduced by this change was fixed in this branch; see below.
  • Fixed: an open fork PR named release-X.Y.Z blocked every Release run (low, introduced by this change).
    • Cause: recovery::inspect (src/release/recovery.rs) treated any open PR whose head is named release-* as a live unfinished release. That included a fork PR from any GitHub account, even when origin had no branch or tag for that version.
    • Proof: tests/release/native.rs an_open_fork_pr_on_a_release_branch_name_does_not_block_releases opens such a fork PR and then runs thirdshift release 0.2.0.
      • On HEAD before the fix it failed: "unfinished release 9.0.0 is live; finish it with thirdshift release 9.0.0 before releasing 0.2.0".
      • At the merge base, the same scenario against scripts/release.sh passed.
      • After the fix it passes, and it stays as a regression test.
    • Fix: a cross-repository PR counts only when origin already holds that version's branch or unpublished tag. A fork PR that shadows a real release branch is still refused (a_resumed_bump_pr_from_another_repository_is_refused).

Security review outcome

No unaddressed introduced findings.

Merge Danger

Door: two-way for the code. Each release a scheduled Pass publishes is a one-way door: tags on GitHub Releases are never moved.

Blast Radius: release pipeline.

  • Merging enables unattended releases from main.
  • The release.yml tag guard refuses tags not pushed by JacobStephens2 or not an exact version bump of a main ancestor.
  • Review and Fix Passes stop touching release-* PRs.
  • Console release starts need admin; /api/v1/start-options now accepts run or admin.
  • The tag ruleset still has to be provisioned by hand with scripts/setup-release-tags.sh.

Tickets

Built with claude 2.1.317 · claude-opus-5-5 · high

Fence release bump Passes and tag publication
A Release checkout and add ran inside a worktree of the launch repository, so its clean and smudge filters and core.fsmonitor ran even with hooks disabled. Release git now uses a temporary repository whose origin is that repository's effective URL, and launch-repository reads disable hooks and fsmonitor.

The regression test plants a filter and fsmonitor in the launch repository and requires a successful release that does not run them.
The native Release happy-path test requires Bubblewrap and socat, but Rust CI did not prepare its sandbox dependencies or Ubuntu namespace permission. A sandbox startup failure correctly falls back to generated notes, causing the summary assertion to fail.

Install the dependencies, grant bwrap its scoped AppArmor namespace permission, and probe startup before tests. Document the prerequisites and include command stderr in the unchanged summary assertion.
Add Release runs for a specified version
A release-* branch pusher can amend an exact-bump commit message. Discovery treated that trailer as authority to drop another live cut, publish the planted version, and close the other pull request. Two live cuts now stay blocked. A recut in the current process still records the version it replaced.
Resume native releases and add Experiment and review controls
Refuse a fresh cut whose tag is already on origin before candidate
selection, name gh pr ready when a draft bump PR blocks the merge after
publication, start over when a closed bump PR's branch is gone, keep the
failed-workflow message when its job details cannot be read, and name the
PR when its head cannot be fetched. Restore the ported tests' specific
assertions, the polling and local-tag scenarios and the declined Experiment
override; remove the fake gh's script-only jq branches; correct the manual,
testing and navigation docs and the homepage override rule.
Complete the native release cutover
Spec axis:
- A Release Pass without --with-experiment no longer publishes a version
  pinned for an Experiment; it skips naming the resume command.
- A Pass waiting on pending main push CI keeps a live Command record, so
  it shows in Working and Stop reaches it; a later skip ends it skipped.
- Release bump-PR, mergeability and release-workflow waits use a
  five-minute grace again, as the Spec and the retired script did.
- An unavailable summary leaves the bump PR's markers empty, so the
  GitHub Release lists the generated notes once with no placeholder.
- A refused release Experiment prints its per-suite results, Advisory
  suites included, on stderr before the override or failure.
- The Day shift ruleset wizard and the workflow generator no longer make
  a scheduled release eligible.
- Tests cover draft GitHub Release recovery, quiet skips with email on,
  and Cron jobs recognition of versionless release entries.

Standards axis:
- Correct the start-options scope, the start API's with_experiment field,
  the release grace and the Cron jobs note in the manual and site pages.
- Record the Release summary sandbox exception in ADR 0013 and the Pass's
  shipped-path filter in docs/ci-performance.md.
- Insert release.yml's tag guard above plan's own comment.
- Run release tests under the fixture bounds, Experiment-bearing ones
  under the Experiment bound, and drop stale script-era wording, dead
  branches and the echoed Candidate previous tag.

Refs #2054
@JacobStephens2
JacobStephens2 marked this pull request as ready for review November 14, 2026 16:09
Resolve conflicts with main:
- Start work panel keeps main's compact `start-work` class and this
  branch's data-start-run / data-start-release attributes.
- console/tests/start.test.cjs keeps both the Release start tests and
  main's closed Start work layout test.
- The guide's `admin` scope row keeps Release starts; the `report` row
  takes main's plan-usage report route.

Main gave command_limit::early_skip the Pass's Harness choice for its
Plan usage gate (ADR 0031: every scheduled Pass skips when no route has
Plan usage), so the Release Pass now passes its own choice.
The site test added for #2054 rejects retired scripts/release.sh mentions;
two Fix and Review Pass sentences arriving via main still named it.
@JacobStephens2

Copy link
Copy Markdown
Owner Author

Delivery completed on agent-authored or reviewed head df76d0f.

@JacobStephens2 JacobStephens2 added in-progress A Claim: thirdshift has taken this issue and removed in-progress A Claim: thirdshift has taken this issue labels Nov 14, 2026
@JacobStephens2
JacobStephens2 marked this pull request as draft November 14, 2026 22:17
- A resumed bump PR whose body still names its content base keeps its
  summary (including one edited with --review) instead of regenerating it.
- The --with-experiment pin fetches from the run's isolated repository by
  path, so the Launch directory's origin settings never run, and is removed
  once the release branch is on origin, so an abandoned version no longer
  blocks later Passes.
- Recovery hints from the bump-PR wait carry --with-experiment when the
  run asked for it.
- The manual names the Plan usage skip and the kept summary.
- Test hygiene: Console start requests use the 45 s helper, pending-CI waits
  use WAIT_BOUND, and the gh fake's unreachable jq branch is gone; doc
  comments on Request, run and Kind::ReleaseSummary, and a note on why
  Release Passes keep held lines.
@JacobStephens2 JacobStephens2 removed the in-progress A Claim: thirdshift has taken this issue label Nov 14, 2026
Anyone can open a fork pull request whose head branch is named
release-X.Y.Z. Release discovery treated such a PR as an unfinished
release, so one outside PR blocked every Release run until it was closed.
A fork PR now counts only when origin already holds that version's
branch or unpublished tag.
@JacobStephens2
JacobStephens2 marked this pull request as ready for review November 15, 2026 02:25
@JacobStephens2
JacobStephens2 merged commit ea4bd94 into main Nov 15, 2026
25 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-2054 branch November 15, 2026 02:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant