Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
0af0b68
feat(skills): add release-captain skill for the weekly release cadence
matthewgrossman Oct 12, 2026
46e8ab0
docs(skills): document starting a release train after tag-push triggers
matthewgrossman Oct 14, 2026
b50bb2b
docs(skills): create lightweight release tags regardless of signing c…
matthewgrossman Oct 14, 2026
50223f1
docs(skills): capture release captain lessons from starting 0.1.5
matthewgrossman Oct 14, 2026
e3d0caa
docs(skills): add recovery for pre-releases failing on new RustSec ad…
matthewgrossman Oct 14, 2026
7ec8363
docs(skills): recommend fixing forward to a green pre-release
matthewgrossman Oct 14, 2026
f8fbb86
docs(skills): describe the triage channel and thread replies for pre-…
matthewgrossman Oct 15, 2026
4e99714
docs(skills): drop Codex Security from release captain run matching
matthewgrossman Oct 15, 2026
a5834d2
docs(skills): describe where and how to post the release coordination…
matthewgrossman Oct 19, 2026
58a369a
docs(skills): make seeding the next train the outgoing captain's handoff
matthewgrossman Oct 19, 2026
9a3f3ae
docs(skills): update release captain for the Release workflow and add…
matthewgrossman Oct 19, 2026
2000195
docs(skills): explain what the release cutoff guarantees
matthewgrossman Oct 19, 2026
c17528e
docs(skills): add release notes review and v0.1.5 release day lessons
matthewgrossman Oct 20, 2026
2a5e536
docs(skills): make starting the next train the captain's final duty
matthewgrossman Oct 20, 2026
9137d48
docs(skills): trim the release captain skill to its durable steps
matthewgrossman Oct 20, 2026
993e33a
Merge remote-tracking branch 'origin/main' into feat/4433-release-cap…
matthewgrossman Oct 20, 2026
218c413
docs(ci): document Release Auto-Tag and pre-release notes previews
matthewgrossman Oct 20, 2026
4c5a04b
docs(skills): add a template for the release announcement reply
matthewgrossman Oct 20, 2026
bace38b
docs(skills): watch pre-releases in the week before your turn
matthewgrossman Oct 20, 2026
9d46fc5
docs(skills): remove internal channel references from release captain
matthewgrossman Oct 20, 2026
a3fd474
docs(skills): fix pre-release failures in a new number via Auto-Tag
matthewgrossman Oct 20, 2026
72d8340
Merge remote-tracking branch 'origin/main' into feat/4433-release-cap…
matthewgrossman Oct 21, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
228 changes: 228 additions & 0 deletions .agents/skills/release-captain/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,228 @@
---
name: release-captain
description: Run the weekly OpenShell release captain checklist. Monday checks the pre-release train, triages failures and the compatibility review, and drafts the coordination post. Tuesday promotes the qualified pre-release after approval, verifies publication, and starts the next train. Trigger keywords - release captain, weekly release, Tuesday release, cut the release, release checklist, pre-release status, release train, start next train, stable release.
metadata:
internal: true
---

# Release Captain

OpenShell publishes a stable release every Tuesday from a qualified
pre-release ([RFC 0014](../../../rfc/0014-release-stability/README.md)). The
release captain keeps the week's pre-release train green, coordinates the last
merges, cuts the release, and starts the next train.

## How to use this skill

| When | Ask | What happens |
|---|---|---|
| The week before your turn | "I'm release captain next week. Is the train ready?" | Checks that the next version's `-pre.1` exists. If not, ask this week's captain to start the train (Tuesday step 6). Then check each weekday that Auto-Tag's pre-release ran and passed (`gh run list --workflow release-prerelease.yml --limit 3`), and fix failures as they happen (Monday step 3) instead of on Monday. |
| Monday | "Run the release captain Monday checklist." | Monday steps 1–4. |
| A pre-release failed | "`vX.Y.Z-pre.N` failed, help me fix it." | Monday step 3 and Recovery. |
| Tuesday | "Cut the release." | Tuesday steps 1–6, ending with starting the next train. |

## Guardrails

Get explicit approval, with the exact command and target SHA, before pushing a
tag, dispatching, rerunning, or cancelling a workflow, merging or approving a
PR, or posting in Slack or GitHub. Never repoint a stable tag, delete a `-pre.N`
tag, or start a second `Pre-release` run for a tag; it has no concurrency guard.

## How the train works

- A train has one base version, such as `v0.1.4`, fixed by its `-pre.1` tag. A
patch train can't take breaking changes to Stable interfaces.
- `Release Auto-Tag` runs on weekdays at 14:00 Europe/Zurich. If `main` has
commits since the latest `-pre.N`, it tags the next one and dispatches
`Pre-release` (`release-prerelease.yml`, about 45 minutes). It skips when the
train has no `-pre.1` or the base version is already stable.
- `Pre-release` builds, then publishes and qualifies in parallel, and writes a
`release-notes` preview artifact. Pushing a `-pre.N` tag by hand starts it.
- `Release` (`release.yml`) is dispatched by hand with the stable version. It
picks the highest `-pre.N` on `main` whose whole `Pre-release` run passed,
creates the stable tag on that commit, and publishes without requalifying.

## Setup

```shell
export GH_REPO=NVIDIA/OpenShell
git fetch origin main --tags --force
PREV=$(git tag -l 'v*' --sort=-v:refname | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | head -1)
git tag -l 'v*' --sort=-v:refname | grep -E -- '-pre\.[0-9]+$' | head -5
```

Find the `Pre-release` run for a tag. A tag pushed by hand runs on the tag; an
Auto-Tag run is dispatched on `main`:

```shell
TAG=vX.Y.Z-pre.N
gh run list --workflow release-prerelease.yml --branch "$TAG"
gh run list --workflow release-prerelease.yml --event workflow_dispatch --commit "$(git rev-parse "$TAG^{commit}")"
```

Preview the candidate `Release` would promote, read-only:

```shell
VERSION=vX.Y.Z
wt=$(mktemp -d)
git worktree add -q --detach "$wt" origin/main
(cd "$wt" && GITHUB_REPOSITORY=NVIDIA/OpenShell uv run -q --no-project python tasks/scripts/select-release-candidate.py "$VERSION" --branch main | jq .)
git worktree remove --force "$wt"
```

## Monday: pre-flight

1. **Check the candidate.** Run the preview. If the latest `-pre.N` isn't the
candidate, its run failed; read why:

```shell
RUN=<run-id>
gh run view "$RUN" --json jobs --jq '.jobs[] | select(.conclusion == "failure" or .conclusion == "timed_out") | "\(.name)\t\(.url)"'
gh run view "$RUN" --log-failed
```

If the next version has no `-pre.1`, start the train (Tuesday step 6).

2. **Triage the compatibility review** for the candidate's run. It's advisory,
and a green job doesn't mean it found no breaks:

```shell
ATTEMPT=$(gh run view "$RUN" --json attempt --jq .attempt)
dir=$(mktemp -d)
gh run download "$RUN" -n "compatibility-review-run-$RUN-attempt-$ATTEMPT" -D "$dir"
jq '{status, verdict: .review.verdict}' "$dir/report.json"
cat "$dir/report.md"
```

Only `complete` with `no_unaddressed_breaks` is clean. A real break needs a
fix, a revert, a minor train, or a maintainer-approved exception with upgrade
steps in the release notes.

3. **Fix forward.** A failed pre-release doesn't end the train. Find the cause,
and let the failed run finish rather than cancelling it.

A rerun repeats the same commit and workflow, so it only helps with a
transient failure (see Recovery). Anything that needs a code or workflow
fix ships in a new number: that `-pre.N` stays failed, and `Release` skips
it. Merge the fix, then cut the next `-pre.N` from the tip of `main` after
approval:

```shell
gh workflow run release-auto-tag.yml --ref main
```

Tag a specific commit by hand (as in Tuesday step 6) only when the tip of
`main` isn't the commit you want.

4. **Draft the coordination post.** Start a thread in the OpenShell channel of
the CNCF Slack. The release ships from `main` with no release branch, so the
thread is for getting ready PRs reviewed and merged before the cutoff. List
approved PRs:

```shell
gh pr list --base main --state open --search "review:approved" --json number,title,author --jq '.[] | "#\(.number) \(.title) (@\(.author.login))"'
```

The release ships the final pre-release's commit, if it passes. Pick a
cutoff before Tuesday's Auto-Tag run, or a later one after which you tag the
final pre-release by hand. Don't promise that everything merged by the
cutoff ships, or name the next version.

```text
:thread: OpenShell vX.Y.Z release, Tuesday <date>. I'm release captain this week.

Status: vX.Y.Z-pre.N (<commit>) passed qualification. <blockers and owners, or "No blockers.">

Reply here with any ready PRs you'd like in vX.Y.Z. Merge cutoff is <HH:MM UTC> (<local time>) Tuesday. <"Tuesday's automatic pre-release picks up everything merged by then" | "I'll cut the final pre-release after the cutoff">, and vX.Y.Z ships from it if it passes qualification. Anything that misses it goes in the next release.
```

If the pre-release that ships was cut before the posted cutoff, say so in
the thread, with its commit and time.

## Tuesday: cut the release

1. **Select the final candidate.** For a late cutoff, cut the final `-pre.N`
after the cutoff as in Monday step 3. When it finishes, run the preview and
repeat Monday step 2 for its run.

2. **Request approval.** Present the candidate tag, commit, qualification run,
compatibility assessment, commits since `$PREV`, and release notes preview:

```shell
git log --oneline --no-merges "$PREV..<candidate_tag>"
gh run download "$RUN" -n release-notes -D "$dir/notes" && cat "$dir/notes/release-notes.md"
```

The notes come from each PR's "Release Notes" section. To fix them, edit the
PR, then preview with
`gh workflow run release.yml --ref main -f tag="$VERSION" -f notes-only=true`.

3. **Dispatch the release.** After approval:

```shell
gh workflow run release.yml --ref main -f tag="$VERSION"
```

4. **Watch and verify.** Check that `Select Qualified Prerelease` names the
approved candidate, the GitHub Release has assets, no job failed, and the
wheel reached PyPI:

```shell
gh run watch <run-id> --exit-status
gh release view "$VERSION" --json isPrerelease,publishedAt,assets --jq '{isPrerelease, publishedAt, assets: (.assets | length)}'
```

5. **Close the loop.** Draft a short reply to the coordination thread. Point
to the release notes rather than repeating them:

```text
:white_check_mark: :ship: OpenShell vX.Y.Z is out: https://github.com/NVIDIA/OpenShell/releases/tag/vX.Y.Z

It shipped from vX.Y.Z-pre.N (<commit>). I'll be starting the <next version>-pre.* jobs shortly for next week.

Please check the release notes for upgrade notes that operators might care about. I'm also moving everything still open in the X.Y.Z milestone to <next milestone>.

Thanks everyone!
```

After approval, move the milestone's open items to the next milestone.

6. **Start the next train.** This is this week's captain's last duty. Until the
next `-pre.1` exists, Auto-Tag skips. Agree with the maintainers on patch
(the default) or minor
([version selection](../../../rfc/0014-release-stability/release-version-selection.md)).
Seed it on a commit after `$VERSION` with a passing `Release Dev` run:

```shell
for sha in $(git rev-list --max-count=10 "$VERSION..origin/main"); do
state=$(gh run list --workflow release-dev.yml --commit "$sha" --json status,conclusion --jq '.[0] | if .status == "completed" then .conclusion else .status end')
echo "${sha:0:9} ${state:-none} $(git log -1 --format=%s "$sha")"
done
```

After approval, push the tag. The `-c` and `/dev/null` keep `git` from
signing the tag and opening an editor:

```shell
NEXT=vX.Y.Z-pre.1 NEXT_SHA=<commit>
git -c tag.gpgSign=false tag "$NEXT" "$NEXT_SHA" </dev/null
git push origin "refs/tags/$NEXT"
```

You're done when that `Pre-release` run passes, or its failure is handed to
next week's captain.

## Recovery

| Symptom | Action |
|---|---|
| Transient infrastructure failure in a pre-release, including a registry push | After approval, `gh run rerun <run-id> --failed`. `Release` won't select the run until all of it passes. |
| Pre-release run ends in `startup_failure` | The workflow file is invalid at that commit. Fix it in a new `-pre.N` (Monday step 3). |
| Code defect or `Cargo Deny` advisory | Rerunning won't help. Fix it in a new `-pre.N` (Monday step 3). |
| `Release` fails after selection | After approval, rerun only the failed jobs that haven't published. |

## Related

- `release-openshell`: the short procedure behind Tuesday steps 1–4.
- `watch-github-actions`: general `gh run` monitoring and log retrieval.
- `CI.md`: release workflows, qualification profile, and scanner policies.
1 change: 1 addition & 0 deletions .agents/skills/sync-agent-infra/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ Run after adding, removing, or renaming a skill or crate; changing issue or PR c
| Inference providers or native model endpoints | `debug-inference`, `openshell-cli` |
| TUI behavior | `tui-development` |
| Release artifacts or smoke coverage | `test-release-canary` |
| Release cadence, tagging, or qualification workflows | `release-captain` |
| CI diagnostics | `watch-github-actions` |
| Gator supervision | `launch-openshell-gator` |
| SBOM workflow | `sbom` |
Expand Down
1 change: 1 addition & 0 deletions CI.md
Original file line number Diff line number Diff line change
Expand Up @@ -637,6 +637,7 @@ These workflows run after merge to publish dev/tagged artifacts and verify them.
|---|---|
| `.github/workflows/release-dev.yml` | Publishes the rolling `dev` build on every push to `main`. Builds gateway, sandbox, and supervisor images and binaries, packages, wheels, and pushes the Helm chart as `oci://ghcr.io/nvidia/openshell/helm-chart:0.0.0-dev` (plus an immutable `0.0.0-dev.<sha>` pin). Also dispatchable manually. |
| `.github/workflows/release.yml` | Manually promote the latest qualified prerelease for a version and branch, create its stable tag, then rebuild and publish. Support releases preserve shared latest channels. Publishes the prerelease's notes preview, retagged, or generates notes when it is unavailable. The notes-only mode generates a release notes preview artifact. |
| `.github/workflows/release-auto-tag.yml` | Weekdays at 14:00 Europe/Zurich, tags the next `-pre.N` on `main` when commits landed since the latest one, then dispatches Pre-release. It skips when the train has no `-pre.1` or its version is already stable, so push each train's `-pre.1` tag by hand. |
| `.github/workflows/release-prerelease.yml` | Pre-release orchestrator, triggered by prerelease tag pushes or dispatched by Release Auto-Tag or manually: build, then publish and qualify in parallel. Qualification failure fails the run and sends the failure notification; published artifacts remain available. |
| `.github/workflows/release-build.yml` | Shared version computation, binary and image builds, and package builds. Validates that the tag matches the caller's release kind. |
| `.github/workflows/release-publish.yml` | Shared artifact assembly, OCI image tags and Helm chart publication, and package publication. Stable releases also publish to GitHub Releases, snap, npm, PyPI, and Fern. |
Expand Down
1 change: 1 addition & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@ Contributor and maintainer skills live in `.agents/skills/`. They are marked int
| Platform | `build-openshell-mxc-windows` | Maintain and validate the build-only x64 and ARM64 Windows MSVC lane |
| Documentation | `update-docs-from-commits` | Scan recent commits and draft doc updates for user-facing changes |
| Maintenance | `sync-agent-infra` | Detect and fix drift across agent-first infrastructure files |
| Maintenance | `release-captain` | Run the weekly Monday pre-flight and Tuesday stable release checklist |
| Reference | `sbom` | Generate SBOMs and resolve dependency licenses |

### Issue Workflow
Expand Down
Loading