ci(release): promote qualified prereleases to stable

.agents/skills/release-openshell/SKILL.md

@@ -7,15 +7,46 @@ metadata:
 
 # Release OpenShell
 
-For the intended release version, pick the latest prerelease whose Release
-Qualification job passed. Confirm that its qualification summary matches the
-tag and commit. Present the candidate and changes since the previous stable
-release to the maintainer; once approved, tag that exact commit as `vX.Y.Z`.
+For the intended release version and branch, inspect the latest passing
+prerelease and changes since the previous stable release. Present the candidate
+to the maintainer. Once approved, dispatch Release from that branch:
+
+```shell
+gh workflow run release.yml --ref main -f tag=vX.Y.Z
+```
+
+Use the support branch instead of `main` for a previous minor version. Release
+selects the highest-numbered `vX.Y.Z-pre.N` reachable from that branch with a
+successful Pre-release run triggered by that tag or dispatched on the same branch,
+and matching passing `release-tag-v1` qualification evidence. Missing or expired evidence blocks
+promotion. Inspect the selected candidate in the promotion job summary.
+The entire Pre-release run must succeed, including publication. If qualification
+passed but publication failed, rerun the failed jobs before promoting the candidate.
+
+Release creates `vX.Y.Z` at that exact commit, then rebuilds and publishes all
+stable artifacts without rerunning qualification. Embedded versions change, so
+stable artifacts are not byte-identical to the qualified prerelease. An existing
+stable tag may be retried only if it already points to the selected commit.
+Stable images are saved as OCI archives during the build and pushed under the
+stable version tags during publication. Their SBOMs are verified before updating
+`latest`; the qualified prerelease's commit-SHA image tags remain unchanged.
 
 If no prerelease exists, agree on the version and source commit, then create and
-push `vX.Y.Z-pre.1`. Wait for qualification to pass before promoting it.
+push `vX.Y.Z-pre.1`. Prerelease tag pushes start Pre-release. If the tag was
+created using `GITHUB_TOKEN`, dispatch Pre-release explicitly on the intended branch:
+
+```shell
+gh workflow run release-prerelease.yml --ref main -f tag=vX.Y.Z-pre.1
+```
+
+Wait for the entire Pre-release run to pass before promoting it. Stable tag pushes do not
+start Release. Release Auto-Tag creates and dispatches prereleases on `main`.
+
+Stable releases serialize across branches. A release older than the highest
+stable Git tag publishes versioned artifacts without moving GHCR `latest`, the
+GitHub latest release, npm `latest`, or snap `latest/stable`. Its npm package
+uses `release-X.Y`; snap files remain available in its GitHub Release rather than
+updating a Snap Store support track. Docs retain an
+immutable version and advance latest only for a newer version.
 
-Prerelease tag pushes start Pre-release; stable tag pushes start Release.
-Release Auto-Tag uses an explicit dispatch because its `GITHUB_TOKEN` pushes do
-not trigger workflows.
 Watch the stable release run and report its result.

.agents/skills/sbom/SKILL.md

@@ -17,6 +17,10 @@ SBOMs are **release artifacts only** -- they are generated on demand and not com
 
 Pushed gateway, sandbox, and supervisor images carry an SPDX SBOM and minimal SLSA provenance as OCI attestations. Branch E2E, Release Dev, Release, and Pre-release image binaries embed cargo-auditable metadata, so their image SBOMs include linked Rust crates.
 
+Stable Release builds retain images and attestations in OCI archive artifacts.
+Publication copies every platform and attestation to GHCR with preserved digests,
+then runs the SBOM verification before updating `latest`.
+
 ## Prerequisites
 
 - `mise install` has been run (installs Syft and other tools)

.agents/skills/watch-github-actions/SKILL.md

@@ -125,8 +125,9 @@ gh run list --json databaseId,status,headBranch,url --jq '.[] | {id: .databaseId
 
 ## View Job Logs
 
-`Release Tag` temporarily omits Codex Security from its `Security Scan` job.
-Other scanners and qualification gates still run. Use the standalone
+`Pre-release` temporarily omits Codex Security from its `Security Scan` job.
+`Release` uses the passing prerelease qualification result. Other scanners and
+qualification gates still run. Use the standalone
 `Codex Security Release Qualification` workflow for manual investigation.
 
 For `Codex Security`, an exit code of 2 with partial coverage is an incomplete
@@ -167,10 +168,13 @@ During `0.x`, a minor train permits compatibility findings
 as warnings; a patch train or no active train rejects them. Compare the current
 train's version with the latest stable release; commit messages are irrelevant.
 Compilation, baseline, and tool errors remain fatal. The `protobuf_compatibility` suite participates in
-the `release-tag-v1` qualification profile. Release and Pre-release call the reusable
-`release-qualification.yml` workflow to run the checks and collect their results.
-Release builds, qualifies, then publishes. Pre-release builds, then publishes
-and qualifies in parallel in the same run. Failed, cancelled, or skipped required suites fail
+the `release-tag-v1` qualification profile. Pre-release calls the reusable
+`release-qualification.yml` workflow after building, in parallel with publication. Release
+selects the latest passing prerelease for the requested version and branch,
+verifies its qualification evidence, creates the stable tag at its exact commit,
+then rebuilds and publishes without rerunning qualification. The promotion job
+summary and `stable-promotion-*` artifact record the candidate and evidence.
+Failed, cancelled, or skipped required suites fail
 qualification; stable publication is blocked, while published pre-release
 artifacts remain available. The Pre-release failure notification includes the
 run link. Both callers share `release-build.yml` and `release-publish.yml`.

.github/actions/build-docker-image/action.yml

@@ -2,7 +2,7 @@
 # SPDX-License-Identifier: Apache-2.0
 
 name: Build Docker Image
-description: Build, push, and export a multi-platform OpenShell image
+description: Build and export a multi-platform OpenShell image, optionally pushing it
 
 inputs:
   component:
@@ -17,6 +17,9 @@ inputs:
   image-tag:
     description: Docker image tag
     required: true
+  push:
+    description: Push the image to GHCR during the build
+    default: "true"
   github-token:
     description: Token used to push the image
     required: true
@@ -29,6 +32,7 @@ runs:
         buildkitd-config: /etc/buildkit/buildkitd.toml
 
     - name: Log in to GHCR
+      if: inputs.push == 'true'
       uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
       with:
         registry: ghcr.io
@@ -60,8 +64,13 @@ runs:
       env:
         COMPONENT: ${{ inputs.component }}
         IMAGE_TAG: ${{ inputs.image-tag }}
+        PUSH_IMAGE: ${{ inputs.push }}
       run: |
         install -d artifacts/images
+        push_args=()
+        if [[ "${PUSH_IMAGE}" == "true" ]]; then
+          push_args+=(--output "type=image,push=true,oci-mediatypes=true,oci-artifact=true")
+        fi
         docker buildx build \
           --builder openshell \
           --platform linux/amd64,linux/arm64 \
@@ -72,11 +81,12 @@ runs:
           --cache-to type=gha,mode=max,scope=${COMPONENT} \
           --provenance=mode=min \
           --attest type=sbom \
-          --output type=image,push=true,oci-mediatypes=true,oci-artifact=true \
-          --output type=oci,dest=artifacts/images/${COMPONENT}.tar \
+          "${push_args[@]}" \
+          --output "type=oci,dest=artifacts/images/${COMPONENT}.tar,oci-mediatypes=true,oci-artifact=true" \
           .
 
     - name: Verify SBOM attestation
+      if: inputs.push == 'true'
       shell: bash
       env:
         IMAGE_REF: ghcr.io/nvidia/openshell/${{ inputs.component }}:${{ inputs.image-tag }}

.github/actions/check-latest-release/action.yml

@@ -0,0 +1,46 @@
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+name: Check Latest Release
+description: Check current remote stable tags before updating a shared latest channel
+
+inputs:
+  release-tag:
+    description: Stable version being published
+    required: true
+
+outputs:
+  make-latest:
+    description: Whether this is still the highest stable version in the repository
+    value: ${{ steps.latest.outputs.make-latest }}
+
+runs:
+  using: composite
+  steps:
+    - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
+      id: latest
+      env:
+        RELEASE_TAG: ${{ inputs.release-tag }}
+      with:
+        script: |
+          const tag = process.env.RELEASE_TAG;
+          const parse = value => /^v([0-9]+)\.([0-9]+)\.([0-9]+)$/.exec(value)?.slice(1).map(BigInt);
+          const version = parse(tag);
+          if (!version) {
+            throw new Error(`Invalid stable release tag: ${tag}`);
+          }
+          const tags = await github.paginate(github.rest.repos.listTags, {
+            ...context.repo,
+            per_page: 100,
+          });
+          if (!tags.some(candidate => candidate.name === tag)) {
+            throw new Error(`Stable release tag ${tag} is missing from the repository`);
+          }
+          const newer = tags.find(candidate => {
+            const other = parse(candidate.name);
+            if (!other) return false;
+            const index = other.findIndex((part, i) => part !== version[i]);
+            return index !== -1 && other[index] > version[index];
+          });
+          core.info(newer ? `${tag} is superseded by ${newer.name}; preserving latest` : `${tag} is the latest stable version`);
+          core.setOutput('make-latest', !newer);

.github/workflows/branch-docs.yml

@@ -12,6 +12,7 @@ on:
       - "tasks/scripts/check_docs_nav_test.py"
       - ".github/workflows/branch-docs.yml"
       - ".github/workflows/release-tag.yml"
+      - ".github/workflows/release.yml"
       - ".github/workflows/release-prerelease.yml"
       - ".github/workflows/release-publish.yml"
 

.github/workflows/build-images.yml

@@ -14,6 +14,11 @@ on:
         required: false
         type: string
         default: ""
+      push-images:
+        description: Push images to GHCR during the build
+        required: false
+        type: boolean
+        default: true
 
 permissions:
   contents: read
@@ -39,6 +44,7 @@ jobs:
           binary: ${{ env.BINARY }}
           target-suffix: ${{ env.TARGET_SUFFIX }}
           image-tag: ${{ inputs.image-tag || github.sha }}
+          push: ${{ inputs.push-images }}
           github-token: ${{ secrets.GITHUB_TOKEN }}
 
   sandbox:

.github/workflows/release-build.yml

@@ -10,6 +10,11 @@ on:
         description: Git tag to build and publish
         required: true
         type: string
+      expected-source-sha:
+        description: Qualified source commit that the release tag must resolve to
+        required: false
+        default: ""
+        type: string
       is-prerelease:
         description: Whether the caller is publishing a pre-release
         required: true
@@ -47,11 +52,9 @@ jobs:
     name: Compute Versions
     runs-on: linux-amd64-cpu8
     timeout-minutes: 5
-    container:
-      image: ghcr.io/nvidia/openshell/ci:9cb72baa2e61a1b5f12407e6e82da7fdba0aa722@sha256:67a9a0c32cb99825e6d3e9d9eec45d67149ea1ff7c11a1b1e1b3480d2d3df684
-      credentials:
-        username: ${{ github.actor }}
-        password: ${{ secrets.GITHUB_TOKEN }}
+    defaults:
+      run:
+        shell: nix develop -c bash -euo pipefail {0}
     outputs:
       python_version: ${{ steps.v.outputs.python }}
       cargo_version: ${{ steps.v.outputs.cargo }}
@@ -71,8 +74,9 @@ jobs:
           persist-credentials: false
           ref: ${{ inputs.release-tag }}
 
-      - name: Mark workspace safe for git
-        run: git config --global --add safe.directory "$GITHUB_WORKSPACE"
+      - uses: ./.github/actions/setup-nix
+        with:
+          prepare-shell: "true"
 
       - name: Fetch tags
         run: git fetch --tags --force
@@ -81,8 +85,13 @@ jobs:
         id: v
         env:
           EXPECTED_PRERELEASE: ${{ inputs.is-prerelease }}
+          EXPECTED_SOURCE_SHA: ${{ inputs.expected-source-sha }}
         run: |
           set -euo pipefail
+          if [[ -n "${EXPECTED_SOURCE_SHA}" && "$(git rev-parse HEAD)" != "${EXPECTED_SOURCE_SHA}" ]]; then
+            echo "Release tag ${RELEASE_TAG} does not match the qualified commit ${EXPECTED_SOURCE_SHA}." >&2
+            exit 1
+          fi
           if [[ "${RELEASE_TAG}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+-pre\.[1-9][0-9]*$ ]]; then
             is_prerelease=true
           elif [[ "${RELEASE_TAG}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
@@ -97,12 +106,12 @@ jobs:
           fi
           {
             echo "is_prerelease=${is_prerelease}"
-            echo "python=$(uv run python tasks/scripts/release.py get-version --python)"
-            echo "cargo=$(uv run python tasks/scripts/release.py get-version --cargo)"
-            echo "npm=$(uv run python tasks/scripts/release.py get-version --npm)"
-            echo "deb=$(uv run python tasks/scripts/release.py get-version --deb)"
-            echo "rpm_version=$(uv run python tasks/scripts/release.py get-version --rpm-version)"
-            echo "rpm_release=$(uv run python tasks/scripts/release.py get-version --rpm-release)"
+            echo "python=$(uv run --no-project python tasks/scripts/release.py get-version --python)"
+            echo "cargo=$(uv run --no-project python tasks/scripts/release.py get-version --cargo)"
+            echo "npm=$(uv run --no-project python tasks/scripts/release.py get-version --npm)"
+            echo "deb=$(uv run --no-project python tasks/scripts/release.py get-version --deb)"
+            echo "rpm_version=$(uv run --no-project python tasks/scripts/release.py get-version --rpm-version)"
+            echo "rpm_release=$(uv run --no-project python tasks/scripts/release.py get-version --rpm-release)"
             echo "semver=${RELEASE_TAG#v}"
             echo "source_sha=$(git rev-parse HEAD)"
           } >> "$GITHUB_OUTPUT"
@@ -147,21 +156,19 @@ jobs:
       packages: write
     uses: ./.github/workflows/build-images.yml
     with:
-      image-tag: ${{ needs.compute-versions.outputs.source_sha }}
+      image-tag: ${{ inputs.is-prerelease && needs.compute-versions.outputs.source_sha || needs.compute-versions.outputs.semver }}
+      push-images: ${{ inputs.is-prerelease }}
       checkout-ref: ${{ needs.compute-versions.outputs.source_sha }}
 
   build-python-wheel:
     name: Build Python Wheel
     needs: [compute-versions]
     runs-on: linux-amd64-cpu8
     timeout-minutes: 20
-    container:
-      image: ghcr.io/nvidia/openshell/ci:9cb72baa2e61a1b5f12407e6e82da7fdba0aa722@sha256:67a9a0c32cb99825e6d3e9d9eec45d67149ea1ff7c11a1b1e1b3480d2d3df684
-      credentials:
-        username: ${{ github.actor }}
-        password: ${{ secrets.GITHUB_TOKEN }}
+    defaults:
+      run:
+        shell: nix develop -c bash -euo pipefail {0}
     env:
-      MISE_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       SETUPTOOLS_SCM_PRETEND_VERSION_FOR_OPENSHELL: ${{ needs.compute-versions.outputs.python_version }}
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -170,16 +177,20 @@ jobs:
           persist-credentials: false
           ref: ${{ needs.compute-versions.outputs.source_sha }}
 
-      - name: Mark workspace safe for git
-        run: git config --global --add safe.directory "$GITHUB_WORKSPACE"
+      - uses: ./.github/actions/setup-nix
+        with:
+          prepare-shell: "true"
 
       - name: Sync Python dependencies
-        run: uv sync
+        run: uv sync --frozen
 
       - name: Build Python wheel
         run: |
           set -euo pipefail
-          mise run python:build
+          uv run --frozen python tasks/scripts/generate_python_proto.py
+          uv build --wheel --out-dir target/wheels
+          uv run --no-project python tasks/scripts/verify-python-wheel.py target/wheels \
+            --expected-version "${SETUPTOOLS_SCM_PRETEND_VERSION_FOR_OPENSHELL}"
           ls -la target/wheels/*.whl
 
       - name: Upload wheel artifact