Release workflow fails for backport tags because check-release-notes runs from the tag checkout

Description

When running the CI: Release workflow from main for a backport release tag, the check-release-notes job can fail before it actually checks the release notes.

Example workflow run:

https://github.com/NVIDIA/cuda-python/actions/runs/26492939119

The run was manually dispatched from main with:

git-tag: v12.9.7
component: cuda-bindings

The job failed with:

Run python ci/tools/check_release_notes.py \
+ python ci/tools/check_release_notes.py --git-tag v12.9.7 --component cuda-bindings
python: can't open file '/home/runner/work/cuda-python/cuda-python/ci/tools/check_release_notes.py': [Errno 2] No such file or directory
Error: Process completed with exit code 2.

Root cause

The check-release-notes job checks out the release tag:

- name: Checkout Source
  uses: actions/checkout@...
  with:
    ref: ${{ inputs.git-tag }}

For v12.9.7, that means the job checks out the 12.9.x backport branch commit:

v12.9.7 -> a4e89b567d7fa367414b2a1fb1e2bcb706124967

That commit does not contain ci/tools/check_release_notes.py. The checker was added later on main in:

CI: Check versioned release notes exist before releasing #1907

As a result, the release workflow definition from main tries to run a helper script that exists on main, but the job workspace has already been replaced with the older backport tag checkout.

Why this is different from #2062

This is related to, but different from:

[BUG] CI: "Self-test release-notes checker" fails due to root conftest.py import error #2062

Issue #2062 was about the Self-test release-notes checker step failing because pytest ci/tools/tests imported the repo-level conftest.py, which required cuda.pathfinder. That issue was worked around by commenting out the self-test step in:

[no-ci] Prepare for cuda.core v1.0.1 release #2073

The current failure happens after that step is skipped/commented out. It fails in the actual Check versioned release notes exist step because the checker script is not present in the checked-out release tag.

Expected behavior

The release workflow should be able to validate release notes for older/backport tags, even when the helper script was introduced after the release branch diverged.

In this scenario, it should either:

run the checker from the workflow branch/ref, while checking release-note files in the release-tag checkout; or
avoid running the checker for tags whose checked-out source tree does not contain the checker; or
otherwise make the check-release-notes job robust for backport releases.

Suggested fix

Use separate checkouts for the workflow tooling and the release source tree. For example:

Checkout the workflow ref/current branch normally so ci/tools/check_release_notes.py is available.
Checkout ${{ inputs.git-tag }} into a subdirectory such as release-src.
Run the checker from the workflow checkout, but pass the release checkout as --repo-root:

python ci/tools/check_release_notes.py \
  --repo-root release-src \
  --git-tag "${{ inputs.git-tag }}" \
  --component "${{ inputs.component }}"

This keeps the validation logic on main, while validating the release-note files that actually belong to the release tag.

Additional context

The current v12.9.7 release attempt needs this path because v12.9.7 is a backport release from 12.9.x, while the release workflow being dispatched from main includes newer release-checking infrastructure.

Issue 2141 Planned Fix

Scope

This plan is based on the current main and 12.9.x branches.

The goal is to fix release.yml so backport releases such as v12.9.7 can be
released cleanly, while mainline releases force an explicit decision about the
corresponding backport release notes.

Current Situation

On main, .github/workflows/release.yml has a check-release-notes job that:

checks out inputs.git-tag;
runs python ci/tools/check_release_notes.py;
fails if the selected component's versioned release notes file is missing or
empty under the checked-out tree.

On 12.9.x, ci/tools/check_release_notes.py does not exist. When the main
workflow is dispatched for git-tag: v12.9.7, the checkout moves the workspace
to the v12.9.7 tag, and the checker script is missing. That is the direct
cause of the FileNotFoundError / can't open file failure.

There is a second, separate policy issue: even if the script were available, the
current checker would look for 12.9.7-notes.rst in the v12.9.7 source tree.
That is not the right source of truth. The 12.9.x release notes are maintained
on main, and the current main branch already contains:

cuda_bindings/docs/source/release/12.9.7-notes.rst
cuda_python/docs/source/release/12.9.7-notes.rst

Docs Publishing Analysis

Release docs are published by .github/workflows/build-docs.yml, called from
release.yml after check-release-notes succeeds. For release builds, it:

checks out inputs.git-tag;
builds docs for the selected component;
removes build/html/latest;
deploys the remaining versioned docs into docs/ on gh-pages with clean: false.

The public URL depends on the selected component:

component: cuda-python publishes top-level docs, e.g. https://nvidia.github.io/cuda-python/12.9.7/.
component: cuda-bindings publishes component docs, e.g. https://nvidia.github.io/cuda-python/cuda-bindings/12.9.7/.

The version selector is driven by nv-versions.json, which is committed in the
docs source and copied into the built site. The published nv-versions.json
already advertises 12.9.7, but the gh-pages branch currently has 12.9.6
and 13.3.0 directories, not 12.9.7. That explains the selector-to-404
behavior: metadata for 12.9.7 is present, but the versioned docs directory has
not been generated and deployed yet.

latest and 13.3.0 docs already contain 12.9.7-notes.html. The missing
piece is a successful release.yml docs job for v12.9.7 and the relevant
component.

Proposed Workflow Design

Add a new workflow_dispatch input to .github/workflows/release.yml:

backport-git-tag:
  description: "Backport release tag for release-note planning, or 'not planned'"
  required: false
  type: string
  default: ""

GitHub Actions cannot make this conditionally required based on the selected
component, so the repository checker should enforce the policy.

For cuda-bindings and cuda-python mainline releases:

backport-git-tag must be provided.
The accepted explicit no-backport value is exactly not planned.
If a tag is provided, it must be a valid bare tag for the same tag namespace,
for example v12.9.7.
If a tag is provided, the checker must verify that the matching release notes
exist on main, not on the release tag.

For cuda-bindings and cuda-python backport releases:

missing release notes should not block publishing;
the checker should emit a highly visible GitHub Actions warning and append a
step summary reminder if notes are missing from the workflow checkout;
invalid tag formats should still fail.

For cuda-core and cuda-pathfinder:

keep the current release-notes check behavior unless a separate policy change
is desired;
do not require backport-git-tag.

Implementation Plan

Update ci/tools/check_release_notes.py.

Add support for:
- optional --backport-git-tag;
- a way to identify the current backport branch, preferably from
  ci/versions.yml or an explicit workflow argument;
- distinguishing a mainline release from a backport release for the
  v<version> tag namespace;
- warning-only behavior for missing notes on backport releases.
Update .github/workflows/release.yml.

Change check-release-notes so it checks out the workflow branch source, not
inputs.git-tag, before running the checker. The release tag checkout is
still needed by the docs and upload workflows, but not by the release-note
policy check.

Pass the new input through to the checker:
```
python ci/tools/check_release_notes.py \
  --git-tag "${{ inputs.git-tag }}" \
  --component "${{ inputs.component }}" \
  --backport-git-tag "${{ inputs.backport-git-tag }}"
```
Add tests in ci/tools/tests/test_check_release_notes.py.

Cover:
- mainline cuda-bindings release fails without a backport decision;
- not planned is accepted;
- planned backport tag requires notes on main;
- backport primary release missing notes returns success with a warning path;
- existing cuda-core / cuda-pathfinder validation remains unchanged;
- invalid tags still return an argument error.
Validate locally.

Run the focused checker tests, then inspect the workflow diff. A useful
minimal validation command is:
```
pixi run python -m pytest ci/tools/tests/test_check_release_notes.py
```
If the pixi environment lacks pytest, use the existing local test environment
that has already been used for this repository.

Expected Release Operations After The Fix

For a mainline release such as v13.3.0:

choose component: cuda-bindings or component: cuda-python;
set git-tag: v13.3.0;
set backport-git-tag: v12.9.7 if the backport is planned, or
backport-git-tag: not planned if it is not.

For the actual backport release such as v12.9.7:

run release.yml from main or from the fix branch containing the updated
workflow;
use git-tag: v12.9.7;
run once for component: cuda-bindings to generate
/cuda-bindings/12.9.7/;
run once for component: cuda-python to generate /12.9.7/.

After those docs jobs complete, the existing version selector entries for
12.9.7 should stop pointing at 404s.

It seems that check-release-notes was added recently (between 12.9.6 and 12.9.7), in #1907:

12.9.6 (success): https://github.com/NVIDIA/cuda-python/actions/runs/22958254416
12.9.7 (failed): https://github.com/NVIDIA/cuda-python/actions/runs/26492939119

Usually, to work around issues hit at the release time, we create a release/<version> branch and apply patches there. Once the release is out, we forward-port the patches to the main branch.

Another potential gotcha (not hit here) is it is better to trigger the release workflow from the main branch, instead of the backport branch, but it's not the issue here.

A similar issue happened in the past: #1120.

Navigation Menu

Repository navigation

Release workflow fails for backport tags because `check-release-notes` runs from the tag checkout #2141

Description

Root cause

Why this is different from #2062

Expected behavior

Suggested fix

Additional context

Issue 2141 Planned Fix

Scope

Current Situation

Docs Publishing Analysis

Proposed Workflow Design

Implementation Plan

Expected Release Operations After The Fix

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Release workflow fails for backport tags because check-release-notes runs from the tag checkout #2141

Description

Description

Root cause

Why this is different from #2062

Expected behavior

Suggested fix

Additional context

Activity

rwgk commented on May 27, 2026

Issue 2141 Planned Fix

Scope

Current Situation

Docs Publishing Analysis

Proposed Workflow Design

Implementation Plan

Expected Release Operations After The Fix

leofang commented on May 29, 2026

leofang commented on May 29, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Release workflow fails for backport tags because `check-release-notes` runs from the tag checkout #2141