ci: check documentation links on PRs and nightly with a shared cache

[rwgk]

Summary and narration

This PR completes the workflow changes for replacing pre-commit.ci with GitHub Actions, following the direction in #2652. #2994 already introduced the standalone Linux and Windows pre-commit checks and monthly Dependabot hook updates. This PR gives documentation link checking its own Linux workflow and connects it to the existing nightly workflow.

The main change is when link checking can run. Previously, checking rendered documentation was a step in the wheel-based documentation build reached through ci.yml. The new workflow runs directly on PR updates, so it can check the proposed documentation without waiting for copy-pr-bot, wheel builds, or GPU CI. It checks both the documentation we write and the HTML that Sphinx generates. Building that HTML matters: generated API pages, cross-package references, and anchors need to be checked in the assembled documentation, beyond what the local pre-commit hook sees in checked-in Markdown and reStructuredText.

Nightly runs use the same checker and check links afresh. They publish successful link results that subsequent PR runs can reuse for up to one day, reducing repeated requests to external documentation sites. Each PR still builds its own HTML and checks local files and anchors against its own revision.

The checker also gives recognized temporary HTTP failures time to recover. When those are the only remaining failures, it waits and tries again using the successful results already cached, with a limit of ten passes including the first. A site returning HTTP 429 can recover without requiring us to rerun an entire workflow or repeat requests that have already succeeded. A broken link or unclassified error stops additional passes immediately, and a temporary failure that persists through the limit still fails the check.

The existing wheel-based documentation build and preview deployment stay in place. They exercise the documentation built from CI's wheels; the independent workflow exercises documentation built from the checked-out sources. Lychee moves out of build-docs.yml, and the old Windows pre-commit job moves out of ci.yml, since #2994 already supplies its replacement. This avoids repeating those checks when the PR is copied to pull-request/<number> for full CI.

Merging this PR completes the workflow implementation. The administrative cutover follows separately: require the three replacement checks, remove the old pre-commit.ci requirement, then remove pre-commit.ci from the GitHub Apps (under Settings → Installations).

The expandable sections below cover the implementation for reviewers, the validation evidence, and the exact Rulesets changes for that cutover.

Details for all reviewers

How the workflows fit together

Entry point	What runs
PR opened, reopened, or updated	The standalone Linux/Windows pre-commit workflow from #2994 and the new Linux documentation-link workflow. Draft and fork PRs are included, subject to GitHub's normal fork-workflow approval settings.
Push to copy-pr-bot's pull-request/<number> ref	Existing full CI, including wheel-based documentation rendering and preview deployment. The replacement workflows have no push trigger for these refs.
Existing nightly schedule	The reusable documentation-link workflow, with a fresh link sweep and cache publication, alongside the existing nightly suite.
Manual workflow dispatch	Run the checker in cache-reader or cache-refresh mode, or exercise the nightly integration in documentation-only mode.

The direct PR checks run independently of /ok to test and a [no-ci] PR title. The workflow triggers have no path filters, so their required-check names can be reported for every PR update. Superseded runs are cancelled within the same PR and mode; manual cache modes and nightly test modes have separate concurrency groups.

What the new checker validates

.github/workflows/lychee.yml runs two independent jobs, Lychee (authored) and Lychee (rendered), followed by the aggregate Documentation links check. The authored job checks tracked .md and .rst files outside qa/, skipping symlinks. The rendered job checks HTML throughout the assembled cuda-python, cuda-bindings, cuda-core, and cuda-pathfinder documentation, excluding _static assets and enabling full fragment validation.

The new source-build helper is exposed as docs-build-all-latest in cuda_core's Pixi manifest. It uses the existing locked docs environment and its local source dependencies. It verifies that the three sibling libraries import from this checkout and agree with their distribution metadata, installs the metapackage with --no-deps to retain those local dependencies, and uses the existing documentation scripts to assemble all four latest trees under artifacts/docs. Generated GitHub source links point to the checked-out revision. The checkout retains Git history and tags for SCM-derived versions.

Input selection produces deterministic absolute-path lists, preserves spaces, and rejects empty lists or filenames containing line breaks. Its six tests cover those cases and the authored/rendered selection rules.

The workflow uses Python 3.14, hosted Linux runners, read-only repository permissions, and checkout without persisted credentials. Lychee stays pinned to v0.24.2, matching the local hook. Existing URL exclusions in lychee.toml are unchanged.

Cache behavior and failure handling

The cache contains link-check results. PR runs restore the newest accessible matching snapshot and do not publish one. Nightly runs skip restoration, check afresh, and publish a new immutable snapshot keyed by run ID and attempt. Authored and rendered checks have separate namespaces incorporating the Lychee version and a hash of the checking policy. Documentation edits therefore retain reusable external-link results, while a policy change selects a new namespace.

Successful external checks expire after one day. Lychee v0.24.2 omits failed responses from its persisted cache and does not cache filesystem checks. A nightly sweep can publish its successful results even when some URLs fail; the failing URLs are checked again on the next run. Local files and anchors are checked against the fresh build each time.

After merge, nightly's main-scoped snapshots can supply a common baseline to PRs, including fork PRs. Manual branch refreshes stay within their branch's cache scope. This follows GitHub Actions' cache access rules; it does not create a globally writable cache shared by PRs.

The existing rendered-check throttling and retries are retained: 16 concurrent requests overall, two per host, a 250 ms host request interval, three retries, and a 30-second request timeout. Both jobs reject empty input and require an explicit successful checker result. Documentation links runs even if a dependency fails and requires both link-check jobs to succeed. Reports are available in job summaries and as seven-day artifacts.

Bounded retries within each job

The new retry helper reads Lychee's JSON report and retries only when every remaining failure is a recognized transient HTTP(S) failure: status 408, 429, or 5xx, or an HTTP(S) timeout. A 404, missing local file, invalid anchor, unsupported or unclassified failure, or a mixture of temporary and permanent failures stops additional passes immediately.

The initial action pass counts toward a maximum of ten complete link-check passes. For an eligible failure, the helper waits 60 seconds before the second pass and 120 seconds before each subsequent pass. Every pass uses the same pinned binary, input list, configuration, request limits, token, and .lycheecache. Successful external checks remain cached; unsuccessful checks and local files are checked again. This adds a cooldown between passes while retaining Lychee's existing per-request retries.

The action's first pass uses fail: false so the helper can handle its result. Its Markdown-only empty-report guard is also disabled for JSON output; the helper requires a positive checked-link count and validates the counts, failure-map shape, and agreement with the CLI exit code. Exit code zero with no reported failures is required for success. Setup, configuration, argument, missing-report, and malformed-report failures stop immediately. Neither exhausting the retry limit nor disabling the action's initial guard can turn an unsuccessful checker result into a passing job.

Every attempt's JSON report is retained as an artifact. The helper produces a readable Markdown summary with attempt exit codes and the final counts and failure details, and publishes it in the job summary and artifacts. A recovered error remains visible in earlier reports without appearing as the current result. The retry tests cover transient classification, mixed failures, recovery, exhausted retries, cache and argument reuse, non-link failures, JSON validation, and the attempt limit. The helper is included in the cache-policy hash so changes to this behavior select a new baseline namespace.

Integration and contributor impact

ci.yml removes the duplicate Windows pre-commit job and its gate dependency. build-docs.yml removes the embedded Lychee and cache steps. ci-nightly.yml calls the reusable checker independently of wheel discovery and requires its success in the nightly status gate. Its documentation-links-only mode also runs the standalone CI-tool tests and verifies that wheel/GPU jobs remain skipped.

Check job status continues to summarize heavyweight CI's own jobs. The independent pre-commit and documentation checks become merge requirements through Rulesets, as described below. This follows the existing pattern of requiring the independently triggered PR metadata check.

For contributors, pre-commit install and pre-commit run --all-files retain their existing roles. Local Lychee still checks authored documentation. CONTRIBUTING.md describes the Actions checks and keeps the hook-installation reminder focused on developer machines. Per-host local caching remains a follow-on pending a stable Lychee release with --cache-location; coordinated version updates remain necessary across the hook revision, hook argument, and CI pin.

Validation and full CI coverage

Validation at the reviewed revision

The reviewed PR head is d15a151. All linked runs below passed at this revision, including JSON-based transient-failure classification. Full CI finished with all 107 jobs successful, including the final Check job status gate.

Evidence	Result	What it covers
Direct PR documentation checks	Passed	Authored inputs, the complete source documentation build, rendered links and fragments, JSON report validation, retry-helper execution, final summaries, and the aggregate Documentation links gate.
Standalone pre-commit	Passed	Linux and Windows hook execution, including formatting and linting of the retry helper and tests.
Lockfile freshness	Passed	The unchanged locked docs environment remains consistent with its manifests.
Approved full CI	Passed on attempt 2	The complete build/test plan on pull-request/2993, wheel-based documentation rendering and preview deployment without embedded Lychee, package builds/tests, and the updated Check job status dependency set.
Manual fresh link check	Passed	Both input kinds checked without restoring a baseline, followed by publication of branch-scoped successful-result snapshots.
Nightly documentation-only mode	Passed on attempt 3	The reusable nightly call, fresh cache publication, CI-tool tests, the nightly status gate, and expected skipped wheel/GPU jobs.
Cache-reader mode	Passed	Restoration of both matching snapshots from the fresh-cache run and successful checking without publishing another baseline.

Both direct PR link jobs passed on their first checker pass. The successful nightly attempt also passed rendered links on its first checker pass, checking 740,779 occurrences of 13,440 unique links with zero errors or timeouts. Final JSON and Markdown reports were verified, and that attempt published a new cache baseline.

The nightly run's first two workflow attempts exercised the failure limit: each exhausted all ten checker passes with only the same Conda documentation link still returning HTTP 429. The link job and aggregate gates failed, while successful checks continued to be reused from cache. Rerunning the affected job and its gates recovered without a code change. Reaching the attempt limit did not relax the success requirement.

Full CI's initial attempt completed 105 jobs successfully, with one Windows L4 GPU job waiting for a runner. After confirming that no jobs were executing, the run was cancelled and that job and its dependent gate were requeued. The targeted rerun retained all 105 successful results and their execution timestamps, without repeating the whole matrix or changing the commit. The Windows test passed once a runner became available; the final gate then passed, completing all 107 jobs successfully.

The full CI run exercises the existing build, test, and documentation integration. Direct PR checks exercise the link checker that intentionally runs outside full CI, while the focused nightly/cache runs exercise the reusable call and snapshot paths. Local failure-path tests cover the errors that must keep a check red. Check job status reports the heavyweight CI result; the independent Documentation links result remains a separate requirement. The administrative cutover still requires successful actual PR checks after merge.

Local tests and failure-path checks

Local validation used Pixi already on PATH with Python 3.14. All 164 standalone CI-tool tests and 36 subtests passed, including the six input-selection tests and 65 focused retry cases. Full pre-commit run --all-files also passed, including local Lychee and actionlint. The retry cases cover HTTP 408/429/5xx and timeout classification, immediate stopping for permanent or mixed failures, recovery, exhaustion at ten total attempts, argument and cache reuse, non-link failures, malformed or missing JSON, count/map inconsistencies, and disagreement between exit codes and reported failures.

Separate checks used the pinned Lychee binary against a localhost HTTP server. A 429 response recovered on the second pass, with request counts confirming that successful HTTP checks were reused from cache. A 404 response stopped after the first pass. These deterministic tests exercise the real CLI and JSON format without depending on an external site's current availability; hosted runs still exercise the actual documentation and GitHub Actions integration.

Reproducing the focused checks

For the same source documentation build on Linux:

PIXI_LOCKED=true pixi run --manifest-path cuda_core/pixi.toml -e docs docs-build-all-latest

For hosted cache-refresh, cache-reader, and nightly-integration tests, select the PR branch explicitly:

gh workflow run lychee.yml --repo NVIDIA/cuda-python --ref rwgk/maint/lychee-pr-nightly -f refresh-cache=true
gh workflow run lychee.yml --repo NVIDIA/cuda-python --ref rwgk/maint/lychee-pr-nightly -f refresh-cache=false
gh workflow run ci-nightly.yml --repo NVIDIA/cuda-python --ref rwgk/maint/lychee-pr-nightly -f documentation-links-only=true

After merge, run the nightly documentation-only mode on main and confirm that a subsequent PR restores both matching baseline caches. This final check verifies default-branch warming and reuse across PRs, which branch-scoped tests cannot establish before merge. Manual runs validate workflow execution; the Rulesets cutover should use successful checks from actual PR events.

Details for admin updating the Rulesets

What changes, and why

The workflows report their results; Rulesets decide which results are required for merging. The current Prevent committing without PR ruleset requires pre-commit.ci - pr, Check job status, and PR has assignee, labels, and milestone. It applies to main, 12.9.x, 11.8.x, 13.4.x, and release/**/*.

Add the three replacement requirements in a new main-only Ruleset, then remove the old pre-commit.ci requirement from the shared Ruleset. This limits the new requirements to the branch containing the replacement workflows. GitHub combines applicable Rulesets, so main will have five required check contexts across the two Rulesets. This is a one-time manual cutover; the helper preserved in closed #3002 is not needed.

Cutover steps

1. Verify the replacement before changing requirements. After ci: check documentation links on PRs and nightly with a shared cache #2993 is merged, confirm that pre-commit.yml and lychee.yml are on main. Obtain successful Pre-commit (Linux), Pre-commit (Windows), and Documentation links checks on the current commit of a representative PR targeting main. Use its actual pull_request runs, including a green rerun of any failed check. GitHub evaluates required checks on the latest commit, and manually dispatched workflow jobs do not satisfy PR requirements. Keep pre-commit.ci installed and required during this verification. Required-check guidance

2. Create the additional main-only Ruleset. In repository Settings, open Rulesets, then New ruleset > New branch ruleset. Name it, for example, Main pre-commit and documentation checks, set enforcement to Active, and target only the exact branch main. Enable Require status checks to pass before merging and add Pre-commit (Linux), Pre-commit (Windows), and Documentation links. Select GitHub Actions as the expected source for each, rather than any source; its App integration ID is 15368. Match the existing status-check policy: leave Require branches to be up to date before merging unchecked and keep branch creation exempt. Leave the bypass list empty and enable only the status-check rule. Save the Ruleset. Creating a Ruleset and choosing the expected check source

3. Remove the retired requirement from the existing Ruleset. Edit Prevent committing without PR and remove only pre-commit.ci - pr from its required checks, then save. Its old App integration ID is 68672. Preserve the other two checks, branch targets, review requirements, and all other protections. Creating the new requirement first keeps main protected throughout the transition.

4. Confirm the effective merge requirements. A PR targeting main should now show the five checks listed below as required, all from GitHub Actions, with no pre-commit.ci - pr requirement. On a temporary test PR, introduce a formatting or documentation-link error, verify that the corresponding required check fails and blocks merging, then repair it and verify success. Confirm the required-check result in the merge box; a PR being blocked merely because it is draft is not proof of enforcement. The release branches covered by the shared Ruleset retain their existing two Actions requirements; they do not gain the three new checks until the workflows are deliberately backported.

5. Retire pre-commit.ci for cuda-python. After the required-check verification, open repository Settings > Integrations > GitHub Apps, choose Configure for pre-commit.ci, and remove only cuda-python from its selected repository access, then save. This opens the NVIDIA installation's settings, which also control access to other repositories. Preserve their access. If the installation currently covers All repositories, coordinate with an organization owner to retain the other repositories when changing to selected access. Use repository access rather than suspending or uninstalling the shared installation. App repository-access instructions

Expected result on main

Required check	Ruleset	Expected source
Check job status	Existing shared Ruleset	GitHub Actions
PR has assignee, labels, and milestone	Existing shared Ruleset	GitHub Actions
Pre-commit (Linux)	New main-only Ruleset	GitHub Actions
Pre-commit (Windows)	New main-only Ruleset	GitHub Actions
Documentation links	New main-only Ruleset	GitHub Actions

For an optional read-only confirmation of the effective requirements:

gh api repos/NVIDIA/cuda-python/rules/branches/main --jq '.[] | select(.type == "required_status_checks") | .parameters.required_status_checks[] | {context, integration_id}'

The final output should contain the five contexts above, each with integration ID 15368. Rule editing requires repository admin access or permission to edit repository rules; changing the organization's App installation may require additional installation-management access.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[rwgk]

EDIT: The links below are outdated, but the guidance in general is still useful.

I asked codex to guide me through reviewing this PR myself. Sharing what it gave me (in part because I want to click the links here myself):

Review Files changed against #2994; that isolates this layer. I’d use six passes, each with one question to answer before moving on.

1. When does it run, and what does it check?
   Start with lychee.yml’s triggers and checkout. Trace a PR update, nightly call, and manual run. There is no push trigger, so copy-pr-bot’s branch pushes do not add another run. Checkout selects the PR’s head SHA.

2. Are we selecting the intended documentation?
   Read the input selector, then its tests. Authored inputs are tracked Markdown/reST outside qa/; rendered inputs are HTML outside _static. Pay particular attention to skipping README symlinks: checking the same content from another directory can misinterpret relative links. Empty inputs must fail.

3. Does rendered checking use this PR’s sources?
   Follow the build helper. The important safeguards are verifying that all three libraries import from the checkout and match their installed metadata. Installing metapackage metadata with --no-deps preserves those local packages.

4. Does caching preserve the checking policy?
   Read cache preparation through publication. Trace both values of refresh-cache: PRs restore; nightly checks afresh and publishes. Version, policy, and authored/rendered kind separate the namespaces; successful URL checks expire after one day. Shared visibility relies on publishing from main, under GitHub’s cache scope rules.

5. Can failures accidentally produce a green check?
   Examine the checker and aggregate gate, then the nightly gate. Think through a failed docs build, one failed matrix leg, cancellation, and documentation-only mode. That mode requires successful link/tool checks and skipped wheel/GPU jobs.

6. Do the deletions have complete replacements?
   Finish with build-docs.yml, ci.yml, and the documentation changes. The dedicated workflow replaces inline lychee; [no-ci] ci: run pre-commit on Linux and Windows with Dependabot hook updates #2994 replaces Windows pre-commit. Live required-check migration belongs to ci: add required-check migration helper #3002.

[rwgk]

/ok to test 27b3cf9

[github-actions]

Doc Preview CI
🚀 View preview at https://nvidia.github.io/cuda-python/pr-preview/pr-2993/
https://nvidia.github.io/cuda-python/pr-preview/pr-2993/cuda-core/
https://nvidia.github.io/cuda-python/pr-preview/pr-2993/cuda-bindings/
https://nvidia.github.io/cuda-python/pr-preview/pr-2993/cuda-pathfinder/
Preview will be ready when the GitHub Pages deployment is complete.

[rwgk]

/ok to test ed21592

[rwgk]

/ok to test d15a151