Skip to content

ci(release): decouple prerelease publication from qualification - #4420

Merged
SDAChess merged 2 commits into
mainfrom
ci/release-orchestration/SDAChess
Oct 15, 2026
Merged

SDAChess merged 2 commits into
mainfrom
ci/release-orchestration/SDAChess

Conversation

@SDAChess

@SDAChess SDAChess commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Summary

Publish pre-release artifacts independently of qualification so a flaky qualification run cannot prevent candidate availability and a publication failure cannot skip qualification. The same Pre-release run calls qualification, fails when a required suite fails, and sends the existing failure notification while published artifacts remain available.

Stacked on #4419; merge the qualification extraction first, then retarget this PR to main.

Related Issue

Direct maintainer request; no tracking issue requested.

Changes

  • Split orchestration into Release (build → qualify → publish) and Pre-release (build → publish and qualify in parallel), sharing callable build, publication, and qualification workflows. Pre-release supports tag pushes and manual dispatch; Release Auto-Tag dispatches Pre-release.
  • Preserve the existing build steps, package contents, qualification suites, security settings, and stable publication destinations. Codex review serialization is provided by refactor(ci): extract release qualification and serialize Codex reviews #4419.
  • Use native job dependencies for failure propagation. Upload passing and failing qualification summaries to Actions; OCI copies are best effort, with bounded setup/login/upload steps and a visible warning on failure.
  • Upload installer-visible packages after images and the Helm chart. The pre installer alias accepts published candidates even while qualification runs or after it fails, and retains support for artifacts from the previous workflow.
  • Update workflow documentation, contributor guidance, and existing installer/docs workflow tests. Stable artifact promotion and test retries remain outside this change.

Testing

  • Actionlint, including ShellCheck, on all seven affected workflows; offline Zizmor reports no high-severity findings.
  • Installer regression tests, qualification-summary tests, and all 29 docs website tests pass.
  • Local extraction audit verifies unchanged build/publication steps and qualification suites, ordering, tag-mode rejection, failure-notification payloads with a stubbed sender, and summary failure enforcement.
  • Docs navigation, Markdown lint, Ruff, skill/crate inventory checks, and git diff --check pass.
  • Standalone ShellCheck compared against the parent: no new diagnostics; existing installer/test-harness diagnostics remain.
  • Release dependency checks confirm parallel pre-release publication/qualification, the stable qualification gate, and failure-notification dependencies.
  • No release workflow was dispatched; live publishing and full qualification will be checked with a real prerelease after merge, without a throwaway tag.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture/workflow documentation updated

@github-actions

Copy link
Copy Markdown

@SDAChess
SDAChess force-pushed the refactor/release-qualification/SDAChess branch from f05c4ed to 4b9eeb9 Compare October 15, 2026 11:32
@SDAChess
SDAChess force-pushed the ci/release-orchestration/SDAChess branch from ab9dea1 to 72da79a Compare October 15, 2026 11:34
@SDAChess
SDAChess force-pushed the refactor/release-qualification/SDAChess branch from 4b9eeb9 to e8d4b29 Compare October 15, 2026 13:36
Base automatically changed from refactor/release-qualification/SDAChess to main October 15, 2026 13:59
@SDAChess
SDAChess force-pushed the ci/release-orchestration/SDAChess branch from 72da79a to b5c6491 Compare October 15, 2026 16:28
@github-actions

Copy link
Copy Markdown

Compatibility review (advisory)

New or updated breaking changes; see the full report for all findings.

1. cli — No explicit risk acceptance found

OPENSHELL_VERSION=pre previously skipped prerelease artifacts unless their Release Tag run completed successfully, but now accepts artifacts from running or failed Pre-release qualification runs.

  • Impact: Automation using the floating pre alias can install a candidate the prior contract rejected, including one whose required qualification suite failed.
  • Action: Keep the success filter for pre, or expose unqualified candidates behind a new opt-in alias and reserve the behavior change for a minor release with migration guidance.
  • PR: ci(release): decouple prerelease publication from qualification #4420
Review details
  • Execution: complete
  • Candidate: PR #4420 (1c1920a7747a2cce322b99eaa62656a1c0c93064)
  • Baseline: 2804f68501fc74da71ff3783b7074682a7a3e276 (2804f68501fc74da71ff3783b7074682a7a3e276)
  • Reviewed: rust-sdk, python-sdk, go-sdk, typescript-sdk, api-behavior, cli, configuration, policy, helm, state

Advisory review; publication gates are unchanged.

Branch check run

@SDAChess

Copy link
Copy Markdown
Member Author

Compatibility review (advisory)

New or updated breaking changes; see the full report for all findings.

1. cli — No explicit risk acceptance found

OPENSHELL_VERSION=pre previously skipped prerelease artifacts unless their Release Tag run completed successfully, but now accepts artifacts from running or failed Pre-release qualification runs.

* **Impact:** Automation using the floating `pre` alias can install a candidate the prior contract rejected, including one whose required qualification suite failed.

* **Action:** Keep the success filter for `pre`, or expose unqualified candidates behind a new opt-in alias and reserve the behavior change for a minor release with migration guidance.

* **PR:** [ci(release): publish prereleases before qualification #4420](https://github.com/NVIDIA/OpenShell/pull/4420)

Review details

Advisory review; publication gates are unchanged.

Branch check run

This is okay, we actually defined that in RFC0014

@SDAChess SDAChess added the test:e2e Requires end-to-end coverage label Oct 15, 2026
@github-actions

Copy link
Copy Markdown

Label test:e2e applied for b5c6491. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway, sandbox, and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

@matthewgrossman matthewgrossman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice split overall. The shared build and publish workflows make the two release orders easy to follow, and the early tag-type check in release-build.yml is a good guard. My comments are inline, plus two things outside the diff:

  • Stale references to the old workflow. .agents/skills/watch-github-actions/SKILL.md:128 still says "Release Tag temporarily omits Codex Security". scripts/baseline_workflow_metrics.py:55 tracks release-tag.yml but not release-prerelease.yml. Since nightlies now run under Pre-release, almost all release runs will disappear from the metrics. Per sync-agent-infra, these should be updated in this branch.
  • Live test. The description says no release workflow was dispatched. A test Pre-release dispatch on a throwaway tag before merge would exercise the new permissions between the calling and called workflows and the actions: read artifact downloads in release-publish.yml.

Comment thread .github/workflows/release-tag.yml
Comment thread install.sh
Comment thread .github/workflows/release-prerelease.yml Outdated
Comment thread .github/workflows/release-qualification.yml
@SDAChess SDAChess changed the title ci(release): publish prereleases before qualification ci(release): decouple prerelease publication from qualification Oct 15, 2026
@SDAChess

Copy link
Copy Markdown
Member Author

Pushed a6f708f with the agreed tag-trigger support, parallel prerelease publication/qualification, and best-effort OCI summary publication. Replied on each inline thread with the changes and the intentional installer behavior.

Validation: Actionlint, qualification-summary tests, release dependency checks, docs navigation, and git diff --check pass. Live publication and qualification will be checked using a real prerelease after merge; no throwaway tag or release workflow was dispatched for this review.

The two infrastructure follow-ups from the review overview remain open: the older Release Tag wording at .agents/skills/watch-github-actions/SKILL.md:128 and adding release-prerelease.yml to scripts/baseline_workflow_metrics.py. Separating qualification from snap/wheel build completion also remains outside this patch.

@github-actions

Copy link
Copy Markdown

Compatibility review (advisory)

New or updated breaking changes; see the full report for all findings.

1. cli — Risk explicitly accepted

OPENSHELL_VERSION=pre previously required a completed successful Release Tag run but now accepts packages from running or failed Pre-release qualification runs.

  • Impact: Automation using the floating pre alias can install a candidate the prior contract rejected, including one whose required qualification suite failed.

  • Action: Although the PR author accepted the risk after the prior report, v0.1.5 must retain successful-run filtering or expose unqualified candidates through a new opt-in alias deferred to a minor release.

  • PR: ci(release): decouple prerelease publication from qualification #4420

  • Acceptance: comment

Review details
  • Execution: complete
  • Candidate: PR #4420 (ae9e8a5cecfa97d90249867457390f36c1986746)
  • Baseline: 2804f68501fc74da71ff3783b7074682a7a3e276 (2804f68501fc74da71ff3783b7074682a7a3e276)
  • Reviewed: rust-sdk, python-sdk, go-sdk, typescript-sdk, api-behavior, cli, configuration, policy, helm, state

Advisory review; publication gates are unchanged.

Branch check run

@SDAChess
SDAChess enabled auto-merge October 15, 2026 16:55

@matthewgrossman matthewgrossman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm: thanks for the quick turnaround. Follow-ups (fine in a separate PR):

  • .agents/skills/watch-github-actions/SKILL.md:128 still says "Release Tag temporarily omits Codex Security…"; it should name Release and Pre-release.
  • Add release-prerelease.yml to WORKFLOWS in scripts/baseline_workflow_metrics.py, otherwise nightly runs drop out of the metrics.
  • Start qualification without waiting for the snap and wheel builds, as discussed.

@SDAChess SDAChess removed the test:e2e Requires end-to-end coverage label Oct 15, 2026
Signed-off-by: Simon Scatton <sscatton@nvidia.com>
Signed-off-by: Simon Scatton <sscatton@nvidia.com>
@SDAChess
SDAChess added this pull request to the merge queue Oct 15, 2026
Merged via the queue into main with commit 2538844 Oct 15, 2026
81 of 82 checks passed
@SDAChess
SDAChess deleted the ci/release-orchestration/SDAChess branch October 15, 2026 17:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants