Repository navigation
Release workflow test gate is dead — continue-on-error forces conclusion=success; flip to outcome + make CI suite honestly green #255
Description
Is your feature request related to a problem? Please describe.
PROBLEM
The release workflow's test gate is dead code and has never fired. .github/workflows/release.yml runs tests with continue-on-error: true (line 44) and gates tag deletion on steps.test.conclusion == 'failure' (line 48) — but under continue-on-error a failed step's conclusion is forced to success; only outcome records the failure. Verified on real runs: v1.4.0 and v1.4.1 both had failing tests in Run Tests, conclusion=success, Delete-tag skipped, release published.
SCOPE BOUNDARY
The release workflow's gate, plus the suite hygiene needed to make flipping it safe. Not PR-time CI, which is its own concern. Do not just flip the field — see the caveat.
GOALS
-
steps.test.conclusion→steps.test.outcome, so the gate is real. (Landed onrelease/2.0via Fix dead release test gate: use outcome, not conclusion #306.) - Partition the CI suite so "green" is meaningful: env-dependent tests skip cleanly via pytest markers or fixture-presence gating, rather than failing.
- Companion failures resolved — TS golden-fixture collection errors, Java source-code-path crash.
- Same fix applied to
main, not onlyrelease/2.0.
CAVEATS AND KNOWN RISKS
- Flipping the field alone blocks every release. The gate becomes real against a suite that is not currently CI-green, so the first tag after the flip fails and stays failing. That is why the flip and the suite work are one issue and not two.
continue-on-errormust stay. Removing it fails the job immediately and the tag-deletion step never runs — the opposite of the intent. The two settings are a pair, and Fix dead release test gate: use outcome, not conclusion #306 comments them as such.- A gate that fires deletes the pushed tag. Worth exercising on a scratch tag before trusting it on a real release.
DEFINITION OF DONE
- Gate fires on a deliberately broken test, verified on a scratch tag or a
workflow_dispatchdry run — demonstrated, not reasoned about. - Release workflow green on an honest, fully-run suite; tests skipped by gating are visibly reported rather than silently absent.
- Both release lines carry the fix.
Describe the solution you'd like
Not stated in the original issue.
Describe alternatives you've considered
Not stated in the original issue.
Activity
rahlk commentedon Jul 14, 2026
Scope consolidation (2026-07-14): this issue now owns the full path to a meaningful CI-green + live gate:
Fix dead release test gate: use outcome, not conclusion (#306)
[-]ci: release workflow test gate is dead — continue-on-error forces conclusion=success; flip to outcome + make CI suite honestly green[/-][+]Release workflow test gate is dead — `continue-on-error` forces `conclusion=success`; flip to outcome + make CI suite honestly green[/+]