Skip to content

ci: run cli tests in their own job so runtime tests stop timing out - #6326

Merged
miguel-heygen merged 4 commits into
mainfrom
ci/split-cli-tests
Oct 25, 2026
Merged

miguel-heygen merged 4 commits into
mainfrom
ci/split-cli-tests

Conversation

@miguel-heygen

Copy link
Copy Markdown
Collaborator

The runtime test job ran cli, core and engine tests one after another and took 7 to 9.5 minutes against a 10 minute limit, so some PRs were cancelled right after their tests passed.

The cli tests now run in a parallel Test (cli) job; Test (runtime) keeps core and engine. The required Test check waits on both and fails if either does not succeed. No required check names change.

Before and after job times are in this PR's own CI run.

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 151 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES · 2 blocker · 0 important · 0 nits · head 562c323e

The shard split itself is right. Test now waits on test-cli and fails closed, and in this run Test (cli) took 5m15s and Test (runtime) 4m39s. There are two blockers, both inline: the required Format check is red, and the new job does not install the numpy that #6327 needs.

Scope

Read the full diff and grepped for other test-runtime / Test (runtime) consumers; Test is the only aggregator. Reproduced the oxfmt 0.41.0 output locally. git merge-tree of this head with #6327 d6ee1cfc merges cleanly, so git will not flag the numpy break.

Comment thread .github/workflows/ci.yml Resolved
Comment thread .github/workflows/ci.yml Resolved

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE — 2 resolved, 0 open, 0 new · 562c323e → a378cafc. CI is green: Test (runtime) took 4m35s and Test (cli) 4m52s.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 25, 2026
Merged via the queue into main with commit 8214cdc Oct 25, 2026
69 checks passed
@miguel-heygen
miguel-heygen deleted the ci/split-cli-tests branch October 25, 2026 19:13
@heygengenesis heygengenesis Bot mentioned this pull request Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant