Repository navigation
ci: run one UI test class ten times per OS on request - #156
Open
BartvHelvert wants to merge 1 commit into
Open
BartvHelvert wants to merge 1 commit into
BartvHelvert wants to merge 1 commit into
Conversation
2 of 5 tasks
Reading a flake takes repeats, and until now each one cost a whole matrix: ten throwaway branches, ten dispatches, and the unit tests, the plugin verifier and the other shard run along with it, for one class. The Escape investigation did that twice. `workflow_dispatch` now takes the class to repeat, as a `--tests` filter. Given one, the run is twenty UI jobs -- ten per OS -- and nothing else, and it keeps every run's IDE logs, the passing ones too, so a failure can be read against the runs either side of it. Given nothing, which is every push and pull request, the workflow is what it was. The filter reaches the script through the environment rather than an expression in it: it is text whoever dispatched the run chose. Taken from the two investigations that each grew their own copy with the class written into it (#146, #147), so neither has to.
BartvHelvert
force-pushed
the
ci/repeat-one-ui-test
branch
from
November 11, 2026 21:33
ba2e083 to
9cc075a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
workflow_dispatchtakes the UI test class to repeat, as a--testsfilter. Given one, the runis twenty UI jobs — ten per OS — and nothing else, and it keeps every run's IDE logs, the passing
ones too, so a failure can be read against the runs either side of it. Given nothing, which is
every push and pull request, the workflow is exactly what it was.
Reading a flake takes repeats, and until now each one cost a whole matrix. The Escape
investigation (#113) did this twice: ten throwaway
ci-flake/escape-*branches, ten dispatches,and each run carried the unit tests, the plugin verifier and the other UI shard along with it —
for one test class. Twenty full matrices to get five failures.
Where it comes from
#149 and #150 each grew their own copy of this, differing only in the test class written into the
job name and the Gradle arguments:
They would have conflicted had both landed, and a third investigation would have needed a third
copy. This is that mechanism, extracted once, with the class as an input.
Two changes from those copies:
inputs.repeat_ui_test && … || …keeps working unchanged while the value also serves as the--testsargument. One input instead of two, and no value can select a class the dispatcherdid not name.
env:, rather than an expression spliced into it. Itis text whoever dispatched the run chose, and it should not be able to become part of the
command.
How it was tested
actionlintpasses. Its one remaining complaint,SC2064ontrap "kill $!" EXIT, ispre-existing on master — and expanding
$!at trap time is what that line wants.workflow_dispatchinputs only take effect once the workflow file is on thedefault branch, so this cannot be exercised from a PR branch. The expressions are the ones
already proven by test(completion): trace a missing snippet selection #149's and test(debugger): capture startup subscription failures #150's batches
(42728115535,
42728125991), which ran
twenty jobs each and uploaded per-repeat artifacts; what differs here is the input's type and
how it reaches the shell.
names as before.
Checklist
A new feature has a UI test.CI only.No product change.docs/is updated where behaviour changed,docs/status.mdespecially.Third-party material keeps its notice and is listed inNone added.NOTICE.md.