Repository navigation
Conversation
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
shuvroroy
force-pushed
the
ci/optimize-workflow-runtime
branch
from
November 8, 2026 19:04
d5faae9 to
da7f952
Compare
shuvroroy
force-pushed
the
ci/optimize-workflow-runtime
branch
2 times, most recently
from
November 8, 2026 20:26
d80559a to
48e9c4d
Compare
Build the Linux coverage suite and Windows suite once, then distribute Nextest archives across six Linux and four Windows runners. Cache Windows archives for the exact revision, compiler, image, and profile to avoid recompilation on reruns while still executing every partition. Preserve the existing aggregate checks and Windows policy. Cache release benchmark artifacts for exact source and build inputs, reuse matching baselines, and share the head server with scaling and memory checks. Pair manifests in measurement workers rather than queueing a preparation job. Keep normal Criterion sampling and 30% thresholds. Pin Rust, test tools, and runner families; populate scoped PR caches; and report cache hits, compiler timings, and runner queue time. Add manual 8/12/16-worker timing runs. Wait for native diagnostic passes before memory sampling so temporary parses do not enter the result. Give the optional semantic-export integration tests their own target so lint does not compile the full integration suite a second time for ten tests. Validated with actionlint, 41 benchmark-helper tests, default and feature Clippy checks, formatting, all 11 semantic-export tests, and archived coverage execution. Both Windows harnesses passed the same 20,096 cases. The archive-based revision passed the full PR workflow in 9m 29s, with Windows completing in 7m 22s before adding its rerun cache and isolating the feature tests. Further cold and warm timings are recorded in the PR.
shuvroroy
force-pushed
the
ci/optimize-workflow-runtime
branch
from
November 8, 2026 20:30
48e9c4d to
6430430
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI previously took 23m 44s, with repeated Rust compilation and benchmark work on the critical path. This PR shares compiled artifacts, parallelizes independent builds, and reuses exact matching builds. The final upstream PR run passed in 10m 08s; a controlled same-commit warm run passed in 5m 14s.
Every existing check still runs on every PR. The 5–6 minute target is demonstrated for a warm rerun; a new commit still exceeds it.
Before and after
Build reuse and cache boundaries
Benchmark artifact keys include the exact source revision, build recipe, compiler, runner image, native toolchain, and build flags. They store benchmark executables, their manifest, the release server, and build timings. A cache miss builds the requested revision; there are no approximate artifact restore keys. Main-branch builds publish entries that later PRs can reuse for a matching base, while PR-created entries remain in their GitHub cache scope.
The Windows archive key includes the exact commit, compiler, runner image, target, and profile settings. Cache hits reuse compilation only: every test partition and every benchmark measurement still executes. Linux coverage continues to build its instrumented archive on each run. Dependency caches remain separate from these exact artifact caches, and Composer downloads are cached for memory measurements.
PR benchmarks still use the actual first parent of GitHub's test merge as the baseline. Each worker measures base and head sequentially on the same VM, using each revision's own fixtures. Criterion sampling, case accounting, and the 30% latency and memory regression thresholds are unchanged. The existing non-blocking Windows policy is also preserved.
Memory measurement correction
The previous fixed delay could sample RSS while background diagnostics still held temporary parse data. The same code produced a 161.2 MiB WordPress sample in one run and 73.1 MiB in another. Measurements now wait for both native diagnostic publications for the opened file before the existing settling period and RSS sampling. This changes synchronization in the benchmark helper; it leaves server diagnostics and regression thresholds intact. Tests cover waiting for the slow pass, ignoring unrelated files, and enforcing the timeout.
Measured results
The final three runs use commit
6430430913e062abef5bb9f3555cc66bfe8f907d. Their cache state and queue conditions differ, so these are observed outcomes rather than a guarantee for every PR.In the warm run, each benchmark build job took 41s and the Windows archive job took 33s, with Windows compilation skipped on the exact cache hit. The complete Windows path, measured from workflow creation through its aggregate check, fell from 11m 41s in the first optimization to 7m 51s in the final upstream run and 3m 01s in the warm run. The warm run still rebuilt Linux coverage (3m 11s) and ran lint (4m 56s).
New source revisions must compile changed project code, and cache misses, runner image updates, and queue contention can increase duration. The manual fork runs execute the complete test and benchmark workload and generate coverage; only the external Codecov upload is skipped when that repository has no token. Normal PR and push uploads retain their existing behavior. All checks on the final upstream PR run are green.
Validation
The successful upstream workflow covers formatting, default and semantic-export Clippy, Linux coverage, Windows tests, semantic-export tests, Python helpers, WebAssembly Clippy/build/smoke test, PHP lint and existing runtime assertions, scaling, and latency and memory comparisons.
Additional validation completed during development:
cargo +1.99.1 clippy --fix --allow-dirty -- -D warnings, followed bycargo +1.99.1 fmt; default and semantic-export all-targets Clippy also passed without warnings.git diff --checkpassed.Checklist
CHANGELOG.mdREADME.md,docs/,examples/)config-schema.json) — not applicable