CI perf: test (windows/macos) — redundant cargo build step + full debuginfo linking ~240 test binaries (~930 job-min/day, ~300 macOS) #1173

Measurement

Window: 2026-10-08 02:16Z → 19:36Z, all CI runs.

test job Build step (cargo build --workspace)

test job duration, 7 days, p50 / p90

Merge_group run 37828932401

• windows shard 2:
  • Build took 84 s.
  • Run tests took 13.6 min: 5 min 28 s compiling and linking (only socket-patch-core/socket-patch-cli show Compiling, so deps came from the rust-cache) plus 8.2 min running 159 test binaries.
• windows shard 1:
  • Build took 91 s.
  • Core lib unit tests took 210 s, against 57 s on macOS and 48 s under coverage.
• macOS shard 1 / macOS shard 2: Build took 85 s / 74 s.

Where the time goes

• Build compiles the workspace bins with the dev profile. cargo test in the next step recompiles the same crates as test targets and links them.
• Linking about 240 integration-test binaries with full DWARF/PDB debuginfo dominates the 5.5 min compile phase on Windows.

Root cause

• ci.yml test job, line ~285: - name: Build / run: cargo build --workspace runs before scripts/ci-test-shard.py.
• cargo test already builds every [[bin]] that integration tests use via CARGO_BIN_EXE_*, so the separate build only helps if some test locates target/debug/socket-patch by path.
• The dev profile's default debug = true makes every test binary carry full debuginfo, which CI never uses. Panics still give file:line with line-tables-only.

Proposed fix

1. Check that no test needs a pre-built binary outside CARGO_BIN_EXE_*: grep for target/debug / env!("CARGO_BIN_EXE under crates/*/tests. Then delete the Build step from the test job.
   • On push to main it can stay (or become cargo test --no-run) if the rust-cache save wants bin artifacts.
2. Set CARGO_PROFILE_DEV_DEBUG: line-tables-only in the test job env, and in e2e-build if it isn't already off. This shrinks link inputs for the ~240 test binaries on Windows and macOS.
3. Follow-up, separate: find out why core lib unit tests take 210 s on Windows against 57 s on macOS. Likely filesystem-heavy tests on NTFS/Defender, so check for tempdir churn.

Expected saving

• Measured: removing Build saves ~628 Windows + ~306 macOS job-min/day, and about 1.5 min off each test leg's wall time.
• Estimated, not yet measured: line-tables-only should save another 1–2 min per Windows/macOS leg, i.e. about 400–800 Windows and 180–360 macOS job-min/day. Please measure this on the PR.
• test legs aren't on the ci-ok critical path today (Gradle e2e is), so the critical-path effect is about 0. The gain is macOS/Windows runner time and slot occupancy.

Coverage and risk

• No test is removed. If some test does depend on a pre-built binary, step 1 shows that immediately as a test failure on the PR.
• line-tables-only keeps backtrace line numbers.
• Required checks ci-ok and clippy are unchanged.

Effort

S

ROI

ROI = weighted saving × confidence / effort.

• Weighted saving: (2 × 0.63 Windows + 3 × 0.31 macOS) = 2.2k job-min/day.
• Confidence: 0.6.
• Effort: S, scored as 1.
• ROI = 1.3