Skip to content

test: give historical Rust pin Trial more time - #1429

Merged
JacobStephens2 merged 1 commit into
mainfrom
issue-1422
Oct 27, 2026
Merged

JacobStephens2 merged 1 commit into
mainfrom
issue-1422

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Oct 27, 2026 •

Copy link
Copy Markdown
Owner

Summary

Give the historical Rust pin test enough Trial time for Python process startup on loaded macOS runners.

 trial_installs_its_historical_rust_pin_before_the_session
-    fixture.execution(5)
+    fixture.execution(30)

Both Arms still run concurrently, and other tests' timeouts stay unchanged.

Closes #1422

Evidence

  • Before: Historical Rust pin Trial test times out on loaded macOS runners #1422 records both Arms timing out with the 5-second budget on a loaded macOS runner. The original test also passed locally, consistent with the reported flake.
    After: cargo test --bin thirdshift evals::experiment::execution::tests::trial_installs_its_historical_rust_pin_before_the_session -- --exact passes with the 30-second budget.
  • The same exact test fails when the install step is temporarily removed (pinned Rust 1.99.0 download refused by Trial proxy) and when it is temporarily called twice ("install\ninstall\n" versus "install\n"). Both mutations were restored before committing.
  • cargo test --bin thirdshift evals::experiment::execution::tests: 21 passed.
  • cargo fmt --check: passed.
  • /usr/bin/time -p scripts/affected-tests main: passed the full Rust suite via the cargo test fallback; 1434.61 seconds wall time, including 64 seconds of compilation. Unit tests: 1529 passed, 3 ignored; all integration suites passed.

Test seam: Trial execution through Execution::prepare and Execution::run, using the existing fake Arms and Rust tool fixtures.

Merge Danger

Door: two-way

Blast Radius: tests

One test can now wait up to 30 seconds when its Trial stalls. The change is easy to revert.

Review coverage

Reviewed against main with thirdshift-code-review: Standards 0 findings; Spec 0 findings. Both reviewers read the only changed file, src/evals/experiment/execution/tests.rs. Unread changed files: none. Generated-output exclusions: none.

Unaddressed findings

  • Standards: none.
  • Spec: none.
  • Security: none.

Built with codex codex-cli 0.164.0 · gpt-6.1-sol · xhigh

Security review outcome

No unaddressed introduced findings.

@JacobStephens2
JacobStephens2 merged commit cb5b880 into main Oct 27, 2026
18 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-1422 branch October 27, 2026 05:12
This was referenced Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Historical Rust pin Trial test times out on loaded macOS runners

1 participant