Skip to content

Remove Scenario fixture roots after killed test harnesses - #2117

Merged
JacobStephens2 merged 4 commits into
mainfrom
issue-2093
Nov 12, 2026
Merged

JacobStephens2 merged 4 commits into
mainfrom
issue-2093

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Summary

Closes #2093. Give each Scenario root a directory guardian that survives harness-only and process-group SIGKILL. Previously, Run guardians ended descendants but only the killed harness's TempDir could remove the fixture.

Scenario drops or harness dies
  Run guardians kill and reap owned descendants
  last writer closes the private completion pipe
  directory guardian removes only its captured Scenario root

Preserve process-only OwnedRuns callers. Document the contract in docs/testing.md and verify normal completion, assertion unwinding, both SIGKILL patterns, and concurrent Scenarios.

Evidence

Execution evidence

  • Before: cargo test --locked --test scenario_lifetime -- --nocapture with the new root-absence assertion, before implementation:
    Scenario root survived its fixture
    test result: FAILED. 2 passed; 4 failed; 2 ignored
    
    All recorded descendants had exited; all four harness-only/group SIGKILL cases retained their Scenario directory before the supervisor's outer TempDir cleanup. Normal completion and assertion unwinding passed.
    After: cargo test --locked --test scenario_lifetime -- --nocapture:
    test result: ok. 7 passed; 0 failed; 3 ignored
    
  • Review regression: cargo check --locked --test quota_wait initially failed with E0061 at tests/quota_wait.rs:309; preserving OwnedRuns::new() and attaching directory ownership separately makes the exact command pass. cargo test --locked --test scenario_lifetime --test quota_wait passes 7 lifecycle tests and 8 quota tests.
  • Cleanup race: cargo test --locked --test console codex_patches:: -- --test-threads=8 failed before bounded removal retries (Directory not empty, 5 passed / 1 failed), then passed all 6 tests after the correction. Transient missing entries count as success only when the captured root is absent.
  • Final local validation: RUST_TEST_THREADS=3 scripts/affected-tests main — PASS: 4,418 tests passed, 0 failed, 197 ignored subprocess/platform cases across 108 targets. The selector's cargo test fallback took 2,548.68 seconds, including compilation. The thread limit follows docs/testing.md for loaded hosts; a Console scoring assertion in the initial default-parallel run passed alone, and all 321 Console tests passed with three threads. Full output is retained in .affected-tests/cargo-test-af3t1fuw.log beside the Session evidence.
  • CI: all checks are green on 6b41029c, including all four Rust x86 partitions and their coverage gate, site, and Console browser tests.

Seams under test: the Scenario's observable root and owned process lifetime, exercised through Scenario::new, run, run_until, and drop in separately supervised harnesses. The supervisor records exact PIDs and the exact root, reaps the directory guardian, and asserts root absence before its emergency-cleanup TempDir is dropped. The concurrency case drops one Scenario while another remains usable.

Confirmed cause: SIGKILL bypasses TempDir::drop. The new exec'd directory guardian has its own process group and waits for EOF on a private close-on-exec pipe held by the harness and every Run guardian, ordering directory removal after descendant cleanup.

Review coverage

Standards: 0 findings. Spec: 1 finding, fixed and verified with the reviewer's exact command. Both axes read all four changed files; no changed file was left unread, and no generated output was excluded. Reports and counts are retained in .thirdshift-review-dYuGzK/ beside the Session evidence.

Unaddressed findings

Standards

None.

Spec

None. The constructor finding was addressed; no findings were skipped.

Security

None.

Security review outcome

No unaddressed introduced findings.

Merge Danger

Door: two-way

Revert the test-infrastructure changes to restore the previous lifetime behavior.

Blast Radius: tests

Each Scenario starts one extra cleanup process. Deletion is confined to its explicitly captured fixture root; no historical directory cleanup or generic temporary-directory sweeping is added.

Built with codex codex-cli 0.167.2 · gpt-6.1-sol · xhigh

@JacobStephens2
JacobStephens2 merged commit 70835ae into main Nov 12, 2026
12 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-2093 branch November 12, 2026 15:45
@JacobStephens2 JacobStephens2 mentioned this pull request Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Integration-test Scenario fixtures leak into /tmp: about 650 directories (4.9 GB) per host since Nov 4

1 participant