Repository navigation
Give every Trial and Grader a memory-capped scope - #1225
Merged
Merged
Conversation
Each Trial and each model Grader runs in a scope of its own, from systemd-run --user --scope, with MemoryHigh, MemoryMax and MemorySwapMax from the new [trial] User config section (defaults 1G, 2G, 1G from the #1009 Harness measurements). The scope raises its own oom_score_adj to the maximum; a Trial killed by its cap dies alone and is void with memory_cap, from the scope's memory.events or journal evidence. Without a reachable user manager Trials run directly, still raised, with a warning naming the loginctl enable-linger step.
Route unscopped Trial and Grader fallbacks through the raising wrapper, so a Grader without a user manager still raises its oom_score_adj. Callers supply the machine caps instead of the launchers loading them, and one Attempt bundle replaces the eight-argument trial command. Merge the two launch types, name the shared oom-kill cause once, and share the scope-availability check across the test suite.
…id=0e04ac92-0789-4332-9896-26905b35c395]: Subscription quota exhausted. Your usage window resets at 2026-10-24T03:28:16Z. (rate_limit_error)) 2026-10-24T02:31:03Z, host stephens. Uncommitted work at the time of failure is included in this commit.
Preserve Trial and Grader memory scopes alongside the Trial network sandbox and allowlisting proxy. Resolve code-navigation and Experiment documentation to describe both changes. Validation: cargo fmt --check; cargo clippy --all-targets -- -D warnings; scripts/affected-tests origin/issue-977 with the systemd user bus. All 44 site tests and 2,541 Rust tests passed; 58 existing Rust skips. Affected checks took 1,756.79 seconds including compilation (Nextest execution: 1,641.170 seconds).
This was referenced Oct 24, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every scope raises its own
oom_score_adjand usesOOMPolicy=kill, so a tool's OOM cannot leave its Trial successful. Missing user-manager access stops dispatch; logged-out launchers discover the user's runtime directory. User config documents the1G/2G/1Gdefaults, and repeated cap kills remain Arm findings.Closes #1015
Evidence
Before: the independent reviewers' targeted reproductions failed: code Graders inherited OOM score
0; an unreachable manager dispatched uncapped Trials; a cap-killed tool left a succeeded, passed Trial;2GGwas accepted; and both defaults examples omitted[trial]. The follow-up also reproduced a live-test failure without bus environment variables.After: each reproduction passed after its fix, and the regressions are retained. The live 300 MiB Trial cap test also passed: only the Candidate died, its three attempts were Void, the Baseline finished, and the model Grader retained OOM score
1000.Final validation passed:
scripts/affected-tests issue-977selected all 44 site tests and the full Rust suite: 2,518 passed, 58 skipped. The real user-bus environment enabled the live scope tests./usr/bin/time -p cargo nextest runmeasured 2,039.87 seconds, including compilation (Nextest execution: 1,934.676 seconds) on this shared host. CI is fully green one698c21d, including formatting, Clippy and the Linux Rust suite. Focusedcargo check, Experiment and User config tests also passed; documentation defaults were checked in both source and generated HTML.Test seams: the User config parser and CLI rejection before work; the Experiment CLI's retained Trial outcomes and Grader Scores; the scope launcher with real systemd caps; the internal
--oom-execcommand's inherited process score.Both independent reviewers read all 23 changed files, including generated HTML directly. Changed files left unread: none on either axis. Generated-output exclusions: none. Reports, including each axis's final files-read list, are retained with the Session trace in
.thirdshift-review-IS1GIg/standards.mdand.thirdshift-review-IS1GIg/spec.md.Unaddressed findings
Standards
None. All five Standards findings are addressed.
Spec
None. All three product findings and the follow-up validation finding are addressed.
Security
None. Security review of the issue-1015 diff (new
src/scope.rsplus the Trial and Grader launch paths) against the resource-exhaustion and local-IPC guidance found no introduced finding with a failing proof-of-concept, so no Security fix was made here. Cap values reachsystemd-runas separate argv entries under an allowlist validator enforced at config parse and env override; scope dispatch refuses to run uncapped when the user manager is unreachable; amemory_capVoid requires manager-owned kernel/journal evidence, never the agent's word.Security review outcome
No unaddressed introduced findings.
Merge Danger
Door: two-way
Revert the code and remove the new
[trial]settings before using an older binary that does not understand them.Blast Radius: Experiments
Experiments require a reachable systemd user manager; enable lingering for logged-out work. Cap-killed Graders stop the Experiment for a cap adjustment and retry. Defaults use #1009's resident-size snapshot; a Trial's footprint beside two building Runs remains unmeasured.
Built with codex · gpt-6.1-sol · xhigh