Skip to content

Resource limits checked after collecting the resource #13

Open

Description

@cblanquera

Timing: Follow-up after trial/3 merges into main; defer implementation until then.

Review baseline: Local trial/3 at a094418baf6388e3eeca23821df481a7f0a8751c. This branch is not yet on the remote. Source locations below refer to that snapshot and should be rechecked after merge.

Problem

The generation formatter collects complete subprocess stdout and stderr before checking the generated-output byte ceiling. Its final artifact limit therefore does not bound captured diagnostics, peak memory, or execution time. There is no formatter deadline.

Evidence

  • crates/idea/idea-transformer/src/output/format.rs:42: wait_with_output() collects both streams.
  • crates/idea/idea-transformer/src/output/format.rs:50-52: the 1 MiB output check runs after collection.
  • A controlled formatter probe used 975,000 input bytes and a temporary configuration with many unknown-setting warnings. It exited successfully while producing 2,058,890 stderr bytes, which the current collection pattern would retain before discarding them.
  • The separate pipe-order experiment did not establish a rustfmt deadlock; this issue makes no such claim.

Acceptance criteria

  • Enforce finite limits while collecting subprocess stdout and stderr, and define a formatter time policy.
  • Kill and reap the child on limit, timeout, and IO-failure paths without hanging or leaking a worker/process.
  • Preserve formatted artifact compatibility, generated-file ownership, and no-install-on-format-failure behavior.
  • Verify oversized diagnostics/output and stalled-child behavior with watchdogs or controlled subprocess fixtures.
  • State separately the final artifact budget and the subprocess acquisition/runtime limits.

Code examples and suggested direction

Example bounded stream collector, to be integrated into a supervised subprocess design:

fn read_capped(
    mut input: impl std::io::Read,
    limit: usize,
) -> std::io::Result<Vec<u8>> {
    let mut output = Vec::new();
    let mut chunk = [0_u8; 8192];
    loop {
        let count = input.read(&mut chunk)?;
        if count == 0 {
            return Ok(output);
        }
        if count > limit.saturating_sub(output.len()) {
            return Err(std::io::Error::other("formatter stream limit exceeded"));
        }
        output.extend_from_slice(&chunk[..count]);
    }
}

The check happens before retaining each chunk. This helper alone does not provide a deadline or concurrent pipe draining; a supervisor must stop/reap the child when either collector fails.

Activity

  1. cblanquera commented on Oct 26, 2026

    @cblanquera
    MemberAuthor

    Suggested investigation and fix notes

    Deferred until trial/3 is merged into main. Recheck the merged source and tests first. The snippets are examples/proposals, not an approved patch or a change to frozen contracts.

    1. Separate final artifact size, per-stream captured bytes, aggregate run budget, and elapsed-time policy. Select finite defaults explicitly; a 1 MiB final-output ceiling is not a subprocess memory/time ceiling.
    2. Replace unrestricted collection with concurrent stdin writing and bounded stdout/stderr readers under a single completion/timeout owner. The deadline must cover startup/input/collection, not just a wait after input is written.
    3. On overflow, reader/writer failure, or timeout, notify the supervisor immediately, stop the child, reap it, and account for all worker handles. A reader that simply stops consuming can leave its child blocked on the pipe.
    4. Keep formatting outside output-install authority. A failed formatter must not modify managed output or leave a newly acquired writer lock. Preserve normal rustfmt arguments and formatted bytes.
    5. Inject the formatter command/runner through a narrow private test seam. Avoid mutating process-global PATH in parallel tests.
    6. Test exact-limit and one-byte-over cases, large stderr with otherwise valid stdout, slow/no-output children, early exit/broken stdin, and repeated failures. Use watchdogs and verify no child/reader remains alive.
    7. Keep the experiment's conclusion precise: a rustfmt deadlock was not reproduced. Concurrent draining is preventive design, not a claim that the original review established one.

    Collector component example:

    assert_eq!(read_capped(&b"abc"[..], 3)?, b"abc");
    assert!(read_capped(&b"abcd"[..], 3).is_err());
    assert_eq!(read_capped(&b""[..], 0)?, b"");

    Start with cargo test -p stackpress-idea-transformer --locked, especially src/output/tests.rs. Coordinate error categories and retained causes with #3, #10, and #11.

    Implementation workflow: After the merge, follow .agents/workflows/rust-programming.md: prove the affected behavior, complete the three test-verified review rounds, and record relevant formatting/lints/docs, feature/platform scope, and applicable development coverage. Preserve unrelated changes.

  2. cblanquera commented on Oct 26, 2026

    @cblanquera
    MemberAuthor

    Fixed on trial/3 in 407b4d6 + df443b7 (merged as ff7a251, task 00010 lane 2; not yet on main, so leaving open).

    • The formatter subprocess's stdout/stderr are now capped while reading, with a deadline; the child is killed and reaped on limit, timeout or IO failure.
    • The final 1 MiB artifact budget and the subprocess limits are stated separately; a failed format still installs nothing.
    • Controlled fixtures assert completion under one second and, on Unix, that the PID no longer exists.

    Coordinator-verified on the combined code: fmt, Clippy -D warnings, Rustdoc -D warnings, cargo +1.96.0 check, 466 workspace tests and 857 doctests pass (467 tests on merged trial/3). Coverage: server 91.61%, CLI 93.15%, transformer 93.62%.

  3. added a commit that references this issue on Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions