feat: add runtime memory-aware build concurrency control #802

Motivation

mcpp can still exceed the available physical memory while compiling, including when mcpp builds itself. This has caused repeated OOM failures. The current controls only set a static concurrency limit:

• ~/.mcpp/config.toml [build] default_jobs
• MCPP_JOBS / --jobs
• project-level [build] jobs

The existing jobs = "auto" calculation uses a startup estimate of available memory, but it cannot react when compiler processes reach their peak RSS or when other processes consume memory during the build.

Proposed behavior

Add runtime memory-aware admission control for compiler processes. The default policy should reserve a configurable proportion of the physical memory available at build startup, excluding swap; the proposed default threshold is 80%.

A possible configuration is:

[build]
memory_threshold_percent = 80

with an environment override such as MCPP_MEMORY_THRESHOLD_PERCENT. The exact names can be discussed.

The memory policy should act as a dynamic upper bound, not as a replacement for jobs configuration:

actual concurrency = min(configured jobs, currently memory-safe concurrency)

When memory pressure exceeds the threshold, new compiler processes should wait or the effective concurrency should temporarily decrease. Already-running compiler processes should not be killed. Hysteresis should be used so that concurrency does not oscillate, and at least one job should remain possible when memory readings are supported but conservative.

Explicit --jobs / MCPP_JOBS should remain an upper bound rather than disabling the memory safety mechanism.

Important: bmi_schedule = "on"

The implementation must cover the detached code-generation path, not only ordinary Ninja -j scheduling.

With GCC and bmi_schedule = "on":

• Ninja uses an expanded ninjaJobs value.
• The BMI edge finishes as soon as the BMI is published.
• The supervisor and compiler continue running in the background.
• The real compiler concurrency is controlled by compilerCap and the .mcpp-sched token mechanism, not by Ninja -j alone.

Therefore, changing Ninja -j dynamically would not be sufficient. The memory-aware admission check must be shared by ordinary compilation and detach-codegen, and a detached compiler token must remain held until the entire compiler/code-generation process exits, not merely until the BMI is published. BMI publication and the subsequent detached code generation must both count toward memory pressure.

Possible implementation direction

1. Extend mcpp.platform.capacity with a runtime physical-memory snapshot.
   • Linux: MemAvailable, with cgroup v2 limits respected when present.
   • macOS: physical free/inactive memory.
   • Windows: physical available memory from the platform API.
   • Swap should not be counted as available build memory.
2. Add a shared dynamic compiler admission controller based on the existing scheduler/token mechanism.
3. Route ordinary compiler starts and detached supervisors through that controller.
4. Preserve Ninja -j and compilerCap as hard maximums, while allowing the admission controller to apply a lower runtime limit.
5. Add unit and integration tests for memory-pressure throttling, recovery, stale-token cleanup, and bmi_schedule = "on".
6. Document the new configuration and the distinction between static jobs limits and runtime memory protection in both English and Chinese documentation.

Acceptance criteria

• A build does not continue starting compiler processes after the configured physical-memory threshold is exceeded.
• The behavior works with ordinary scheduling and with bmi_schedule = "on" / GCC detached code generation.
• Detached code generation remains accounted for until the compiler fully exits.
• --jobs, MCPP_JOBS, project [build] jobs, and machine default_jobs continue to define the maximum concurrency and retain their existing precedence.
• Memory detection degrades safely when platform metrics are unavailable.
• The implementation is covered on Linux, macOS, and Windows where the relevant APIs are available.