Add opt-in incremental GC with automatic pacing

Large live heaps currently require one uninterrupted major mark. Retain mark queues between stopped-world slices, record mutations with the existing generated barriers, and resume array and map scans through managed owners.

Keep the current collector as the default. Add library scheduling hooks, independent reachability verification, epoch recovery, allocator range preservation, conventional regressions and CI stress coverage.

Record matched frame and allocation benchmarks. Tail latency improves on the measured workloads, but throughput and finishing-pause gates remain open in #372. Concurrent marking in #327 stays deferred.

Author: dimensionscape

.github/workflows/main.yml

@@ -224,6 +224,20 @@ jobs:
       working-directory: test/gc_stress
       run: haxe compile.hxml -D HXCPP_M64 -D HXCPP_SANITIZE=${{ matrix.sanitize }} && bin/Stress 10
 
+    # The new collector has its own deterministic scheduling and barrier paths.
+    - name: Run incremental GC regressions
+      if: ${{ !cancelled() && steps.setup.outcome == 'success' }}
+      working-directory: test/regression
+      run: |
+        for test in GcMutationRoots GcContainerMutation IncrementalGcSlices IncrementalGcContainers IncrementalGcFinishBoundary GcWeakMapChain GenerationalFinalizerLifecycle GenerationalLargeBufferLifecycle; do
+          (cd "$test" && haxe -main Main -cpp bin -lib hxcpp -D HXCPP_M64 -D HXCPP_GC_INCREMENTAL -D HXCPP_GC_INCREMENTAL_VERIFY -D HXCPP_GC_INCREMENTAL_TEST -D HXCPP_SANITIZE=${{ matrix.sanitize }} && bin/Main > out.txt && diff <(tr -d '\r' < stdout.txt) <(tr -d '\r' < out.txt))
+        done
+
+    - name: Run incremental GC stress
+      if: ${{ !cancelled() && steps.setup.outcome == 'success' }}
+      working-directory: test/gc_stress
+      run: haxe compile.hxml -lib hxcpp -D HXCPP_M64 -D HXCPP_GC_INCREMENTAL -D HXCPP_GC_INCREMENTAL_VERIFY -D HXCPP_SANITIZE=${{ matrix.sanitize }} && bin/Stress 10 8 2012663045
+
     - name: Show the sanitizer reports
       if: ${{ !cancelled() }}
       run: cat /tmp/sanitizer/report.* 2>/dev/null || echo "No sanitizer reports"

.github/workflows/test.yml

@@ -363,6 +363,49 @@ jobs:
           done
           [ -z "$failed" ] || { echo "Failed modes:$failed"; exit 1; }
 
+  incremental:
+    if: inputs.scope == 'full' && startsWith(inputs.haxe, '4.3.')
+    runs-on: ${{ inputs.os }}
+    name: incremental gc
+    timeout-minutes: 90
+    defaults:
+      run:
+        shell: bash
+    steps:
+      - name: checkout
+        uses: actions/checkout@v7
+      - name: setup
+        uses: ./.github/workflows/setup
+        with:
+          haxe: ${{ inputs.haxe }}
+      - name: deterministic phases and semantics
+        working-directory: test/regression
+        run: |
+          failed=""
+          for test in GcMutationRoots GcContainerMutation IncrementalGcSlices IncrementalGcContainers IncrementalGcFinishBoundary IncrementalGcVerifier GcWeakMapChain GenerationalFinalizerLifecycle GenerationalLargeBufferLifecycle; do
+            echo "::group::incremental gc: $test"
+            # The lifecycle fixtures' hxml selects generational mode; compile
+            # the same Main with incremental mode here instead.
+            if ! (cd "$test" && haxe -main Main -cpp bin -lib hxcpp -D HXCPP_GC_INCREMENTAL -D HXCPP_GC_INCREMENTAL_VERIFY -D HXCPP_GC_INCREMENTAL_TEST -D $HXCPP_ARCH_FLAG && bin/Main > out.txt && diff <(tr -d '\r' < stdout.txt) <(tr -d '\r' < out.txt)); then
+              failed="$failed $test"
+              echo "::error::incremental gc failed: $test"
+            fi
+            echo "::endgroup::"
+          done
+          [ -z "$failed" ] || { echo "Failed tests:$failed"; exit 1; }
+      - name: incremental stress
+        if: ${{ !cancelled() }}
+        working-directory: test/gc_stress
+        run: |
+          haxe compile.hxml -lib hxcpp -D HXCPP_GC_INCREMENTAL -D HXCPP_GC_INCREMENTAL_VERIFY -D $HXCPP_ARCH_FLAG
+          bin/Stress 30 8 2012663045
+      - name: incremental stress (big blocks)
+        if: ${{ !cancelled() }}
+        working-directory: test/gc_stress
+        run: |
+          haxe compile.hxml -lib hxcpp -D HXCPP_GC_INCREMENTAL -D HXCPP_GC_INCREMENTAL_VERIFY -D HXCPP_GC_BIG_BLOCKS -D $HXCPP_ARCH_FLAG
+          bin/Stress 30 8 1337
+
   cppia:
     if: inputs.scope == 'full'
     runs-on: ${{ inputs.os }}

BENCHMARKS.md

@@ -97,6 +97,7 @@ These numbers are from one Windows machine; Linux and macOS results are still to
 | `alloc_strings` | Int-to-String conversion plus a concatenation per string |
 | `alloc_arrays` | Arrays of 1 to 64 Ints grown by `push`, plus a 256 KB buffer every 4000 arrays |
 | `gc_pauses` | Collection pauses as the program sees them: a large live graph stays reachable while the program allocates in fixed work units, and each unit's duration is recorded. The tail holds the units that included a collection |
+| `gc_frames` | 2,000 frames over 150,000 retained entities with graph and inventory mutation, temporary particle chains and event strings, fixed-size spatial cells and periodic allocation bursts. Records frame latency and checks retained payloads. Uses automatic GC without frame hooks or sleeps |
 | `map_int` | `IntMap`: on a fresh map, N sets, N hits, N misses, N/2 removes, then iterating the N/2 values left |
 | `map_string` | `StringMap`, the same cycle |
 | `map_object` | `ObjectMap`, the same cycle |

Changes.md

@@ -2,6 +2,7 @@
 libhxcpp 6.0.0 (unreleased)
 ------------------------------------------------------------
 
+* Added an opt-in incremental marking collector for native Haxe 4 C++ builds (`-lib hxcpp -D HXCPP_GC_INCREMENTAL`), with automatic allocation pacing, optional `hxcpp.gc.Incremental` scheduling hooks, resumable array and map scans, an independent verifier and deterministic regression tests. The existing collector remains the default. Slice budgets are targets, and synchronous completion and memory pressure can still cause longer pauses; see docs/IncrementalGc.md
 * Emscripten programs let memory grow and get a 5 MB stack by default. They linked with Emscripten's fixed 16 MB of memory, which aborts with "Aborted(OOM)" as soon as a program holds more, and since Emscripten 3.1.27 a 64 KB stack, which Haxe recursion and the pointers that spill-pointers keeps on the stack soon overflow. HXCPP_LINK_NO_MEMORY_GROWTH and HXCPP_LINK_NO_STACK_SIZE restore Emscripten's settings, and HXCPP_LINK_STACK_SIZE sets another size
 * Fixed converting a long or a size_t to Dynamic failing to compile on Emscripten, where long is 32 bits but not int, as an Int default of -2147483648 does (HaxeFoundation/hxcpp#621)
 * Fixed cppia hosts built with HXCPP_DEBUGGER failing to compile on Emscripten, which has no cppia JIT: the debugger's frame reader asked whether a function was jitted through a field that only exists with the JIT

bench/results/incremental-gc/README.md

@@ -0,0 +1,63 @@
+# Incremental GC development measurements
+
+These results compare the existing libhxcpp collector with the experimental incremental mode in the same source tree. They do not replace the stock-hxcpp comparison in [BENCHMARKS.md](../../../BENCHMARKS.md). Both collectors use the same benchmark sources, optimized compiler settings and payload checks.
+
+The raw process results, sample times, latency summaries, checksums, peak working sets, reserved heap sizes and incremental counters are in [windows-x64-20261026.json](windows-x64-20261026.json). The containing commit identifies the implementation; the JSON also records the base revision and source hashes.
+
+## Results
+
+Elapsed time is the median of the 15 measured samples. Latency columns show the largest per-process p99.9 and the largest observed work-unit duration across the three processes; the p99.9 column is not a pooled percentile. Memory columns show the largest process value.
+
+| Workload | Collector | Elapsed, ms | Worst p99.9, ms | Maximum, ms | Peak RSS, MiB | Reserved heap, MiB |
+| --- | --- | ---: | ---: | ---: | ---: | ---: |
+| `gc_frames` | Existing | 240.246 | 7.742 | 9.728 | 128.76 | 115.36 |
+| `gc_frames` | Incremental | 294.961 | 0.969 | 1.858 | 131.98 | 118.29 |
+| `gc_pauses` | Existing | 145.429 | 5.055 | 8.289 | 135.29 | 121.55 |
+| `gc_pauses` | Incremental | 256.843 | 2.718 | 5.232 | 135.52 | 122.51 |
+
+All six `gc_frames` processes returned checksum 1106626610; all six `gc_pauses` processes returned 458711266. Incremental mode reduced tail latency on these workloads, with 22.8% more elapsed time for `gc_frames` and 76.6% more for `gc_pauses`. That corresponds to 18.5% and 43.4% less throughput, respectively. Neither meets the proposed maximum 10% throughput loss.
+
+The largest budgeted collector invocation was 1.346 ms in `gc_frames` and 1.783 ms in `gc_pauses`, against a requested 0.5 ms. Cumulative unbudgeted maxima were 13.553 ms and 9.971 ms, respectively, including explicit collections outside measured frames. These results do not establish a sub-millisecond collector or eliminate synchronous interruptions.
+
+## Protocol
+
+- Windows x64, AMD Ryzen 9 9950X, Haxe 4.3.7 and MSVC 19.44.35213.
+- Default block size, existing collector defaults versus `HXCPP_GC_INCREMENTAL` with its default 0.5 ms target and 1 ms preferred interval.
+- No `HXCPP_GC_INCREMENTAL_VERIFY`, `HXCPP_GC_INCREMENTAL_TEST`, scriptable mode or explicit engine scheduling hooks.
+- Three processes per workload and collector, two warmups and five measured samples per process. Collector order alternates between processes. Compilation, stress and sanitizer runs finished before measurement.
+- `gc_frames` retains 150,000 entities and runs 2,000 frames per sample. Frame timing includes graph and inventory changes, temporary particles and event strings, fixed-size spatial cells and periodic bursts. Payload validation runs after the frame loop and is included in elapsed sample time.
+- `gc_pauses` retains a large graph and records 3,000 allocation work units per sample. A work unit is not a game frame or an isolated GC pause.
+- The harness requests a full collection before each sample, outside its timed section. Cumulative GC counters include setup, warmups, these explicit collections and validation. Their maxima cannot be presented as maxima of GC work inside measured frames.
+- Checksums must agree across all processes of a workload. The JSON contains elapsed sample times and per-process latency summaries, rather than a trace of individual frames.
+
+## Reproduction
+
+After pointing haxelib at this checkout as described in CONTRIBUTING.md, build each mode sequentially:
+
+```sh
+haxe -cp bench/src -main Main -cpp build/gc-default -lib hxcpp -D HXCPP_M64
+haxe -cp bench/src -main Main -cpp build/gc-incremental -lib hxcpp -D HXCPP_M64 -D HXCPP_GC_INCREMENTAL
+```
+
+For each workload, run the following pair three times, reversing the order in the second pair. On Windows the executable is `Main.exe`.
+
+```sh
+build/gc-default/Main gc_frames --warmup 2 --samples 5
+build/gc-incremental/Main gc_frames --warmup 2 --samples 5
+build/gc-default/Main gc_pauses --warmup 2 --samples 5
+build/gc-incremental/Main gc_pauses --warmup 2 --samples 5
+```
+
+## Correctness validation
+
+Local Windows validation includes all nine phase and lifecycle fixtures listed in [IncrementalGc.md](../../../docs/IncrementalGc.md), with the independent verifier and deterministic controls, at normal and big block sizes. Each block size also passed three 30-second stress runs with eight threads and seeds 2012663045, 1337 and 8675309. The full runtime suite passed 1,100 assertions with incremental verification and big blocks. The new slice, container and finishing fixtures also pass with the existing collector, and the container and finishing fixtures pass in single-threaded incremental builds.
+
+Linux under WSL, Haxe 4.3.3 and GCC 13.3 passed ASan and UBSan builds of the slice, container and finishing fixtures, plus a 30-second stress run with eight threads under each sanitizer. Leak detection is disabled because these runs exercise the runtime's conservative stack scanning configuration. They are not leak measurements. The final GCC define matrix passed all 79 configurations, including incremental, verification/test and single-threaded variants at both block sizes.
+
+The independent verifier fixture deliberately omits a reachable descendant's mark in a child process and requires rejection. The epoch recovery assertion failed before the recovery fix, and the allocation cursor assertion failed before preserving allocator ranges across partial slices.
+
+Local TSan did not reach a usable test run: the installed GCC runtime rejected the repository suppression syntax and, without suppressions, failed under WSL before the workload ran. TSan and other platform results remain for CI. CI configuration includes incremental regressions and stress in the sanitizer jobs and a native incremental job on supported Haxe 4 configurations.
+
+## Remaining gates
+
+This is experimental. The throughput target in [#372](https://github.com/openfl/libhxcpp/issues/372) is still open. Weak closure, finalizers, reclamation, root capture, thread rendezvous and forced completion can exceed the slice target. Longer soaks, additional memory-pressure and native-integration coverage, pause distributions for all stopped threads, progress and metadata high-water counters, and cross-platform results are still needed before considering a default change.