Skip to content

refactor(cuda): keep inflight pinned buffers unwrapped - #10878

Merged
0ax1 merged 1 commit into
developfrom
ad/cuda-pinned-inflight-unwrap
Nov 3, 2026
Merged

0ax1 merged 1 commit into
developfrom
ad/cuda-pinned-inflight-unwrap

Conversation

@0ax1

@0ax1 0ax1 commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Simplifies inflight pinned-buffer tracking by removing InflightPinnedBuffer,
a wrapper containing only a PinnedByteBuffer.

The pool now stores PinnedByteBuffer directly. Transfer completion checks,
buffer reclamation, and synchronization are unchanged. The existing test
still checks that an inflight buffer retains its original allocation.

Changes

  • refactor(cuda): keep inflight pinned buffers unwrapped

Benchmark

BenchPress comparison against develop, using 8,388,608 Vortex block rows.

Mainline: 9384250 (origin/develop)
HEAD: 52dbd35

Times are means in ms. Negative change = faster on HEAD.
Samples are measured reads per revision, excluding warmup.

+-------+-------------------------+---------+----------+----------+---------+
| Cache | Dataset                 | Samples | develop  | HEAD     | Change  |
+-------+-------------------------+---------+----------+----------+---------+
| Cold  | Synthetic, 2M rows      |      10 |    2.482 |    2.495 |  +0.53% |
| Cold  | Numeric, 2M rows        |      20 |    1.727 |    1.723 |  -0.25% |
| Cold  | Lineitem SF10, decimal  |       2 |  111.025 |  111.160 |  +0.12% |
| Cold  | Lineitem SF10, float64  |       2 |  113.025 |  110.628 |  -2.12% |
+-------+-------------------------+---------+----------+----------+---------+
| Hot   | Synthetic, 2M rows      |       2 |    1.533 |    1.548 |  +1.00% |
| Hot   | Numeric, 2M rows        |       2 |    0.741 |    0.738 |  -0.42% |
| Hot   | Lineitem SF10, decimal  |       2 |   79.811 |   79.283 |  -0.66% |
| Hot   | Lineitem SF10, float64  |       2 |   81.693 |   83.126 |  +1.75% |
+-------+-------------------------+---------+----------+----------+---------+

Cold synthetic uses a 10-sample rerun. Cold numeric combines two
10-sample processes per revision in develop -> HEAD -> HEAD -> develop
order, with one excluded warmup per process. Other rows use the original
two-sample runs.

The original cold numeric run contained a 10.257 ms outlier. Subsequent
follow-ups varied in direction, so the final ABBA comparison above was
used to reduce run-order bias. It did not reproduce the slowdown:
combined means differ by -0.25%, and medians by +0.44%. The original
cold synthetic slowdown also did not reproduce in its longer rerun.

These measurements show no consistent performance regression, but do not
establish statistical significance or exact performance equivalence.

All benchmark result and cache checks passed. Cold/hot refers to the OS
file cache, not GPU or storage-controller caches.

GPU: GH200. CUDA: 13.1.
Vortex block rows: 8,388,608.
I/O: native. GPU allocator: cuda_async.
Lineitem SF10: 59,986,052 rows.

@0ax1 0ax1 added the changelog/chore A trivial change label Oct 27, 2026
@0ax1

0ax1 commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

@claude review this pr in depth

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @0ax1's task in 35s —— View job


PR review: refactor(cuda): keep inflight pinned buffers unwrapped

  • Read the PR diff and the surrounding code in vortex-cuda/src/pinned.rs
  • Check that the refactor keeps the same behaviour (completion fencing, reclaim, drop, failure paths)
  • Look for leftover references and unused code
  • Post the review

Verdict

I found no issues. The PR is ready to merge. This is a mechanical cleanup and nothing it touches behaves differently afterwards.

Why the behaviour is unchanged

  • No new memory-safety risk: after perf(cuda): reuse pinned buffer completion events #10713, InflightPinnedBuffer held only buffer: PinnedByteBuffer. Storing Vec<PinnedByteBuffer> directly keeps the same ownership, drop behaviour and memory layout.
  • Completion check: reclaim_completed (pinned.rs:257) still checks PinnedByteBuffer::event. That event is recorded on the transfer stream in copy_to_device (:393, and through the SyncOnDrop guard at :119). The fence is the same one as before.
  • Drop path: if a pool is dropped while transfers are still running, each PinnedByteBuffer in inflight still runs its own Drop (:138-147). That Drop synchronizes on the event before calling free_host. The wrapper had no Drop of its own, so dropping it was the same as dropping its field.
  • Failure paths: the error and leak handling in retain_after_transfer (:402-414) is untouched. Only the successful push at :417 changed.
  • Leftover references: InflightPinnedBuffer no longer appears anywhere in the workspace. The test change (:552) is the matching field-access update and still checks the same allocation pointer.
  • New doc comment: the comment on inflight (:158) correctly says why the buffers are held and when they are freed.

Optional follow-ups (outside this PR's scope)

These are not blockers. Both are older than this PR, and this refactor makes them easier to spot.

  1. Simpler reclaim loop. The manual idx / swap_remove loop at :255-263 could use Vec::extract_if, which is stable since Rust 1.87 (MSRV is 1.95):
    for completed in inflight.extract_if(.., |b| b.event.is_complete()) {
        self.put(completed);
    }
    This would also keep the order of inflight, which swap_remove does not. The order doesn't matter today.
  2. Shorter lock hold. reclaim_completed calls put while still holding the inflight lock. put then takes buckets, and if the bucket is full it can call cuMemFreeHost (through drop(overflow)) with inflight still held. You could collect the completed buffers, release inflight, then call put. That keeps cuMemFreeHost and the buckets lock outside the inflight lock, as the comment at :279 already intends for buckets. It isn't a deadlock: nothing takes buckets and then inflight.

Checks

I did not compile or run any tests, per the repo's guidance that verification is up to the user. This review is based on reading the code. CI covers cargo clippy -p vortex-cuda and the tests.

@codspeed

codspeed Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

1 warning

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

✅ 1883 untouched benchmarks
⏩ 468 skipped benchmarks1
🗄️ 47 archived benchmarks run2


Comparing ad/cuda-pinned-inflight-unwrap (52dbd35) with develop (952c43b)

Open in CodSpeed

Footnotes

  1. 468 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. 47 benchmarks were run, but are now archived. If they were deleted in another branch, consider rebasing to remove them from the report. Instead if they were added back, click here to restore them. ↩

`InflightPinnedBuffer` held its completion event next to the buffer
until #10713 reused the buffer's own event, leaving a wrapper around a
single field. Store the buffers directly.

Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
@0ax1
0ax1 force-pushed the ad/cuda-pinned-inflight-unwrap branch from 7180163 to 52dbd35 Compare November 3, 2026 15:22
@0ax1
0ax1 requested review from mprammer and robert3005 November 3, 2026 15:25
@0ax1
0ax1 marked this pull request as ready for review November 3, 2026 15:26
@0ax1
0ax1 enabled auto-merge (squash) November 3, 2026 15:27
@0ax1
0ax1 disabled auto-merge November 3, 2026 15:27
@0ax1
0ax1 enabled auto-merge (squash) November 3, 2026 15:27
@0ax1
0ax1 merged commit 1214bcf into develop Nov 3, 2026
128 of 129 checks passed
@0ax1
0ax1 deleted the ad/cuda-pinned-inflight-unwrap branch November 3, 2026 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants