Repository navigation
refactor(cuda): avoid access tracking for address-only lookups - #10884
Conversation
|
@claude review this pr in depth |
|
Claude finished @0ax1's task in 1m 17s —— View job Review of PR #10884:
|
| Site | Allocation | Stream used | Cross-stream write possible? |
|---|---|---|---|
fsst.rs:244, fsst.rs:338 |
fresh ctx.device_alloc |
ctx.stream() (= owner) |
No, no writes yet |
onpair.rs:498 |
fresh ctx.device_alloc |
ctx.stream() (= owner) |
No |
zstd.rs:131, zstd_buffers.rs:88 |
fresh ctx.device_alloc |
ctx.stream() (= owner) |
No |
dynamic_dispatch tests (643, 2313, 2351) |
stream.clone_htod |
same stream | No, the write is on the owner |
None of these sites could wait, so dropping the tracked read keeps ordering identical. The real writes are still tracked where they happen:
- FSST and OnPair pass the buffer through
.arg(&mut output)/.arg(&mut bytes)on the launch builder. - Zstd keeps
record_device_output = device_output.device_ptr_mut(stream)alive acrossdecompress_async(zstd.rs:269,zstd_buffers.rs:121).
The change also removes a small side effect. On the owner stream, after(Read) sets allocation_stream_pending = true. In the dispatch tests that flag had just been cleared by the clone_htod write. Leaving it set would make a later cross-stream access record a snapshot event that isn't needed. The new code avoids that, which is more accurate.
The new method is a reasonable addition to the API. CudaDeviceBuffer::offset_ptr / cuda_device_ptr() already hand out untracked addresses (e.g. zstd.rs:124, zstd.rs:155-158), so this adds the same thing for a bare CudaSlice. The rustdoc is clear about the caller's responsibility, and the intra-doc link to DevicePtr::device_ptr resolves because the trait is in the same module.
Suggestions (nits)
-
Explain why
device_buffer.rs:118keeps the tracked read. The PR body explains whyCudaDeviceBuffer::newstill callsdevice_ptr: it makes the allocation stream wait for writes on other streams. But there's no comment in the code, andlet (device_ptr, _) = cuda_slice.device_ptr(cuda_slice.stream())is now the only(x, _)discard left invortex-cuda. Someone tidying up later will probably switch it todevice_address()and silently lose that ordering. A one-line comment would prevent that. Fix this → -
The performance description is slightly off. At every changed site the stream is the owner and nothing has been written from another stream, so "may make the stream wait" never actually happened here. The saving per call is two
bind_to_threadcalls (cuCtxGetCurrent, pluscuCtxSetCurrentif the context isn't current) and twoparking_lotlock acquisitions, one each inbeforeandafter, not just "a lock". Each call happens once per decode, so the gain is real but small.changelog/performanceis fine, but the PR reads more naturally as a cleanup that also saves a little work. -
Optional: the comments in
zstd.rs:129-130andzstd_buffers.rs:86-87could be shorter. "Only the allocation address builds the pointer metadata" reads a little oddly. Since the method name now says what's going on, something like// The write is tracked by record_device_output around decompress_async.would be enough. This is purely cosmetic. -
Optional:
CudaViewhas no equivalent. That's fine since no caller needs one yet. If one comes up, anoffset-awaredevice_addresson the views would mirror this method.
Checks
I didn't compile or run anything; this review comes from reading the code. CUDA tests need a GPU runner, so CI will cover them.
Merging this PR will not alter performance1 warning
Comparing Footnotes
|
Several kernels and tests only needed an allocation's address, to check its alignment or to build pointer metadata, and got it from `device_ptr`, discarding the guard right away. Since #10722, every such call is a tracked read that locks the access state, binds the context and may make the stream wait, all for an access that never happens. Add `CudaSlice::device_address`, which only returns the address, and use it at those sites. `CudaDeviceBuffer::new` keeps its tracked read, whose wait orders the allocation stream after writes on other streams. Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
4cde242 to
19829c2
Compare
Summary
Adds CudaSlice::device_address for callers that only need an allocation's
address, without ordering or recording a device access.
Use it when building output-pointer metadata in FSST, OnPair, and Zstd,
and for address checks in dynamic-dispatch tests. Actual GPU reads and
writes remain tracked through launch arguments or device-pointer guards.
Changes
their access guards.
stream after pending writes on other streams. Document why its guard
can be dropped immediately.
Each replaced lookup avoids two CUDA context checks, two access-state
mutex acquisitions, and unnecessary read-tracking updates. These sites
did not require cross-stream waits, so this removes bookkeeping rather
than synchronization needed by the workload.
No builds, tests, or benchmarks were run for this change.