Repository navigation
cuda(perf): export struct fields concurrently - #11680
Merged
Merged
Conversation
Exporting a struct to Arrow decoded and exported its fields one after another on one stream. String fields read small results back to the host, such as the total length that sizes their output, and the GPU sat idle during every round trip because no other field had work queued. Export the fields concurrently, each on a context forked onto another stream from the session's pool, so that other fields keep the GPU busy while one waits for a readback. The struct's stream waits on an event recorded after each field's export, so the struct's sync event still covers all fields, and every exported field is released if a sibling fails or the export is cancelled. A GPU read of TPC-H SF10 lineitem through the CUDA FFI takes 38.9 ms instead of 41.9 ms with a hot page cache. Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
0ax1
enabled auto-merge (squash)
November 20, 2026 10:44
0ax1
disabled auto-merge
November 20, 2026 10:56
Contributor
|
So we didn't do it on a cpu as we think this is benchmaxxing. SF10 on an h200 is definitely very little data. I am happy to approve this on gpu since we have A LOT more cores but we should consider later if this is truly necessary |
robert3005
approved these changes
Nov 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Arrow export serializes struct fields: each child is decoded and exported before the next one starts. When a field triggers a device-to-host readback (string sizes, run-end totals, FSST symbol tables), the GPU idles while the host waits for the result, and no other field's work can fill that gap.
Each field now exports on its own CUDA stream via a forked execution context. The struct's stream waits for every field event before recording the final sync, so downstream consumers still see a single ordered export. Error and cancellation cleanup uses per-field
PendingArrowChildrenguards.Measured end to end with BenchPress (TPC-H SF10
lineitem, GPU read through the CUDA FFI, hot page cache, 5 iterations, two runs each):