Skip to content

cuda(perf): export struct fields concurrently - #11680

Merged
0ax1 merged 1 commit into
developfrom
ad/cuda-concurrent-export
Nov 20, 2026
Merged

0ax1 merged 1 commit into
developfrom
ad/cuda-concurrent-export

Conversation

@0ax1

@0ax1 0ax1 commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

Arrow export serializes struct fields: each child is decoded and exported before the next one starts. When a field triggers a device-to-host readback (string sizes, run-end totals, FSST symbol tables), the GPU idles while the host waits for the result, and no other field's work can fill that gap.

Each field now exports on its own CUDA stream via a forked execution context. The struct's stream waits for every field event before recording the final sync, so downstream consumers still see a single ordered export. Error and cancellation cleanup uses per-field PendingArrowChildren guards.

Measured end to end with BenchPress (TPC-H SF10 lineitem, GPU read through the CUDA FFI, hot page cache, 5 iterations, two runs each):

Before After
Read time 41.85 ms 38.85 ms
Throughput 196.0 GB/s 211.4 GB/s

Exporting a struct to Arrow decoded and exported its fields one after another
on one stream. String fields read small results back to the host, such as
the total length that sizes their output, and the GPU sat idle during every
round trip because no other field had work queued.

Export the fields concurrently, each on a context forked onto another stream
from the session's pool, so that other fields keep the GPU busy while one
waits for a readback. The struct's stream waits on an event recorded after
each field's export, so the struct's sync event still covers all fields, and
every exported field is released if a sibling fails or the export is
cancelled.

A GPU read of TPC-H SF10 lineitem through the CUDA FFI takes 38.9 ms instead
of 41.9 ms with a hot page cache.

Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
@0ax1 0ax1 added the changelog/performance A performance improvement label Nov 20, 2026
@0ax1
0ax1 enabled auto-merge (squash) November 20, 2026 10:44
@0ax1
0ax1 requested a review from robert3005 November 20, 2026 10:44
@0ax1
0ax1 disabled auto-merge November 20, 2026 10:56
@robert3005

Copy link
Copy Markdown
Contributor

So we didn't do it on a cpu as we think this is benchmaxxing. SF10 on an h200 is definitely very little data. I am happy to approve this on gpu since we have A LOT more cores but we should consider later if this is truly necessary

@0ax1
0ax1 merged commit 79e75c1 into develop Nov 20, 2026
105 of 106 checks passed
@0ax1
0ax1 deleted the ad/cuda-concurrent-export branch November 20, 2026 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/performance A performance improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants