Skip to content

perf(cuda-ffi): late string expansion for GPU reads - #11422

Merged
0ax1 merged 9 commits into
developfrom
ad/cudf-late-string-expansion
Nov 17, 2026
Merged

0ax1 merged 9 commits into
developfrom
ad/cudf-late-string-expansion

Conversation

@0ax1

@0ax1 0ax1 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Enables ~20% higher hot GPU read throughput in the companion BenchPress integration. Builds on #11421 (merged).

Adds an optional cuDF helper that combines dictionary batches before expanding strings, with a fast path for fixed-width string payloads. Supports differing dictionary orders, nulls, and integer code widths; adds standalone C++ tests. No dataset-specific assumptions, file-format changes, or cuDF dependency in the Rust FFI.

Performance

  • Combined integration: +20.3% / +21.1% hot throughput (Decimal / Float64).
  • Fixed-width string fast path alone: +8.7% / +12.7% over the preceding native implementation, measured separately—not additive.
  • Cold reads: 1.3% / 1.2% slower than decoded Vortex. File sizes unchanged.
Dataset | Reader           | Hot         | Hot/PQ | Cold        | Cold/PQ
--------+------------------+-------------+--------+-------------+--------
Decimal | Vortex baseline  | 156.68 GB/s |  2.51x |  33.32 GB/s |  1.21x
Decimal | Vortex optimized | 188.52 GB/s |  3.02x |  32.88 GB/s |  1.19x
Decimal | Parquet          |  62.39 GB/s |  1.00x |  27.56 GB/s |  1.00x
Float64 | Vortex baseline  | 154.77 GB/s |  2.53x |  33.10 GB/s |  1.25x
Float64 | Vortex optimized | 187.39 GB/s |  3.07x |  32.72 GB/s |  1.23x
Float64 | Parquet          |  61.10 GB/s |  1.00x |  26.57 GB/s |  1.00x

Baseline expands strings per batch before concatenation. Optimized uses late expansion plus companion BenchPress changes: one final synchronization and direct numeric/date copies. Those companion changes are not in this PR.

GB/s measures uncompressed logical output, not disk bandwidth. Both Vortex modes read the same files and produce fully materialized GPU tables inside the timer.

Measurement

  • GH200, libcudf 26.06.01; SF10 lineitem, ~60M rows / 8.205 GB logical data.
  • A-B-B-A visits, each with two warmups and 2 hot + 2 cold reads; four samples per Vortex mode/dataset/cache, two for Parquet. Reported throughput uses median elapsed time.
  • Hot: host-cached input, zero physical reads. Cold: full-file reads from RAM-backed null_blk capped at ~7 GB/s—not a real NVMe model.
  • All 40 final measured outputs matched CSV values/nulls outside timing.
  • Measured on Vortex 5bfa7ceec8 + BenchPress b4d5bcd, before restacking; benchmarks have not been rerun after the rebase or review changes.

@0ax1

0ax1 commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

@claude review

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @0ax1's task in 3m 4s —— View job


Review: late string expansion helper (vortex_cudf_dictionary.hpp)

  • Read the diff against origin/ad/cuda-device-batches
  • Review the header, the tests, CMake and the README
  • Post feedback

I found no correctness bugs. The header is careful: overflow is checked using host metadata only, empty batches are type-checked and then skipped, the null sentinel works together with NULLIFY, and the documented lifetime and stream rules are explicit. The points below are mostly about test coverage and cuDF conventions. I did not build or run anything: this runner has no GPU or libcudf, and checks are opt-in under CLAUDE.md.

1. The typed code-width test only exercises the fast path (test gap)

concatenate_string_dictionaries_test.cpp:68-75: every key in CodeWidths ("z", "a", "b") is 1 byte and non-null. So fixed_string_width returns 1, and every TypeParam goes through concatenate_fixed_width_strings. As a result, concatenate_dictionary_codes (the binary_operation ADD into INT32) is never run with uint8_t, uint16_t or uint32_t codes. The other tests reach the general path only with int8, int16, int32, int64 and uint64 codes.

That ADD with mixed operand and output types is the riskiest cuDF call in the general path, so it needs coverage. Add a variable-width or nullable variant to the typed suite, for example keys {"zz", "a", "zz"}, so that each code width goes through both paths.
Fix this →

2. Temporaries are allocated from the caller's mr

cuDF's convention is that mr is only for memory that is returned to the caller. Temporaries go to cudf::get_current_device_resource_ref(). This header allocates all of these from buffers.mr:

  • all_keys
  • the count_bytes lengths and the minmax scalars
  • the bases scalars and adjusted columns
  • all_codes and nonnull_codes
  • the zero / step sequence scalars

If a caller passes a limited or tracking resource meant for output only, it will see peak usage of roughly 2–3× the output. I suggest using the current device resource for intermediates and passing mr only to the final make_strings_column, gather and decode calls that build the result. Note that in the fast path values becomes the result's chars buffer, so it has to stay on mr.

3. fixed_string_width synchronizes the host twice

vortex_cudf_dictionary.hpp:152-155: each .value(stream) call is a device-to-host copy plus a sync. batch.has_nulls() and all_keys->has_nulls() are host-only here, which is fine. Another option is to compute min == max on the device, or copy both bounds in one cudaMemcpyAsync followed by a single sync. This happens once per column per read, so the saving is small, but this PR is about performance.

4. The fast path relies on chars_begin ignoring the view offset

concatenate_fixed_width_strings reads chars_begin(stream) from all_keys and steps through it by keys_size() * width. That is correct only because all_keys is a freshly concatenated column, so its offset is 0, its first string offset is 0, and its buffer is RMM-aligned for the UINT64 reinterpretation. A one-line comment there would stop a future refactor, such as reusing a sliced keys view for a single batch, from silently breaking it.

5. Minor and nits

  • Missing includes: add explicit includes for what the header uses directly: <rmm/cuda_stream_view.hpp>, <rmm/resource_ref.hpp>, <cudf/utilities/default_stream.hpp> and <cudf/utilities/memory_resource.hpp>. Today they only arrive transitively.
  • Gather policy: gather_dictionary_strings uses out_of_bounds_policy::NULLIFY even when all_codes has no nulls. DONT_CHECK skips the bounds check in that case, and valid codes are already part of the contract.
  • Tests not in CI: tests/CMakeLists.txt is opt-in and not wired into CI, so the header can drift from libcudf releases unnoticed. Consider a follow-up job on a GPU runner, or at least a note in the README saying the tests are run manually.
  • Formatting: in the test file there is a stray blank line between <string> and <utility>, and another inside OwnedOutputAndDefaultsAcrossTranslationUnits after strings single {...}. clang-format CI covers vortex-cuda, so it is worth running on the three new C++ files.
  • PR description: it says "Includes the fixed-width string fast path". Since the measured numbers predate the rebase, noting that the fast-path result in item 1 is the one carrying most of the 8.7%/12.7% gain helps reviewers focus. Optional.

Things I checked that look correct

  • Overflow preflight: the check size > limit - rows and the metadata(0, max) tests correctly make sure inflated views never reach the device.
  • Sliced batches: get_indices_annotated() handles parent offsets and validity, and the slice tests cover both paths.
  • Null codes: replacing them with the key_count sentinel plus NULLIFY is sound, and the INT32 sentinel type matches all_codes.
  • Fast-path guard: the requirements are no key nulls, no row nulls, a uniform width of 1, 2, 4 or 8, and rows * width fitting in INT32 offsets. That is enough for DONT_CHECK, given valid codes.
  • Ownership: temporaries live in buffers until return and are released stream-ordered, including when an exception is thrown. The lifetime contract for inputs is documented.

0ax1 added a commit that referenced this pull request Nov 13, 2026
…11421)

## Summary

First of two stacked drafts: expose dictionary-preserving GPU batches so
consumers can postpone string expansion until after combining batches.
This PR adds the export API only; the companion PR #11422 adds an
optional cuDF materialization helper and reports the ~20% end-to-end
hot-read gain.

A fixed-schema Arrow device stream cannot describe batches whose
dictionary encodings or index widths differ. The new iterator returns an
independently owned schema and device array for each batch, using the
existing `DictionaryExport::Preserve` policy.

## Changes

- Add `vx_cuda_scan_path_device_batches`, `vx_cuda_device_batches_next`,
and `vx_cuda_device_batches_free`.
- Reuse the existing scan options, ordered top-level projection, direct
I/O, and batch splitting.
- Keep the existing fixed-schema Arrow stream API and its decoded output
unchanged.
- Document ownership, stream synchronization, thread affinity,
end-of-stream, and terminal-error behavior; regenerate the C header.
- Add 10 regression tests for varying dictionary/index types, plain
batches, nulls, empty batches, projection, independent output lifetimes,
invalid inputs, and terminal errors.

---------

Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
Base automatically changed from ad/cuda-device-batches to develop November 13, 2026 11:03
@0ax1
0ax1 force-pushed the ad/cudf-late-string-expansion branch from 5084784 to 2abc5a7 Compare November 13, 2026 11:18
@0ax1
0ax1 force-pushed the ad/cudf-late-string-expansion branch from 71ed9e4 to 976b4c9 Compare November 16, 2026 16:19
@0ax1
0ax1 marked this pull request as ready for review November 16, 2026 16:33
@0ax1 0ax1 changed the title perf(cuda-ffi): late string expansion for ~20% faster hot GPU reads perf(cuda-ffi): late string expansion for GPU reads Nov 16, 2026
@0ax1 0ax1 added the changelog/performance A performance improvement label Nov 16, 2026
0ax1 added a commit that referenced this pull request Nov 17, 2026
## What changed

Remove the duplicate image definitions from `.github/runs-on.yml` so
Vortex uses the shared configuration.

The local definitions were hiding the shared GPU images. That caused
[the cuDF tests in
#11422](https://github.com/vortex-data/vortex/actions/runs/43908865241/job/133445095829)
to boot a stock image without Pixi, even though the custom image was
ready. Linux and Windows image selections stay the same.

## Testing

- Checked that the removed definitions match the shared ones; YAML and
whitespace checks passed.
- The custom x64 image already [passed all 29 cuDF integration
tests](https://github.com/vortex-data/vortex/actions/runs/43891010084/job/133381881332)
with an explicit AMI override.

After merge, rerun #11422’s cuDF job without the override to confirm it
picks the custom image. RunsOn reads this configuration from `develop`,
so we can only verify that part after merging.

Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
0ax1 and others added 8 commits November 17, 2026 11:37
Add an optional cuDF adapter with stream-ordered concatenation and a runtime-detected fixed-width payload fast path. Keep dictionary key order independent across batches and cover nullable, sliced, mixed-width, and empty inputs.

Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
The ubuntu24-gpu-x64 image behind runner=gpu now resolves to the cuDF GPU
AMI (vortex-data/.github-private#54), which bakes the vortex-cuda Pixi
environment into /opt/cudf. Drop the ami= override and the dispatch input
that existed only to try that image before the switch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
"CUDA tests (cudf)" already runs the cudf-test-harness integration
tests, so calling this job a cuDF integration test was ambiguous. It
builds and runs the GTest suite for vortex_cudf_dictionary.hpp, so name
the job, runner tag, step and build directory after that header.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
@0ax1
0ax1 force-pushed the ad/cudf-late-string-expansion branch from 2bc1008 to 4516ed7 Compare November 17, 2026 11:37
@codspeed

codspeed Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

1 warning

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

✅ 2012 untouched benchmarks
⏩ 6 skipped benchmarks1


Comparing ad/cudf-late-string-expansion (6c1f969) with develop (4c3c318)

Open in CodSpeed

Footnotes

  1. 6 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

0ax1 added a commit that referenced this pull request Nov 17, 2026
## What changed

Point the GPU compression benchmark at `ubuntu24-gpu-x64-pre`, the
distinct custom image alias introduced in
vortex-data/.github-private#55, instead of RunsOn’s stock image name.

Also replace the unverified inheritance explanation in
`.github/runs-on.yml` with a simple note about where the shared
definitions live. This does not change configuration behavior.

## Validation

YAML parsing, a cross-check against the proposed shared image alias, and
`git diff --check` passed. The benchmark’s other settings are unchanged.

Merge after vortex-data/.github-private#55 lands and #11422’s cuDF
integration job confirms that the shared GPU runner selects the custom
AMI. No AMI rebuild is needed.

Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
@0ax1
0ax1 merged commit 2931f45 into develop Nov 17, 2026
82 of 83 checks passed
@0ax1
0ax1 deleted the ad/cudf-late-string-expansion branch November 17, 2026 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/performance A performance improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants