Skip to content

Read array slots through generated accessors and drop duplicates - #11430

Merged
robert3005 merged 1 commit into
developfrom
claude/inspiring-ritchie-cwpkl4
Nov 19, 2026
Merged

robert3005 merged 1 commit into
developfrom
claude/inspiring-ritchie-cwpkl4

Conversation

@robert3005

@robert3005 robert3005 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #11418 and #11423. The #[array_slots] accessors now return references borrowed for the array, or for the ArrayView's lifetime 'a. So there's no longer a reason to index slots() by XSlots constants and unwrap by hand, or to keep hand-written accessors that duplicate the generated ones.

API Changes

  • Removed PrimitiveArrayExt::validity_child, DecimalArrayExt::validity_child, InterleaveArrayExt::array_indices, InterleaveArrayExt::row_indices and ChunkedArrayExt::chunk_offsets_array. The generated accessors replace them, so callers need the encoding's …ArraySlotsExt trait in scope. Use chunk_offsets() in place of chunk_offsets_array().
  • Newly public: InterleaveSlots, InterleaveSlotsView and InterleaveArraySlotsExt from arrays::interleave, and ChunkedArraySlotsExt from arrays::chunked.

@robert3005 robert3005 added the changelog/break A breaking API change label Nov 12, 2026 — with Claude
@codspeed

codspeed Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Merging this PR will improve performance by 11.24%

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 1 benchmark is dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚡ 2 improved benchmarks
✅ 1956 untouched benchmarks
⏩ 53 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ Walltime decode_blocked_bitpacked_chunked_neon[u32, 524288] 30 µs 26.9 µs +11.69%
⚡ Walltime starts_with_avx512[64, VarBinView] 128.6 µs 116.1 µs +10.8%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing claude/inspiring-ritchie-cwpkl4 (0b3030d) with develop (3dc8d93)

Open in CodSpeed

Footnotes

  1. 53 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@robert3005
robert3005 enabled auto-merge (squash) November 13, 2026 10:57
Now that the #[array_slots] accessors return references borrowed for the
array (or the ArrayView's lifetime 'a), there is no reason to index
slots() by XSlots constants and unwrap by hand, or to hand-write accessors
that duplicate the generated ones.

Replace direct slot indexing with the generated accessors:

- validity children read via slots()[XSlots::VALIDITY_CHILD].as_ref()
  across Bool, Primitive, Decimal, List, ListView, FixedSizeList, VarBin,
  VarBinView, Masked, ByteBool, OnPair, ParquetVariant, Pco and Zstd now
  call validity_child();
- required slots read via .clone().vortex_expect(..) (List, ListView,
  FixedSizeList, Sparse) and the BitPacked block offsets, Variant
  shredded child and Sparse patch chunk offsets now use their accessors;
- the Chunked and Interleave ext methods read chunks and values through
  the variadic SlotSlice accessors;
- Dict, DateTimeParts and Patched build their slot view with
  slots_view() instead of XSlotsView::from_slots(array.slots()).
- the Delta chunk decoder reads its deltas with deltas(), which now
  borrows for the view's lifetime, and the packed-deltas probe reads a
  Slice's child with child();
- Bool validity children read by position (slots()[0]) in canonical.rs
  and the Patched compare kernel, and Zstd's in the CUDA kernel and
  benchmark, now call validity_child().

Remove hand-written accessors identical to the generated ones:

- PrimitiveArrayExt::validity_child and DecimalArrayExt::validity_child;
- InterleaveArrayExt::array_indices and row_indices. InterleaveSlots is
  now pub, and InterleaveArraySlotsExt, InterleaveSlots and
  InterleaveSlotsView are re-exported from arrays::interleave, so the
  accessors stay public;
- ChunkedArrayExt::chunk_offsets_array, replaced by the generated
  chunk_offsets(). ChunkedArraySlotsExt is now re-exported from
  arrays::chunked.

ChunkedArrayExt::chunks is kept: it returns an owned Vec<ArrayRef>, unlike
the generated SlotSlice accessor of the same name, so ChunkedArrayExt calls
the generated accessor through its trait path. Callers of the removed
methods now need the encoding's ArraySlotsExt trait in scope.

Generic whole-slot rewrites that clone or map every slot, and validation
of raw slot slices before an array exists, are left as they are.

Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SWbErpV8gppbsHrfq7fJD
@robert3005
robert3005 force-pushed the claude/inspiring-ritchie-cwpkl4 branch from 775dcea to 0b3030d Compare November 16, 2026 15:08
@robert3005
robert3005 merged commit 1fcb628 into develop Nov 19, 2026
80 of 83 checks passed
@robert3005
robert3005 deleted the claude/inspiring-ritchie-cwpkl4 branch November 19, 2026 12:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/break A breaking API change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants