Skip to content

feat(array): extend BETWEEN execution across encodings - #10378

Closed
joseph-isaacs wants to merge 1 commit into
developfrom
ji/between-remaining-encodings
Closed

joseph-isaacs wants to merge 1 commit into
developfrom
ji/between-remaining-encodings

Conversation

@joseph-isaacs

@joseph-isaacs joseph-isaacs commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Range predicates on encoded arrays can fall back to materializing the entire input. Add encoding-aware BETWEEN paths so the predicate can run on compressed children, dictionary values, or sequence metadata.

  • Extension: delegate to storage when the input and both bounds have matching extension metadata; preserve nullability and Kleene null-bound semantics.
  • Dictionary and run-end: evaluate constant bounds once per dictionary entry or run value, then expand the boolean result through the existing codes or run ends.
  • Masked: push the predicate into the child and apply the original validity mask to the result.
  • Sequence: derive the matching index interval from monotone sequence metadata using i128 arithmetic, including descending steps and unsigned values above i64::MAX.
  • FoR: translate constant bounds into the encoded domain for a shared reference, splitting an interval when wrapping subtraction crosses the type boundary.
  • Patched, VarBin, DateTimeParts, FSST, and OnPair: reuse the existing comparison kernels through a shared adaptor. Both comparisons must be handled; otherwise the normal canonical fallback remains available.

Extracts the extension implementation, tests, and benchmark from #10339. The existing primitive, decimal, constant, ALP, sparse, bit-packed, and chunked paths remain in place. DecimalByteParts is already covered by merged #10152.

Validation: 154 targeted BETWEEN tests passed across the seven affected crates. All-target/all-feature Clippy with -D warnings, pinned-nightly Rust formatting, and patch whitespace checks passed. Regression cases cover strictness, nulls, row-bound fallback, dictionary codes, run offsets, signed and unsigned sequence limits, FoR wraparound, string bounds, patched values, and non-midnight timestamps.

The extension microbenchmark is included. The additional encoding paths have not yet been benchmarked; cross-platform CI is still running.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature stale This PR is stale and will be auto-closed soon

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant