Skip to content

Keep only fastlanes.delta in the unstable edition - #11595

Merged
joseph-isaacs merged 4 commits into
developfrom
ji/unstable-delta-only
Nov 18, 2026
Merged

joseph-isaacs merged 4 commits into
developfrom
ji/unstable-delta-only

Conversation

@joseph-isaacs

@joseph-isaacs joseph-isaacs commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The unstable edition now contains only fastlanes.delta. Every other member leaves it and belongs to no edition, so a writer that enforces editions no longer writes them. An audit found the other members not ready:

  • fastlanes.bitpacked.v2 and fastlanes.for.v2: per-block bit widths and references have no slice, filter or take kernels, so those operations decode the whole array.
  • vortex.patched: gated by VORTEX_EXPERIMENTAL_PATCHED_ARRAY rather than by the edition.
  • vortex.union: nothing produces it.
  • vortex.decimal_byte_parts.v2: between and aggregates fall back to canonical decoding when lower parts exist.
  • vortex.list layout: the V2 scan cannot read it.
  • vortex.bloom_filter.sbbf: never used for pruning.
  • vortex.sum_v2: partial invariants are not validated on read.

Changes

  • The declaration in vortex-edition/src/declarations/unstable/mod.rs now lists only fastlanes.delta.
  • vortex/editions/unstable/unstable2026.10.0.toml is regenerated with cargo run -p xtask -- generate-editions.
  • Two tests hardcoded the old members:
    • enabling_unstable_permits_only_persisted_aggregates no longer lists the aggregates unstable adds. It keeps its check that no vortex.all_* aggregate is enabled.
    • The vortex-file decimal edition test no longer expects UNSTABLE to enable decimal_byte_parts.v2.

The fuzzers enable UNSTABLE, so they stop writing the components that left it.

Benchmark changes that enable unstable in every benchmark are stacked on this PR in #11534.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UeniBDnhofVptQwrj9sj7o

bitpacked.v2, for.v2, decimal_byte_parts.v2, patched, union, the list
layout, the SBBF bloom filter and sum_v2 leave the unstable edition and
belong to no edition, so a writer that enforces editions no longer writes
them. An audit found them not ready: blocked bitpacked.v2 and for.v2 have
no slice, filter or take kernels, patched is gated by an environment
variable, nothing produces union, the V2 scan cannot read the list
layout, and the bloom filter is never used for pruning.

The edition record is regenerated with
`cargo run -p xtask -- generate-editions`.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBDnhofVptQwrj9sj7o
joseph-isaacs and others added 3 commits November 18, 2026 14:30
The unstable edition never freezes, so a test listing its members fails on
every membership change. Keep only the invariant that enabling it never
enables the vortex.all_* aggregates.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBDnhofVptQwrj9sj7o
Drop the added doc comments and the btrblocks comment change, leaving the
one-line declaration change, the regenerated edition record, and the two
tests that hardcoded the old members.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBDnhofVptQwrj9sj7o
The test hardcoded whether UNSTABLE permits decimal_byte_parts.v2, so it
broke whenever the unstable edition changed. It now checks whether the
session's enabled editions permit the v2 ID.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBDnhofVptQwrj9sj7o
@joseph-isaacs
joseph-isaacs merged commit 5dd206d into develop Nov 18, 2026
103 of 104 checks passed
@joseph-isaacs
joseph-isaacs deleted the ji/unstable-delta-only branch November 18, 2026 14:47
robert3005 added a commit that referenced this pull request Nov 19, 2026
#11595 removed `vortex.decimal_byte_parts.v2` from the `unstable`
edition because `between` and aggregates decode to canonical form when
lower parts exist. That is a performance gap, not a wire problem, and
`unstable` has no compatibility guarantee, so the encoding goes back in
and the decompression kernels can follow. `vortex.widen` gets a wire ID
in #11597. Adding it to `unstable` lets writers that enforce editions
emit integer `Widen` arrays and decimal `Widen` arrays over encoded
children. Without it, those writes return an error.

---------

Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants