Skip to content

Harden serde of unstable encodings and the list layout, and document wire-contract design - #11442

Draft
joseph-isaacs wants to merge 8 commits into
developfrom
ji/unstable-serde-hardening
Draft

joseph-isaacs wants to merge 8 commits into
developfrom
ji/unstable-serde-hardening

Conversation

@joseph-isaacs

@joseph-isaacs joseph-isaacs commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This applies the serialized-contract pattern from the Delta PR (#11441) to every other UNSTABLE component that has children, and writes the pattern down as general guidance. The two PRs are independent.

The pattern:

  1. Explicit positions. Every child's position follows from the metadata and dtype alone. Nothing is inferred from children.len() or located relative to the end of the child list.
  2. Trailing children. Readers require at least the declared children and ignore any after them. Each component documents the rule for extending it:
    • a later writer may append a child under the same ID only if decoding without it yields the same values (a statistic or an index);
    • a child or metadata field that changes the decoded values needs a new ID.
  3. Corrupt input. Bad input returns an error rather than panicking or allocating without bound. The checks run before children.get, because the in-memory ArrayChildren asserts.

These IDs are all in the UNSTABLE edition, so the contracts change in place under the same IDs. Frozen v1 IDs (fastlanes.bitpacked, fastlanes.for, vortex.decimal_byte_parts) and the vortex.list array are untouched.

Changes

Docs

  • New page docs/specs/versioning/wire-contracts.md ("Designing wire contracts") covers how to make a format forward compatible:
    • locating children;
    • appending children;
    • protobuf metadata, including always decoding it as a message and choosing defaults;
    • which changes need a new wire ID;
    • validating input;
    • the tests each contract needs.
  • The page is linked from the versioning index, its toctree, and the "New formats" section of design.md.
  • AGENTS.md gains one line pointing to the page for anyone adding or changing a serialized format.

Encodings

Component Wire change Hardening
fastlanes.bitpacked.v2 Child order changes from [patches…, validity?, block_offsets] to [block_offsets, patches…, validity?]; metadata gains has_validity (tag 4); serialize builds the child list explicitly integer dtype; has_validity needs a nullable dtype; unsigned block-offset ptype; offset + len overflow; packed buffer size bound; patch metadata bounds (count, indices/chunk-offset ptypes, chunk-offset count); host block offsets validated in release builds
fastlanes.for.v2 None integer dtype; offset + len overflow
vortex.decimal_byte_parts.v2 None (validity lives in the MSP) every metadata ptype checked before any child is read; MSP must be a signed int, lower parts unsigned
vortex.patched Values child must be non-nullable (all writers already do this) 0 buffers; primitive dtype; n_lanes a power of two ≤ 128 (0 used to divide by zero); offset < 1024; checked lane-offsets length; child dtypes and lengths in validate
vortex.union Metadata is decoded as a field-less protobuf message instead of being required to be empty, so a later field doesn't break old readers. The bytes are unchanged: zero bytes at-least child count before any child is read
vortex.list layout None (validity present exactly when the list is nullable) integer offsets ptype; list dtype checked first; offsets row count exactly row_count + 1, with overflow check

Each component gets tests for:

  • a round-trip, including sliced and nullable cases where relevant;
  • a trailing extra child being ignored;
  • negative cases for every new check.

bitpacked.v2 also gets a new golden file, goldenfiles/bitpacked_v2.metadata. Its bytes were computed by hand with every field set to a non-default value.

Patched's inline tests moved to patched/vtable/tests.rs.

Left for follow-up

These need a data scan, or kernel bounds checks, to detect:

  • corrupt patched lane-offset values;
  • non-monotonic list offsets;
  • out-of-range union type ids, which still panic on access;
  • underflow in Patches::new on corrupt indices.

Checks

None have been run yet, so this is a draft:

  • no build, clippy, tests or Sphinx build;
  • formatting was done with stable rustfmt, not the pinned nightly;
  • the hand-computed golden file has not been checked against check_metadata.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UeniBBYkMiCtjPPKVBhjRj

joseph-isaacs and others added 8 commits November 13, 2026 13:22
Accept trailing children for `fastlanes.for.v2` so a later writer can
append children that old readers may safely ignore, and document the
rule on `FoRV2Metadata`. Children that change the decoded values still
need a new encoding ID. Both v2 children are required, so no metadata
field is needed to locate them and the wire bytes are unchanged.

Reject corrupt input before fetching children:
- non-integer dtypes
- an `offset + len` that overflows usize, which would panic when
  counting the reference chunks

`fastlanes.for` (v1) is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBBYkMiCtjPPKVBhjRj
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
`vortex.union` already locates every child from the dtype alone: the
type IDs, then one child per variant, with outer nulls carried by the
type IDs. Document that wire contract on `Union` and accept trailing
children so a later writer can append children that old readers may
safely ignore. Children that change the decoded values still need a new
encoding ID.

Add tests for the round trip, an ignored trailing child, and each
deserialize guard: too few children, a non-union dtype, non-empty
metadata or buffers, and decoding against the wrong length or variant
dtypes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBBYkMiCtjPPKVBhjRj
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Readers previously rejected any non-empty metadata, which would stop a
later writer from adding a field that describes an appended child old
readers may ignore. Decode it as a field-less protobuf message instead,
so unknown fields are ignored and malformed bytes are still rejected.
The serialized form is unchanged: the message encodes to zero bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBBYkMiCtjPPKVBhjRj
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Accept trailing children under `vortex.decimal_byte_parts.v2` so a later
writer can append children that old readers may safely ignore, and
document the rule on `DecimalBytePartsV2Metadata`. Children that change
the decoded values still need a new encoding ID. Every child position
already follows from the metadata: the MSP carries validity, and
`lower_part_ptypes` declares one child per lower part.

Validate the MSP and every lower-part ptype before reading any child,
since the in-memory children panic on a dtype mismatch, and name the MSP
in the error for an out-of-range ptype.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBBYkMiCtjPPKVBhjRj
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Document the `vortex.list` layout wire contract on `ListLayoutMetadata`:
the children are `[elements, offsets]`, followed by a validity child
exactly when the list dtype is nullable, so each child's position is a
function of the dtype alone. Readers now require at least that many
children and ignore trailing ones, so a later writer can append children
that old readers may safely skip, such as an index. Children or metadata
that change the decoded values still need a new layout ID.

Reject corrupt input with errors:
- offsets ptypes that are not integers, such as f32
- offsets whose row count is not exactly the list row count plus one;
  the old check saturated and accepted an empty offsets child
- list row counts that overflow when adding the extra offset

The writer already emits exactly the contracted children, and keeps its
exact-count check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBBYkMiCtjPPKVBhjRj
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Document the `vortex.patched` wire contract on `PatchedMetadata` and accept
trailing children so a later writer can append children that old readers may
safely ignore. Children that change the decoded values still need a new
encoding ID. Patched has no optional children, so the metadata fields and their
encoding are unchanged.

Read the patch values child with the non-nullable outer dtype. Patching ignores
the validity of the patch values, and every writer already stores them as
non-nullable, so a nullable values child could only drop nulls silently.

Reject corrupt input before reading children:
- any buffers, or fewer than four children
- non-primitive dtypes
- lane counts that are not a power of two between 1 and 128, which included
  zero lanes that panicked on `index % n_lanes` in `scalar_at`
- offsets outside the first chunk
- `offset + len` and the lane offsets length overflowing usize

`PatchedData::validate` now also checks the dtypes of the lane offsets, patch
indices and patch values and the length of the lane offsets. Serialize rejects
arrays whose lane offsets length a reader could not derive from the metadata.

Move the vtable tests into `vtable/tests.rs` and add a nullable sliced
round-trip and negative tests for each guard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBBYkMiCtjPPKVBhjRj
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Record whether a validity child is present in `BitPackedV2Metadata`
instead of inferring it from the child count, and write the block offsets
first so no child is located relative to the end. The children are now the
block offsets, then the patch indices, values and optional chunk offsets
when the metadata has patches, then the validity when `has_validity` is
set. Readers accept trailing children so a later writer can append
children that old readers may safely ignore. Children that change the
decoded values still need a new encoding ID. `fastlanes.bitpacked.v2` is
only in the unstable edition, so the contract changes in place;
`fastlanes.bitpacked` is unchanged.

Reject corrupt input before reading any child, so it can neither panic on
the in-memory child assertions nor force large allocations:
- non-integer dtypes, which previously panicked in `as_ptype`
- a validity child declared for a non-nullable dtype
- non-unsigned block offset types
- `offset + len` overflowing, and packed buffers that are not whole
  128-byte rows or exceed the blocks at the type's full bit width
- patch counts outside `1..=len`, signed chunk offset types, and chunk
  offset counts outside `1..=len.div_ceil(1024) + 1`

Host-resident block offsets are also validated after they are read, in
all builds, where debug builds previously panicked on invalid boundaries.

Pin the metadata encoding in a new `bitpacked_v2.metadata` golden file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBBYkMiCtjPPKVBhjRj
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Add a versioning page that sets out the rules the serde hardening applies:
child positions follow from metadata and dtype, readers accept trailing
children that are safe to ignore, metadata is always decoded as a protobuf
message, value-changing extensions take a new wire ID, and readers reject
malformed input with errors. Link it from the versioning index, the design
page's section on new formats, and AGENTS.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBBYkMiCtjPPKVBhjRj
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs changed the title Harden serde of unstable encodings and the list layout ahead of stabilization Harden serde of unstable encodings and the list layout, and document wire-contract design Nov 13, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant