Repository navigation
Validate List and ListView arrays with an explicit execution context - #10594
Conversation
7281c06 to
8105637
Compare
8105637 to
cbb2829
Compare
The List and ListView constructors validated their offsets and sizes, and `offset_at`, `size_at`, `list_elements_at` and `sliced_elements` read them, with the hidden global `legacy_session()`. Take an `ExecutionCtx` instead, as do Map `entries_at` and `entry_count_at`, and validate during deserialization with a context from the session. The List builder, the Arrow and DuckDB list imports and the spatial storage builders build valid lists by construction and use the unchecked constructors. Plugin-aware Arrow imports now check that an importer preserves the length of the array it imports, since list, list view and map imports reuse Arrow's offsets into the imported child without checking them again. Copying a CUDA list view to the host keeps the checked constructor, because device-resident offsets and sizes skip range validation when the list is built. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y1zHpE1K1Y9HvyU4kt24Tq
931c193 to
5389bac
Compare
Merging this PR will regress 4 benchmarks
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | bench_compare_sliced_dict_primitive[(2000, 10000)] |
13.9 µs | 68 µs | -79.57% |
| ❌ | Simulation | density_sweep_dense_runs[0.001] |
44.2 µs | 62.7 µs | -29.41% |
| ❌ |
Simulation | test_repeated_scan_scalar_at[int] |
244.2 µs | 279 µs | -12.48% |
| ❌ | WallTime | words_gather_dispatch_neon[65536] |
1.8 µs | 2 µs | -10.01% |
| ⚡ | Simulation | column_x_constant |
580.4 µs | 153.3 µs | ×3.8 |
| ⚡ | Simulation | column_x_column |
169.2 µs | 107.9 µs | +56.92% |
| ⚡ | Simulation | nullable_columns |
183.6 µs | 124.5 µs | +47.5% |
| ⚡ |
Simulation | random_i128[0.01] |
5.9 µs | 4.4 µs | +34.79% |
| ⚡ | WallTime | bitpack_blocked_compress_avx2 |
7.6 µs | 6.7 µs | +13.11% |
| ⚡ | Simulation | eight_points |
639.9 µs | 573.6 µs | +11.55% |
| ⚡ |
Simulation | test_scan_filter[int] |
3.4 ms | 3.1 ms | +11.51% |
| ⚡ | Simulation | sixty_four_points |
711 µs | 639.8 µs | +11.12% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing rk/legacy-session-list (68b6865) with develop (7216cb5)
Footnotes
-
12 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
The C++ field accessor no longer takes a session (#10611). Same change as #10638. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y1zHpE1K1Y9HvyU4kt24Tq
A plugin that declines an import hands back an array that the canonical conversion then imports. Check that it kept the original length too, since list, list view and map imports reuse the Arrow offsets into it. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y1zHpE1K1Y9HvyU4kt24Tq
## Summary Develop doesn't compile vortex-file's tests. #10533 added `nested_empty_list_roundtrip`, which calls `ListArray::try_new` with three arguments, but #10594 had already added a fourth, the execution context. Both passed CI on their own. Since #10533 merged, the Rust build, lint and test jobs fail on develop and on every PR. ## Changes - `nested_empty_list_roundtrip` creates an execution context and passes it to both `ListArray::try_new` calls, as `nested_list_of_list_roundtrip` just above it already does. ## Testing - `cargo nextest run -p vortex-file -E 'test(/nested_empty_list_roundtrip|nested_list_of_list_roundtrip/)'` (5 passed) - `cargo clippy -p vortex-file --all-targets --all-features -- -D warnings` - `cargo +nightly-2026-09-10 fmt -p vortex-file --check` Signed-off-by: Matt Katz <mhkatz97@gmail.com>
ListArray::{new, try_new}andListViewArray::{new, try_new}take anExecutionCtx. They validate offsets and sizes (sorted and in bounds) with it, instead of withlegacy_session().offset_at,size_at,list_elements_at,sliced_elementsentries_at,entry_count_atSAFETYcomment:ArrowImportVTablereturns as many elements as it was given. The list, list-view and map imports reuse Arrow's offsets into the imported child, so a plugin that changed the child's length would otherwise produce an invalid list.ListViewArray::try_new, now with an explicit context. Device-resident offsets and sizes skip range validation when the list is built, so this is where invalid ranges are caught.part of #9446