Repository navigation
Shrink compiled cast kernels and speed up validated casts - #11512
Conversation
Merging this PR will improve performance by 63.89%1 warning
Performance Changes
Tip Curious why performance improved? Comment Comparing Footnotes
|
Primitive-to-primitive, primitive-to-decimal, decimal-to-decimal and decimal-to-integer casts each compiled a fused, checked loop for every (source, target) type pair, with separate fallible, masked and rescale variants: about 1.7 MB of code across the two cast modules. Split every cast into validation and conversion: - Validation is a range check compiled once per source type, with the target's bounds passed at runtime. Only integer targets can reject a value. For rescaling casts, the target range is translated back into the source domain; exact division divides once and checks that the quotient multiplies back. - Conversion then cannot fail. Integer resizing only depends on the widths and the source signedness, so it is shared by primitive int-to-int casts, decimal storage and upcast_decimal_values. Conversions involving floats stay direct: routing them through an integer intermediate measured 50-60% slower. The new IntegerCast in primitive/compute/integer_cast.rs holds the shared integer machinery. Existing fast paths are kept: buffer reuse for same-width casts, WidenArray for same-scale decimal widening, and constant nulls for all-null input (now also for unchecked conversions of AllInvalid arrays). Cached min/max skip NaN, so they no longer prove that a float array fits an integer type unless the NaN count is known to be zero; such NaNs previously cast silently to 0. The cast code shrinks to ~267 KB. Validated casts get faster since the range check vectorizes and avoids per-value i256 arithmetic, e.g. u32->u8 12.4us -> 3.7us (8K values) and decimal->i64 1.6ms -> 35us (16K values). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018SWbEuaJSq7kXEfEUZcoiy Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
d3d61f5 to
6278a48
Compare
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
Benchmarks: PolarSignals Profiling 📖Commits: PR datafusion / vortex-file-compressed / ns (1.030x ➖, 0↑ 1↓)
No file size changes detected. |
Benchmarks: FineWeb NVMe 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.034x ➖, 0↑ 1↓)
datafusion / parquet / ns (1.002x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.008x ➖, 0↑ 1↓)
duckdb / parquet / ns (1.021x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-H SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.995x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.000x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.003x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.997x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-H SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.001x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.999x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.999x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.000x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-DS SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.007x ➖, 0↑ 1↓)
datafusion / parquet / ns (1.004x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.997x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.994x ➖, 1↑ 0↓)
No file size changes detected. |
Benchmarks: Clickbench on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.987x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.997x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.991x ➖, 1↑ 1↓)
duckdb / parquet / ns (0.988x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-H SF=1 on S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.933x ➖, 2↑ 2↓)
datafusion / parquet / ns (1.052x ➖, 0↑ 2↓)
duckdb / vortex-file-compressed / ns (1.060x ➖, 0↑ 2↓)
duckdb / parquet / ns (0.970x ➖, 1↑ 0↓)
|
Benchmarks: TPC-DS SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.004x ➖, 0↑ 1↓)
datafusion / parquet / ns (0.999x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.994x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.991x ➖, 0↑ 0↓)
No file size changes detected. |
Adapt the cast kernels to the decimal changes on develop: decimal arrays always use the canonical storage width, constructors check precision, and upcast_decimal_values is gone. Cast results are built with try_new_handle_from_trusted_values, since the cast has already checked every valid value against the target precision. Also from review: - An overflowing scale factor only accepts zero, matching the scalar cast, also for truncating division (unreachable for valid scales). - Keep dispatch macro bodies in integer_cast to a single generic call. - Record the new benchmark counts for CodSpeed. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018SWbEuaJSq7kXEfEUZcoiy Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
| /// A cast into decimals of `decimal_dtype`, stored as its canonical values type. | ||
| pub(crate) fn to_decimal( | ||
| from: IntegerType, | ||
| from_scale: i8, | ||
| decimal_dtype: DecimalDType, | ||
| ) -> Self { | ||
| let precision = usize::from(decimal_dtype.precision()); | ||
| Self { | ||
| min: i256::MIN_BY_PRECISION[precision], | ||
| max: i256::MAX_BY_PRECISION[precision], | ||
| ..Self::new( | ||
| from, | ||
| IntegerType::of_decimal(decimal_dtype.canonical_values_type()), | ||
| i16::from(decimal_dtype.scale()) - i16::from(from_scale), | ||
| Remainder::Reject, | ||
| ) | ||
| } | ||
| } |
There was a problem hiding this comment.
I think we need to be careful here, are we sure we want to be casting integers to decimals and back?
There was a problem hiding this comment.
Also the robots found an issue related to this:
Details
Why an integer conversion runs during a Decimal cast. A caller can cast an unsigned quantity column to a schema field declared Decimal(9, 0). The primitive cast kernel routes a DType::Decimal target to cast_to_decimal. That function uses IntegerCast because Decimal(9, 0) stores its values as unscaled i32s. For [1u32, 2], both values fit the nine-digit precision and have the same bytes as [1i32, 2]. Cast dispatch · Canonical Decimal storage
The concrete failing input. Start with a typed buffer so the pointer is definitely four-byte aligned. Then use a byte view with a one-byte recorded guarantee. PrimitiveArray::from_buffer_handle preserves that guarantee.
let bytes = buffer![1u32, 2]
.into_byte_buffer()
.aligned(Alignment::none());
let source = PrimitiveArray::from_buffer_handle(
BufferHandle::new_host(bytes),
PType::U32,
Validity::NonNullable,
);
let target = DType::Decimal(DecimalDType::new(9, 0), Nullability::NonNullable);
let result = source.into_array().cast(target)?.execute::<DecimalArray>(&mut ctx);Alignment::none() means the buffer promises one-byte alignment. Here .aligned(...) keeps the same pointer and changes the recorded guarantee. Buffer alignment implementation · Primitive handle storage
After the values pass the range check, the zero-scale cast reaches this branch. Both widths are four bytes, so the clone carries the one-byte guarantee forward.
let (from_width, to_width) = (from.byte_width(), to.byte_width());
if from_width == to_width {
return values.clone();
}The Decimal caller then wraps those bytes in BufferHandle::new_host(casted) and passes them to try_new_handle_from_trusted_values. Its layout check reads the recorded guarantee through BufferHandle::is_aligned_to:
vortex_ensure!(
buffer.is_aligned_to(Alignment::of::<D>()),
"decimal buffer alignment {:?} is invalid for values type {:?}",
buffer.alignment(),
D::DECIMAL_TYPE,
);For D = i32, the check compares one byte against the required four and returns an error even though the pointer is four-byte aligned. The base allocated a BufferMut<i32> for this unsigned input, so the output carried the required guarantee. The regression is in the metadata passed to the constructor. Re-establishing the target alignment on the reused buffer can keep this case zero-copy. Decimal layout check · Handle alignment check
There was a problem hiding this comment.
On the question: this PR doesn't add any casts. Integer→decimal (cast_to_decimal) and decimal→integer (cast_to_integer) were already supported before it; IntegerCast only replaces how they are implemented, and their results are checked against the scalar cast.
The alignment issue is real and fixed in ebfad2c on #11676, where this change now lives. A same-width cast returned a clone of the source buffer with the source's recorded alignment, so u32 → Decimal(9, 0) failed the decimal layout check when the buffer only recorded byte alignment. The reused buffer now records the target type's alignment, which stays zero-copy whenever the pointer is aligned. The repro from the report is a regression test (cast_integer_with_unrecorded_alignment_to_same_width_decimal); it failed before the fix.
Generated by Claude Code
| let mut values = resize(values, self.from, self.to).into_mut(); | ||
| match_each_integer_type!(self.to, |T| { | ||
| multiply_in_place::<T>(values_as_mut(&mut values), factor); | ||
| }); | ||
| values.freeze() |
There was a problem hiding this comment.
Apparently this is the cause of an actual regression
There was a problem hiding this comment.
This was the same-width multiply: it cloned the buffer, copied it, then multiplied in place, so two passes where the old kernel had one. A later commit on this branch added multiply_within, which checks the range and writes the products in one pass into a new buffer when the source and target widths match, so convert no longer takes that path for equal widths. It's included in #11676.
Generated by Claude Code
Scaling up into storage of the source's width resized by cloning the shared source buffer, so into_mut copied it, and then range-checked and multiplied in two more passes. Check the range and write the products into a new buffer in one pass, with one kernel per source type. Signed-off-by: Robert Kruszewski <github@robertk.io>
…de/clever-hawking-qreyad Signed-off-by: Robert Kruszewski <github@robertk.io> # Conflicts: # scripts/codspeed_bench_counts.json # vortex-array/benches/cast_decimal.rs # vortex-array/benches/cast_primitive.rs
|
I was restacking these prs and it merged it is benchmark branch 😢 |
Stacked on #11676
Primitive and decimal casts compiled a fused, checked loop for every (source, target) pair, about 1.7 MB of code. Each cast is now a range check compiled once per source type plus a conversion that cannot fail; start the review with
primitive/compute/integer_cast.rs. Float casts keep one loop per pair, because an integer intermediate was 50–60% slower.The cast code shrinks to about 267 KB, and u32→u8 validated casts go from 12.4 µs to 3.7 µs for 8K values.
Behaviour change: float-to-integer casts reject NaN even with cached min/max stats (they cast to
0before), out-of-range casts always reportvalues exceed target range, and all-null input to a converting cast returns a constant null array.🤖 Generated with Claude Code