Repository navigation
Compare DecimalByteParts decode at the canonical width - #11510
Conversation
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Merging this PR will degrade performance by 66.73%2 warnings
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
| fn decode_at_canonical_width(array: ArrayRef, ctx: &mut ExecutionCtx) -> DecimalArray { | ||
| let decoded = array | ||
| .execute::<DecimalArray>(ctx) | ||
| .vortex_expect("decode DecimalByteParts"); | ||
| let decimal_dtype = decoded.decimal_dtype(); | ||
| let canonical_type = decimal_dtype.canonical_values_type(); | ||
| if decoded.values_type() == canonical_type { | ||
| return decoded; | ||
| } | ||
|
|
||
| let validity = decoded.validity().vortex_expect("decimal validity"); | ||
| let mask = validity | ||
| .execute_mask(decoded.len(), ctx) | ||
| .vortex_expect("execute decimal validity"); | ||
| let values = match_each_decimal_value_type!(decoded.values_type(), |T| { | ||
| copy_to_canonical_width(decoded.buffer::<T>().as_ref(), &mask, decimal_dtype) | ||
| .vortex_expect("copy decoded values to the canonical width") |
There was a problem hiding this comment.
who does this differ from execution::<Canonical> or CanonicalRec?
There was a problem hiding this comment.
Do we not have a single width per decimal precision now?
There was a problem hiding this comment.
Not yet because we need to get DecimalByteParts canonicalization into the correct width first, which requires this entire stack
There was a problem hiding this comment.
so is this change undone at the end of the stack?
There was a problem hiding this comment.
Yeah I will remove it in the next PR
Summary
decodeandfiltercurrently stop at whatever storage width DecimalByteParts returns. That makes a decoder that returns narrow storage look faster than one that returns the canonical decimal width, even though the results have different physical layouts.Changes
Both benchmarks now end at the canonical width, with a copy only when the decoded width differs. Names and cases stay the same so CodSpeed can compare the same workloads across the decoder change.
CodSpeed is expected to report slower
decodeandfilterresults on this PR because these benchmarks now include the widening copy that callers need. #11459 can then be compared against the same endpoint.