Repository navigation
Require canonical storage width for decimal arrays - #11571
Conversation
b9342a4 to
a0a4cb3
Compare
a0a4cb3 to
9363839
Compare
There was a problem hiding this comment.
This change means that this will no longer build 2 versions of each of the decimal fixtures.
9363839 to
65d506c
Compare
Merging this PR will regress 1 benchmark2 warnings
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
65d506c to
6101c3f
Compare
ac9430c to
87b5498
Compare
| let fill = fill | ||
| .cast::<T>() | ||
| .vortex_expect("fill scalar fits the array's decimal precision"); | ||
| let mut buffer = array.buffer::<T>().into_mut(); | ||
| for index in is_invalid.set_indices() { | ||
| buffer[index] = fill; | ||
| } |
There was a problem hiding this comment.
can you pull this into a func
2809bf3 to
6a04391
Compare
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
6a04391 to
a5fe0e8
Compare
The primitive, VarBinView, and decimal `fill_null` kernels each write one fill value at the set positions of a validity bitmap with their own loop, and a review on #11571 asked for the decimal loop to move into a function. The loop does not depend on the element type, so `BufferMut::fill_where` provides it for any `Copy` element, and the primitive and VarBinView kernels use it here. The decimal kernel switches to it after #11571 lands. `fill_where` fills each run of set bits with one `slice::fill`, which is the loop the VarBinView kernel already used and `varbinview_fill_null` measures. The primitive kernel used to write one index at a time, and no benchmark calls it directly. Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
A decimal dtype can currently have six canonical representations because a caller can choose the width of its values buffer. The precision now determines that width. Typed constructors report a mismatch, and builders and iterator constructors convert their input at construction. A narrow physical representation uses a
Widenover a narrower decimal child.DecimalBuilderconstructors lose their width parameter. The decimal reader still accepts older storage widths, including narrow nodes emitted by the current writer.