Repository navigation
Conversation
A slice over a run-end array had no reduce rule, so the optimizer kept lower(slice(runend)) and evaluated the function once per row. Slicing host-resident primitive ends now produces a run-end array directly, so the existing run-end scalar function rule pushes the function into the run values, and through a dictionary into its values. Co-authored-by: Amp <amp@ampcode.com> Signed-off-by: Alfonso Subiotto <alfonso.subiotto@dash0.com>
Contributor
|
FWIW I think the generic fix for this issue is #11310 |
Contributor
Author
|
Thanks! Our problem is that we would like |
Contributor
|
Right so I had the same issue where I have |
Contributor
Author
|
Yes, closing this and opening a different stack. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A reader that reads a row range of a chunk slices the array, and a scalar function such as
loweris often applied on top. For a run-end array the optimizer keptlower(slice(runend)), because run-end slicing exists only as an execute kernel that needs an execution context. The existing run-end scalar function rule never saw a run-end child, so the function ran once per row instead of once per run, or once per dictionary value when the run values are a dictionary.Changes
SliceReduceforRunEnd, registered withSliceReduceAdaptorin the same way asDict. When the ends are an unsigned primitive array in host memory, it finds the first and last kept run with a binary search, slices the ends and the values, and keeps the logical offset. In all other cases it returnsNone, and the existing execute kernel slices as before.lowerover a slice ofrunend(dict)now optimizes torunend(dict(lower(values)))and returns the expected values. The test fails without the change.This PR was written with the help of an AI coding agent. I reviewed the change and the tests.