You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[C++] Potential regression on FieldRef/FieldPath non-flattening Get methods #36892
@benibus Would you have time to take a look at this?
It would be nice to add benchmarks for the non-flattening methods. They will also help measuring the slowdown before/after that PR.
@benibus@westonpace I was planning on waiting a couple days to create the next RC (Tuesday?) to see if this was able to make it to 13.0.0. If this sounds totally unreasonable let me know and I can create the new RC sooner as discussed on Zulip we might want to ship 13.0.0 with this known issue and apply the fix targeting 14.0.0.
Should have a fix for this by the end of the week, provided my diagnosis is actually complete.
At the very least, the PR introduced substantial overhead for calling FieldPath::Get on record batches specifically. It seems this is due to a few instances where usages of RecordBatch::column_data() were replaced with columns(), which is far less trivial (it loads every column atomically on each invocation).
That being said, the two aren't exactly interchangeable in the implementation so there's some restructuring in order.
Activity
pitrou commentedon Jul 26, 2023
@benibus Would you have time to take a look at this?
It would be nice to add benchmarks for the non-flattening methods. They will also help measuring the slowdown before/after that PR.
raulcd commentedon Jul 28, 2023
@benibus @westonpace I was planning on waiting a couple days to create the next RC (Tuesday?) to see if this was able to make it to 13.0.0. If this sounds totally unreasonable let me know and I can create the new RC sooner as discussed on Zulip we might want to ship 13.0.0 with this known issue and apply the fix targeting 14.0.0.
raulcd commentedon Jul 31, 2023
@benibus any news here? I plan to create the new RC tomorrow unless there's possibility for this to be solved soon.
benibus commentedon Aug 1, 2023
Should have a fix for this by the end of the week, provided my diagnosis is actually complete.
At the very least, the PR introduced substantial overhead for calling
FieldPath::Geton record batches specifically. It seems this is due to a few instances where usages ofRecordBatch::column_data()were replaced withcolumns(), which is far less trivial (it loads every column atomically on each invocation).That being said, the two aren't exactly interchangeable in the implementation so there's some restructuring in order.
GH-36892: [C++] Fix performance regressions in `FieldPath::Get` (#37032)
GH-36892: [C++] Fix performance regressions in `FieldPath::Get` (#37032)
apacheGH-36892: [C++] Fix performance regressions in `FieldPath::Get` (…