Skip to content

Push down list element casts and bool/int/uint casts to Float64 in DataFusion - #11161

Merged
mhk197 merged 3 commits into
developfrom
mk/datafusion-float-casts
Nov 3, 2026
Merged

mhk197 merged 3 commits into
developfrom
mk/datafusion-float-casts

Conversation

@mhk197

@mhk197 mhk197 commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The DataFusion converter now pushes down integer and boolean casts to Float64, plus list casts whose element cast is supported. These were left out of the cast allowlist from #9785, but Arrow and Vortex convert them the same way. array_sum needs this: DataFusion always plans it as array_sum(CAST(xs AS List<Float64>)).

Changes

  • Boolean and every integer type → Float64: both engines use as f64 (round to nearest, never fails; booleans become 1.0/0.0).
  • Lossless unsigned → wider signed casts, such as UInt32 → Int64.
  • List/LargeList casts are classified by their element cast. FixedSizeList → List and narrowing element nullability stay in DataFusion.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/performance A performance improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants