Repository navigation
Conversation
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
|
||
| ### `ASOF` Joins | ||
|
|
||
| DataFusion 56 adds `ASOF JOIN`, a long-standing feature request ([#318]) |
| [Variant shredding]: https://parquet.apache.org/docs/file-format/types/variantshredding/ | ||
| [datafusion-variant roadmap]: https://github.com/datafusion-contrib/datafusion-variant/issues/69 | ||
|
|
||
| ### Geospatial Data |
There was a problem hiding this comment.
FYI @Xuanwo I was hoping to feature geodatafusion for this blog post -- do you think that is a good idea / would the datafusion 56 integration be ready relatively shortly after datafusion 56 is released?
| projects in the DataFusion ecosystem that are made possible by the | ||
| extension points in DataFusion 56. | ||
|
|
||
| ### Semi-Structured Data (e.g. JSON) |
There was a problem hiding this comment.
FYI @sdf-jkl I am hoping to higlight Variant here
|
FYI You are mentioned in the post, let me know if you would like any changes or see any omissions @2010YOUY01 @AdamGS @adriangb @andygrove @brancz @cblp @cetra3 @comphead @cyberbeam524 @Dandandan @friendlymatthew @gadomski @geoffreyclaude @getChan @haohuaijin @ianthetechie @jay-dee7 @jayzhan211 @jonathanc-n @kosiew @Kristofferjoha @kumarUjjawal @kylebarron @lyne7-sc @M-Tesla @martin-g @matthewmturner @mbutrovich @neilconway @Omega359 @osipovartem @paleolimbot @pepijnve @peterxcli @QinXi-ai @raghuvanshraj @Rich-T-kid @sdf-jkl @sharkinsspatial @SubhamSinghal @Surajsuthar @TTyChud @viirya @Xuanwo @xudong963 @yashrb24 @yiaany @zhuqi-lucas |
| implement `OFFSET` pushdown, which is useful for paging through results | ||
| ([#25404]). Thanks to [@SubhamSinghal], [@peterxcli], and [@martin-g]. | ||
|
|
||
| ### Faster Joins |
There was a problem hiding this comment.
For JOINs we also added apache/datafusion#26249 which improved high partitioned hash joins by ~2x in TPC-DS benches.
|
Shall we also mention PWMJ related improvements apache/datafusion#23870, apache/datafusion#25467, apache/datafusion#25840, apache/datafusion#26287 etc. |
See rendered preview here: https://alamb.github.io/datafusion-site/blog/2026/11/20/datafusion-56.0.0/
56.0.0release datafusion#2720156.0.0(Oct/Nov 2026) datafusion#24461