Issues

• Adopt DataFusion MemoryDriftTracker after upgrading DataFusion

  apache/datafusion-ballista#2531 · Filter by author andygrove opened on Oct 3, 2026

• Support rolling upgrades without an outage window

  enhancement
  apache/datafusion-ballista#2525 · Filter by author andygrove opened on Oct 2, 2026
  1

• Sort-shuffle writer compacts view columns even when they're already dense

  apache/datafusion-ballista#2521 · Filter by author NoahKusaba opened on Oct 1, 2026

• Performance review: shuffle, executor, scheduler and client findings (tracking)

  apache/datafusion-ballista#2520 · Filter by author NoahKusaba opened on Oct 1, 2026

• Coalesce small batches in the passthrough shuffle writer

  apache/datafusion-ballista#2517 · Filter by author NoahKusaba opened on Oct 1, 2026

• perf: shuffle writers use one blocking thread per output partition and compress outside the vcore budget

  apache/datafusion-ballista#2509 · Filter by author NoahKusaba opened on Sep 29, 2026
  1

  

• Discussion: move the expensive CI jobs to a merge queue, like Comet

  development-process
  apache/datafusion-ballista#2500 · Filter by author andygrove opened on Sep 27, 2026

• TPC-H SF1000 results for Ballista main (DataFusion 55.1.0): 0.72x of Spark 3.5, 1.45x of Spark + Comet, 15% of time in planning (#2497)

  performance
  apache/datafusion-ballista#2499 · Filter by author andygrove opened on Sep 27, 2026
  1

• Proposal: Decoupled Result Service for Ballista

  apache/datafusion-ballista#2484 · Filter by author villebro opened on Sep 23, 2026
  1

• AQE replan can swap a broadcast exchange onto the probe side of a CollectLeft join, pinning the join stage to one task

  apache/datafusion-ballista#2454 · Filter by author NoahKusaba opened on Sep 14, 2026
  1

  

• Design note: composing multi-partition tasks with plan-level AQE in DataFusion

  apache/datafusion-ballista#2445 · Filter by author NoahKusaba opened on Sep 12, 2026

• Emit the final (post-AQE) physical plan to the event log

  apache/datafusion-ballista#2422 · Filter by author andygrove opened on Sep 3, 2026

• CI: Python client tests do not run on Rust changes and do not test the working tree

  enhancement
  apache/datafusion-ballista#2372 · Filter by author andygrove opened on Aug 25, 2026

• Python client silently forces the static planner (pinned to ballista 54.0.0 defaults)

  bug
  apache/datafusion-ballista#2371 · Filter by author andygrove opened on Aug 25, 2026
  4

• Ballista 55.0.0 Release

  enhancement
  apache/datafusion-ballista#2369 · Filter by author andygrove opened on Aug 25, 2026
  2

• TPCH benchmark fails with Failed to open partition file at "/var/folders/..../data.arrow"

  bug
  apache/datafusion-ballista#2367 · Filter by author martin-g opened on Aug 25, 2026
  3

  

• Graceful executor shutdown terminates active Flight streams

  apache/datafusion-ballista#2357 · Filter by author DanilSerd opened on Aug 23, 2026

• [EPIC] Add public APIs required for scheduler high availability

  apache/datafusion-ballista#2349 · Filter by author phillipleblanc opened on Aug 20, 2026
  4

• Document HA scheduler

  documentationenhancement
  apache/datafusion-ballista#2347 · Filter by author milenkovicm opened on Aug 19, 2026
  2

• Allow a stage to start on partially complete inputs instead of waiting for the whole upstream stage

  enhancementperformance
  apache/datafusion-ballista#2320 · Filter by author andygrove opened on Aug 16, 2026

• Shuffle affinity: prototype a task distribution policy that places consumer tasks where their input already lives

  enhancementperformance
  apache/datafusion-ballista#2319 · Filter by author andygrove opened on Aug 16, 2026
  1

  

• Buffer shuffle output in memory and spill to disk only when a configured budget is exceeded

  enhancementperformance
  apache/datafusion-ballista#2318 · Filter by author andygrove opened on Aug 16, 2026

• Add PartialReduce operator / optimization

  enhancement
  apache/datafusion-ballista#2314 · Filter by author Dandandan opened on Aug 16, 2026

• Create a new roadmap for remainder of 2026

  enhancement
  apache/datafusion-ballista#2299 · Filter by author andygrove opened on Aug 14, 2026
  18

• [DISCUSSION] Restore Flight SQL support as a pluggable query frontend

  apache/datafusion-ballista#2298 · Filter by author andygrove opened on Aug 14, 2026
  2