Skip to content

Bind a request once per (expression, dtype) in each session - #11701

Draft
gatesn wants to merge 2 commits into
ngates/v2-plan-file-sourcesfrom
ngates/v2-plan-bind-cache
Draft

gatesn wants to merge 2 commits into
ngates/v2-plan-file-sourcesfrom
ngates/v2-plan-bind-cache

Conversation

@gatesn

@gatesn gatesn commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Scan requests are unbound, and each source binds them against its own dtype in scan(). An engine that opens one source per file would therefore bind and optimize the same projection and filter once per file. A small cache on the session binds each (expression, dtype) pair once and serves every source scanned in that session.

Contracts to check:

  • Key. An entry is keyed by the request expression and the dtype, compared by equality. Children and fields sit behind Arcs whose equality compares pointers first, so a lookup is mostly a few pointer checks.
  • Dynamic expressions. They compare by identity, so an entry is never reused for another query's dynamic value.
  • Bounds. The cache keeps 32 entries, least recently used first out. A miss costs only the bind it would have saved.

Stacked on #11700.

🤖 Generated with Claude Code

Requests stay unbound, as vortex#11654 settled: each source binds them
against its own dtype in scan(). An engine that opens a plan source per
file, as DuckDB does, therefore bound and optimized the same projection
and filter once per file, where the V2 DuckDB path bound them once per
query. A small per-session cache keyed by expression and dtype restores
binding once per (expression, dtype).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn gatesn added the changelog/performance A performance improvement label Nov 20, 2026
Signed-off-by: Nicholas Gates <nick@nickgates.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/performance A performance improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant