Repository navigation
Closed
Description
Context
A query's WindowImplementationCandidate list (PlanningQuery.window_implementations) is built from several sources and then rewritten in several places. This issue records how that works today and lists open questions, with the goal of simplifying the interface.
Where candidates come from
| # | Source | Code |
|---|---|---|
| S1 | Snapshot implementation.window_candidates: HashMap<PromQL, Vec<WindowImplementationCandidate>> (provider-priced, never rewritten) |
control_plane/src/physical/compiler.rs:226, :547, :621 |
| S2 | HTTP PhysicalPlanQueryRequest.window_implementations (compile-and-publish / cost-manifests), passed through as-is |
control_plane/src/main.rs:578, :959 |
| S3 | derived_window_candidates (one production caller: planning_request), used when S1 has no entry for the query |
control_plane/src/physical/compiler.rs:2435 |
S3 today, for each range-selector window W (or the lookback if the query has none), with evaluation interval E:
E < W && W % E == 0: Sliding, slide = E, two candidates:Pane{E}andFullWindow- otherwise: Tumbling, slide = W, one candidate:
Pane{W} - Cost:
derived_window_costscales the snapshot's singleimplementation_costtemplate.
Rewrites and filters
| # | What | Code |
|---|---|---|
| R1 | After logical selection, for S3 queries only: candidates at derived-maintenance windows are collapsed to one Tumbling Pane{W} (…-derived-cohort) and repriced |
compiler.rs:684-727 |
| R2 | Hybrid mode drops states with no candidate at their window; each state keeps only candidates at its own window | compiler.rs:1082, :1188 |
| R3 | Pane sharing makes shared-pane-<fp>, priced from window_implementations.first(), S3 queries only |
physical/pane_reuse.rs:80 |
Consumption: validate_window_implementations → ControlPlaneCostModel::with_window_implementation_costs → select_lifecycle → Planner plan_summary_maintenance_lifecycles → selected_window_implementation_id.
Questions
- Can S1 be deleted? No production caller sets
implementation.window_candidates, and no checked-in snapshot does either. Its only users are tests:compiler.rs(supplied_window_candidates_are_not_replaced_by_the_derivation, ~:5482), anddata_plane/tests/asapquery_compatibility_process_e2e.rs:789(run_shared_dashboard(multi_pane=true)), which injects a 5s pane with a hand-set cost to force the choice. Deleting it removeswindow_candidates, the key check at:547, thecontains_keybranches at:621/:673/:741, and makessynthesized_window_queriesalways cover every query. What should replace provider-priced pane quotes in those tests? Should the backend keep a way to accept measured per-pane costs, or is the derived formula the intended source of truth? - Should S2 derive candidates too? The HTTP path is the only entry for
DistributedCollectors(the snapshot path requiresBackendLocalRemoteWrite). Its only in-repo caller isdata_plane/tests/backend_process_e2e.rs. Should it keep caller-supplied candidates, or share S3? - Intermediate pane sizes. Proposal: also offer pane
pwithE < p < W,p % E == 0,W % p == 0. ButWindowMaterializationLayout::validate(crates/asap_types/src/aggregation_config.rs:50) requires the pane to divide both window and slide. With slide = E, a flatPane{p}withp > Eis rejected, and readouts at every E tick could not align with pane ends anyway. Options:- a)
HierarchicalRollup { base_pane_secs: E, levels_secs: [p] }. The compiler accepts it only asExtension("backend.exact-hierarchical-rollup.v1")(compiler.rs:2561), and the data plane has no implementation today. - b) Finer panes
p | gcd(E, W),p < E. These never beatPane{E}for one query, but could enable sharing across queries with different E. - c) Change the slide to
p. That changes evaluation alignment, so off-tick evaluations fall back to exact.
Which is intended?
- a)
- Tumbling fallback when
W % E != 0orE > W. Example: W = 300s, E = 45s producesTumbling Pane{300}, slide 300. Evaluations every 45s rarely line up with pane ends, so most reads should fail the serving pane-alignment check and go to exact. Should S3 usePane{gcd(E, W)}with slidegcd(E, W)instead? - R1 / R3 only apply to S3 queries. If S1 goes away, is the "synthesized" distinction still needed at all?
Metadata
Metadata
Assignees
Labels
No labels
Activity
milindsrivastava1997 commentedon Sep 13, 2026
pls check @zzylol