Repository navigation
[Refactor] Centralize planner and backend accuracy formulas #707
Description
What to build
Centralize the shared CMS, HLL, KLL, and DDSketch accuracy formulas in asap_types, with a small interface over normalized family parameters. The control plane and data plane should retain only their source-specific parameter extraction and any backend-only augmentation, then derive the same AccuracyProfile through the shared formulas.
This removes the current duplicated policy logic while preserving the physical-plan compile, publish, install, and serving behavior.
Acceptance criteria
- Each shared sketch-family formula has one authoritative implementation in
asap_types. - Control-plane planning and data-plane serving derive equivalent
AccuracyProfilevalues for identical normalized CMS, HLL, KLL, and DDSketch inputs. - Backend-specific behavior remains outside the shared formula module and is covered by focused tests.
- A cross-crate parity test covers representative parameters for every supported shared family.
- Existing physical-plan and query-serving tests continue to pass.
Blocked by
None - can start immediately
Metadata
Metadata
Assignees
Labels
No labels
Activity
milindsrivastava1997 commentedon Sep 13, 2026
pls check @zzylol if this makes sense
milindsrivastava1997 commentedon Sep 13, 2026
Implementation references:
control_plane/src/accuracy.rs:1-92owns planner derivation fromSketchParams; the module header at lines 3-8 explicitly requires lockstep with the backend. CMS, HLL, KLL, and DDSketch formulas are at lines 52-89.data_plane/src/storage_engines/sketch_db/accuracy.rs:50-259owns backend derivation from installedAggregationConfig. The shared formula branches are CMS at 103-121, HLL at 199-215, KLL at 218-231, and DDSketch at 233-246.deriveadds the backend-only GOS staleness term at 72-83.crates/asap_types/src/accuracy.rs:1-55already owns the sharedAccuracyProfile/AccuracyKindwire contract and is the intended home for normalized formula primitives.crates/asap_types/src/accumulator_spec.rsdefines the typedAccumulatorSpecconversion fromAggregationConfig; assess whether it can supply normalized family parameters without altering fingerprinted legacy wire fields.Keep backend-only aggregation families and the GOS augmentation outside the shared formula module.