You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Query engine: roll-ups over keyed aggregations (aggregated_labels) #846
Follow-up to #839, which covers roll-ups only for single-subpopulation configs (KLL, DDSketch, HLL, Sum, MinMax: grouping_labels = G, one sketch per store key).
Goal
Let the engine serve a query grouped by Q from a keyed aggregation whose aggregated_labels ⊋ Q. Keyed configs use grouping = [] and aggregated = G, with a DeltaSet key aggregation or a self-keyed value: MultipleSum, MultipleMinMax, MultipleIncrease, CountMinSketch, CountMinSketchWithHeap and HydraKLL.
Why it's a different mechanism
The subpopulations live inside one sketch, so a roll-up can't merge store keys. Instead it has to enumerate the G-keys (from the DeltaSet, or the accumulator's get_keys), project them onto Q, and combine the per-key answers:
CMS: sum the point estimates. The error grows with the fan-in, so it has to be priced or bounded.
MultipleIncrease, CMS+heap: not mergeable across groups. The counter-reset merge only works across time, and the top-k heaps drop items.
HydraKLL: the engine wraps a single-key Hydra. sketchlib's full Hydra framework expands each record into its subpopulation subsets, which would answer subsets natively.
Open
Is this worth doing natively in the engine, beyond what query_time_aggregations already covers for the exact types?
Activity