Skip to content

Add production-process differential coverage for non-DDSketch families #489

Open

Description

@zzylol

Context

Found while reviewing #478.

The PR documents differential-all as the complete sketch/query differential inventory. In practice:

  • promql_differential_process_e2e.rs is production-process + independent raw-value oracle, but DDSketch-only.
  • KLL, HLL, CountSketch, Count-Min Sketch, and heap-backed top-k live in e2e_controller_plans_and_backend_serves.rs, which constructs the controller/backend HTTP stack in-process and is not differential against an independent raw-data oracle.
  • Eight of those fifteen matrix scenarios currently fail.

The matrix is useful and should remain strict, but it does not yet provide the same process boundary and oracle strength for non-DDSketch families.

Current commands

./scripts/e2e.sh differential
./scripts/e2e.sh differential-all

Expected coverage

Add production-process scenarios for:

  • KLL: quantile with an exact rank oracle and justified tolerance
  • HLL: cardinality from a deterministic raw distinct set, including multi-SID merge
  • CountSketch and Count-Min Sketch: frequency estimates from deterministic keyed events
  • heap-backed CountSketch/CMS: top-k labels, ordering, and approximate counts
  • instant/range forms where the query language supports both

Each scenario should:

  1. ingest through modified OTLP into the production data-plane binary;
  2. query only the public PromQL HTTP API;
  3. prove ASAP-local execution rather than fallback;
  4. compare labels, timestamps, shape, and values with a raw-data oracle;
  5. remain executable and non-ignored when it exposes a product regression.

Done when

The all-sketch differential command no longer relies on in-process-only coverage for non-DDSketch families, and its documentation distinguishes stable acceptance scenarios from known failing diagnostics.

Activity

  1. zzylol commented on Sep 3, 2026

    @zzylol
    ContributorAuthor

    PR #478 now adds all_sketches_process_oracle_e2e.rs and the manual ./scripts/e2e.sh sketch-oracles target. DDSketch, KLL, HLL, CountSketch, and Count-Min Sketch each cross a real production data_plane child-process boundary and compare the public PromQL response with an oracle computed independently from raw values/keys. The sketch library is used only for fixture encoding. Current result: DDSketch passes; the four non-DDS families strictly fail with ASAP-local No result for query and remain non-ignored. Keeping this issue open for the still-missing production-process multi-SID and additional instant/range variants from its full acceptance list.

  2. added
    enhancementNew feature or request
    area: data planeRuntime ingestion, storage, and query-serving data paths
    area: query enginePromQL lowering, execution, result shaping, and fallback behavior
    feature: PromQLPromQL compatibility and execution semantics
    feature: warm summariesASAP warm-tier materialization, readiness, and serving
    feature: sketch algorithmsSketch-family configuration, lifecycle, merge, and readout behavior
    test: end-to-endProcess-boundary, differential, conformance, and full-stack tests
    on Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: data planeRuntime ingestion, storage, and query-serving data pathsarea: query enginePromQL lowering, execution, result shaping, and fallback behaviorenhancementNew feature or requestfeature: PromQLPromQL compatibility and execution semanticsfeature: sketch algorithmsSketch-family configuration, lifecycle, merge, and readout behaviorfeature: warm summariesASAP warm-tier materialization, readiness, and servingtest: end-to-endProcess-boundary, differential, conformance, and full-stack tests

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions