You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
rate/increase functions do no counter-reset correction #476
Root cause: IncreaseAccumulator keeps only starting + last_seen and query(Increase) returns last − first (no reset detection), so any window spanning a counter restart is undercounted — often negative.
What causes a reset
A Prometheus/VM counter is in-process and goes back to 0 whenever the target restarts: deploys/rollouts, crashes/OOM-kills, autoscaling, node drains/preemptions, pod reschedules. (Counter overflow is also a "reset" but is rare with float64.) So a "reset within the window" simply means "the exporter restarted during [d]."
Testing
Drop this into the test module of asap-query-engine/src/precompute_operators/increase_accumulator.rs (it isolates the bug to the accumulator, which is exactly what MultipleIncrease stores per key):
#[test]
fn counter_reset_increase_is_wrong() {
use crate::data_model::Measurement;
use promql_utilities::query_logics::enums::Statistic;
// One counter, sampled 100 -> 150 -> (restart) 10 -> 60 within the window.
let mut acc = IncreaseAccumulator::new(Measurement::new(100.0), 0,
Measurement::new(100.0), 0);
acc.update(Measurement::new(150.0), 1_000);
acc.update(Measurement::new(10.0), 2_000); // reset: 150 -> 10
acc.update(Measurement::new(60.0), 3_000);
let inc = acc.query(Statistic::Increase, None).unwrap();
// BUG: last - first = 60 - 100 = -40 (negative!)
// VictoriaMetrics/Prometheus reset-correct this to +110.
assert_eq!(inc, -40.0);
}
#Run:
cargo test -p query_engine_rust --lib counter_reset_increase_is_wrong -- --nocapture
Expected Result
It passes asserting -40.0, which is the bug: rate shows the same (negative value_diff/time_diff). Prometheus/VictoriaMetrics' reset-corrected value here is +110 (its removeCounterResetsMaybeNaNs rebuilds the monotonic series [100,150,160,210] → 210−100).
Activity
chore(query-engine): apply cargo fmt to increase accumulators