Skip to content

console v0.9.0: read the probe's origin-pressure signals and the purge's spare-visited pass - #137

Merged
harper-joseph merged 1 commit into
mainfrom
chore/console-catchup-0.57
Aug 25, 2026
Merged

harper-joseph merged 1 commit into
mainfrom
chore/console-catchup-0.57

Conversation

@harper-joseph

Copy link
Copy Markdown
Contributor

The console last caught up at plugin v0.55.0. Plugin v0.56.0 and v0.57.0 added three prerender_ops series and reshaped both node-local passes, and the console read none of it.

It shipped invisible because the metric-coverage guard exempts metrics.changeProbe — it names its series with a probe_${series} template, which the guard's regex structurally cannot read. Green suite on both sides. One of the three missed series is the one the catalog says to alert on.

Change probe

series what it means why it needed a surface
probe_throttled origin pushback — 429/502/503/504, connect/read timeouts The only signal that the probe is loading an origin that cannot take it. The sweep answers by halving its own rate, so it covers less of the corpus per pass while the change rate, failure share and trigger count all keep exactly the shape they had.
probe_fresh rows skipped because a baseline was younger than reprobeAfter Explains why probed drops after a restart. Sustained, it means reprobeAfter sits too close to sweepInterval and real cadence is quietly two sweep intervals.
probe_unreadable registry rows whose key would not decode Those targets are never probed and appear in no other count.
  • throttled gets its own alarm, a head pill, and a per-node column — the probe rate is agreed and held per node, so pushback is a fact about one node's conversation with the origin, and a cluster total names nobody to go and look at.
  • fresh is counted against the rows a pass considered, not against its probes: a skipped URL was never attempted, so it sits outside probed entirely.
  • unreadable is stated as a storage-layer escalation, because no setting on that page reaches those rows.
  • The three ways a pass can stop early are no longer one Interrupted label. Standing down for a reseed is routine; giving up on a refusing origin means the slice was never covered and the counters are a partial count.

Discovered-target purge

  • skipVisited, defaulted ON in the console although the plugin defaults it off. A plugin default that changed behaviour for existing callers would be the wrong kind of change; a console default is a suggestion to a human. Sent on the census as well as the purge, so the number an operator approves is the number that happens.
  • visitedSkipped and errors are now terms in the "not reached" subtraction. Without them a pass that spared 40% of a prefix on purpose reported that 40% as keyspace it never got to — a completed pass rendered as an interrupted one. (unreadable is deliberately not a term: the walk skips those rows before anything reads their sitemapUrl, so they never entered discovered.)
  • abortedOnErrors and the delete-failure samples, because a failed delete leaves a row in the corpus that appears in no render metric, no serve metric, and nowhere else.

Cluster merge

Counters that sum, sum. The pacing window is taken as the worst node's, never a mean — averaging one throttled node against three healthy ones answers "was the probe being held back" with a number that was true nowhere. Nodes that gave up on a refusing origin, or ran a different delete predicate over the same prefix, are named rather than folded into a total.

Guard

A second metric contract test covers the probe family via the catalog's own values list — the source the regex-based guard cannot reach. Verified by hand to fail when a series goes unread, and to pass again when restored.

The other dynamic families (sitemap_*, demand_*, queue_health gauges, invalidation_*) stay unguarded, and the test says so explicitly. They are legitimately unread from analytics — the queue gauges come from the overview endpoint, the demand series have no panel — so guarding them means a waiver list stating a decision per series. That belongs with whoever makes those decisions, not in a catch-up.

Not in this PR

changeProbeStatus() does not expose a rule's statusSignals, so the probe view's rules table cannot show which rules treat a 4xx as an observation rather than a failure. That needs a plugin field and therefore a plugin release train; the console's failure-share narrative stays correct without it. Worth a follow-up.

Verification

  • packages/console: 210 pass, 0 fail (was 185 — 25 new tests)
  • packages/plugin: 850 pass, 0 fail, untouched
  • npm run lint and npm run format:check clean at the root

🤖 Generated with Claude Code

…'s spare-visited pass; v0.9.0

The console last caught up at plugin v0.55.0. Since then v0.56.0 and v0.57.0 added
three prerender_ops series and reshaped both node-local passes, and none of it was
being read — the metric-coverage guard exempts `metrics.changeProbe` because it names
its series with a template, so three new signals shipped invisible with a green suite
on both sides. One of them is the one the catalog says to ALERT on.

Change probe
  - probe_throttled: origin pushback (429/502/503/504, connect/read timeouts). This is
    the only signal that the probe is loading an origin that cannot take it, and the
    sweep answers it by halving its own rate — so it covers less of the corpus per pass
    while the change rate, the failure share and the trigger count all keep the shape
    they had. Its own alarm, a head pill, and a per-node column, since the probe rate is
    agreed and held PER NODE and a cluster total names nobody to go and look at.
  - probe_fresh: rows skipped because a baseline was younger than reprobeAfter. Counted
    against the rows a pass CONSIDERED, not against its probes — a skipped URL was never
    attempted — and flagged past half, which means reprobeAfter sits too close to
    sweepInterval and real cadence is quietly two sweep intervals.
  - probe_unreadable: registry rows whose key would not decode. A storage-layer
    escalation, said as one: no setting on that page reaches them.
  - The three ways a pass can stop early are no longer one "Interrupted" label. Standing
    down for a reseed is routine; giving up on a refusing origin means the slice was
    never covered, and the counters are a partial count.

Discovered-target purge
  - skipVisited, defaulted ON in the console though the plugin defaults it off — a
    plugin default that changed behaviour for existing callers would be the wrong kind
    of change, a console default is a suggestion to a human. Sent on the census as well
    as the purge, so the number an operator approves is the number that happens.
  - visitedSkipped/errors are now terms in the "not reached" subtraction. Without them a
    pass that spared 40% of a prefix on purpose reported that 40% as keyspace it never
    got to — a completed pass rendered as an interrupted one. (unreadable is not a term:
    those rows never entered `discovered`.)
  - abortedOnErrors and the delete-failure samples, because a failed delete leaves a row
    in the corpus that appears in no render metric, no serve metric and nowhere else.

The merge learns the same distinctions: counters that sum, a pacing window taken as the
worst node's rather than a mean that was true nowhere, and the nodes that gave up or ran
a different delete predicate named instead of folded into a total.

Guard: a second metric contract test covers the probe family by the catalog's own
values list, which is what the regex-based one structurally cannot see. Verified to fail
when a series goes unread. The other dynamic families stay unguarded and the test says
so — they need a per-series decision that does not belong in a catch-up.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request bumps the version of @harperfast/prerender-console to 0.9.0 and introduces several UI and aggregation improvements to the management console. Key additions include support for sparing bot-visited targets (skipVisited) during discovery purges, handling and displaying origin pushback/throttling, tracking skipped fresh rows from resumable sweeps, and surfacing unreadable registry rows. It also updates the aggregation logic to correctly merge these new metrics across cluster nodes and adds comprehensive test coverage for these features. I have no feedback to provide as there are no review comments.

@harper-joseph
harper-joseph merged commit 5b3d30e into main Aug 25, 2026
@harper-joseph
harper-joseph deleted the chore/console-catchup-0.57 branch August 25, 2026 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant