Skip to content

feat: surface v0.50.0's prioritisation signals, bucket the median, and pin the metric contract - #122

Merged
harper-joseph merged 3 commits into
mainfrom
chore/console-catchup-0.50
Aug 21, 2026
Merged

harper-joseph merged 3 commits into
mainfrom
chore/console-catchup-0.50

Conversation

@harper-joseph

@harper-joseph harper-joseph commented Aug 21, 2026 •

Copy link
Copy Markdown
Contributor

Catching the console up to plugin v0.50.0, plus the three plugin-side items that needed a restart
to be worth doing. Two packages: @harperfast/prerender v0.51.0 and
@harperfast/prerender-console v0.7.0.
Deploy the plugin first — the console degrades cleanly
against an older one, not the other way round.

1. v0.50.0 added four queue_health series and nothing read them

ready_sweep_ms, ready_published, ready_cadence, claim_granted. The catalog is explicit
about why the last one needs a panel:

the only series that shows whether prioritisation is engaging, since it reorders a fixed amount
of work and moves no total. All-index is the failure to look for and is indistinguishable from
health elsewhere.

That is exactly why the gap was invisible: throughput, backlog, claim-scan time and render outcomes
all read the same whether the ready set is working perfectly or not running at all. A node quietly
serving every claim from the fallback index looked healthy in every number this console displayed.

New Render prioritisation panel — prioritised share of grants, ready supply per sweep, sweep
time, and the effectiveInterval backfill gauge, over a stacked ready-vs-index timeline. It names
the three failures METRICS.md documents: all-index grants (with the reminder that
queue.ready.capacity is restart-scoped, where a change that looks applied but is not would
otherwise hide), a capped sweep (which orders the oldest part of the backlog and skips exactly
the recently-due rows the ordering exists to protect), and a dry sweep against a non-empty backlog.
ready_cadence is presented as the migration gauge it is, not an alarm.

2. A guard, because this gap was invisible on both sides

The route contract is pinned across all three layers by a test — that is why sweep-orphans
shipping unreachable in v0.48.0 cannot recur. Metrics had no equivalent.

The new test classifies every recordAnalytics call by its third slot: a quoted literal is a
series the console must chart or waive with a written reason; anything else must be a named
exception — a metric whose path slot carries a dimension (bot_serve's source), or a series named
at the call site (queueHealth(value, gauge), the sitemap_${series} templates). An emitter in an
unrecognised shape fails the test rather than being skipped by it. Both halves verified by breaking
them.

3. Plugin: bucket the median (v0.51.0)

bucketize built per-bucket means and p95s but kept the median as a scalar — so a dashboard
could put a median in a tile and never in a trend, which is how every chart ends up being a tail
chart and a p95 gets read as "what a crawler normally gets". One more array per value combo, in a
payload that already carries two. The console's cluster merge carries it through as well, or the
line would exist on one node and vanish under cluster scope, which is the default view.

The staleness chart now draws median + p95 — the same two statistics as its tile, so the trend and
the headline cannot disagree. Version-detected, not assumed: against a pre-0.51.0 plugin it
falls back to the mean line and says why, because this console runs against whatever plugin a
deployment has.

4. Plugin: stop asking the page cache about sitemaps

Every entry of a sitemap index is a child sitemap, and entryState was answering "is this URL
cached and scheduled" for each one — an XML document that is never prerendered — at two point reads
plus a route classification each. A page of 50 spent ~100 reads on a worker that also serves bot
traffic, to produce verdicts (no target, filtered) that are misleading and that console v0.5.0+
already stopped displaying.

Index entries now return lastmod instead — the one field the sitemapindex schema carries, and a
column this console already renders when present, which is what makes an older console safe against
the change. The invalidation read that only entryState needed is skipped too, and targetCount is
null rather than 0: an index owns no targets by construction (a walk attributes every Target to
the child that listed the URL), and the truthful 0 reads as "this sitemap covers nothing" on the
largest document in a deployment.

5. Plugin: the catalog's machine-readable list had drifted

v0.50.0 added the four series to the prose description and to METRICS.md but not to path.values —
which is the half GET /prerender_admin/metrics serves, so the endpoint under-reported its own
series.

A bug found on the way in

count is the number of emits, not what was recorded. claim_granted emits once per claim pass
carrying a job count, so summing counts answers "how many claim passes ran": 83% where the true
prioritised share was 84% — close enough to look right, wrong for a reason nobody would go
looking for. sumValues() reconstructs the recorded total as Σ(mean × count), because total is
populated for boolean counters only: every distribution row carries total: 0 beside a
perfectly good mean (verified on production — 82 of 142 series had a non-zero total, not one a
distribution). Now documented at the point of read in analyticsRead.js, and the fixtures use a
count that differs from their value so the two cannot be confused again.

Needed no work

  • queue.ready.* — the Queue settings card is prefix-based and picked up all five options on
    its own. Its description now covers them, including the restart-scoped one.
  • No new admin routes, so the route contract test stayed green.
  • The Metrics reference view renders the live catalog, so it lists the new series already.

Verification

Plugin 776, console 127 → 136, lint + format clean. The new guards were each verified by breaking
what they protect: removing a panel's read of claim_granted fails the coverage test, and an
emitter in an unparseable shape fails the accounting.

Deploy order

Plugin 0.51.0 first, console 0.7.0 second. The console tolerates an older plugin (it detects
the missing per-bucket medians and says so); a 0.51.0 plugin with an older console is also fine —
the extra payload field is simply unread.

… contract; v0.7.0

Plugin v0.50.0 added four `queue_health` series and nothing on this side read them.
One of them, `claim_granted`, is described in the catalog as the ONLY way to see whether
render prioritisation is engaging — the ready set reorders a fixed amount of work and
moves no total, so throughput, backlog, claim-scan time and render outcomes all read
identically whether it is working perfectly or not running at all. A node quietly
serving every claim from the fallback index looked exactly like a healthy one.

New Render prioritisation panel on the Queue view: prioritised share of grants, ready
supply per sweep, sweep time, and the effectiveInterval backfill gauge, over a stacked
ready-vs-index timeline. It names the three failures METRICS.md documents — all-index
grants, a capped sweep (which orders the OLDEST part of the backlog and skips exactly
the recently-due rows the ordering exists to protect), and a dry sweep against a
non-empty backlog — and says that `queue.ready.capacity` is restart-scoped, which is
where a change that looks applied but is not would otherwise hide.

AND A GUARD, because this gap was invisible on both sides. The route contract is pinned
across all three layers by a test; metrics had no equivalent, so a new series ships with
a green suite and no panel. The new test extracts every literal (metric, series) pair
from the plugin's emit sites — ground truth, not the catalog's `values` list, which had
already drifted — and requires each to be charted or waived with a written reason.
Verified it fails when a series goes unread.

Fixed while writing it: `count` on these is the number of EMITS, not what was recorded.
`claim_granted` emits once per claim pass carrying a job count, so summing counts
answers "how many claim passes ran" — 83% where the real prioritised share was 84%,
which is the worst kind of wrong. `sumValues()` reconstructs the recorded total as
Σ(mean × count), because the `total` column is populated for boolean counters only:
every distribution row in a live payload carries `total: 0` beside a perfectly good mean
(checked against production — 82 of 142 series had a non-zero total, none a
distribution). The test fixture now uses a count that differs from its value, so the two
cannot be confused again.

`queue.ready.*` needed no work: the Queue settings card is prefix-based and picked up
all five options on its own. Its description now covers them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new "Render prioritisation" panel to the queue management console to track how the render order is decided, adding a new sumValues helper to reconstruct the true sum of value metrics and updating the stackBy charting function. Corresponding unit tests, asset validation tests, and documentation updates are also included. The review feedback suggests using a safePct helper to prevent division-by-zero or null-coercion bugs when calculating percentages, and improving the robustness of the metric-matching regex in tests to support both single and double quotes.

Comment on lines +149 to +167
stat(
'Prioritised grants',
pct(fromReady, grants),
`${num(fromReady)} of ${num(grants)} jobs came from the ready set`,
// The failure this panel exists for. Only meaningful once something was granted.
{ warn: grants > 0 && fromReady === 0 }
),
stat('Ready set supply', perSweep === null ? '—' : fmtCount(perSweep), 'entries published per sweep', {
warn: perSweep === 0 && backlog > 0,
}),
stat('Sweep time', fmtMs(weighted(sweeps, 'mean')), sweeps.length ? 'mean per sweep' : 'no sweep in window'),
stat(
'Cadence carried',
pct(carried, carried + resolved),
'rows scored against their own cadence',
// Not an alarm on its own — see the header. Flagged only once the backfill has had
// a chance and still has not happened.
{ warn: carried + resolved > 0 && carried === 0 }
),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When calculating ratios or dividing metrics by a yardstick/cadence, we must avoid the Number(null) === 0 trap (where null / number evaluates to 0 instead of null or NaN). Instead of duplicating validation or fallback logic in the caller with inline ternary checks (e.g., grants > 0), we should use a shared helper function (like safePct) that explicitly validates that both the value and the yardstick are finite numbers and that the yardstick is greater than 0, returning null otherwise.

Suggested change
stat(
'Prioritised grants',
pct(fromReady, grants),
`${num(fromReady)} of ${num(grants)} jobs came from the ready set`,
// The failure this panel exists for. Only meaningful once something was granted.
{ warn: grants > 0 && fromReady === 0 }
),
stat('Ready set supply', perSweep === null ? '—' : fmtCount(perSweep), 'entries published per sweep', {
warn: perSweep === 0 && backlog > 0,
}),
stat('Sweep time', fmtMs(weighted(sweeps, 'mean')), sweeps.length ? 'mean per sweep' : 'no sweep in window'),
stat(
'Cadence carried',
pct(carried, carried + resolved),
'rows scored against their own cadence',
// Not an alarm on its own — see the header. Flagged only once the backfill has had
// a chance and still has not happened.
{ warn: carried + resolved > 0 && carried === 0 }
),
stat(
'Prioritised grants',
safePct(fromReady, grants) ?? '—',
`${num(fromReady)} of ${num(grants)} jobs came from the ready set`,
// The failure this panel exists for. Only meaningful once something was granted.
{ warn: grants > 0 && fromReady === 0 }
),
stat('Ready set supply', perSweep === null ? '—' : fmtCount(perSweep), 'entries published per sweep', {
warn: perSweep === 0 && backlog > 0,
}),
stat('Sweep time', fmtMs(weighted(sweeps, 'mean')), sweeps.length ? 'mean per sweep' : 'no sweep in window'),
stat(
'Cadence carried',
safePct(carried, carried + resolved) ?? '—',
'rows scored against their own cadence',
// Not an alarm on its own — see the header. Flagged only once the backfill has had
// a chance and still has not happened.
{ warn: carried + resolved > 0 && carried === 0 }
)
References
  1. When calculating ratios or dividing metrics by a yardstick/cadence in JavaScript, avoid the Number(null) === 0 trap (where null / number evaluates to 0 instead of null or NaN). Use a helper function to explicitly validate that both the value and the yardstick are finite numbers and that the yardstick is greater than 0 before performing the division, returning null otherwise.
  2. Avoid duplicating validation or fallback logic in the caller if the callee's contract already guarantees safety (e.g., checking for non-positive or non-finite values) to prevent redundant code and maintain a single source of truth.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dismissing this one — the guard you are describing already exists and is already applied here.

pct is the shared helper:

export const pct = (part, whole) => (whole ? `${Math.round((part / whole) * 100)}%` : '—');

The yardstick is validated (0, null, undefined and NaN all take the '—' branch), so the
inline grants > 0 you are reading is not a fallback — it is the warn condition, deciding
whether "no prioritised grants" is alarming or just "nothing was granted this window". Those are
different questions and the tile needs both.

The numerator half cannot fire here either: sumValues() is a reduce seeded at 0, so it
returns a number or nothing at all. The trap you are pointing at is real in this codebase — it is
why ratioOf() exists, added last release for exactly the case where the numerator is nullable
(weighted() returns null when no row carried a finite stat). That guard belongs where a null
can actually arrive; adding a second one where it cannot would make the two disagree about which
is load-bearing.

Comment on lines +203 to +205
const emitted = [...metricsSource.matchAll(/recordAnalytics\([^,]+,\s*'([a-z_]+)',\s*'([a-z_0-9]+)'/g)].map(
([, metric, series]) => `${metric}.${series}`
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The regex used to match recordAnalytics calls strictly expects single quotes. If any future metric emission in the plugin uses double quotes or template literals, this test will silently skip those emit sites. We should make the regex more robust by supporting both single and double quotes.

Suggested change
const emitted = [...metricsSource.matchAll(/recordAnalytics\([^,]+,\s*'([a-z_]+)',\s*'([a-z_0-9]+)'/g)].map(
([, metric, series]) => `${metric}.${series}`
);
const emitted = [...metricsSource.matchAll(/recordAnalytics\([^,]+,\s*['"]([a-z_]+)['"],\s*['"]([a-z_0-9]+)['"]/g)].map(
([, metric, series]) => `${metric}.${series}`
);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taken, though not as suggested — the fix you proposed would be false insurance, and the concern
behind it deserved more.

Double quotes cannot reach this file: prettier runs over the repo with single quotes enforced, so a
double-quoted emit site fails format:check long before this test sees it. Accepting both would
look like protection while protecting nothing.

The real weakness is the one your comment names: a scanner that silently skips what it cannot
parse reports "all covered" while covering less every time someone writes an emitter in a new
shape.
Template literals already exist in that file (`sitemap_${series}`), so the blind
spot is not hypothetical.

So the accounting is now total (28f1960). Every recordAnalytics call is classified by its third
slot: a quoted literal is a series the console must chart, and anything else must be a named
exception — either a metric whose path slot carries a dimension rather than a series
(bot_serve's source, page_age's bot) or a series named at the call site (queueHealth(value, gauge), the templates). A new emitter in a shape the scanner does not recognise now fails with a
message telling the author which of the two it is. Verified by adding one:

metrics.newThing names its series dynamically (`series`). If that slot is a dimension rather than
a series, add it to DYNAMIC_SERIES_SLOT; if it is a series, this test cannot see it and the
console needs checking by hand

The quote class is flexible now too — it costs nothing — but it is the accounting, not the regex,
that closes the hole.

harper-joseph and others added 2 commits August 21, 2026 17:11
… and fix the catalog's own list; v0.51.0

Three things the console could not do from outside, now that a restart is on the table.

BUCKET THE MEDIAN. `bucketize` built per-bucket means and p95s but kept the median as a
scalar, so a dashboard could put a median in a tile and never in a trend — which is how
every chart ends up being a tail chart and a p95 gets read as "what a crawler normally
gets". One more array per value combo, in a payload that already carries two. The
console's cluster merge carries it through as well, or the line would exist on a single
node and vanish under cluster scope, which is the default view.

STOP ASKING THE PAGE CACHE ABOUT SITEMAPS. Every entry of a sitemap INDEX is a child
sitemap, and `entryState` was answering "is this URL cached and scheduled" for each one
— an XML document that is never prerendered — at two point reads plus a route
classification each. A page of 50 spent ~100 reads on a worker that also serves bot
traffic to produce verdicts (`no target`, `filtered`) that are misleading and that
console v0.5.0+ already stopped displaying. Index entries now return `lastmod` instead,
which is the one field the sitemapindex schema actually carries; the console shows that
column only when present, so an older console is unaffected. The invalidation read that
only `entryState` needed is skipped too, and `targetCount` is null rather than 0 — an
index owns no targets by construction, since a walk attributes every Target to the child
that listed the URL, and the truthful 0 reads as "this sitemap covers nothing" on the
largest document in a deployment.

FIX THE CATALOG'S MACHINE-READABLE LIST. v0.50.0 added four `queue_health` series to the
prose description and to METRICS.md but not to `path.values` — which is the half that
`GET /prerender_admin/metrics` serves, so the endpoint under-reported its own series.

Also documented at the point of read: `total` is counter-only. Harper fills it for a
boolean emit and leaves it 0 on every distribution row (82 of 142 series on production
had a non-zero total; not one was a distribution), so the sum a value metric recorded
has to be reconstructed as Σ(mean × count) — reading `count` instead answers "how many
times did this record" with a plausible wrong number, which cost a real bug in the
console's prioritisation panel before it was caught.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…a regex matches

Review asked for the scanner to accept double quotes. Prettier forbids them here, so
that alone would be false insurance — but the concern behind it is right: a scanner that
silently skips what it cannot parse reports "all covered" while covering less every time
someone writes an emitter in a new shape.

So every recordAnalytics call is now classified by its third slot. A quoted literal is a
series the console must chart; anything else is a metric whose path slot carries a
dimension, or a series named at the call site, and it has to be a NAMED exception. A new
emitter in an unrecognised shape fails the test instead of vanishing from it (verified).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@harper-joseph harper-joseph changed the title feat(console): surface v0.50.0's prioritisation signals, and pin the metric contract feat: surface v0.50.0's prioritisation signals, bucket the median, and pin the metric contract Aug 21, 2026
@harper-joseph
harper-joseph merged commit 170d316 into main Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant