Skip to content

feat(plugin): per-route demandFloor — bound the demand ladder per route; v0.55.0 - #128

Merged
harper-joseph merged 1 commit into
mainfrom
feat/demand-floor
Aug 24, 2026
Merged

harper-joseph merged 1 commit into
mainfrom
feat/demand-floor

Conversation

@harper-joseph

Copy link
Copy Markdown
Contributor

Why

The demand ladder's rungs are global, but corpora aren't symmetric: a breadth-sweeping crawler that recrawls an entire route daily makes "visited in consecutive windows" true for everything it touches, so any fast global rung gets granted at corpus scale to that route — measured on a production deployment, over a million product targets held the fastest rung, several multiples of fleet capacity from one rung alone. The operator's only lever was removing the fast rungs globally, which also takes them away from the small corpus that genuinely earns them (listing pages that churn intraday).

What

ingress.routes[].demandFloor (ms, prerender routes only): the fastest cadence the ladder may grant that route's pages. Rungs faster than the floor are unreachable for the route; a floor at/above the granted cadence leaves it resting there (single-rung). Live like renderInterval — consulted per ladder decision, no data migration.

Mechanics:

  • effectiveLadder(base, floor) filters the offered rungs; memo key is now base|floor (both from a handful of config values, so the memo stays single-digit).
  • decideInterval derives the floor from the matched route itself (demandFloorFor, a memoized-compile linear scan on a ~20×/s path) — no call-site changes.
  • resolveEffectiveInterval clamps a stored rung UP to the floor, mirroring its existing ceiling clamp and for the mirrored staleness reason: raise a floor and old faster stamps must not be credited by priority math while they wait to re-stamp.
  • Sub-floor stamps heal on the next decision: rungIndexOf snaps into the floored list (the existing off-ladder mechanism).

Verification

  • 823/823 unit tests (3 new: field compile rules, demandFloorFor/effective-interval clamps, and the decideInterval floor behavior — promotion cap, sub-floor heal, single-rung, unrouted URLs unaffected).
  • Lint + format clean. No repo CI on PRs — verified locally per repo policy.

For the human reviewer

  • demandLadder now imports routeClass (acyclic: routeClass imports only config); the floor is derived inside decideInterval rather than threaded from callers, trading one cheap classify per decision for zero call-site churn.
  • The one behavioral edge: a route whose floor ≥ its granted cadence is deliberately a silent single-rung (documented in the schema text), not a config warning — the base varies per target (stored/changefreq/default), so compile time can't judge it.

Review coverage

Authored by Claude (Fable). Self-review + domain pass only — no outside-model leg ran; flagging per policy.

🤖 Generated with Claude Code

…te; v0.55.0

A breadth-sweeping crawler that recrawls everything daily makes 'visited in
consecutive windows' true corpus-wide, so global ladder rungs grant the fast
cadence at corpus scale to whichever route that crawler sweeps. demandFloor
makes the fastest reachable rung per-route policy: rungs faster than a route's
floor are unreachable for its pages, so fast global rungs can serve a small
churning corpus (listing pages) without dragging a large swept one (product
pages) onto them.

Consulted per ladder decision (live, like renderInterval); a stored rung below
a newly-raised floor reads as the floor immediately (resolveEffectiveInterval)
and re-stamps on its next decision (rungIndexOf snaps into the floored list).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new demandFloor configuration option for prerender routes, which bounds the fastest cadence the demand ladder can grant to a route's pages. The changes update the configuration schema, integrate the floor logic into the demand ladder and route classification utilities, and add comprehensive unit tests. The feedback suggests simplifying the null/undefined check for raw.demandFloor in routeClass.js to a more idiomatic loose inequality check.

// Optional per-route demand-ladder floor (ms) — the FASTEST cadence the ladder may grant
// this route's pages (see util/demandLadder.js). Same drop-the-FIELD rule as the two above.
let demandFloor = null;
if (raw.demandFloor !== undefined && raw.demandFloor !== null) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This check can be simplified to raw.demandFloor != null to cleanly handle both null and undefined in a single, idiomatic check.

Suggested change
if (raw.demandFloor !== undefined && raw.demandFloor !== null) {
if (raw.demandFloor != null) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dismissing: != null would trip this repo's lint. The shared config sets eqeqeq: 'warn' with no null exemption (@harperdb/code-guidelines/eslint.config.mjs), so loose equality warns even in the idiomatic null-check position — I hit exactly that warning on a == null in a test stub while preparing the follow-up PR and had to spell it out there too.

The explicit form also matches the two sibling fields compiled in the same function (renderInterval, discoverTargets), which all deliberately treat null as "not set" rather than as a bad value. Consistency across the three matters more here than the character count.

(This PR is already merged and released as 0.55.0, so noting rather than changing.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant