Repository navigation
feat(plugin): per-route demandFloor — bound the demand ladder per route; v0.55.0 - #128
Conversation
…te; v0.55.0 A breadth-sweeping crawler that recrawls everything daily makes 'visited in consecutive windows' true corpus-wide, so global ladder rungs grant the fast cadence at corpus scale to whichever route that crawler sweeps. demandFloor makes the fastest reachable rung per-route policy: rungs faster than a route's floor are unreachable for its pages, so fast global rungs can serve a small churning corpus (listing pages) without dragging a large swept one (product pages) onto them. Consulted per ladder decision (live, like renderInterval); a stored rung below a newly-raised floor reads as the floor immediately (resolveEffectiveInterval) and re-stamps on its next decision (rungIndexOf snaps into the floored list). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request introduces a new demandFloor configuration option for prerender routes, which bounds the fastest cadence the demand ladder can grant to a route's pages. The changes update the configuration schema, integrate the floor logic into the demand ladder and route classification utilities, and add comprehensive unit tests. The feedback suggests simplifying the null/undefined check for raw.demandFloor in routeClass.js to a more idiomatic loose inequality check.
| // Optional per-route demand-ladder floor (ms) — the FASTEST cadence the ladder may grant | ||
| // this route's pages (see util/demandLadder.js). Same drop-the-FIELD rule as the two above. | ||
| let demandFloor = null; | ||
| if (raw.demandFloor !== undefined && raw.demandFloor !== null) { |
There was a problem hiding this comment.
Dismissing: != null would trip this repo's lint. The shared config sets eqeqeq: 'warn' with no null exemption (@harperdb/code-guidelines/eslint.config.mjs), so loose equality warns even in the idiomatic null-check position — I hit exactly that warning on a == null in a test stub while preparing the follow-up PR and had to spell it out there too.
The explicit form also matches the two sibling fields compiled in the same function (renderInterval, discoverTargets), which all deliberately treat null as "not set" rather than as a bad value. Consistency across the three matters more here than the character count.
(This PR is already merged and released as 0.55.0, so noting rather than changing.)
Why
The demand ladder's rungs are global, but corpora aren't symmetric: a breadth-sweeping crawler that recrawls an entire route daily makes "visited in consecutive windows" true for everything it touches, so any fast global rung gets granted at corpus scale to that route — measured on a production deployment, over a million product targets held the fastest rung, several multiples of fleet capacity from one rung alone. The operator's only lever was removing the fast rungs globally, which also takes them away from the small corpus that genuinely earns them (listing pages that churn intraday).
What
ingress.routes[].demandFloor(ms, prerender routes only): the fastest cadence the ladder may grant that route's pages. Rungs faster than the floor are unreachable for the route; a floor at/above the granted cadence leaves it resting there (single-rung). Live likerenderInterval— consulted per ladder decision, no data migration.Mechanics:
effectiveLadder(base, floor)filters the offered rungs; memo key is nowbase|floor(both from a handful of config values, so the memo stays single-digit).decideIntervalderives the floor from the matched route itself (demandFloorFor, a memoized-compile linear scan on a ~20×/s path) — no call-site changes.resolveEffectiveIntervalclamps a stored rung UP to the floor, mirroring its existing ceiling clamp and for the mirrored staleness reason: raise a floor and old faster stamps must not be credited by priority math while they wait to re-stamp.rungIndexOfsnaps into the floored list (the existing off-ladder mechanism).Verification
demandFloorFor/effective-interval clamps, and the decideInterval floor behavior — promotion cap, sub-floor heal, single-rung, unrouted URLs unaffected).For the human reviewer
demandLaddernow importsrouteClass(acyclic: routeClass imports only config); the floor is derived insidedecideIntervalrather than threaded from callers, trading one cheap classify per decision for zero call-site churn.Review coverage
Authored by Claude (Fable). Self-review + domain pass only — no outside-model leg ran; flagging per policy.
🤖 Generated with Claude Code