Skip to content

feat(plugin): per-route renderInterval — render cadence by route type; v0.22.0 - #58

Merged
harper-joseph merged 2 commits into
mainfrom
feat/route-render-interval
Aug 3, 2026
Merged

harper-joseph merged 2 commits into
mainfrom
feat/route-render-interval

Conversation

@harper-joseph

Copy link
Copy Markdown
Contributor

Why

Different route classes deserve different render cadences: the homepage changes intraday and is what bots re-crawl most; section/listing pages sit behind a fixed CDN edge TTL that caps how much freshness a faster render can even deliver; detail pages are the long tail where the default is right. One global defaultInterval can't express that.

What

ingress.routes entries accept an optional renderInterval (ms, prerender routes only):

routes: [
  { match: 'exact',  path: '/',        queryParams: [],    renderInterval: 2 * HOUR },
  { match: 'prefix', path: '/catalog/', queryParams: ['CN'], renderInterval: 6 * HOUR },
  { match: 'prefix', path: '/product/prd-', queryParams: [] },   // default cadence
]

Precedence, resolved at schedule time on every cycle: matched route renderInterval → the target's stored interval (sitemap <changefreq> / explicit API write) → render.defaultInterval. Applied consistently at all four scheduling decision points: post-render reschedule, redirect-source reschedule, Target.put initial jitter, and reconcile repair.

Design decisions (deliberate, please push back if wrong)

  • Route beats stored. Stored intervals are creation-time data; if they won, a route-cadence change would only affect targets discovered after the config change — the existing ~800k rows would keep stamped values forever, and retroactivity would require a registry sweep. Route-first means a cadence change takes effect on each URL's next render cycle, no migration. Per-URL exceptions stay expressible as an exact route above the class prefix (first-match-wins); a route that should defer to sitemap changefreq sets no interval.
  • Discovery no longer stamps defaultInterval onto new targets — that stamp is what froze creation-time config into data. Existing stamped rows are unaffected in behavior: route resolution supersedes them where a route sets a cadence, and they resolve to their stored value (same as today) where none does.
  • A bad renderInterval value drops the field, never the route — rejecting the entry over a cadence typo would silently change how the path is served (prerender → unclassified), a much worse failure than a wrong cadence.
  • Passthrough routes reject the field (warned) — never scheduled, no cadence to set.

Operational note (documented in config.js)

If the CDN edge-caches a route class with a fixed property-settings TTL (not driven by our response headers), that TTL and the route's renderInterval must be kept aligned by hand: rendering much faster than the edge TTL burns renders the edge never serves; much slower and the edge re-fetches stale content. Neither side can observe the other drift.

Tests

315 pass (6 new): compile validation (valid kept / invalid drops field not route / passthrough rejected), full resolver precedence including invalid stored values, exact-above-prefix per-URL exception, and processJobResult wiring (route cadence beats stored on reschedule; stored still drives when the route is silent; page expiresAt stays coupled to nextRenderTime).

🤖 Generated with Claude Code

…ved at schedule time; v0.22.0

ingress.routes entries accept an optional renderInterval (ms, prerender routes
only — a bad value drops the field with a warning, never the route, so a
cadence typo can't unroute a served path).

Cadence precedence, resolved on EVERY scheduling decision (post-render
reschedule, redirect-source reschedule, Target.put initial jitter, reconcile
repair): matched route renderInterval > the target's stored interval (sitemap
changefreq / explicit API write) > render.defaultInterval.

Route beats stored deliberately: stored intervals are creation-time data, and
if they won, a route-cadence change would apply only to targets discovered
after the config change — retroactivity would mean sweeping the registry.
Resolving route-first makes a cadence change take effect on each URL's next
render cycle with no data migration. A per-URL exception is an exact route
above its class prefix (first match wins); a route that defers to sitemap
changefreq simply sets no interval.

Discovery no longer stamps render.defaultInterval onto new targets — that
stamp froze creation-time config into data and is what made interval changes
non-retroactive.

Config docs carry the operational note: when the CDN edge-caches a route class
with a fixed property TTL, that TTL and the route's renderInterval must be
kept aligned by hand.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a route-specific renderInterval configuration to allow setting a custom render cadence on a per-route basis. This new cadence is resolved dynamically at schedule time with a precedence of route-level configuration over stored target intervals and default intervals. The review feedback highlights critical robustness improvements: first, in resolveRenderInterval, storedInterval should be coerced to a Number to correctly handle BigInt values returned by HarperDB and prevent silent fallbacks; second, raw.renderInterval should be guarded against null values to avoid redundant warnings, and any potential BigInt values must be handled safely to prevent JSON.stringify from throwing a process-crashing TypeError.

Comment on lines +274 to +278
export const resolveRenderInterval = (url, storedInterval) => {
const { entry } = classifyUrl(url);
if (entry && entry.renderInterval !== null && entry.renderInterval !== undefined) return entry.renderInterval;
return Number.isFinite(storedInterval) && storedInterval > 0 ? storedInterval : config.render.defaultInterval;
};

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

In HarperDB, Long columns (such as renderInterval in the Target table) can be returned as BigInt values. Since Number.isFinite returns false for BigInt values without coercing them, any target with a custom interval stored as a BigInt will be ignored, causing the scheduler to silently fall back to config.render.defaultInterval. Coercing storedInterval to a Number before performing the Number.isFinite check ensures that custom intervals are correctly respected. Additionally, since resolveRenderInterval is an exported function, we should guard it against invalid inputs (such as a non-string url) to ensure the robustness of the module's public API.

Suggested change
export const resolveRenderInterval = (url, storedInterval) => {
const { entry } = classifyUrl(url);
if (entry && entry.renderInterval !== null && entry.renderInterval !== undefined) return entry.renderInterval;
return Number.isFinite(storedInterval) && storedInterval > 0 ? storedInterval : config.render.defaultInterval;
};
export const resolveRenderInterval = (url, storedInterval) => {
if (typeof url !== 'string') {
throw new TypeError('Expected url to be a string');
}
const { entry } = classifyUrl(url);
if (entry && entry.renderInterval !== null && entry.renderInterval !== undefined) return entry.renderInterval;
const interval = storedInterval !== undefined && storedInterval !== null ? Number(storedInterval) : NaN;
return Number.isFinite(interval) && interval > 0 ? interval : config.render.defaultInterval;
};
References
  1. Guard exported functions against invalid inputs (e.g., non-string types) to ensure the robustness of the module's public API.

Comment on lines +84 to +99
let renderInterval = null;
if (raw.renderInterval !== undefined) {
if (mode === PASSTHROUGH) {
warn(
`ignoring renderInterval on passthrough route "${raw.match} ${raw.path}" — a passthrough route is ` +
`never scheduled, so it has no render cadence`
);
} else if (Number.isFinite(raw.renderInterval) && raw.renderInterval > 0) {
renderInterval = raw.renderInterval;
} else {
warn(
`ignoring renderInterval on route "${raw.match} ${raw.path}" — expected a positive number of ` +
`milliseconds, got ${JSON.stringify(raw.renderInterval)}`
);
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

There are two issues here:

  1. If raw.renderInterval is explicitly configured as null (a common pattern to explicitly disable or clear a setting), the current check raw.renderInterval !== undefined will evaluate to true, leading to an unnecessary warning being logged. Checking for both undefined and null avoids this warning.
  2. If raw.renderInterval is a BigInt (which can happen if passed from a custom JS configuration), JSON.stringify will throw a TypeError: Do not know how to serialize a BigInt and crash the process. Safely handling BigInt serialization prevents this crash.
Suggested change
let renderInterval = null;
if (raw.renderInterval !== undefined) {
if (mode === PASSTHROUGH) {
warn(
`ignoring renderInterval on passthrough route "${raw.match} ${raw.path}" — a passthrough route is ` +
`never scheduled, so it has no render cadence`
);
} else if (Number.isFinite(raw.renderInterval) && raw.renderInterval > 0) {
renderInterval = raw.renderInterval;
} else {
warn(
`ignoring renderInterval on route "${raw.match} ${raw.path}" — expected a positive number of ` +
`milliseconds, got ${JSON.stringify(raw.renderInterval)}`
);
}
}
let renderInterval = null;
if (raw.renderInterval !== undefined && raw.renderInterval !== null) {
if (mode === PASSTHROUGH) {
warn(
"ignoring renderInterval on passthrough route \"" + raw.match + " " + raw.path + "\" — a passthrough route is " +
"never scheduled, so it has no render cadence"
);
} else if (Number.isFinite(raw.renderInterval) && raw.renderInterval > 0) {
renderInterval = raw.renderInterval;
} else {
const displayValue = typeof raw.renderInterval === 'bigint' ? raw.renderInterval + "n" : JSON.stringify(raw.renderInterval);
warn(
"ignoring renderInterval on route \"" + raw.match + " " + raw.path + "\" — expected a positive number of " +
"milliseconds, got " + displayValue
);
}
}

…the resolver; null route interval means unset; crash-safe warn

The BigInt one is the known Long-column trap: Number.isFinite rejects BigInt,
so a stored interval surfaced as BigInt silently fell to the default cadence.
The resolver now owns the coercion (reconcile's call-site Number() dropped —
one home). Number(null)=0 / Number(undefined)=NaN still fall through.

Declined the suggested TypeError on a non-string url: this module degrades
gracefully by convention (bad entries drop, unparseable URLs classify as
unclassified) — throwing would turn a data problem into a scheduler crash.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@harper-joseph

Copy link
Copy Markdown
Contributor Author

Review comments addressed in c2fd401:

  • BigInt stored interval (high): accepted — this is the known Harper Long-column trap. The resolver now coerces before the finite check (and reconcile.js's call-site Number() moved into the resolver so the guard has one home). Test added with a BigInt stored value.
  • null route interval + BigInt-safe warning (medium): accepted both — null now means "not set" (no warning, defers to stored/default; test added), and the warn path uses String() instead of JSON.stringify() so it can never throw on a BigInt.
  • Declined the suggested TypeError on a non-string url in resolveRenderInterval: this module's convention is graceful degradation (invalid entries drop at compile, unparseable URLs classify as unclassified) — throwing would convert a bad data row into a scheduler crash, which is strictly worse than falling back to the default cadence. classifyUrl already handles non-string input safely.

316 tests pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant