[harper-joseph]

Adds two options to the negative cache (render.negative), for an origin that wants each dead URL asked about once a day:

• expiry: 'midnight' (with expiryTimezone): a stored 404 answers until the next local midnight after the origin last confirmed it. No background re-checks.
• replicate: true: one copy for the cluster in a new replicated table, SharedNegativePage, instead of one per node in NegativePage.

Both default off, so a deployment that sets neither behaves as before.

Why

An origin asked for 404s to be held until its nightly change. A replay of one week of a 4-node production deployment's dead-URL requests (~790k a day) compared policies:

Policy	Origin requests/day	Stale 404 answers vs today
2h15m fresh window, per node (current)	466k	1.0×
Midnight, per node	322k (−31%)	~3.1×
Midnight, replicated	268k (−43%)	~4.2×

The stale figures are estimates. Successful requests aren't logged, so they use each URL's prior request rate.

What changes

• Freshness under midnight: a row is fresh until getNextTimeOfDay('00:00', expiryTimezone, checkedAt), then expired; there is no revalidate window. The boundary is computed at read time from checkedAt, so switching applies to rows already stored. getNextTimeOfDay gains an optional nowMs (backward compatible); rawExpiresAt now passes its nowMs through.
• Staggered deletion: a midnight row's @expiresAt is its boundary plus fnv1a32(key) % 1 day. Every row stored on one day shares one boundary, so without the spread Harper's cleanup would delete a whole day of blob rows (~270k per node on the measured deployment) in one pass after midnight. Serving still stops at the boundary.
• Table choice: SharedNegativePage lives in the existing negative_cache database. That database already has a replicated anchor, so this adds no new replication connections, and it stays isolated from page_cache and raw_cache copies. Each mode reads only its own table, so switching needs no migration.
• New negative_cache expired-live count: a row whose window had ended, found live by the next proxied request. It is counted and not timed as served_wrong, because the page may have come back after the row stopped answering. Under midnight expiry this is the stale-404 signal; recheck-live no longer fires.
• A bad expiryTimezone fails closed: nothing answers and stores count as write-failed, rather than throwing on the request path. Config flags it as render.negative.expiryTimezone. The lifeMs warnings no longer fire under midnight expiry.

Capacity check on the target cluster (measured before building)

	Added per node	Current
Replicated stores	~268k/day (~3/s, ~7/s peak in the first hour after midnight)	raw_cache already replicates ~834k/day of the same size (+32%)
Bytes written	~9.4 GB/day (404 bodies average ~35 KB gzipped)	disk writes 2.2–2.7 MB/s (+~4–10%)
Disk	≤ ~19 GB live, counting the deletion spread	~300 GB free of 576 GB
CPU / network	negligible / +~0.1 MB/s in	load 0.5–0.9 on 16 cores / 1.4–2.2 MB/s in
Replication	—	all sockets connected, 0 copying, backpressure ≤ 6%, 0 blob-stream timeouts in 24h

Origin 404 fetches in the first hour after midnight rise ~42% (peak 6.3/s → 6.9/s). Crawlers' re-requests spread across the hour, so there is no spike. The main thing to watch after enabling is memory: swap at the post-midnight write wave.

Test plan

• node --test in packages/plugin: 2,106 pass. New tests cover midnight freshness, body age, expiry spread, the bad-timezone path, answering with no re-check, replicate table routing and a replicated drop, expired-live, schema parity between the two tables, the config warnings, and getNextTimeOfDay with an explicit now across a DST change.
• npm run lint and npm run format:check are clean.
• Not yet run on a real Harper cluster: replication of SharedNegativePage alongside the node-local NegativePage in one database, and reclaim of spread @expiresAt rows.

Rollout

• Schema: the new table needs a worker restart per node.
• Enabling: set render.negative.expiry: midnight, expiryTimezone, and replicate: true. Either can be flipped at runtime once the table exists.
• After enabling, watch: negative_cache expired-live, the stores, and swap around midnight.

Follow-ups (not in this PR)

• Console: the negative-cache card's help text still describes the interval windows, and it doesn't chart expired-live yet.

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request introduces a new 'expiry: midnight' policy and a cluster-wide replication option ('replicate') for the negative cache in the prerender plugin. The 'midnight' policy allows stored 404/410 responses to remain valid until the next midnight in a specified timezone, avoiding background re-checks. The 'replicate' option stores negative cache entries in a new replicated 'SharedNegativePage' table rather than node-local tables, reducing origin requests across clusters. Additionally, a reclaim spread mechanism is introduced to distribute database deletions throughout the day. Review feedback suggests improving the 'isTimeZone' helper in 'packages/plugin/src/config.js' to explicitly verify that the timezone parameter is a non-empty string, preventing 'undefined' or invalid values from silently passing validation by falling back to the system timezone.

[gemini-code-assist]

The isTimeZone helper returns true if zone is undefined because Intl.DateTimeFormat defaults to the system timezone when timeZone is omitted or undefined. To prevent invalid or missing timezone configurations from silently passing validation, explicitly verify that zone is a non-empty string before checking it with Intl.

const isTimeZone = (zone) => {
	if (typeof zone !== 'string' || !zone) {
		return false;
	}
	try {
		new Intl.DateTimeFormat('en-US', { timeZone: zone });
		return true;
	} catch {
		return false;
	}
};

References

1. Validation layers should explicitly check for invalid values (such as null or undefined) rather than relying on preceding layers (like merge layers) to filter them out, ensuring the validator remains robust and self-contained.