[harper-joseph]

What

A 404/410 verdict suppresses the target and rechecks it at gone.recheckInterval (14d), deleting only after gone.maxStrikes (2) consecutive verdicts. This makes the strike ceiling depend on sitemap attribution: a gone verdict on a URL no sitemap lists now retires the target on the first verdict (render.suppression.gone.maxStrikesUnlisted, default 1). Sitemap-listed targets are unchanged.

Why

Measured on a ~850k-URL deployment (one node, 22.8h of render/outcome/*):

outcome	count
rendered/stored	210,719
suppressed/http-error	88,992
suppressed/canonical-mismatch	12,618

~29% of all render outcomes are http-error suppressions, steady at 3–5.8k per 30-min bucket all day — roughly 356k/day fleet-wide. Each dead URL costs two renders (strike 1 suppresses, strike 2 fourteen days later deletes). That is a third of the render budget spent re-proving that dead URLs are dead.

Why the ceiling depends on attribution

The two cases are not symmetric, and the asymmetry runs opposite to the intuition:

• No attribution — nothing re-creates it. Discovery mints a target only on a 200 from the origin: maybeSchedule checks the status, and isPrerenderCandidate checks it again. A URL the origin answers 404/410 for cannot be re-minted by a crawler hit, so retiring it is terminal. The suppressed row it would otherwise hold blocks nothing, and the row + schedule row + recheck render are pure waste.
• Sitemap-listed — the refresh re-creates it. actionForExisting returns CREATE for an absent target and SKIP for a suppressed one, so the suppressed row is the only thing holding the walk off. Deleting trades one recheck per recheckInterval for a full render per sitemap.refreshInterval — at a 6h refresh against a 14d recheck, ~28× more render work, not less.

Scoped to gone verdicts deliberately: noindex and canonical-mismatch verdicts come from pages the origin serves 200 for, so discovery re-mints them on the next bot request (a <meta> noindex is not even visible to isCandidateFromHeaders, which reads headers) and retiring those on sight would loop the same way.

Shape

• New packages/plugin/src/util/suppression.js — gradeSuppression() / isGoneVerdict(), pure and config-reading like util/failureBackoff.js, so the decision is unit-testable (Target.js subclasses a table at import time and cannot be loaded without a live Harper).
• Target.suppress reads the grading instead of inlining it. sitemapUrl was already in its projection, so there is no extra read.
• New option render.suppression.gone.maxStrikesUnlisted (default 1, min: 1). Set it equal to gone.maxStrikes to restore the pre-0.67.0 behaviour.

A URL that departs a sitemap is patched to sitemapUrl: null first, so it falls into the retire-immediately class — which is what lets this compose with the sitemap-departure work rather than fight it.

Tests

npm test in packages/plugin: 1058 pass, 0 fail. Lint and format:check clean.

• New test/suppression.test.js (6 tests) over the grading.
• test/suppressionStatus.test.js: seedSource gains sitemapUrl; the four existing gone tests now seed a listed target (the suppression path they were always exercising), plus three new tests for the unlisted path — unlisted 404 and 410 retire on the first verdict, and an unlisted non-gone verdict still suppresses rather than retiring.

Risk

The retirement is on a path that already deleted the cached pages at strike 1, so nothing that was being served stops being served any sooner. What changes is that the target row and its schedule row go too, and the 14-day recheck render never happens. If a 404 was wrong (an origin blip), a sitemap-listed URL is untouched by this change, and an unlisted one is restored by the next bot request that gets a 200 — which is how it was discovered in the first place.

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request introduces a new configuration option, maxStrikesUnlisted, to optimize the suppression and retirement of prerender targets. Unlisted targets that return a 404 or 410 ('gone') status are now retired on the first verdict by default, preventing wasteful rechecks for URLs that cannot be re-created by discovery. Sitemap-listed targets continue to use the standard maxStrikes threshold to avoid redundant rendering work. These changes are implemented via a new suppression.js utility, integrated into the Target resource, and verified with comprehensive unit tests. There are no review comments, and I have no additional feedback to provide.