[harper-joseph]

Stacked on #163 — review that first. GitHub will retarget this to main when #163 merges. The commit here is the second one in the diff.

What

A refresh unlinks a departed target (sitemapUrl -> null) and leaves it rendering on its own cadence. This adds a per-route ingress.routes[].departureAction so a route can say that departure means something:

action	effect
none	unlink and nothing else — the default, and the previous behaviour
expire	hard-expire the cached pages; the URL re-renders on its own cadence
render	hard-expire and file the URL to render at the current minute

Why

A product that sells out or is withdrawn leaves the product sitemap the same day — measured on one deployment, 57/58 sold-out products absent from that day's sitemap, 77/77 in-stock present. Its page keeps serving the snapshot taken while it was still available until the next scheduled render, which on a slow cadence is most of a day of serving markup the origin no longer agrees with.

Per route, because only some URL classes carry that meaning. A product URL leaving a product sitemap says something about that product; a listing URL leaving a paginated sitemap usually means the catalog was re-bucketed, and acting on it would expire pages that are perfectly current.

A re-check, not a retirement. Departure is a statement about the declaration, not the page — the origin typically still serves the URL, 200, with out-of-stock markup. The render's own verdict decides what follows: a page that is merely unavailable re-renders with correct markup and stays; one the origin has actually retired answers 404/410 and is retired by the suppression path on the first verdict, because the target is now unlinked (#163).

Two details that are load-bearing

The expiry is hard. Past page.swrTtl, not to now. A plain Date.now() expiry leaves the page swr (util/pageFreshness.js) — i.e. still serving for another swrTtl. The swr window exists to smooth over a late re-render of content presumed still right, which is exactly what a departed product page is not. Same reasoning, and same arithmetic, as changeProbe.triggerRevalidate; Target.revalidate keeps its plain expiry deliberately.

The check runs after the whole walk. This is not an optimisation. A paginated corpus shears across child boundaries: children are walked in order, so a URL moving to an earlier child is re-attached before the child it left is pruned and never looks departed — but one moving to a later child is pruned first and looks departed until the child that now claims it is reached. Acting at prune time would fire on every URL that shifted forward, which on a fixed-size paginated sitemap is every URL after an insertion. Candidates are collected during the walk, re-read once it ends, and anything that picked up an attribution is dropped and counted as reattached.

That counter is also the diagnostic: a large departure_reattached share means the corpus is shearing and the raw removed count is not a departure count.

Bounded and observable

• sitemap.departure.dryRun defaults true. The first thing any deployment needs is the number a real walk would act on — a number nobody has until they look, and the one that decides whether maxActions is a ceiling or a no-op.
• maxActions (5000) caps actions per walk, so one child sitemap that fetches truncated but still parses as valid XML cannot expire a large slice of the cache. Overflow is counted as capped and left for the next walk.
• maxCandidates (50000) caps the in-memory list separately — it bounds memory across a walk that can prune millions.
• sitemap.departure.enabled is a master switch for incident response without editing the route list.
• Every candidate is decided and counted by outcome, on the result, the SitemapRefresh progress row, and a sitemap_departure_* series each.
• With no route opted in, departureCap is 0 and addRemoved allocates nothing — the feature costs an untouched deployment one boolean per walk.

Shape

• New util/sitemapDeparture.js — the decision table, pure, so every branch is testable without a database.
• departureAction is compiled and validated in compileEntry alongside renderInterval / discoverTargets / demandFloor, under the same drop-the-FIELD rule: a typo costs that route its departure check, never how the path is served. (This is why the field needed compiler support at all — compileEntry normalizes to an explicit allowlist, so an unknown field is silently dropped.)
• createRefreshRun gains a bounded candidate list and an outcome tally. Its addRemoved loop had a break at removedSampleCap that would have truncated the candidate list to the sample size; that is now two independent caps, with a test pinning it.
• The post-walk pass is guarded: a failure there must not fail a refresh that correctly ingested a million URLs.

Tests

npm test in packages/plugin: 1073 pass, 0 fail. Lint and format:check clean. 15 new tests cover the route opt-in and the master switch, an invalid action compiling away to none, the re-attached/suppressed/target-gone/route-opted-out skips, and the two independent caps.

The existing queueFunnel guard caught a hardcoded fromSitemap: false in the new schedule write; it now derives from the live target, which is what that guard asks for and stays correct if the shear guard above it ever moves.

Rollout

Ships inert: no route opts in, and dryRun is on. The sequence is to set departureAction: 'render' on the product route, read one walk's departure_would_render and departure_reattached, size maxActions against them, then turn dryRun off.

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request introduces a new sitemap departure feature (version 0.68.0) that handles URLs leaving a sitemap. It adds configuration options (departureAction per route, and a global departure configuration group), implements a post-walk departure check to safely handle boundary-shearing in paginated sitemaps, and allows hard-expiring cached pages or triggering immediate re-renders for departed URLs. Comprehensive unit tests and metrics tracking have also been added. There are no review comments, and the implementation appears robust and well-tested, so I have no feedback to provide.