[harper-joseph]

Companion to prerender-plugin#139 (pageCheck), and a direct response to review feedback on it: don't make the plugin decompress and parse the whole document to recover something the renderer already has in a live DOM.

Why

#139 needs to know what a rendered page CLAIMS about price and availability, so the probe can ask "does the cached page still agree with the origin" — the question that catches a value which changed and changed back between two probes (measured at ~2.7% of served product pages on one deployment).

The plugin could recover those values from the stored HTML, but that means a regex scan plus a JSON parse of a ~1MB commerce PDP on the hottest write path in the plugin, to reconstruct data this process had structured in front of it. Doing it here is one page.evaluate against a DOM that is already parsed and already settled.

What

RenderJob.structuredOffers — the page's schema.org Product offers, flattened to [price, currency, availability] triples, posted in the result metadata.

• Read off the settled DOM, before postProcess, which may strip nodes: the field must describe the page as rendered.
• Sorted with code-unit comparison, so unchanged content yields a byte-identical sequence render to render and pod to pod (the consumer compares claims derived from it; localeCompare would have made the order depend on the browser's locale/ICU).
• Availability reduced to its last path segment (https://schema.org/InStock → InStock), matching the plugin's extraction contract exactly, including trailing slashes.
• The 200-triple cap is a refusal threshold, not a sample size: a page carrying more offers posts null — no claim, never a truncated one. A deterministic DOM-order sample that omits the offer the consumer's endpoint reports would disagree with it on every comparison, and a systematic disagreement means the consumer expires and re-renders that page forever. The cap holds across @graph nodes and script blocks (it originally leaked one triple per Product node past the cap — an unbounded array from a single crafted block), and every field is byte-bounded at 64 chars (the cap bounds count; without this one megabyte-long price string still inflated every posted result).
• The wire contract is three-valued and deliberate: an array is a claim; null is posted as null and means the extraction ran and the page declared no Product offers (or it failed benignly); an absent field means the renderer predates this feature — which is the case the consumer alarms on hourly. Collapsing null into undefined would make every offerless page impersonate an outdated renderer.
• Only for pages that will be stored (indexable or sitemap-listed) and best-effort: a failure here can never cost the render its content. Content is only ever produced in the same branch that runs the extraction, so a content-bearing result with no structuredOffers field genuinely is an old renderer.

Compatibility

Additive metadata field, ignored by consumers that don't read it (including plugin <= 0.57.0 — deploy order between the two packages does not matter).

@harperfast/prerender >= 0.58.0 requires it for pageCheck: there is no fallback to parsing the HTML on that side, so pageCheck is inert until this ships, and logs hourly to say so. Neither package blocks the other on deploy; the feature simply does nothing until both are in place.

Verification

• 7 tests against real headless Chrome (the renderOnce harness): flattening and sort order, @graph documents plus a single non-array offer, one malformed JSON-LD block not costing the page its other blocks, a no-offers page posting null (present on the wire, distinct from absent), a trailing-slash availability URL still yielding its verdict, a 250-Product @graph posting null rather than a truncated sample, and a 100k-char price being sliced to 64.
• Full browser suite green (153 tests), lint and format clean, tsc builds.

For the human reviewer

• The extractor is a self-contained page.evaluate function (no imports, no module-scope closure), matching extractIndexSignals beside it.
• Placement is deliberate: after the indexability verdict, before postProcess. If postProcess ever gains the ability to rewrite structured data, this ordering is what keeps the field honest about what was stored.
• The 200-triple threshold is a judgement call — variant-heavy PDPs legitimately carry dozens; a page that exceeds it loses pageCheck coverage (posts no claim), which is the deliberate trade against ever posting a claim built from a sample.

Review coverage

Authored by Claude (Fable). Two self-review rounds plus a fresh-context Claude adversarial pass over both PRs' diffs with full repo access (it found and executably reproduced the cap leak and the byte-unbounded field, both fixed here). Outside-model legs were attempted and are unavailable on this machine (Codex CLI: burned refresh token, needs codex logout && codex login; Gemini CLI: no API key configured), so model coverage is Claude-only and degraded, flagged per policy.

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request bumps the package version to 1.20.0 and introduces the extraction of schema.org Product offers (structuredOffers) directly from the settled DOM during rendering, which avoids expensive parsing downstream. The extracted offers are flattened into [price, currency, availability] triples, capped at 200, and sorted. Feedback suggests improving the robustness of the availability URL parsing to handle trailing slashes and optimizing the sorting logic to avoid redundant JSON.stringify calls.

[gemini-code-assist]

If offer.availability is a URL with a trailing slash (e.g., https://schema.org/InStock/), split('/').pop() will return an empty string "" instead of "InStock". Filtering out empty segments before popping ensures robust extraction of the last path segment.

Suggested change

	const availability = typeof offer.availability === 'string' ? offer.availability.split('/').pop()! : null;
	const availability = typeof offer.availability === 'string' ? offer.availability.split('/').filter(Boolean).pop() ?? null : null;

[gemini-code-assist]

Sorting the triples by calling JSON.stringify on every comparison is inefficient, as it results in O(N log N) stringifications. For up to 200 triples, this can lead to thousands of redundant stringifications. Comparing the elements of the triples directly is much more efficient and avoids any stringification overhead.

triples.sort((a, b) => {
	for (let i = 0; i < 3; i++) {
		const valA = a[i];
		const valB = b[i];
		if (valA === valB) continue;
		if (valA === null) return -1;
		if (valB === null) return 1;
		return valA.localeCompare(valB);
	}
	return 0;
});

[harper-joseph]

Both findings taken — fixed and pushed.

Trailing-slash availability: https://schema.org/InStock/ popped the empty segment and read as no availability at all. Now filter(Boolean) before pop, with a regression test.

JSON.stringify in the sort comparator: sorting is O(n log n) comparisons, so that serialised each triple many times over. Field-wise comparison now.

151/151 browser tests.

[harper-joseph]

Second self-review round plus a fresh-context adversarial pass (coverage details in the body). Two commits since the last review:

f717aaf — null is posted as null. The call site collapsed the extraction's null into undefined, so every offerless page was omitted from the payload and would have tripped the consumer's hourly 'renderer too old' alarm. On the wire: array = claim, null = extraction ran and found nothing, absent = renderer predates the feature. Also swapped localeCompare for code-unit comparison — the sort exists for cross-render/cross-pod determinism and collation varies with ICU.

28843f1 — the cap now actually caps, and refuses instead of sampling. The adversarial pass reproduced both: (1) the cap didn't hold across @graph nodes — one block with 10k single-offer Products posted 10k triples; (2) fields were byte-unbounded — a megabyte price string rode through whole (now sliced at 64 chars). And the worse latent design flaw: past the cap the old code posted a truncated DOM-order sample, and a deterministic sample omitting the offer the consumer's endpoint reports would disagree on every comparison — the consumer would expire and re-render that page forever. Overflow now posts null: no claim, never a truncated one.

153/153 tests against real headless Chrome. Body rewritten to current state.