[harper-joseph]

What

Every rendered variant result now includes pageFacts: what the cached page itself claims. The renderer reads it in the page off the settled DOM, next to structuredOffers.

pageFacts: {
  canonical: string | null        // .href of the first <link rel="canonical"> (absolute)
  title: string | null            // document.title, trimmed
  metaDescription: string | null  // content of the first <meta name="description">
  h1: string | null               // first <h1> textContent, whitespace collapsed, trimmed
  product: null | {               // first JSON-LD node typed Product or ProductGroup (see "Sourcing")
    name, brand, image: string | null
    rating: [number | null, number | null] | null               // [ratingValue, ratingCount ?? reviewCount]
    offers: Array<[sku, price, currency, availability]> | null  // document order, NOT sorted
  }
  breadcrumbs: string[] | null    // first BreadcrumbList, ordered by position
}

The extractor, its types and PAGE_FACT_BOUNDS live in src/pageFacts.ts. renderer.ts calls it in the same branch as structuredOffers.

Why

The plugin will compare a cached page's own claims against a product API, and re-render only a page that is wrong. It never parses HTML on its hot write path, so every fact it compares has to come from the renderer. The plugin side is a separate, later PR written against this contract.

Wire contract

• Absent vs null works exactly as for structuredOffers. ABSENT means the renderer predates the field. null means the extraction ran and failed benignly (the page was not evaluable).
• Failure never fails the render. A throw becomes null. The extraction runs as a separate page.evaluate, so a throw in one extraction never costs the other.
• It only runs where content is produced. Absence therefore means something only on a variant that has content, as with structuredOffers.
• It is posted on every path that posts structuredOffers:
  • the legacy flat envelope (resultMetadata)
  • every multi-device variant
  • the result-build-failed variant (unreportableMetadata)
• Bounds are refusals, not truncations:
  • a string longer than 2048 characters becomes null
  • more than 200 offers makes offers: null, counted across an AggregateOffer's list and across ProductGroup variants
  • more than 30 breadcrumbs makes breadcrumbs: null
  • an offer field longer than 64 characters becomes null, for that field only
  • Worst case is about 125 KB per variant; a typical PDP adds well under 2 KB.
• Types are exported from the package entry: PageFacts, PageFactsProduct, PageFactOffer, PageFactBounds.

Sourcing

The rules below decide which node and which values feed the fields. The shape above never changes.

• Where JSON-LD is searched:
  • every <script type="application/ld+json">, in document order
  • a top-level array or @graph counts as a list of nodes
  • a block that does not parse is skipped without costing the others
• One level of nesting. Used only when no top-level node matches:
  • A page node (WebPage or one of its schema.org subtypes, e.g. ItemPage, CollectionPage, SearchResultsPage) has its mainEntity (an object or an array) and its mainEntityOfPage (only if it is an object) searched for the Product/ProductGroup.
  • Its breadcrumb object is searched for the BreadcrumbList. That object must be typed BreadcrumbList.
  • A top-level node always wins, wherever it sits on the page. Nothing deeper than one level is read, and non-page nodes are never searched.
• Offers:
  • They come from the node's offers: an object, an array, or an AggregateOffer's offers list.
  • Availability is reduced exactly as in structuredOffers: https://schema.org/InStock becomes InStock.
  • A numeric sku or price becomes its string form. Any other non-string value is null.
• sku on the product:
  • A product with exactly one offer that has no sku (the sku is missing, null or "") takes the product's own sku, under the same 64-character refusal.
  • It never applies with several offers.
  • An offer sku that is present but refused stays null; the product sku does not replace it.
• ProductGroup variants:
  • A ProductGroup that states no offers of its own reports the offers of its hasVariant products. For each offer, sku = offer.sku ?? variant.sku.
  • The same 200-offer refusal applies. A group whose own offers are refused does not fall through to its variants.
  • name, brand, image and rating stay the group's. They come from the first variant only where the group states none.
  • Each value is chosen first and bounded second. An overlong group value is therefore refused, not swapped for the variant's.
• Other choices the contract left open:
  • An empty string is null; there is one way to say "no claim".
  • rating is null when neither value is numeric. Numeric strings are accepted, and a null count never becomes Number(null) === 0.
  • breadcrumbs:
    • A crumb with no name is skipped. A list with no named crumbs is null, not [].
    • One name over the bound makes the whole trail null, because a string[] cannot hold a null.
    • Crumbs without a position sort after the positioned ones, in document order.
  • The meta description is matched case-insensitively. HTML metadata names are ASCII case-insensitive.
  • The canonical uses the same link[rel="canonical"] selector as extractIndexSignals, so it reports the canonical the indexability verdict judged.

Plugin compatibility: checked, it ignores the field

In packages/plugin/src/resources/RenderQueue.js:

• normalizeJobResult copies each variant into an in-memory object with ...variant.
• processDecodedJobResult reads only named fields: outcome, statusCode, headers, isIndexable, structuredOffers, readiness, and so on.
• PrerenderedPage.put stores an explicit field list.
• Nothing validates or rejects unknown keys, and nothing stores the variant as a whole.

So pageFacts is carried in memory and dropped.

Local tooling

• reuseParityCheck: snapshots carry pageFacts. The report prints page facts differ (informational): …, listing the differing keys (product.* per field). This is outside pass, so the verdict is unchanged.
• renderOnce / renderMatrix: they expose it as result.job.pageFacts, alongside structuredOffers.

Tests

• The full suite is 346/346; the baseline was 317.

  • test/pageFacts.test.ts has 27 tests. Most run the real extractor through page.evaluate against small example.com fixtures.
  • test/jobResult.test.ts adds 2 wire tests.

• What the fixtures cover:

  • canonical resolved to absolute; title trim; meta description verbatim and case-insensitive; h1 whitespace collapse; the first element wins
  • an all-null page, and empty values reading as null
  • Product vs ProductGroup, @graph, a top-level array, @type as an array, the first product winning
  • AggregateOffer; brand as an object or a string; image as a string, an array or an ImageObject
  • ratingCount vs reviewCount, and a null count not read as 0
  • breadcrumbs out of order
  • every refusal bound, at the bound and one past it
  • malformed, null and string JSON-LD blocks next to valid ones
  • sku on the product: a single offer, a single offer through an AggregateOffer, the offer's own sku winning, several offers, a refused offer sku, a refused product sku
  • ProductGroup: variant offers with the variant-sku fallback; first-variant fallback for the fields; the group's own offers and fields winning; a refused group value not swapped; the 200 limit across variants; a refused group not falling through
  • Nesting: mainEntity as an object and as an array, a mainEntityOfPage object, breadcrumb on a CollectionPage, top-level beating nested, and nothing read two levels deep, under non-page nodes, from a string mainEntityOfPage or from an untyped breadcrumb

• End to end through renderOnce:

  • job.pageFacts is set and structuredOffers is unchanged.
  • A page whose document.title getter throws still renders ok, keeps its offers, and posts pageFacts: null, not absent.

• Revert check (patched dist/, reran). Each change below fails the tests noted:

  Change to dist/	Failing tests
  no renderer call	the 2 end-to-end tests
  no pageFacts in the builders	the 2 wire tests
  truncating instead of refusing	the 3 bound tests
  no @graph search	the ProductGroup test
  no product-sku inheritance	the sku test
  inheriting on multi-offer products	the sku test
  no hasVariant offers	2 tests
  no first-variant field fallback	the variant test
  a refused group falling through	the bound test
  no nested search	2 tests
  nested preferred over top-level	the precedence test
  any node's mainEntity searched	the one-level test

• npm run lint, npm run format:check and tsc are clean.

Known limitation, kept on purpose

structuredOffers still truncates. It slices an over-long offer field to 64 characters, which contradicts the refusal rule this PR follows. It is left as is because changing it would change the claim values the change probe already stores. The README says so.

Rollout

The change is additive and does nothing until a plugin reads it. The fleet can deploy before the plugin. An older plugin ignores the field (see above), and a newer plugin will treat its absence as "renderer predates it".

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request introduces the extraction of SEO facts (pageFacts) from the settled DOM, including canonical URLs, titles, meta descriptions, headings, product JSON-LD, and breadcrumbs. It implements strict boundaries where values exceeding limits are treated as refusals (null) to avoid continuous re-rendering. The review feedback suggests optimizing key merging in reuseParity.ts to avoid temporary array allocations, and wrapping the page.evaluate call in renderer.ts within a promise chain to catch potential synchronous throws.

[gemini-code-assist]

Avoid using the spread operator to merge keys into a Set as it allocates an unnecessary temporary array. Instead, initialize the Set with one object's keys and iteratively add the other's keys.

	const keys = (x: object, y: object) => {
		const set = new Set(Object.keys(x));
		for (const key of Object.keys(y)) {
			set.add(key);
		}
		return [...set];
	};

References

1. Avoid using the spread operator to merge map keys into a Set when processing large datasets, as it allocates an unnecessary temporary array. Instead, initialize the Set with one map's keys and iteratively add the other map's keys.

[harper-joseph]

Applied in 8ce233d: the key set is built from one object's keys and the other's are added in a loop. (This is local audit tooling that compares two ~6-key objects, so it doesn't matter for performance, but it's no less clear.)

[gemini-code-assist]

When executing asynchronous operations that may throw synchronous errors (such as page.evaluate when the page or browser context is closed), wrap the execution in a promise chain starting with Promise.resolve().then(...) to ensure any synchronous throws are caught by the .catch() block instead of escaping.

Suggested change

	job.pageFacts = await page.evaluate(extractPageFacts, PAGE_FACT_BOUNDS).catch(() => null);
	job.pageFacts = await Promise.resolve()
	.then(() => page.evaluate(extractPageFacts, PAGE_FACT_BOUNDS))
	.catch(() => null);

References

1. When executing asynchronous operations that may throw synchronous errors, wrap the execution in a promise chain starting with Promise.resolve().then(...) to ensure any synchronous throws are caught by the .catch() block instead of escaping.

[harper-joseph]

Not changing this one. Puppeteer's Page.evaluate is declared async evaluate(pageFunction, ...args) (puppeteer-core/lib/esm/puppeteer/api/Page.js), so a closed page or a destroyed context comes back as a rejection, never a synchronous throw, and .catch(() => null) already handles it. page here is always a live Page, never an arbitrary value. The line also mirrors the existing structuredOffers call right above it on purpose, so the two extractions keep identical failure handling. The test an extraction that throws posts pageFacts: null covers the in-page throw path from start to finish.

[harper-joseph]

Pushed 8ce233d. The pageFacts shape is unchanged; this changes where product facts are sourced from:

• A product with exactly one sku-less offer now takes the product's own sku.
• A ProductGroup with no offers of its own reports its hasVariant offers.
• When no top-level node matches, one level of mainEntity / mainEntityOfPage / breadcrumb under a page node is searched.

The Set-merge suggestion is applied. The Promise.resolve() suggestion is declined, with the reason in its thread. The PR body is rewritten to the current design, including the revert-check results. 346/346 tests pass, and lint, format and tsc are clean.