Skip to content

perf(browser): one in-page evaluate for structuredOffers and pageFacts — one CDP round trip and one JSON-LD parse per render, outputs unchanged; v1.38.0 - #212

Merged
harper-joseph merged 1 commit into
mainfrom
perf/browser-single-extract
Sep 24, 2026
Merged

harper-joseph merged 1 commit into
mainfrom
perf/browser-single-extract

Conversation

@harper-joseph

Copy link
Copy Markdown
Contributor

What

After settle, the renderer read the page's two claims with two page.evaluate calls: extractStructuredOffers (→ job.structuredOffers) and extractPageFacts (→ job.pageFacts, added in 1.37.0 / #208). Each call was its own CDP round trip, and each queried and JSON.parsed every JSON-LD block itself.

extractPageClaims (in src/pageFacts.ts) reads both in one evaluate and returns { structuredOffers, pageFacts }:

  • the <script type="application/ld+json"> blocks are queried once and each is parsed at most once. Parsing happens when the first reader asks for a block, and never goes past the furthest block either reader needs. (pageFacts stops early once it has a product and a trail; structuredOffers stops at its cap.)
  • the two readers are the 1.37.0 extractors verbatim apart from the block loop. git diff -w / ?w=1 shows that: most of the pageFacts.ts diff is indentation.
  • extractStructuredOffers and STRUCTURED_OFFER_CAP move out of renderer.ts. The renderer now makes one call:
    const claims = await page.evaluate(extractPageClaims, STRUCTURED_OFFER_CAP, PAGE_FACT_BOUNDS).catch(() => null);
    job.structuredOffers = claims?.structuredOffers ?? null;
    job.pageFacts = claims?.pageFacts ?? null;

What does not change

  • Every output value, byte for byte. That includes structuredOffers' existing behaviour of slicing an over-long field to 64 characters, which is left alone on purpose: the plugin's change probe stores those values. pageFacts keeps its refusal bounds and sourcing rules (product-level sku fallback, ProductGroup hasVariant, one level of nested mainEntity/breadcrumb, top-level beats nested).
  • Independence. Each reader runs in its own in-page try/catch, so a throw in one still nulls only that claim.
  • Absent vs null. Both fields are assigned at the same point and under the same condition as before. A failed evaluate posts null for both, never undefined, so the legacy envelope, the multi-device variants and the result-build-failed variant are unchanged.

The one intended difference

A failure of the evaluate itself now posts null for both claims. Before, only the claim whose evaluate failed was null. The extractor's own code cannot cause this: its outputs are fresh, bounded arrays and primitives. Only two things can:

  • a destroyed context or target, which also fails the postProcess evaluate that follows;
  • a page that monkeypatches Array.prototype so a result cannot be returned.

A new test pins the contract: both claims null, both present, and the render still ok. Related side effect: both claims are now read from one synchronous DOM snapshot. Before, a live page could change between the two evaluates.

Parity evidence

Fixtures: test/pageClaimsParity.test.ts

The test keeps the two 1.37.0 extractors verbatim (renamed only). Each side is mapped exactly as its renderer maps it: .catch(() => null) per evaluate before; one evaluate plus ?? null after. The merged output must be deepStrictEqual and identical as JSON bytes on every input below:

corpus documents result
every fixture of the structuredOffers suite 7 identical
every fixture of the pageFacts suite 38 identical
new edge cases where the two walks diverge: facts stop early while offers read on; offers overflow while facts read on; exactly the cap across two products; cap+1 across two blocks; ProductGroup (offers null, facts read the variants); JSON primitives, empty and whitespace blocks, @graph as an object; over-long strings everywhere; sort with nulls, duplicates, non-ASCII, 1e21 and -0; 60 small blocks 9 identical
hostile pages that break one reader: document.title throws, querySelector throws, Array.prototype.flat throws, querySelectorAll throws, one block's textContent throws 5 each claim null exactly where 1.37.0 made it null
seeded fuzz: random Product / ProductGroup / BreadcrumbList / WebPage graphs, AggregateOffers, offer counts around the cap, malformed and primitive blocks, strings around every bound 400 identical (63 with offers, 159 with a product, 75 with product offers, 27 with breadcrumbs, 100 over the offer cap)

To check that the gate is not vacuous, I broke the merged extractor three ways. Each mutation failed it:

  • the offers walk skips the last block;
  • one try around both readers;
  • the facts early-stop ignores breadcrumbs.

Real pages

I rendered 10 product pages and 3 listing pages from a production retail catalog. Each ran through renderOnce with that deployment's own config and renderer, on 1.37.0 and on this build (installed from npm pack). Renders were sequential, the build order alternated per page, and the total was 26 renders.

Two comparisons:

  • across builds: job.structuredOffers and job.pageFacts from the two separate renders, plus outcome, statusCode and isIndexable;
  • same DOM: inside the new-build render, straight after the real extraction, the shipped 1.37.0 extractors (taken from the 1.37.0 dist/) and the merged one ran back to back on that one settled DOM.

HTML bytes were not compared: third-party widgets change them from run to run, and the extraction is read-only.

page kind outcome status indexable offer triples product offers crumbs JSON-LD blocks / KB across builds same DOM
P1 product ok 200 true 50 50 4 2 / 15.4 identical identical
P2 product ok 200 true 1 1 6 2 / 2.0 identical identical
P3 listing ok 200 true — — — 1 / 8.1 identical identical
P4 product ok 200 true 2 2 5 2 / 2.2 identical identical
P5 product ok 200 true 1 1 6 2 / 2.0 identical identical
P6 product ok 200 true 1 1 6 2 / 1.7 identical identical
P7 listing ok 200 true — — — 1 / 7.9 identical identical
P8 product ok 200 true 17 17 5 2 / 7.0 identical identical
P9 product ok 200 true 1 1 5 2 / 1.6 identical identical
P10 product ok 200 true 37 37 6 2 / 13.4 identical identical
P11 listing ok 200 true — — — 1 / 8.5 identical identical
P12 product ok 200 true 1 1 5 2 / 1.7 identical identical
P13 product ok 200 true 5 5 6 2 / 3.5 identical identical

Result: 13 / 13 identical, 0 differences. Listing pages post structuredOffers: null and product: null, with canonical, title, meta description and h1 present, the same on both builds.

Cost

This laptop is shared with other benchmarks. Every measurement was interleaved (ABAB / BABA) on one DOM.

Real pages (13 pages, measured on the same DOM as above):

1.37.0 pair merged paired saving
round trip, ms (15 rounds × 13 pages) median 0.775 (p25–p75 0.70–0.89) median 0.558 (0.52–0.62) median 0.20 (0.14–0.26); merged faster in 183 of 195 rounds
in-page compute only, ms (batches of 25) median 0.112 (0.092–0.188) median 0.064 (0.052–0.092) median 0.048 (0.040–0.072)
products / listings, in-page median ms 0.096 / 0.220 0.054 / 0.116 —

The in-render extraction step was also timed in the separate renders (one sample per render, so noisier): median 1.58 ms before, 1.38 ms after.

CPU per extraction step, on a static local page with no network (12 KB JSON-LD: 50 offers, 5 crumbs). 4,000 calls per variant in interleaved blocks of 250, repeated 3 times; ranges span the 3 runs:

µs per call 1.37.0 pair merged
renderer main thread (TaskDuration) 217–221 148–155
Node process 153–159 129–140
Chrome browser process 262–268 238–250

The browser-process figure includes background work that scales with wall time, so its share of the saving is an upper bound.

Bottom line: about 0.2 ms wall and roughly 0.1 ms CPU per device render, from one fewer CDP round trip and one fewer JSON-LD parse. A render costs seconds of CPU, so this is small. It is worth doing because it is free and proven not to change any output, not because it changes capacity.

Considered, not done

  • Folding extractIndexSignals into the same evaluate would save one more round trip. Its failure has to fail the render; the claims' failure never may. Merging them would make a failed evaluate either fail the render or lose the verdict, so it stays separate.
  • Removing comments from the in-page function source, which is sent on every evaluate, would need a build change. Out of scope.
  • Settle and timing logic is untouched.

Checks

  • npm run build --workspace @harperfast/prerender-browser: clean
  • node --test in packages/browser: 353 / 353 pass (5 new parity tests, 2 new renderer contract tests)
  • npm run lint && npm run format:check: clean
  • Version: @harperfast/prerender-browser 1.38.0

🤖 Generated with Claude Code

…s — one CDP round trip and one JSON-LD parse per render, outputs unchanged; v1.38.0

The renderer read the page's two claims with two separate page.evaluate calls, each
querying and JSON-parsing every JSON-LD block itself. extractPageClaims reads both in
one evaluate: the blocks are queried once and each parsed at most once, on the first
reader's demand and never past the furthest block either reader needs.

The two readers' bodies are the 1.37.0 extractors verbatim apart from the block loop,
so every output is unchanged, including structuredOffers' 64-char field slicing (the
plugin's change probe stores those values). Each reader runs in its own in-page
try/catch, so a throw in one still nulls only that claim; a failure of the evaluate
itself posts null for both, never a failed render and never an absent field.

test/pageClaimsParity.test.ts keeps both 1.37.0 extractors verbatim and holds the
merged one to deep-equal and byte-identical output over every fixture of both suites,
edge cases where the two walks diverge, hostile pages that break one reader, and a
400-document seeded fuzz corpus. Three deliberate mutations of the merged extractor
each fail it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request optimizes the extraction of page claims (both structuredOffers and pageFacts) by combining them into a single in-page evaluation pass (extractPageClaims). This change reduces Chrome DevTools Protocol (CDP) round trips and avoids redundant JSON-LD parsing. A comprehensive parity test suite has been added to ensure the new single-pass implementation produces identical output to the legacy separate extractors, alongside updated unit tests for error handling. There are no review comments, so I have no feedback to provide.

@harper-joseph
harper-joseph merged commit ea54f46 into main Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant