Repository navigation
perf(browser): one in-page evaluate for structuredOffers and pageFacts — one CDP round trip and one JSON-LD parse per render, outputs unchanged; v1.38.0 - #212
Merged
Conversation
…s — one CDP round trip and one JSON-LD parse per render, outputs unchanged; v1.38.0 The renderer read the page's two claims with two separate page.evaluate calls, each querying and JSON-parsing every JSON-LD block itself. extractPageClaims reads both in one evaluate: the blocks are queried once and each parsed at most once, on the first reader's demand and never past the furthest block either reader needs. The two readers' bodies are the 1.37.0 extractors verbatim apart from the block loop, so every output is unchanged, including structuredOffers' 64-char field slicing (the plugin's change probe stores those values). Each reader runs in its own in-page try/catch, so a throw in one still nulls only that claim; a failure of the evaluate itself posts null for both, never a failed render and never an absent field. test/pageClaimsParity.test.ts keeps both 1.37.0 extractors verbatim and holds the merged one to deep-equal and byte-identical output over every fixture of both suites, edge cases where the two walks diverge, hostile pages that break one reader, and a 400-document seeded fuzz corpus. Three deliberate mutations of the merged extractor each fail it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request optimizes the extraction of page claims (both structuredOffers and pageFacts) by combining them into a single in-page evaluation pass (extractPageClaims). This change reduces Chrome DevTools Protocol (CDP) round trips and avoids redundant JSON-LD parsing. A comprehensive parity test suite has been added to ensure the new single-pass implementation produces identical output to the legacy separate extractors, alongside updated unit tests for error handling. There are no review comments, so I have no feedback to provide.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
After settle, the renderer read the page's two claims with two
page.evaluatecalls:extractStructuredOffers(→job.structuredOffers) andextractPageFacts(→job.pageFacts, added in 1.37.0 / #208). Each call was its own CDP round trip, and each queried andJSON.parsed every JSON-LD block itself.extractPageClaims(insrc/pageFacts.ts) reads both in one evaluate and returns{ structuredOffers, pageFacts }:<script type="application/ld+json">blocks are queried once and each is parsed at most once. Parsing happens when the first reader asks for a block, and never goes past the furthest block either reader needs. (pageFacts stops early once it has a product and a trail; structuredOffers stops at its cap.)git diff -w/?w=1shows that: most of thepageFacts.tsdiff is indentation.extractStructuredOffersandSTRUCTURED_OFFER_CAPmove out ofrenderer.ts. The renderer now makes one call:What does not change
hasVariant, one level of nestedmainEntity/breadcrumb, top-level beats nested).try/catch, so a throw in one still nulls only that claim.nullfor both, neverundefined, so the legacy envelope, the multi-device variants and theresult-build-failedvariant are unchanged.The one intended difference
A failure of the evaluate itself now posts null for both claims. Before, only the claim whose evaluate failed was null. The extractor's own code cannot cause this: its outputs are fresh, bounded arrays and primitives. Only two things can:
postProcessevaluate that follows;Array.prototypeso a result cannot be returned.A new test pins the contract: both claims null, both present, and the render still
ok. Related side effect: both claims are now read from one synchronous DOM snapshot. Before, a live page could change between the two evaluates.Parity evidence
Fixtures:
test/pageClaimsParity.test.tsThe test keeps the two 1.37.0 extractors verbatim (renamed only). Each side is mapped exactly as its renderer maps it:
.catch(() => null)per evaluate before; one evaluate plus?? nullafter. The merged output must bedeepStrictEqualand identical as JSON bytes on every input below:@graphas an object; over-long strings everywhere; sort with nulls, duplicates, non-ASCII,1e21and-0; 60 small blocksdocument.titlethrows,querySelectorthrows,Array.prototype.flatthrows,querySelectorAllthrows, one block'stextContentthrowsTo check that the gate is not vacuous, I broke the merged extractor three ways. Each mutation failed it:
tryaround both readers;Real pages
I rendered 10 product pages and 3 listing pages from a production retail catalog. Each ran through
renderOncewith that deployment's own config and renderer, on 1.37.0 and on this build (installed fromnpm pack). Renders were sequential, the build order alternated per page, and the total was 26 renders.Two comparisons:
job.structuredOffersandjob.pageFactsfrom the two separate renders, plusoutcome,statusCodeandisIndexable;dist/) and the merged one ran back to back on that one settled DOM.HTML bytes were not compared: third-party widgets change them from run to run, and the extraction is read-only.
Result: 13 / 13 identical, 0 differences. Listing pages post
structuredOffers: nullandproduct: null, with canonical, title, meta description and h1 present, the same on both builds.Cost
This laptop is shared with other benchmarks. Every measurement was interleaved (ABAB / BABA) on one DOM.
Real pages (13 pages, measured on the same DOM as above):
The in-render extraction step was also timed in the separate renders (one sample per render, so noisier): median 1.58 ms before, 1.38 ms after.
CPU per extraction step, on a static local page with no network (12 KB JSON-LD: 50 offers, 5 crumbs). 4,000 calls per variant in interleaved blocks of 250, repeated 3 times; ranges span the 3 runs:
TaskDuration)The browser-process figure includes background work that scales with wall time, so its share of the saving is an upper bound.
Bottom line: about 0.2 ms wall and roughly 0.1 ms CPU per device render, from one fewer CDP round trip and one fewer JSON-LD parse. A render costs seconds of CPU, so this is small. It is worth doing because it is free and proven not to change any output, not because it changes capacity.
Considered, not done
extractIndexSignalsinto the same evaluate would save one more round trip. Its failure has to fail the render; the claims' failure never may. Merging them would make a failed evaluate either fail the render or lose the verdict, so it stays separate.Checks
npm run build --workspace @harperfast/prerender-browser: cleannode --testinpackages/browser: 353 / 353 pass (5 new parity tests, 2 new renderer contract tests)npm run lint && npm run format:check: clean@harperfast/prerender-browser1.38.0🤖 Generated with Claude Code