[leorivastech]

What

A graded <video> inside a page-side shader transition no longer stalls the render. findInjectedRenderFrame now looks for the render frame among the element's following siblings before falling back to document.getElementById.

Why

During a transition, the page-side compositor clones the outgoing and incoming scenes, ids included, into staging layers. Color grading picks up the copied <video> and grades it too, and it finds the frame to draw from through findInjectedRenderFrame, which used getElementById. With the id repeated, the copy got the live scene's <img>. Both grading entries then kept moving their canvas right after that same <img> (keepCanvasAboveSource), each move re-triggering the other through the MutationObserver, which starves the page. The capture stops advancing at the transition's first frame, as in the report.

Related work

Fixes #3994

How

The injector always inserts the frame right after its <video> (insertBefore(img, video.nextSibling)), and the staged copy carries its own copy of it, so the element's following siblings are the right place to look first. The id lookup stays as the fallback, and when the frame isn't in the document at all (preview) it returns right away, as before. For everything that calls findInjectedRenderFrame (color grading, media sync, the media proxy, the WebGPU video texture shim) the answer only changes when an id is duplicated.

Test plan

• Unit tests:
  • renderFrameSibling.test.ts: a staged copy of a scene resolves to its own frame (fails on main), a frame past a node inserted between them, and a frame that isn't a sibling is still found by id.
  • colorGrading.test.ts: a graded video plus a staged copy with the same ids; each canvas has to end up right after its own frame. On main the canvases keep trading places (the test caps it at 100 moves and fails); here it settles after a couple.
  • packages/core 3736 passed, packages/shader-transitions 57 passed. tsc, lint, format, the comments check, the comment ratchet and fallow audit are clean.
• Manual testing with the gist from the issue, rendered with the CLI built from this branch and the repo's current shader-transitions bundle:
  • main stops at 653/1200 with --workers 2, same as the report (I stopped it after 3 minutes).
  • This branch finishes in 33 s with --workers 2, 53 s with 1 worker and 25 s with 4.
  • Until the clip ends at 2.0 s, the transition frames are the same picture as the ungraded render with the warm grade on top (see the first note below for what happens after).
  • Without data-color-grading, main and this branch produce the same 1200 frames (identical framemd5).
• Documentation updated (not applicable)
• Comments follow CONTRIBUTING.md "Comments"

Two related things I found, not changed here

Happy to send either of these as a follow-up if you'd like them:

1. A clip whose window ends mid-transition. In the gist the clip ends at 2.0 s, halfway through the transition. From there the graded render shows nothing for it, which matches what grading and the injector intend (a clip outside its window isn't shown). The ungraded render instead shows the clip's first frame: prepareComposite force-shows every [data-start] element in the copy, which also revives the hidden native <video>, a fresh element sitting at time 0. Skipping <video>s that have a __render_frame__ sibling in that loop is a two-line change, with a unit test in engineModePageComposite.test.ts. I tried it on a copy and from 2.0 s on both renders measure the same, frame by frame.
2. --no-page-side-compositing renders a graded <video> black during the transition (luma 16 against about 140 ungraded), on main too. In queryElementStacking, getEffectiveOpacity counts the opacity: 0 !important that grading puts on the source <video>, so selectDomLayerShowIds drops the clip and the scene buffer stays empty. Ignoring the element's own opacity when it carries data-hf-color-grading-source-hidden, as injectVideoFramesBatch already does, is about three lines, with a unit test on the stacking query. On a patched copy the video shows up with the grade applied.

[jrusso1020]

Approving at 9d60ac64, adding to the existing approval at this head. Clear write-up, and a fix sized to the cause.

Can the sibling-first lookup change the answer when ids are unique? No. renderFrameSibling.ts:57-61 now requires document.getElementById(frameId) to be non-null before looking at siblings, and only accepts a sibling whose id === frameId. With unique ids, that sibling is the getElementById result, so the returned element is identical. When the frame isn't in the document (preview), it still returns null right away, as before. The answer can only differ when the id is repeated, which is the staged-copy case.

The four callers

• media.ts:35,568 and mediaProxy.ts:72 only use the result as a boolean. Non-null still requires the id to resolve in the document, so their answers change only in the duplicate case, and there only in which <img> gets picked.
• video-texture-compat.ts:30-41 already tries video.nextElementSibling with the __render_frame__ class before calling the helper. So the helper now also covers a frame that isn't the immediate sibling, which is consistent with what it already did.
• colorGrading.ts:2476 (findRenderFrameImage) is the caller that was actually wrong. Pairing the copy with the live <img> is what made the two grading canvases trade places in keepCanvasAboveSource.

Tests. I ran renderFrameSibling.test.ts at this head (all pass). I then restored renderFrameSibling.ts from main and ran it again: resolves a staged copy of a scene to the copy's own frame (#3994) fails, as the PR says. I couldn't run the colorGrading.test.ts case locally because of stale dependencies. From reading it, the 100-move cap on insertBefore and the < 10 moves assertion are the right shape to catch the ping-pong without hanging.

CI at this head: 41 checks pass and none fail. 18 were still pending (Windows jobs and the studio shards) when I checked.

On the two follow-ups: both are worth sending.

1. prepareComposite reviving the native <video> (engineModePageComposite.ts:299-302). Forcing opacity/visibility on every [data-start] in the copy also un-hides a <video> that the injector hid behind its __render_frame__. After the clip's window ends, the ungraded render then shows the clip's first frame while the graded one shows nothing. Skipping <video>s that have a render-frame sibling makes both paths agree. Small change, real correctness fix.
2. --no-page-side-compositing renders a graded video black. getEffectiveOpacity (videoFrameInjector.ts:580) multiplies in the element's own opacity, which color grading sets to 0 on the source <video>. So the stacking query drops the clip. It's already broken on main, and ignoring the element's own opacity when it carries data-hf-color-grading-source-hidden matches what injectVideoFramesBatch does.

Verdict: APPROVE
Reasoning: The lookup is provably unchanged for unique ids, it fixes the duplicate-id pairing that stalled the render, and the regression test fails on main as claimed.

— Rames Jusso