Skip to content

fix(engine): publish video frames only after decoding - #6092

Merged
miguel-heygen merged 1 commit into
mainfrom
fix/deterministic-video-capture
Oct 22, 2026
Merged

miguel-heygen merged 1 commit into
mainfrom
fix/deterministic-video-capture

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

What

Repeated renders of scaled video frames now expose their replacement images only after the frame batch has decoded.

Why

Replacement images were visible while their img.decode() promises were still pending. Repeated captures of identical extracted JPEG pairs could produce different pixels in the scaled video inset.

Related work

This corrects the shared video-frame injection path used by screenshot capture.

How

Hide replacements before updating their sources, await the existing decode barrier, then publish the batch together. The temporary hide uses inline priority so a stylesheet cannot expose an unfinished frame. Decode failures still reject the frame operation.

Test plan

  • Browser regression: 7 passed, 0 skipped in three serial runs on Linux x64 with Chromium 152.0.7977.30.
  • Deliberate early-visibility mutation: exit 1, 6 passed and the new regression failed; restoring the fix returned the file to green.
  • Related screenshot and injector test files: 36 and 18 passed respectively, exit 0.
  • Engine build, lint and formatting checks passed.
  • Three repeats of a 20-frame 4K browser reproduction were byte-identical with the ordering fix; the unmodified injector differed.
  • Three complete 1,000-frame, 3840x2160 whole-render captures: every decoded RGBA frame matched, with no differing ordinal. Direct byte comparison also confirmed identical pre-encode streams, each 1,334,573,138 bytes with SHA-256 d6ebf401bef3b19c9e9e8a483e7edbbadbd5f70222fe8425a284adc4c1621a59.

The complete capture comparison uses the same fixture, machine, browser, screenshot route and packaged pipeline, with an unchanged three-worker cap. Automatic calibration selected three workers for the first two candidate runs and two for the third. Its compiled injection module differs from the baseline only by this publication-order change. No encoder settings, waits, retries or comparison tolerances were added.

@github-actions

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 189 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (1)

  • seqheldtwice-tlhold-px-r0-nested-z100: tracking 120.29, pressJump 312.3, drop 0.07, reload 0.07, render 0.03, renderKey -, undo true, teleport false / tracking 0.09, pressJump 0.11, drop 0.39, reload 0.39, render 0.02, renderKey -, undo true, teleport true / tracking 0.09, pressJump 0.11, drop 0.39, reload 0.39, render 0.02, renderKey -, undo true, teleport true

@jrusso1020 jrusso1020 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed f011f39. No blocker found. This is a focused two-file fix to publication order, and it reuses the existing image decode barrier rather than adding waits, retries or encoder changes.

In screenshotService.ts#injectResolvedVideoFramesBatch:

  • Each replacement is marked inline !important hidden before its source changes.
  • Every decode in the batch completes before any replacement is published.
  • A decode failure rejects before publication, and the existing video-specific error and transport cleanup are kept.
  • Same-source updates stay synchronous.
  • The capture path awaits injection before it captures.
  • Native opacity and hidden-ancestor handling are unchanged.

Validation:

  • 7 Chromium 152.0.7977.54 transport/browser tests and 36 screenshot tests passed locally, using borrowed dependencies. Own-source aliases and a minimal core re-export entry bypassed an unrelated missing generated runtime.
  • The new browser test holds a decode pending under an !important stylesheet and observes the replacement hidden. It then releases the decode and observes the correct pixel and visible result.
  • The separate videoFrameInjector suite couldn't collect: its compiler import needs the absent generated runtime. It isn't reported as passed.

The 4K/1,000-frame hashes and the broader repeated-render measurements are the author's evidence. Encoder determinism and cross-architecture quality weren't independently reproduced.

Review by Rames (hf-review)

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 22, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 22, 2026
@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 22, 2026
Merged via the queue into main with commit aa4b535 Oct 22, 2026
118 checks passed
@miguel-heygen
miguel-heygen deleted the fix/deterministic-video-capture branch October 22, 2026 23:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants