[miguel-heygen]

What

A cloud render chunk (Lambda, Cloud Run) now starts its capture browsers while it is still extracting its video frames, instead of after. Output is unchanged.

Why

A chunk's fixed work runs one step after another: boot the preflight browser and measure each video's drawn size, extract the video frames, then start the capture browsers, then capture. Starting the capture browsers takes a few seconds and does not depend on the extracted frames, so it can run alongside the extraction. Every chunk of every render with video saves that time, and so does the compute a render is billed for.

How

• Extraction starts, capture starts. renderChunk begins the deferred video extraction and goes straight into the capture stage instead of waiting for it. The frame lookup becomes a promise.
• A session waits on its first frame. createChunkVideoFrameInjectorFactory also takes a lookup that is still being built: each session's injector awaits it on its first injection, then behaves exactly as before (one stateful injector per session). The existing synchronous callers are unchanged.
• Failures keep their cause. If capture fails while extraction failed too, the chunk throws the extraction's own error (a VideoSourceExtractionError, for example), not the capture stage's wrapper, so callers and Step Functions routing see the same error as before. Extraction is always settled before the chunk's work directories are cleaned up, so no ffmpeg keeps writing into a removed directory.

Test plan

• Unit tests added/updated
  • renderChunkFallback.test.ts: an injector built from a lookup still being extracted holds its first injection until the lookup settles, and surfaces the extraction's failure; each session gets its own injector.
  • renderChunk.test.ts: a chunk whose video is read by URL from an unreachable host fails with VideoSourceExtractionError while its capture browsers start (without the rethrow it surfaced as a capture-stage error).
  • The existing chunk tests (deferred video by file and by URL renders the same chunk, shown-size measurement, frame ranges) pass unchanged.
  • Each new test passed 3 runs in a row and was shown to fail with its code path disabled (factory not waiting, no rethrow).
• Manual testing performed
  • Heavy 4K fixture (full-screen screen recording plus camera bubble) on Lambda, this branch against main on the same base, two cold runs each: output byte-identical (same video and audio stream checksums), each chunk finished sooner and compute billed per render went down in both runs.
• Documentation updated (if applicable): not applicable.
• Comments follow CONTRIBUTING.md "Comments"

Not exercised: Cloud Run end to end (same renderChunk path).

[github-actions]

Edit accuracy: accurate 271 (base branch 271), smooth 154 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (1)

• seqplainkeys-none-px-r0-root-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo false, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true

[vanceingalls]

Reviewed at 08a8a114. I started at c42fa39a. 127fd9aa was a pure rebase onto a main that includes #6026 (git range-diff identical). 08a8a114 adds one commit on top of it, which deletes three comments and renames the capture .catch parameter to captureError, with no behaviour change. There are no other reviews.

What holds

• A first frame that waits on extraction can't trip a timeout on the chunk's capture path.
  • Chunks capture to disk. executeDiskCaptureWithAdaptiveRetry calls executeParallelCapture with no onFrameBuffer (renderOrchestrator.ts:1244-1264), so resolveParallelWorkerTimeoutMs returns 0 and the 30 s per-phase deadline is off (parallelCoordinator.ts:541-547, 846-848).
  • The 15 s drawElement frame deadline wraps only captureFrameToBuffer (frameCapture.ts:4421), the streaming path. Disk capture goes through captureFrame (:4367), which has no deadline.
  • So a slow 4K extraction read by URL holds the first injection without being mistaken for a stall.
• An extraction failure doesn't set off the screenshot retry. shouldRetryChunkCaptureWithScreenshot (renderChunk.ts:297) retries only on BeginFrame messages. A hook that rejects with the extraction error goes straight to the .catch, which awaits the extraction and throws its error. VideoSourceExtractionError carries its own kind and retryable (videoFrameExtractor.ts:445), so routing still sees the source's real retryability.
• The injector change is confined to cloud chunks.
  • Only renderChunk passes the factory a promise. The synchronous branch (chunkFrames.ts:59) is unchanged for every other caller.
  • Each session still gets its own stateful injector. The injector is created on the first injection, and a session injects frames one at a time, so two first frames can't race.
  • The hook is non-null only where it already was (a non-empty lookup), so static-frame dedup and drawElement self-verify see the same hook as before.
• Extraction is always settled before teardown.
  • On failure, the capture .catch awaits it before rethrowing.
  • On success, await deferredVideoFrames follows the capture stage.
  • finally covers anything thrown in between.
  • Extraction writes to ${workDir}.video-frames, beside workDir, so the screenshot retry's rmSync(workDir) can't delete frames still being written.
• Unhandled rejections: extraction's promise always has a handler before the first await. When the lookup is built, its .then is that handler. The validation throw that could reject the extraction with no lookup built ("read by URL but no deferred video uses it", chunkFrames.ts:221) can't be reached, because assertRemoteSources (planV2.ts:1104-1107) rejects such a source at materialization.
• Verified locally:
  • At 08a8a114 (and before that at 127fd9aa), renderChunk.test.ts + renderChunkFallback.test.ts pass 45/45.
  • At c42fa39a, the new extraction-failure test ran for real; it was not skipped for lack of Chrome. One URL byte-range test failed once in a full run at c42fa39a. It passed when run alone and passed in the full run at 127fd9aa.
• Mutation checks:
  • These each turn a test red:
    • dropping the await deferredVideoFrames in the capture .catch (the new chunk test then gets the capture error);
    • making the factory skip the wait (the new fallback test).
  • These each stay green, as expected, because they are defensive layers that the outer .catch and the .then handler already cover:
    • dropping the finally settle;
    • dropping videoFrameLookup?.catch;
    • dropping the success-path await.

Non-blocking

• This branch depends on #6037, which it doesn't contain. Booting browsers during extraction is the load that made a passing BeginFrame probe's slow page close flip one worker to screenshot mode. That fails the chunk's "one capture mode" check (renderChunk.ts, observedModes.size !== 1). #6037 (merged into main as a8b576d1) fixes it in probeBeginFrameSupport. The branch is still based on d67e81c7, so it is safe only as merged onto main: 08a8a114 merges cleanly with main at a8b576d1, which is #6037's own merge commit.
• Peak memory with #6026 on the same base isn't measured yet.
  • Before this PR, parallel chunks closed the preflight browser and ran ffmpeg alone before booting workers.
  • With #6026, a 10 GB function now boots 5 capture browsers at 2 GB each while the deferred 4K extraction runs, and #6026's 2 GB budget didn't count ffmpeg.
  • The first A/B in the body was on the pre-#6026 base (c42fa39a's parent). If the -4% heavy-4K figure was measured on this head, its Max Memory Used would settle it.
• The ChunkResult stage-timing contract is now out of date (renderChunk.ts:151-165). Extraction used to sit in the "remainder of durationMs". It now overlaps the capture stage, so its wait shows up inside captureStageMs. Fleet cost models that compare captureStageMs across versions will see a step change. One line in the doc comment would cover it.
• A capture failure now waits for the full extraction before surfacing. For example, a Chrome launch failure on a chunk with a long URL read now takes as long as the extraction. Before, extraction finished first, so total time is no worse than main, but the failure no longer comes back early. Aborting extraction on a capture failure could be a follow-up.

CI: the required checks are green at 08a8a114: Build, Render on windows-latest, Semantic PR title, Test, Test: runtime contract, Tests on windows-latest, Typecheck, regression, Studio and player captures, Test reachability and Studio: edit accuracy gate. Skills: manifest in sync was skipped by its path filter. No check failed; Comments passes now that the comments are trimmed.

Verdict: APPROVE
Reasoning: The first-frame wait holds no deadline on the disk capture path, extraction failures keep their own type and retryability, and extraction always settles before teardown. Both new tests are shown to catch their regressions. What's left is an unmeasured memory peak alongside #6026 and a timing doc.

— Via