Skip to content

refactor(engine): one builder for the video frame filter chain - #6908

Merged
miguel-heygen merged 2 commits into
mainfrom
feat/video-layer-passthrough
Nov 1, 2026
Merged

miguel-heygen merged 2 commits into
mainfrom
feat/video-layer-passthrough

Conversation

@miguel-heygen

Copy link
Copy Markdown
Collaborator

What

Moves the ffmpeg filter chain that decides which source frame a video slot shows and how its colour is converted (frame sampling, tick-rounding fix, aspect ratio, HDR/SDR conversion) out of the extraction function into one exported builder, videoFilterChain. Extraction calls it and produces the same filters in the same order as before.

Why

A follow-up (refs #6881) lets ffmpeg composite the source video itself instead of round-tripping every frame through JPEG and the page. That path has to pick the same frames and colours as extraction, so both must call one builder rather than keep two copies that drift.

Tests

No behaviour change: the 9 existing videoFrameExtractor test files pass (289 passed, 1 skipped), typecheck and format clean.

Size

About 70 changed lines, one file.

@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 105 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (2)

  • seqplainkeys-none-center-r0-root-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true
  • seqplainkeys-none-px-r0-root-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo false, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes at 25ee0f12. The refactor itself is correct, but it turns CI red.

Blocker: the CI Fallow audit job fails, and this PR causes it. The report finds 2 new dead-code issues:

  • fallow/unused-type: VideoFilterChainInput (videoFrameExtractor.ts:1125)
  • fallow/unused-export: videoFilterChain (videoFrameExtractor.ts:1139)

Both are exported, but nothing outside this module imports them until the #6881 follow-up lands. Either fix is enough:

  • Drop export from both for now. The follow-up can add it back when it imports them.
  • Or mark them the way the repo already does: // fallow-ignore-next-line unused-export (used 9 times in packages/) and unused-type for the interface.

Fallow is not a required check, so the merge loop would merge with it red. That is why I'm asking for the fix here rather than letting it land.

The refactor is equivalent (checked):

  • Order. Before: [nv12?] + sampling(seek) + [SAR, sdr→hdr, tone-map, sdr canvas (+ halve-to-cover, bt601) for jpg], with sampling spliced in at samplingAt (0 or 1). After: head + sampling(seek) + tail, which gives the same filters in the same order.
  • Tone map. toneMapFrom !== undefined matches the caller's toneMappedToSdr exactly. readFirstFrameColour → parseFirstFrameColour always returns an object (at worst {}) and never undefined. It is still read only when tone-mapping, at the same point as before.
  • Tests. The 9 videoFrameExtractor* test files give 289 passed and 1 skipped, which matches the PR body. tsc --noEmit on the engine package is clean.

Nit, not blocking: the shortened sdrToHdrTransfer comment drops the reason it is safe today ("the remap only applies to SDR sources, nv12 only to HDR sources, so the flags are mutually exclusive"). That line is worth keeping.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at b45a5812. My blocker from 25ee0f12 is resolved.

  • What changed since 25ee0f12. The only edits are dropping export from videoFilterChain and VideoFilterChainInput, plus the comment. The PR diff is still the single file videoFrameExtractor.ts. No filter logic changed, so the equivalence I checked last round still holds.
  • Fallow is green on this head, and so is Lint.
  • The comment nit is addressed. The sdrToHdrTransfer note again says the remap never meets nv12, because nv12 is HDR-only.
  • Local checks are unchanged. The 9 videoFrameExtractor* test files give 289 passed and 1 skipped, and tsc --noEmit on the engine package is clean.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Nov 1, 2026
Merged via the queue into main with commit 94849bf Nov 1, 2026
90 checks passed
@miguel-heygen
miguel-heygen deleted the feat/video-layer-passthrough branch November 1, 2026 01:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant