Skip to content

fix(engine): passthrough keeps frame count and accepts films whose audio outlasts the picture - #6920

Merged
miguel-heygen merged 4 commits into
mainfrom
fix/video-layer-tail-frames
Nov 1, 2026
Merged

miguel-heygen merged 4 commits into
mainfrom
fix/video-layer-tail-frames

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

What

Fixes two ways the opt-in video layer passthrough (#6914, off by default) mishandled a real film whose audio track is a little longer than its picture, which most exported films are (refs #6881).

  1. A video shorter than the timeline by even two frames was refused (source_unsupported), so passthrough never ran on such films. Up to half a second of shortfall is now allowed, and the video's last frame is held for the rest, as extraction does.
  2. When it did run, the output was one frame short of today's (2649 against 2650 on an 88 s film). A -t after the video input is an output option and cut the file at the video's length. It is removed, and the encoder now ends when the page frames end (overlay=eof_action=endall), so the output length is decided by the page frames alone.

fails without the fix:

  • videoLayerInput.test.ts › "accepts it, holds the last frame and leaves the output length to the page frames" (expected undefined to be defined on main)
  • videoLayerInput.test.ts › "encodes exactly one output frame per page frame when the video is shorter": a real ffmpeg run with 74 page frames and a 2.0 s video gives 74 frames with the fix and expected 73 to be 74 with the -t, padding and end rule taken out (the output was cut at floor(duration x fps) frames)

Real-footage check

Devbox, one worker and four pinned cores in both arms, three runs per arm, each run started once load was under 8 (box load was 9 to 12 by the end of each run, so treat the times as indicative). Each film is the full-canvas source under a caption every five seconds. The sources are public films hosted for the OSS docs; none is from a private project.

Film Source path Today (s) Passthrough (s) Frames today / passthrough PSNR avg (min) SSIM
88 s, 1920x1080, 30 fps static.heygen.ai/hyperframes-oss/homepage/showcase/hq/explainer-code2video-bench-v1.mp4 231, 248, 225 103, 98, 132 2650 / 2650 50.4 (43.7) 0.9974
55 s, 1920x1200, 60 fps screen recording static.heygen.ai/hyperframes-oss/docs/studio-demos/first-video-demo-v2.mp4 317, 384, 343 95, 89, 96 3302 / 3302 48.1 (43.8) 0.9975
43 s, 1920x1080, 30 fps static.heygen.ai/hyperframes-oss/docs/images/prompting/variant-pr-changelog.mp4 112, 126, 111 37, 40, 37 1293 / 1293 52.3 (49.3) 0.9987

I looked at the frames at 25, 50 and 75 percent of each film side by side; they match, and sampled pixel values in the caption boxes agree within a few levels.

The repo and catalog carry no real 1080p, 4K or long footage of their own, so these three hosted films stand in. There is no 4K film in this check.

Size

Under 100 changed lines plus one test file.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE · 0 blocker · 1 important · 0 nits · head 88f065c5

The named test fails at main for fix 1. Fix 2, the frame count, has no test. ↓ inline

Scope

Read the full diff, plus qualifyVideoLayer and planVideoLayerPassthrough (the only caller). The caller passes the composition duration, and only a video that starts at 0 and runs to the end qualifies, so removing -t can't play the source past its element. videoLayerInput, streamingEncoder and videoLayerPassthrough tests pass locally (99/99). I checked the filter graph directly with ffmpeg, using a transparent page stream over a 2 s film with 2.4 s audio and over a 5 s film: 72 page frames → 72 output frames, 61 → 61, and the short film's tail holds its last frame. No prior reviews. CI was still running at review time; nothing had failed.

@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 121 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (1)

  • seqplainkeys-none-pct-r0-root-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo false, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE — 1 resolved, 0 open, 0 new · 88f065c5 → 15294c68. The frame-count fix is now covered by a real ffmpeg encode: with eof_action=endall removed it fails with 90 ≠ 74; videoLayerInput and streamingEncoder tests pass at this head (91/91).

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE — 0 resolved, 0 open, 0 new · 15294c68 → 5294ff7b. The only change adds -- before the ffprobe input in the test. The test still passes at this head and still fails (90 ≠ 74) without eof_action=endall.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Nov 1, 2026
Merged via the queue into main with commit 58428b4 Nov 1, 2026
91 checks passed
@miguel-heygen
miguel-heygen deleted the fix/video-layer-tail-frames branch November 1, 2026 10:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant