Skip to content

fix(engine): read a video's length from its packets when headers state none - #7233

Merged
miguel-heygen merged 4 commits into
mainfrom
dcloudpro/headerless-duration
Nov 5, 2026
Merged

miguel-heygen merged 4 commits into
mainfrom
dcloudpro/headerless-duration

Conversation

@miguel-heygen

Copy link
Copy Markdown
Collaborator

Problem

A WebM or Matroska file written without a duration header (screen and browser recorders write to a pipe) probes with no duration. The engine took that as "no video stream length", refused the clip with invalid_media, and the render failed with VIDEO_SOURCE_UNRENDERABLE before capture started. The same VP9 clip with a duration header renders.

Change

extractMediaMetadata now reads the length from the video packets (last packet end) when both the container and the stream state none. Files that state a duration are untouched; the scan only runs when both are missing.

Verified

  • Real repro with the released CLI: a VP9 gbrp Matroska clip written to a pipe fails extraction with invalid_media=1; with a duration header it extracts.
  • With this change the same file probes as 3 s (gbrp vp9 3 3).
  • ffprobe.test.ts: 129 pass; the new test fails on main (expected +0 to be close to 2.9).
  • Formatted with oxfmt and typechecked the engine package; the repo's lint hook was not run (no local install).

fails without the fix: a video whose headers state no duration is measured from its packets, so a recorder's WebM or Matroska is not refused

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes: one blocker at e0a5533d.

The fails without the fix: line is present and checks out. I ran the new test against main's ffprobe.ts and it fails with expected +0 to be close to 2.9. It passes on this head.

Blocker: scanPacketDuration can return a length many times too long for any clip longer than about 100 s

runFfprobe(..., { retainTail: true, maxChars: 64 * 1024 }) keeps only the last 64 KiB with stdout.slice(-maxChars), so the first retained line is usually cut partway through. The other tail callers only read .at(-1), so a cut first line doesn't affect them. This function takes Math.max over every line, so the cut line counts. If the cut lands just after a decimal point, a pts like 1x.3000 becomes 3000, and that wins the max.

At 30 fps, each pts_time,duration_time line is about 19 chars, so any clip longer than about 110 s goes over 64 KiB. That covers most real screen recordings, which is the case this fix targets.

To reproduce it, I used the real ffprobe packet output from a VP9 Matroska file piped out of ffmpeg, cut to 5,993 frames (199.77 s). I fed it through extractMediaMetadata using the PR's own spawn-mock pattern:

AssertionError: expected 3000.033 to be close to 199.766, received difference is 2800.267

So a 3-minute recording gets measured as 50 minutes. Other lengths land wherever the cut falls: some come out exact, some are wildly too long.

Fix: when the output reached the cap, drop the first (partial) line, for example if (stdout.length >= 64 * 1024) lines.shift(). Or have runFfprobe's tail mode trim to the first \n after slicing. Please add a test with more than 64 KiB of packet lines whose cut lands after a .. The fixture above is an easy way to build one: real packet lines up to frame 5,993.

Non-blocking

  • Matroska SimpleBlocks with no DefaultDuration (common in browser MediaRecorder output) can report duration_time=N/A on every packet. The filter would then throw away every line and the clip would still be refused. Consider falling back to the max pts_time plus one frame (1/avg_frame_rate) when no packet has a duration.
  • CI at this head: Comments failed, and CI, regression, Windows render, and CodeQL were still running when I reviewed.

@miguel-heygen

Copy link
Copy Markdown
Collaborator Author

Thanks, good catch. Fixed at 300ee06: the scan drops the first line when output reached the cap, and a packet with no stated duration (N/A) counts as one frame (1/fps), so recorder files with N/A on every packet are measured too. New test builds >64 KiB of N/A packet lines with the cut landing right after a decimal point: without the line-drop it reads 633333 s for a 167 s clip, with it 166.9 s. ffprobe.test.ts 130 pass.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at 300ee066. The blocker from my last review is fixed.

  • The cut-off first line is now dropped. I ran my 199.77 s fixture again: real ffprobe packet output, 5,993 frames, cut after a .. The last head measured it as 3000.033. This head measures it as 199.77 s. The new half line test fails on e0a5533d (expected +0 to be close to 166.9) and passes here. A run that ends exactly at the cap without being cut loses one complete first line. That's harmless, because the max is always near the end.
  • A packet with no duration now counts as one frame (1/fps), which covers MediaRecorder SimpleBlocks. A line whose pts is N/A is still skipped. An empty line parses as 0 + frame, which never wins the max.
  • ffprobe.test.ts locally: 127 pass. The 4 HDR-PNG/cICP failures happen the same way on main in my sandbox, so they come from my environment, not this PR.

Non-blocking:

  • The Comments check is red: comment-ratchet reports ffprobe.ts rising from 13.2% to 13.3% (6 comment lines added). It isn't one of main's required checks. Cutting a couple of comments would turn it green.
  • Nit: the /** The video stream's length… */ JSDoc now sits on PACKET_TAIL_CHARS instead of scanPacketDuration.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at 1430b87f. I diffed this head against 300ee066, which I approved: the only change is to comments in ffprobe.ts. Two inline comments are removed, and the scanPacketDuration JSDoc is moved back onto the function, which settles my earlier nit. No code or test lines changed, so my verification from 300ee066 still applies. Comments is now green.

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 267 (base branch 267), smooth 103 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (8)

Unstable (1)

  • seqplainkeys-none-pct-r0-nested-z100: tracking 0.05, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo false, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at c9035284. I diffed this head against 1430b87f, the last head I approved. The only change rewraps one line in ffprobe.test.ts, the destructured await import("./ffprobe.js"). No code or test logic changed, so my earlier verification still applies. oxfmt --check (0.41.0) passes on both changed files at this head.

On the previous head, the failing jobs were CI Format (format:check), plus the Preflight (lint + format) jobs in the Windows render and regression runs. The Windows and regression jobs downstream failed on their Require … preflight / Check results gates. That matches the format-only explanation.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Nov 5, 2026
Merged via the queue into main with commit 2af37ac Nov 5, 2026
90 checks passed
@miguel-heygen
miguel-heygen deleted the dcloudpro/headerless-duration branch November 5, 2026 12:11
@heygengenesis heygengenesis Bot mentioned this pull request Nov 5, 2026
mitanshu450 pushed a commit to mitanshu450/hyperframes-video-generation that referenced this pull request Nov 5, 2026
* chore: release v0.9.16

* chore: format v0.9.16 release notes

* chore: add heygen-com#7228 and heygen-com#7231 to v0.9.16 notes

* chore: add heygen-com#7217 heygen-com#7224 heygen-com#7225 heygen-com#7226 heygen-com#7229 heygen-com#7233 to v0.9.16 notes
SanHsien added a commit to SanHsien/hyperframes that referenced this pull request Nov 6, 2026
…eview

Adopts or partially adopts 11 groups: multi-worker tween replay behind
the timeline stand-in (heygen-com#7231), template entries in the shared compile
stage (heygen-com#7217), HYPERFRAMES_CACHE_DIR and HYPERFRAMES_CONFIG_DIR for
every per-user cache and settings file (heygen-com#7224, heygen-com#7229; HeyGen
credentials stay under HEYGEN_CONFIG_DIR), WebKit audio clock (heygen-com#7225),
packet-based video length (heygen-com#7233), HLG tone mapping (heygen-com#7234), fetch
network cause (heygen-com#7236), .hyperframesignore in lint (heygen-com#7239), info
--media (heygen-com#7235), lint priority (heygen-com#7244), check --budget (heygen-com#7245) and the
unknown-command answer (heygen-com#7247). bun.lock gains only the lint `ignore`
edge from heygen-com#7239, already resolved. Watermark advances to 0a6746e.

Co-authored-by: Jerrai <jerrai@heygen.com>
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
Co-authored-by: WaterrrForever <miao.yang@heygen.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant