Skip to content

fix(studio-server): proxy MPEG-4 Part 2 and MJPEG video the browser cannot decode - #7310

Merged
miguel-heygen merged 2 commits into
mainfrom
drenderq/proxy-undecodable-codecs
Nov 6, 2026
Merged

miguel-heygen merged 2 commits into
mainfrom
drenderq/proxy-undecodable-codecs

Conversation

@miguel-heygen

Copy link
Copy Markdown
Collaborator

What changed

BROWSER_HOSTILE_CODECS gains mpeg4 (MPEG-4 Part 2: OpenCV's default mp4v, ffmpeg -c:v mpeg4) and mjpeg, both with no representative mime and pre-warm on, as prores is. A project video in either codec now gets a browser-playable copy instead of "no copy" (browser_safe_codec).

Why

Chrome refuses both inside an mp4 (<video> error code 4, readyState 0), but the table lists only hevc, prores, theora, av1 and vp9, so the studio server answers the copy request with 404 and the player stays blank. Reproduced: ffmpeg -f lavfi -i testsrc2=size=640x360:rate=30 -t 1 -c:v mpeg4 m.mp4 (and -c:v mjpeg -pix_fmt yuvj420p) in Chrome 156 on macOS: error code 4; the same clip as H.264 plays. H.264 4:4:4 and High 10 also play, so they are not added.

What I measured

  • fails without the fix: packages/studio-server/src/helpers/mediaCodecMap.test.ts, 4 failures with mediaCodecMap.ts at main (the mpeg4 and mjpeg mime and pre-warm cases), 41 pass with it (bun install, build, vitest on a Linux box).
  • Browser check as above, one Chrome, macOS.

What I did NOT exercise

  • No end-to-end proxy transcode of an mpeg4 or mjpeg file through the server route; the table is what decides eligibility.
  • Other codecs Chrome refuses (mpeg2video, h263, wmv, vc1, dv) are not added: not tested here.

Size

About 14 changed lines in 2 files: a lone table change with its test; nothing open on this table to carry it.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at a34b58f. Requesting changes. There's one blocker.

Blocker: the Comments workflow fails. In mediaCodecMap.ts, comment lines go from 87 to 88 (19.4% → 19.5%) because of the new // MPEG-4 Part 2 … Chrome answers error code 4. line. Drop it. The PR body and the test names already carry that context.

The rest checks out:

  • mjpeg doesn't catch cover art or stills. probeCodecFacts only runs when kind === "video", the kind comes from the file path, and metadataFromProbe skips attached_pic streams. So an mp3 or m4a with embedded art, or a .jpg, won't trigger a transcode.
  • representativeMime: null means Safari, which can play MPEG-4 Part 2, also gets a copy. That's the same trade-off prores already makes, so it's acceptable.
  • The fails without the fix: line names mediaCodecMap.test.ts and its new mpeg4/mjpeg cases, which fail against main's table.

@miguel-heygen

Copy link
Copy Markdown
Collaborator Author

Head a3371e9 drops the comment line; the table change and its test are unchanged.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at a3371e9. Approving.

  • The only change since a34b58f removes the one comment line that the Comments check counted. mediaCodecMap.ts now adds no comment lines over its base.
  • My earlier findings still hold. mjpeg doesn't catch cover art or stills, because the codec is only probed for video-kind files and attached_pic streams are skipped. The fails without the fix: line is valid.

All workflows, including Comments, were still queued on this head when I reviewed. This approval doesn't stand in for them.

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 267 (base branch 267), smooth 114 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (8)

@miguel-heygen
miguel-heygen added this pull request to the merge queue Nov 6, 2026
Merged via the queue into main with commit f3e45b4 Nov 6, 2026
81 checks passed
@miguel-heygen
miguel-heygen deleted the drenderq/proxy-undecodable-codecs branch November 6, 2026 10:16
SanHsien added a commit to SanHsien/hyperframes that referenced this pull request Nov 6, 2026
…view to heygen-com#7250

Adopts heygen-com#7310 (MPEG-4 Part 2 / MJPEG get browser-playable copies), heygen-com#7316
(check_report counts the copies a check waited for) and heygen-com#7317 (render reads
a composition src like the preview and refuses paths outside the project).
Defers heygen-com#7309 (needs heygen-com#6912); heygen-com#7314 does not apply. PR axis advances to heygen-com#7250
(heygen-com#7243 closed unmerged, heygen-com#7249 open and pinned as monitored; monitored 96->97).

Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant