Repository navigation
fix(producer): compile probes each unique media source once however many elements use it - #6441
Conversation
There was a problem hiding this comment.
Verdict: REQUEST CHANGES · 1 blocker · 0 important · 0 nits · head 9d791823
The compiler change is correct: I reran the new test against main's htmlCompiler.ts (24 probes for one source, fails) and at this head (passes), and the mediaType, hiddenProbe and naturalDuration suites pass locally. The one blocker is that the new test runs in the wrong CI lane, so the producer unit tests fail. ↓ inline
Scope
Read all of htmlCompiler.ts diff, its 3 call sites and the advisory pass. The probe map is keyed on the absolute resolved path and scoped to one compile. The probe never rejects, so nothing goes unhandled. withSrcContext still wraps each element separately. No prior reviews. Other CI jobs were still running when I posted this.
— Claude
Edit accuracy: accurate 271 (base branch 271), smooth 156 of thoseThe gate passes. Quarantined, measured but not gated (0) Unstable (1)
|
fbac394 to
8979eca
Compare
8979eca to
2deec47
Compare
2deec47 to
532737b
Compare
What was wrong
Compiling a composition probed a media source once per element, not once per file. A source with no content identity (anything the compiler reads in place from a URL) shares only its in-flight probe, and the shared four-slot limiter queues the other probes, so none of them is in flight together and each one starts a new ffprobe. The compile then runs those probes in rounds of four, each over the network, and a long composition with many video elements of the same few files spends its time re-reading headers, so a slow link multiplies the wait by the element count.
Measured on a project of 40 sub-compositions, each with one muted
<video>over 3 sources served over HTTP (the new test): 92 ffprobe runs on main, 9 here (two per remote file: one format read, one keyframe scan, plus one for each absent local path the advisory pass tried).Fix
compileForRendercreates oneSourceProbesmap and hands it to every pass that resolves a media duration (the page, each sub-composition, the audio/video clamp). A unique(tag, resolved source)is probed once behind the existing four-slot limiter; every element reads its own duration from that result, and the per-element checks (media type of the element, the error naming itssrc) still run per element. The advisory keyframe pass skips a source it already queued.assetMediaType.tsalready groups by unique path (byPath), which is the pattern this follows.Design calls:
[src=...].Test
htmlCompiler.probeCount.test.tsbuilds the 40-element project above and counts the ffprobe processes per source through a logging shim: at most 2 per file. On main it received 24 for one source (expected at most 2); with the fix 3 runs in a row pass. The existing limiter test now uses a distinct source per sub-composition, since it relied on one file being probed eight times.htmlCompilersuites (mediaType, hiddenProbe, scriptOrder, naturalDuration, animatedGif, parity, fontEmbed, moduleScripts, main): all pass.