[miguel-heygen]

What

At a cut between two <audio> clips, the incoming clip now starts at its in-point and is never seeked while it sounds. Its volume lane and fades stay on the audio they shape, including after pause and play, a seek, a rate change and a loop.

Why

A user's detailed report measured this on edits with one <video> and one <audio> per take, with overlapping audio windows at each join:

• The incoming clip started about 55 ms late and was then seeked forward 20-125 ms once or twice in its first 300 ms. This happened at 27 of 30 joins, and the seeks were audible as clicks or clipped syllables.
• The data-automation volume lane landed up to 310 ms early, and up to 350 ms late after pause and play. Edge fades and make-up gain applied to the wrong stretch of audio.

There were three causes:

1. Only <video> was pre-seeked to its in-point before its window opened. Audio got its first seek at window open and started late.
2. Playing audio that drifted past 40 ms was corrected with a seek, which jumps the sounding signal.
3. A routed clip's lane and fades were booked once on the audio-context clock when the clip was routed, up to 10 s ahead. Nothing re-aimed them when the element started late, was seeked or was steered. On a rate change, the context-clock plan was re-aimed instead of the element.

Related work

Depends on #6432. This branch is stacked on it: every currentTime write here goes through its mediaElementTime, including the dipped seek this PR adds. Merge #6432 first, then this; until then the diff includes #6432's commits.

Refs #5344 (same family: the audio gap at a split). Builds on #5633 (video pre-seek) and #4511 / #6198 (rate steering). The player's parent-frame proxy (#6417) imports the drift policy exported here (@hyperframes/core/runtime/media-steer).

Design calls

1. Getting the incoming audio to its in-point

• (a) Pre-seek upcoming <audio> in the same preparation path video uses.
• (b) Have the decoded-buffer route own clips that share a file.
• Picked (a). It is the existing owner, it is one condition removed, and it keeps the pitch-preserving media-element route. (b) would split one file's clips across two routes with different rate behaviour.

2. Correcting drift on audio that is sounding

• (a) Seek, as before. This is the click.
• (b) Steer by rate with video's steps (3% / 10% / 25%). At 25%, speech audibly changes tempo.
• (c) Nudge by rate, never past 10%. When the lag holds past 100 ms (more than 10% closes in about a second), seek under a 15 ms gain dip: down, seek, and back up when the element lands.
• Picked (c), the best-sounding option. With the pre-seek in place this should almost never fire.
  • The dip lives on a gain stage the routed graph owns.
  • An element not routed through Web Audio has no stage to dip, so it is seeked plainly.
  • The dipped seek waits for the lag to stop growing, so an element stalled on data is not chased with seeks (the same rule strict sync uses).
  • Video keeps its steps.

3. Keeping a routed clip's lane and fades on its audio

• (a) Write the lane per tick. This is 50 ms apart, which is coarser than a 30 ms fade and is the stepped sound fades were moved off.
• (b) Move routed clips to decoded buffers, which are sample-accurate. This loses pitch-preserving rate and rate curves.
• (c) Keep the envelopes booked on the audio clock, and re-aim them at the element's own position whenever it starts, is seeked or changes rate.
• Picked (c). The element owns the clip's time, so it owns the mapping too.
  • followElement is the single place that maps the element's position to clip seconds.
  • setRate no longer re-aims routed clips on the context clock, so the clip has one writer, not two.
  • A looping element's wrapped position is placed on the pass the transport is in, so a looping bed's ducking lane follows too.
  • A pause needs nothing, because no audio flows until playing re-books.

4. Re-aiming without side effects

reanchor cleared limiter and compressor state, which is right for a new pass but audible on every rate nudge. The FX handle now has retime, which re-aims the envelopes only. reanchor is a reset followed by retime.

How

• media.ts:
  • prepareUpcomingMedia (was prepareUpcomingVideo) pre-seeks any paused clip within MEDIA_PRESEEK_SECONDS of its start. init.ts's indexed pass now visits upcoming audio too.
  • Playing media is steered by rate. For audio the steer is capped at 10%, and a held lag past 100 ms calls seekUnderDip.
• webAudioTransport.ts:
  • The media-element route puts a dip gain first after its source.
  • followElement re-books the lane and fades on playing, seeked and ratechange. The timing is timeAtSourceTime(spec, currentTime - mediaStart) at playbackRate / rateAt(...), with loops unwrapped.
  • seekUnderDip ramps the dip down over 15 ms, seeks, and ramps back up when the element lands (or errors). If the clip is stopped in the meantime, it does nothing.
  • setRate handles decoded sources only.
• audioFx.ts: adds retime.

The decoded-buffer route is unchanged. Its source and its lanes already share the context clock.

Test plan

• Unit tests added/updated. On origin/main, the new audio cases fail and every video variant passes:
  • audio pre-seek: sync-level and real init/index-level
  • playing audio nudged, not seeked
  • lane re-booked on a late start, on seek, on rate steer, and on a looping pass
  • fades re-booked
  • the 10% cap
  • the dipped seek, its plain fallback, and its stop guard
• One mutation per guard, each failing its test:
  • retime resetting effect state
  • the paused guard
  • unfollow on stop
  • the fade re-book
  • the loop unwrap
  • a second writer on rate change
  • the dip latch
  • the dip stop guard
  • the 10% cap
  • the dip while the lag grows
  • the plain-seek fallback
• packages/core src/runtime passed 3 runs in a row: 104 files, 2561 tests, exit 0. Core tsc --noEmit exits 0. oxlint reports 0 warnings and 0 errors on src/runtime. oxfmt --check is clean.
• Manual testing performed (see below)
• Comments follow CONTRIBUTING.md "Comments"

The local pre-commit lint, format, typecheck, fallow and commitlint hooks were skipped because this checkout has no node_modules. lint, format and typecheck ran clean on a remote copy (above). fallow and commitlint are left to CI.

What I measured

Measured in headless Chrome on Linux with this branch's runtime and with origin/main's (the "base" rows), built from the same tree.

The compositions are shaped like the report's two edits, built with our own media:

• One joined file. Each take has one muted <video> and one <audio> on that file.
• Audio windows overlap each cut by 125 ms before and 165 ms after.
• Each <audio> has a data-automation volume lane with 30 ms edge fades and a make-up gain of 1.36-2.04.
• 3-take edit: 24.8 s at 30 fps. 10-take edit: 53.08 s at 25 fps. Keyframe every 1 s.

The modes:

• play: straight through.
• pause: pause 0.3 s before the second cut, hold 0.8 s, then play.
• seek: while playing, jump to 1 s before the second cut.

The method follows the report:

• Every <audio> seek is logged with the playhead at that moment.
• Every AudioParam booking is logged.
• Each element's currentTime is sampled against AudioContext.currentTime every animation frame.
• A lane point is scored when it actually fires: its booked context time is compared with where its element was at that moment.
• Points that fire while the element is not yet advancing shape no audio, so they are not scored. paused turns false at play(), before audio flows.

edit mode build	runs	joins at in-point 50 ms early	joins with a mid-play seek in 600 ms	lane points (sounding)	median / p95 / max error ms	over 10 ms
ten play base	3	0/27	15/27	68	46.0 / 302.7 / 315.1	62
ten play branch	3	27/27	0/27	89	0.8 / 1.9 / 1.9	0
three pause base	3	0/6	2/6	6	60.7 / 70.6 / 70.6	6
three pause branch	3	6/6	0/6	12	0.5 / 2.6 / 2.6	0
three play base	5	0/10	1/10	34	44.2 / 106.3 / 106.4	28
three play branch	5	10/10	0/10	44	0.6 / 1.3 / 1.4	0
three seek base	3	0/6	1/6	6	71.4 / 93.0 / 93.0	6
three seek branch	3	6/6	0/6	11	0.6 / 2.5 / 2.5	0

Across all runs, seeks of a playing <audio> drop from 59 on base to 1 on this branch.

• The one left is the play after the pause. The transport lands the element 8 ms before it sounds; the seeking event fires after play() has already un-paused it.
• The dipped seek never fired, which is the expected case once incoming clips are pre-seeked.

Before

One join of the 10-take edit on main, measured as above. On main, 15 of 27 joins on this edit had a mid-play seek; this is the first of them in run 0.

• Top: the incoming audio falls 52 ms behind its schedule and is seeked again 93 ms after its window opens, while it sounds.
• Bottom: every volume-lane point that shaped audio in that run. They land up to 303 ms early.

After

The same join and run with this branch's runtime. The incoming audio stays within about 7 ms of its schedule with no seek, and every lane point lands within 2 ms of its audio.

What I did NOT exercise

• Output-burst energy at joins (the report's 12 dB check) was not measured. That needs a recording of the destination, which the shared playback harness will add.
• Safari and Firefox.
• A real audio device. Headless Chrome ran on Linux with no sound card.

[github-actions]

Edit accuracy: accurate 271 (base branch 271), smooth 153 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (1)

• seqplainkeys-none-pct-r0-root-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo false, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true

[heygengenesis]

Verdict: APPROVE · 0 blocker · 1 important · 0 nits · head ed235f55

• important (follow-up, not a regression): packages/player/src/parent-media.ts:236. When iframe autoplay is blocked, the parent proxy owns audible playback, and mirrorTime still seeks a playing <audio> after 2 samples over 50 ms. That is the click this PR removes from the runtime. steerMedia is now exported through ./runtime/media-steer for this, but nothing calls it yet. Worth wiring in a follow-up.

Scope

Read the full diff (13 files). The 6 touched runtime suites pass locally, 339/339. Mutations each fail a test: the dip stop guard, unfollow on dispose, the audio pre-seek, playing audio routed through strict sync, and setRate writing to routed fx. All 9 workflows on this head are green (69 jobs). No other callers of the renamed VIDEO_PRESEEK_SECONDS / prepareUpcomingVideo. Not run live in a browser.

— Rames

[heygengenesis]

Verdict: APPROVE · 0 resolved, 1 open, 0 new · ed235f55 → 290483a6

Rebased onto #6432 (09f15140). range-diff: commits 1 and 5 differ from ed235f55 only in their #6432 context. Commits 7 and 8 send the dipped seek through mediaElementTime and update its tests. Runtime suites pass locally, 344/344. PR captures is red: #6432's player files are in this diff, and this body has no Before/After. It should clear once #6432 merges and this rebases. Merge #6432 first. Merging this now would squash #6432 in with it.

— Rames

[heygengenesis]

Verdict: APPROVE · 0 resolved, 1 open, 0 new · 290483a6 → c791f520

range-diff: all 8 commits identical. Only #6432 moved (2364387b, mediaElementTime now leads by 10 µs). Runtime suites pass locally, 344/344. PR captures is still red from #6432's player files, as before. Merge #6432 first.

— Rames

[heygengenesis]

Verdict: APPROVE · 0 resolved, 1 open, 0 new · c791f520 → 5b520a7a

range-diff: all 8 commits identical. Only #6432 moved (36602a8d, generator and serializer files, none shared with this PR). Runtime suites pass locally, 343/343. All 9 workflows are green, PR captures included. #6432 is still unmerged, so merge it first.

— Rames

[heygengenesis]

Verdict: REQUEST CHANGES · 0 resolved, 1 open, 1 new · 5b520a7a → 67a685a2

The range-diff shows all 8 commits identical. Runtime suites pass locally (2582/2582), and #6432 has merged.

• New blocker — CI Build fails on this head (run 40399621376, verify:packed-manifests): @hyperframes/inspector packed size 817397 bytes is over its budget of 808905 by more than 1%. Main 933ae3eb passes Build, and this PR's runtime code is bundled into inspector, so the rebase pushes it over. The fix is a budget raise in scripts/package-size-budgets.json with the reason, as in #6457. Measured size plus 5% gives packed 858267. Re-measure unpacked too. Studio's edit-accuracy failure (0 of 5 shards) comes after Build.

[miguel-heygen]

Head 661a098 (rebased on main 1df003d). The Build failure reproduces: this PR's runtime is bundled into the inspector package, whose packed size is 817397 bytes, over its 808905 budget. Raised the inspector packed budget to 858267 (measured plus 5%, as in #6457); verify:packed-manifests now passes on a full build ('Verified 14 packages are within their size budgets'). Unpacked was not flagged, so it is unchanged. Runtime suites still 106 files / 2582 tests.

[heygengenesis]

Verdict: APPROVE · 1 resolved, 1 open, 0 new · 67a685a2 → 661a0982

The range-diff shows commits 1–8 identical. Main 933ae3eb..1df003df touches none of this PR's files. The new commit raises the inspector packed budget to 858267, which is the failing 817397 × 1.05, rounded up. Build's verify:packed-manifests now passes, and all 9 workflows are green. Windows engine-cli failed once on the known chromeOrphans.process.test.ts flake and passed when the failed job was re-run.

[miguel-heygen]

Head 512a3d1, rebased on main 75bee42. Dropped my inspector budget edit: #6460 merged with 858046, which covers this PR's measured 817397 packed bytes. No other change; runtime suites 106 files / 2589 tests.

[heygengenesis]

Verdict: APPROVE · 0 resolved, 1 open, 0 new · 661a0982 → 512a3d1f

The range-diff shows commits 1–8 identical. The inspector budget commit was dropped because #6460 raised that budget on main to 858046. Main's only change to this PR's files is #6460's compositionFps in init.ts, which is independent of this PR. src/runtime passes locally (2589/2589), and all 9 workflows are green, Build included.