Skip to content

Fix decoder repeating the first frame after a seek (preview N−1, trims shifted) - #162

Draft
StuartCameronCode wants to merge 1 commit into
mainfrom
agentbob/4-seek-alignment-decoder-repeats-the-first
Draft

StuartCameronCode wants to merge 1 commit into
mainfrom
agentbob/4-seek-alignment-decoder-repeats-the-first

Conversation

@StuartCameronCode

Copy link
Copy Markdown
Owner

Fixes #157.

Problem

The worker's source decoders write rawvideo, and ffmpeg gives that output constant-rate timing. After an input -ss to (N − 0.5)/fps, the first kept frame arrives half a frame late. On many time bases (MP4 1/2997, MOV 1/11988, MKV 1/1000, TS 1/90000), that timing emitted the frame twice. The effects:

  • Preview: with every pass off, the preview of frame N showed N−1, next to the app's correct "before" frame N (forum report).
  • Trimmed encodes: a trim of 41..52 came out 41, 41, 42 … 51. The first frame was doubled, the last was lost, and the picture ran a frame late against the audio.
  • Detectors: crop_detect, noise_halo and native_res seek the same way. Each noise pair was the same frame twice, so the noise score was under-reported.

Preview and encode shifted together, so the existing preview-vs-encode tests never saw it.

Fix

The fix is in worker/src/source_decode.rs:

  • setpts=PTS-STARTPTS (RESET_PTS_FILTER) goes first in the decoder's single -vf, and only when the decoder seeks.
  • A decoder that doesn't seek passes no -ss at all, not even -ss 0. Untrimmed encodes are byte-for-byte unchanged, and preview frame 0 is the encode's frame 0.
  • The VFR timeline decoder is untouched.
  • -fps_mode passthrough was deliberately not used, because it would drop the soft-telecine pulldown repeats that IVTC expects.

Other changes:

  • PreviewGenerator.beforeFrameArgs is factored out of getFrameAtIndex, so the test can check the app's own "before" frame.
  • Report: docs/reports/issue-157-seek-alignment/, with its probe scripts. Also an ENGINEERING_NOTES entry, a reports index row, and a CLAUDE.md rule under "Source Frame Size".

Known limitation

This only affects in-points inside leading AVI null frames. There, the reset rebases to the first stored frame after the gap, so the trim's tail can run up to 2 frames early. This is tracked with the other seek-position bugs below.

Measured:

  • Mid-stream gaps: on an FFV1 AVI with frames 100–104 dropped, the old decoder and the reset give identical frames from every start 97–106 (scripts/gap_probe.py).
  • Alternative tried: shifting by a fixed half frame (setpts=PTS-0.5/fps/TB). It fixes the same cases but rounds early in 1/25 time bases and breaks starts just before a mid-stream gap, so it was rejected.

Because of this, integration_frame_mapping_test's "from an in-point" case moves its in-point from 3 to 6, past the fixture's leading null frames. The offsets and padding phases are the same, and a comment explains the move.

Still out of scope: MPEG-PS/TS, MPEG-2 in AVI, soft telecine and leading null frames still seek to the wrong picture, for reasons unrelated to this fix. The report covers them, and each has its own follow-up.

Testing (macOS arm64, deps 1.16.0)

  • cargo test --lib source_decode: 11/11 pass. The new tests pin the reset ahead of the size guard, the single -vf, and the unchanged args when the decoder doesn't seek.
  • integration_seek_alignment_test.dart (new, heavy): 12/12 pass.
    • Fixtures: small_clip.mp4 (1/2997), seek_timebase_2997.mov (1/11988), and face_restore_test.mp4 as a 1/30000 control.
    • Checks: preview frames 10/41/77 and the app's before frame, plus a lossless trim of 41..52. Everything is compared against an independent select=eq(n,N) decode, never one pipeline path against another.
    • With only the reset disabled, exactly the 8 tests on affected sources fail and the 4 control tests pass.
  • Other heavy suites through the seeking decoders: 39/39 pass (--exclude-tags gpu): integration_frame_mapping, _vfr_source, _crop_detect, _noise_halo, _descale, _clean_aperture, _stabilize.
  • Not run locally: the rest of the heavy suite, and Windows/Linux. The decoder args are platform-independent, and nightly covers the rest.

🤖 Generated with Claude Code

…rst frame (#157)

The worker's preview and trimmed-encode decoders seek to (N - 0.5)/fps and
write rawvideo, whose constant-rate timing emitted the first kept frame twice
on many time bases (MP4 1/2997, MOV 1/11988, MKV 1/1000): the preview showed
N-1 and a trim of 41..52 came out 41, 41, 42 .. 51. setpts=PTS-STARTPTS now
leads the decoder's single -vf whenever it seeks; a decoder that doesn't seek
passes no -ss and is unchanged.

- PreviewGenerator.beforeFrameArgs factored out of getFrameAtIndex
- integration_seek_alignment_test.dart (heavy) checks preview, before frame
  and a trimmed FFV1 encode against an independent select=eq(n,N) decode
- new fixture seek_timebase_2997.mov (1/11988) + generator script
- frame-mapping in-point test moved past interlaced_test.avi's leading null
  frames (3 -> 6), the one case the reset changes (task 16)
- report docs/reports/issue-157-seek-alignment, ENGINEERING_NOTES, CLAUDE.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Preview one frame behind and trimmed encodes shifted on many MP4/MOV/TS sources (decoder repeats the first frame after a seek)

1 participant