Repository navigation
[feat] Turn reference videos into editable shot breakdowns #171
Description
I'd like to take the video-understanding part of item 7 in #107.
The first version would turn a reference video into an editable shot list:
- Shot boundaries, timestamps and representative frames.
- Subject, action and framing descriptions from the configured vision model, kept separate from source metadata and user edits.
- Shared UI/Agent editing, saved in the project and preserved after reopening.
I'll check it against hand-annotated clips, including rapid cuts and camera motion, and report boundary errors, description quality, runtime and peak GPU memory.
This touches media inspection, project documents/commands and the canvas/Agent editors. I'd like to settle the shot format with maintainers first; per-shot generation can follow from the saved breakdown.
Activity
prettygirlisnotme commentedon Nov 9, 2026
I have a working draft for the reference-video breakdown. I’d like to check the saved format before opening the PR.
The draft uses a project document linked to the source video. Each shot keeps the detected time range and representative frame, an optional still-frame suggestion, and a separate editable draft. The canvas and Agent use the same save command.
Does that shape fit the project? I can open the draft PR once we agree on where the breakdown should live.