Skip to content

feat(ai-edition): introduce AI Edition editor and timeline workflow - #61

Closed
EtienneLescot wants to merge 141 commits into
mainfrom
feat/ai-edition
Closed

EtienneLescot wants to merge 141 commits into
mainfrom
feat/ai-edition

Conversation

@EtienneLescot

@EtienneLescot EtienneLescot commented Jun 30, 2026 •

Copy link
Copy Markdown
Member

Pull Request: Implement the AI Edition Editor

This pull request implements the AI Edition—a next-generation screen recording editor for OpenScreen that ports the powerful data model and UX patterns of Axcut onto OpenScreen's native primitives. It introduces a comprehensive AI-assisted editing workflow, a modernized workbench shell, and multi-track timeline capabilities.


High-Level Overview of the AI Edition

The AI Edition transforms OpenScreen from a simple recording utility into a state-of-the-art video editor, featuring:

1. Modernized Workbench Shell

  • 4-Row Resizable Grid Layout: A highly polished, responsive layout consisting of a global Titlebar, a 7-column main workspace (with collapsible Left/Right panels and resize handles), and a bottom Timeline area.
  • Dual-Theme Support: Built on a unified CSS variables design system offering sleek, premium Light and Dark themes.
  • Project Management & Safety: Inline project renaming, save/save-as options, keyboard shortcuts (e.g., Undo/Redo, quick tool selections), and safety prompts that prevent accidental loss of unsaved changes.

2. Multi-Track Timeline & Region Editor

  • Timeline Tracks: Reimplemented the timeline with multi-track support for zoom, trim, speed adjustments, and caption/text annotation tracks.
  • Virtual Playback Engine: Real-time virtual timeline playback that seamlessly stitches kept clips together, skipping trimmed/cut segments.
  • Contextual Region Inspector: Instantly inspects and modifies selected timeline elements—allowing live edits to zoom depth, playback speed, and text annotation styles (color, size, typewriter/pulse animations).

3. Integrated AI & Auto-Captioning

  • Provider Registry: Secure integration with 8 LLM providers (including OpenAI, Anthropic, Google Gemini, and local Ollama) using safe OS-level credential storage.
  • AI Chat Assistant: An interactive AI chat pane that allows users to query, edit, and orchestrate timeline modifications via conversational prompting.
  • Automated Captioning: One-click speech-to-text transcription and automatic timeline caption generation with customizable word-wrap constraints.

Architectural & Data Model Alignment

  • Zustand State Store: Implements a unified projectStore tracking project ID, file assets, timeline revisions, and document dirty states.
  • Ported Axcut Schemas: Seamlessly maps Axcut's data format (e.g., annotation regions, zoom zones) directly onto OpenScreen’s local database layout, completely in-tree without external Python sidecar dependencies.

Verification & Testing

  • Typecheck: npx tsc --noEmit completes with 0 errors.
  • Linting & Formatting: Verified clean Biome formatting across all modified files.
  • Unit Tests: Passed all 353 tests across 44 test files in the test suite.

Summary by CodeRabbit

  • New Features
    • Enabled the AI-assisted editor as the default workflow, including project/media management, AI chat with provider settings, cursor/webcam preview overlays, virtual timeline editing, and compact export flow.
  • Bug Fixes
    • Improved handling of empty/invalid recordings during session saving.
    • Forwarded renderer console logs to the main process for easier diagnostics.
  • Documentation
    • Published/updated OpenScreen, Axcut, and AI-Edition design and architecture/merge documentation.
  • Tests
    • Added coverage for AI-edition document persistence and key editor UI components.
  • Chores
    • Updated scanning rules to ignore design/worktree files.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant