diff --git a/README.md b/README.md index 201c017d..18bb7317 100644 --- a/README.md +++ b/README.md @@ -1,150 +1,240 @@ -# OpenDirector · 本地创作画布 +

+ + + OpenDirector — animated clapperboard logo + +

-在画布上整理素材、生成图片和视频、编辑媒体,并通过时间线编排和导出视频。 +

+ English | 简体中文 +

-## 安装与启动 +# OpenDirector -准备以下环境: +**From an idea to a downloadable short film: generate through chat, organize on a canvas, edit media, and export from a timeline.** -- Node.js 24.22.1 或更新版本。 -- Python 3.11 或更新版本。 -- `ffmpeg` 和 `ffprobe`,安装后可在终端直接运行。 +OpenDirector is an AI video creation workspace that runs locally and is accessed through your browser. Discuss ideas with the assistant and generate assets, organize and iterate on them on the canvas, edit images, video, and audio, then arrange your material on a timeline and export an MP4. -下载项目并安装依赖: +Use it to explore visual ideas, refine assets, and make short films. Projects stay on your machine; generation uses the provider APIs or self-hosted model services you choose. + +[Quick start](#quick-start) · [Connect models](#connect-models) · [Make your first short film](#make-your-first-short-film) · [Troubleshooting](#troubleshooting) · [More documentation](#more-documentation) + +## What you can do + +- **Work with AI**: Describe what you want to create, reference canvas assets, and ask the assistant to save text or generate content with your configured models. +- **Canvas and reference assets**: Upload files, connect nodes, or use `@` to add references. Organize assets in one place and review generation history. +- **Media editing**: Crop, adjust colors, and work with layers and clips. AI tools such as image extension, erasing, and background removal depend on the connected models' capabilities. +- **Timeline editing**: Arrange assets, adjust start and end points, speed, and volume, then preview and render an MP4. You can also export JSON, FCPXML, and asset bundles. +- **Local projects, your choice of models**: The canvas saves automatically to your machine. Configure multiple models for a service and switch between them as needed. + +## Quick start + +### 1. Prepare your environment + +| Dependency | Requirement | +| --- | --- | +| Node.js and npm | Node.js `24.22.1` or later, with npm | +| Python | `3.11` or later | +| FFmpeg | Both `ffmpeg` and `ffprobe` must be available in your terminal for media processing | +| Model service | A working service URL, Model ID, and any API key or other authentication information required by the service | + +Installing dependencies and calling remote models require a network connection. Prepare self-hosted model services separately; starting this application does not install or deploy models. + +First, check that the media tools are available: ```sh -git clone --branch rea --single-branch https://github.com/ThinkFlowLab/OpenDirector.git +ffmpeg -version +ffprobe -version +``` + +### 2. Get the source + +```sh +git clone https://github.com/ThinkFlowLab/OpenDirector.git cd OpenDirector +``` + +You can also download and extract the source. Run all the following commands from the repository root. + +### 3. Install and start + +#### macOS, Linux, and WSL + +```sh npm ci python3 -m venv .venv .venv/bin/python -m pip install -r requirements.txt npm run build +.venv/bin/python scripts/opendirector.py serve ``` -启动应用: +#### Windows (PowerShell) -```sh -.venv/bin/python scripts/opendirector.py serve +```powershell +npm.cmd ci +py -3 -m venv .venv +.\.venv\Scripts\python.exe -m pip install -r requirements.txt +npm.cmd run build +.\.venv\Scripts\python.exe scripts/opendirector.py serve ``` -在浏览器打开终端显示的地址,默认是 `http://127.0.0.1:8000`。使用期间保持终端运行,按 `Ctrl+C` 退出。 +If the `py` command is unavailable but Python is on your PATH, create the virtual environment with `python -m venv .venv`. -Windows 将上述 `.venv/bin/python` 替换为 `.venv\Scripts\python.exe`。如果系统使用 `python` 命令,创建虚拟环境时使用 `python -m venv .venv`。 +Open the address shown in the terminal in your browser. The default is . Keep the terminal running while using the application; press `Ctrl+C` to stop it. -端口被占用时可以更换: +- **Start again**: Run the final `serve` command for your operating system from the repository root. +- **Update the source**: Stop the application and back up your projects first. After updating, reinstall dependencies, rebuild the frontend, and start the application. +- **Port conflict**: Append `--port 8001` to the corresponding `serve` command and open : ```sh +# macOS, Linux, and WSL .venv/bin/python scripts/opendirector.py serve --port 8001 ``` -## 创建和管理画布 +```powershell +# Windows (PowerShell) +.\.venv\Scripts\python.exe scripts/opendirector.py serve --port 8001 +``` + +## Connect models + +Click the gear at the bottom left to open **Settings (设置)**, then open **Model connections (模型接入)**. Add models for chat, image, video, or audio generation as needed. + +1. Choose the connection type and API protocol. +2. Enter the service URL, Model ID, and the credentials or authentication method required by the service. +3. For video models, check the adapter, supported modes, resolutions, and durations. +4. Save the model, then select it in the generation panel or chat. + +Use the following guide to check the configuration fields: + +| Field | What to enter | +| --- | --- | +| Connection type | Choose **Provider API (服务商 API)** or **Self-hosted service (自部署服务)** to match your service | +| API protocol | Choose a protocol that matches the service's actual API and the model category; matching names alone do not guarantee media compatibility | +| Service URL (Base URL) | Enter an address reachable from the machine running the application; for an SSH tunnel, use the locally forwarded address | +| API Key | Enter the key required by the service. If the service does not require authentication, select **No key required (无需密钥)**; a placeholder key is unnecessary | +| Model ID | Use the model name actually exposed by the service; select it with **Fetch models (获取模型)** or add it manually | + +A service can provide multiple models within the same category. After entering its URL and authentication settings, click **Fetch models (获取模型)**, search for and select the models you need, then save. Use **Add manually (手动添加)** for models that are not listed. Chat models can be tested individually in the list. The service URL and credentials are stored once and can be updated together by editing the service. Deselecting a model removes its configuration; wherever that configuration was selected, you will need to select a model again. -首次打开时会创建“未命名画布”,之后打开应用可恢复最近使用的项目。 +The service URL and Model ID must match your model service. Self-hosted service URLs typically include `/v1`; use the model name actually exposed by the service as the Model ID. For third-party services, follow the information provided by the service. -- 点击左上角项目名称修改名称,按 `Enter` 或点击其他位置保存,按 `Esc` 取消。 -- 点击名称右侧的箭头切换项目,选择“新建画布项目”创建项目。 -- 通过左侧工具栏或双击空白画布创建文本、图片、视频、音频、时间线或主体节点。 -- 右键空白画布可新建节点、粘贴、撤销或重做。 -- 点击左侧“上传”或将文件拖入画布添加素材;“资产库”可选择已有素材。 -- 双击节点名称重命名;双击节点内容进入相应编辑或预览界面。 +For detailed video settings, see [Video model capability configuration (Chinese)](docs/guides/video-model-capabilities.md). When no explicit output specification is provided, use **Auto (自动)** or **Service default duration (服务默认时长)**. -画布会自动保存。关闭或切换前,请确认左上角显示“已保存到本机”;若显示保存失败,保留页面并按提示处理。 +Automatic region selection in images and video matting require a model with segmentation capabilities. Image generation and editing still require the corresponding generation or editing model. -## 接入模型 +In **General (通用)** settings, you can switch themes and adjust the watermark removal option. Watermark settings affect only content generated afterward, and the actual effect depends on support from the model service. -点击左下角齿轮进入“设置”,打开“模型接入”,按用途添加对话、图片、视频或音频模型。 +## Make your first short film -1. 选择接入方式和接口协议。 -2. 填写服务地址、Model ID,以及服务要求的密钥或鉴权方式。 -3. 对视频模型检查适配方式、支持的模式、分辨率和时长。 -4. 保存模型,然后在生成面板或对话框中选择它。 +Start by generating and exporting a single shot, then expand to more clips. -同一类别下,一个服务可启用多个模型:填写地址和鉴权后,点击“获取模型”,搜索并勾选需要的模型后保存。未列出的模型可通过“手动添加”补充;对话模型可在列表中逐个测试。服务地址和密钥只保存一份,编辑服务时可统一更新;取消勾选会移除对应模型配置,已选用该配置的位置需要重新选择。 +1. **Connect models**: In **Settings → Model connections (设置 → 模型接入)**, configure the image and video models you need. Add an audio model if you want narration. +2. **Prepare an image**: Create an image node and generate an image, or upload one you already have. For example, try “a street on a morning after rain, soft morning light, reflections on the ground.” +3. **Generate a video**: Create a video node, choose a video model, and connect the image node containing your image as a reference. Describe the motion using a mode the model supports, then generate. Check progress and results in **Generation history (生成历史)**. +4. **Refine your assets**: Check the result and edit the image or video as needed. For narration, create an audio node, enter the text to be read aloud, and choose a voice supported by the service. +5. **Arrange the timeline**: Create a timeline node and open its editor. Add your prepared visuals and audio, adjust their order, start and end points, speed, and volume, then preview the result. +6. **Download your film**: Click **Render MP4 (合成 MP4)** and keep the page open until it finishes. Download the result or add it to the canvas. -服务地址和 Model ID 应与自己的模型服务一致。自部署服务的地址通常包含 `/v1`,Model ID 使用服务实际公开的模型名称;第三方服务按其提供的信息填写。 +You can also [chat with AI](#chat-with-ai) first to develop ideas, save text, or generate assets, then continue editing on the canvas and timeline. If something fails, check the [generation task status](#monitor-and-manage-generation-tasks) and identify the cause before deciding whether to submit again. -视频模型的详细配置见 [视频模型能力配置](docs/guides/video-model-capabilities.md)。未提供明确输出规格时,使用“自动”或“服务默认时长”。 +## Create and manage canvases -自动识别图片选区或视频抠像需要具备分割能力的模型;生成图片、编辑图片仍需选择对应的生成或编辑模型。 +The first launch creates an untitled canvas (未命名画布). Later launches can restore the most recently used project. -在设置的“通用”页中可切换主题、调整去除水印选项。水印设置仅影响后续生成的内容,实际效果取决于模型服务是否支持。 +- Click the project name at the top left to rename it. Press `Enter` or click elsewhere to save, or press `Esc` to cancel. +- Click the arrow next to the name to switch projects. Choose **New canvas project (新建画布项目)** to create one. +- Use the left toolbar or double-click empty canvas space to create text, image, video, audio, timeline, or subject nodes. +- Right-click empty canvas space to create a node, paste, undo, or redo. +- Click **Upload (上传)** on the left or drag files onto the canvas to add assets. Use **Asset library (资产库)** to choose existing assets. +- Double-click a node's name to rename it. Double-click its content to open the appropriate editor or preview. -## 生成内容与添加参考 +The canvas saves automatically. Before closing it or switching projects, check that the top-left status reads **Saved locally (已保存到本机)**. If saving fails, keep the page open and follow the error message. -1. 创建文本、图片、视频或音频节点,点击节点打开生成面板。 -2. 选择模型,填写提示词,并设置可用的参数。 -3. 将有内容的来源节点连接到目标节点,或在输入框中输入 `@` 选择参考素材。 -4. 检查参考和参数,点击生成。完成后在节点或“生成历史”中查看结果。 +## Generate content and add references -输入 `@` 后可用上下方向键选择素材、左右方向键切换分类,按 `Enter` 插入,按 `Esc` 关闭选择。引用以整体形式插入正文;删除同一来源的最后一次引用会移除相应参考连线。 +1. Create a text, image, video, or audio node, then click it to open the generation panel. +2. Choose a model, enter a prompt, and set the available parameters. +3. Connect a source node that has content to the target node, or type `@` in the input field to select reference assets. +4. Check the references and parameters, then generate. When the task finishes, view its result in the node or **Generation history (生成历史)**. -空白节点需要先上传、保存文本或生成内容,才能作为其他节点的有效参考。向已有节点上传本体素材时,文件类型应与节点一致;添加参考请使用连线、`@` 或生成面板的添加入口。 +After typing `@`, use the up and down arrow keys to select an asset, the left and right arrow keys to switch categories, `Enter` to insert, and `Esc` to close the picker. References are inserted into the text as single units. Removing the last reference to a source also removes its corresponding reference connection. -视频生成时按模型能力选择模式: +An empty node must first receive an upload, saved text, or generated content before it can serve as a valid reference for other nodes. When uploading a node's own content, the file type must match the node. To add references, use connections, `@`, or the reference controls in the generation panel. -- **全能参考**:添加模型支持的图片、视频、音频或文字参考。 -- **首尾帧**:最多连接两个图片来源,按槽位选择首帧和尾帧;能否使用尾帧以模型能力和界面提示为准。 -- **接续**:选择支持该功能的模型和来源视频。服务重启或来源不再可用时,可能需要重新生成来源视频。 +For video generation, choose a mode supported by the model: -音频生成时,在提示词中填写需要朗读的正文,在参数中选择服务支持的音色。 +- **Multimodal references (全能参考)**: Add image, video, audio, or text references supported by the model. +- **First and last frames (首尾帧)**: Connect up to two image sources and assign them to the first- and last-frame slots. Last-frame support depends on the model's capabilities and the guidance in the interface. +- **Continuation (接续)**: Select a model that supports continuation and a source video. If the service restarts or the source is no longer available, you may need to regenerate the source video. -## 查看和处理生成任务 +For audio generation, enter the text to be read aloud in the prompt and choose a voice supported by the service in the parameters. -打开右上角“生成历史”,查看任务进度、结果和失败原因。 +## Monitor and manage generation tasks -- 排队或执行中的任务可请求取消。 -- 可恢复的中断任务可选择“继续查询原任务”。 -- 提交状态不明时,先核对服务端是否已接受任务,再决定是否重新生成,避免重复提交。 -- 切换服务地址、模型或协议后,旧任务可能无法继续查询。 +Open **Generation history (生成历史)** at the top right to view task progress, results, and failure details. -## 与 AI 对话 +- You can request cancellation of queued or running tasks. +- For recoverable interrupted tasks, choose **Continue checking the original task (继续查询原任务)**. +- If the submission status is unknown, first check whether the server accepted the task before deciding to generate again. This avoids duplicate submissions. +- After changing the service URL, model, or protocol, you may no longer be able to query older tasks. -点击右下角“与 AI 对话”,选择对话模型后输入创作要求。可上传、拖放、粘贴素材,或输入 `@` 引用画布中的内容。助手可按要求保存文本,并调用已配置的生成模型制作素材。 +## Chat with AI -- `Enter` 发送,`Shift+Enter` 换行。 -- 回复过程中可停止输出。 -- 点击右上角收起按钮隐藏对话框;拖动左边缘调整宽度。 +Click **Chat with AI (与 AI 对话)** at the bottom right, select a chat model, and describe what you want to create. You can upload, drag and drop, or paste assets, or type `@` to reference canvas content. The assistant can save text as requested and call configured generation models to create assets. -素材生成可能在回复结束后继续进行,可在“生成历史”查看进度。 +- Press `Enter` to send, or `Shift+Enter` for a new line. +- You can stop the response while it is being generated. +- Click the collapse button at the top right to hide the chat panel. Drag its left edge to resize it. -## 编辑图片、视频和音频 +Asset generation may continue after the reply ends. Check its progress in **Generation history (生成历史)**. -选中节点,通过工具栏选择所需操作。 +## Edit images, video, and audio -- **图片**:裁剪、旋转、翻转、调色、图层、文字、形状、标注和选区编辑。 -- **视频与音频**:取帧、裁剪、音轨提取和片段处理。 -- **AI 工具**:扩图、消除、抠图、智能改图、超清等,是否可用取决于已接入模型。 +Select a node, then choose an action from its toolbar. -需要局部修改时先标记区域或创建选区,再填写要求并确认执行。缺少所需模型能力时,按工具提示补充配置;灰色入口以悬停提示为准。 +- **Images**: Crop, rotate, flip, adjust colors, work with layers, add text, shapes, or annotations, and edit selections. +- **Video and audio**: Extract frames, trim, extract audio tracks, and process clips. +- **AI tools**: Extend images, erase content, remove backgrounds, edit images with AI, upscale, and more. Availability depends on the connected models. -## 编排和导出视频 +For a local edit, mark an area or create a selection first, then enter your instructions and confirm the action. If a required model capability is missing, follow the tool's instructions to configure it. Hover over a disabled control for an explanation. -1. 创建时间线节点并打开编辑器。 -2. 添加图片、视频和音频,调整顺序、起止点、速度及音量。 -3. 预览并检查画面和声音。 -4. 点击“合成 MP4”,完成后下载,或将结果加入画布。 +## Arrange and export videos -时间线还支持 JSON、FCPXML 和素材包导出,按用途选择。若浏览器提示不支持所需编码能力,请使用支持的浏览器后重试。 +1. Create a timeline node and open its editor. +2. Add images, videos, and audio. Adjust their order, start and end points, speed, and volume. +3. Preview and check the picture and sound. +4. Click **Render MP4 (合成 MP4)**. When rendering finishes, download the result or add it to the canvas. -合成期间保持页面打开。关闭编辑器或取消合成会停止本次处理。 +The timeline also supports JSON, FCPXML, and asset bundle exports. Choose the format that fits your needs. If your browser reports that a required encoding capability is unavailable, try again in a browser that supports it. -## 保存、备份和下载 +Keep the page open while rendering. Closing the editor or canceling the render stops the current operation. -- 节点中的下载入口用于保存对应素材。 -- 右上角“导出画布”用于保存画布 JSON 副本。 -- 本地项目保存在项目目录下的 `projects/` 文件夹。需要保留完整项目时,停止应用后备份该目录。 -- 模型配置保存在本机,包含凭证时请妥善保管。 +## Save, back up, and download -发送给模型的参考素材会传递至所选服务,请根据素材用途选择合适的服务。 +- Use a node's download control to save its asset. +- Use **Export canvas (导出画布)** at the top right to save a JSON copy of the canvas. +- Local projects are stored in the `projects/` folder inside the repository directory. To preserve complete projects, stop the application and back up that folder. +- Model settings are stored in `.opendirector/models.json` and may contain API keys. Keep backups secure and exclude these credentials when sharing projects or logs. -## 常见问题 +Saving projects locally does not imply offline inference. Generation prompts and selected reference assets are sent to the configured model service. Usage costs and data handling depend on that service; choose one appropriate for your assets. -**无法打开应用或提示服务不可用**:确认启动终端仍在运行,使用终端显示的地址打开,并检查启动错误。只运行 `npm run dev` 不能提供完整应用服务。 +## Troubleshooting -**模型不能生成或工具置灰**:检查模型是否已保存、是否在面板中选中,以及它是否支持当前输入和功能。 +| Symptom | What to check first | +| --- | --- | +| The page will not open, or the service is unavailable | Check that the startup terminal is still running, use the address it displays, and check for startup errors or port conflicts. Running only `npm run dev` does not start the complete application service | +| A message asks you to build the frontend first | Run `npm run build` from the repository root, or `npm.cmd run build` on Windows, then restart | +| A model cannot generate content, or a tool is disabled | Check that the model is saved and selected in the panel, and that the service URL, protocol, Model ID, authentication, and required capabilities match | +| An upload is rejected | Check that the file type matches the node or reference slot, then check the size, format, and duration against the error message | +| Media processing fails | Run `ffmpeg -version` and `ffprobe -version` in your terminal to check that the tools are available; restart the application after installing them | +| A video cannot be rendered or exported | Check your browser's encoding support and whether the assets can be read and played correctly. Keep the page and editor open while rendering | +| A task is interrupted, or its submission status is unknown | Check **Generation history (生成历史)**. Use **Continue checking the original task (继续查询原任务)** for recoverable tasks. If the status is unknown, check whether the server accepted the task before submitting again | +| Changes have not been saved | Keep the page open, check the **Saved locally (已保存到本机)** status and any error messages, and resolve the save failure before closing or switching | -**上传被拒绝**:检查文件类型是否匹配节点或槽位,并按提示检查文件大小、格式和时长。 +## More documentation -**视频无法导出**:检查浏览器的编码支持,以及素材是否可正常读取和播放。 +- [Video model capability configuration (Chinese)](docs/guides/video-model-capabilities.md): Configure video adapters, modes, and output specifications. +- [Documentation index (Chinese)](docs/README.md): Find the project architecture, development guidance, and validation guide. -**修改没有保存**:保持页面打开,检查“已保存到本机”状态和错误提示,避免在保存失败时直接关闭页面。 +To report a problem, open an [issue](https://github.com/ThinkFlowLab/OpenDirector/issues) with your operating system, relevant tool versions, reproduction steps, and the expected and actual results. Remove credentials and sensitive assets before sharing logs. diff --git a/README.zh-CN.md b/README.zh-CN.md new file mode 100644 index 00000000..1555d718 --- /dev/null +++ b/README.zh-CN.md @@ -0,0 +1,240 @@ +

+ + + OpenDirector 动画场记板 Logo + +

+ +

+ English | 简体中文 +

+ +# OpenDirector + +**从创作想法到可下载的短片:对话生成、画布管理、媒体编辑、时间线导出。** + +OpenDirector 是一个在本地运行、通过浏览器使用的 AI 视频创作工作台。你可以与助手讨论创意并生成素材,在画布上整理和迭代内容,用编辑工具调整图片、视频和音频,再通过时间线编排并导出 MP4。 + +适合验证视觉创意、反复调整素材和制作短片。项目保存在本机,生成内容使用你选择的服务商 API 或自部署模型服务。 + +[快速开始](#快速开始) · [接入模型](#接入模型) · [制作第一支短片](#制作第一支短片) · [故障排查](#故障排查) · [更多文档](#更多文档) + +## 能做什么 + +- **与 AI 协作**:描述创作要求,引用画布素材,让助手保存文本并调用已配置的模型生成内容。 +- **画布与参考素材**:上传文件、连接节点或用 `@` 添加参考,集中整理素材并查看生成历史。 +- **媒体编辑**:裁剪、调色、处理图层和片段;扩图、消除、抠图等 AI 工具按已接入模型的能力提供。 +- **时间线编排**:安排素材顺序、起止点、速度和音量,预览并合成 MP4,也可导出 JSON、FCPXML 和素材包。 +- **本地项目,自选模型**:画布自动保存到本机,同一服务可配置多个模型,按用途切换。 + +## 快速开始 + +### 1. 准备环境 + +| 依赖 | 要求 | +| --- | --- | +| Node.js 与 npm | Node.js `24.22.1` 或更新版本,附带 npm | +| Python | `3.11` 或更新版本 | +| FFmpeg | `ffmpeg` 与 `ffprobe` 均可在终端运行,用于媒体处理 | +| 模型服务 | 准备可用的服务地址、Model ID,以及服务要求的 API Key 或其他鉴权信息 | + +安装依赖和调用远程模型需要网络。自部署模型服务需另外准备,启动本应用不会自动安装或部署模型。 + +先确认媒体工具可用: + +```sh +ffmpeg -version +ffprobe -version +``` + +### 2. 获取源码 + +```sh +git clone https://github.com/ThinkFlowLab/OpenDirector.git +cd OpenDirector +``` + +也可下载源码并解压。以下命令均在仓库根目录执行。 + +### 3. 安装并启动 + +#### macOS、Linux 和 WSL + +```sh +npm ci +python3 -m venv .venv +.venv/bin/python -m pip install -r requirements.txt +npm run build +.venv/bin/python scripts/opendirector.py serve +``` + +#### Windows(PowerShell) + +```powershell +npm.cmd ci +py -3 -m venv .venv +.\.venv\Scripts\python.exe -m pip install -r requirements.txt +npm.cmd run build +.\.venv\Scripts\python.exe scripts/opendirector.py serve +``` + +如果系统没有 `py` 命令,但已将 Python 加入 PATH,创建虚拟环境时使用 `python -m venv .venv`。 + +在浏览器打开终端显示的地址,默认是 。使用期间保持终端运行,按 `Ctrl+C` 退出。 + +- **再次启动**:在仓库根目录运行对应系统的最后一条 `serve` 命令。 +- **更新源码**:先停止应用并备份项目,更新后重新安装依赖、构建前端并启动。 +- **端口冲突**:在对应系统的 `serve` 命令后追加 `--port 8001`,并访问 : + +```sh +# macOS、Linux 和 WSL +.venv/bin/python scripts/opendirector.py serve --port 8001 +``` + +```powershell +# Windows(PowerShell) +.\.venv\Scripts\python.exe scripts/opendirector.py serve --port 8001 +``` + +## 接入模型 + +点击左下角齿轮进入“设置”,打开“模型接入”,按用途添加对话、图片、视频或音频模型。 + +1. 选择接入方式和接口协议。 +2. 填写服务地址、Model ID,以及服务要求的密钥或鉴权方式。 +3. 对视频模型检查适配方式、支持的模式、分辨率和时长。 +4. 保存模型,然后在生成面板或对话框中选择它。 + +配置时可按下面的说明核对字段: + +| 字段 | 如何填写 | +| --- | --- | +| 接入方式 | 按实际服务选择“服务商 API”或“自部署服务” | +| 接口协议 | 选择与服务实际接口及模型类别匹配的协议;仅名称相同不代表媒体能力兼容 | +| 服务地址(Base URL) | 填写运行应用的机器能够访问的地址;通过 SSH 隧道连接时使用本地转发地址 | +| API Key | 填写服务要求的密钥;服务确实不要求鉴权时,可勾选“无需密钥”,无需填写占位值 | +| Model ID | 使用服务实际公开的模型名称,通过“获取模型”选择或手动添加 | + +同一类别下,一个服务可启用多个模型:填写地址和鉴权后,点击“获取模型”,搜索并勾选需要的模型后保存。未列出的模型可通过“手动添加”补充;对话模型可在列表中逐个测试。服务地址和密钥只保存一份,编辑服务时可统一更新;取消勾选会移除对应模型配置,已选用该配置的位置需要重新选择。 + +服务地址和 Model ID 应与自己的模型服务一致。自部署服务的地址通常包含 `/v1`,Model ID 使用服务实际公开的模型名称;第三方服务按其提供的信息填写。 + +视频模型的详细配置见 [视频模型能力配置](docs/guides/video-model-capabilities.md)。未提供明确输出规格时,使用“自动”或“服务默认时长”。 + +自动识别图片选区或视频抠像需要具备分割能力的模型;生成图片、编辑图片仍需选择对应的生成或编辑模型。 + +在设置的“通用”页中可切换主题、调整去除水印选项。水印设置仅影响后续生成的内容,实际效果取决于模型服务是否支持。 + +## 制作第一支短片 + +建议先完成一个镜头的生成与导出,再扩展到多段素材。 + +1. **接入模型**:在“设置 → 模型接入”配置需要的图片、视频模型;需要旁白时再配置音频模型。 +2. **准备画面**:创建图片节点并生成一张图,或上传已有图片。例如先生成“雨后清晨的街道,柔和晨光,地面有倒影”。 +3. **生成视频**:创建视频节点,选择视频模型,将有内容的图片节点连接为参考,按模型支持的模式填写运动描述并生成。在“生成历史”查看进度和结果。 +4. **调整素材**:检查画面,按需编辑图片或视频;需要旁白时创建音频节点,填写朗读正文并选择服务支持的音色。 +5. **编排时间线**:创建时间线节点并打开编辑器,添加准备好的画面和音频,调整顺序、起止点、速度和音量,再预览检查。 +6. **下载短片**:点击“合成 MP4”,保持页面打开直到完成,然后下载结果或将其加入画布。 + +也可以先通过[与 AI 对话](#与-ai-对话)整理创意、保存文本或生成素材,再继续在画布和时间线中编辑。遇到失败时先查看[生成任务状态](#查看和处理生成任务),核对原因后再决定是否重新提交。 + +## 创建和管理画布 + +首次打开时会创建“未命名画布”,之后打开应用可恢复最近使用的项目。 + +- 点击左上角项目名称修改名称,按 `Enter` 或点击其他位置保存,按 `Esc` 取消。 +- 点击名称右侧的箭头切换项目,选择“新建画布项目”创建项目。 +- 通过左侧工具栏或双击空白画布创建文本、图片、视频、音频、时间线或主体节点。 +- 右键空白画布可新建节点、粘贴、撤销或重做。 +- 点击左侧“上传”或将文件拖入画布添加素材;“资产库”可选择已有素材。 +- 双击节点名称重命名;双击节点内容进入相应编辑或预览界面。 + +画布会自动保存。关闭或切换前,请确认左上角显示“已保存到本机”;若显示保存失败,保留页面并按提示处理。 + +## 生成内容与添加参考 + +1. 创建文本、图片、视频或音频节点,点击节点打开生成面板。 +2. 选择模型,填写提示词,并设置可用的参数。 +3. 将有内容的来源节点连接到目标节点,或在输入框中输入 `@` 选择参考素材。 +4. 检查参考和参数,点击生成。完成后在节点或“生成历史”中查看结果。 + +输入 `@` 后可用上下方向键选择素材、左右方向键切换分类,按 `Enter` 插入,按 `Esc` 关闭选择。引用以整体形式插入正文;删除同一来源的最后一次引用会移除相应参考连线。 + +空白节点需要先上传、保存文本或生成内容,才能作为其他节点的有效参考。向已有节点上传本体素材时,文件类型应与节点一致;添加参考请使用连线、`@` 或生成面板的添加入口。 + +视频生成时按模型能力选择模式: + +- **全能参考**:添加模型支持的图片、视频、音频或文字参考。 +- **首尾帧**:最多连接两个图片来源,按槽位选择首帧和尾帧;能否使用尾帧以模型能力和界面提示为准。 +- **接续**:选择支持该功能的模型和来源视频。服务重启或来源不再可用时,可能需要重新生成来源视频。 + +音频生成时,在提示词中填写需要朗读的正文,在参数中选择服务支持的音色。 + +## 查看和处理生成任务 + +打开右上角“生成历史”,查看任务进度、结果和失败原因。 + +- 排队或执行中的任务可请求取消。 +- 可恢复的中断任务可选择“继续查询原任务”。 +- 提交状态不明时,先核对服务端是否已接受任务,再决定是否重新生成,避免重复提交。 +- 切换服务地址、模型或协议后,旧任务可能无法继续查询。 + +## 与 AI 对话 + +点击右下角“与 AI 对话”,选择对话模型后输入创作要求。可上传、拖放、粘贴素材,或输入 `@` 引用画布中的内容。助手可按要求保存文本,并调用已配置的生成模型制作素材。 + +- `Enter` 发送,`Shift+Enter` 换行。 +- 回复过程中可停止输出。 +- 点击右上角收起按钮隐藏对话框;拖动左边缘调整宽度。 + +素材生成可能在回复结束后继续进行,可在“生成历史”查看进度。 + +## 编辑图片、视频和音频 + +选中节点,通过工具栏选择所需操作。 + +- **图片**:裁剪、旋转、翻转、调色、图层、文字、形状、标注和选区编辑。 +- **视频与音频**:取帧、裁剪、音轨提取和片段处理。 +- **AI 工具**:扩图、消除、抠图、智能改图、超清等,是否可用取决于已接入模型。 + +需要局部修改时先标记区域或创建选区,再填写要求并确认执行。缺少所需模型能力时,按工具提示补充配置;灰色入口以悬停提示为准。 + +## 编排和导出视频 + +1. 创建时间线节点并打开编辑器。 +2. 添加图片、视频和音频,调整顺序、起止点、速度及音量。 +3. 预览并检查画面和声音。 +4. 点击“合成 MP4”,完成后下载,或将结果加入画布。 + +时间线还支持 JSON、FCPXML 和素材包导出,按用途选择。若浏览器提示不支持所需编码能力,请使用支持的浏览器后重试。 + +合成期间保持页面打开。关闭编辑器或取消合成会停止本次处理。 + +## 保存、备份和下载 + +- 节点中的下载入口用于保存对应素材。 +- 右上角“导出画布”用于保存画布 JSON 副本。 +- 本地项目保存在项目目录下的 `projects/` 文件夹。需要保留完整项目时,停止应用后备份该目录。 +- 模型配置保存在 `.opendirector/models.json`,可能包含 API Key。备份时请妥善保管,分享项目或日志时不要包含这些凭证。 + +本地保存项目并不意味着离线推理。生成时的提示词和所选参考素材会发送给配置的模型服务;调用费用和数据处理方式取决于该服务,请根据素材用途选择。 + +## 故障排查 + +| 现象 | 优先检查 | +| --- | --- | +| 页面无法打开或提示服务不可用 | 确认启动终端仍在运行,使用终端显示的地址,并检查启动错误和端口冲突;只运行 `npm run dev` 不能提供完整应用服务 | +| 提示先构建前端 | 在仓库根目录运行 `npm run build`,Windows 使用 `npm.cmd run build`,然后重新启动 | +| 模型无法生成或工具置灰 | 检查模型是否已保存并在面板选中,以及服务地址、协议、Model ID、鉴权和所需能力是否匹配 | +| 上传被拒绝 | 检查文件类型是否匹配节点或参考槽位,并按提示检查大小、格式和时长 | +| 媒体处理失败 | 在终端运行 `ffmpeg -version` 和 `ffprobe -version` 检查工具是否可用,安装后重启应用 | +| 视频无法合成或导出 | 检查浏览器的编码支持,以及素材是否能正常读取和播放;合成期间保持页面和编辑器打开 | +| 任务中断或提交状态不明 | 查看“生成历史”;可恢复任务使用“继续查询原任务”,状态不明时先核对服务端是否已接收,避免重复提交 | +| 修改没有保存 | 保持页面打开,检查“已保存到本机”状态和错误提示,处理保存失败后再关闭或切换 | + +## 更多文档 + +- [视频模型能力配置](docs/guides/video-model-capabilities.md):了解视频适配、模式及输出规格的设置。 +- [文档索引](docs/README.md):查阅项目架构、开发指导和验证指南。 + +遇到问题可在 [Issues](https://github.com/ThinkFlowLab/OpenDirector/issues) 中提供操作系统、相关工具版本、复现步骤,以及预期和实际结果。分享日志前请移除密钥和敏感素材。 diff --git a/src/assets/opendirector-readme-logo.gif b/src/assets/opendirector-readme-logo.gif new file mode 100644 index 00000000..4ab300f9 Binary files /dev/null and b/src/assets/opendirector-readme-logo.gif differ diff --git a/src/assets/opendirector-readme-logo.png b/src/assets/opendirector-readme-logo.png new file mode 100644 index 00000000..0d138afc Binary files /dev/null and b/src/assets/opendirector-readme-logo.png differ