Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Use it to explore visual ideas, refine assets, and make short films. Projects st
- **Timeline editing**: Arrange assets, adjust start and end points, speed, and volume, then preview and render an MP4. You can also export JSON, FCPXML, and asset bundles.
- **Local projects, your choice of models**: The canvas saves automatically to your machine. Configure multiple models for a service and switch between them as needed.

If image generation repeatedly times out or disconnects through a system proxy, open **Settings → Model connections → Edit service → Advanced connection** and set **Network connection mode** to **Direct connection**. This affects that service's model requests and result downloads without changing the system proxy. Check the provider's records before starting another attempt when request acceptance is unknown.
If image generation repeatedly times out or disconnects through a system proxy, open **Settings → Model connections → Edit service → Advanced connection** and set **Network connection mode** to **Direct (no proxy)**. This affects that service's model requests and result downloads without changing the system proxy. Check the provider's records before starting another attempt when request acceptance is unknown.

## Example film: We Can Still Win

Expand Down
2 changes: 1 addition & 1 deletion README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ OpenDirector 是一个在本地运行、通过浏览器使用的 AI 视频创作
- **时间线编排**:安排素材顺序、起止点、速度和音量,预览并合成 MP4,也可导出 JSON、FCPXML 和素材包。
- **本地项目,自选模型**:画布自动保存到本机,同一服务可配置多个模型,按用途切换。

若图片生成经系统代理反复超时或连接中断,可在「设置 → 模型接入 → 编辑服务 → 高级连接」将「网络连接方式」设为「直连」。此设置仅影响该服务的模型请求和结果下载,不修改系统代理。提交结果不明的任务应先核对供应商记录,再决定是否新建尝试。
若图片生成经系统代理反复超时或连接中断,可在「设置 → 模型接入 → 编辑服务 → 高级连接」将「网络连接方式」设为「直连(不使用代理)」。此设置仅影响该服务的模型请求和结果下载,不修改系统代理。提交结果不明的任务应先核对供应商记录,再决定是否新建尝试。

## 短片示例:《我们还能赢》

Expand Down
2 changes: 2 additions & 0 deletions apps/api/src/opendirector_api/localization.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@
from opendirector.domain.creation.commands import CommandReceipt

ENGLISH_BY_CODE = {
'tool_concept_translation': 'Use a short English subject description, or add a chat model in Models to translate it automatically.',
'tool_no_video_match': 'The requested subject was not found in the video. Try a different description.',
'voice_upload_type': 'Upload the voice recording using multipart/form-data.',
'voice_upload_form': 'Check the recording, form fields and text lengths, then try the upload again.',
'voice_model_unsupported': 'This checkpoint does not accept uploaded voices. Choose a Base model for voice cloning or a compatible built-in voice.',
Expand Down
6 changes: 6 additions & 0 deletions docs/development/lessons.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,9 @@ When work exposes a consequential risk or a recurring problem, add a lesson or u
- **Trigger:** A relevant test fails and comparison shows that it also fails on the baseline.
- **Lesson:** Commits `28d61dce` and `0a5630ac` each recorded the same generation test failing on the baseline. That historical record does not establish whether the test still fails today.
- **Check:** Record the versions and results used in the comparison. Link an existing issue or create one when appropriate. Report the check as failed; a baseline failure is not a pass.

## Provider result URLs are an independent trust boundary

- **Trigger:** Accepting a cross-origin result URL from a configured model service.
- **Lesson:** [PR #210](https://github.com/ThinkFlowLab/OpenDirector/pull/210) added compatible image CDN downloads without provider credentials, but initially left arbitrary local and private hosts reachable. Credential isolation and bounded responses do not constrain the network destination.
- **Check:** Preserve explicitly configured local service origins; validate all DNS answers for unconfigured hosts, connect to a checked public IP while retaining the original Host and TLS identity, and revalidate every redirect. Cover local metadata addresses, IPv6, mixed DNS answers, redirect loops, and recovery without another model submission.
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,28 @@ def freeze_inputs(service, project_id, value):
targetIntent=target_intent(snapshot, target) if target else None)


def _numbered(title):
head, separator, tail = title.rpartition(' ')
return (head, int(tail)) if separator and head and tail.isascii() and tail.isdigit() else (title, None)


def result_title(snapshot, base, used=(), exclude=None):
"""Keep result names tied to their tool; a repeated run takes the next number."""
family, index = _numbered(base)
occupied = [item.title for item in snapshot.canvasItems
if item.visible and item.title and (exclude is None or item.id != exclude)]
taken = (*occupied, *used)
if base not in taken:
return base
numbers = [0]
for title in taken:
head, number = _numbered(title)
if head != family:
continue
numbers.append(1 if number is None else number)
return f'{family} {max(numbers) + 1}'


def release_placeholder(service, task, session=None):
"""Release only the accepted task's unchanged target; never infer UI defaults."""
snapshot = service.projects.snapshot(task['projectId'])
Expand Down Expand Up @@ -156,6 +178,7 @@ def expect(kind, obj):
expect('canvas_item', item)
if policy == 'if_current' and not matching:
return unplaced()
titles = []
for index, output in enumerate(task['outputs']):
item_id = UUID(output_item_id(task, index, output['kind']))
item = next((item for item in snapshot.canvasItems if item.id == item_id), None)
Expand All @@ -181,17 +204,23 @@ def expect(kind, obj):
expect('canvas_item', item)
operations.append(dict(type='canvas.show', payload=dict(item=item.model_dump(mode='json'))))
continue
# The placeholder already carries the tool/preset name in the UI language.
title = result_title(snapshot, item.title if item else output['title'],
titles, item.id if item else None)
titles.append(title)
if item:
editable(snapshot, item.id)
expect('canvas_item', item)
if not item.visible:
operations.append(dict(type='canvas.show', payload=dict(item=item.model_dump(mode='json'))))
if item.title != title:
operations.append(dict(type='canvas.update', payload=dict(itemId=item.id, title=title)))
operations.append(dict(type='canvas.materialize', payload=dict(itemId=item.id, asset=asset_ref(asset).model_dump(mode='json'), discardGenerationDraft=True)))
else:
width, height = media_card_size(asset.kind, asset.metadata.width, asset.metadata.height,
source.width if source else 320, source.height if source else 240)
operations.append(dict(type='canvas.show', payload=dict(item=dict(id=item_id, projectId=snapshot.project.id,
kind=asset.kind, title=output['title'], position=dict(x=(source.position.x + source.width + 120 if source else 80) + index * 360, y=source.position.y if source else 80),
kind=asset.kind, title=title, position=dict(x=(source.position.x + source.width + 120 if source else 80) + index * 360, y=source.position.y if source else 80),
width=width, height=height, binding=dict(role='material', objectRef=asset_ref(asset).model_dump(mode='json'))))))
if asset.kind == 'image':
editor_id = uuid5(UUID(task['id']), f'output-editor-{item_id}')
Expand Down
14 changes: 12 additions & 2 deletions packages/core/src/opendirector/application/tool_context.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,14 @@ class Output:
suffix: str
title: str


class ToolInputError(ProjectError):
"""A tool rejection with stable localization identity and interpolation data."""

def __init__(self, message, *, code, label):
super().__init__(message, 422, code=code)
self.params = {'label': label}

class ToolContext:
"""What a recipe may use: inputs, model capabilities and progress reporting."""

Expand Down Expand Up @@ -121,7 +129,9 @@ async def concepts(self, text: str, default: tuple[str, ...]) -> list[tuple[str,
except ProjectError:
english = None
if not english or not english.isascii():
raise ProjectError(f'无法识别「{part}」,请改用英文描述,或在“模型接入”中添加对话模型以自动翻译。', 422)
raise ToolInputError(
f'无法识别「{part}」,请改用英文描述,或在“模型接入”中添加对话模型以自动翻译。',
code='tool_concept_translation', label=part)
pairs.append((english, part))
return pairs

Expand Down Expand Up @@ -160,7 +170,7 @@ async def video_from_frame(self, frame: Image.Image, prompt: str, seconds: float
CONCEPTS = {
'人': 'person', '人物': 'person', '人像': 'person', '人群': 'person', '主体': 'person', '男人': 'man', '女人': 'woman',
'小孩': 'child', '孩子': 'child', '儿童': 'child', '脸': 'face', '手': 'hand', '头发': 'hair',
'动物': 'animal', '猫': 'cat', '狗': 'dog', '鸟': 'bird', '马': 'horse', '鱼': 'fish',
'动物': 'animal', '猫': 'cat', '猫咪': 'cat', '小猫': 'cat', '狗': 'dog', '鸟': 'bird', '马': 'horse', '鱼': 'fish',
'车': 'car', '汽车': 'car', '车辆': 'vehicle', '交通工具': 'vehicle', '自行车': 'bicycle', '摩托车': 'motorcycle',
'船': 'boat', '飞机': 'airplane', '树': 'tree', '花': 'flower', '植物': 'plant', '建筑': 'building', '房子': 'house',
'天空': 'sky', '桌子': 'table', '椅子': 'chair', '沙发': 'sofa', '家具': 'furniture', '杯子': 'cup', '瓶子': 'bottle',
Expand Down
13 changes: 9 additions & 4 deletions packages/core/src/opendirector/application/tool_image_edits.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,8 @@ async def smart_edit(ctx: ToolContext):
input_image = marked
instruction += '。图上的笔画、箭头与文字是编辑指示'
instruction += ',完成修改后移除这些辅助标记'
if mode != 'annotation' and ctx.mask:
instruction += '。这是原图上的局部修改,请严格保持原图构图、镜头、主体位置、大小和姿势,使编辑结果与原图逐像素对齐'
edited = await ctx.edit(input_image, instruction + '。保持未提及的内容不变', 0.78, references=references)
if mode != 'annotation' and ctx.mask:
# A selection is an exact editing boundary: pixels outside it stay original.
Expand Down Expand Up @@ -116,7 +118,7 @@ async def inpainting_remove(ctx: ToolContext):
mask = media.mask_from(ctx.mask, image.size)
await ctx.progress('正在消除并填补')
drawn = await ctx.edit(media.soften_region(image, mask), '移除涂抹区域中的物体与文字,用周围环境自然填补,画面干净完整', 0.8)
return [image_output(media.blend(image, drawn, media.feather(mask, max(3, min(image.size) / 120))), '消除')]
return [image_output(media.blend(image, drawn, media.feather(mask, max(3, min(image.size) / 120))), '消除笔')]

def _yaw_text(yaw):
names = ['正面', '右前方', '右侧面', '右后方', '背面', '左后方', '左侧面', '左前方']
Expand Down Expand Up @@ -176,19 +178,22 @@ class ImagePreset:
'保持参考图画风,纯净浅灰背景、均匀柔光、弱透视,无剧情场景、文字、标签、水印、分割线或额外视图。',)),
'face-three-view': ImagePreset('面部三视图', (
'以参考图人物的面部为唯一身份依据,生成同一人物的面部三视图参考板。'
'超宽横版画面分为三个等宽区域,从左至右严格排列:正面、左侧90度、右侧90度。'
'3:1超宽横版画面均分为三个等宽区域,从左至右严格排列:正面、左侧90度、右侧90度。'
'第二格(左侧90度)鼻尖朝向画面右方,第三格(右侧90度)鼻尖朝向画面左方;'
'两张侧脸朝向相反、左右镜像对称,不要输出两张朝向相同的侧脸。'
'每个区域只放一个居中的头肩像,三个头像之间留有自然间隔,不接触、不重叠、不粘连。'
'每个视图只呈现完整头部至肩颈,头顶、下巴、耳朵均完整,不出现背面或全身视图。'
'保持同一身份、年龄、脸型、骨相、五官比例、肤色、发际线、发型及识别性细节,左右侧面符合真实结构。'
'三个头部大小一致、眼睛基线对齐、中性表情、视线随朝向自然变化,面部不被头发遮挡。'
'保持参考图画风,纯净浅灰背景、均匀柔光,无美颜改脸、夸张透视、文字、标签、水印或额外人物。',), (2688, 1152)),
'保持参考图画风、纯净浅灰背景、均匀柔光,无美颜改脸、夸张透视、文字、标签、水印或额外人物。',), (3456, 1152)),
'product-three-view': ImagePreset('产品三视图', (
'以参考图中的产品为唯一设计依据,生成同一产品的工业设计三视图参考板。'
'横版画面分为三个等宽区域,从左至右严格排列:正视图、右侧90度侧视图、后视图。'
'采用近正交投影、无广角畸变,同一比例尺、同一高度与底部基线,完整展示产品外轮廓,不裁切、不重叠。'
'三个视图保持结构、尺寸关系、材质、颜色、表面纹理、接口、按钮、接缝及标识一致,细节在视角之间符合物理对应关系。'
'不可见部位依据已知结构作合理且一致的补全,不增加功能件或改变产品设计。'
'产品独立展示,纯白背景、均匀棚拍光,无人物、手、包装道具、剧情场景、说明文字、尺寸标注、水印或额外视图;保留原有品牌标识。',)),
'storyboard': ImagePreset('分镜', ('把镜头拉远成远景建立镜头,完整交代周围环境,主体在画面中较小',
'storyboard': ImagePreset('连续分镜图', ('把镜头拉远成远景建立镜头,完整交代周围环境,主体在画面中较小',
'改为中景镜头,主体占画面一半高度,动作清晰',
'改为近景镜头,镜头推近到主体胸口以上,突出表情',
'改为特写镜头,只拍主体面部或关键细节,背景虚化')),
Expand Down
4 changes: 3 additions & 1 deletion packages/core/src/opendirector/application/tool_tasks.py
Original file line number Diff line number Diff line change
Expand Up @@ -443,7 +443,9 @@ async def preserve():
if current.get('cancelRequested'):
await self._stopped(task['projectId'], task['id'])
else:
await self._update(directory, task, status='failed', localState='stopped', phase=None, error=str(failure), finishedAt=_now())
await self._update(directory, task, status='failed', localState='stopped', phase=None,
error=str(failure), errorDetails={'code': failure.code, 'params': getattr(failure, 'params', {})},
finishedAt=_now())
except Exception: # noqa: BLE001 — a recipe bug must still end the task visibly.
current = self.get(task['projectId'], task['id'])
if current.get('cancelRequested') or self.closing:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

from PIL import Image, ImageChops, ImageFilter

from opendirector.application.tool_context import Output, Source, ToolContext
from opendirector.application.tool_context import Output, Source, ToolContext, ToolInputError
from opendirector.domain.capabilities import EditMask, Tracks
from opendirector.domain.projects import ProjectError
from opendirector.media import processing as media
Expand Down Expand Up @@ -111,7 +111,9 @@ async def video_matting(ctx: ToolContext):
await ctx.progress(f"正在识别并跟踪「{'、'.join(label for _, label in pairs)}」")
tracks = await ctx.track(clip.read_bytes(), tuple(concept for concept, _ in pairs))
if not any(tracks.frames):
raise ProjectError(f"视频中没有识别到「{'、'.join(label for _, label in pairs)}」,请换一种描述后重试。", 422)
label = '、'.join(label for _, label in pairs)
raise ToolInputError(f'视频中没有识别到「{label}」,请换一种描述后重试。',
code='tool_no_video_match', label=label)
await ctx.progress('正在合成抠像视频')
await blocking(_write_mattes, tracks, work.path / 'mattes')
data, suffix = await blocking(media.matte, clip, str(work.path / 'mattes' / '%05d.png'), info['fps'],
Expand Down
16 changes: 11 additions & 5 deletions packages/core/src/opendirector/integrations/model_adapters.py
Original file line number Diff line number Diff line change
Expand Up @@ -286,11 +286,17 @@ async def work():
result = value['data'][0]
if result.get('b64_json'):
return base64.b64decode(result['b64_json'], validate=True)
# Download only from the configured service, never a third-party host.
if urlsplit(result['url']).netloc != urlsplit(config['url']).netloc:
raise protocol_failure('download')
async with client.stream('GET', result['url'], headers=auth_headers(config)) as response:
return await bounded(response, stage='download', secrets=secrets)
# Compatible APIs can return a CDN URL even when base64 was requested.
# Reuse the generation downloader, which sends no API key to the CDN.
async def download():
target, base = urlsplit(result['url']), urlsplit(config['url'])
if (target.scheme, target.netloc) != (base.scheme, base.netloc):
_, data = await VolcengineGeneration(None, transport=self.transport).resume_result(
config, 'png', result['url'], public_only=True)
return data
async with client.stream('GET', result['url'], headers=auth_headers(config)) as response:
return await bounded(response, stage='download', secrets=secrets)
return await _call('download', download)

return _decoded(await _call('submit', work))

Expand Down
10 changes: 7 additions & 3 deletions packages/core/src/opendirector/integrations/providers.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,16 +88,20 @@ async def resume_result(self, config, suffix, url):
if not config or config.get('protocol') in {None, 'volcengine'}:
return await super().resume_result(config, suffix, url)
# Only send a service token back to that exact configured origin.
base, target = urlsplit(config['url']), urlsplit(url)
headers = auth_headers(config) if (base.scheme, base.netloc) == (target.scheme, target.netloc) else {}
try:
base, target = urlsplit(config['url']), urlsplit(url)
same_origin = (base.scheme, base.netloc) == (target.scheme, target.netloc)
async with httpx.AsyncClient(event_hooks=MODEL_HTTP_HOOKS, transport=self.transport, trust_env=trust_environment(url, config=config), timeout=httpx.Timeout(240, connect=15)) as client:
async with client.stream('GET', url, headers=headers, follow_redirects=False) as response:
if not same_origin:
return suffix, await self._download(client, url, secrets=(config.get('apiKey', ''),), public_only=True)
async with client.stream('GET', url, headers=auth_headers(config), follow_redirects=False) as response:
return suffix, await bounded(response, stage='download', secrets=(config.get('apiKey', ''),))
except httpx.TimeoutException:
raise transport_failure('download', timeout=True) from None
except httpx.HTTPError:
raise transport_failure('download') from None
except (httpx.InvalidURL, ValueError, TypeError, KeyError, AttributeError):
raise protocol_failure('download') from None

async def generate(self, config, task, submitted, *, result_ready=None):
adapter = self._profile_adapter(config)
Expand Down
Loading