Repository navigation
[tracking] providers: API integrations #240
Description
Goal
Put the user experience first. For each provider targeting the full creation workflow, ask whether a user can enter one API key once and use it for every core scenario in OpenDirector: assistant chat and script writing, image generation, video generation, and narration with a preset voice. The user should be able to complete the workflow in one project without switching providers or configuring another key.
Track which workflows are available and which gaps prevent that experience. Track mainland China and international endpoints separately when credentials, model availability, or protocols differ. Specialist providers can be tracked for a single modality.
Integration matrix
Initial snapshot based on upstream main 6310097 and existing GitHub issues.
Video priority uses the Artificial Analysis v2.1 Overall With Audio snapshot checked on 2026-11-09. Sort by image-to-video (I2V) point rank, with text-to-video (T2V) shown separately. P1 covers I2V ranks 1–10 and P2 ranks 11–25 in this matrix. Unranked rows lack a verified match between a tracked video integration and an AA entry. Ranks with overlapping confidence intervals may change. Entries marked as unconfirmed are conditional priorities pending verification of the exact model and endpoint.
AA image-to-video · AA text-to-video
| Provider | Endpoint scope | Text | Image | Video | TTS / voices | Tracking |
|---|---|---|---|---|---|---|
| Volcengine Ark Agent Plan | mainland China | — | ✅ Integrated | ✅ Integrated | ✅ Integrated · audio path | — |
| MiniMax | mainland China | ✅ Integrated · chat preset | ❌ Gap · official image API | ✅ Integrated · H3 V2 | ❌ Gap · official speech and voice APIs | H3 PR #78 |
| Kling | endpoint scope to confirm | — | — | ⏳ Tracked | — | #142 |
| international | — | ⏳ Tracked · Imagen | ⏳ Tracked · Veo | — | #144 | |
| OpenAI | international | ❌ No dedicated preset | ⏳ Tracked · dedicated Images integration | ❌ No preset | ❌ No dedicated preset | Images #140 |
| Black Forest Labs | international | — | ⏳ Tracked · FLUX | — | — | #141 |
| Zhipu BigModel | mainland China | ⏳ Tracked | ⏳ Tracked | ⏳ Tracked · CogVideoX-3 | ⏳ Tracked | RFC #234 |
| MOSS | mainland China | — | ✅ Integrated preset · verify scope | — | ✅ Integrated preset · voice work tracked | Image #138 · Voices #205 |
| DeepSeek | hosted API | ✅ Integrated · chat preset | — | — | — | — |
Pending consideration
| Provider | Endpoint scope | Text | Image | Video | TTS / voices | Tracking |
|---|---|---|---|---|---|---|
| fal | international | — | ❌ No preset | ❌ No preset | ❌ No preset | Check model-specific inputs and queued result handling. |
| Runway | international | — | — | ⏳ Tracked | — | #143 |
| Replicate | international | — | ❌ No preset | ❌ No preset | ❌ No preset | Check prediction lifecycle and model-specific input schemas. |
| SiliconFlow | mainland China | ❌ No preset | ❌ No preset | ❌ No preset | ❌ No preset | Test compatible chat and speech routes. Verify image and asynchronous video contracts separately. |
| SiliconFlow | international | ❌ No preset | ❌ No preset | ❌ No preset | ❌ No preset | Verify the international catalog and credentials independently. |
| Kimi / Moonshot | mainland China | ❌ No preset · chat | — | — | — | Verify the mainland endpoint through the generic chat configuration. |
| Kimi / Moonshot | international | ❌ No preset · chat | — | — | — | Verify streaming and tool calls through the generic chat configuration. |
| OpenRouter | international | ❌ No preset · chat | ❌ No preset · image generation | ❌ No preset · async video | ❌ No preset · TTS | Verify generic chat compatibility and the media request, polling, and result formats. |
| MiniMax | international | ❌ No preset | ❌ No preset | ❌ No preset | ❌ No preset | Separate endpoint and account validation |
| Alibaba Cloud Model Studio | mainland China | ❌ No preset | ❌ No preset | ❌ No preset | ❌ No preset | Verify the Wan 3.0 workflow in OpenDirector. |
| Alibaba Cloud Model Studio | international | ❌ No preset | ❌ No preset | ❌ No preset | ❌ No preset | Verify model availability and regional endpoints. |
| xAI Grok API | international | ❌ No preset · Grok chat | ❌ No preset · Grok Imagine | ❌ No preset · Grok Imagine | ❌ No preset · standalone speech API | Verify chat compatibility and native media workflows. |
| Google Gemini Developer API | international | ❌ No preset · Gemini chat | ❌ No preset · Gemini image generation | ❌ No preset · Veo | ❌ No preset · Gemini TTS | Verify the Gemini Developer API separately from the Vertex AI endpoint tracked in #144. |
| Z.ai | international | ❌ No preset | ❌ No preset | ❌ No preset | ❌ No preset | Separate endpoint and account validation |
In the capability columns, ✅ indicates a dedicated option in the OpenDirector provider dropdown. ⏳ indicates an open implementation issue. ❌ indicates no dedicated dropdown option. The generic OpenAI-compatible option does not count as a provider preset.
A dash indicates a modality outside the current tracking scope for that row. Self-hosted vLLM-Omni is tracked separately in #224.
Work to track
- Deliver the mainland Zhipu BigModel workflow through [RFC] providers: add Zhipu BigModel for text, image, video, and speech generation #234.
- Fill the mainland MiniMax minimum-workflow gaps with official image generation and synchronous HTTP TTS with preset voices. Track voice management separately when its scope is defined.
- Confirm international MiniMax and Z.ai endpoint, credential, and model coverage before marking any capability verified.
- Track specialist providers through the existing issues in the matrix.
Evidence required to mark a capability verified
Record the endpoint region, model ID, implementation PR, and tested workflow. Verify generated media can be opened and adopted in the project. For full-workflow providers, run all four modalities with one account and key. Include screenshots in each implementation PR showing the workflow and result in OpenDirector.
Activity
wuli666 commentedon Nov 9, 2026
Hi @FayeSpica, I'd like to take the mainland MiniMax gap: official image generation (
image-01) and synchronous TTS with preset voices (speech-2.8).I'll send two PRs, image first, and verify all four modalities with one minimax.cn key. Voice cloning and management stay out of scope. Happy to open sub-issues if you prefer.
PCcoding666 commentedon Nov 10, 2026
Hi @FayeSpica, I’d like to take the mainland China Alibaba Cloud Model Studio (百炼) chat entry. Could I start with a Qwen assistant preset for the Beijing region, using
qwen3.8-maxand an editable workspace endpoint?I tested the existing assistant locally with simulated Qwen responses. It called an OpenDirector tool to write a text asset, received the tool result, and continued its reply. For the PR, I’d verify that same flow with a real Beijing-region account, including a streamed reply and the saved asset, and include screenshots of the Chinese and English setup.
Does that scope work for a first PR? I’m happy to track it in a dedicated sub-issue if that’s easier.
FayeSpica commentedon Nov 10, 2026
Thanks, @wuli666! assigned.
Image first sounds good. Feel free to open the PR and reference #240. If anything needs discussion, please open a sub-issue under #240.
FayeSpica commentedon Nov 10, 2026
Thanks for testing the tool call, @PCcoding666! Go ahead with the Beijing Qwen preset and reference #240 in the PR.
If you’re interested, feel free to open a separate sub-issue under #240 to track full Alibaba Cloud Model Studio support.
FayeSpica commentedon Nov 10, 2026
Hi @wuli666, any updates on the image API integration? MiniMax-H3-Image is out now, so could we prioritize adding support for it?
wuli666 commentedon Nov 11, 2026
Hi @FayeSpica, I saw #280 already landed MiniMax-H3-Image, thanks! I'll take the remaining part: synchronous TTS with preset voices (
speech-2.8).