Skip to content

feat(providers): discover capabilities and validate generation inputs - #266

Draft
bread-lxy wants to merge 24 commits into
ThinkFlowLab:mainfrom
bread-lxy:codex/v2-capabilities
Draft

bread-lxy wants to merge 24 commits into
ThinkFlowLab:mainfrom
bread-lxy:codex/v2-capabilities

Conversation

@bread-lxy

@bread-lxy bread-lxy commented Nov 10, 2026 •

Copy link
Copy Markdown

Summary

Discover optional model capabilities before submitting work, and explain incompatible requests without discarding the user's configuration, inputs or selections.

  • Share capability lookup and request checks across connection settings, generation, the assistant, API, CLI and Agent tools. Match the exact model, route, inputs, parameters and required output.
  • Reject new work only on fresh, applicable negative evidence. Preserve existing validation for older services and failed or unknown discovery. Keep accepted-task recovery and replay unchanged.
  • Check image counts without truncation. Region-editing tools use native masks when both the declared route and local adapter support them, preserving requested dimensions and final compositing.
  • Check the exact parameters sent by image adapters, including upstream MiniMax's fixed output/response formats and adjusted tool dimensions. Unsupported mask transport cannot silently discard an input.
  • Keep Chinese and English states current, including generation history after a language change. Explain busy services and blocked result downloads without weakening download security.

Linked issue

Refs #224 (V2). The companion Omni S2 PR supplies the discovery endpoint.

Labels

type:feat, area:provider, area:canvas, area:agent.

Test plan

Windows, Python 3.12.7 and Node 24.22.1. Integrated upstream 4d15b4e; final code a0fbd18.

  • 244 passed across image adapters, request parameters, capability decisions, creation acceptance, native-mask tools, image edits and saved-task behavior. MiniMax checks compare checked parameters with serialized requests, reject incompatible formats through service/HTTP/Agent with zero jobs and zero generation requests, and preserve upstream ordered-image, download-recovery and no-resubmission behavior.
  • 24 passed for the subsequently integrated GLM chat mappings and assistant conversations; 50 passed in the related frontend checks.
  • Contract generation, import boundaries, bilingual catalogs, browser manifest and production build passed.
  • The final capability browser workflow passed in both languages: save/refresh, unchanged drafts, exact-input rejection, successful UI submission and saved media, pause expiry, stale-response handling and preserved history. Earlier voice-library and history-language browser checks also passed.

Focused test entry points: tests/test_minimax_image.py, tests/test_model_availability.py, tests/test_request_capability_acceptance.py, tests/test_tool_native_mask.py and tests/browser/capability_discovery.mjs. Setup and commands are in the validation guide.

Upstream Z-Image sampling defaults apply only to text-to-image jobs. Capability validation checks the saved sampling values; editing and accepted-task recovery retain their applicable parameters. MiniMax retains its existing routes, image ordering, 16-image limit and result-download checks.

Evidence that it works

Example capability cards from the controlled UI regression:

English capability card

中文能力卡

Provider verification

The normal application completed real GLM chat and MOSS speech requests. The saved speech file played successfully; its transcription matched the submitted text. A failed capability lookup against a legacy service was followed by a successful real chat request without changing configuration or existing tasks.

Capability-state and image-edit regressions use controlled services and fixture media. Real-model mask adherence remains unverified.

Bilingual UI

Both zh-CN and en-US were checked in the browser, including live language switching, rejection messages and retained voice selections.

AI assistance

This contribution was made with assistance from Codex.

## summary

- Apply declared string enum normalization only while comparing values,
  retaining original input and treating unknown rules as unknown.
- Count content references without treating metadata or cached video
  continuations as uploaded media; retain endpoint pause enforcement.
- Share specific constraint errors across admission and execution, keep
  nested receipt codes, and present image geometry failures bilingually.
- Document accepted image request policies and cover actual speech
  payloads, continuation job IDs, and rejection before task creation.

## validation

- Targeted capability, Agent and cache checks: 106 passed, 1 skipped.
- Actual request projection and H3 checks: 61 passed, model doubles.
- Frontend unit tests: 763 passed with PYTHONUTF8=1 on Windows.
- TypeScript/build, i18n, browser manifests, generated contracts and
  import-layer checks passed. Full Python suite is still running.
## summary

Honor a globally disabled exact HTTP endpoint when a continuation
profile is undeclared, without borrowing another profile's inputs.
Keep route_disabled as the rejection reason during a fresh pause.
Verify the public command API creates no continuation job or request
on the disabled endpoint and preserves the original remote job ID
when the endpoint is available.

## validation

The projection and public HTTP regressions both failed before the fix.
The three affected test modules now pass all 110 cases with model
doubles and real temporary media; no real model inference was run.
## summary

Apply the exact global HTTP endpoint closure after profile matching so
both a missing profile and missing profile-route details still reject
new work. Preserve already explicit operation and profile rejections.

## validation

The missing-profile-route regression failed before the fix. All 111
capability projection, availability and H3 tests pass afterward,
including public command rejection without a new model request.
@FayeSpica FayeSpica added type:feat New capability or behaviour area:provider Model providers and generation protocols area:canvas Canvas, nodes and direct manipulation area:agent Agent runtime, assistant chat and tools labels Nov 10, 2026
@Xingkai98

Copy link
Copy Markdown

看过了,整体扎实,判定核心和 fail-open 的骨架是对的(我在本地跑了你的能力子集:243 passed / 1 skipped、node 测试全绿、check:i18n 干净)。

三点想确认:

  1. observed_at 没做年龄校验,只有 30s 缓存 TTL。我用 21 天前观测、support: unsupported 的文档实测,仍会硬拒;建议给声明加年龄上限,超龄归 unknown。
  2. 部署级 routes 只按 (method, path) 键控且优先级最高——profile 写 enabled、全局写 disabled 时仍会拒。共享路径(TTS 与音乐同走 /v1/audio/speech)想按 stage 分别关闭,目前表达不了。
  3. stage_ids 解析了但全仓没有读取方,requests_for() 按 kind+protocol 硬编码 profile 名;ASR / 扒谱走空列表 → adapter_not_mapped。

V1 我只动 providerPresets.ts 和文案,ModelServiceForm.tsx 等你合了再 rebase。

关于第 2、3 点我先出了个 PR 探形状:#278 —— 7 个新文件 + docs/README.md 一行索引、零依赖(不 import 你任何符号,可独立合),内容是「上游 (operation, profile) → 本仓 capability id」的映射表 + 三档 fixture + 69 个测试。我按上游 #10102 的 d1104f8 取证过(现已到 43ff7b9,词表未变):词表是 6 个 operation、12 个 profile,speech 一条路径上挂着 text_to_speech/lyrics_to_music/caption_to_audio 三个 profile,所以键必须是 (operation, profile),光靠 route 或 stage 都不成立(stages[] 只有 id/type/depends_on,且每个 model 拿到的是同一份全量列表,拿它判定会全员同状态)。

你看这个形状能接受吗?如果接受,你那边 requests_for() 可以改成吃 capabilities_for(operation, profile),我就不再另造 operation 常量;不接受我们换个切法。


Read through #266 — the core is solid: the admission logic and fail-open skeleton hold up (I ran your capability subset locally: 243 passed / 1 skipped, node tests green, check:i18n clean).

Three things I'd like to confirm:

  1. observed_at is never age-checked — only the 30s cache TTL. With a document observed 21 days ago declaring support: unsupported, I still get a hard rejection. Suggest a max declaration age falling back to unknown.
  2. Deployment-level routes is keyed by (method, path) only and wins over the profile: a profile route marked enabled is still rejected when the global entry says disabled. Per-stage closure on a shared path (TTS vs music on /v1/audio/speech) can't be expressed today.
  3. stage_ids is parsed but never read, and requests_for() hard-codes profile ids from kind+protocol; ASR/score return [] → adapter_not_mapped.

On my side V1 only touches providerPresets.ts and copy, so we won't collide; I'll rebase on ModelServiceForm.tsx after you merge.

For points 2 and 3 I opened a small PR to test the shape: #278 — 7 new files plus a one-line index entry in docs/README.md, zero dependencies (imports nothing from your code, can merge on its own). It is the (operation, profile) → capability id table plus three fixture tiers and 69 tests, keyed to upstream #10102 at d1104f8 (now 43ff7b9; the vocabulary is unchanged): six operations, twelve profiles, and speech carries text_to_speech / lyrics_to_music / caption_to_audio on one path — so a route-only or stage-only key cannot work (stages[] only has id/type/depends_on, and every model receives the same full list, so keying on it collapses all models to one state).

Would that shape work for you? If so, requests_for() could consume capabilities_for(operation, profile) and neither of us needs a second set of operation constants.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agent Agent runtime, assistant chat and tools area:canvas Canvas, nodes and direct manipulation area:provider Model providers and generation protocols type:feat New capability or behaviour

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants