Skip to content

relay#384: refuse a raw-model start whose model its kind does not offer - #499

Merged
barelyworkingcode merged 4 commits into
mainfrom
fix/384-raw-model-offer
Oct 19, 2026
Merged

barelyworkingcode merged 4 commits into
mainfrom
fix/384-raw-model-offer

Conversation

@barelyworkingcode

Copy link
Copy Markdown
Owner

Closes #384

What changed

A fresh chat, codex or pi start with a raw model and no agent entry is checked against the models its kind offers before launch. This is the check agent-entry starts already run. It sits in AuthorizeLaunch, the one core behind POST /api/sessions, relay session start (flags and --file), the bridge session.start op and the Chief of Staff start.

  • A model the kind does not offer is refused with 400 model_not_offered. The message names the model and the kind, and lists the offered models the project allows (at most 50), spelled as --model takes them.
  • A bare codex or pi gets a spelling hint: codex/<model>, pi/relay-router/<model>. A bare claude gets haiku, sonnet or opus.
  • When relay cannot read the kind's model list, it refuses with 503 models_unavailable, the same as entry starts. Fails closed.
  • Claude aliases and full Claude tier ids are not checked. Resume, entry, terminal and drop-in starts keep their checks. Every earlier refusal keeps its code.
  • session.launch and chief_of_staff.start end denied with the code as reason and the requested model.
  • Docs: routes.md, cli.md, events.md, FEATURES rows G1.01 and G1.26.

What a person notices: asking the Chief of Staff for a "codex session" now gets a refusal that lists real choices. It no longer gets a session that dies with "prompt could not be delivered".

Proof

  • Red on main: on commit 78c7589 (main's code plus the tests), four tests fail at the refusal assertion. They get exit 0, 201, 502 launch failed and 502 prompt_not_delivered.
  • Break-and-restore: making checkRawModel return nil turns TestBug384RawModelNotOfferedRefused red (--model codex exited 0, want 1). The restore is byte for byte. The entry is in e2e/coverage/proof/breaks.json.
  • Real-app check: the installed main build answers relay session start --model codex with started session …, exit 0. The head build answers with model "codex" is not offered for chat; models: …; for a codex session use codex/<model>, exit 1. nosuchmodel-xyz is refused the same way. haiku still starts.

Notes

  • Fix attempts: 1. Review rounds: 1, on the default model, with no blocking findings. Two minor findings are fixed in 33b0a42: the hint named codex/codex, which no model offers, and the events doc. The third minor finding stays: one kind assertion in TestBug384RawCodexAndPiModelNotOffered repeats another, harmless. Tokens: about 560k across subagents.
  • Small deviation within ownership: endRawModelRefusal ends a raw-model 503 denied, which criterion 14 requires. Entry-start 503s still end error. events.md says so.
  • Follow-on not taken: fakerelay still accepts any raw model. No eve test needs the refusal yet.
  • Run report: barelyworkingcode/claude-process#53

Review: no blocking (round 1, default model; minors fixed in 33b0a42)
Red on main: TestBug384RawModelNotOfferedRefused and three more fail on main's code at the refusal assertion (exit 0 / 201 / 502)
Real-app check: seen before on the installed main build (--model codex starts a session); gone on the head build (refused, names codex, lists offered models)
Break-and-restore: checkRawModel returns nil -> TestBug384RawModelNotOfferedRefused red; restored byte for byte
Journey: n/a: the stack api run is green; no journey starts a raw model the catalogue lacks

🤖 Generated with Claude Code

barelyworkingcode and others added 4 commits October 18, 2026 21:25
…rs launches anyway

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@barelyworkingcode

Copy link
Copy Markdown
Owner Author

Verified: 2c5e97b · repo-suite: ok relaye2e/features 478.564s · devboxverify: SUMMARY pass=5 fail=0 blocked=0 notrun=2 (stack r384, --phase api) · real-app: seen before on main (--model codex started a session), gone on head (refused, model_not_offered with choices; haiku still starts) · log: test machine /tmp/r384-e2e.log, /tmp/r384-dbv.log, /tmp/r384-realapp.log

@barelyworkingcode
barelyworkingcode merged commit 9ff41de into main Oct 19, 2026
@barelyworkingcode
barelyworkingcode deleted the fix/384-raw-model-offer branch October 19, 2026 04:42
@barelyworkingcode

Copy link
Copy Markdown
Owner Author

merge-check: merged. approved, target, verified, contract, evidence, owner, weakening

@barelyworkingcode

Copy link
Copy Markdown
Owner Author

Merged: approved, target, verified, contract, evidence PASS; ci NA; no gate or weakening hits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A raw-model start with an unknown model launches anyway and fails with 'prompt could not be delivered'

1 participant