Skip to content

fix(llm): accept list-shaped message content from chat completions - #1663

Open
localhostdotexe wants to merge 4 commits into
usestrix:mainfrom
localhostdotexe:fix/content-block-messages
Open

localhostdotexe wants to merge 4 commits into
usestrix:mainfrom
localhostdotexe:fix/content-block-messages

Conversation

@localhostdotexe

Copy link
Copy Markdown

Closes #1357

Mistral's reasoning models (and some other OpenAI-compatible servers) send message.content as a list of blocks instead of a string, both in full responses and in every stream delta:

[{"type": "thinking", "thinking": [{"type": "text", "text": "..."}]}, {"type": "text", "text": "OK"}]

The SDK's chat-completions converter assumes a string, so the first call dies with a ResponseOutputText (or ResponseTextDeltaEvent when streaming) validation error and Strix reports LLM CONNECTION FAILED.

This flattens that content before the SDK reads it. StrixProvider.get_model now returns BlockTolerantChatCompletionsModel (new strix/llm/content_blocks.py) in place of the stock OpenAIChatCompletionsModel. It overrides _fetch_response, so streaming and non-streaming are handled in one spot. Text blocks become the message text, thinking blocks go to reasoning_content so the SDK keeps them as a reasoning item, and unknown block types are skipped. String content is untouched, and the LiteLLM route and custom subclasses are left alone.

I went with a subclass rather than patching chatcmpl_converter so nothing changes globally, and it's easy to drop if the SDK starts handling this itself.

tests/test_content_blocks.py runs against a local server that replies the way Mistral does. It checks that the stock model still fails (so the test really exercises the bug), that both paths return the text plus a reasoning item, that a thinking-only turn keeps its tool call, that string content is unchanged, and that the provider wiring is in place. ruff and mypy are clean, and the rest of the suite has the same failures here as on main.

…sestrix#1357)

Mistral and other OpenAI-compatible endpoints can send message content as a list of typed blocks (thinking + text) instead of a string. The SDK expects a string and fails validation on the first call. Add a chat-completions model that flattens that content: text blocks become the message text, thinking blocks become reasoning_content, unknown blocks are skipped.
StrixProvider.get_model now swaps the stock OpenAIChatCompletionsModel for BlockTolerantChatCompletionsModel. Custom subclasses and the LiteLLM route are unchanged.
Local server that answers like Mistral's reasoning models. Shows the stock SDK model failing on both paths and the tolerant model returning the text with the thinking kept as a reasoning item.
@greptile-apps

greptile-apps Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The follow-up changes appear safe to merge.

Summary

This PR lets chat-completions models accept replies made of content blocks while keeping text and reasoning separate.

  • Chat completions now turn content blocks into text and reasoning.

Reviews (2) · Last reviewed commit: "test: don't leak globals from the provid..." · Reviewed by Greptile

Comment thread tests/test_content_blocks.py Resolved
configure_sdk_model_defaults sets litellm.api_key/api_base/headers, the SDK default route and OPENAI_API_KEY process-wide. Register them with monkeypatch so later tests don't inherit the fake endpoint.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] ResponseOutputText mishandles thinking text in blocks

1 participant