Repository navigation
Dato Agent: avoid Anthropic output-limit failures on large writes - #160
Merged
roger-datocms merged 7 commits intoNov 4, 2026
Merged
Conversation
- Use the 64k output budget (bounded by the model maximum) at every effort; adaptive thinking shares max_tokens with the reply and tool input. - When a response stops at max_tokens, discard it (none of its tool calls ran) and retry once with an instruction to split the work into smaller scripts. Co-authored-by: Cursor <cursoragent@cursor.com>
Member
|
(Claude here [Opus 5.5]) TriageStatus: still relevant. Master ( Mergeability:
ReviewThe diagnosis matches the code. Adaptive thinking shares Correct:
Minor points (non-blocking):
RecommendationMerge after a maintainer review. Address the |
This was referenced Oct 19, 2026
Every step of a tool loop resends the tools, the system prompt and the whole history. Without cache_control, each step paid full input price for all of it, so splitting a large write into smaller scripts multiplied input cost and latency. Mark the system prompt (which also caches the tools before it) and set top-level automatic caching, so each step reads the previous step's prefix. A live smoke turn with five steps dropped from $0.0104 to $0.0041. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Live tests run the real runtime (createAgentRuntime) against a real provider and the real DatoCMS Remote MCP, in a forked environment of a seed project, and assert on the project's end state. Evals compare request variants and write tables of rates and costs. Neither runs in `npm test` or gates anything: `npm run test:live` and `npm run eval`. - e2e/setup: seed the project schema; mint a project-scoped MCP token - e2e/lib: env loading, fetch tap for usage and request rewrites, environment forks, the approve-everything turn driver, spend ledger with a per-key cap (default $10) - smoke and output-limit live tests: one write; a 30-record large write; the max_tokens discard-and-retry path under a forced low ceiling - output-limit eval: limit x script guidance x task budget x caching Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
max_tokens is a hard cutoff the model cannot see, so capping it below what the model supports only truncates large requests; script size is the system prompt's job. Use the maximum discovered from the Models API (128k on current models) and fall back to 64k, the smallest maximum of any active Claude model. Anthropic bills only generated tokens, and max_tokens does not count toward output rate limits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a reply hits the output limit, the runtime now shows a status line saying nothing in it ran and that it is redoing the work in smaller steps, so the extra approvals that follow make sense. When the retry also fails, or the chat outgrows the context window, the error names a remedy the editor can act on: ask for fewer records at a time, or start a new chat. OpenAI's incomplete reasons (max_output_tokens, content_filter) get the same treatment instead of a raw reason code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The config screen is developer-facing, and few people know a model's limits offhand. Anthropic's model list reports both, so label each option, e.g. "Claude Haiku 5.5 · 1M context · 128K output". OpenAI's list reports neither, so its options keep their plain labels. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Member
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Fixes #158.
What
resolveAnthropicMaxOutputTokensnow uses the 64k budget at every reasoning effort, still capped by the model's advertised maximum (and 16k when the maximum is unknown). Adaptive thinking sharesmax_tokenswith the reply and the tool input, so the previous 16k cap at low/medium/high left too little room for multi-record write scripts. Anthropic bills only generated tokens.max_tokens, the runtime discards it instead of ending the turn. The truncated assistant message is not kept, and none of its tool calls were dispatched. It then retries once, with a user-turn instruction to keep reasoning brief and split large writes into smaller scripts. Any partial visible text stays in the transcript, followed by a paragraph break. A second truncation in the same turn still ends with the existing retryableincompleteerror.model_context_window_exceededis unchanged.Tests
tool_useis discarded, MCP is never called, and no approval is raised. The retry request has no assistant turn and carries the recovery instruction.max_tokensstops with the retryableincompleteerror.npm run typecheck,npm run lint,npm test(952 passed).