Repository navigation
feat: runtime thinking config switching across REPL, ACP, and HTTP - #564
Merged
Merged
Conversation
Store thinking/reasoning configuration as a mutable dict reference
in ctx.config, read per-turn by providers without requiring pipeline
rebuild. This means /thinking changes take effect on the very next
turn with zero rebuilding overhead.
Core architecture:
- ctx.config['thinking_config'] holds {'enabled': bool, 'effort': str}
- This dict is a shared mutable reference: REPL context, session
config, and pipeline ctx all reference the same object
- Providers read thinking_config from stream() kwargs each turn
- No RuntimeReplacementService rebuild needed for thinking-only changes
Per-provider translation:
- Anthropic: effort -> budget_tokens (low=4k, medium=16k, high=32k)
- OpenAI compat: effort -> reasoning_effort parameter
- Codex/DeepSeek: ignore (model-controlled reasoning)
REPL:
- /thinking on|off controls enabled flag
- /thinking effort low|medium|high sets model reasoning depth
- /thinking with no args displays current state
- Backward compatible: context dict structure unified
HTTP:
- PATCH /sessions/{id}/runtime-config accepts 'thinking' field
- Thinking-only changes skip pipeline rebuild (fast path)
- Schema: ThinkingConfigSchema {enabled: bool, effort: str}
ACP:
- AcpMode gains optional 'thinking' field
- CLI shorthand: --acp-mode 'id:name:provider:model::on:high'
- session/set_mode applies thinking directly to session dict
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Require explicit output of decisions and rationale at every decision point before proceeding — choosing approaches, trade-offs, design patterns, scope deferrals, or ambiguous requirements. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (12)
📝 WalkthroughWalkthroughThis PR adds thinking configuration throughout the system, enabling users to control reasoning behavior via session modes, CLI commands, and HTTP APIs. The configuration flows from session storage through runtime context and pipeline into provider implementations, with support for per-call overrides mapping effort levels to provider-specific parameters. ChangesThinking Configuration System
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Poem
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Avoid redundant second fetch of the session object when applying thinking config. The first call on line 461 now captures the return value, used by both the thinking update and the runtime change path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Store thinking/reasoning configuration as a mutable dict reference in ctx.config, read per-turn by providers without requiring pipeline rebuild. This means
/thinkingchanges take effect on the very next turn with zero rebuilding overhead — thinking is not structurally tied to provider/model identity.Problem
Previously:
/thinkingonly controlled UI visibility of thinking chunks (RichConsumer callbacks); it never reached the provider API callthinking: dictat construction time but LLMProviderPlugin never passed itSolution
Single mutable dict
{"enabled": bool, "effort": str}shared across all layers:Per-provider translation
thinking: {type, budget_tokens}reasoning_effortWhy no rebuild
thinking is a per-turn flag, not a structural runtime property. The dict reference pattern means changing thinking doesn't require full pipeline reconstruction — it's immediately visible to the provider on the next stream() call. This is consistent with how RichConsumer already used mutable callbacks for the same state.
Changes
11 files, +158/-40 lines
providers/anthropic.py— stream() reads thinking_config from kwargsproviders/openai_compat.py— stream() reads thinking_config, sets reasoning_effortpipeline.py— passes ctx.config["thinking_config"] to provider.stream()session_manager.py— Session gains mutable thinking_config dict fieldruntime_preparation.py— wires session.thinking_config into ctx.configcommands.py— /thinking uses single dict, writes to shared referencerepl.py— context["thinking_config"] shared with pipeline ctxschemas.py— ThinkingConfigSchema, RuntimeConfigUpdateRequest.thinkinghttp_server.py— PATCH applies thinking, thinking-only skips rebuildacp/server.py— AcpMode.thinking, set_mode applies itacp_command.py— parse thinking shorthand in --acp-modeVerification
Usage
REPL:
HTTP:
ACP:
🤖 Generated with Claude Code
Co-Authored-By: Claude Fable 5 noreply@anthropic.com
Summary by CodeRabbit
New Features
Documentation