Repository navigation
feat(server): continue long idle threads in a fresh provider session - #64
Draft
ScriptedAlchemy wants to merge 8 commits into
Draft
ScriptedAlchemy wants to merge 8 commits into
ScriptedAlchemy wants to merge 8 commits into
Conversation
|
ScriptedAlchemy
force-pushed
the
feat/cache-aware-context-snip
branch
from
November 5, 2026 05:01
aac7be6 to
c506c63
Compare
Cache reads dominate list-price spend, but any cut has to repay the uncached rebuild it causes. These scripts replay local transcripts against T3's own pricing code to price cuts, compaction, and seed selection, with no model calls. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A long thread re-sends its whole history on every call. Once its prompt cache has expired, the next request is uncached anyway, so that is the cheapest moment to drop old history. With the new setting on, a person's message to a thread over 100k tokens, after an hour idle, skips resume and continues through the existing context handoff. Commands, agent messages, and threads with background work are left alone. The setting is off by default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resume the existing native thread before deciding whether to start fresh, preserving external-writer locks and background work discovered during resume. Route the environment-wide setting through the correct settings search scope.
ScriptedAlchemy
force-pushed
the
feat/cache-aware-context-snip
branch
from
November 5, 2026 05:41
26e1b68 to
5a95650
Compare
…eeded Builds a labelled set from local transcripts, split by session, then scores and fine-tunes decision models on it and compares them with plain recency. The set holds chat text, so it stays local. The fit wrapper works around a one-row batch crash in mlx-decision 1.1.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ScriptedAlchemy
marked this pull request as draft
November 5, 2026 07:15
The replay and the relevance scoring read private chats and now run next to that data in a private repo. The cold-return setting does not depend on them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bring the idle-return branch up to date with the merged worktree cleanup fixes and pstack model expectations. Preserve T3_MOBILE_TEST_PLATFORM in the shared provider test environment so the Android preview regression tests run with the intended platform. The existing idle-return changes remain intact. Validation passed 82 ProviderTurnStartService tests, 176 selected check:changed tests, formatting, lint, both knip scans, GraphQL, and React compiler checks. The aggregate typecheck launcher is blocked by the environment's seccomp supervisor.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Cache reads are 70% of list-price spend across 30 days of real sessions, because every call re-sends the whole thread. Once a thread's prompt cache has expired, the next request is uncached anyway, so a fresh session with less history costs little extra and makes every later call cheaper.
What changed
freshSessionOnColdReturnserver setting is off by default. When it is on, a person's message to a thread that is over 100k tokens and has been idle for an hour skips resume. The thread continues in a new native session through the existing context handoff.Scope
Tradeoffs
clear_tool_useslost money or roughly broke even at every setting replayed.Blast radius
The setting is off by default, so nothing changes until someone turns it on. When it is on, the agent loses older context in that session. The agent can read omitted history back through the thread-reading tool, but nobody has measured the effect on answer quality.
Verification
ProviderTurnStartService.test.tstests pass, including three added regression tests. They cover a recent completion after old telemetry, an oversized replacement handoff, and background work that appears during handoff preparation. All three failed before the fixes.vp run check:changedpasses on the latest commit: formatting, lint, both Knip passes, GraphQL, typecheck across 36 packages, seven test files, and React compiler checks.GPT-6 Astra Pro in ChatGPT did the follow-up review and fixes.
Claude Opus 5.5 in Claude Code, through T3 Code.
🤖 Generated with Claude Code