Repository navigation
/cheap: keep the 1-hour prompt cache warm while you are away - #28
Merged
Merged
Conversation
On a 1-hour cache, returning after the hour rewrites the whole context at the write rate (about 80x a read on Fable). With /cheap on, Nexus now sends a one-turn ping in the last 4 minutes before expiry, which reads the cache and restarts the hour. - The ping resumes the session with --fork-session --no-session-persistence --max-turns 1, so no transcript is written and the conversation is unchanged. It reuses the last real launch (model, effort, system prompt, skill overrides, MCP config), so it hits the same cached prefix. - Hooks are off for the ping only; they are not part of the cached prefix. - Only on a 1h TTL, with 20k+ context, idle, and within 6h of the last message. A 5-minute cache is cheaper to rebuild than to keep warm. - Priced from the ping's own usage: on a resumed run total_cost_usd and modelUsage are the session's running totals. - /cheap status shows the refresh count and their cost. Measured on Haiku: the ping read the full ~27k context from cache, wrote 50 tokens, left the original transcript untouched and saved no fork. 14-day replay of real traffic at Fable rates: about $43 net saved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With /cheap on (claude engine), Nexus keeps a 1-hour prompt cache warm while you are away, so coming back does not rewrite the whole context at the write rate.
How
--fork-session --no-session-persistence --max-turns 1. Nothing is saved and the conversation is unchanged.usage, becausetotal_cost_usdandmodelUsageare cumulative on a resumed run./cheapstatus shows the count and cost.Verified
ok. The original transcript has no trace of it, and no fork file was created.HOME=/tmp/nxhome-full npm test: all suites pass (8 new keep-warm tests).