Skip to content

/cheap: keep the 1-hour prompt cache warm while you are away - #28

Merged
Darknode-Official merged 1 commit into
mainfrom
feat/cheap-keep-warm
Nov 11, 2026
Merged

Darknode-Official merged 1 commit into
mainfrom
feat/cheap-keep-warm

Conversation

@Darknode-Official

Copy link
Copy Markdown
Owner

With /cheap on (claude engine), Nexus keeps a 1-hour prompt cache warm while you are away, so coming back does not rewrite the whole context at the write rate.

How

  • In the last 4 minutes before expiry, a one-turn ping resumes the session with --fork-session --no-session-persistence --max-turns 1. Nothing is saved and the conversation is unchanged.
  • It reuses the last real launch options so it hits the same cached prefix. Hooks are off for the ping only.
  • Runs only on a 1h TTL, with at least 20k tokens of context, while idle, and up to 6h after your last message.
  • Priced from the ping's own usage, because total_cost_usd and modelUsage are cumulative on a resumed run. /cheap status shows the count and cost.

Verified

  • Real Haiku run: the ping read the full ~27k context from cache, wrote 50 tokens, and returned ok. The original transcript has no trace of it, and no fork file was created.
  • 14-day replay of real traffic at Fable rates: about $43 net saved.
  • HOME=/tmp/nxhome-full npm test: all suites pass (8 new keep-warm tests).

On a 1-hour cache, returning after the hour rewrites the whole context at
the write rate (about 80x a read on Fable). With /cheap on, Nexus now sends
a one-turn ping in the last 4 minutes before expiry, which reads the cache
and restarts the hour.

- The ping resumes the session with --fork-session --no-session-persistence
  --max-turns 1, so no transcript is written and the conversation is
  unchanged. It reuses the last real launch (model, effort, system prompt,
  skill overrides, MCP config), so it hits the same cached prefix.
- Hooks are off for the ping only; they are not part of the cached prefix.
- Only on a 1h TTL, with 20k+ context, idle, and within 6h of the last
  message. A 5-minute cache is cheaper to rebuild than to keep warm.
- Priced from the ping's own usage: on a resumed run total_cost_usd and
  modelUsage are the session's running totals.
- /cheap status shows the refresh count and their cost.

Measured on Haiku: the ping read the full ~27k context from cache, wrote
50 tokens, left the original transcript untouched and saved no fork.
14-day replay of real traffic at Fable rates: about $43 net saved.
@Darknode-Official
Darknode-Official merged commit b410e8e into main Nov 11, 2026
6 checks passed
@Darknode-Official
Darknode-Official deleted the feat/cheap-keep-warm branch November 11, 2026 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant