Proposal: Context Budget Optimization & Memory Architecture for Squad #1037

TL;DR

Squad agents get slower the more you use them. Not because of model changes — because of state bloat. After a few weeks of heavy usage, .squad/ files grow until agents are burning most of their context window loading memories instead of doing work. This is a systemic problem that will hit every heavy Squad user eventually. Here's a 3-phase plan to fix it, inspired by Karpathy's wiki architecture and validated by Cloudflare's production Agent Memory system.

The Problem (a.k.a. "Why Are You So Slow Today?")

Here's what happens to a Squad repo after a few weeks of active use:

Most models have a 200K-600K context window. You do the math. Your agents have barely anything left for actual work. And this happens gradually — files grow ~15-20% every two weeks, so you adapt without realizing your team is suffocating.

Proposed Solution: Three-Layer Context Architecture

Inspired by Karpathy's wiki-based memory model — the idea that LLMs should curate knowledge at write time, so runtime loads a clean wiki instead of raw history.

Phase 1: Index + Load Guidance (Quick Win)

Add a .squad/index.md file — a catalog of every .squad/ file with explicit load-guidance tags:

• 🔴 [ALWAYS] — always loaded, total <20K tokens. Team identity, routing, current priorities.
• 🟡 [ON-DEMAND] — loaded only when the task needs it. Recent decisions, domain-specific context.
• 🟢 [ARCHIVE] — never pre-loaded. Grep when you need history.

Add a "Context Loading — Start Here" section to copilot-instructions.md so agents know to read index.md first and respect the tags.

Plus basic hygiene: archive old decisions, trim agent histories, keep only recent entries active.

Expected impact: ~95% token reduction per spawn. The 1.6MB decision archive is still there — you just don't load it every time.

Phase 2: Workstream Isolation

Right now, a blog post agent loads infrastructure decisions and vice versa. That's wasteful.

.squad/workstreams/
├── content/        # Blog, marketing, social
├── infrastructure/ # K8s, CI/CD, deployment  
├── research/       # Studies, experiments
└── security/       # Compliance, privacy

Each workstream gets its own now.md, routing.md, decisions.md. Agents only load their domain. This could push from ~25K to ~9K mandatory tokens (98% reduction from unoptimized baseline).

Phase 3: Knowledge Graph + Automated Hygiene

• Run Graphify on the decision archive → queryable knowledge graph instead of flat files
• Build a lint script (could be a squad doctor extension) that detects bloat, staleness, duplicate entries, orphaned files
• Enforce a write protocol: agents curate wiki at write time, not at runtime

What Cloudflare Taught Us

Cloudflare recently shipped a managed memory service for Workers AI agents. Their production architecture validates this direction:

Squad already has the beginning of this — the Scribe + inbox pattern separates decision ingestion from storage, and an index.md would add a retrieval abstraction layer. What's missing: dedup at ingest, supersession chains, typed classification, and any kind of semantic retrieval.

We don't need to build Cloudflare's full vector database. But the ingestion pipeline concept (verify before storing, deduplicate, classify) could be implemented with simple conventions and a lint step.

Concrete Contribution Ideas

Quick Wins (PR-able today)

1. index.md template — generated at squad init, catalogs all .squad/ files with [ALWAYS]/[ON-DEMAND]/[ARCHIVE] tags
2. Load-guidance convention — document in squad.agent.md so agents respect the tags (relates to Split squad.agent.md to reduce system-prompt overhead via lazy loading #1036)
3. "Read index.md first" in copilot-instructions.md — one line that prevents the whole problem
4. History archival threshold — when history.md exceeds N KB, Scribe auto-archives old entries

Needs Design Discussion

5. Workstream isolation — .squad/workstreams/ as a first-class concept
6. Memory lint / squad doctor — detect bloat, staleness, and budget overruns before they bite
7. Token budget monitoring — warn at init time when mandatory context exceeds 30% of window
8. Scribe archive protocol — auto-archive stale decisions, not just accumulate forever

Research / Long-term

9. Graphify integration — knowledge graph over decision archive for semantic queries. Should ship as an optional Squad extension.
10. Write protocol — formal convention for how agents commit to shared memory (verify → classify → store)
11. Memory API layer — thin abstraction between agents and .squad/ files, inspired by Cloudflare's approach

Why This Belongs Upstream

All of this should be built into Squad itself — not left to individual users to figure out. Every heavy Squad user will eventually hit context budget starvation; the only variable is when. The framework should handle memory management the same way it handles routing and ceremonies: as a first-class concern.

• squad init should generate index.md with load-guidance tags out of the box
• squad.agent.md should enforce "read index.md first" so agents never load the full archive by accident
• Scribe's archive protocol should be default behavior, not an opt-in discovery
• squad doctor should detect bloat before it becomes a performance problem
• Graphify/knowledge-graph tooling should ship as an optional Squad extension

Related Upstream Issues

The Punchline

Your agents aren't getting dumber. They're getting fuller. And full agents are slow agents.

The fix isn't bigger context windows (though those help). The fix is memory management — the same discipline we apply to any system that accumulates state over time. Garbage collection for AI brains, basically.

Happy to dig into any of these with contributors. Pick a number from the list above and let's talk.