feat: Cooperative Rate Limiting & Predictive Circuit Breaker for Multi-Agent Deployments #515

Feature: Cooperative Rate Limiting & Predictive Circuit Breaker for Multi-Agent Deployments

Problem

Issue #450 added a circuit breaker template for model rate limiting — great foundation. But in production with multiple Ralphs across machines (or pods on K8s), single-instance circuit breakers aren't enough. Each Ralph independently hits API limits, causing cascading failures across the fleet.

The core problems at scale:

1. No coordination — 5 Ralphs each think they have full API quota
2. Thundering herd — All Ralphs retry simultaneously after rate limit resets
3. Priority inversion — Low-priority work exhausts quota before critical work runs
4. Reactive only — Circuit opens AFTER 429, wasting the failed request

Proposed Solution: 6-Pattern Rate Limiting Architecture

1. Traffic Light (RAAS) — GREEN/AMBER/RED throttling from response headers:

• GREEN (>20% remaining): Normal operation
• AMBER (5-20%): Only P0 agents proceed
• RED (<5%): Block all except emergency P0

2. Cooperative Token Pool (CMARP) — Shared ~/.squad/rate-pool.json:

• P0 agents (Lead) get 40% of quota
• P1 agents (specialists) get 35%
• P2 agents (Ralph, Scribe) get 25%
• Stale leases auto-recovered after 5 minutes

3. Predictive Circuit Breaker (PCB) — Opens BEFORE 429:

• Tracks last 10 (timestamp, remaining) pairs from response headers
• Linear regression predicts ETA to quota exhaustion
• Opens circuit when ETA < 120 seconds
• Prevents the 429 from ever happening

4. Priority Retry Windows (PWJG) — Non-overlapping jitter:

• P0: 500ms–5s retry window
• P1: 2s–30s retry window
• P2: 5s–60s retry window
• Higher priority agents recover first

5. Resource Epoch Tracker (RET) — Heartbeat-leased allocations:

• Each agent registers a 5-minute lease
• Crashed agent allocations auto-recycle

6. Cascade Dependency Detector (CDD) — Backpressure from downstream failures

Implementation Available

I have all 6 patterns implemented and running in production:

• rate-limit-manager.ps1 (200+ lines) — Cooperative rate pooling
• Predictive circuit breaker in ralph-watch.ps1
• KEDA integration via keda-copilot-scaler (Go, 51 tests) — scales pods to zero when rate limited
• Running with 7+ concurrent agents on AKS

Builds on top of the existing ralph-circuit-breaker.md template from #450/#451.

References

• Extends Feature: Circuit breaker for model rate limiting in Ralph #450 (circuit breaker template — merged as docs-only)
• Related to feat: persistent Ralph — wire squad watch + enable heartbeat cron #236 (persistent Ralph)