diff --git a/.squad-templates/ralph-circuit-breaker.md b/.squad-templates/ralph-circuit-breaker.md new file mode 100644 index 000000000..87be26015 --- /dev/null +++ b/.squad-templates/ralph-circuit-breaker.md @@ -0,0 +1,313 @@ +# Ralph Circuit Breaker — Model Rate Limit Fallback + +> Classic circuit breaker pattern (Hystrix / Polly / Resilience4j) applied to Copilot model selection. +> When the preferred model hits rate limits, Ralph automatically degrades to free-tier models, then self-heals. + +## Problem + +When running multiple Ralph instances across repos, Copilot model rate limits cause cascading failures. +All Ralphs fail simultaneously when the preferred model (e.g., `claude-sonnet-4.6`) hits quota. + +Premium models burn quota fast: +| Model | Multiplier | Risk | +|-------|-----------|------| +| `claude-sonnet-4.6` | 1x | Moderate with many Ralphs | +| `claude-opus-4.6` | 10x | High | +| `gpt-5.4` | 50x | Very high | +| `gpt-5.4-mini` | **0x** | **Free — unlimited** | +| `gpt-5-mini` | **0x** | **Free — unlimited** | +| `gpt-4.1` | **0x** | **Free — unlimited** | + +## Circuit Breaker States + +``` +┌─────────┐ rate limit error ┌────────┐ +│ CLOSED │ ───────────────────► │ OPEN │ +│ (normal)│ │(fallback)│ +└────┬────┘ ◄──────────────── └────┬────┘ + │ 2 consecutive │ + │ successes │ cooldown expires + │ ▼ + │ ┌──────────┐ + └───── success ◄──────── │HALF-OPEN │ + (close) │ (testing) │ + └──────────┘ +``` + +### CLOSED (normal operation) +- Use preferred model from config +- Every successful response confirms circuit stays closed +- On rate limit error → transition to OPEN + +### OPEN (rate limited — fallback active) +- Fall back through the free-tier model chain: + 1. `gpt-5.4-mini` + 2. `gpt-5-mini` + 3. `gpt-4.1` +- Start cooldown timer (default: 10 minutes) +- When cooldown expires → transition to HALF-OPEN + +### HALF-OPEN (testing recovery) +- Try preferred model again +- If 2 consecutive successes → transition to CLOSED +- If rate limit error → back to OPEN, reset cooldown + +## State File: `.squad/ralph-circuit-breaker.json` + +```json +{ + "state": "closed", + "preferredModel": "claude-sonnet-4.6", + "fallbackChain": ["gpt-5.4-mini", "gpt-5-mini", "gpt-4.1"], + "currentFallbackIndex": 0, + "cooldownMinutes": 10, + "openedAt": null, + "halfOpenSuccesses": 0, + "consecutiveFailures": 0, + "metrics": { + "totalFallbacks": 0, + "totalRecoveries": 0, + "lastFallbackAt": null, + "lastRecoveryAt": null + } +} +``` + +## PowerShell Functions + +Paste these into your `ralph-watch.ps1` or source them from a shared module. + +### `Get-CircuitBreakerState` + +```powershell +function Get-CircuitBreakerState { + param([string]$StateFile = ".squad/ralph-circuit-breaker.json") + + if (-not (Test-Path $StateFile)) { + $default = @{ + state = "closed" + preferredModel = "claude-sonnet-4.6" + fallbackChain = @("gpt-5.4-mini", "gpt-5-mini", "gpt-4.1") + currentFallbackIndex = 0 + cooldownMinutes = 10 + openedAt = $null + halfOpenSuccesses = 0 + consecutiveFailures = 0 + metrics = @{ + totalFallbacks = 0 + totalRecoveries = 0 + lastFallbackAt = $null + lastRecoveryAt = $null + } + } + $default | ConvertTo-Json -Depth 3 | Set-Content $StateFile + return $default + } + + return (Get-Content $StateFile -Raw | ConvertFrom-Json) +} +``` + +### `Save-CircuitBreakerState` + +```powershell +function Save-CircuitBreakerState { + param( + [object]$State, + [string]$StateFile = ".squad/ralph-circuit-breaker.json" + ) + + $State | ConvertTo-Json -Depth 3 | Set-Content $StateFile +} +``` + +### `Get-CurrentModel` + +Returns the model Ralph should use right now, based on circuit state. + +```powershell +function Get-CurrentModel { + param([string]$StateFile = ".squad/ralph-circuit-breaker.json") + + $cb = Get-CircuitBreakerState -StateFile $StateFile + + switch ($cb.state) { + "closed" { + return $cb.preferredModel + } + "open" { + # Check if cooldown has expired + if ($cb.openedAt) { + $opened = [DateTime]::Parse($cb.openedAt) + $elapsed = (Get-Date) - $opened + if ($elapsed.TotalMinutes -ge $cb.cooldownMinutes) { + # Transition to half-open + $cb.state = "half-open" + $cb.halfOpenSuccesses = 0 + Save-CircuitBreakerState -State $cb -StateFile $StateFile + Write-Host " [circuit-breaker] Cooldown expired. Testing preferred model..." -ForegroundColor Yellow + return $cb.preferredModel + } + } + # Still in cooldown — use fallback + $idx = [Math]::Min($cb.currentFallbackIndex, $cb.fallbackChain.Count - 1) + return $cb.fallbackChain[$idx] + } + "half-open" { + return $cb.preferredModel + } + default { + return $cb.preferredModel + } + } +} +``` + +### `Update-CircuitBreakerOnSuccess` + +Call after every successful model response. + +```powershell +function Update-CircuitBreakerOnSuccess { + param([string]$StateFile = ".squad/ralph-circuit-breaker.json") + + $cb = Get-CircuitBreakerState -StateFile $StateFile + $cb.consecutiveFailures = 0 + + if ($cb.state -eq "half-open") { + $cb.halfOpenSuccesses++ + if ($cb.halfOpenSuccesses -ge 2) { + # Recovery! Close the circuit + $cb.state = "closed" + $cb.openedAt = $null + $cb.halfOpenSuccesses = 0 + $cb.currentFallbackIndex = 0 + $cb.metrics.totalRecoveries++ + $cb.metrics.lastRecoveryAt = (Get-Date).ToString("o") + Save-CircuitBreakerState -State $cb -StateFile $StateFile + Write-Host " [circuit-breaker] RECOVERED — back to preferred model ($($cb.preferredModel))" -ForegroundColor Green + return + } + Save-CircuitBreakerState -State $cb -StateFile $StateFile + Write-Host " [circuit-breaker] Half-open success $($cb.halfOpenSuccesses)/2" -ForegroundColor Yellow + return + } + + # closed state — nothing to do +} +``` + +### `Update-CircuitBreakerOnRateLimit` + +Call when a model response indicates rate limiting (HTTP 429 or error message containing "rate limit"). + +```powershell +function Update-CircuitBreakerOnRateLimit { + param([string]$StateFile = ".squad/ralph-circuit-breaker.json") + + $cb = Get-CircuitBreakerState -StateFile $StateFile + $cb.consecutiveFailures++ + + if ($cb.state -eq "closed" -or $cb.state -eq "half-open") { + # Open the circuit + $cb.state = "open" + $cb.openedAt = (Get-Date).ToString("o") + $cb.halfOpenSuccesses = 0 + $cb.currentFallbackIndex = 0 + $cb.metrics.totalFallbacks++ + $cb.metrics.lastFallbackAt = (Get-Date).ToString("o") + Save-CircuitBreakerState -State $cb -StateFile $StateFile + + $fallbackModel = $cb.fallbackChain[0] + Write-Host " [circuit-breaker] RATE LIMITED — falling back to $fallbackModel (cooldown: $($cb.cooldownMinutes)m)" -ForegroundColor Red + return + } + + if ($cb.state -eq "open") { + # Already open — try next fallback in chain if current one also fails + if ($cb.currentFallbackIndex -lt ($cb.fallbackChain.Count - 1)) { + $cb.currentFallbackIndex++ + $nextModel = $cb.fallbackChain[$cb.currentFallbackIndex] + Write-Host " [circuit-breaker] Fallback also limited — trying $nextModel" -ForegroundColor Red + } + # Reset cooldown timer + $cb.openedAt = (Get-Date).ToString("o") + Save-CircuitBreakerState -State $cb -StateFile $StateFile + } +} +``` + +## Integration with ralph-watch.ps1 + +In your Ralph polling loop, wrap the model selection: + +```powershell +# At the top of your polling loop +$model = Get-CurrentModel + +# When invoking copilot CLI +$result = copilot-cli --model $model ... + +# After the call +if ($result -match "rate.?limit" -or $LASTEXITCODE -eq 429) { + Update-CircuitBreakerOnRateLimit +} else { + Update-CircuitBreakerOnSuccess +} +``` + +### Full integration example + +```powershell +# Source the circuit breaker functions +. .squad-templates/ralph-circuit-breaker-functions.ps1 + +while ($true) { + $model = Get-CurrentModel + Write-Host "Polling with model: $model" + + try { + # Your existing Ralph logic here, but pass $model + $response = Invoke-RalphCycle -Model $model + + # Success path + Update-CircuitBreakerOnSuccess + } + catch { + if ($_.Exception.Message -match "rate.?limit|429|quota|Too Many Requests") { + Update-CircuitBreakerOnRateLimit + # Retry immediately with fallback model + continue + } + # Other errors — handle normally + throw + } + + Start-Sleep -Seconds $pollInterval +} +``` + +## Configuration + +Override defaults by editing `.squad/ralph-circuit-breaker.json`: + +| Field | Default | Description | +|-------|---------|-------------| +| `preferredModel` | `claude-sonnet-4.6` | Model to use when circuit is closed | +| `fallbackChain` | `["gpt-5.4-mini", "gpt-5-mini", "gpt-4.1"]` | Ordered fallback models (all free-tier) | +| `cooldownMinutes` | `10` | How long to wait before testing recovery | + +## Metrics + +The state file tracks operational metrics: + +- **totalFallbacks** — How many times the circuit opened +- **totalRecoveries** — How many times it recovered to preferred model +- **lastFallbackAt** — ISO timestamp of last rate limit event +- **lastRecoveryAt** — ISO timestamp of last successful recovery + +Query metrics with: +```powershell +$cb = Get-Content .squad/ralph-circuit-breaker.json | ConvertFrom-Json +Write-Host "Fallbacks: $($cb.metrics.totalFallbacks) | Recoveries: $($cb.metrics.totalRecoveries)" +```