Skip to content

Feature: Circuit breaker for model rate limiting in Ralph #450

New issue
New issue
Closed
#451
Closed
Feature: Circuit breaker for model rate limiting in Ralph#450
#451
Labels
go:needs-researchNeeds investigationpriority:p2Next sprintsquadSquad triage inbox — Lead will assign to a memberstatus:backlogBacklog itemtype:featureNew capability

Description

@tamirdresher
tamirdresher
opened on Mar 18, 2026

Problem

When running multiple Ralph instances, Copilot model rate limits can cause cascading failures. All Ralphs fail simultaneously when the preferred model hits its quota.

Proposed Solution

Add a circuit breaker pattern that automatically falls back to free-tier models when rate limited, then recovers:

  • CLOSED (normal): Use preferred model (configurable, default claude-sonnet-4.6)
  • OPEN (rate limited): Fall back through model chain: gpt-5.4-mini → gpt-5-mini → gpt-4.1
  • HALF-OPEN (testing): After cooldown, try preferred model. 2 consecutive successes → close circuit.

State tracked in .squad/ralph-circuit-breaker.json with metrics (total fallbacks, recoveries).

Why this matters

  • Free-tier models (gpt-4.1, gpt-5-mini, gpt-5.4-mini) have 0x multiplier — unlimited usage
  • Premium models (Sonnet=1x, Opus=10x, GPT-5.4=50x) burn quota fast with multiple Ralphs
  • Without fallback, rate limits cause complete Ralph outage
  • With circuit breaker, Ralph degrades gracefully and self-heals

Implementation

This follows the classic circuit breaker pattern (Netflix Hystrix / Polly / Resilience4j) applied to model selection. The fallback chain acts like a CDN tier — always prefer premium, degrade to free tier when unavailable, recover when possible.

Real-world validation

Tested with 8 concurrent Ralphs across 8 repos. Rate limit hit after ~2 hours of intensive polling. Circuit breaker kicked in, Ralphs continued on gpt-5.4-mini, recovered to preferred model after 10-minute cooldown.

Activity

  1. tamirdresher
    added a commit that references this issue on Mar 18, 2026

    Add circuit breaker for model rate limiting in Ralph

    f466cca
  2. bradygaster
    added
    type:featureNew capability
    priority:p2Next sprint
    status:backlogBacklog item
    squad:eecom
    squadSquad triage inbox — Lead will assign to a member
    on Mar 18, 2026
  3. github-actions
    added
    squad:fido
    on Mar 18, 2026

  4. github-actions commented on Mar 18, 2026

    @github-actions
    github-actionsBot
    on Mar 18, 2026 – with GitHub Actions
    Contributor

    🏗️ Squad Triage — Flight (Lead)

    Issue: #450 — Feature: Circuit breaker for model rate limiting in Ralph
    Assigned to: FIDO (Quality Owner)
    Reason: Issue relates to testing/quality work

    @copilot evaluation: 🔴 Not suitable — issue involves work outside the coding agent's capability profile.

    Team roster:

    • Flight (Lead) → label: squad:flight
    • Procedures (Prompt Engineer) → label: squad:procedures
    • EECOM (Core Dev) → label: squad:eecom
    • FIDO (Quality Owner) → label: squad:fido
    • PAO (DevRel) → label: squad:pao
    • CAPCOM (SDK Expert) → label: squad:capcom
    • CONTROL (TypeScript Engineer) → label: squad:control
    • Surgeon (Release Manager) → label: squad:surgeon
    • Booster (CI/CD Engineer) → label: squad:booster
    • GNC (Node.js Runtime) → label: squad:gnc
    • Network (Distribution) → label: squad:network
    • RETRO (Security) → label: squad:retro
    • INCO (CLI UX & Visual Design) → label: squad:inco
    • GUIDO (VS Code Extension) → label: squad:guido
    • Telemetry (Aspire & Observability) → label: squad:telemetry
    • VOX (REPL & Interactive Shell) → label: squad:vox
    • DSKY (TUI Engineer) → label: squad:dsky
    • Sims (E2E Test Engineer) → label: squad:sims
    • Handbook (SDK Usability) → label: squad:handbook
    • Ralph (Work Monitor) → label: squad:ralph
    • @copilot (Coding Agent) → label: squad:copilot

    To reassign, remove the current squad:* label and add the correct one.

  5. github-actions
    added
    go:needs-researchNeeds investigation
    on Mar 18, 2026

  6. github-actions commented on Mar 18, 2026

    @github-actions
    github-actionsBot
    on Mar 18, 2026 – with GitHub Actions
    Contributor

    📋 Assigned to EECOM (Core Dev)

    Issue: #450 — Feature: Circuit breaker for model rate limiting in Ralph

    EECOM will pick this up in the next Copilot session.

    For Copilot coding agent: If enabled, this issue will be worked automatically.
    Otherwise, start a Copilot session and say:
    EECOM, work on issue #450

  7. bradygaster
    added a commit that references this issue on Mar 20, 2026

    Add circuit breaker for model rate limiting in Ralph

    336a0a0
  8. bradygaster
    closed this as completedin #451on Mar 20, 2026
  9. bradygaster
    added a commit that references this issue on Mar 20, 2026

    Add circuit breaker for model rate limiting in Ralph (#451)

    04a44fa
  10. chrislomonico
    added a commit that references this issue on Mar 26, 2026

    fix: unify status vocabulary and context-aware thinking labels (githu…

    c5b1bdb
  11. bradygaster
    removed
    squad:eecom
    squad:fido
    on Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    go:needs-researchNeeds investigationpriority:p2Next sprintsquadSquad triage inbox — Lead will assign to a memberstatus:backlogBacklog itemtype:featureNew capability

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions