Skip to content

Pause agents and warn in UI when Claude usage limit is reached - #301

Open
zwanski-start wants to merge 2 commits into
AutoForgeAI:masterfrom
zwanski-start:swarm/issue-19-ada
Open

zwanski-start wants to merge 2 commits into
AutoForgeAI:masterfrom
zwanski-start:swarm/issue-19-ada

Conversation

@zwanski-start

Copy link
Copy Markdown

Closes #19

Problem

When a Claude subscription limit is hit, Claude Code returns text like
Limit reached · resets 6pm (America/New_York) · /upgrade to Max or turn on /extra-usage.
None of the existing rate-limit patterns matched that, so the agent treated it as a normal session and printed "auto-continue in 3s". On top of that, agents spawned by the orchestrator (the UI path) run a single session and exit, so the orchestrator kept respawning new sessions that failed right away.

Changes

Detection (rate_limit_utils.py)

  • New patterns for usage-limit messages: limit reached · resets …, You've hit your limit …, Claude (AI) usage limit reached. A bare "limit reached" still doesn't match (existing false-positive tests like "File size limit reached" still pass).
  • parse_limit_reset() turns resets 6pm (America/New_York) / reset at 6:30 PM (TZ) / the legacy usage limit reached|<epoch> into seconds until reset. It uses local time when there's no timezone, rolls to tomorrow if the time has already passed, and caps the wait at 7 days. This replaces the inline regex in agent.py.
  • format_rate_limit_marker() / parse_rate_limit_marker() produce and read a small [RATE_LIMIT] retry_in=<s|unknown> line that agents send to the orchestrator.

Agent (agent.py)

  • Prints a prominent warning banner that includes Claude's own message and the resume time.
  • Single-session agents (feature, batch and testing) print the marker and exit instead of retrying. Standalone runs sleep until the reset time, as before.

Orchestrator (parallel_orchestrator.py)

  • When it sees the marker, it pauses spawning of all agents, coding and testing, until the reset. If several agents hit the limit, the pause can get longer but never shorter. When the reset time is unknown it uses exponential backoff. Once the pause ends it resumes on its own and prints Rate limit pause over - resuming agents.

UI warning

  • OrchestratorTracker (server/websocket.py) turns the orchestrator's pause and resume lines into an orchestrator_update with state rate_limited and a rateLimitedUntil field. The state stays set while other events (such as agents exiting) come in.
  • Mission Control shows a red "Usage limit - paused" badge. The Maestro card shows a warning banner with the resume time.

Verification

  • ruff check . is clean. mypy reports no new errors in the changed files; the existing errors are in other files.
  • pytest test_rate_limit_utils.py test_client.py test_dependency_resolver.py test_ws_security.py: 77 passed, including new tests for the usage-limit patterns, reset-time parsing (timezones, 12am/pm, rollover, epoch) and the marker round-trip. python test_security.py passes.
  • npm run lint shows 0 errors (only the existing warnings). npm run build passes.
  • Ran a scripted end-to-end check:
    • A mocked single-feature agent that receives the exact message from the issue prints the warning and the marker, then exits without sleeping.
    • ParallelOrchestrator._read_output on a real subprocess that prints the marker sets a pause of about 1234s, and a later unknown marker does not shorten it.
    • OrchestratorTracker emits rate_limited and keeps it through Feature #3 completed, then clears it on resume.
  • UI: ran the real backend plus the Vite dev server and used Playwright to inject a rate_limit_pause update on the project WebSocket. The Mission Control badge and the warning banner both render.

Assumptions / notes

  • An agent that stops because of a limit exits with code 0, so the stop doesn't count toward MAX_FEATURE_RETRIES. The feature's in_progress flag is cleared as usual and the feature is picked up again after the pause.
  • Agents that are already running when the pause starts are not killed. They hit the same limit and exit on their own.
  • The initializer still handles limits in-process by sleeping until the reset, because it isn't a single-session agent.

🤖 Generated with Claude Code

zwanski-start and others added 2 commits October 22, 2026 09:10
The 'Limit reached · resets 6pm (America/New_York)' message from Claude Code
did not match any rate-limit pattern, so agents treated it as a normal
session and immediately started new ones.

- rate_limit_utils: match usage-limit messages, add parse_limit_reset()
  (time-of-day + timezone, legacy epoch format) and a [RATE_LIMIT] marker
  used to report limits to the orchestrator
- agent: print a prominent warning; single-session (orchestrated) agents
  emit the marker and exit instead of being respawned; standalone runs
  sleep until the reset time

Refs AutoForgeAI#19

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Orchestrator parses the agents' [RATE_LIMIT] marker and stops spawning
  coding and testing agents until the limit resets (exponential backoff
  when the reset time is unknown), then resumes automatically
- OrchestratorTracker emits a sticky 'rate_limited' state with the
  resume time; Mission Control shows a warning banner and badge

Closes AutoForgeAI#19

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zwanski-start

Copy link
Copy Markdown
Author

🔍 QA report: ✅ Passed

Tester: Barbara (cubefarm agent, in a fresh session) · Round: 1 · Author: Ada

The PR fixes issue #19. The exact 'Limit reached · resets 6pm (America/New_York)' message is now detected, feature and testing agents print a warning plus a [RATE_LIMIT] marker and exit instead of auto-continuing, and the orchestrator stops spawning coding and testing agents until the parsed reset time, then resumes on its own. The UI shows a red 'Usage limit - paused' badge and a warning banner with the resume time, which clear on resume. Lint and build pass; pytest failures and mypy errors match master. Minor non-blocking notes: Stop during a pause can wait up to 60s before reacting to SIGTERM, exception-path limits don't parse the reset time, and the banner is cramped on phone widths because of the existing Maestro card layout.

Check Details
✅ Issue repro: exact 'Limit reached · resets 6pm (America/New_York)' message is detected and parsed is_rate_limit_error() returns True; parse_limit_reset() returns 49602s until 2026-10-22 18:00 America/New_York. 'File size limit reached' and 'Max retries limit reached - resetting' are still not matched. Note: free text like 'The rate limit reached its reset point' does match, which could cause a short spurious pause; low risk.
✅ Single-session feature/testing agents stop instead of auto-continuing Ran run_autonomous_agent with mocked create_client/run_agent_session returning the issue message. feature_id=3 coding and testing agents print the warning banner (including Claude's message and the resume time) and '[RATE_LIMIT] retry_in=49602', then exit without asyncio.sleep. A status='rate_limit' exception path sends 'retry_in=unknown'.
✅ Standalone agent still waits until the reset Same mock with no feature_id: prints the banner and 'Agent will auto-continue in 49592s', then sleeps 49592s (the old behavior, now capped at 7 days instead of 24h).
✅ Orchestrator pauses all spawning and resumes on its own Ran the real ParallelOrchestrator.run_loop on a temporary features.db, with spawn methods replaced by real subprocesses that print the marker and the real _read_output. 'Pausing all agents for 3s' was printed and nothing was spawned during the pause; then 'Rate limit pause over - resuming agents' and the next features started. Later unknown markers used backoff (18s) and 'Extending' to 31s, never shortening. Exit code 0, so MAX_FEATURE_RETRIES is not affected.
✅ Shutdown during a rate-limit pause Minor: with a pause active, the loop waits up to min(remaining, 60s) in _wait_for_agent_completion, so a SIGTERM shutdown flag is not seen within 5s in my probe. The UI Stop uses kill_process_tree with a 10s timeout, so it still stops, but by force rather than gracefully. Not blocking.
✅ WebSocket OrchestratorTracker emits rate_limited state The pause line becomes rate_limit_pause with state=rate_limited and rateLimitedUntil='2026-10-22 18:00 EDT'. 'Feature #3 completed' keeps state=rate_limited, Extending updates the time, and 'Rate limit pause over' gives state=scheduling with rateLimitedUntil=None.
✅ UI warning renders and clears (real browser) Ran the backend on 127.0.0.1:5443 with an isolated HOME and a temp project. Used Playwright routeWebSocket to inject orchestrator_update messages in the tracker's format. The Mission Control badge 'Usage limit - paused' and the role=alert banner with the time render; the alert stays after an agent-exit update and disappears after the resume update (alert count 1 -> 1 -> 0). No console errors or warnings.
✅ Responsive layout at 390px No horizontal page scroll (scrollWidth 390). The Maestro card's content column is very narrow at phone width; this layout predates the PR (the status message is also squashed), but the new banner's text spills past its border there. Cosmetic.
✅ Kanban and dependency graph still render Kanban shows the features; the graph view renders 4 react-flow nodes; no console errors.
✅ Security / allowlist / localhost middleware untouched The diff doesn't touch security.py or the middleware; test_security.py prints ALL TESTS PASSED. No secrets are logged; the marker only carries the retry seconds.
⏭️ UI Playwright e2e suite The Playwright chromium-headless-shell browser isn't installed on this host and 'npx playwright install' timed out after 300s, so all 15 tests failed at browser launch, before running. Environmental, not caused by the PR. The UI change was checked directly in a real browser instead.
🧪 Commands run
Command Result
ruff check . All checks passed
python -m pytest -q 156 passed, 21 failed (test_devserver_security.py); the same 21 failures on origin/master
python -m pytest -q test_rate_limit_utils.py test_client.py test_dependency_resolver.py 67 passed
python test_security.py ALL TESTS PASSED
mypy . Same 24 errors as origin/master; 1 extra from the local venv/activate_this.py, not from the PR
npm --prefix ui run lint 0 errors, 6 warnings
npm --prefix ui run build tsc -b && vite build succeeded
python probe_agent.py (mocked agent loop with the issue message) Feature and testing agents print the marker and exit without sleeping; standalone sleeps until the reset
python probe_orch.py (real run_loop with subprocesses printing the marker) Pauses, extends, and resumes correctly; no spawns during the pause
OrchestratorTracker line probe rate_limited is kept across agent exits and cleared on resume
HOME=<scratch> uvicorn server.main:app --host 127.0.0.1 --port 5443 Health OK; project registered; UI checked in Playwright; server stopped afterwards
CI=1 npm --prefix ui run test:e2e 15 failed at browser launch (Chromium headless shell missing; install timed out); environmental

No browser screenshots were taken in this round.

Posted by cubefarm · merges automatically once GitHub's checks pass

cabana8471-arch pushed a commit to cabana8471-arch/autoforge that referenced this pull request Nov 8, 2026
…tive (AutoForgeAI#19)

Adaptations: the [RATE_LIMIT] marker counts only as a whole line (anchored regex, matched on the stripped line), so model text quoting it no longer pauses every agent; _note_rate_limit clamps retry_in to MAX_LIMIT_RESET_SECONDS (an oversized value raised OverflowError in the output reader thread). Kept my-features' _print_system_message/_print_result_message next to _print_limit_warning in agent.py. New tests: marker mid-line ignored, oversized retry_in clamped, tracker rate_limited state, feature agent emits the marker and stops.

Integrates AutoForgeAI#301

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AeH26pcsqoxPb3CDs8g1VN
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

When Claude limit is reached, don't keep re-querying

1 participant