Skip to content

Surface Anthropic credit-balance health as a diagnosable state, not just a run-time failure #773

Closed

Description

@proagentstore

PROBLEM — A coding_loop_start run against this instance failed immediately with: "UserAiProviderError: Anthropic (400): Your credit balance is too low to access the Anthropic API." There was no way to see this coming: billing_status returned {active:false, status:"none", hasBillingAccount:false} — correctly showing no PAGS-side billing issue, because this is an Anthropic-account-level credit balance, not a PAGS billing state. Nothing in coding_diagnostics or coding_loop_status surfaced it either; the only signal was the run dying.

EXPLICIT NON-GOAL — Do NOT add a balance check before every dispatch or every action. This is a rare failure mode and a pre-flight check on every call would be wasted latency/cost for something that almost never happens. The ask is purely for VISIBILITY on demand and clearer failure reporting, not a gate.

PROPOSAL — 1) Add a way to check current provider credit health on demand (e.g. via coding_diagnostics or a small dedicated read), so a caller CAN check when they suspect something's wrong, without it being mandatory. 2) When a run does fail for this specific reason, make the stop reason unambiguous and distinguishable from other provider errors, so the caller (human or orchestrator) immediately understands "top up credits" is the fix, rather than treating it like a generic transient failure worth retrying.

CONTEXT — This surfaced mid-session while triaging a red production deploy on ProAgentStore/platform main; the fix-forward run could not even start because of this, and only became diagnosable by manually re-running coding_loop_status and reading the raw error text.

Activity

  1. added
    enhancementNew feature or request
    mcpMCP server surface — the operator's instrument panel, not the product
    coderThe Coder wedge agent (#68) — Engine, Pilot, Co-pilot, Loop, Overseer
    observabilityA displayed value the code cannot produce, or that means something other than its label
    on Sep 5, 2026
  2. serge-ivo commented on Sep 8, 2026

    @serge-ivo
    Contributor

    Delivered to main and deployed.

    What was implemented

    Part 1 — on-demand credit visibility (no per-dispatch gate)

    • GET /v1/instances/:id/coding/diagnostics (MCP system_status / coding_diagnostics) now carries a providerAccount block: the newest account-class run death from the error log compared against the Anthropic key's last_used_at. state is failing (failure newer than last success), recovered (key has succeeded since), or no_failure_recorded (nothing on record — explicitly not an all-clear). While failing, a warn issue names the top-up page and the verify endpoint. Rows filed before this change as provider_credentials are re-read from their stored sentence, so historical credit failures classify correctly. New helper: workers/api/src/lib/provider-account-health.ts.
    • POST /v1/keys/anthropic/verify is now accepted (previously Cloudflare only). It spends one four-token request against the stored key and returns the failure classified (failureClass, remedy) in the same vocabulary a dead run files. 404 when no Anthropic key is stored, rather than silently probing Workers AI. This is the explicit live check — called when a caller suspects a problem, never ahead of every run, per the issue's non-goal.

    Part 2 — unambiguous stop reason

    • New failure class provider_credit, split out of provider_credentials in coding-failure.ts. Matches credit balance / insufficient_funds / insufficient credit / insufficient_quota. Non-retryable, non-resumable (DRIVER_RESUME_POLICY entry with the top-up remedy), logs at warn. An invalid key stays provider_credentials.
    • New LoopStopReason member provider_credit, mapped by statusFor to needs_human. The loop-run row, board card, chat bubble (**Loop stopped** (provider_credit)) and coding_loop_status all report it instead of failed.
    • codingCrashReport gained an owner-action table beside the interruption table. The detail leads with "Stopped because the Anthropic account has no credit left, not by the objective — top up at console.anthropic.com/settings/billing, then start the run again; retrying before that will fail the same way", with the vendor's sentence last where the 300-char card truncates.
    • runAnthropic appends "Insufficient Anthropic credit balance. Top up at console.anthropic.com/settings/billing" to the raw 400 when the message is a credit failure, so the chat path reads the remedy too. The vendor text is kept verbatim because it is what the classifier matches on.
    • MCP descriptions for system_status and coding_loop_status name the new reason and say not to retry.

    Tests: classifier split, crash-report path (5 cases), health helper (9 cases), diagnostics route (3 cases), user-ai hint + classifier agreement, and the stop-reason / failure-class denominator guards updated.

    Commits

    • 19c7eca — feat: surface Anthropic credit-balance as diagnosable provider_credit state (#773)
    • f6c741b — docs: record delivery mode (straight to main) in CLAUDE.md
    • 88c596c — test(user-ai): type the caught rejection so the #773 billing-hint test compiles under tsconfig.test.json

    CI / deploy status at HEAD 88c596c

    Workflow Result
    CI ✅ success
    Deploy API Worker ✅ success
    Deploy MCP Worker ✅ success (on 19c7eca; MCP unchanged since)

    Note: CI on 19c7eca and f6c741b went red on the test-typecheck step only (Typecheck worker + SDK TESTS, #599) — a .catch((e) => e) in the new test left err typed unknown under tsconfig.test.json. The API deploy on 19c7eca had already passed its own test run and succeeded; 88c596c cleared CI with no runtime change.

  3. serge-ivo commented on Sep 8, 2026

    @serge-ivo
    Contributor

    Closed: delivered on main at 88c596c, CI and deploys green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    coderThe Coder wedge agent (#68) — Engine, Pilot, Co-pilot, Loop, OverseerenhancementNew feature or requestmcpMCP server surface — the operator's instrument panel, not the productobservabilityA displayed value the code cannot produce, or that means something other than its label

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions