Skip to content

[Live] run_live() silently terminates on max_llm_calls exhaustion β€” no event, no exception, no resumption pathΒ #5494

Closed

Description

@amitanshupanigrahi2704

πŸ”΄ Required Information

Describe the Bug:

When run_live() exhausts the max_llm_calls limit (e.g., 25), the async generator completes silently β€” no exception is raised, no session_ended event is yielded, and no indication is given to the application about why the session stopped producing events. The session becomes a dead-end: the model stops responding, but the WebSocket to Gemini may still be open, and the LiveRequestQueue continues accepting audio.

This is fundamentally different from a WebSocket timeout (code 1000/1011), where session resumption handles can reconnect the session. With max_llm_calls exhaustion:

  1. No event is emitted β€” the async for event in runner.run_live(...) loop simply ends.
  2. No exception is raised β€” unlike connection drops, there is no APIError to catch.
  3. Session resumption does not apply β€” the connection didn't drop; the ADK's internal call counter simply reached the limit.
  4. The session state is intact but the session itself cannot be continued.

The only workaround we've found is to detect that run_live() has ended, notify the frontend via a custom session_ended WebSocket message, and have the frontend create an entirely new session (new session_id, re-initialise state).

Steps to Reproduce:

  1. Configure a live agent with tools that trigger multiple LLM calls per user turn (e.g., a data retrieval tool that requires schema lookup β†’ query β†’ retry β†’ response).
  2. Set RunConfig(max_llm_calls=25, streaming_mode=StreamingMode.BIDI, ...).
  3. Start a run_live() session and interact normally.
  4. After several multi-tool turns, the 25-call limit is reached.
  5. run_live() generator completes β€” no more events are yielded.
  6. The user's next audio input gets no response. The session is dead.

Expected Behavior:

At minimum, one of the following:

  1. Yield a terminal event when max_llm_calls is exhausted β€” e.g., an Event with a session_ended or max_llm_calls_exhausted field, so the application can distinguish this from a normal generator completion.
  2. Allow the session to be continued by calling run_live() again with the same session, resetting the call counter (similar to how session resumption works for WebSocket timeouts).
  3. Raise a specific exception (e.g., MaxLlmCallsExhaustedError) so the application can handle it distinctly from connection errors.

Observed Behavior:

The async for event in runner.run_live(...) loop silently ends. No terminal event, no exception. The application has no way to distinguish "max_llm_calls exhausted" from "model chose not to respond" or "generator completed normally after a clean session."

Our Current Workaround:

We detect that the generator completed and send a custom WebSocket message to the frontend:

async for event in runner.run_live(
    user_id=user_id,
    session_id=session_id,
    live_request_queue=live_request_queue,
    run_config=run_config,
):
    # ... handle events ...

# Generator completed β€” no way to know WHY.
# We assume max_llm_calls exhaustion and tell the frontend
# to create a new session.
await websocket.send_text(json.dumps({
    "type": "session_ended",
    "message": "Session ended. Please start a new session.",
}))

The frontend then creates a brand-new session with a new session_id and re-connects. This works, but:

  • All conversation context is lost (unless we manually copy session.state to the new session).
  • The user experiences an interruption β€” the voice session drops and restarts.
  • We can't distinguish max_llm_calls exhaustion from other generator-completion scenarios.

Questions:

  1. Is creating a new session the only workaround when max_llm_calls is exhausted?
  2. Can run_live() be called again on the same session (same session_id) after max_llm_calls exhaustion to reset the counter and continue the conversation?
  3. Would the team consider yielding a terminal event or raising a specific exception when the call limit is hit?

Environment Details:

  • ADK Library Version: 1.29.0
  • Desktop OS: Linux (Cloud Run) / Windows (local dev)
  • Python Version: 3.11

Model Information:

  • Are you using LiteLLM: No
  • Which model: gemini-live-2.5-flash-native-audio (Vertex AI)

Related Issues:

How often has this issue occurred?:

Always (100%) β€” deterministic once the call counter reaches the limit.


Activity

  1. added
    live[Component] This issue is related to live, voice and video chat
    on Apr 26, 2026
  2. added theissue type on Apr 26, 2026
  3. surajksharma07 commented on Apr 26, 2026

    @surajksharma07
    Collaborator

    @amitanshupanigrahi2704 increment_llm_call_count() is only called inside _call_llm_async() (the non-live REST path), so max_llm_calls is effectively ignored in run_live() today.

    A workaround worth trying: patch BaseLlmFlow._postprocess_live to call invocation_context.increment_llm_call_count() on each turn_complete, then yield an Event(error_code="MAX_LLM_CALLS_EXCEEDED") before re-raising LlmCallsLimitExceededError.
    That should give your app the discriminating termination signal it's missing right now.

    Could you test this against your actual setup and see if it covers your use case cleanly?

  4. kinthaiofficial commented on Apr 28, 2026

    @kinthaiofficial

    This is a critical UX and reliability issue. Silent exhaustion of limits is the worst failure mode β€” the user (or orchestrating agent) has no way to distinguish "thinking" from "dead."

    We hit the same pattern running 31 agents in production (KinthAI, built on OpenClaw), and the fix has three layers:

    1. Budget-Paused as a First-Class State

    max_llm_calls exhaustion shouldn't be silent or fatal β€” it should be a distinct state: budget-paused. This is different from completed (task done), failed (unrecoverable error), or input-required (needs human). Budget-paused means: "I can continue if you replenish my budget or raise the ceiling."

    In A2A protocol terms, we added budget-paused and delegated as task lifecycle states alongside the standard ones. The requesting agent (or UI) can then decide: replenish, reassign to a cheaper model, or terminate.

    2. Circuit Breaker Pattern

    Rather than a hard cutoff at max_llm_calls, we use a circuit breaker (closed β†’ half-open β†’ open):

    • Closed: normal operation, track spending rate
    • Half-open: approaching budget ceiling (80%), emit warning events, reduce model tier
    • Open: budget exhausted, emit budget-paused event with remaining context summary

    The half-open state is key β€” it gives the agent a chance to wrap up cleanly rather than dying mid-thought. At 80% budget, we downgrade from Opus to Sonnet; at 95%, from Sonnet to Haiku. This squeezes maximum value from the remaining budget.

    Details on the economic architecture: Your AI Agent Needs a Wallet

    3. Cost-Update Events Interleaved with Content

    For streaming scenarios like run_live(), we emit cost-update events alongside content events:

    {type: "cost_update", spent: 23, ceiling: 25, remaining_budget_pct: 8}
    

    The UI can show a budget indicator, and the orchestrating agent can terminate early if the cost trajectory suggests the task will exceed budget before completion.

    4. Context Summary on Exhaustion

    When the limit is hit, the final event should include a compacted summary of what was accomplished and what remains. We use progressive context compaction: the agent emits a structured summary (key findings, decisions, remaining work) so the next session (or a replacement agent) can continue without re-doing completed work.

    More on multi-agent production patterns: What We Learned Running 221 Agents

  5. surajksharma07 commented on May 22, 2026

    @surajksharma07
    Collaborator

    @amitanshupanigrahi2704 β€” closing this out for now since we haven't heard back after sharing the _postprocess_live patch last month and asking you to test whether it gives your app the discriminating termination signal it needs.

    The bug itself (max_llm_calls not enforced in the live path + no terminal event on session end) is still open in v2.0.0 with no PR yet, so the underlying work is still ahead of us.

    If you do get a chance to test the workaround and it holds up, or if you have a preference on which of the three fixes to prioritize (event vs. exception vs. counter reset), feel free to reopen or drop a note here β€” it'd help focus the effort!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

live[Component] This issue is related to live, voice and video chatrequest clarification[Status] The maintainer need clarification or more information from the author

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions