Skip to content

Runner.run() silently terminates the event generator when run_async raises — exception reaches stderr but no signal to caller #5567

Closed

Description

@ivanmkc

Summary

Runner.run() (the sync wrapper in runners.py) returns a clean empty generator when its underlying run_async() coroutine raises. The exception does reach threading.excepthook and stderr (so a careful operator can see it), but the consumer-facing iterator yields zero events with no exception raised on the caller's thread — the consumer-side signal is lost.

This is the same class of bug as previously reported against other runner surfaces — see #4244 (run_sse, closed stale) and #5494 (run_live, open). Filing this for the sync run() surface specifically since the existing tickets cover different entrypoints.

Reproducer (≤15 lines, no GCP needed)

from unittest.mock import AsyncMock, MagicMock
from google.adk.runners import Runner
from google.adk.sessions.in_memory_session_service import InMemorySessionService
from google.genai import types
import asyncio

agent = MagicMock(name="agent", sub_agents=[])
svc = InMemorySessionService()
session = asyncio.run(svc.create_session(app_name="x", user_id="u"))
runner = Runner(agent=agent, app_name="x", session_service=svc)

# Make run_async raise — simulates an auth error / model construction failure
runner.run_async = lambda **kw: (_ for _ in ()).throw(RuntimeError("boom"))

events = list(runner.run(user_id="u", session_id=session.id,
                         new_message=types.Content(role="user",
                                                   parts=[types.Part(text="hi")])))
print(f"events={events}")  # → events=[]   (bug: no exception raised, just empty)

Mechanism

In runners.py (around lines 345-362):

event_queue = queue.Queue()

async def _invoke_run_async():
  try:
    async with Aclosing(self.run_async(...)) as agen:
      async for event in agen:
        event_queue.put(event)
  finally:
    event_queue.put(None)

def _asyncio_thread_main():
  try:
    asyncio.run(_invoke_run_async())
  finally:
    event_queue.put(None)

thread = create_thread(target=_asyncio_thread_main)
thread.start()

while True:
  event = event_queue.get()
  if event is None: break
  yield event

thread.join()  # ← does not propagate the in-thread exception

Path: any exception from run_async → propagates out of _invoke_run_async → reaches _asyncio_thread_main → thread dies → default threading.excepthook writes traceback to stderr. But:

  • The finally clause put None on the queue first, so the consumer's while True loop sees None and exits cleanly (yielding zero events)
  • thread.join() returns normally — Thread.join() does not surface the exception that killed the thread
  • The caller has no programmatic way to know anything went wrong; their generator just returned empty

This is especially bad for the documented "local testing and convenience" use case the docstring calls out, since silent failure during local dev is exactly the worst case for a debug-friendly API. We hit it in production with a benchmark harness running 12 concurrent cases — debugging took hours because the symptom (empty event stream) is indistinguishable from a real "agent quality" failure.

Suggested fix (minimal)

Capture the in-thread exception and re-raise after thread.join():

error_box: list[BaseException] = []

def _asyncio_thread_main():
  try:
    asyncio.run(_invoke_run_async())
  except BaseException as e:
    error_box.append(e)
  finally:
    event_queue.put(None)

# ... after the consumer loop:
thread.join()
if error_box:
  raise error_box[0]

Alternatively, move to concurrent.futures.ThreadPoolExecutor and call future.result(), which propagates exceptions naturally.

Severity

Medium-High. Documented as "for local testing and convenience" — silent failure in this surface is exactly the wrong default for the debug audience. Easy 5-line fix.

Workaround

We're shipping a local workaround that drives run_async() directly via asyncio.run (drops the entire thread+queue+sentinel machinery). Would happily delete it once an upstream fix lands.

Activity

  1. added
    core[Component] This issue is related to the core interface and implementation
    on May 1, 2026
  2. added theissue type on May 1, 2026
  3. ivanmkc commented on May 1, 2026

    @ivanmkc
    CollaboratorAuthor

    Filed in error from a downstream debugging session. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    core[Component] This issue is related to the core interface and implementation

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions