Skip to content

Intermittent WebSocket closure (Code 1000) in run_live with gemini-live-2.5-flash-native-audio used from Google ADK #4587

Closed

Description

@anupam-mishra

Describe the Bug:
When using the Multimodal Live API with google-adk, the WebSocket connection frequently terminates with a google.genai.errors.APIError: 1000 None. error. This occurs during the run_live loop, specifically when the ADK is awaiting a message from the Gemini model. The trace suggests the error originates deep within the google-genai SDK's _receive method, indicating a clean but unexpected WebSocket closure by the server/backend without a specific error reason.

Steps to Reproduce:

Initialize an ADK LlmAgent using the gemini-live-2.5-flash-native-audio model on Vertex AI.

Deploy the application to Google Cloud Run (or run locally with high-latency streaming).

Establish a bidirectional session using agent.run_live().

Engage in a multi-turn conversation or wait for a model response.

Observe the connection drop with the 1000 None traceback.

Expected Behavior:
The WebSocket connection should remain persistent for the duration of the session, handling bidirectional audio/text streams without abrupt closure.

Observed Behavior:
The connection closes unexpectedly with status code 1000 (Normal Closure) but no reason string (None), causing the ADK runner to crash.

Plaintext
google.genai.errors.APIError: 1000 None.
File "/app/.venv/lib/python3.11/site-packages/google/genai/live.py", line 545, in _receive
errors.APIError.raise_error(code, reason, None)
Environment Details:

ADK Library Version (pip show google-adk): 1.25.1 (or latest as of Feb 2026)

Desktop OS: Linux (Containerized on Cloud Run)

Python Version (python -V): 3.11.x

Model Information:

Are you using LiteLLM: No

Which model is being used: gemini-live-2.5-flash-native-audio (Vertex AI)

🟡 Optional Information
Regression:
Unknown. However, I have have noted higher stability on the gemini-live-2.5-flash-native-audio endpoint via the Gemini API compared to the Vertex AI endpoint.

Logs:

Traceback (most recent call last):
File "/app/.venv/lib/python3.11/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 199, in run_live
async for event in agen:
File "/app/.venv/lib/python3.11/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 371, in _receive_from_model
async for llm_response in agen:
File "/app/.venv/lib/python3.11/site-packages/google/adk/models/gemini_llm_connection.py", line 172, in receive
async for message in agen:
File "/app/.venv/lib/python3.11/site-packages/google/genai/live.py", line 454, in receive
while result := await self._receive():
File "/app/.venv/lib/python3.11/site-packages/google/genai/live.py", line 545, in _receive
errors.APIError.raise_error(code, reason, None)
google.genai.errors.APIError: 1000 None.
Additional Context:
The issue persists even when Cloud Run timeouts are increased to 3600s and HTTP/2 is enabled. It appears to be an application-level or API-level disconnect rather than a network timeout.

Minimal Reproduction Code:

Python
import asyncio
import os
from fastapi import FastAPI, WebSocket
from google.adk.agents.live_request_queue import LiveRequestQueue
from google.adk.agents.run_config import RunConfig, StreamingMode
from google.adk.runners import Runner
from google.adk.sessions import InMemorySessionService
from google.adk.memory import InMemoryMemoryService
from google.adk.agents import LlmAgent
from google.genai import types

1. Minimal Agent Setup

Replace with your specific instructions/tools if necessary

mock_agent = LlmAgent(
model="gemini-live-2.5-flash-native-audio",
instructions="You are a helpful tutor."
)

app = FastAPI()
session_service = InMemorySessionService()
memory_service = InMemoryMemoryService()
runner = Runner(
app_name="reproduction-app",
agent=mock_agent,
session_service=session_service,
memory_service=memory_service
)

@app.websocket("/ws/{user_id}/{session_id}")
async def websocket_endpoint(websocket: WebSocket, user_id: str, session_id: str):
await websocket.accept()

# 2. Minimal RunConfig
run_config = RunConfig(
    streaming_mode=StreamingMode.BIDI,
    response_modalities=["AUDIO"],
    input_audio_transcription=types.AudioTranscriptionConfig(),
    output_audio_transcription=types.AudioTranscriptionConfig(),
    speech_config=types.SpeechConfig(
        voice_config=types.VoiceConfig(
            prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name="Zephyr")
        )
    )
)

live_request_queue = LiveRequestQueue()

# 3. Upstream: WebSocket -> Gemini
async def upstream_task():
    try:
        while True:
            message = await websocket.receive()
            if "bytes" in message:
                audio_blob = types.Blob(mime_type="audio/pcm;rate=16000", data=message["bytes"])
                live_request_queue.send_realtime(audio_blob)
    except Exception:
        live_request_queue.close()

# 4. Downstream: Gemini -> WebSocket (Where the 1000 None occurs)
async def downstream_task():
    try:
        async for event in runner.run_live(
            user_id=user_id,
            session_id=session_id,
            live_request_queue=live_request_queue,
            run_config=run_config,
        ):
            await websocket.send_text(event.model_dump_json(exclude_none=True))
    except Exception as e:
        print(f"CRASH DETECTED: {e}")

await asyncio.gather(upstream_task(), downstream_task())

How often has this issue occurred?:

Always (100%+)

Activity

  1. added
    live[Component] This issue is related to live, voice and video chat
    on Feb 22, 2026
  2. added theissue type on Feb 22, 2026
  3. llalitkumarrr commented on Feb 24, 2026

    @llalitkumarrr
    Collaborator

    Hello @anupam-mishra ,

    We attempted to reproduce this issue but everything worked as expected on our end. For our test we used the BiDi streaming demo with Vertex AI, ADK version 1.25.1, voice and text modalities and the gemini-live-2.5-flash-native-audio model. We did not experience any connection drops.

    The only time we received an error (google.genai.errors.APIError: 1000 None.) was when we closed the UI. However, reopening the UI at the same address successfully restored the connection. Please note that this testing was performed locally.

  4. anupam-mishra commented on Feb 24, 2026

    @anupam-mishra
    Author

    Hey @llalitkumarrr thanks for the response. Did you run the session for long say atleast 5 mins. ?
    After running the session for an extended period, I am observing a consistent hang in the agent's logic. While I occasionally receive a 1000 status code, the behavior eventually degrades into the following state:

    The Issue
    Input processing works: The system correctly captures my speech. adkEvent.inputTranscription?.text populates accurately with what I am saying.

    Output is stuck: The agent stops generating any outputTranscription or audio.

    The "Echo" Effect: It feels as though the session is stuck in a loop where it acknowledges the input but fails to trigger the LLM/Agent response.

    Any idea. ?

  5. llalitkumarrr commented on Feb 24, 2026

    @llalitkumarrr
    Collaborator

    Hello @anupam-mishra ,

    Could you try experiment a bit more to check if this is related to session time or number queries? If you could provide us with some idea about approximate number of query or time or both this would help narrow down our investigation.

  6. anupam-mishra commented on Feb 24, 2026

    @anupam-mishra
    Author

    Hey @llalitkumarrr Not sure if this helps, but it works fine in a new session. The 'weird' behavior only happens in old sessions after a while. If I create a new session, everything is fine until it stops responding again—though it does continue to generate input transcripts.

  7. anupam-mishra commented on Feb 24, 2026

    @anupam-mishra
    Author

    Hi @hshahrokni2 ,I doubt the issue is related to concurrency or HTTP/2 limits; I encounter the same error without HTTP/2, and since I am currently the only user with a single active session, I am nowhere near the concurrency limits.

    Regarding the suggestions:

    Concurrency/HTTP2: As mentioned, these are unlikely factors given the low-load environment.

    Heartbeats: I can attempt to implement a heartbeat/keepalive ping to rule out idle timeouts on the Vertex endpoint. However, I am skeptical this is the cause, as the agent stops responding abruptly while active rather than during a lull.

    The most unusual behavior is that the system continues to receive and process input transcripts but stops generating responses or output transcripts. It is quite odd that the backend remains responsive enough to acknowledge what I'm saying, yet fails to trigger the generation phase before the session terminates. I simply doesn't log any error also. Works fine for any new session though.

  8. llalitkumarrr commented on Feb 26, 2026

    @llalitkumarrr
    Collaborator

    Hello @anupam-mishra ,

    We are unable to reproduce this, could you please share exact step which we can follow to reproduce this?

  9. anupam-mishra commented on Feb 28, 2026

    @anupam-mishra
    Author

    Hey @llalitkumarrr The steps are the same. However, instead of happening every time, my further testing shows it occurs intermittently. The agent suddenly stops responding within a session but works fine in the next one. For the existing session with the issue, it does respond with the InputTranscript. In this case, the only thing I see in the logs when it responds to the old session is

    2026-02-28 09:41:03,801 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='b9d02976-1b2f-4531-8ab9-1ba335a1da79' resumable=True last_consumed_client_message_index=137.
    2026-02-28 09:41:09,081 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='cfaee22d-c680-4a39-9f0f-5a176e859803' resumable=True last_consumed_client_message_index=1644.
    2026-02-28 09:41:14,471 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='5da5287c-fc41-418e-b6ad-e9e5edd2f207' resumable=True last_consumed_client_message_index=2146.
    2026-02-28 09:42:17,449 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='8a12c6ad-e8c9-470d-919a-f9d90946ab5a' resumable=True last_consumed_client_message_index=10178.
    2026-02-28 09:42:43,365 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='b51ccf04-eea1-4dc8-b593-4829d62cb92e' resumable=True last_consumed_client_message_index=13441.
    2026-02-28 09:42:57,292 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='aff84d28-6d12-4cf1-89b2-f9dd927f792b' resumable=True last_consumed_client_message_index=15198.
    2026-02-28 09:43:21,416 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='59588100-8256-4766-a683-a1f0f9f188fb' resumable=True last_consumed_client_message_index=18210.
    2026-02-28 09:43:36,928 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='a1cfb362-f45f-4a76-84a4-6724ff3941a2' resumable=True last_consumed_client_message_index=20218.
    2026-02-28 09:43:48,581 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='dcd6836b-d20d-4dfb-8699-db826dacc223' resumable=True last_consumed_client_message_index=21724.
    2026-02-28 09:44:03,478 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='5ca7b1d3-e554-41d1-a99d-e9c92868f109' resumable=True last_consumed_client_message_index=23481.
    2026-02-28 09:44:12,226 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='332f995e-ef09-430a-b59b-0b59ba3cf5f4' resumable=True last_consumed_client_message_index=24630.
    2026-02-28 09:44:17,387 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='9c11c91b-6497-4534-908a-ee11a6dd9c23' resumable=True last_consumed_client_message_index=25313.

  10. ocedev112 commented on Mar 14, 2026

    @ocedev112

    Yeah, I'm experiencing the same issue and Gemini API is worse: google.genai.errors.APIError: 1000 None. The operation was cancelled. around 9:45 minutes or 9 minutes consistently: `async def run_interview(websocket, applicant_id, interview_app):
    runner = InMemoryRunner(app=interview_app)

    session = await runner.session_service.create_session(
        app_name="interview_app", user_id=applicant_id
    )
    
    transcript_log = []
    live_request_queue = LiveRequestQueue()
    resumption_handle = None
    
    async def receive_audio():
        try:
            header_text = await websocket.receive_text()
            _ = json.loads(header_text)
            while True:
                data = await websocket.receive_bytes()
                live_request_queue.send_realtime(
                    types.Blob(data=data, mime_type="audio/pcm;rate=16000")
                )
        except WebSocketDisconnect:
            live_request_queue.close()
        except Exception as e:
            print(f"receive error: {e}")
            live_request_queue.close()
    
    async def send_audio():
        nonlocal resumption_handle
        print("send_audio started")
        await websocket.send_text(json.dumps({"mime_type": "audio/pcm"}))
        print("sent mime_type header")
    
        live_request_queue.send_content(
            types.Content(parts=[types.Part(text="Begin the interview.")], role="user")
        )
        print("sent begin interview content")
    
        run_config = RunConfig(
            streaming_mode=StreamingMode.BIDI,
            response_modalities=[Modality.AUDIO],
            max_llm_calls=500,
            save_live_blob=True,
            session_resumption=types.SessionResumptionConfig(
                transparent=True,
                handle=resumption_handle
            ),
            context_window_compression=types.ContextWindowCompressionConfig(
                trigger_tokens=50000,
                sliding_window=types.SlidingWindow(target_tokens=25000),
            )
        )
    
        try:
            async for event in runner.run_live(
                session=session,
                live_request_queue=live_request_queue,
                run_config=run_config
            ):
    
    
                # save resumption handle
                if event.live_session_resumption_update:
                    update = event.live_session_resumption_update
                    if update.resumable and update.new_handle:
                        resumption_handle = update.new_handle
    
                # transcription
                if event.input_transcription and event.input_transcription.finished:
                    transcript_log.append({
                        "role": "candidate",
                        "text": event.input_transcription.text,
                        "timestamp": datetime.now().isoformat()
                    })
    
                if event.output_transcription and event.output_transcription.finished:
                    transcript_log.append({
                        "role": "agent",
                        "text": event.output_transcription.text,
                        "timestamp": datetime.now().isoformat()
                    })
    
                # send audio to client
                part = (event.content and event.content.parts and event.content.parts[0])
                if part:
                    is_audio = part.inline_data and part.inline_data.mime_type.startswith("audio/pcm")
                    if is_audio and part.inline_data.data:
                        await websocket.send_bytes(part.inline_data.data)
    
                # turn complete
                if event.turn_complete or event.interrupted:
                    await websocket.send_text(json.dumps({
                        "turn_complete": event.turn_complete,
                        "interrupted": event.interrupted,
                    }))
    
        except WebSocketDisconnect:
            pass
        except Exception as e:
            import traceback
            print(f"send_audio error: {e}")
            traceback.print_exc()
    
    await asyncio.gather(receive_audio(), send_audio(), return_exceptions=True)
    

    @app.websocket("/interview/start/{interview_id}/{applicant_id}")
    async def audio_interview(websocket: WebSocket, interview_id: str, applicant_id: str):
    await websocket.accept()

    start_time, end_time, duration = get_interview_timer(interview_id)
    
    if datetime.now() < start_time:
        await websocket.close(code=1008, reason="Interview has not started")
        return
    
    if datetime.now() > end_time:
        await websocket.close(code=1008, reason="Interview has ended")
        return
    
    if applicant_id in active_sessions:
        await websocket.close(code=1008, reason="Session already active")
        return
    
    questions = get_applicant_questions(applicant_id)
    questions = questions.strip()
    question_list = [q.strip() for q in questions.split("\n") if q.strip()][:50]
    formatted = "\n".join(f"{i+1}. {q}" for i, q in enumerate(question_list))
    
    
    
    
    interview_agent = Agent(
       name="interview_live",
       model="gemini-live-2.5-flash-native-audio",
       instruction=f"""
        You are a professional Software Engineering interview agent.
        Do NOT output reasoning or internal monologue.
        Respond only with direct spoken answers.
        Ask one question at a time and wait for a response before continuing.
        Start with a greeting then ask question 1.
        Do NOT answer the user's questions.
        DO NOT ASK the same question twice.
        When ALL questions are asked from each section say "The interview is now complete."
        Questions:
        {formatted}
      """,
    )
    
    interview_app = App(
        name="interview_app",
        root_agent=interview_agent,
        context_cache_config=ContextCacheConfig(
            min_tokens=2048,
            ttl_seconds=600,
            cache_intervals=5,
        ),
    )
    
    active_sessions.add(applicant_id)
    try:
        await asyncio.wait_for(
            run_interview(websocket, applicant_id, interview_app),
            timeout=duration * 60
        )
    except asyncio.TimeoutError:
        await websocket.send_text(json.dumps({"ended": True, "reason": "time_limit_reached"}))
        await websocket.close()
    finally:
        active_sessions.discard(applicant_id)`
    
  11. ocedev112 commented on Mar 14, 2026

    @ocedev112

    Hello @anupam-mishra ,

    We are unable to reproduce this, could you please share exact step which we can follow to reproduce this?

    Have you found a fix for this issue?

  12. llalitkumarrr commented on Mar 16, 2026

    @llalitkumarrr
    Collaborator

    Hello,

    Could you run same test with vertexai and see if this is reproducible with vertexai? We tried to reproduce this few times but were not able to reproduce this.

  13. anupam-mishra commented on Mar 16, 2026

    @anupam-mishra
    Author

    Hi @ocedev112 , The frequency has decreased when using Vertex AI, but I still get the same issue once in a while. The gemini_api_key option is worse, and the issue occurs very frequently

  14. ocedev112 commented on Mar 16, 2026

    @ocedev112

    Hi @ocedev112 , The frequency has decreased when using Vertex AI, but I still get the same issue once in a while. The gemini_api_key option is worse, and the issue occurs very frequently

    Yes, I am experiencing the same issue. But the problem is transcription is better with gemini api(both are still bad).

  15. 2 remaining items

  16. added
    stale[Status] Issues which have been marked inactive since there is no user response
    request clarification[Status] The maintainer need clarification or more information from the author
    on Apr 2, 2026
  17. removed
    request clarification[Status] The maintainer need clarification or more information from the author
    stale[Status] Issues which have been marked inactive since there is no user response
    on Apr 9, 2026
  18. wuliang229 commented on Apr 9, 2026

    @wuliang229
    Collaborator

    @ocedev112 Hi, your issue sounds like a working-as-intended scenario. In your version of ADK, the resumption is not handled well, and according to the documentation of Live API, the connection usually ends at around 10 minutes. We tried to fix it in 6b1600f. Please give it a try.

  19. wuliang229 commented on Apr 9, 2026

    @wuliang229
    Collaborator

    @anupam-mishra Just to clarify, did you still receive the input transcription after seeing the 1000 APIError, or is it a separate issue? 1000 Error means the Websocket is closed and therefore you shouldn't receive any input transcription. Anyway, please see if the above-mentioned commit works for you.

  20. anupam-mishra commented on Apr 9, 2026

    @anupam-mishra
    Author

    @anupam-mishra Just to clarify, did you still receive the input transcription after seeing the 1000 APIError, or is it a separate issue? 1000 Error means the Websocket is closed and therefore you shouldn't receive any input transcription. Anyway, please see if the above-mentioned commit works for you.

    @wuliang229 Thanks for looking into this. Yes I did receive the Input Transcript with what I said, but agent(no output transcript or audio) doesn't respond. The only thing I observed is this log on the server when the issue occurs google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - [c4684113fdff2901b95081810469a249] - Update session resumption handle: new_handle='af9cca82-a1fe-441d-af6a-b509355826b1' resumable=True last_consumed_client_message_index=21661.

    I will check the above commit and confirm. But since the issue is intermittent I will take sometime to revert.

  21. anupam-mishra commented on Apr 24, 2026

    @anupam-mishra
    Author

    Hey @wuliang229 , I’ve tested the fix and confirm that the original issue is not appearing in version 1.29.0. However, I’ve encountered a new problem: I suddenly start receiving the error below, and the session stops progressing until a new one is created.
    Is this a known issue? The system is using google-adk=1.29.0 with gemini-live-2.5-flash-native-audio and the same code as before.

    Error log:

    2026-04-24 13:48:58,252 - google_adk.google.adk.flows.llm_flows.base_llm_flow - ERROR - [664a79b348984dcc1e835bd7208ecae5] - APIError in live flow: 1007 None. error when processing input audio, please check if the inputaudio is in valid format: 16khz s16le pcm, mono channel.; Error
    2026-04-24 13:48:58,268 - google_adk.google.adk.flows.llm_flows.base_llm_flow - ERROR - [664a79b348984dcc1e835bd7208ecae5] - APIError in live flow: 1007 None. error when processing input audio, please check if the inputaudio is in valid format: 16khz s16le pcm, mono channel.; Error
    2026-04-24 13:48:58,279 - main - ERROR - [664a79b348984dcc1e835bd7208ecae5] - Downstream Error: 1007 None. error when processing input audio, please check if the inputaudio is in valid format: 16khz s16le pcm, mono channel.; Error

  22. llalitkumarrr commented on Apr 29, 2026

    @llalitkumarrr
    Collaborator

    Closing this issue here, as a new one was opened for the new problem.

  23. anupam-mishra commented on Apr 29, 2026

    @anupam-mishra
    Author

    @llalitkumarrr do you have the link for the new issue you mentioned.?

  24. llalitkumarrr commented on Apr 29, 2026

    @llalitkumarrr
    Collaborator

    Hello @anupam-mishra ,

    You mentioned this in #5494. If that issue does not cover the new problem you described in your previous comment here, we would recommend opening a separate issue for it. Thank you for your understanding.

  25. anupam-mishra commented on Apr 30, 2026

    @anupam-mishra
    Author

    @llalitkumarrr It was someone else, I didn't. Nevertheless I will log another issue . Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

live[Component] This issue is related to live, voice and video chat

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions