Repository navigation
Description
Describe the Bug:
When using the Multimodal Live API with google-adk, the WebSocket connection frequently terminates with a google.genai.errors.APIError: 1000 None. error. This occurs during the run_live loop, specifically when the ADK is awaiting a message from the Gemini model. The trace suggests the error originates deep within the google-genai SDK's _receive method, indicating a clean but unexpected WebSocket closure by the server/backend without a specific error reason.
Steps to Reproduce:
Initialize an ADK LlmAgent using the gemini-live-2.5-flash-native-audio model on Vertex AI.
Deploy the application to Google Cloud Run (or run locally with high-latency streaming).
Establish a bidirectional session using agent.run_live().
Engage in a multi-turn conversation or wait for a model response.
Observe the connection drop with the 1000 None traceback.
Expected Behavior:
The WebSocket connection should remain persistent for the duration of the session, handling bidirectional audio/text streams without abrupt closure.
Observed Behavior:
The connection closes unexpectedly with status code 1000 (Normal Closure) but no reason string (None), causing the ADK runner to crash.
Plaintext
google.genai.errors.APIError: 1000 None.
File "/app/.venv/lib/python3.11/site-packages/google/genai/live.py", line 545, in _receive
errors.APIError.raise_error(code, reason, None)
Environment Details:
ADK Library Version (pip show google-adk): 1.25.1 (or latest as of Feb 2026)
Desktop OS: Linux (Containerized on Cloud Run)
Python Version (python -V): 3.11.x
Model Information:
Are you using LiteLLM: No
Which model is being used: gemini-live-2.5-flash-native-audio (Vertex AI)
🟡 Optional Information
Regression:
Unknown. However, I have have noted higher stability on the gemini-live-2.5-flash-native-audio endpoint via the Gemini API compared to the Vertex AI endpoint.
Logs:
Traceback (most recent call last):
File "/app/.venv/lib/python3.11/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 199, in run_live
async for event in agen:
File "/app/.venv/lib/python3.11/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 371, in _receive_from_model
async for llm_response in agen:
File "/app/.venv/lib/python3.11/site-packages/google/adk/models/gemini_llm_connection.py", line 172, in receive
async for message in agen:
File "/app/.venv/lib/python3.11/site-packages/google/genai/live.py", line 454, in receive
while result := await self._receive():
File "/app/.venv/lib/python3.11/site-packages/google/genai/live.py", line 545, in _receive
errors.APIError.raise_error(code, reason, None)
google.genai.errors.APIError: 1000 None.
Additional Context:
The issue persists even when Cloud Run timeouts are increased to 3600s and HTTP/2 is enabled. It appears to be an application-level or API-level disconnect rather than a network timeout.
Minimal Reproduction Code:
Python
import asyncio
import os
from fastapi import FastAPI, WebSocket
from google.adk.agents.live_request_queue import LiveRequestQueue
from google.adk.agents.run_config import RunConfig, StreamingMode
from google.adk.runners import Runner
from google.adk.sessions import InMemorySessionService
from google.adk.memory import InMemoryMemoryService
from google.adk.agents import LlmAgent
from google.genai import types
1. Minimal Agent Setup
Replace with your specific instructions/tools if necessary
mock_agent = LlmAgent(
model="gemini-live-2.5-flash-native-audio",
instructions="You are a helpful tutor."
)
app = FastAPI()
session_service = InMemorySessionService()
memory_service = InMemoryMemoryService()
runner = Runner(
app_name="reproduction-app",
agent=mock_agent,
session_service=session_service,
memory_service=memory_service
)
@app.websocket("/ws/{user_id}/{session_id}")
async def websocket_endpoint(websocket: WebSocket, user_id: str, session_id: str):
await websocket.accept()
# 2. Minimal RunConfig
run_config = RunConfig(
streaming_mode=StreamingMode.BIDI,
response_modalities=["AUDIO"],
input_audio_transcription=types.AudioTranscriptionConfig(),
output_audio_transcription=types.AudioTranscriptionConfig(),
speech_config=types.SpeechConfig(
voice_config=types.VoiceConfig(
prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name="Zephyr")
)
)
)
live_request_queue = LiveRequestQueue()
# 3. Upstream: WebSocket -> Gemini
async def upstream_task():
try:
while True:
message = await websocket.receive()
if "bytes" in message:
audio_blob = types.Blob(mime_type="audio/pcm;rate=16000", data=message["bytes"])
live_request_queue.send_realtime(audio_blob)
except Exception:
live_request_queue.close()
# 4. Downstream: Gemini -> WebSocket (Where the 1000 None occurs)
async def downstream_task():
try:
async for event in runner.run_live(
user_id=user_id,
session_id=session_id,
live_request_queue=live_request_queue,
run_config=run_config,
):
await websocket.send_text(event.model_dump_json(exclude_none=True))
except Exception as e:
print(f"CRASH DETECTED: {e}")
await asyncio.gather(upstream_task(), downstream_task())
How often has this issue occurred?:
Always (100%+)
Activity
llalitkumarrr commentedon Feb 24, 2026
Hello @anupam-mishra ,
We attempted to reproduce this issue but everything worked as expected on our end. For our test we used the
BiDi streaming demowithVertex AI, ADKversion 1.25.1,voice and text modalitiesand thegemini-live-2.5-flash-native-audiomodel. We did not experience any connection drops.The only time we received an error (
google.genai.errors.APIError: 1000 None.) was when we closed the UI. However, reopening the UI at the same address successfully restored the connection. Please note that this testing was performed locally.anupam-mishra commentedon Feb 24, 2026
Hey @llalitkumarrr thanks for the response. Did you run the session for long say atleast 5 mins. ?
After running the session for an extended period, I am observing a consistent hang in the agent's logic. While I occasionally receive a 1000 status code, the behavior eventually degrades into the following state:
The Issue
Input processing works: The system correctly captures my speech. adkEvent.inputTranscription?.text populates accurately with what I am saying.
Output is stuck: The agent stops generating any outputTranscription or audio.
The "Echo" Effect: It feels as though the session is stuck in a loop where it acknowledges the input but fails to trigger the LLM/Agent response.
Any idea. ?
llalitkumarrr commentedon Feb 24, 2026
Hello @anupam-mishra ,
Could you try experiment a bit more to check if this is related to session time or number queries? If you could provide us with some idea about approximate number of query or time or both this would help narrow down our investigation.
anupam-mishra commentedon Feb 24, 2026
Hey @llalitkumarrr Not sure if this helps, but it works fine in a new session. The 'weird' behavior only happens in old sessions after a while. If I create a new session, everything is fine until it stops responding again—though it does continue to generate input transcripts.
anupam-mishra commentedon Feb 24, 2026
Hi @hshahrokni2 ,I doubt the issue is related to concurrency or HTTP/2 limits; I encounter the same error without HTTP/2, and since I am currently the only user with a single active session, I am nowhere near the concurrency limits.
Regarding the suggestions:
Concurrency/HTTP2: As mentioned, these are unlikely factors given the low-load environment.
Heartbeats: I can attempt to implement a heartbeat/keepalive ping to rule out idle timeouts on the Vertex endpoint. However, I am skeptical this is the cause, as the agent stops responding abruptly while active rather than during a lull.
The most unusual behavior is that the system continues to receive and process input transcripts but stops generating responses or output transcripts. It is quite odd that the backend remains responsive enough to acknowledge what I'm saying, yet fails to trigger the generation phase before the session terminates. I simply doesn't log any error also. Works fine for any new session though.
llalitkumarrr commentedon Feb 26, 2026
Hello @anupam-mishra ,
We are unable to reproduce this, could you please share exact step which we can follow to reproduce this?
anupam-mishra commentedon Feb 28, 2026
Hey @llalitkumarrr The steps are the same. However, instead of happening every time, my further testing shows it occurs intermittently. The agent suddenly stops responding within a session but works fine in the next one. For the existing session with the issue, it does respond with the InputTranscript. In this case, the only thing I see in the logs when it responds to the old session is
2026-02-28 09:41:03,801 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='b9d02976-1b2f-4531-8ab9-1ba335a1da79' resumable=True last_consumed_client_message_index=137.
2026-02-28 09:41:09,081 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='cfaee22d-c680-4a39-9f0f-5a176e859803' resumable=True last_consumed_client_message_index=1644.
2026-02-28 09:41:14,471 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='5da5287c-fc41-418e-b6ad-e9e5edd2f207' resumable=True last_consumed_client_message_index=2146.
2026-02-28 09:42:17,449 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='8a12c6ad-e8c9-470d-919a-f9d90946ab5a' resumable=True last_consumed_client_message_index=10178.
2026-02-28 09:42:43,365 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='b51ccf04-eea1-4dc8-b593-4829d62cb92e' resumable=True last_consumed_client_message_index=13441.
2026-02-28 09:42:57,292 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='aff84d28-6d12-4cf1-89b2-f9dd927f792b' resumable=True last_consumed_client_message_index=15198.
2026-02-28 09:43:21,416 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='59588100-8256-4766-a683-a1f0f9f188fb' resumable=True last_consumed_client_message_index=18210.
2026-02-28 09:43:36,928 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='a1cfb362-f45f-4a76-84a4-6724ff3941a2' resumable=True last_consumed_client_message_index=20218.
2026-02-28 09:43:48,581 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='dcd6836b-d20d-4dfb-8699-db826dacc223' resumable=True last_consumed_client_message_index=21724.
2026-02-28 09:44:03,478 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='5ca7b1d3-e554-41d1-a99d-e9c92868f109' resumable=True last_consumed_client_message_index=23481.
2026-02-28 09:44:12,226 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='332f995e-ef09-430a-b59b-0b59ba3cf5f4' resumable=True last_consumed_client_message_index=24630.
2026-02-28 09:44:17,387 - google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - Update session resumption handle: new_handle='9c11c91b-6497-4534-908a-ee11a6dd9c23' resumable=True last_consumed_client_message_index=25313.
ocedev112 commentedon Mar 14, 2026
Yeah, I'm experiencing the same issue and Gemini API is worse: google.genai.errors.APIError: 1000 None. The operation was cancelled. around 9:45 minutes or 9 minutes consistently: `async def run_interview(websocket, applicant_id, interview_app):
runner = InMemoryRunner(app=interview_app)
@app.websocket("/interview/start/{interview_id}/{applicant_id}")
async def audio_interview(websocket: WebSocket, interview_id: str, applicant_id: str):
await websocket.accept()
ocedev112 commentedon Mar 14, 2026
Have you found a fix for this issue?
llalitkumarrr commentedon Mar 16, 2026
Hello,
Could you run same test with
vertexaiand see if this is reproducible withvertexai? We tried to reproduce this few times but were not able to reproduce this.anupam-mishra commentedon Mar 16, 2026
Hi @ocedev112 , The frequency has decreased when using Vertex AI, but I still get the same issue once in a while. The gemini_api_key option is worse, and the issue occurs very frequently
ocedev112 commentedon Mar 16, 2026
Yes, I am experiencing the same issue. But the problem is transcription is better with gemini api(both are still bad).
2 remaining items
wuliang229 commentedon Apr 9, 2026
@ocedev112 Hi, your issue sounds like a working-as-intended scenario. In your version of ADK, the resumption is not handled well, and according to the documentation of Live API, the connection usually ends at around 10 minutes. We tried to fix it in 6b1600f. Please give it a try.
wuliang229 commentedon Apr 9, 2026
@anupam-mishra Just to clarify, did you still receive the input transcription after seeing the 1000 APIError, or is it a separate issue? 1000 Error means the Websocket is closed and therefore you shouldn't receive any input transcription. Anyway, please see if the above-mentioned commit works for you.
anupam-mishra commentedon Apr 9, 2026
@wuliang229 Thanks for looking into this. Yes I did receive the Input Transcript with what I said, but agent(no output transcript or audio) doesn't respond. The only thing I observed is this log on the server when the issue occurs google_adk.google.adk.flows.llm_flows.base_llm_flow - INFO - [c4684113fdff2901b95081810469a249] - Update session resumption handle: new_handle='af9cca82-a1fe-441d-af6a-b509355826b1' resumable=True last_consumed_client_message_index=21661.
I will check the above commit and confirm. But since the issue is intermittent I will take sometime to revert.
anupam-mishra commentedon Apr 24, 2026
Hey @wuliang229 , I’ve tested the fix and confirm that the original issue is not appearing in version 1.29.0. However, I’ve encountered a new problem: I suddenly start receiving the error below, and the session stops progressing until a new one is created.
Is this a known issue? The system is using google-adk=1.29.0 with gemini-live-2.5-flash-native-audio and the same code as before.
Error log:
2026-04-24 13:48:58,252 - google_adk.google.adk.flows.llm_flows.base_llm_flow - ERROR - [664a79b348984dcc1e835bd7208ecae5] - APIError in live flow: 1007 None. error when processing input audio, please check if the inputaudio is in valid format: 16khz s16le pcm, mono channel.; Error
2026-04-24 13:48:58,268 - google_adk.google.adk.flows.llm_flows.base_llm_flow - ERROR - [664a79b348984dcc1e835bd7208ecae5] - APIError in live flow: 1007 None. error when processing input audio, please check if the inputaudio is in valid format: 16khz s16le pcm, mono channel.; Error
2026-04-24 13:48:58,279 - main - ERROR - [664a79b348984dcc1e835bd7208ecae5] - Downstream Error: 1007 None. error when processing input audio, please check if the inputaudio is in valid format: 16khz s16le pcm, mono channel.; Error
llalitkumarrr commentedon Apr 29, 2026
Closing this issue here, as a new one was opened for the new problem.
anupam-mishra commentedon Apr 29, 2026
@llalitkumarrr do you have the link for the new issue you mentioned.?
llalitkumarrr commentedon Apr 29, 2026
Hello @anupam-mishra ,
You mentioned this in #5494. If that issue does not cover the new problem you described in your previous comment here, we would recommend opening a separate issue for it. Thank you for your understanding.
anupam-mishra commentedon Apr 30, 2026
@llalitkumarrr It was someone else, I didn't. Nevertheless I will log another issue . Thanks