Skip to content

Tool Hallucination when using ContextWindowCompression and BIDI Live #4231

Closed

Description

@GrahamGGreig

Running into an issue when using ContextWindowCompression whereby I see agents hallucinate tools they do not have access to. This is a transient issue and can be hard to reproduce. I have only ever seen this when using context window compression and it does not happen otherwise (Instead I hit a context overflow error).

I have two agents defined as such (note: I'm excluding some context for brevity):

def get_general_responder_agent(live_api: bool = False):
    return LlmAgent(
        name=NAME,
        model=select_model(live_api),
        description=get_description(),
        instruction=get_instructions(),
        planner=_get_planner(),
        tools=[get_context_tool()],
        generate_content_config=_get_generate_content_config(),
    )

def get_persona_responder_agent(live_api: bool = False):
    return LlmAgent(
        name=NAME,
        model=select_model(live_api),
        description=get_description(),
        instruction=get_instructions(),
        tools=[get_persona_tool()],
        generate_content_config=_get_generate_content_config(),
    )

def get_context_tool():
    return FunctionTool(func=set_context) # function definition left out for brevity

def get_persona_tool():
    return FunctionTool(func=set_persona_instructions) # function definition left out for brevity

def configure_root_agent(live_api: bool = False) -> LlmAgent:
    # agents
    general_responder = get_general_responder_agent(live_api)
    general_responder = configure_user_facing_agent(general_responder)

    persona_responder = get_persona_responder_agent(live_api)
    persona_responder = configure_user_facing_agent(persona_responder)

    # routing
    general_responder.sub_agents = [persona_responder]
    persona_responder.parent_agent = general_responder

    general_responder.before_agent_callback = initialize_state_variables

    return general_responder


def initialize_state_variables(
    callback_context: CallbackContext,
) -> Optional[types.Content]:
    callback_context = initialize_language_selector_state_variables(callback_context)
    callback_context = initialize_persona_state_variables(callback_context)
    callback_context = initialize_context_state_variables(callback_context)


def configure_user_facing_agent(agent: LlmAgent) -> LlmAgent:
    agent.tools.extend(get_default_tools())
    agent.instruction += language_instructions()
    agent.instruction += general_instructions()

    return agent

def get_default_tools():
    return [
        get_general_user_assistance_tool(),
        get_language_selector_tool(),
    ]

and using run config:

    run_config = RunConfig(
        streaming_mode=StreamingMode.BIDI,
        response_modalities=[types.Modality.AUDIO],
        speech_config=types.SpeechConfig(
            voice_config=types.VoiceConfig(
                prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name=VOICE_NAME)
            )
        ),
        output_audio_transcription=types.AudioTranscriptionConfig(),
        input_audio_transcription=types.AudioTranscriptionConfig(),
        session_resumption=types.SessionResumptionConfig(),
        save_live_blob=False,
        context_window_compression=types.ContextWindowCompressionConfig(
            trigger_tokens=60000, sliding_window=types.SlidingWindow() # Our application currently draws in a lot of contexts. I imagine this would need to be smaller to trigger on a simpler application.
        )
    )

Using the model: gemini-live-2.5-flash through Vertex AI. Using ADK 1.22.0. The error I can transiently get is:

Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 199, in run_live
    async for item in agen:
      yield item
  File "/usr/local/lib/python3.13/site-packages/google/adk/agents/base_agent.py", line 327, in run_live
    async for event in agen:
      yield event
  File "/usr/local/lib/python3.13/site-packages/google/adk/agents/llm_agent.py", line 492, in _run_live_impl
    async for event in agen:
      self.__maybe_save_output_to_state(event)
      yield event
  File "/usr/local/lib/python3.13/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 199, in run_live
    async for item in agen:
      yield item
  File "/usr/local/lib/python3.13/site-packages/google/adk/agents/base_agent.py", line 327, in run_live
    async for event in agen:
      yield event
  File "/usr/local/lib/python3.13/site-packages/google/adk/agents/llm_agent.py", line 492, in _run_live_impl
    async for event in agen:
      self.__maybe_save_output_to_state(event)
      yield event
  File "/usr/local/lib/python3.13/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 199, in run_live
    async for item in agen:
      yield item
  File "/usr/local/lib/python3.13/site-packages/google/adk/agents/base_agent.py", line 327, in run_live
    async for event in agen:
      yield event
  File "/usr/local/lib/python3.13/site-packages/google/adk/agents/llm_agent.py", line 492, in _run_live_impl
    async for event in agen:
      self.__maybe_save_output_to_state(event)
      yield event
  File "/usr/local/lib/python3.13/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 152, in run_live
    async for event in agen:
    ...<59 lines>...
        return
  File "/usr/local/lib/python3.13/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 329, in _receive_from_model
    async for event in agen:
    ...<19 lines>...
      yield event
  File "/usr/local/lib/python3.13/site-packages/google/adk/flows/llm_flows/base_llm_flow.py", line 643, in _postprocess_live
    function_response_event = await functions.handle_function_calls_live(
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        invocation_context, model_response_event, llm_request.tools_dict
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/google/adk/flows/llm_flows/functions.py", line 476, in handle_function_calls_live
    function_response_events = await asyncio.gather(*tasks)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/google/adk/flows/llm_flows/functions.py", line 509, in _execute_single_function_call_live
    tool, tool_context = _get_tool_and_context(
                         ~~~~~~~~~~~~~~~~~~~~~^
        invocation_context, function_call, tools_dict
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/google/adk/flows/llm_flows/functions.py", line 754, in _get_tool_and_context
    tool = _get_tool(function_call, tools_dict)
  File "/usr/local/lib/python3.13/site-packages/google/adk/flows/llm_flows/functions.py", line 729, in _get_tool
    raise ValueError(error_msg)
ValueError: Tool 'set_context' not found.
Available tools: transfer_to_agent, set_persona_instructions, general_user_assistance_tool, Language_Logging_Tool

Possible causes:
  1. LLM hallucinated the function name - review agent instruction clarity
  2. Tool not registered - verify agent.tools list
  3. Name mismatch - check for typos

Suggested fixes:
  - Review agent instruction to ensure tool usage is clear
  - Verify tool is included in agent.tools list
  - Check for typos in function name
2026-01-16 21:57:33,696 - ERROR - Exception in agent_to_client_sse: Tool 'set_context' not found.

To Reproduce
Run an application with multiple agents where they each have different tools. Interact with the agents using the live api model (gemini-live-2.5-flash through Vertex AI). Have a long enough conversation that context is reset. Ask questions that require the tool use of the opposite agent. See if it hallucinates the tool instead of performing an agent transfer.

Expected behavior
To transfer to the correct agent that has the tool instead of hallucinating the tool.

Desktop (please complete the following information):

  • OS: Linux container
  • Python version(python -V): 3.13.7
  • ADK version(pip show google-adk): 1.22.1

Model Information:

  • Are you using LiteLLM: No
  • Which model is being used(e.g. gemini-2.5-pro): gemini-live-2.5-flash through VertexAI

Activity

  1. added
    core[Component] This issue is related to the core interface and implementation
    on Jan 22, 2026
  2. added theissue type on Jan 22, 2026
  3. klateefa commented on Jan 22, 2026

    @klateefa

    @GrahamGGreig Thanks for reporting
    This issue originates from the ADK runtime rather than model behavior or user configuration.

    When ContextWindowCompression (SlidingWindow) is applied, the model may carry forward references to tools that were valid earlier in the session. However, after an agent transition, the active agent’s tools_dict is rebuilt and no longer includes those tools. As a result, the model can emit a function call that is no longer executable in the current agent context, causing _get_tool to fail.
    The fix is to enforce tool ownership at execution time. If a requested tool is not available on the active agent, the flow should redirect via agent transfer instead of raising an error,this restores correct multi-agent routing semantics and ensures safe use of context compression in BIDI live mode.
    Thank you.

  4. self-assigned this
    on Jan 22, 2026
  5. added
    answered[Status] This issue has been answered by the maintainer
    on Jan 22, 2026
  6. GrahamGGreig commented on Jan 22, 2026

    @GrahamGGreig
    Author

    Hi @klateefa is there a current way to perform that fix? Or is this still an open issue? From my understanding I do not have the ability to enforce tool ownership at execution time but perhaps I am wrong on that.

  7. GrahamGGreig commented on Feb 17, 2026

    @GrahamGGreig
    Author

    Hi @klateefa following up on this. Is there a known work around at this time or is this in the roadmap to be fixed?

  8. added
    live[Component] This issue is related to live, voice and video chat
    and removed
    core[Component] This issue is related to the core interface and implementation
    on Apr 16, 2026
  9. aaravanmay commented on Jun 10, 2026

    @aaravanmay

    per-test thresholds would be a big improvement over the global pass rate. One thing that helped me reduce how many repeats I need at all: splitting checks into "answer quality" (genuinely non-deterministic, needs repeats like you describe) vs "did the agent take the right action" (tool called, right args, sane values). The second kind can be asserted deterministically against the tool-call trace, so it doesn't need 8/10 sampling - it either took the action or it didn't. Shrinks the flaky surface a lot.

  10. pr-maia commented on Jun 24, 2026

    @pr-maia

    Had the exact same issue just now. It was a bizarre hallucination, my agent used a few tools to interact with my knowledge database and out of nowhere it tried to create a file with this (truncated):

    Tool: create_file
    Args:
    {
      "title": "report.json",
      "content": {
        "category-verdicts": {
          "task-quality": {"verdict": "OK", "summary": "The task is realistic, difficult, and verifiable. It requests the implementation of a new data source (TikTok Ads) for an ingestion framework."},
          "statement-quality": {"verdict": "OK", "summary": "The problem statement is self-contained, unambiguous..."},
    

    My agent doesnt write code or interact with tiktok ads in any capacity. It was a complete hallucination by the model.

  11. llalitkumarrr commented on Jul 30, 2026

    @llalitkumarrr
    Collaborator

    Hello @GrahamGGreig ,

    Apologies for the late response. Could you please test the latest version and let us know if this issue is still relevant?

  12. added
    request clarification[Status] The maintainer need clarification or more information from the author
    and removed
    answered[Status] This issue has been answered by the maintainer
    on Jul 30, 2026
  13. adk-bot commented on Aug 7, 2026

    @adk-bot
    Collaborator

    This issue has been automatically marked as stale because it has not had recent activity for 7 days after a maintainer requested clarification. It will be closed if no further activity occurs within 7 days.

  14. added
    stale[Status] Issues which have been marked inactive since there is no user response
    on Aug 7, 2026
  15. adk-bot commented on Aug 14, 2026

    @adk-bot
    Collaborator

    This has been automatically closed because it has been marked as stale for over 7 days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

live[Component] This issue is related to live, voice and video chatrequest clarification[Status] The maintainer need clarification or more information from the authorstale[Status] Issues which have been marked inactive since there is no user response

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions