Repository navigation
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
Models re-emit an identical tool call while a slow tool is still running or repeat it in a later step; the reporter of google#3940 saw production tools loop this way, and long-running tools re-fire while the model waits. Wrap the tool execution step of the shared tool-calling pipeline in an invocation-scoped single-flight cache keyed by agent, branch, tool name and canonical arguments: the first call runs the tool and publishes a snapshot that concurrent and later duplicates reuse as their own copy. It is opt-in via RunConfig.dedupe_tool_calls and always on for LongRunningFunctionTool; callbacks still run per call, a failed run is evicted so a later call retries, and a result that transfers, escalates or requests confirmation or auth is never shared. Implemented with Claude Code (Claude Fable 5.1); the author reviewed and tested the change. Closes google#3940
e8e4ebe to
c92aac2
Compare
|
Thank you so much for the thorough implementation, detailed docstrings, and comprehensive test suite, as well as your patience carrying this over from #4189! After reviewing this with the team, we've decided not to add a run-wide
import json
CACHEABLE_TOOLS = {"expensive_readonly_lookup"}
def cache_before_tool(tool, args, tool_context):
if tool.name not in CACHEABLE_TOOLS:
return None
cache = tool_context.state.get("temp:_tool_cache", {})
key = f"{tool.name}:{json.dumps(args, sort_keys=True, default=str)}"
return cache.get(key)
def cache_after_tool(tool, args, tool_context, tool_response):
if tool.name in CACHEABLE_TOOLS:
cache = dict(tool_context.state.get("temp:_tool_cache", {}))
key = f"{tool.name}:{json.dumps(args, sort_keys=True, default=str)}"
cache[key] = tool_response
tool_context.state["temp:_tool_cache"] = cache
return NoneUsing Thanks again for your contribution and effort! |
Successor of #4189 (closed for merge conflicts), re-implemented on the current tool pipeline instead of rebased. Gemini's two findings on #4189 (duplicated dedupe predicate, duplicated cache-hit metadata block) are single helpers here.
Link to Issue or Description of Change
1. Link to an existing issue (if applicable):
Problem: models re-emit an identical tool call while a slow tool is still running, or repeat it in a later step, and every repeat runs the tool again.
Solution:
RunConfig.dedupe_tool_calls(defaultFalse). Within one invocation, identical calls (same agent, branch, tool name and canonical arguments) share one tool execution through an invocation-scoped single-flight cache around Step 3 of_execute_single_prepared_call, so async and live modes use the same hook.custom_metadata['adk_tool_call_cache_hit'] = True.stop_streaming, live streaming tools,_defers_responsetools, and calls carrying a confirmation answer.LongRunningFunctionToolcalls are always deduped. Deliberate default; happy to make it opt-in too.LongRunningFunctionTool: Step 3 is the unchangedawait tool_runner().Testing Plan
Unit Tests:
33 new tests in
tests/unittests/flows/llm_flows/tools/test_functions_dedupe.py,test_caller.py,test_batch_executor.pyandtests/unittests/agents/test_run_config.py: duplicates across steps and within one step, default off, long-running default, argument identity (type and key order), side effects applied once, eviction on failure including concurrent waiters, per-call callbacks, confirmation and rejection, transfers, per-agent and per-branch scope.Full
tests/unittestssuite, fresh venv per interpreter with thetestextra,pytest -n auto:pre-commit run --files <touched files>: all hooks pass.mypywith the repo's strict config on the touched modules: no issues.Manual End-to-End (E2E) Tests:
Real model (
gemini-3.6-flash), this branch,InMemoryRunner, each scenario with the flag off and on:output_schema+output_key+ "call exactly once", 12 s tool), five variants incl. the exact prompt,ParallelAgentandLongRunningFunctionToolWhen the model repeats a call, the duplicate is answered from the first execution and the run is one tool latency shorter. Without duplicates the flag changes nothing. The loop described on #3940 did not reproduce on current
mainwith this model; this change removes the redundant execution when a model does repeat a call.Checklist
Additional context
docs/guides/runners/runner/index.mdgains theRunConfigrow; the user-facing reference in adk-docs will follow in a separate PR.