Skip to content

feat(flows): dedupe identical tool calls within an invocation - #7193

Closed
jayy-77 wants to merge 1 commit into
google:mainfrom
jayy-77:fix-tool-dedup-3940-v2
Closed

jayy-77 wants to merge 1 commit into
google:mainfrom
jayy-77:fix-tool-dedup-3940-v2

Conversation

@jayy-77

@jayy-77 jayy-77 commented Sep 18, 2026 •

Copy link
Copy Markdown

Successor of #4189 (closed for merge conflicts), re-implemented on the current tool pipeline instead of rebased. Gemini's two findings on #4189 (duplicated dedupe predicate, duplicated cache-hit metadata block) are single helpers here.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

Problem: models re-emit an identical tool call while a slow tool is still running, or repeat it in a later step, and every repeat runs the tool again.

Solution: RunConfig.dedupe_tool_calls (default False). Within one invocation, identical calls (same agent, branch, tool name and canonical arguments) share one tool execution through an invocation-scoped single-flight cache around Step 3 of _execute_single_prepared_call, so async and live modes use the same hook.

  • Only the tool execution is shared. Before/after-tool callbacks run per call; each duplicate gets its own deep copy of the result and its own response event, marked custom_metadata['adk_tool_call_cache_hit'] = True.
  • Never shared: failed runs (evicted, the next identical call retries), results whose run recorded any action beyond state or artifact deltas (transfer, escalate, auth or confirmation request), stop_streaming, live streaming tools, _defers_response tools, and calls carrying a confirmation answer.
  • LongRunningFunctionTool calls are always deduped. Deliberate default; happy to make it opt-in too.
  • Flag off and no LongRunningFunctionTool: Step 3 is the unchanged await tool_runner().

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

33 new tests in tests/unittests/flows/llm_flows/tools/test_functions_dedupe.py, test_caller.py, test_batch_executor.py and tests/unittests/agents/test_run_config.py: duplicates across steps and within one step, default off, long-running default, argument identity (type and key order), side effects applied once, eviction on failure including concurrent waiters, per-call callbacks, confirmation and rejection, transfers, per-agent and per-branch scope.

Full tests/unittests suite, fresh venv per interpreter with the test extra, pytest -n auto:

Python Result
3.10 15186 passed, 0 failed
3.11 15195 passed, 0 failed
3.12 15186 passed, 0 failed
3.13 15186 passed, 0 failed
3.14 15186 passed, 0 failed

pre-commit run --files <touched files>: all hooks pass. mypy with the repo's strict config on the touched modules: no issues.

Manual End-to-End (E2E) Tests:

Real model (gemini-3.6-flash), this branch, InMemoryRunner, each scenario with the flag off and on:

Scenario Flag Runs Model calls Tool executions Cache hits Wall
Agent instructed to call a 4 s tool twice off 3 2 2 0 14.3 s
same on 3 2 1 1 9.3 s
#3940 reporter's agent (output_schema + output_key + "call exactly once", 12 s tool), five variants incl. the exact prompt, ParallelAgent and LongRunningFunctionTool off / on 14 / 14 1 per agent 1 per agent 0 17-19 s

When the model repeats a call, the duplicate is answered from the first execution and the run is one tool latency shorter. Without duplicates the flag changes nothing. The loop described on #3940 did not reproduce on current main with this model; this change removes the redundant execution when a model does repeat a call.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

Additional context

docs/guides/runners/runner/index.md gains the RunConfig row; the user-facing reference in adk-docs will follow in a separate PR.

@google-cla

google-cla Bot commented Sep 18, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Models re-emit an identical tool call while a slow tool is still running
or repeat it in a later step; the reporter of google#3940 saw production tools
loop this way, and long-running tools re-fire while the model waits.

Wrap the tool execution step of the shared tool-calling pipeline in an
invocation-scoped single-flight cache keyed by agent, branch, tool name
and canonical arguments: the first call runs the tool and publishes a
snapshot that concurrent and later duplicates reuse as their own copy.
It is opt-in via RunConfig.dedupe_tool_calls and always on for
LongRunningFunctionTool; callbacks still run per call, a failed run is
evicted so a later call retries, and a result that transfers, escalates
or requests confirmation or auth is never shared.

Implemented with Claude Code (Claude Fable 5.1); the author reviewed and
tested the change.

Closes google#3940
@jayy-77
jayy-77 force-pushed the fix-tool-dedup-3940-v2 branch from e8e4ebe to c92aac2 Compare September 18, 2026 17:53
@xuanyang15

Copy link
Copy Markdown
Collaborator

Thank you so much for the thorough implementation, detailed docstrings, and comprehensive test suite, as well as your patience carrying this over from #4189!

After reviewing this with the team, we've decided not to add a run-wide RunConfig.dedupe_tool_calls flag into the core tool execution pipeline, so we'll be closing this PR:

  1. Idempotency is a property of individual tools, not an entire run: When a model emits multiple calls to the same tool within one step or across steps, it is often intentional—for example, rolling a die multiple times (roll_die(sides=6) in our hello_world sample), polling get_current_time(), or re-querying state after a mutation (get_balance() -> transfer() -> get_balance()). A run-wide flag silently returns duplicate random/time values, drops per-call state_deltas on the second call, and returns stale results across state-mutating steps.
  2. Achievable today via before_tool_callback / after_tool_callback (or BasePlugin) without core changes: ADK's callback and plugin hooks already sit directly around tool execution and allow scoping memoization strictly to an explicit allowlist of pure, idempotent tools:
import json

CACHEABLE_TOOLS = {"expensive_readonly_lookup"}

def cache_before_tool(tool, args, tool_context):
  if tool.name not in CACHEABLE_TOOLS:
    return None
  cache = tool_context.state.get("temp:_tool_cache", {})
  key = f"{tool.name}:{json.dumps(args, sort_keys=True, default=str)}"
  return cache.get(key)

def cache_after_tool(tool, args, tool_context, tool_response):
  if tool.name in CACHEABLE_TOOLS:
    cache = dict(tool_context.state.get("temp:_tool_cache", {}))
    key = f"{tool.name}:{json.dumps(args, sort_keys=True, default=str)}"
    cache[key] = tool_response
    tool_context.state["temp:_tool_cache"] = cache
  return None

Using temp: state keys keeps the cache scoped to the current invocation automatically without adding new public fields to RunConfig or extra branching in _caller.py and _batch_executor.py.

Thanks again for your contribution and effort!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tool Call has been called multiple times by ADK

2 participants