Repository navigation
Description
Environment
- Python 3.13
google-adk==2.0.0- Default ADK feature:
PROGRESSIVE_SSE_STREAMING(on by default) - Triggered under
RunConfig(streaming_mode=StreamingMode.SSE)
Bug description
Under RunConfig(streaming_mode=SSE), ADK's StreamingResponseAggregator runs in
PROGRESSIVE_SSE_STREAMING mode (default-on) and marks every intermediate chunk
partial=True — including the chunk that carries a task-delegation FunctionCall. The
non-partial aggregate that re-carries the FC is only produced by aggregator.close()
after the model stream fully drains.
ADK's chat-mode wrapper, google.adk.workflow._llm_agent_wrapper.run_llm_agent_as_node,
extracts task-delegation FCs from every yielded event with no partial check,
dispatches the specialist, and breaks out of the agent's event generator. Two defects
follow:
- The FC event is never persisted. The Runner only appends events with
partial is not Trueto the session; thebreakcloses the generator before the
non-partial aggregate is ever yielded. The synthesized taskFunctionResponseis
persisted — so the very next contents build sees an orphaned FR (a function response
with no matching function call) and raises
ValueError: No function call event found for function responses ids: {...}. The
session is permanently poisoned: every later turn replays the same history and
re-raises. - Dispatch can fire with truncated args. Progressive SSE streams FC arguments
across chunks; extracting from the first partial chunk that names the FC can dispatch
the specialist with incomplete (even empty) input.
This defect is the dominant producer of the orphaned-function-response poisoned
state reported in the companion draft
docs/upstream-reports/02-orphaned-function-response-poisoning.md: the wrapper
synthesizes and persists a task-delegation function response while never persisting
the matching function call.
Reproduction
A self-contained Python repro drives a real Runner + coordinator(mode='chat') +
task specialist whose stub coordinator LLM streams the delegation FC in the exact
progressive-SSE shape (a partial=True chunk first, then the partial=False aggregate
re-carrying the same FC with the same id). No downstream workaround is installed — this
is the raw ADK 2.0.0 behaviour.
"""
Minimal repro: ADK 2.0.0 progressive-SSE task-dispatch defect.
Run: cd <repo with google-adk==2.0.0> && python repro_partial_sse.py
Expected: the coordinator's re-entry contents build raises the
orphaned-response ValueError; the persisted session shows the task FR with no
matching task FC.
"""
from __future__ import annotations
import asyncio
import traceback
from collections.abc import AsyncIterator
from typing import Any
from google.adk.agents import LlmAgent
from google.adk.models.base_llm import BaseLlm
from google.adk.models.llm_response import LlmResponse
from google.genai.types import Content, FunctionCall, Part
_COORDINATOR_CALLS: list[int] = []
_SPECIALIST_CALLS: list[int] = []
class _StreamingDispatchCoordinatorLlm(BaseLlm):
"""Coordinator stub emitting the progressive-SSE dispatch shape.
Call 1 yields the task FC twice, exactly as ADK's StreamingResponseAggregator
does under PROGRESSIVE_SSE_STREAMING: a partial=True chunk first, then the
partial=False aggregate re-carrying the same FC (same id).
"""
model: str = "streaming_dispatch_stub"
@classmethod
def supported_models(cls) -> list[str]:
return ["streaming_dispatch_stub"]
async def generate_content_async( # type: ignore[override]
self, llm_request: Any, stream: bool = False
) -> AsyncIterator[LlmResponse]:
_COORDINATOR_CALLS.append(1)
if len(_COORDINATOR_CALLS) == 1:
fc = FunctionCall(
name="ga_specialist",
args={"request": "Top landing pages for June 2026"},
id="fc-dispatch-1",
)
# 1. partial chunk carrying the task FC — every intermediate
# progressive-SSE chunk is marked partial=True.
yield LlmResponse(
content=Content(role="model", parts=[Part(function_call=fc)]),
partial=True,
)
# 2. non-partial aggregate re-carrying the same FC (same id) —
# only this is persisted by the Runner.
yield LlmResponse(
content=Content(role="model", parts=[Part(function_call=fc)]),
partial=False,
turn_complete=True,
)
else:
yield LlmResponse(
content=Content(
role="model",
parts=[Part.from_text(text="Here are your top landing pages.")],
),
partial=False,
turn_complete=True,
)
class _FinishingSpecialistLlm(BaseLlm):
"""Task specialist stub: immediately calls finish_task with its result."""
model: str = "finishing_specialist_stub"
@classmethod
def supported_models(cls) -> list[str]:
return ["finishing_specialist_stub"]
async def generate_content_async( # type: ignore[override]
self, llm_request: Any, stream: bool = False
) -> AsyncIterator[LlmResponse]:
_SPECIALIST_CALLS.append(1)
fc = FunctionCall(
name="finish_task",
args={"result": "1. /pricing 2. /blog 3. /home"},
id=f"fin-{len(_SPECIALIST_CALLS)}",
)
yield LlmResponse(
content=Content(role="model", parts=[Part(function_call=fc)]),
turn_complete=True,
)
async def _run_repro() -> None:
from google.adk.runners import Runner
from google.adk.sessions import InMemorySessionService
from google.genai import types as genai_types
specialist = LlmAgent(
name="ga_specialist",
model=_FinishingSpecialistLlm(),
mode="task",
instruction="GA specialist",
disallow_transfer_to_parent=True,
)
coordinator = LlmAgent(
name="coordinator",
model=_StreamingDispatchCoordinatorLlm(),
mode="chat",
instruction="Coordinator",
tools=[],
sub_agents=[specialist],
)
session_service = InMemorySessionService()
session = await session_service.create_session(
app_name="partial_dispatch_repro", user_id="test_user"
)
runner = Runner(
agent=coordinator,
app_name="partial_dispatch_repro",
session_service=session_service,
)
print("Running a progressive-SSE-shaped dispatch turn (raw ADK, no guard)...\n")
try:
async for event in runner.run_async(
user_id=session.user_id,
session_id=session.id,
new_message=genai_types.Content(
role="user",
parts=[genai_types.Part.from_text(text="Top landing pages June 2026?")],
),
):
pass
print("NO RAISE — the defect may be fixed upstream; re-check on this ADK version.")
except ValueError as exc:
print("RAISED ValueError (orphaned task FR — the progressive-SSE poison):\n")
print(f" type: {type(exc).__module__}.{type(exc).__name__}")
print(f" message: {exc}\n")
print("Traceback (key frames):")
print("----")
print(traceback.format_exc())
print("----")
persisted_session = await session_service.get_session(
app_name="partial_dispatch_repro",
user_id=session.user_id,
session_id=session.id,
)
persisted = list(persisted_session.events) if persisted_session else []
def _fc_ids(name: str) -> list[str]:
out = []
for e in persisted:
for fc in e.get_function_calls():
if fc.name == name:
out.append(fc.id)
return out
def _fr_ids(name: str) -> list[str]:
out = []
for e in persisted:
for fr in e.get_function_responses():
if fr.name == name:
out.append(fr.id)
return out
print("\n── Persisted session events (post-turn) ──")
for i, e in enumerate(persisted):
role = getattr(e.content, "role", "?") if e.content else "?"
parts = []
for p in (e.content.parts if e.content else []):
if p.function_call:
parts.append(f"FC:{p.function_call.name}(id={p.function_call.id})")
elif p.function_response:
parts.append(f"FR:{p.function_response.name}(id={p.function_response.id})")
elif p.text:
parts.append(f"text:{p.text[:40]!r}")
print(f" [{i}] role={role} partial={e.partial} parts={parts}")
fc_ids = _fc_ids("ga_specialist")
fr_ids = _fr_ids("ga_specialist")
print()
print(f"Persisted coordinator task FC ids: {fc_ids}")
print(f"Persisted synthesized task FR ids: {fr_ids}")
print(f"Specialist dispatched {len(_SPECIALIST_CALLS)}x for one task FC")
print()
if not fc_ids and fr_ids:
print(
"BUG CONFIRMED: the coordinator's task-delegation FunctionCall event\n"
"was NOT persisted (the wrapper broke out on the partial=True chunk,\n"
"before the non-partial aggregate was ever yielded), while the\n"
"synthesized task FunctionResponse WAS persisted — an orphaned FR.\n"
"Every later contents build raises the orphaned-response ValueError.\n"
"Defect 3 is the dominant producer of defect 2's poisoned state."
)
elif fc_ids and fr_ids:
print(
"No orphan — the FC was persisted. If this is ADK 2.0.0 with the\n"
"guard removed, the defect may be fixed upstream; re-check."
)
if __name__ == "__main__":
asyncio.run(_run_repro())Observed output (run against google-adk==2.0.0, Python 3.13):
Running a progressive-SSE-shaped dispatch turn (raw ADK, no guard)...
RAISED ValueError (orphaned task FR — the progressive-SSE poison):
type: builtins.ValueError
message: No function call event found for function responses ids: {'fc-dispatch-1'}
Traceback (key frames):
----
File ".../google/adk/workflow/_llm_agent_wrapper.py", line 380, in run_llm_agent_as_node
async for event in run_iter:
...
break
File ".../google/adk/flows/llm_flows/base_llm_flow.py", line 935, in _preprocess_async
async for event in agen:
yield event
File ".../google/adk/flows/llm_flows/contents.py", line 74, in run_async
llm_request.contents = _get_contents(
File ".../google/adk/flows/llm_flows/contents.py", line 637, in _get_contents
result_events = _rearrange_events_for_latest_function_response(
filtered_events
)
File ".../google/adk/flows/llm_flows/contents.py", line 224, in _rearrange_events_for_latest_function_response
raise ValueError(
...
)
ValueError: No function call event found for function responses ids: {'fc-dispatch-1'}
----
── Persisted session events (post-turn) ──
[0] role=user partial=None parts=["text:'Top landing pages June 2026?'"]
[1] role=model partial=None parts=['FC:finish_task(id=fin-1)']
[2] role=user partial=None parts=['FR:finish_task(id=fin-1)']
[3] role=user partial=None parts=['FR:ga_specialist(id=fc-dispatch-1)']
[4] role=? partial=None parts=[]
Persisted coordinator task FC ids: []
Persisted synthesized task FR ids: ['fc-dispatch-1']
Specialist dispatched 1x for one task FC
BUG CONFIRMED: the coordinator's task-delegation FunctionCall event
was NOT persisted (the wrapper broke out on the partial=True chunk,
before the non-partial aggregate was ever yielded), while the
synthesized task FunctionResponse WAS persisted — an orphaned FR.
Every later contents build raises the orphaned-response ValueError.
Defect 3 is the dominant producer of defect 2's poisoned state.
The persisted-event dump is the smoking gun: event [3] is FR:ga_specialist(id=fc-dispatch-1)
with no matching FC:ga_specialist anywhere in the session (Persisted coordinator task FC ids: []). The specialist dispatched exactly once, so the orphan is not a double
dispatch — it is the wrapper breaking on the partial=True chunk before the
partial=False aggregate (the only event the Runner persists) was ever yielded.
Root cause
google/adk/workflow/_llm_agent_wrapper.py, function _extract_task_delegation_fcs
(line 56) has no partial check:
def _extract_task_delegation_fcs(
event: Event, tools_dict: dict
) -> list[types.FunctionCall]:
"""Return task-delegation FCs from this event."""
from ..tools.agent_tool import _TaskAgentTool
return [
fc
for fc in event.get_function_calls() # ← no `event.partial` gate
if fc.id
and fc.name in tools_dict
and isinstance(tools_dict[fc.name], _TaskAgentTool)
]It is called from run_llm_agent_as_node (line 382) inside the loop that breaks on any
extracted task FC (line 388):
async with aclosing(run_method) as run_iter:
async for event in run_iter:
yield event
task_fcs = _extract_task_delegation_fcs(event, tools_dict) # line 382
for fc in task_fcs:
output = await _dispatch_task_fc(agent, fc, ctx)
yield _synthesize_task_fr_event(fc, output)
if task_fcs:
had_task_fc = True
break # line 388 — closes run_iter before the non-partial aggregateUnder progressive SSE, the first chunk carrying the task FC is partial=True. The wrapper
extracts from it, dispatches, synthesizes the task FR, and breaks. The Runner only
persists events with partial is not True, so:
- the synthesized task
FunctionResponse(yielded by the wrapper itself, not by the
inner generator'spartial=Truechunk) is persisted, but - the coordinator's
FunctionCallevent is only ever carried by thepartial=False
aggregate, which thebreaknever reached → it is never persisted.
The result is an orphaned function response, which the next _preprocess_async contents
build raises on (contents.py:224 via _get_contents at contents.py:637).
Expected behaviour
Task-delegation FC extraction should be gated on non-partial events, exactly as ADK's
own process_llm_agent_output does for the non-streamed path — dispatch from the final
aggregate, which the Runner persists and which carries the complete FC args. Concretely,
_extract_task_delegation_fcs (or its caller) should return no FCs for events whose
partial is True, so the break only fires on the non-partial aggregate.
Observed behaviour
Without the gate, the wrapper dispatches from the partial=True chunk, breaks before
the non-partial aggregate is yielded, and the session is left with a synthesized task FR
and no matching task FC — permanently poisoned by the orphaned-response raise documented
in the companion report.
Related
- Consumption defect (companion report):
docs/upstream-reports/02-orphaned-function-response-poisoning.md
— the orphaned-FRValueErrorthis defect produces is the same raise that turns any
orphaned FR into a dead session. This report is the producer; that report is the
consumer. Fixing the wrapper's partial gate removes the dominant producer; fixing the
contents processor's fatal raise (companion) removes the fatal consumption. - Mixed-turn FC drop (companion report):
docs/upstream-reports/01-mixed-turn-fc-drop.md
— the samerun_llm_agent_as_nodebreak(line 388) is also the root of the mixed-turn
regular-FC drop. Both defects share the wrapper's break-on-task-FC design; a single
"drain remaining events beforebreak/ gate on non-partial" fix sketch addresses
both.
Activity
bferanmi806-sketch commentedon Aug 4, 2026
I’d like to work on this. I plan to add a narrow guard so task-delegation function calls are dispatched only from non-partial events, then add an offline progressive-SSE regression test verifying complete arguments, single dispatch, balanced persisted FC/FR history, and a usable subsequent turn. Please let me know if this direction aligns with the intended behavior.
a2105z commentedon Aug 4, 2026
@bferanmi806-sketch @kenwilly - also interested, let me know if I can help - maybe we can work together.
bferanmi806-sketch commentedon Aug 5, 2026
Thanks I see @llalitkumarrr is assigned now. I’d already investigated the progressive-SSE failure path and outlined a regression-test plan above. I’ll avoid duplicating the implementation, but I’m happy to share context, review the patch, or help test complete arguments, single dispatch, balanced persisted FC/FR history, and a usable subsequent turn.
llalitkumarrr commentedon Aug 6, 2026
Hello @bferanmi806-sketch and @a2105z,
Could you please share the details of your analysis with us? This information would be very helpful.
Additionally, @kenwilly mentioned version adk=2.0.0. Do we get the same results with version 2.6.0?
bferanmi806-sketch commentedon Aug 6, 2026
Thanks for asking. Here is the failure path I traced:
Under progressive SSE,
StreamingResponseAggregator.process_response()marks every intermediate model chunkpartial=True, including chunks carrying function-call names or partial arguments.The completed function call, with fully accumulated arguments, is emitted only by
StreamingResponseAggregator.close()as apartial=Falseaggregate.In
run_llm_agent_as_node, the chat-mode loop yields each event and then immediately passes it to_extract_task_delegation_fcs(). That helper currently has noevent.partialguard.When a task-delegation FC is present on a partial chunk, the wrapper dispatches the child task, synthesizes its FR, and then breaks out of the inner generator.
This closes the stream before the non-partial aggregate is reached. The partial FC event is not persisted, while the synthesized task FR is persisted.
On coordinator re-entry, contents reconstruction sees the task FR without its matching FC and raises:
ValueError: No function call event found for function responses ids: {...}The same early extraction can also dispatch with incomplete streamed arguments.
For the version question: I checked the
v2.6.0source tag. The missing partial-event guard and the earlybreakare still present, and the progressive-SSE aggregator still emits complete FC data only in its non-partial close result.The same code path is also present in
v2.6.2and currentmain, so the issue appears to remain applicable at source level.I have not yet run the complete reproduction script against an installed 2.6.0 environment, so I would distinguish this from a confirmed runtime rerun.
The narrow fix I had in mind is to ignore task-delegation FCs on
partial=Trueevents and dispatch only from the final non-partial aggregate.The regression test should verify:
I’m also happy to review or run the reproduction against the proposed patch.
a2105z commentedon Aug 6, 2026
Hello @llalitkumarrr — sharing the analysis you asked for (and answering the 2.6.0 question). Aligning with @bferanmi806-sketch’s earlier write-up; happy to coordinate so we don’t duplicate the patch.
Version check (2.0.0 / 2.6.0 / current
main)I re-ran kenwilly’s progressive-SSE-shaped repro against current
main/ installedgoogle-adk==2.6.2(editable checkout).Producer defect still reproduces on current
main. After one turn:[]['fc-dispatch-1']Dropping function responses with no matching function call: ['fc-dispatch-1']So the wrapper still breaks on the
partial=Truechunk before the Runner-persisted aggregate is yielded.Consumption symptom changed after 2.0.0. On 2.0.0 the next contents build raised:
ValueError: No function call event found for function responses ids: {...}On current
main,contents.pycan prune orphaned FRs (Dropping function responses…, from73e86254) instead of always hard-failing on every path. That softens the “permanently poisoned session” raise, but it does not fix the producer: history is still missing the task FC, dispatch can still fire from a truncated partial chunk, and the prune path is a warning / history rewrite rather than correct FC/FR pairing.Source-level, the missing
event.partialgate and the earlybreakare still present on thev2.6.0tag,v2.6.2, and currentmain.Failure path (pinpoint)
Progressive SSE emission (
StreamingResponseAggregatorunderPROGRESSIVE_SSE_STREAMING, default-on):partial=True, including chunks that already name a task-delegationFunctionCall(args may still be streaming / incomplete).close()/ final yield).Persistence rule (
runners.py):event.partial is not True.Chat wrapper extraction has no partial gate —
src/google/adk/workflow/_llm_agent_wrapper.py:run_llm_agent_as_node(~L470–487 on currentmain):Resulting session shape (smoking gun):
main).Second defect on the same path: if the first partial chunk has incomplete streamed args, the specialist can be dispatched with truncated / empty input even when a later aggregate would have carried the full args.
Contrast with existing ADK pattern
process_llm_agent_outputin the same module already skips partial events:Task-delegation extraction should follow that same rule: never act on
partial=Truemodel chunks.What needs to be fixed
Narrow, surgical fix (producer):
In
_extract_task_delegation_fcs(or immediately at the call site before dispatch), return no FCs whenevent.partialis true:That forces dispatch +
breakto wait for the non-partial aggregate that the Runner actually persists, with complete args.Out of scope for this bug (related but separate):
break, different missing drain.contents.pyprune) — consumer safety net; does not replace the partial gate.Regression checks the fix should prove
Status / coordination
I’ve confirmed the above on current
main, implemented the partial gate + unit/e2e coverage on branchfix/progressive-sse-task-fc-partial-gate, and can open a PR whenever that’s useful.Since you’re assigned (@llalitkumarrr): happy to open the PR, hand the patch over, or just review/test yours — whatever avoids double work. @bferanmi806-sketch’s direction matches this; I just wanted the version re-run + exact call-site pin for the clarification label.
llalitkumarrr commentedon Aug 7, 2026
Hello @a2105z and @bferanmi806-sketch,
Thank you for sharing your input. We looked into this further based on your suggestions. Implementing a partial gate in
_extract_task_delegation_fcslooks like a clean way to resolve this issue.@a2105z, your proposed implementation seems to address this and we verified with the reproduction code provided above that it should fix the issue. Please feel free to open a pull request. We will review it and it can be merged once approved.
a2105z commentedon Aug 7, 2026
Ok thank you, I will immediately get started and open a PR by EOD.
fix(workflow): Skip task dispatch on progressive-SSE partial chunks
a2105z commentedon Aug 7, 2026
Hello @llalitkumarrr — thanks for the green light and for verifying the partial-gate approach against the repro.
PR is up here: #6638
It implements the
event.partialgate in_extract_task_delegation_fcs, plus unit + progressive-SSE e2e coverage (complete args, single dispatch, matched FC/FR ids, usable next turn).cc @bferanmi806-sketch @kenwilly — happy for eyes/review if useful; tried to keep this aligned with the analysis already shared on this thread.
fix(workflow): do not act on function calls in partial events