Skip to content

[LiteLLM] role='tool_responses' for Gemma 4 breaks tool calling on Vertex AI Model Garden MaaS (400 INVALID_ARGUMENT) #7977

Closed

Description

@rladeira

🔴 Required Information

Describe the Bug:
_is_gemma4_model in google/adk/models/lite_llm.py causes ADK to emit tool results with role="tool_responses" whenever the model string matches gemma-?4, except when the effective provider is in _STANDARD_TOOL_ROLE_PROVIDERS ({"openai", "azure", "lm_studio"}, added in 7d190e6e for #6482).

When calling Gemma 4 on Vertex AI Model Garden MaaS via LiteLLM (model="vertex_ai/google/gemma-4-26b-a4b-it-maas"), LiteLLM routes the request to Vertex AI's OpenAI-compatible chat-completions endpoint (.../endpoints/openapi/chat/completions), which strictly validates OpenAI message roles. Because "vertex_ai" is not in _STANDARD_TOOL_ROLE_PROVIDERS, ADK sends role="tool_responses" and Vertex AI rejects every tool-result turn with HTTP 400 INVALID_ARGUMENT:

Unrecognized 'role' field for a message; found: 'tool_responses'.

Steps to Reproduce:

  1. Configure ADC with a GCP project that has Vertex AI API enabled.
  2. Instantiate LiteLlm(model="vertex_ai/google/gemma-4-26b-a4b-it-maas", vertex_project="<gcp-project>", vertex_location="global").
  3. Send a tool-calling round trip (user prompt -> model FunctionCall -> user FunctionResponse).
  4. Observe litellm.BadRequestError (400 INVALID_ARGUMENT).

Expected Behavior:
ADK sends tool results with role="tool" for vertex_ai Gemma 4 models, and Vertex AI returns a 200 OK completion grounded in the tool result.

Observed Behavior:
ADK sends role="tool_responses" and Vertex AI returns:

{
  "error": {
    "code": 400,
    "message": "Unrecognized 'role' field for a message; found: 'tool_responses'.",
    "status": "INVALID_ARGUMENT"
  }
}

Environment Details:

  • ADK Library Version (pip show google-adk): 2.13.0 and main (76627586)
  • Desktop OS: Linux
  • Python Version (python -V): Python 3.12

Model Information:

  • Are you using LiteLLM: Yes
  • Which model is being used: vertex_ai/google/gemma-4-26b-a4b-it-maas

🟡 Optional Information

Regression:
Introduced when gemma-?4 matching was broadened in 49301558 (#6334 / #6340); 7d190e6e (#6482 / #7042) excluded openai, azure, and lm_studio, but did not include vertex_ai.

Logs:

roles sent: ['user', 'assistant', 'tool_responses']
EXCEPTION: BadRequestError litellm.BadRequestError: Vertex_aiException BadRequestError - [{
  "error": {
    "code": 400,
    "message": "Unrecognized 'role' field for a message; found: 'tool_responses'.",
    "status": "INVALID_ARGUMENT"
  }
}]

Additional Context:
We also verified a self-deployed Vertex AI Model Garden Gemma 4 vLLM endpoint (google/gemma4@gemma-4-e2b-it via vertex_ai/gemma/... :predict with @requestFormat="chatCompletions" and dedicated /chat/completions): it accepts role="tool" with 200 OK and produces the expected final answer without re-calling the tool. Adding "vertex_ai" to _STANDARD_TOOL_ROLE_PROVIDERS therefore fixes Vertex MaaS without regressing self-deployed Vertex Model Garden endpoints, while preserving role="tool_responses" for ollama/... and hosted_vllm/....

Minimal Reproduction Code:

import asyncio
from google.adk.models.lite_llm import LiteLlm
from google.adk.models.llm_request import LlmRequest
from google.genai import types


async def main() -> None:
  model = "vertex_ai/google/gemma-4-26b-a4b-it-maas"
  llm = LiteLlm(
      model=model,
      vertex_project="<gcp-project>",
      vertex_location="global",
  )
  weather = types.FunctionDeclaration(
      name="get_weather",
      description="Returns the weather for a city.",
      parameters=types.Schema(
          type=types.Type.OBJECT,
          properties={"city": types.Schema(type=types.Type.STRING)},
          required=["city"],
      ),
  )
  req = LlmRequest(
      model=model,
      contents=[
          types.Content(
              role="user",
              parts=[types.Part(text="What is the weather in Paris?")],
          ),
          types.Content(
              role="model",
              parts=[
                  types.Part(
                      function_call=types.FunctionCall(
                          id="call_1",
                          name="get_weather",
                          args={"city": "Paris"},
                      )
                  )
              ],
          ),
          types.Content(
              role="user",
              parts=[
                  types.Part(
                      function_response=types.FunctionResponse(
                          id="call_1",
                          name="get_weather",
                          response={"result": "sunny, 21 C"},
                      )
                  )
              ],
          ),
      ],
      config=types.GenerateContentConfig(
          tools=[types.Tool(function_declarations=[weather])],
          max_output_tokens=64,
      ),
  )
  async for r in llm.generate_content_async(req):
    print(r.finish_reason, r.content)


asyncio.run(main())

How often has this issue occurred?:

  • Always (100%)

Activity

  1. added a commit that references this issue on Nov 3, 2026
    d5cace7
  2. surajksharma07 commented on Nov 4, 2026

    @surajksharma07
    Collaborator

    @rladeira Reproduced this on 2.13.0, and vertex_ai is indeed missing from _STANDARD_TOOL_ROLE_PROVIDERS, so Gemma 4 on MaaS still gets role="tool_responses".

    Until #7978 lands, a workaround you could try is extending that set at startup: ll._STANDARD_TOOL_ROLE_PROVIDERS |= {"vertex_ai"} (with import google.adk.models.lite_llm as ll). It fixed the round trip in my local test.

    Could you confirm it works on your side too, including the self-deployed vLLM endpoint and longer multi-turn tool flows? Also, which LiteLLM version are you on?

    Once that’s tested on your end we’ll move the review forward on #7978.

  3. added a commit that references this issue on Nov 6, 2026
    b33b43e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions