Skip to content

LiteLLM wrapper doesn't support GCS file URIs (gs://) when using proxy #3131

Closed

Description

@antoremin

Describe the bug
When using ADK's LiteLlm wrapper with a LiteLLM proxy and GCS file URIs (e.g., gs://bucket/file.pdf), the file URI is silently dropped and not passed to the model.

This works correctly with:

  • Native Gemini + GCS URIs
  • LiteLLM proxy called directly with OpenAI format
  • ADK + LiteLLM proxy + inline data (Part.from_bytes())

But fails when combining:

  • ❌ ADK LiteLlm wrapper + LiteLLM proxy mode + Part.from_uri(file_uri="gs://...")

The file reference is silently dropped in the _get_content() function because there's no handler for part.file_data.file_uri.

To Reproduce

import os
import asyncio
from google.adk.agents import LlmAgent
from google.adk.runners import Runner
from google.adk.sessions import InMemorySessionService
from google.adk.models.lite_llm import LiteLlm
from google.genai import types
import litellm

# Setup (replace with your config)
os.environ["GOOGLE_CLOUD_PROJECT"] = "your-project"
os.environ["GOOGLE_CLOUD_LOCATION"] = "us-central1"
os.environ["LITELLM_PROXY_API_KEY"] = "your-key"
os.environ["LITELLM_PROXY_API_BASE"] = "http://localhost:4000"

# Enable proxy mode
litellm.use_litellm_proxy = True

# Create agent with LiteLLM proxy
agent = LlmAgent(
    name="test_agent",
    model=LiteLlm(model="vertex-gemini"),  # Proxy model alias
    instruction="Summarize PDFs."
)

session_service = InMemorySessionService()
runner = Runner(agent=agent, app_name="test", session_service=session_service)

async def test():
    await session_service.create_session(app_name="test", user_id="u", session_id="s", state={})
    
    # Create message with GCS file URI
    pdf_part = types.Part.from_uri(
        file_uri="gs://your-bucket/document.pdf",
        mime_type="application/pdf"
    )
    msg = types.Content(
        role="user",
        parts=[types.Part(text="Summarize this."), pdf_part]
    )
    
    async for event in runner.run_async(user_id="u", session_id="s", new_message=msg):
        if hasattr(event, 'content') and event.content and event.content.parts:
            for part in event.content.parts:
                if hasattr(part, 'text') and part.text:
                    print(part.text)
                    break

asyncio.run(test())

Expected behavior
The model should receive the GCS file URI and process the PDF content, producing a summary like:

"This document discusses [actual content from the PDF]..."

Desktop (please complete the following information):

  • OS: macOS 24.6.0
  • Python version: Python 3.12
  • ADK version: 1.15.1
  • LiteLLM version: 1.77.7

Model Information:

  • Are you using LiteLLM: Yes
  • Which model is being used: gemini-2.5-pro
  • Proxy: LiteLLM proxy with vertex_ai/gemini-2.5-pro configured

Additional Information.
In google/adk/models/lite_llm.py, the _get_content() function (lines ~226-293) converts ADK's types.Part to LiteLLM format but only handles:

  • part.text → text content
  • part.inline_data → base64 encoded data (images, videos, PDFs)

Missing: Handler for part.file_data.file_uri (GCS URIs, HTTP URLs, etc.)

When Part.from_uri("gs://...") is used, it creates a Part with file_data.file_uri populated, but _get_content() has no code path to handle it, so the file reference is silently dropped from the message sent to the proxy.

Proposed Fix

Add a handler for part.file_data in the _get_content() function:

def _get_content(parts: Iterable[types.Part]) -> Union[OpenAIMessageContent, str]:
    content_objects = []
    for part in parts:
        if part.text:
            # ... existing text handling ...
        
        elif part.inline_data and part.inline_data.data and part.inline_data.mime_type:
            # ... existing inline_data handling ...
        
        # 🆕 ADD THIS: Handle file_data with URIs (GCS, HTTP URLs)
        elif part.file_data and part.file_data.file_uri:
            content_objects.append({
                "type": "file",
                "file": {
                    "file_id": part.file_data.file_uri,  # gs://... or https://...
                    "format": part.file_data.mime_type
                }
            })
    
    return content_objects

Activity

  1. added
    models[Component] This issue is related to model support
    on Oct 9, 2025
  2. added theissue type on Oct 9, 2025
  3. GWeale commented on Oct 9, 2025

    @GWeale
    Collaborator

    Hi thanks for pointing this out! Have you tested in your use case if you add this handler that it works in your use case?

  4. antoremin commented on Oct 9, 2025

    @antoremin
    Author

    Hi thanks for pointing this out! Have you tested in your use case if you add this handler that it works in your use case?

    Yup! Adding the handler fixed the issue.

  5. GWeale commented on Oct 10, 2025

    @GWeale
    Collaborator

    Thank you again for pointing this added a fix for this, please re open if there are more issues!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

models[Component] This issue is related to model support

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions