Skip to content

BIDI Input and Output Transcriptions Added as NULL Events to DataBaseSessionService #3172

Closed

Description

@GrahamGGreig

Was previously using ADK 1.11.0 and wanted to upgrade to make use of the changes seen in commit
ab69ef8, ideally reducing latency with the ability to prevent audio from being uploaded to our SQL DB. However, upon upgrading to 1.16.0, I have seen all messages removed from the SQL DB whether or not the RunConfig value save_live_audio is set to True. This is despite the description seen here where it is expected that audio data will at least be saved to the session.

Image

As well, all of the transcriptions are now missing from the database making it impossible to resume a conversation at a later date. From reading the commit, it looks like transcriptions have been moved out of contents and I can see transcriptions are taking place so that part is working as expected:

Event: content=None grounding_metadata=None partial=None turn_complete=None finish_reason=None error_code=None error_message=None interrupted=None custom_metadata=None usage_metadata=None live_session_resumption_update=None input_transcription=None output_transcription=Transcription( text='Hey' ) avg_logprobs=None logprobs_result=None cache_metadata=None citation_metadata=None invocation_id='e-a0b8e90f-4c9a-4578-a631-5bde953cad6e' author='General_Inquiries_Agent' actions=EventActions(skip_summarization=None, state_delta={}, artifact_delta={}, transfer_to_agent=None, escalate=None, requested_auth_configs={}, requested_tool_confirmations={}, compaction=None, end_of_agent=None, agent_state=None) long_running_tool_ids=None branch=None id='c785ba05-4849-438f-881e-b2235372c627' timestamp=1760474279.826785

But, given that the DataBaseSessionService does not have fields output_transcription and input_transcription. They cannot be seen even if they are being uploaded. It looks like all references to upload to the session database have been removed in the aforementioned commit - ex:

src/google/adk/flows/llm_flows/audio_cache_manager.py

Image

and

‎src/google/adk/flows/llm_flows/transcription_manager.py

Image

I did some digging and found that the transcription events are being saved to the session service in:

src/google/adk/runners.py in _exec_with_plugin(). But, as expected, because they are not being converted to contents they appear as null events in the session DB.

So, my question is: can we add a point where transcriptions are converted back to "contents" such that they are accessible in the session history? Or, is the future plan to have transcriptions only be uploaded as their own columns in the session service (note, I can see this causing some issues if switching back and forth between TEXT and AUDIO modes as TEXT mode is added as contents still).

Also, on another note, not that I need this, but it looks like audio data is only saved if the there is an artifact service:

src/adk/flows/llm_flows/audio_cache_manager.py

Image

I imagine it would be a nice feature to still save to session if desired and not using an artifact service.

Desktop

  • Testing Locally on Windows 11
  • Python version: 3.13.7
  • ADK version: Seen in code base from > 1.14.1

Model Information:

  • Are you using LiteLLM: No
  • Which model is being used: gemini-2.0-flash-live-preview-04-09

Activity

  1. added
    services[Component] This issue is related to runtime services, e.g. sessions, memory, artifacts, etc
    on Oct 14, 2025
  2. added theissue type on Oct 14, 2025
  3. GrahamGGreig commented on Oct 22, 2025

    @GrahamGGreig
    Author

    Also, noticed that with some gemini models, audio is still being uploaded. This is because they are returning a mimetype with a parameter like "audio/pcm;24000"

    The lines in src/google/adk/flows/llm_flows/contents.py - _is_live_model_audio_content() should be changed from:

      for part in event.content.parts:
        if part.inline_data and part.inline_data.mime_type == 'audio/pcm':
          return True
        if part.file_data and part.file_data.mime_type == 'audio/pcm':
          return True
      return False
    

    to something like:

      for part in event.content.parts:
        if part.inline_data and part.inline_data.mime_type.split(';')[0] == 'audio/pcm':
          return True
        if part.file_data and part.inline_data.mime_type.split(';')[0] == 'audio/pcm':
          return True
      return False
    
  4. added this to the 10/27 ADK Week milestone on Oct 27, 2025
  5. hangfei commented on Nov 9, 2025

    @hangfei
    Collaborator

    Also, noticed that with some gemini models, audio is still being uploaded. This is because they are returning a mimetype with a parameter like "audio/pcm;24000"

    The lines in src/google/adk/flows/llm_flows/contents.py - _is_live_model_audio_content() should be changed from:

      for part in event.content.parts:
        if part.inline_data and part.inline_data.mime_type == 'audio/pcm':
          return True
        if part.file_data and part.file_data.mime_type == 'audio/pcm':
          return True
      return False
    

    to something like:

      for part in event.content.parts:
        if part.inline_data and part.inline_data.mime_type.split(';')[0] == 'audio/pcm':
          return True
        if part.file_data and part.inline_data.mime_type.split(';')[0] == 'audio/pcm':
          return True
      return False
    

    for part in event.content.parts:

    Thanks for reporting. We have changed this to "startswith" so the problem is addressed. Please give it try.

  6. hangfei commented on Nov 9, 2025

    @hangfei
    Collaborator

    Regarding your original question.

    Yes, the plan is to save the data into transcription so it's more clear what's the data(transcription). The support for database session is not there yet. I am working on it now.

    Regarding the artifactory, the audio is generally pretty large. So we think it might overload the database so we didn't provide that option. Have you tested if this is not a concern for you and why artifactory is not a better? Like to learn more about your actual use case to help us perfect the design.

  7. added
    live[Component] This issue is related to live, voice and video chat
    and removed
    services[Component] This issue is related to runtime services, e.g. sessions, memory, artifacts, etc
    on Nov 11, 2025
  8. GrahamGGreig commented on Nov 12, 2025

    @GrahamGGreig
    Author

    Hi @hangfei I agree with it being too large and after further testing my previous claim on having the option for session storage does not make sense. Removing audio from the session service makes the bidi much less likely to crash from what I am assuming is context overflow.

  9. hangfei commented on Nov 14, 2025

    @hangfei
    Collaborator

    Sg.

    I will keep this open for the DataBaseSessionService.

  10. added a commit that references this issue on Nov 19, 2025
    3ad30a5
  11. added a commit that references this issue on Nov 27, 2025
    e69c757
  12. added a commit that references this issue on Aug 11, 2026
    ccbc6a4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

live[Component] This issue is related to live, voice and video chat

Type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions