Repository navigation
DatabaseSessionService error : Database Column Length Exceeded for Error Messages #3351
Description
Describe the bug
When using DatabaseSessionService with PostgreSQL as a persistent database for session storage, error messages that exceed the database column length limit (VARCHAR(1024)) cause a StringDataRightTruncation error. Instead of gracefully handling this error, the entire agent execution crashes, preventing proper error reporting and potentially leaving the session in an inconsistent state.
To Reproduce
Steps to reproduce the behavior:
- Set up a PostgreSQL database for session persistence
- Configure
DatabaseSessionServicewith the PostgreSQL connection:
from google.adk.sessions import DatabaseSessionService
from google.adk import Runner
# Using PostgreSQL for persistent session storage
session_service = DatabaseSessionService(
db_url="postgresql://user:password@host:5432/database"
)
# Agent configured with persistent session service
runner = Runner(
agent=agent,
session_service=session_service
)- Run an agent that involves complex data processing (e.g.,
plot_agentwith large datasets) - Trigger a
MALFORMED_FUNCTION_CALLerror with a large payload - Observe the database error and agent crash with this error:
Error: Agent API error: {"error": "(psycopg2.errors.StringDataRightTruncation) value too long for type character varying(1024)\n\n[SQL: INSERT INTO events (...) VALUES (...)]\n[parameters: {..., 'error_message': 'Malformed function call: print(default_api.plot_agent(request='''Create a line graph with this data: [{'transaction_day': '2024-10-24', 'dail ... (40424 characters truncated) ...)', 'interrupted': None}]\n(Background on this error at: https://sqlalche.me/e/20/9h9h)", "timestamp": "2025-10-29T13:00:34.724692", "request_id": "req_1761767910262", "user_id": "2", "session_id": "2025_10_29_19_37_15_A9AA"}
Expected behavior
The agent should handle database insertion errors gracefully and continue operation, rather than crashing when error messages exceed the column size limit. Error logging failures should not cause agent execution failures.
Screenshots
N/A
Desktop (please complete the following information):
- OS: Linux/Ubuntu
- Python version: 3.11+
- ADK version: 1.15.1
Model Information:
- Are you using LiteLLM: No
- Which model is being used: gemini-2.5-flash ( google-adk running in GKE cluster )
Additional context
Root Cause Analysis:
- The
eventstable in the PostgreSQL database used byDatabaseSessionServicedefineserror_messageasVARCHAR(1024) - When using data-intensive agents (especially with
plot_agent), error messages can exceed 40,000 characters DatabaseSessionServicedoes not handle PostgreSQL insertion errors, so the column length violation crashes the agent execution
Current Impact:
- Affects all production deployments using
DatabaseSessionServicewith PostgreSQL - Particularly impacts data-intensive and visualization agents
- Results in agent crashes and potential session state corruption in PostgreSQL
- 97%+ of error context is lost due to truncation
Technical Details:
- Failed SQL Operation:
INSERT INTO events (
id, app_name, user_id, session_id, invocation_id, author,
actions, long_running_tool_ids_json, branch, timestamp,
content, grounding_metadata, custom_metadata,
partial, turn_complete, error_code, error_message, interrupted
) VALUES (...)- Problematic Field:
error_message(VARCHAR(1024)) - Actual Length in Example: ~40,424 characters
- Triggers: plot_agent, large data payloads, MALFORMED_FUNCTION_CALL errors
Temporary Workaround:
Until fixed, users can manually alter their PostgreSQL database:
ALTER TABLE events ALTER COLUMN error_message TYPE TEXT;Proposed Solutions:
- Add error handling in
DatabaseSessionServiceto prevent crashes when PostgreSQL logging fails - Implement smart truncation for oversized error messages
- Update the PostgreSQL schema to use
TEXTtype instead ofVARCHAR(1024) - Add configuration options for max error message length
Dependencies:
- SQLAlchemy 2.0.43
- asyncpg 0.30.0 (async PostgreSQL driver)
- psycopg2-binary 2.9.11 (sync PostgreSQL driver)
- fastapi==0.118.0
- pydantic==2.11.9
- uvicorn==0.37.0
Related Files:
google/adk/sessions/database_session_service.py
Activity
prasadskarmarkar commentedon Nov 1, 2025
** implemented the following temporary fix in our Agent execution environment , as a workaround **
surajksharma07 commentedon Nov 3, 2025
@prasadskarmarkar the error_message column at line 270 in database_session_service.py is defined as String(1024) which is too small for real-world error messages, especially MALFORMED_FUNCTION_CALL errors with large data payloads.
The fix is straightforward: change to
Texttype (like other large fields in the schema) and add error handling around the commit to prevent crashes.Thanks for the detailed analysis and for applying the workaround - this helps production deployments stay stable.
prasadskarmarkar commentedon Nov 7, 2025
how are we planning to fix this bug ?
Here is my analysis :
In this file src/google/adk/sessions/database_session_service.py
The SQLAlchemy create_all() method (called on line 454: Base.metadata.create_all(self.db_engine)) only creates tables and columns that don't exist. It doesn't modify existing column definitions
As per SQLAlchemy documentation :
SQLAlchemy's create_all() method:
This means that if we change a column type from String(1024) to Text in the database_session_service.py file , and the table already exists in the database, SQLAlchemy will NOT alter the column type in the database. It will only use the new definition when creating new tables.
Just changing the datatype from String(1024) to TEXT in your code will not alter the column in the database if an ADK agent with DatabaseSessionService is already running in a production environment.
Since only a small percentage of deployments might have this issue, we recommend:
Let me know your thoughts.
[-]DatabaseSessionService error : Database Column Length Exceeded for Error Messages in Sequential Agent Workflows[/-][+]DatabaseSessionService error : Database Column Length Exceeded for Error Messages[/+]mattmcc-attest commentedon Nov 20, 2025
also see this quite a lot
surajksharma07 commentedon Nov 20, 2025
Thanks @mattmcc-attest for confirming this is widespread.
@prasadskarmarkar you're absolutely right about the SQLAlchemy create_all() limitation - just changing the code to Text won't modify existing columns.
We are looking more into your recommended approach: (1) change schema to Text in code,
(2) provide an Alembic migration script users can run,
(3) document in release notes that existing deployments need manual migration, and
(4) add startup validation to warn if schema doesn't match.
This is the right way to handle it while respecting that only a subset of deployments are affected.
surajksharma07 commentedon Nov 20, 2025
@DeanChensj Please have a look into it.
DeanChensj commentedon Nov 27, 2025
Hi @prasadskarmarkar , thanks for reporting this issue, in 0094eea we migrated the db schema to use json payload to represent the event object. As a side effect this issue should also be fixed. You should be able to migrate to a new DB with
adk migrate session, in place db migration is not supported thoughprasadskarmarkar commentedon Nov 28, 2025
@DeanChensj
The fix looks good.
A note regarding deployment:
The adk migrate session command cannot be executed from within our production GKE application pods, unlike in local environments.
Production GKE Considerations
Most production pods do not include the ADK CLI and do not allow ad-hoc command execution.
The schema migration must be performed once per environment before deploying the updated ADK version.
Required Approach
We will need to run the migration using a one-off Kubernetes Job in the production GKE cluster with the appropriate DB credentials.
This does introduce additional operational overhead for teams running services on their organization’s custom GKE cluster, good to document the same.
mattmcc-attest commentedon Dec 3, 2025
@DeanChensj what would you recommend we do for in production databases where we want to update in the schema in place?
DeanChensj commentedon Dec 3, 2025
Hi @mattmcc-attest , what about you migrate from DB A to DB B, and then migrate back to DB A?
prasadskarmarkar commentedon Dec 3, 2025
I wrote a much cleaner implementation of DB migration so that the code ( migration manager, takes care of the db migration as a part of code execution , no additional adk commands or scripts needed ) but did not create a pull request as the schema totally changed.
but looks like you have reverted these changes now : 9d918d4#diff-529a63417090757c651b62a1f94b04f86b27413415b55a7304637e3f10f66b25
does this mean the bug is still open ?
i will test my changes with the latest code and share my pull request for this bug fix. let me know your thoughts...