Skip to content

SetModelResponseTool: schema validation failure terminates the tool flow instead of returning feedback #1430

Closed

Description

@BlueCatPro

🔴 Required Information

Describe the Bug

SetModelResponseTool currently propagates output-schema validation failures as tool execution errors.

In the current Java implementation, SchemaUtils.validateMapOnSchema(...) is executed inside Single.fromCallable(...). When validation throws an IllegalArgumentException, the Single terminates with an error instead of returning validation feedback as a tool response.

Under the default tool execution flow, no validation-feedback FunctionResponse is returned to the model. As a result, the model cannot correct the arguments and call set_model_response again.

ADK Python now returns structured-output validation failures to the model as tool feedback and records successfully validated output separately. This appears to be a Java/Python parity gap.

There is also a related finalization concern: OutputSchema.getStructuredModelResponse(...) currently promotes a function response named set_model_response to the final structured response. Validation feedback must therefore be distinguishable from successfully validated output.

Steps to Reproduce

Create an output schema containing a required field:

Schema outputSchema =
    Schema.builder()
        .type("OBJECT")
        .properties(
            ImmutableMap.of(
                "field1",
                Schema.builder().type("STRING").build()))
        .required(ImmutableList.of("field1"))
        .build();

Create and invoke SetModelResponseTool without the required field:

SetModelResponseTool tool =
    new SetModelResponseTool(outputSchema);

Map<String, Object> invalidArgs = ImmutableMap.of();

tool.runAsync(invalidArgs, null).blockingGet();

The call terminates with:

java.lang.IllegalArgumentException:
Output args does not contain required field1

Expected Behavior

For parity with ADK Python, an output-schema validation failure from the internal set_model_response tool should be returned to the model as validation feedback:

LLM calls set_model_response
        ↓
schema validation fails
        ↓
validation feedback is returned to the model
        ↓
LLM corrects the arguments
        ↓
LLM calls set_model_response again
        ↓
validated result becomes the final response

An invalid set_model_response call must not become the final structured model response. Only successfully validated output should be finalized.

The exact Java implementation does not need to identify validation feedback solely through an "error" field, since a valid user-defined output schema could legitimately contain a field with that name.

Observed Behavior

SetModelResponseTool currently performs:

return Single.fromCallable(
    () -> {
      SchemaUtils.validateMapOnSchema(
          args, outputSchema, /* isInput= */ false);
      return args;
    });

The validation exception therefore becomes an RxJava error signal and propagates through the tool flow.

The existing upstream SetModelResponseToolTest explicitly verifies this behavior using assertThrows(...), confirming that exception propagation is the current expected Java behavior.

Additionally, OutputSchema.getStructuredModelResponse(...) currently identifies the final structured response using the set_model_response function name. A change that only converts the exception into a Python-style feedback map could therefore incorrectly promote that feedback to the final response unless validated output is tracked or distinguished separately.

Environment Details

  • ADK Java version: commit 2b87d65d9704a61ff4668b8c9482a79fef9fe0d4
  • Java: Microsoft OpenJDK 17.0.15
  • OS: Windows 11
  • Test framework: JUnit 4

Model Information

  • Model: N/A
  • The issue is reproducible deterministically at the tool level without a live model

🟡 Optional Information

Regression

No known Java regression.

This appears to be a behavioral parity gap following the newer structured-output validation recovery behavior in ADK Python.

Logs

IllegalArgumentException: Output args does not contain required field1
    at com.google.adk.SchemaUtils.validateMapOnSchema(SchemaUtils.java:124)
    at com.google.adk.tools.SetModelResponseTool.lambda$runAsync$0(
        SetModelResponseTool.java:63)

Additional Context

The corresponding Python behavior is present on adk-python main in:

google/adk-python@b5b27cb

The change originated from:

google/adk-python#6368

The Python implementation catches structured-output validation failures and returns validation feedback to the model. Successfully validated output is recorded separately and is the only result promoted to the final structured response.

Relevant Java files:

  • core/src/main/java/com/google/adk/tools/SetModelResponseTool.java
  • core/src/main/java/com/google/adk/flows/llmflows/OutputSchema.java
  • core/src/test/java/com/google/adk/tools/SetModelResponseToolTest.java

The ADK Java contribution guide states that ADK Java aims to align with ADK Python and leans on ADK Python as the source of truth for behavioral validation:

https://github.com/google/adk-java/blob/2b87d65d9704a61ff4668b8c9482a79fef9fe0d4/CONTRIBUTING.md#alignment-with-adk-python

Would maintainers agree that ADK Java should align with the current Python recovery behavior here?

How often has this issue occurred?

Always (100%) when SetModelResponseTool receives arguments that fail output-schema validation.

Activity

  1. self-assigned this
    on Aug 14, 2026
  2. added theissue type on Aug 14, 2026
  3. BlueCatPro commented on Aug 14, 2026

    @BlueCatPro
    ContributorAuthor

    I’d like to work on this if no one else is already handling it. Could you confirm whether Java should match the current Python behavior here?

  4. hemasekhar-p commented on Aug 14, 2026

    @hemasekhar-p
    Contributor

    Hi @BlueCatPro, thank you for reporting this issue and for providing such a detailed breakdown. I am able to reproduce this behavior with your inputs. We would be very happy to have you contribute a fix to align the Java implementation with the current ADK Python behavior. Please feel free to proceed with a pull request and let us know once it is ready for review?

  5. added
    waiting on reporterWaiting for reaction by reporter. Failing that, maintainers will eventually closed it as stale.
    on Aug 14, 2026
  6. hemasekhar-p commented on Aug 17, 2026

    @hemasekhar-p
    Contributor

    @BlueCatPro, thank you for your contribution. Both this issue and your corresponding pull request are currently under review by our team. We will keep you posted if we need any further information or updates. Thank you!

  7. added and removed
    waiting on reporterWaiting for reaction by reporter. Failing that, maintainers will eventually closed it as stale.
    on Aug 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions