You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
test(cli): add coverage for adk conformance test command options #6730
The adk conformance test CLI command supports several options and input combinations, but these CLI behaviors should have explicit automated coverage to prevent regressions.
The command currently supports:
Running conformance tests with no path, defaulting to the tests directory.
Running tests against one or more explicitly provided directories.
Selecting replay or live mode.
Generating a Markdown report with --generate_report.
Specifying a custom report directory with --report_dir.
Selecting a streaming mode with --streaming-mode.
This issue proposes adding focused unit tests for the CLI argument handling and dispatch behavior without requiring real LLM execution.
Proposed Changes
Add pytest coverage for the adk conformance test CLI command covering:
No path provided → defaults to the tests directory.
A single test directory is correctly passed to the conformance runner.
Multiple test directories are accepted.
--mode replay is correctly propagated.
--mode live is correctly propagated.
--generate_report is correctly propagated.
--report_dir <path> is correctly propagated.
--streaming-mode values are parsed and propagated correctly.
Invalid --mode values are rejected by Click.
Invalid --streaming-mode values are rejected by Click.
Where appropriate, the conformance runner should be mocked so these tests remain fast and deterministic.
Expected Outcome
The CLI's argument parsing and dispatch behavior should have regression coverage, making future changes to the conformance command less likely to introduce unnoticed CLI regressions.
Testing
Run the relevant CLI unit tests with pytest and ensure the existing ADK test suite continues to pass.
Scope
This should be limited to automated test coverage for the existing CLI behavior. No changes to the underlying conformance execution logic are required.
Picking this one up now — opening a PR shortly. Flagging it here so nobody duplicates the work; if someone is already on it, say so and I will drop mine.
hi maintainer i do like to work on this issue, as i have looked through the issue and planed to , reproduce the failure when one of the provided toolset fails during tool discovery , and also add a regression test covering the partial failure case.
if this approach aligns with the expected behavior , can i plz be a part of it and also can you assign me this issue .
and would be making a pr soon on this as well
Hi @YASHcode-IIITV, thanks for raising a feature request. If you already have a solution in mind, please feel free to open a PR. our team would be happy to review it.
landed in #6874 as 7d9f2d0, all ten cases are in tests/unittests/cli/utils/test_cli_tools_click.py now and pytest -k conformance is 11 passed on main. reopen if a specific case is still uncovered.
Activity
chelsealong commentedon Aug 15, 2026
Picking this one up now — opening a PR shortly. Flagging it here so nobody duplicates the work; if someone is already on it, say so and I will drop mine.
YASHcode-IIITV commentedon Aug 15, 2026
hi maintainer i do like to work on this issue, as i have looked through the issue and planed to , reproduce the failure when one of the provided toolset fails during tool discovery , and also add a regression test covering the partial failure case.
if this approach aligns with the expected behavior , can i plz be a part of it and also can you assign me this issue .
and would be making a pr soon on this as well
YASHcode-IIITV commentedon Aug 15, 2026
@chelsealong i would like to work on this issue
sanketpatil06 commentedon Aug 18, 2026
Hi @YASHcode-IIITV, thanks for raising a feature request. If you already have a solution in mind, please feel free to open a PR. our team would be happy to review it.
YASHcode-IIITV commentedon Aug 18, 2026
@sanketpatil06 thx , I will be posting the pr shortly .
YASHcode-IIITV commentedon Aug 19, 2026
@sanketpatil06 i have opened the pr and would be grateful to get reviewed as well
GWeale commentedon Aug 28, 2026
landed in #6874 as 7d9f2d0, all ten cases are in tests/unittests/cli/utils/test_cli_tools_click.py now and pytest -k conformance is 11 passed on main. reopen if a specific case is still uncovered.