Repository navigation
FEAT: Add A2ATarget for Agent-to-Agent protocol communication - #2771
Richard Lundeen (richlundeen) merged 7 commits into
Conversation
Introduces A2ATarget, enabling PyRIT to red-team multi-agent systems and services communicating over the Agent-to-Agent (A2A) protocol. Key features: - Dual-spec protocol negotiation: supports A2A v0.3 (message/send) with automatic fallback to v0.2 (tasks/send) on JSON-RPC -32601. - Multi-turn conversation task continuity: tracks and preserves active server task IDs per conversation session. - Robust response and refusal handling: captures JSON-RPC error refusals as conversational evidence rather than raising unhandled exceptions. - Agent card discovery: provides get_agent_card_async() to inspect declared agent capabilities at standard discovery paths. - Auth support: handles Bearer token and custom API-Key headers. - Comprehensive unit test suite covering v0.3, v0.2 fallback, task continuity, refusals, and agent card discovery.
|
@microsoft-github-policy-service agree |
5293bb3 to
439d91f
Compare
Richard Lundeen (richlundeen)
left a comment
There was a problem hiding this comment.
Thanks for adding this! We are interested in including A2A support in PyRIT. The overall direction looks good, and the target follows the main PyRIT target contract.
The main change I would like before merging is to use the official A2A Python SDK (a2a-sdk), rather than maintain our own protocol implementation. It is published on PyPI and supports A2A v1.0 with v0.3 compatibility. Using its typed messages, agent-card discovery, and client transport would reduce the version-specific code we need to maintain. A2ATarget can remain a thin adapter that owns PyRIT message mapping, identifiers/capabilities, upstream context tracking, and error mapping. An optional a2a extra would fit our existing dependency pattern.
A few gaps to address during that change:
- Safe polling retries: A 429 from
tasks/getcurrently retries the whole send method, which submits the prompt again with a new message ID. Retry the poll against the existing task instead of creating duplicate agent work. - Task result handling: Failed or canceled tasks currently become ordinary text responses with no error flag. An
input-requiredtask with partial artifacts can also return those artifacts instead of the question in its status message. Handle these states explicitly, and do not usestr(result)as an agent answer when there is no supported text. - Timeouts: The blocking send uses HTTPX's five-second default timeout. The 120-second task timeout only starts after that response arrives, so it does not protect a slow initial request.
- Protocol versions: The
v02branch does not match the published v0.2.5 format, which already usesmessage/sendandkind. The0.3+claim also needs narrowing because v1.0 has breaking changes. This is a strong reason to let the SDK handle protocol compatibility. - Identity and capability limits: Include non-secret protocol/routing settings that affect behavior in the target identifier. If stored history exists but the upstream context is missing, fail explicitly or provide a supported restoration path instead of continuing with only the latest message. Capability overrides should not advertise features the adapter cannot send.
- Integration coverage: We should have a real HTTP integration test against a deterministic SDK-based agent, not only mocked responses. Cover multi-turn context, input-required continuation, failed tasks, and a polling 429 that does not resubmit the prompt. An environment-gated test against an owned Foundry agent would also be useful for real authentication and service interoperability.
Requesting changes mainly for the SDK direction and integration coverage. This is a useful addition, and I would like to get it into PyRIT with the protocol details handled by the maintained SDK.
|
Thanks for the thorough and constructive review Richard Lundeen (@richlundeen)! I completely agree with the direction. Adopting the official I will update the PR with the following changes:
I'll push the updates shortly. Thanks again! |
|
Also for additional context on motivation and broader ecosystem integration: I built agent-redteam-benchmark and actively develop agent-probe, using them to evaluate agent targets across diverse scenarios. I am also preparing a second red-teaming agent with more advanced orchestration strategies that will build upon this A2A target integration. |
|
The updates are pushed in commit 0f76903:
|
Use refreshable HTTP authentication, serialize conversation state, validate capabilities and timeouts, and bound task polling retries. Add official SDK integration coverage and lock optional dependencies. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Closes #2824
Summary
Adds
A2ATargetso PyRIT can test agents exposed through the Agent-to-Agent (A2A) protocol, including Microsoft Foundry incoming A2A endpoints.Implementation
a2a-sdkthrough the optionalpyrit[a2a]extra and theallextra. The SDK handles JSON-RPC serialization and protocol compatibility. The dependency is bounded to>=1.1.5,<2and included inuv.lock.protocol_version="auto". Supports custom card paths. Discovery errors propagate without a silent protocol fallback.timeout=None, validates timeout and polling values, and rejects conflicting timeout settings. Polling HTTP 429 honorsRetry-Afteror uses exponential backoff within the polling deadline. The configured request-rate limit covers submission attempts and polls, excluding discovery.Integration coverage
The local integration tests use the official SDK server over loopback HTTP rather than a hand-written protocol server. They cover 0.3 and 1.0, discovery, persisted history, continuation, failures, and prevention of duplicate submissions after polling or empty results.
sse-starletteis an explicit development dependency required by the SDK server.Integration tests require
RUN_ALL_TESTS=true. The separate Foundry test also requiresA2A_FOUNDRY_ENDPOINT, uses a static token or a refreshable Entra token provider with thehttps://ai.azure.com/.defaultscope, and requires a successful text response. It does not create Azure resources.Target documentation covers installation, authentication, identity, lifecycle, limitations, timeout behavior, and integration-test setup.
Validation
For commit
7e12ce329:RUN_ALL_TESTS=false.The live Foundry test was not run because no endpoint was configured. No Azure resources were created. The author's earlier Foundry results applied to the pre-SDK implementation and do not verify the current SDK adapter.