[delete-cloud]

Summary

• add metadata-only tool.call spans around per-tool execution
• record tool name/call id and error/missing flags without arguments or result content
• cover successful and wrapped-error tool execution spans

Verification

• uv run pytest tests/agentkit/tools/test_toolset.py tests/agentkit/runtime tests/agentkit/observability -q
• uv run basedpyright --level error src/agentkit/tools/toolset.py tests/agentkit/tools/test_toolset.py src/agentkit/runtime/pipeline.py tests/agentkit/runtime/test_pipeline.py src/agentkit/observability tests/agentkit/observability
• uv run ruff check src/agentkit/tools/toolset.py tests/agentkit/tools/test_toolset.py
• uv run ruff format --check src/agentkit/tools/toolset.py tests/agentkit/tools/test_toolset.py
• git diff --check

Summary by CodeRabbit

• New Features

  • Integrated tool execution observability with span-based monitoring to capture and record tool call metrics, including execution status and error details

• Bug Fixes

  • Improved error handling for proxied tool calls to ensure proper error attribution and fallback behavior when execution fails

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR adds observability instrumentation to tool execution in the toolset module. Tool calls are now wrapped in record_span("tool.call") with attributes for tool name and call ID, and span attributes record outcome metadata (error state and type). Tests verify span recording through a new RecordingObservationSink. The diff also includes schema validation and test code reformatting with no semantic changes.

Changes

Tool Execution Observability

Layer / File(s)	Summary
Observability helpers and imports coding-agent/src/agentkit/tools/toolset.py	Module imports ObservationSink and record_span from observability; helper functions extract sink from context config, assemble tool span attributes (name and call ID), and set result outcome attributes (missing/error flags and error type).
Instrumented tool execution coding-agent/src/agentkit/tools/toolset.py	_execute_one wraps the selected tool execution path in record_span, passes observation sink and call attributes, records result outcome on the span before returning. Proxied tool error handling now uses underlying error or falls back to RuntimeError.
Observability test infrastructure and validation coding-agent/tests/agentkit/tools/test_toolset.py	New RecordingObservationSink and ToolContext classes capture and inject observation sinks into tool execution. Added tests verify tool.call spans are recorded with expected attributes on success and error cases (including error type metadata).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• delete-cloud/Coding-Agent-Loop#188: Introduces the foundational observability primitives (ObservationSink, SpanRecord, record_span) that this PR builds upon and integrates into tool execution.

Poem

🐰 A rabbit tools its way through spans,
Recording calls and error plans,
Each tool now glows with metrics bright,
Observability set just right! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'feat: instrument tool call spans' directly and concisely describes the main change: adding observability instrumentation (spans) around tool call executions.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/agentkit-tool-call-spans

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

coding-agent/tests/agentkit/tools/test_toolset.py (1)

1011-1013: 💤 Low value

Use standard return type annotation syntax.

The return type annotation -> (None) is non-idiomatic Python. The parentheses serve no purpose for a simple None return type and should be removed.

📝 Suggested fix

For all three functions, change:

-async def test_function_name() -> (
-    None
-):
+async def test_function_name() -> None:

Also applies to: 1072-1074, 1279-1281

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@coding-agent/tests/agentkit/tools/test_toolset.py` around lines 1011 - 1013,
The test function
test_toolset_call_tool_rejects_invalid_proxy_arguments_before_execution (and the
two other test functions flagged) use the non-idiomatic return annotation "->
(None)"; update each function signature to use the standard annotation "-> None"
(search for function signatures containing "-> (None)" and replace with "->
None") so the return type follows normal Python syntax.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@coding-agent/tests/agentkit/tools/test_toolset.py`:
- Around line 1011-1013: The test function
test_toolset_call_tool_rejects_invalid_proxy_arguments_before_execution (and the
two other test functions flagged) use the non-idiomatic return annotation "->
(None)"; update each function signature to use the standard annotation "-> None"
(search for function signatures containing "-> (None)" and replace with "->
None") so the return type follows normal Python syntax.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 96de8115-9556-4139-8bdc-c8d9ae5980f3

📥 Commits

Reviewing files that changed from the base of the PR and between fa24fc3 and a491790.

📒 Files selected for processing (2)

• coding-agent/src/agentkit/tools/toolset.py
• coding-agent/tests/agentkit/tools/test_toolset.py