[delete-cloud]

Summary

• Add a turn-scoped AgentObservationRecorder with append-only JSONL events for realtime crash/failure localization.
• Record sanitized LLM usage, tool-call, and tool-result shape events without raw prompt, args, stdout/stderr, or results.
• Allow Langfuse langfuse.observation.input/output only for schema-validated sanitized turn projections, documented in ADR-0049.

Test Plan

• uv run pytest tests/ -q
• uv run pytest tests/coding_agent/test_agent_observability.py tests/coding_agent/test_observability.py tests/ui/test_session_manager_runtime.py tests/coding_agent/test_pipeline_adapter.py tests/coding_agent/test_release_observability_contract.py -q
• uv run pytest tests/coding_agent/action_safety/test_action_observability.py tests/coding_agent/action_safety/test_safe_action_smoke.py tests/coding_agent/test_bootstrap.py -q
• uv run pytest tests/ui/test_http_server_failover.py tests/ui/test_session_manager_public_api.py -q
• uv run ruff check src/coding_agent/agent_observability.py src/coding_agent/adapter.py src/coding_agent/server/session_manager.py src/coding_agent/observability.py tests/coding_agent/test_agent_observability.py tests/ui/test_session_manager_runtime.py
• uv run ruff format --check src/coding_agent/agent_observability.py src/coding_agent/adapter.py src/coding_agent/server/session_manager.py tests/coding_agent/test_agent_observability.py tests/ui/test_session_manager_runtime.py
• git diff --check -- .

Summary by CodeRabbit

Release Notes

New Features

• Agent observability system records turn events, LLM token usage, and tool interactions
• Automatic data sanitization filters sensitive information from external observability exports
• Persistent observation logs for audit and compliance

[coderabbitai]

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

This PR introduces a session-turn-scoped observation recording system that captures LLM usage, tool interactions, and turn lifecycle events, sanitizes them to remove sensitive data, persists them locally to JSONL, and allows controlled export to Langfuse and OTLP endpoints.

Changes

Agent Observation Recording

Layer / File(s)	Summary
Observation event model and storage contracts src/coding_agent/agent_observability.py	Immutable AgentObservationEvent with kind/status/identifiers, AgentObservationStore protocol, and JsonlAgentObservationStore for append-only JSONL persistence under observability/runs/<run_id>/observations.jsonl.
Observation recorder and event recording src/coding_agent/agent_observability.py	AgentObservationRecorder manages turn lifecycles with sequence tracking, records eight event types (turn lifecycle + LLM/tool observations) to store and optional sink, suppresses exceptions.
Observation sanitization and projections src/coding_agent/agent_observability.py	Sanitization pipeline filters forbidden substrings and disallowed keys; sanitized_turn_projection converts turns to structured summaries; shape helpers redact sensitive data while preserving metadata (counts, lengths, structural info).
OTLP export sanitization gates src/coding_agent/observability.py	Langfuse allowlists restrict which observation keys/fields can be exported; _otlp_attribute_allowed gates inclusion; recursive payload validators enforce JSON string format and field allowlists for safe OTLP emission.
Adapter event recording integration src/coding_agent/adapter.py	PipelineAdapter retrieves recorder from context, records LLM usage, tool calls, and tool results before emitting deltas during event processing; imports narrowed and typing simplified.
SessionManager observation orchestration src/coding_agent/server/session_manager.py	SessionManager initializes JSONL store, creates per-turn recorder, starts before adapter, completes on success, fails/cancels on error paths; helper methods map status and resolve sink from config.
Tests validating observability behavior tests/coding_agent/test_agent_observability.py, tests/ui/test_session_manager_runtime.py	Unit tests verify JSONL persistence without raw secrets, sanitization removes sensitive strings while preserving metadata, and OTLP exports match schema; integration test confirms turn.started before adapter completion.
ADR documentation docs/adr/0049-agent-observation-recorder.md	ADR 0049 documents design, acceptance criteria, local persistence, event logging, sanitized exports, and schema validation constraints.

Sequence Diagram(s)

The sequence diagram above illustrates the complete observation lifecycle: SessionManager creates and starts a recorder before running the adapter, the adapter logs observations as events occur, and the recorder finalizes with a sanitized turn projection on completion.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

This PR introduces a comprehensive observability layer with multiple interacting components: an event model and storage protocol, a recording system with sanitization logic, export gates, adapter integration, and SessionManager orchestration. The sanitization pipeline includes careful key/value filtering with regex rules and forbidden-substring checks. OTLP export controls add recursive payload validation with Langfuse-specific allowlisting. While individual pieces are straightforward, the cross-file coordination, safety guarantees, and integration points across adapter/session/observability layers demand careful review of the full flow.

Possibly related PRs

• delete-cloud/Coding-Agent-Loop#384: Also modifies SessionManager.run_agent per-turn status/failure handling and exception paths, potentially conflicting with observation recorder lifecycle management.
• delete-cloud/Coding-Agent-Loop#188: Introduces shared ObservationSink abstractions that this PR depends on for external observation emission.

Poem

🐇 A rabbit's ode to safe observability:

Per turn, a recorder takes the stage,
Events flow through time without a rage,
Secrets filtered, shapes distilled,
Local JSONL files proudly filled,
Sanitized spans to distant skies—safe at last! 🌟

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main feature addition: introducing an agent observation recorder for observability, which is the primary focus of the changeset across all modified files.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/agent-observation-recorder

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}