Skip to content

G10 durable runtime observability correlation - #207

Merged
delete-cloud merged 2 commits into
mainfrom
codex/durable-runtime-g10-observability-correlation
May 18, 2026
Merged

delete-cloud merged 2 commits into
mainfrom
codex/durable-runtime-g10-observability-correlation

Conversation

@delete-cloud

@delete-cloud delete-cloud commented May 18, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • add safe durable runtime correlation attributes to AgentKit pipeline stage and LLM generation spans
  • bind HTTP root runs to turn_id and tape_id trace metadata
  • expand observability privacy coverage for sensitive attribute names
  • mark G10 complete in durable runtime progress docs

Verification

  • uv run pytest tests/agentkit/runtime/test_pipeline.py -v
  • uv run pytest tests/agentkit/observability/test_core.py -v
  • uv run pytest tests/ui/test_session_manager_runtime.py -k "run_id or approval or message_snapshot" -v
  • uv run pytest tests/coding_agent/test_observability.py -v
  • uv run ruff check src/agentkit/runtime/pipeline.py src/coding_agent/ui/session_manager.py tests/agentkit/runtime/test_pipeline.py tests/ui/test_session_manager_runtime.py tests/coding_agent/test_observability.py
  • uv run ruff format --check src/agentkit/runtime/pipeline.py src/coding_agent/ui/session_manager.py tests/agentkit/runtime/test_pipeline.py tests/ui/test_session_manager_runtime.py tests/coding_agent/test_observability.py
  • git diff --check

Summary by CodeRabbit

Release Notes

  • New Features

    • Enriched observability spans with safe, low-cardinality runtime correlation attributes (turn IDs, tape IDs, agent IDs) for improved distributed tracing and debugging correlation across system interactions.
  • Documentation

    • Added comprehensive task specification for runtime observability correlation, including implementation constraints and privacy preservation requirements.
  • Tests

    • Updated test coverage to verify propagation of correlation attributes in observability spans and validate filtering of sensitive data in exports.

Review Change Stack

@coderabbitai

coderabbitai Bot commented May 18, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Rate limit exceeded

@delete-cloud has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 54 minutes and 38 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 13d577aa-9e5a-4ac3-9216-5778d2a9bc2e

📥 Commits

Reviewing files that changed from the base of the PR and between 7e1b3db and a1811c3.

📒 Files selected for processing (2)
  • coding-agent/.opencode/prompts/tasks/durable-runtime-observability-correlation.md
  • coding-agent/tests/coding_agent/test_observability.py
📝 Walkthrough

Walkthrough

This PR propagates runtime correlation identifiers into observability spans by adding safe, low-cardinality correlation attributes (tape_id, turn_id, run_id) to pipeline stage and LLM generation spans, filtering trace_metadata by allowlist, and enriching session root run binding with trace metadata—while preserving OTLP/Langfuse privacy filtering that excludes sensitive fields.

Changes

Durable Runtime Observability Correlation

Layer / File(s) Summary
Task Definition and Completion Documentation
coding-agent/.opencode/prompts/tasks/durable-runtime-observability-correlation.md, coding-agent/docs/durable_runtime/GOAL_PROGRESS.md
Prompt task defines goal to propagate durable runtime correlation identifiers into observability spans with low-cardinality attributes, whitelisted trace_metadata, and binding HTTP root runs to turn_id/tape_id. GOAL_PROGRESS marks checkpoint G10 complete and adds verification evidence and target test commands.
Pipeline Span Runtime Correlation Attributes
coding-agent/src/agentkit/runtime/pipeline.py
Introduces internal allowlist of trace-metadata keys and two helper functions: one to filter trace_metadata by key allowlist and primitive value type, another to build correlation attributes dict (run_id, agent_id, parent_run_id, tape_id). Merges filtered correlation attributes into both stage span and LLM generation span attributes.
Session Manager Root Run Trace Metadata Binding
coding-agent/src/coding_agent/ui/session_manager.py
_bind_root_run_identity now copies and enriches run_context.trace_metadata by adding turn_id (from run_id) and tape_id (from session tape), then passes updated trace_metadata into the replaced AgentRunContext.
Pipeline Span Correlation Test Verification
coding-agent/tests/agentkit/runtime/test_pipeline.py
Runtime stage span and LLM generation span tests updated to verify correlation attributes (run_id, tape_id, turn_id, interaction_id) are present in span attributes; explicitly verify sensitive prompt field is excluded from span attributes.
OTLP Privacy Filtering and Session Manager Test Verification
coding-agent/tests/coding_agent/test_observability.py, coding-agent/tests/ui/test_session_manager_runtime.py
OTLP sink test expanded to verify multiple sensitive fields (content, message, result, secret, text, prompt) are omitted from exported spans. Session manager test asserts trace_metadata is populated with turn_id and tape_id.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • delete-cloud/Coding-Agent-Loop#91: Introduced AgentRunContext/PipelineContext.run_context with trace-metadata propagation that this PR builds upon for consistent span correlation attributes.
  • delete-cloud/Coding-Agent-Loop#195: Integrates the runtime correlation attributes (run_id, turn_id, tape_id) added here into OTLP trace/resource grouping logic for session-level trace ID derivation.

Poem

🐰 Hops with joy through correlation trace,
Tape and turn now mark their place,
Secrets safe, no prompt exposed,
Span attributes well-composed!
Durable runtime, glowing bright, ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the primary change: adding durable runtime observability correlation attributes and binding runtime identifiers to observability spans.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/durable-runtime-g10-observability-correlation

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
coding-agent/tests/coding_agent/test_observability.py (1)

94-98: ⚡ Quick win

Strengthen privacy assertion with structured key checks.

Current checks only validate sentinel values are absent from the serialized body. Add assertions that denylisted attribute keys are not present in exported span attributes to catch key-leak regressions too.

Proposed test-hardening diff
     body = request.read().decode()
     assert "llm.generation" in body
     assert "input_tokens" in body
     assert "do-not-send" not in body
     assert "content-do-not-send" not in body
     assert "message-do-not-send" not in body
     assert "result-do-not-send" not in body
     assert "secret-do-not-send" not in body
     assert "text-do-not-send" not in body
+    span = _first_span(_payload(request))
+    exported_keys = {
+        item["key"]
+        for item in span.get("attributes", [])
+        if isinstance(item, dict) and isinstance(item.get("key"), str)
+    }
+    assert {"content", "message", "result", "secret", "text", "prompt"}.isdisjoint(
+        exported_keys
+    )
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@coding-agent/tests/coding_agent/test_observability.py` around lines 94 - 98,
Add assertions to the test that not only check sentinel values are absent from
the serialized body but also verify the denylisted attribute keys are not
present in exported span attributes: after the existing body checks, iterate the
exported spans (e.g., the variable holding exported span payloads or each span's
.attributes) and assert that keys "content", "message", "result", "secret", and
"text" are not in any span.attributes keys to prevent key-leak regressions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@coding-agent/.opencode/prompts/tasks/durable-runtime-observability-correlation.md`:
- Around line 36-41: Update the "Target tests:" block to match PR verification
steps by adding the missing full-suite and observability tests and the ruff
format check: include `uv run pytest tests/agentkit/runtime/test_pipeline.py
-v`, `uv run pytest tests/agentkit/observability/test_core.py -v`, add the `uv
run ruff format --check ...` command used in PR verification, and ensure the `uv
run ruff check` command's file list includes
`tests/coding_agent/test_observability.py`; edit the Target tests section (the
"Target tests:" header and its bullet list) to add these exact commands so the
task definition and PR verification steps are synchronized.

---

Nitpick comments:
In `@coding-agent/tests/coding_agent/test_observability.py`:
- Around line 94-98: Add assertions to the test that not only check sentinel
values are absent from the serialized body but also verify the denylisted
attribute keys are not present in exported span attributes: after the existing
body checks, iterate the exported spans (e.g., the variable holding exported
span payloads or each span's .attributes) and assert that keys "content",
"message", "result", "secret", and "text" are not in any span.attributes keys to
prevent key-leak regressions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 1aeeed9f-6bb7-49a2-8fdb-f01ac34ed738

📥 Commits

Reviewing files that changed from the base of the PR and between 470e8b6 and 7e1b3db.

📒 Files selected for processing (7)
  • coding-agent/.opencode/prompts/tasks/durable-runtime-observability-correlation.md
  • coding-agent/docs/durable_runtime/GOAL_PROGRESS.md
  • coding-agent/src/agentkit/runtime/pipeline.py
  • coding-agent/src/coding_agent/ui/session_manager.py
  • coding-agent/tests/agentkit/runtime/test_pipeline.py
  • coding-agent/tests/coding_agent/test_observability.py
  • coding-agent/tests/ui/test_session_manager_runtime.py

@delete-cloud
delete-cloud merged commit 447272d into main May 18, 2026
1 check passed
@delete-cloud
delete-cloud deleted the codex/durable-runtime-g10-observability-correlation branch May 18, 2026 19:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant