Skip to content

test: add Bee runtime smoke docs - #317

Merged
delete-cloud merged 1 commit into
mainfrom
codex/bee-g101-smoke-docs
May 21, 2026
Merged

delete-cloud merged 1 commit into
mainfrom
codex/bee-g101-smoke-docs

Conversation

@delete-cloud

@delete-cloud delete-cloud commented May 21, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • add final Bee runtime smoke covering manifest -> task store -> lifecycle anchors -> planner -> launch metadata -> console -> metrics
  • add Bee runtime usage guide and implementation report
  • update Bee ledger and ADR acceptance criteria

Verification

  • uv run pytest tests/coding_agent/test_bee_runtime.py -v
  • uv run pytest tests/ui/test_developer_console.py -v
  • uv run pytest tests/coding_agent/test_observability.py -v
  • uv run pytest tests/coding_agent/action_safety/test_safe_action_smoke.py -v
  • uv run pytest tests/coding_agent/test_topic_layer_smoke.py -v
  • uv run pytest tests/coding_agent/test_scheduled_runs.py -v
  • uv run pytest tests/coding_agent/test_observability_platform_smoke.py -v
  • uv run pytest tests/integration/test_durable_runtime_smoke.py -v
  • uv run pytest tests/coding_agent/test_context_system_smoke.py -v
  • uv run pytest tests/coding_agent/evaluation/ -v
  • uv run pytest tests/agentkit/runtime/test_pipeline.py -k "build_context or runtime_stage_spans" -v
  • uv run ruff format --check tests/coding_agent/test_bee_runtime.py
  • uv run ruff check tests/coding_agent/test_bee_runtime.py
  • git diff --check -- .

Summary by CodeRabbit

  • Documentation

    • Added comprehensive Bee Runtime usage documentation including manifest schema, runtime flow, and console/metrics guidance
    • Added implementation report detailing scope, verification evidence, and known boundaries
    • Updated acceptance criteria tracking for Bee Runtime smoke testing
  • Tests

    • Added smoke test suite covering topic launch workflow with console metrics and observability validation

Review Change Stack

@coderabbitai

coderabbitai Bot commented May 21, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Rate limit exceeded

@delete-cloud has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 53 minutes and 51 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 45865a66-aa0c-44d8-9204-7dbacc8325df

📥 Commits

Reviewing files that changed from the base of the PR and between 8b3d5ae and a8137b7.

📒 Files selected for processing (5)
  • coding-agent/docs/adr/0041-bee-workflow-task-runtime.md
  • coding-agent/docs/bee_runtime/GOAL_PROGRESS.md
  • coding-agent/docs/bee_runtime/IMPLEMENTATION_REPORT.md
  • coding-agent/docs/bee_runtime/USAGE.md
  • coding-agent/tests/coding_agent/test_bee_runtime.py
📝 Walkthrough

Walkthrough

This PR marks the completion of Bee Runtime goal G101 by adding comprehensive documentation, updating the ADR acceptance criteria, and implementing a full end-to-end smoke test that validates the runtime's task/node lifecycle, console rendering, and metrics recording.

Changes

Bee Runtime G101 Completion

Layer / File(s) Summary
ADR Acceptance Criteria and Goal Progress Logging
coding-agent/docs/adr/0041-bee-workflow-task-runtime.md, coding-agent/docs/bee_runtime/GOAL_PROGRESS.md
ADR checkmarks updated to confirm test_bee_runtime_smoke_manifest_topic_launch_console_metrics and git diff --check are passing. GOAL_PROGRESS.md extended with G101 ledger entry documenting final Bee runtime smoke coverage, test suite runs (unit/smoke/UI/observability/action-safety/topic/scheduled/integration/context/evaluation/pipeline), and deferred durable-run-conversion risk.
Implementation Report and Usage Documentation
coding-agent/docs/bee_runtime/IMPLEMENTATION_REPORT.md, coding-agent/docs/bee_runtime/USAGE.md
IMPLEMENTATION_REPORT documents Bee Runtime scope, landed goals (G93–G101), system boundaries (inclusions/exclusions of IDs and raw content), G101 smoke-test verification evidence with command, and remaining risks (durable run wiring, TopicPlugin compatibility). USAGE guide covers required safe manifest fields, staged execution/anchoring lifecycle, console/Prometheus labeling restrictions, and smoke-test verification commands.
End-to-End Topic Launch Smoke Test with Observability
coding-agent/tests/coding_agent/test_bee_runtime.py
New async test test_bee_runtime_smoke_manifest_topic_launch_console_metrics imports observability (SpanRecord, PrometheusMetricsRecorder/ObservationSink) and console rendering utilities. Test executes full topic-launch flow: populates fake store with task and validation node dependency, starts/finalizes task via BeeTaskLifecycle, plans launch intents via BeeTaskPlanner, builds metadata, renders developer-console HTML, records runtime metrics span, and asserts anchors, console/metrics consistency while excluding sensitive fields.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

  • delete-cloud/Coding-Agent-Loop#303: Both update Bee-runtime goal documentation in GOAL_PROGRESS.md (303 adds G93 scaffolding; this PR adds G101 completion entry).
  • delete-cloud/Coding-Agent-Loop#291: This PR's topic launch smoke test validates developer-console topic rendering and metrics, directly related to 291's topic_id sanitization and topic console view work.
  • delete-cloud/Coding-Agent-Loop#271: Both add developer-console E2E smoke/no-leak validation (271 in route assertions; this PR in runtime metrics and console content consistency checks).

Poem

🐰 A Bee takes flight at last, with G101 in hand,
Smoke tests and docs now dance across the land,
Console metrics bloom, safe fields stand guard,
The runtime flows—what labor, yet not hard!
Goal complete, the warren cheers with glee.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title 'test: add Bee runtime smoke docs' partially relates to the changeset. While it mentions adding tests and docs, it omits the primary implementation work: a new smoke test file, usage guide, implementation report, and updates to the ledger/ADR files. Consider a more complete title like 'test: add Bee runtime smoke test, usage guide, and implementation report' to better reflect all major changes in the PR.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/bee-g101-smoke-docs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@delete-cloud
delete-cloud force-pushed the codex/bee-g101-smoke-docs branch from 8b3d5ae to a8137b7 Compare May 21, 2026 18:30
@delete-cloud
delete-cloud merged commit cda48aa into main May 21, 2026
1 check passed
@delete-cloud
delete-cloud deleted the codex/bee-g101-smoke-docs branch May 21, 2026 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant