Skip to content

refactor(layout): batch4 topics subsystem - #595

Merged
delete-cloud merged 2 commits into
mainfrom
refactor/layout-batch4
Jun 14, 2026
Merged

delete-cloud merged 2 commits into
mainfrom
refactor/layout-batch4

Conversation

@delete-cloud

@delete-cloud delete-cloud commented Jun 13, 2026 •

Copy link
Copy Markdown
Owner

What

Batch 4 (final) of the package-layout consolidation: relocate the 9-module topic/recall subsystem into a new topics/ package. Old import paths preserved via shims — no external call-site changes.

old path new path
topic_lifecycle.py topics/lifecycle.py
topic_memory.py topics/memory.py
topic_provenance.py topics/provenance.py
topic_range_index.py topics/range_index.py
topic_recall.py topics/recall.py
topic_store.py topics/store.py
context_pack.py topics/context_pack.py
recall_context.py topics/recall_context.py
recall_evaluation.py topics/recall_evaluation.py

Compat mechanism

Each old path is a sys.modules alias shim. Sibling imports inside the moved modules were repointed to coding_agent.topics.*; topics/__init__.py is kept thin to avoid cycles. ~28 external references continue to resolve through the shims unchanged.

Verification (independent, outside codex sandbox)

  • uv run pytest -q: 3725 passed, 8 skipped, 20 xfailed
  • All 9 old paths import in fresh processes and are identity-equal to their new module in both import orderings.

Batch 4 of 4 — completes the consolidation. Spec: .opencode/prompts/tasks/package-layout-consolidation-task-packet.md.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Chore
    • Internal code organization: Core topic management modules reorganized into a new subsystem package for improved maintainability. All existing functionality remains unchanged and backward compatible through compatibility layers.

@coderabbitai

coderabbitai Bot commented Jun 13, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

This PR reorganizes the Coding Agent topic subsystem by moving nine module implementations into a new coding_agent/topics/ package and replacing the old modules with import shims. The new subsystem introduces durable topic lifecycle management, memory candidate modeling, deterministic range indexing and cross-topic recall planning, and PostgreSQL-backed persistence for topics, anchors, and cost tracking.

Changes

Topic Subsystem Reorganization

Layer / File(s) Summary
Deprecation Shims for Backward Compatibility
coding-agent/src/coding_agent/context_pack.py, recall_context.py, recall_evaluation.py, topic_lifecycle.py, topic_memory.py, topic_provenance.py, topic_range_index.py, topic_recall.py, topic_store.py, topics/__init__.py
All nine legacy modules are replaced with simple sys.modules aliases that forward imports to their new topics/ submodule locations, maintaining backward compatibility.
Data Models: Storage and Context Packing
topics/store.py, topics/context_pack.py
Introduces immutable TopicRecord, TopicAnchorRecord, TopicCostRecord dataclasses with strict __post_init__ validation; PostgreSQL persistence layer (PGTopicStore) with lazy schema creation and async CRUD; context pack domain models (EvidenceRef, ContextPackItem, ContextPackSection, ContextPack) with JSON serialization and ContextPackRenderer for system message grounding.
Provenance and Cost Tracking
topics/provenance.py
Adds cost delta computation and store-backed updates, provenance payload builders for evaluation/memory evidence with topic metadata, and safe label normalization for metrics.
Topic Lifecycle Management with Tape Integration
topics/lifecycle.py
Implements durable TopicLifecycle class that creates topics with initial tape anchors, finalizes or aborts with end anchors and optional memory review proposals, and provides utilities to query tape entries and anchors belonging to a topic.
Topic Recall with Token Overlap Scoring
topics/recall.py
Adds deterministic token-overlap recall (recall_topic_summaries) that ranks candidate topics by score, async anchor-based recording of recall links, and context pack rendering for recalled topics.
Topic-Derived Memory Candidates and Review Store
topics/memory.py
Introduces TopicDerivedMemoryCandidate and ReviewedMemoryRecord dataclasses with validation; MemoryReviewStore for JSONL-backed candidate lifecycle management; and proposal functions to extract memory candidates from finalized topics and Bee workspace artifacts.
Deterministic Topic Range Indexing and Search
topics/range_index.py
Implements in-memory TopicRangeIndex with deterministic token-overlap scoring and multi-dimensional filtering (kind, profile, tags, date ranges); includes safe text validation to prevent indexing forbidden markers.
Cross-Topic Recall Planning with Memory Ranking
topics/recall_context.py
Adds TopicRecallPlanner that builds cross-topic search queries, retrieves and filters results excluding the source topic, ranks accepted memory candidates using token overlap plus provenance boosts, and renders recall plans into context packs.
Recall Evaluation across Multiple Variants
topics/recall_evaluation.py
Implements deterministic evaluation of four recall modes (no recall, memory-only, topic-range-only, combined) to measure candidate counts and expected-ID match rates for testing/benchmarking recall effectiveness.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • delete-cloud/Coding-Agent-Loop#224: Adds session_id and tape_entry_id fields to EvidenceRef in the context pack model that enable session-scoped memory context rendering.
  • delete-cloud/Coding-Agent-Loop#358: Implements the TopicRangeIndex, TopicRangeSearchQuery, and search query building logic that is now relocated into topics/range_index.py as part of this reorganization.
  • delete-cloud/Coding-Agent-Loop#362: Adds recall evaluation functionality and tests for the RecallEvalVariant, RecallEvalCase, and evaluate_recall_variants utilities now living in topics/recall_evaluation.py.

Poem

🐰 Hop-skip through the module trees,
Topics bundled, organized with ease—
Shims redirect the old-path calls,
While new subsystems fill the halls;
Recall, memory, and lifecycle now clear,
All in one coherent tier! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 1.14% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'refactor(layout): batch4 topics subsystem' accurately summarizes the main change: moving nine topic-related modules into a new topics subsystem package while preserving import compatibility via sys.modules shims.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch refactor/layout-batch4

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Nitpick comments (4)
coding-agent/src/coding_agent/topics/lifecycle.py (4)

328-330: ⚡ Quick win

Error message could include the actual tape ID for easier debugging.

The error message indicates which tape the topic belongs to, but not which tape was actually provided.

♻️ Proposed fix
 def _require_matching_tape(tape: Tape, topic: TopicRecord) -> None:
     if tape.tape_id != topic.tape_id:
-        raise ValueError(f"topic {topic.topic_id} belongs to tape {topic.tape_id}")
+        raise ValueError(
+            f"topic {topic.topic_id} belongs to tape {topic.tape_id}, "
+            f"not {tape.tape_id}"
+        )
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@coding-agent/src/coding_agent/topics/lifecycle.py` around lines 328 - 330,
The error raised in _require_matching_tape currently only states the topic's
tape_id; update the ValueError message to include both the provided tape's id
(tape.tape_id) and the topic's tape id (topic.tape_id) for clearer debugging,
e.g. change the f-string in _require_matching_tape to mention "provided tape
{tape.tape_id}" and "topic {topic.topic_id} belongs to tape {topic.tape_id}" so
the mismatch is explicit.

312-325: 💤 Low value

Accessing private Tape internals creates tight coupling.

These helpers directly access tape._lock and tape._entries, which are private implementation details. If the Tape class changes its internal structure, these functions will break.

If there's no public API for atomic append/remove, consider documenting this coupling with a comment explaining the dependency and the reason for bypassing encapsulation.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@coding-agent/src/coding_agent/topics/lifecycle.py` around lines 312 - 325,
_helpers _append_anchor and _remove_anchor directly access Tape internals
(tape._lock and tape._entries), creating tight coupling; either update the Tape
class to provide atomic, public methods (e.g., Tape.append_anchor(anchor) -> seq
and Tape.remove_anchor(seq, entry_id) -> None) and use those here, or if adding
APIs is infeasible, add a clear comment above _append_anchor and _remove_anchor
documenting the deliberate coupling: explain that these helpers require atomic
access to the entries and lock, name the specific private attributes accessed
(tape._lock, tape._entries), and state the reason why encapsulation is bypassed
and the risk if Tape internals change so future maintainers can refactor safely.

210-213: ⚡ Quick win

Consider handling memory review store exceptions to prevent masking the successful topic finalization.

If add_candidate raises an exception, the topic is already finalized and the anchor is recorded, but the caller receives an exception suggesting failure. This could lead to retry attempts that fail because the topic is already finalized, or mask that memory proposal specifically failed.

Consider wrapping the memory candidate addition in a try/except to log failures without propagating them, since this is a secondary side effect after the primary operation succeeds.

♻️ Proposed fix
         if close_status == "finalized" and self._memory_review_store is not None:
-            candidate = propose_memory_candidate_from_topic(stored)
-            if candidate is not None:
-                self._memory_review_store.add_candidate(candidate)
+            try:
+                candidate = propose_memory_candidate_from_topic(stored)
+                if candidate is not None:
+                    self._memory_review_store.add_candidate(candidate)
+            except Exception:
+                # Log but don't fail - topic finalization succeeded
+                pass
         return stored
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@coding-agent/src/coding_agent/topics/lifecycle.py` around lines 210 - 213,
The code calls propose_memory_candidate_from_topic(stored) and then
self._memory_review_store.add_candidate(candidate) inside the finalization path,
but if add_candidate raises it will propagate and falsely indicate topic
finalization failed; wrap the add_candidate call in a try/except that catches
Exception, logs the failure (including the exception info and the
candidate/context) via the existing logger or process logger used in this
module, and do not re-raise so the primary operation (topic finalization)
remains successful; keep the candidate None check as-is and only attempt
add_candidate when candidate is not None.

312-325: ⚡ Quick win

Private tape manipulation helpers are exposed across module boundaries.

The tape manipulation functions in lifecycle.py access private Tape internals (_lock, _entries), and recall.py imports these private helpers (_append_anchor, _remove_anchor). This creates a fragile dependency chain: both modules depend on Tape's internal structure, and recall.py additionally depends on lifecycle.py's internal helpers.

  • coding-agent/src/coding_agent/topics/lifecycle.py#L312-L325: Consider making _append_anchor and _remove_anchor public (drop the _ prefix) if they're intended to be shared, or move them to a dedicated internal utility module (e.g., topics/_tape_utils.py).
  • coding-agent/src/coding_agent/topics/recall.py#L17-L17: Update the import to use the public/shared location once the helpers are relocated.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@coding-agent/src/coding_agent/topics/lifecycle.py` around lines 312 - 325,
Create a shared, public helper module and stop exposing Tape internals across
modules: move the implementations of _append_anchor and _remove_anchor from
coding-agent/src/coding_agent/topics/lifecycle.py (lines 312-325) into a new
module topics/_tape_utils.py and rename them to append_anchor and remove_anchor
(they should still operate on Tape under its lock but be public helpers). In
lifecycle.py replace the local private implementations with imports from
topics._tape_utils (or delete them and import append_anchor/remove_anchor) so
lifecycle references the shared helpers. In
coding-agent/src/coding_agent/topics/recall.py (line 17) update the import to
import append_anchor and remove_anchor from coding_agent.topics._tape_utils (or
from lifecycle if you instead made them public there), ensuring no code
elsewhere accesses Tape._lock or Tape._entries directly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@coding-agent/src/coding_agent/topics/context_pack.py`:
- Around line 74-85: The __post_init__ currently normalizes containers (evidence
-> tuple, metadata -> dict) but doesn't validate element types, causing
downstream serialization/rendering errors; update __post_init__ in the
ContextPack dataclass to iterate self.evidence and raise a TypeError if any
element is not the expected type (e.g., not a str) before converting to tuple,
and similarly validate metadata by ensuring every key is a str and every value
is the expected primitive/string type (raise TypeError with a clear message)
before converting to dict; also add the same element-type checks for any other
normalized containers (e.g., if body is a container) referenced in __post_init__
so failures occur fast and close to the source.

In `@coding-agent/src/coding_agent/topics/memory.py`:
- Around line 146-169: In save(), the temporary file created with
NamedTemporaryFile(delete=False) can be orphaned if an exception occurs during
the write loop; wrap the write/close/replace sequence in a try/except/finally
(or try/finally) so that on any exception you close the temp_file (if not
already closed) and remove temp_path (temp_path.unlink() if it exists) before
re-raising, and only call temp_path.replace(self._path) in the successful path;
update the save method (the NamedTemporaryFile block and the temp_path.replace
call) in memory.py to implement this cleanup logic using temp_path.exists()
checks and safe unlinking.

In `@coding-agent/src/coding_agent/topics/recall_context.py`:
- Around line 83-88: The search is currently limited to planner_input.limit
before filtering out the source topic, which can cause fewer than the requested
results; update the code that builds and executes the search (use
build_topic_recall_query and self._topic_index.search) to request one extra
result (e.g., planner_input.limit + 1), then filter out results where
result.topic_id == planner_input.source_topic.topic_id and only then slice to
planner_input.limit to return exactly the requested number of non-source recall
results.

In `@coding-agent/src/coding_agent/topics/recall_evaluation.py`:
- Around line 156-158: The comprehension that builds memory_ids currently
coerces missing candidate IDs to empty strings ("or \"\""), which hides upstream
errors; change the construction in the memory_ids assignment to not coerce
values and instead fail fast: iterate plan.accepted_memories and collect
memory.candidate.candidate_id directly (e.g.,
tuple(memory.candidate.candidate_id for memory in plan.accepted_memories)), and
add an explicit check that raises a clear exception (ValueError) if any
candidate_id is None/empty so invalid upstream IDs are surfaced (alternatively,
if you prefer filtering, filter out None/empty IDs with an explicit filter and
adjust candidate_count accordingly). Ensure you update any downstream logic that
assumes placeholder empty IDs to handle the raised error or the filtered list.

In `@coding-agent/src/coding_agent/topics/store.py`:
- Around line 200-240: The DDL defines topic_anchors, topic_recall_links, and
topic_costs without FKs so dependent rows can reference nonexistent topics; add
foreign key constraints on topic_anchors.topic_id,
topic_recall_links.source_topic_id and topic_recall_links.recalled_topic_id, and
topic_costs.topic_id to reference topics(id). Modify the schema to ALTER/CREATE
these tables with FOREIGN KEY (...) REFERENCES topics(id) (choose ON DELETE
CASCADE if you want dependents removed when a topic is deleted, or ON DELETE
RESTRICT to prevent deleting topics with dependents), and ensure the migration
checks that topics.id exists and that any existing orphan rows are handled
(delete or reassign) before adding the constraints.

---

Nitpick comments:
In `@coding-agent/src/coding_agent/topics/lifecycle.py`:
- Around line 328-330: The error raised in _require_matching_tape currently only
states the topic's tape_id; update the ValueError message to include both the
provided tape's id (tape.tape_id) and the topic's tape id (topic.tape_id) for
clearer debugging, e.g. change the f-string in _require_matching_tape to mention
"provided tape {tape.tape_id}" and "topic {topic.topic_id} belongs to tape
{topic.tape_id}" so the mismatch is explicit.
- Around line 312-325: _helpers _append_anchor and _remove_anchor directly
access Tape internals (tape._lock and tape._entries), creating tight coupling;
either update the Tape class to provide atomic, public methods (e.g.,
Tape.append_anchor(anchor) -> seq and Tape.remove_anchor(seq, entry_id) -> None)
and use those here, or if adding APIs is infeasible, add a clear comment above
_append_anchor and _remove_anchor documenting the deliberate coupling: explain
that these helpers require atomic access to the entries and lock, name the
specific private attributes accessed (tape._lock, tape._entries), and state the
reason why encapsulation is bypassed and the risk if Tape internals change so
future maintainers can refactor safely.
- Around line 210-213: The code calls
propose_memory_candidate_from_topic(stored) and then
self._memory_review_store.add_candidate(candidate) inside the finalization path,
but if add_candidate raises it will propagate and falsely indicate topic
finalization failed; wrap the add_candidate call in a try/except that catches
Exception, logs the failure (including the exception info and the
candidate/context) via the existing logger or process logger used in this
module, and do not re-raise so the primary operation (topic finalization)
remains successful; keep the candidate None check as-is and only attempt
add_candidate when candidate is not None.
- Around line 312-325: Create a shared, public helper module and stop exposing
Tape internals across modules: move the implementations of _append_anchor and
_remove_anchor from coding-agent/src/coding_agent/topics/lifecycle.py (lines
312-325) into a new module topics/_tape_utils.py and rename them to
append_anchor and remove_anchor (they should still operate on Tape under its
lock but be public helpers). In lifecycle.py replace the local private
implementations with imports from topics._tape_utils (or delete them and import
append_anchor/remove_anchor) so lifecycle references the shared helpers. In
coding-agent/src/coding_agent/topics/recall.py (line 17) update the import to
import append_anchor and remove_anchor from coding_agent.topics._tape_utils (or
from lifecycle if you instead made them public there), ensuring no code
elsewhere accesses Tape._lock or Tape._entries directly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 26d9ce54-6fd0-41d5-8d3a-eab0e174684d

📥 Commits

Reviewing files that changed from the base of the PR and between 444531c and 72a14ba.

📒 Files selected for processing (19)
  • coding-agent/src/coding_agent/context_pack.py
  • coding-agent/src/coding_agent/recall_context.py
  • coding-agent/src/coding_agent/recall_evaluation.py
  • coding-agent/src/coding_agent/topic_lifecycle.py
  • coding-agent/src/coding_agent/topic_memory.py
  • coding-agent/src/coding_agent/topic_provenance.py
  • coding-agent/src/coding_agent/topic_range_index.py
  • coding-agent/src/coding_agent/topic_recall.py
  • coding-agent/src/coding_agent/topic_store.py
  • coding-agent/src/coding_agent/topics/__init__.py
  • coding-agent/src/coding_agent/topics/context_pack.py
  • coding-agent/src/coding_agent/topics/lifecycle.py
  • coding-agent/src/coding_agent/topics/memory.py
  • coding-agent/src/coding_agent/topics/provenance.py
  • coding-agent/src/coding_agent/topics/range_index.py
  • coding-agent/src/coding_agent/topics/recall.py
  • coding-agent/src/coding_agent/topics/recall_context.py
  • coding-agent/src/coding_agent/topics/recall_evaluation.py
  • coding-agent/src/coding_agent/topics/store.py

@delete-cloud
delete-cloud merged commit fff0c8c into main Jun 14, 2026
2 checks passed
@delete-cloud
delete-cloud deleted the refactor/layout-batch4 branch June 14, 2026 03:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant