[codex] Render memory context packs

coding-agent/.opencode/prompts/tasks/context-system-g23-memory-context-pack.md

@@ -0,0 +1,43 @@
+Goal:
+Render Coding Agent memory as evidence-backed context-pack reference material instead of legacy memory grounding.
+
+Scope:
+- Convert `MemoryPlugin.build_context` output to a rendered `ContextPack` memory section.
+- Omit memories without evidence from grounding by default.
+- Preserve topic-file filtering and importance ordering for memory selection.
+- Support memory evidence refs that include session and tape-entry identifiers.
+
+Out of scope:
+- AgentKit pipeline, `ContextBuilder`, or directive schema changes.
+- Changing how new memory records are produced or persisted beyond rendering needs.
+- External services, vector stores, or production credentials.
+
+Context:
+- ADR:
+  - `docs/adr/0034-context-system-boundaries-and-evidence.md`
+- Relevant files:
+  - `src/coding_agent/plugins/memory.py`
+  - `src/coding_agent/context_pack.py`
+  - `tests/coding_agent/plugins/test_memory.py`
+  - `tests/coding_agent/test_context_pack.py`
+- Postmortem:
+  - `postmortem/patterns/PM-0009-preserve-neutral-bare-anchor-semantics.md`
+
+Target tests:
+- `uv run pytest tests/coding_agent/plugins/test_memory.py -k "build_context or TopicScopedRecall" -v`
+- `uv run pytest tests/coding_agent/test_context_pack.py -k "memory" -v`
+- `uv run pytest tests/coding_agent/plugins/test_memory.py tests/coding_agent/test_context_pack.py -v`
+- `uv run pytest tests/coding_agent/ -k "context_pack or retrieval or memory or evaluation" -v`
+- `uv run pytest tests/agentkit/runtime/test_pipeline.py -k "build_context" -v`
+
+Loop policy:
+- Engineer writes failing memory-rendering tests first.
+- Engineer implements the smallest coding-agent-only rendering change.
+- Reviewer reports only P1/P2 issues.
+- Engineer fixes accepted P1/P2 findings and reruns target tests.
+
+Stop conditions:
+- Stop if the change requires AgentKit runtime or `ContextBuilder` changes.
+- Stop if memory records need to become system instructions to satisfy tests.
+- Stop if unevidenced memories cannot be omitted without breaking deterministic behavior.
+- Stop after two failed fix iterations for the same failure.

coding-agent/docs/context_system/GOAL_PROGRESS.md

@@ -779,3 +779,79 @@ Remaining risks:
 
 - G22 records evidence but still renders legacy `[Memory]` grounding; G23 owns rendering memory as context-pack reference material and omitting unevidenced memories.
 - Evidence inference is intentionally conservative and file-tag based; richer tape-entry/session/test-failure evidence can be added without breaking the JSONL shape.
+
+## G23 - Memory Retrieval And Injection As Evidence-Backed Reference Context
+
+Status: in progress.
+
+### Before
+
+Goal id: G23
+
+Intended files:
+
+- `src/coding_agent/plugins/memory.py`
+- `src/coding_agent/context_pack.py`
+- `tests/coding_agent/plugins/test_memory.py`
+- `tests/coding_agent/test_context_pack.py`
+- `docs/context_system/GOAL_PROGRESS.md`
+- `.opencode/prompts/tasks/context-system-g23-memory-context-pack.md`
+
+Verification commands:
+
+- `uv run pytest tests/coding_agent/plugins/test_memory.py -k "build_context or TopicScopedRecall" -v`
+- `uv run pytest tests/coding_agent/test_context_pack.py -k "memory" -v`
+- `uv run pytest tests/coding_agent/plugins/test_memory.py tests/coding_agent/test_context_pack.py -v`
+- `uv run pytest tests/coding_agent/ -k "context_pack or retrieval or memory or evaluation" -v`
+- `uv run pytest tests/agentkit/runtime/test_pipeline.py -k "build_context" -v`
+
+Stop criteria:
+
+- Change requires AgentKit runtime, `ContextBuilder`, or directive schema changes.
+- Change renders memory as instructions or policy rather than reference material.
+- Change injects unevidenced memory by default.
+- Change breaks topic-file filtering or importance ordering for memory recall.
+- Deterministic verification cannot be produced.
+- More than two fix iterations fail for the same reason.
+
+Postmortem routing:
+
+- `src/coding_agent/plugins/memory.py`, `src/coding_agent/context_pack.py`, `tests/coding_agent/plugins/test_memory.py`, and `tests/coding_agent/test_context_pack.py` match PM-0009. Release checks: run focused affected tests and review control-flow shape before shipping.
+
+### After
+
+Changed files:
+
+- `src/coding_agent/plugins/memory.py`
+- `src/coding_agent/context_pack.py`
+- `tests/coding_agent/plugins/test_memory.py`
+- `tests/coding_agent/test_context_pack.py`
+- `docs/context_system/GOAL_PROGRESS.md`
+- `.opencode/prompts/tasks/context-system-g23-memory-context-pack.md`
+
+Verification results:
+
+- Baseline: `uv run pytest tests/coding_agent/plugins/test_memory.py -v` passed with 21 passed before G23 edits.
+- Red: `uv run pytest tests/coding_agent/plugins/test_memory.py -k "build_context or TopicScopedRecall" -v` failed with 3 failed, 3 passed, 16 deselected because memory still used legacy `[Memory]` rendering and unevidenced injection.
+- Red: `uv run pytest tests/coding_agent/test_context_pack.py -k "memory_session_evidence" -v` failed with 1 failed, 4 deselected because `EvidenceRef` did not accept session/tape-entry evidence fields.
+- Green: `uv run pytest tests/coding_agent/plugins/test_memory.py -k "build_context or TopicScopedRecall" -v` passed with 6 passed, 16 deselected.
+- Green: `uv run pytest tests/coding_agent/test_context_pack.py -k "memory" -v` passed with 3 passed, 2 deselected.
+- `uv run ruff format --check src/coding_agent/plugins/memory.py src/coding_agent/context_pack.py tests/coding_agent/plugins/test_memory.py tests/coding_agent/test_context_pack.py` passed with all four files already formatted.
+- `uv run ruff check src/coding_agent/plugins/memory.py src/coding_agent/context_pack.py tests/coding_agent/plugins/test_memory.py tests/coding_agent/test_context_pack.py` passed.
+- `git diff --check -- .` passed.
+- `uv run pytest tests/coding_agent/plugins/test_memory.py tests/coding_agent/test_context_pack.py -v` passed with 27 passed.
+- `uv run pytest tests/coding_agent/ -k "context_pack or retrieval or memory or evaluation" -v` passed with 51 passed, 585 deselected.
+- `uv run pytest tests/agentkit/runtime/test_pipeline.py -k "build_context" -v` passed with 7 passed, 30 deselected.
+
+Implementation notes:
+
+- MemoryPlugin now renders recalled memories through `ContextPackRenderer` as a single `Memory references` section.
+- Memory records without evidence are omitted from grounding by default.
+- Topic-file filtering and importance ordering still choose memory candidates before rendering.
+- Memory context-pack source ids are deterministic hashes of normalized summary, tags, and evidence.
+- ContextPack `EvidenceRef` now supports optional `session_id` and `tape_entry_id` fields for memory evidence.
+
+Remaining risks:
+
+- MemoryPlugin still uses simple tag overlap for memory retrieval; richer memory ranking is outside G23.
+- Unevidenced legacy memories remain persisted and loadable but are intentionally invisible to default grounding.

coding-agent/src/coding_agent/context_pack.py

@@ -29,6 +29,8 @@ class EvidenceRef:
     chunk_id: str | None = None
     test_node_id: str | None = None
     command_label: str | None = None
+    session_id: str | None = None
+    tape_entry_id: str | None = None
 
     def __post_init__(self) -> None:
         _require_non_empty(self.kind, "kind")
@@ -48,6 +50,8 @@ def to_dict(self) -> dict[str, JsonValue]:
         _add_optional(payload, "chunk_id", self.chunk_id)
         _add_optional(payload, "test_node_id", self.test_node_id)
         _add_optional(payload, "command_label", self.command_label)
+        _add_optional(payload, "session_id", self.session_id)
+        _add_optional(payload, "tape_entry_id", self.tape_entry_id)
         return payload
 
 
@@ -263,6 +267,10 @@ def _format_evidence(evidence: EvidenceRef) -> str:
         _append_unique(details, evidence.test_node_id)
     if evidence.command_label is not None:
         _append_unique(details, evidence.command_label)
+    if evidence.session_id is not None:
+        _append_unique(details, evidence.session_id)
+    if evidence.tape_entry_id is not None:
+        _append_unique(details, evidence.tape_entry_id)
     return f"{evidence.kind}:{evidence.source_id} ({'; '.join(details)})"
 
 

coding-agent/src/coding_agent/plugins/memory.py

@@ -14,13 +14,22 @@
 
 from __future__ import annotations
 
+import hashlib
+import json
 import re
 from pathlib import PurePosixPath
 from typing import Any, Callable
 
 from agentkit.directive.types import MemoryRecord
 from agentkit.tape.models import Entry
 from agentkit.tape.tape import Tape
+from coding_agent.context_pack import (
+    ContextPack,
+    ContextPackItem,
+    ContextPackRenderer,
+    ContextPackSection,
+    EvidenceRef,
+)
 
 
 class MemoryPlugin:
@@ -112,7 +121,7 @@ def on_session_event(
     def build_context(
         self, tape: Tape | None = None, **kwargs: Any
     ) -> list[dict[str, Any]]:
-        """Grounding mode: inject relevant memories as system messages.
+        """Grounding mode: inject evidence-backed memories as reference context.
 
         If topic file tags are available, filter memories to those with
         overlapping tags. Falls back to importance-sorted top-N otherwise.
@@ -139,16 +148,28 @@ def build_context(
                 self._memories, key=lambda m: m.get("importance", 0.5), reverse=True
             )
 
-        top = sorted_memories[: self._max_grounding]
+        memory_items: list[ContextPackItem] = []
+        for memory in sorted_memories:
+            item = _memory_context_pack_item(memory)
+            if item is None:
+                continue
+            memory_items.append(item)
+            if len(memory_items) == self._max_grounding:
+                break
 
-        grounding_messages = []
-        for mem in top:
-            content = f"[Memory] {mem['summary']}"
-            if mem.get("tags"):
-                content += f" (tags: {', '.join(mem['tags'])})"
-            grounding_messages.append({"role": "system", "content": content})
+        if not memory_items:
+            return []
 
-        return grounding_messages
+        return ContextPackRenderer().render_messages(
+            ContextPack(
+                sections=(
+                    ContextPackSection(
+                        title="Memory references",
+                        items=tuple(memory_items),
+                    ),
+                )
+            )
+        )
 
     def _tags_overlap(self, memory_tags: list[str], topic_files: set[str]) -> bool:
         """Check if any memory tag overlaps with topic file paths."""
@@ -371,6 +392,76 @@ def _normalize_evidence_refs(value: Any) -> list[dict[str, Any]]:
     return _merge_evidence_refs(refs)
 
 
+def _memory_context_pack_item(memory: dict[str, Any]) -> ContextPackItem | None:
+    summary = _memory_summary(memory)
+    if summary is None:
+        return None
+
+    evidence = tuple(
+        evidence_ref
+        for raw_ref in _normalize_evidence_refs(memory.get("evidence", []))
+        if (evidence_ref := _evidence_ref_from_memory_ref(raw_ref)) is not None
+    )
+    if not evidence:
+        return None
+
+    return ContextPackItem(
+        source_kind="memory",
+        source_id=_memory_source_id(memory, summary=summary, evidence=evidence),
+        label=summary,
+        evidence=evidence,
+    )
+
+
+def _memory_summary(memory: dict[str, Any]) -> str | None:
+    summary = memory.get("summary")
+    if not isinstance(summary, str):
+        return None
+    normalized = " ".join(summary.split())
+    return normalized or None
+
+
+def _evidence_ref_from_memory_ref(ref: dict[str, Any]) -> EvidenceRef | None:
+    kind = _non_empty_str(ref.get("kind"))
+    source_id = _non_empty_str(ref.get("source_id"))
+    label = _non_empty_str(ref.get("label"))
+    if kind is None or source_id is None or label is None:
+        return None
+
+    return EvidenceRef(
+        kind=kind,
+        source_id=source_id,
+        label=label,
+        repo_path=_non_empty_str(ref.get("repo_path")),
+        line_start=_positive_int(ref.get("line_start")),
+        line_end=_positive_int(ref.get("line_end")),
+        chunk_id=_non_empty_str(ref.get("chunk_id")),
+        test_node_id=_non_empty_str(ref.get("test_node_id")),
+        command_label=_non_empty_str(ref.get("command_label")),
+        session_id=_non_empty_str(ref.get("session_id")),
+        tape_entry_id=_non_empty_str(ref.get("tape_entry_id")),
+    )
+
+
+def _memory_source_id(
+    memory: dict[str, Any],
+    *,
+    summary: str,
+    evidence: tuple[EvidenceRef, ...],
+) -> str:
+    key = {
+        "summary": summary,
+        "tags": _normalize_tags(memory.get("tags", [])),
+        "evidence": [ref.to_dict() for ref in evidence],
+    }
+    encoded = json.dumps(key, ensure_ascii=True, separators=(",", ":"), sort_keys=True)
+    return f"memory:{hashlib.sha256(encoded.encode('utf-8')).hexdigest()[:16]}"
+
+
+def _non_empty_str(value: Any) -> str | None:
+    return value if isinstance(value, str) and value else None
+
+
 def _positive_int(value: Any) -> int | None:
     if type(value) is int and value > 0:
         return value