From 6d0dcf5460a615da234cb5d8e04aad3064760529 Mon Sep 17 00:00:00 2001 From: Bradley Hilton Date: Tue, 13 Oct 2026 15:07:58 -0500 Subject: [PATCH 01/19] docs: add KG closure RFC and Phase 1 implementation plan MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add the knowledge-graph closure RFC (intent → session → turn → provenance → change → resource, with an OWL 2 RL ontology and SHACL shapes) and a Phase 1 plan based on a survey of the current KG code. The plan covers the canonical IRI and identity model, the full old-to-new term replacement table, seven milestones (ontology, project identity, storage generation, projectors with unified reconcile, in-house RL reasoner, validation, consumer cutover), acceptance tests, and eleven resolved design decisions. Co-Authored-By: Claude Opus 5.5 --- docs/knowledge-graph-closure-phase1-plan.md | 727 +++++++++++ docs/knowledge-graph-closure-rfc.md | 1307 +++++++++++++++++++ 2 files changed, 2034 insertions(+) create mode 100644 docs/knowledge-graph-closure-phase1-plan.md create mode 100644 docs/knowledge-graph-closure-rfc.md diff --git a/docs/knowledge-graph-closure-phase1-plan.md b/docs/knowledge-graph-closure-phase1-plan.md new file mode 100644 index 00000000..b4a21ca1 --- /dev/null +++ b/docs/knowledge-graph-closure-phase1-plan.md @@ -0,0 +1,727 @@ +# Phase 1 Implementation Plan: KG Referential Closure + +> Status: **draft plan.** Implements Phase 1 ("Referential closure") of +> [`knowledge-graph-closure-rfc.md`](./knowledge-graph-closure-rfc.md). +> Branch: `feat/semweb-knowledge-graph`. Survey references verified against `dev` at `5ddcf1b`. + +## 1. Phase 1 scope (from the RFC) + +- Versioned OWL 2 RL ontology, optional OWL 2 DL companion, SHACL shape graph. +- Breaking graph-format cutover to canonical RDF IRIs; no compatibility vocabulary. +- Coverage test: every current predicate / entity type / edge kind has exactly one + canonical replacement. +- Signed stable `ProjectId`, project-qualified mutable-resource IRIs, algorithm-qualified + content-object IRIs. +- Intent / session / turn / provenance / change / attestation nodes. +- Required direct shortcut edges (`implements`, `explainedBy`, `attestedBy`). +- Backfill and projection validation. + +Explicitly **not** Phase 1: `AIContribution` nodes and cost aggregates (Phase 2), +`query ask` evidence envelopes (Phase 3), review closure digests / `reviewsClosure` +(Phase 4), federation and named-graph catalogs (Phase 5), `Execution` nodes (no +structured execution capture exists yet — see D11). + +## 2. Current state (survey findings) + +### 2.1 Storage + +- The KG is a **JSON property graph**, not a triple store: + `KgNode { id, kind, label, summary, source, metadata }` and + `KgEdge { from_id, to_id, kind, metadata }` (`atomic-core/src/pristine/vault.rs:462,516`). +- Tables (`atomic-core/src/pristine/tables.rs:797-830`): `kg_nodes`, `kg_edges` + (key `from\0to\0kind`, `encode_edge_key` L856), `kg_edges_from`, `kg_edges_to`, + `kg_fts`, `kg_fts_by_node`, `kg_index_meta`. No predicate or type index. +- The predicate string is **part of the edge key**, so renaming means rewriting keys. +- `upsert_kg_edge` (`txn/write/triples.rs:380`) accepts any string. `ontology::is_known_predicate` + has no callers. +- `kg_neighbors` (`traits/triples.rs:18-68`) is kind-blind, bidirectional BFS with depth ≤ 2; + it is the main read primitive (query plans, `vault context`, RAG, daemon). + +### 2.2 The KG is entirely derived + +Every node/edge comes from one of three producers, all over authoritative objects: + +| Producer | File | Source tag | Trigger | +|---|---|---|---| +| Vault projection | `atomic-repository/src/repository/vault_triples.rs` | `vault` | every `vault_store` (intent/memory/goal writes, `vault sync`, record w/ vault sync), `query reindex` | +| VCS/AST enrichment | `atomic-repository/src/repository/vault_kg_enrich.rs` | `vcs`, `ast` | clone, pull, git import, record (`kg_enrich_change`), `query enrich` | +| Session ledger | `atomic-core/src/pristine/txn/write/session_kg.rs` | `session_ledger` | turn append/rewrite, session lifecycle, sidecar pull, `session rebuild` | + +**Consequence for the cutover:** nothing in the KG needs in-place rewriting. A storage- +generation bump + full rebuild from canonical objects *is* the migration, and the RFC's +"fail on unclassifiable overloaded edge" rule becomes a projector-level guarantee (each +projector knows its endpoint classes at emit time). + +`vault sync` and `query enrich` share **no** code; they meet only on shared ID strings. + +### 2.3 Vocabulary inventory (`atomic-core/src/pristine/ontology.rs`) + +- 6 prefix labels (no IRIs), 30 `predicate::*` CURIEs, 16 `entity_type::*` (`vault:*`), + 29 `edge_kind::*` SCREAMING_CASE strings. +- Actually emitted: 19 uppercase kinds + `prov:{wasDerivedFrom,used,wasAssociatedWith, + hadPlan,hadMember,generated,wasInformedBy}`, `atom:informedBy`, `vault:{explainedBy,hasTodo}`. +- Defined but never emitted: all `dc:*`, `skos:*`, `rdf:type` (as edge), `vault:{status, + priority,assignedTo,blocks,...}` predicates, `IMPORTS`, `VISIBLE_FROM`, and 12 of 16 + `entity_type` values. +- Typing: `kind` string on every node; `metadata.rdf_type` (prefixed, e.g. `vault:Session`) + only on the 4 session-ledger node kinds. **No real `rdf:type` anywhere.** + +### 2.4 Identity problems + +- Change node IDs are **12-char base32 prefixes** in three places + (`vault_kg_enrich.rs:1089 truncate_hash`, `session_kg.rs:46 change_node_id`, + `vault_triples.rs:1268 rdf_target_to_kg_id`). Full hash survives only in metadata. +- `identity:` IDs are display names (`identity:{author.name}` vs `identity:{fm.developer}`); + intents write `author`/`created_by` which the KG ignores, so intents have no author edge. +- `todo:X` from `urn:atomic:todo:X` collides with the legacy session-global todo form. +- `LINKED_INTENT` targets `intent:{raw}` without uppercasing/manifest resolution. +- **No `ProjectId` exists.** Closest: remote `ProjectInfo.id: Uuid` (server-assigned), + `VaultManifest.intent_prefix` (4-char human code), `repository_identity` DID in + `provenance_gate.rs:76`. + +### 2.5 Overloaded / misused relations (confirmed in code) + +| Current | Uses | Lines | +|---|---|---| +| `DEPENDS_ON` | change→change; intent→intent (fallback for any `:::ref`) | enrich :409,:526; triples :1158 | +| `MODIFIES` | change→file; change→entity **only on record-time path**, not bulk | enrich :436,:554,:624 | +| `ON_VIEW` | change→*working copy's desired view* (not membership!); session→view | enrich :386,:514; session_kg :320 | +| `prov:wasAssociatedWith` | tool_result→goal | triples :1002 | +| `prov:hadPlan` | session→intent **and** turn→intent | session_kg :389-397 | +| `prov:hadMember` | session→turn | session_kg :381 | +| `prov:wasDerivedFrom` | memory→{anything}; session→session (fork) | triples :978; session_kg :456 | + +### 2.6 Facts captured but never projected + +- `Attestation` (`atomic-core/src/change/attestation.rs:153`): `session_id`, + `changes_covered`, `signer` (`did:atomic:…` or `session-mac:{sid}`), `signature`, + `previous_attestation`, `provenance_root`, … Stored as `.attest` files. + **No KG projection.** +- Canonical intent/memory attestations (signed JSON-LD vault entries) — explicitly skipped + by the KG (`vault_triples.rs:53`). +- `SessionTurn` (`atomic-core/src/change/session.rs:418`): `plan_id`, `goal`, + `provenance_hash`, `change_hashes`, `previous_provenance`, `outcome`, boundaries — + partially projected (no outcome/boundaries). +- AI `Provenance` on changes (model/tokens/cost) — Phase 2. + +### 2.7 Consumers that hardcode kind strings + +- `atomic-cli/src/commands/query/mod.rs`: `callers` (`to_uppercase()=="CALLS"` :1231,:1255), + HTML `EDGE_COLORS` (:748-755), `ask` system prompt (:1971-1976), text output. +- `atomic-repository/src/triage/report.rs` (`kind_is` at :35, case-insensitive, ~12 sites), + `atomic-repository/src/repository/triage.rs:231,305`. +- `atomic-repository/src/repository/vault_intent.rs:1770,1776` (`atom:informedBy`). +- `atomic-cli/src/commands/vault/context.rs:470-473` (filters `node.kind == "memory"`). +- libatomic daemon copies: `services_agent.rs:131,3372-3387`, `services_query.rs:1001-1038,1059`. +- Wire: `libatomic/proto/.../knowledge_messages.proto:123` `KGEdge.relation` (free string). +- Out-of-repo: `../noname` (`atomic.ts:700,1928`, `kgHops.ts:202-245`, `BlastTriage.tsx`, + `MemoryView.tsx`), `../the-hive/packages/shared/src/types/kg.ts:40-48`. +- ~110 test assertion sites (largest: `vault_triples.rs` ~40, `ontology.rs` ~31, + `atomic-cli/tests/session_integration_test.rs` ~17). + +### 2.8 Existing Semantic Web pieces and constraints + +- `atomic-canonical` already emits signed JSON-LD with context + `https://atomic.dev/ns/ctx.jsonld` mapping `atom:` → `https://atomic.dev/ns#`, and URNs + `urn:atomic:{intent,memory,change,...}:`. Content hash = `blake3(JCS(node))`, which covers + the `@context` **URL string** but not the term definitions behind it. +- `atomic provenance` projects PROV JSON-LD on demand (`atomic-canonical/src/prov.rs`); + not written to the KG. +- **No RDF/OWL/SHACL crates** in the workspace; no `.ttl` files. +- **Standing decision (2026-07-06, `docs/shacl-oxirs-spike-findings.md`):** no SHACL engine + ships in `atomic`; SHACL runs as a Circuit Breaker (CB) workflow; atomic keeps hand-coded + gates and ships shapes `.ttl` as data. oxirs rejected (~432 crates, silently drops + `sh:node`/`sh:sparql`). `oxttl` (~20 crates) noted as acceptable. +- Schema/upgrade machinery: `SCHEMA_VERSION` only rejects newer DBs (`schema.rs`); + **capability fences** (`capability.rs`, `required-capability/`) make older binaries + fail closed; `kg_index_meta` holds lazy per-index markers (`fts_reverse_schema` = 2 is the + precedent for a KG rebuild-on-write). + +## 3. Target model for Phase 1 + +### 3.1 Namespaces + +| Prefix | IRI | Role | +|---|---|---| +| `atomic:` | `https://atomic.dev/ns/workflow#` | the one workflow ontology | +| `prov:` | `http://www.w3.org/ns/prov#` | reused directly | +| `dcterms:` | `http://purl.org/dc/terms/` | `created`, `modified`, `description`, `subject` | +| `skos:` | `http://www.w3.org/2004/02/skos/core#` | `Concept`, `broader`, `narrower`, `related` | +| `rdf:`/`rdfs:`/`owl:`/`xsd:` | standard | | + +Ontology IRI `https://atomic.dev/ns/workflow`, version IRI +`https://atomic.dev/ns/workflow/1.0.0`. + +### 3.2 Node identity (replaces `kind:local` IDs) + +| Kind | IRI | Notes | +|---|---|---| +| Project | `urn:atomic:project:` | `` from the project identity object (D5) | +| Change | `urn:atomic:change:blake3-256:` | full hash, canonical uppercase base32 per D4; kills `truncate_hash` | +| Provenance graph | `urn:atomic:provenance:blake3-256:` | | +| Attestation | `urn:atomic:attestation:blake3-256:` | hash of `.attest` object | +| Intent (incl. review) | `urn:atomic:project::intent:` | ULID lowercase canonical | +| Goal / Memory / Skill / ToolResult | `urn:atomic:project::{goal,memory,skill,tool-result}:` | | +| AC / Task / ScopeItem / Constraint | `urn:atomic:project::intent::{ac,task,scope,constraint}:` | qualified by parent intent | +| Session / Turn / Todo | `urn:atomic:project::session:[:turn: \| :todo:]` | session-qualified todos only | +| View | `urn:atomic:project::view:` | | +| File / Module | `urn:atomic:project::{file,module}:` | | +| Source entity | `urn:atomic:project::entity:#:[~]` | per D10; file stays in the ID, line becomes a literal shown beside it | +| Identity | `did:atomic:…` when known; else `urn:atomic:project::identity:` + `atomic:Unknown`-style finding | stops display-name identity merging | +| Concept | `urn:atomic:project::concept:` typed `skos:Concept` | | +| Finding | `urn:atomic:project::finding::` | e.g. `MissingHistoricalPlanId` | + +A single `atomic_core::kg::iri` module owns all minting and parsing (percent-encoding per +RFC 3987 path rules); producers never format IDs by hand. The FTS ranking prefix checks +(`txn/write/triples.rs:268-280`, duplicated in `read.rs`) switch to type lookups. + +### 3.3 Edge mapping (complete replacement table) + +Every row below is a test fixture for the coverage test (M1). + +**Classes** — `entity_type::*` and `kind` values → `rdf:type`: + +| Current | Canonical | +|---|---| +| `vault:Goal` / `goal` | `atomic:Goal` | +| `vault:Intent` / `intent` | `atomic:WorkIntent` (+ `FeatureIntent`/`BugIntent`/`ReviewIntent`/… from canonical `kind`; `ManagedIntent` from lifecycle status) | +| `vault:Memory` / `memory` | `atomic:Memory` | +| `vault:Skill` / `skill` | `atomic:Skill` | +| `vault:ToolResult` / `tool_result` | `atomic:ToolResult` | +| `vault:File` / `file` | `atomic:File` | +| `vault:Identity` / `identity` | `atomic:Identity` | +| `vault:Concept` / `concept` | `skos:Concept` | +| `vault:Session` / `session` | `atomic:Session` | +| `vault:SessionTurn` / `session_turn` | `atomic:SessionTurn` (+ `atomic:ManagedTurn` when `plan_id` present) | +| `vault:Todo` / `todo` | `atomic:Todo` | +| `vault:ProvenanceGraph` / `provenance` | `atomic:ProvenanceGraph` | +| `vault:Task` / `task` | `atomic:Task` | +| `vault:AcceptanceCriterion` / `acceptance_criterion` | `atomic:AcceptanceCriterion` | +| `vault:ScopeItem` / `scope` | `atomic:ScopeItem` | +| `vault:Constraint` / `constraint` | `atomic:Constraint` | +| `change` | `atomic:Change` | +| `view` | `atomic:View` | +| `module` | `atomic:Module` | +| `entity` | `atomic:SourceEntity` (language kind stays a literal `atomic:entityKind`) | +| *(new)* | `atomic:Attestation`, `atomic:Finding` | + +**Predicates** — split by endpoint class where overloaded: + +| Current | Endpoints | Canonical | +|---|---|---| +| `prov:hadMember` | session→turn | `atomic:hasTurn` (inverse `atomic:turnOfSession`) | +| `prov:hadPlan` | turn→intent | `atomic:hasPlan` (inverse `atomic:governsTurn`) | +| `prov:hadPlan` | session→intent | **dropped**; derivable as session `hasTurn`/`hasPlan` | +| `prov:generated` | turn→change | `atomic:generatedChange` ⊑ `prov:generated` (inverse `atomic:generatedBy`) | +| `prov:wasInformedBy` | turn→turn | `prov:wasInformedBy` (keep) | +| `prov:wasDerivedFrom` | memory→source; session→session (fork) | `prov:wasDerivedFrom` (keep; fork metadata stays edge-level) | +| `prov:used` | tool_result→file | `prov:used` (keep) | +| `prov:wasAssociatedWith` | tool_result→goal | `atomic:linkedGoal` | +| `atom:informedBy` | intent→memory/source | `atomic:informedBy` ⊑ `prov:wasDerivedFrom` | +| `vault:explainedBy` | turn→provenance | `atomic:explainedBy` | +| `vault:hasTodo` | turn→todo | `atomic:hasTodo` | +| `BLOCKS` / `BLOCKED_BY` | intent↔intent | `atomic:blocks` / `atomic:blockedBy` (inverses; emit one, infer the other) | +| `ASSIGNED_TO` | intent→identity | `atomic:assignedTo` | +| `LINKED_INTENT` | goal→intent | `atomic:linkedIntent` (target now manifest-resolved) | +| `LINKED_GOAL` | intent→goal | `atomic:linkedGoal` | +| `REFERENCES` | entry→memory, entry→file | `atomic:references` | +| `HAS_LABEL` | intent→concept | `atomic:hasLabel` | +| `HAS_TASK` | intent→task | `atomic:hasTask` | +| `HAS_ACCEPTANCE_CRITERION` | intent→AC | `atomic:hasAcceptanceCriterion` | +| `HAS_SCOPE_IN` / `HAS_SCOPE_OUT` | intent→scope | `atomic:hasScopeIn` / `atomic:hasScopeOut` ⊑ `atomic:hasScopeItem` | +| `HAS_CONSTRAINT` | intent→constraint | `atomic:hasConstraint` | +| `SATISFIES` | task→AC | `atomic:satisfies` | +| `TOUCHES` | task→file | `atomic:touches` | +| `SCOPE_OUT_FILE` | intent→file | `atomic:scopeOutFile` | +| `REMEDIATES` | intent→intent | `atomic:remediates` | +| `REVIEWS` | review intent→intent | `atomic:reviews` (inverse `atomic:reviewedBy`) | +| `DEPENDS_ON` | change→change | `atomic:directlyDependsOnChange` ⊑ `atomic:dependsOn` | +| `DEPENDS_ON` | intent→intent | `atomic:dependsOnIntent` ⊑ `atomic:dependsOn` | +| `MODIFIES` | change→file | `atomic:modifiesFile` ⊑ `atomic:modifies` | +| `MODIFIES` | change→entity | `atomic:modifiesEntity` ⊑ `atomic:modifies` (emitted on **both** bulk and record paths) | +| `ON_VIEW` | change→view | `atomic:directlyOnView` — **recomputed from `VIEW_CHANGES[view]` own entries**, not the working copy's desired view | +| `ON_VIEW` | session→view | `atomic:sessionViewContext` | +| `CHILD_OF` | view→view | `atomic:childViewOf` | +| `VISIBLE_FROM` | (never emitted) | `atomic:visibleFrom` — query-time operator only in Phase 1 | +| `DEFINES` | file→entity | `atomic:defines` | +| `EXPORTS` | file→entity | `atomic:exports` ⊑ `atomic:defines` | +| `INCLUDES` | file→file | `atomic:includes` | +| `IMPORTS` | (never emitted) | `atomic:imports` ⊑ `atomic:includes` (declared; extraction split deferred) | +| `CALLS` | entity→entity | `atomic:calls` | +| `PART_OF` | file→module / module→module | `atomic:filePartOfModule` / `atomic:modulePartOfModule` ⊑ `atomic:partOf` | +| `AUTHORED_BY` | change/goal/intent→identity | `atomic:authoredBy` ⊑ `prov:wasAttributedTo` (intents now emit it from `author`/`created_by`) | +| `vault:status`, `vault:priority` | literal | `atomic:status`, `atomic:priority` (datatype properties) | +| `vault:appliesTo` | entity→concept | `atomic:appliesTo` | +| `vault:blocks`… duplicate predicate constants | — | removed (uppercase twins already mapped above) | +| `dc:type` | — | removed (never emitted) | +| `dc:creator` | — | `atomic:authoredBy` | +| `dc:date` | — | `dcterms:created` / `dcterms:modified` from typed source fields | +| `dc:description` | — | `dcterms:description` | +| `skos:broader`/`narrower`/`related` | — | kept | +| `skos:subject` | — | removed; `dcterms:subject` | +| `rdf:type` | — | real type assertions (§3.4) | + +**New Phase 1 edges** (no current equivalent): + +| Edge | Source fact | +|---|---| +| `Attestation atomic:coversChange Change` | `Attestation.changes_covered` | +| `Attestation atomic:attests SessionTurn` | attestation `session_id` + covered change ∈ turn `change_hashes` (D8: exact once the turn is recorded; interim inference only when unique) | +| `Attestation atomic:signedBy Identity` | `Attestation.signer` (`did:atomic` or MAC signer) | +| `Attestation atomic:previousAttestation Attestation` | `previous_attestation` | +| `Attestation atomic:verificationResult atomic:{Verified,Unverifiable,Failed}` | asserted **only** by the verifier pass (never inferred) | +| `Change atomic:implements Intent` | inferred: `generatedBy ∘ hasPlan` | +| `Change atomic:explainedBy ProvenanceGraph` | inferred: `generatedBy ∘ explainedBy` | +| `Change atomic:attestedBy Attestation` | inferred: inverse of `coversChange` | +| `Finding atomic:aboutResource X`, `atomic:findingCode` | historical gaps (`MissingHistoricalPlanId`, `MissingAttestation`, …) | + +### 3.4 Storage shape + +Keep the property-graph tables (they already serve FTS, neighbors, and UI) but make them +RDF-faithful: + +- `KgEdge.kind` → absolute predicate IRI. The write API takes a typed + `Predicate` (ontology constant), not `&str`; unknown IRIs are unrepresentable on write and + rejected on read. +- New field `KgEdge.graph: Asserted | Inferred` (serde default `Asserted`), with + `metadata.derivation = { rule, ontology_version, premises: [edge keys] }` on inferred edges. +- **Types in a dedicated index**, not as `rdf:type` edges: new multimap tables + `kg_types` (node IRI → class IRI) and `kg_types_rev` (class → node). Reason: storing + classes as edge targets would make every node a 2-hop neighbor of every same-typed node + under the kind-blind `kg_neighbors` BFS. RDF export emits them as `rdf:type` triples. + Entailed superclasses are stored with `graph = Inferred`. +- `KgNode.kind` is **removed**; display category derives from the most specific type. + `metadata.rdf_type` is removed. `KgNode.source` stays (it drives `kg_clear_derived`). +- `kg_index_meta["kg_generation"]` records `{ontology_version, projector_version, + inference_profile, built_at}` per the RFC's "projection generation". +- Register new tables in `schema.rs::visit_tables`; retire nothing (same table names, new + contents after rebuild). + +## 4. Milestones + +The RFC forbids a dual-write window, so M3, M4 and M7 land together on this branch and +merge as one release. M1, M2, M5 and M6 are additive and can be reviewed independently. + +### M1 — Ontology artifacts and term registry (additive, no behavior change) + +- `atomic-core/ontology/workflow-1.0.0.ttl`: OWL 2 RL ontology — classes (§3.3), object + and datatype properties with domain/range, inverses, subproperty hierarchy, the two + property chains, `atomic:dependsOn` as `owl:TransitiveProperty`, safe disjointness + (`AIContribution`/`WorkIntent` deferred with Phase 2 classes), and `atomic:Execution` / + `atomic:partOfTurn` declared but not projected (D11). +- `atomic-core/ontology/workflow-1.0.0-dl.ttl`: optional DL companion (can slip). +- Rewrite `atomic-core/src/pristine/ontology.rs` as `atomic_core::kg::ontology`: + `Class` and `Predicate` enums with `iri()`, `curie()`, domain/range metadata, and a + `PREFIXES` table for display compaction. Embed the `.ttl` via `include_str!`. +- `LEGACY_TERMS` table (test-only): every old constant → `Replacement::{One(term), + SplitByEndpoint([(domain, range, term)]), Removed(reason)}`. +- Tests: + - coverage: each of the 30 + 16 + 29 legacy constants appears exactly once in + `LEGACY_TERMS`, and every `One`/`Split` target exists in the TTL; + - TTL ↔ Rust consistency: every Rust term is declared in the TTL with matching + domain/range and vice versa (parse with `oxttl`/`oxrdf` as **dev-dependencies only**); + - no emitted string starts with `vault:`, `atom:`, or is SCREAMING_CASE (enforced again in + M4 over a rebuilt fixture repo). +- Doc: update the AGENTS.md table section; amend the RFC to state that frozen historical + document formats are read-only inputs, not a compatibility vocabulary (D3); publish + dereferenceable docs at the namespace (ops task, tracked separately). + +### M2 — Project identity and IRI minting + +- `atomic-canonical` ctx v2 on `https://atomic.dev/ns/workflow#` with an object + schema-version bump; v1 frozen for verification only; all new writes are v2 (D3). +- `atomic-canonical`: new `ProjectIdentity` node (`issuer` DID, `createdAt`, 256-bit random + id, schema version, proof), gate rules, JCS hash. Project ID = base32 of the random id. +- Storage and transport per D5: its own pristine node type plus a + `PROJECT_IDENTITY` table, carried by clone/push/pull as a repository-level object). + `atomic init` mints it; upgrading an existing repo mints it on the first writable open by + a signing identity, otherwise fails with `KgNeedsProjectIdentity` and an actionable message. +- `atomic project identity show|fork` (fork records `prov:wasDerivedFrom` origin). Rotation + chain deferred, but the object schema reserves `previousIdentity`. +- Import collision check: same id + different verified object → reject. +- `atomic_core::kg::iri`: mint/parse for every row of §3.2; property tests for round trip and + percent-encoding of paths with spaces, `#`, `:`, non-ASCII. +- Strict canonical base32 parser for content-addressed URNs (D4): rejects lowercase, + padding, wrong length and nonzero trailing bits; tested against `Hash::to_base32`. + +### M3 — Storage generation 2 (lands with M4/M7) + +- `KgEdge.graph`, typed `Predicate` write API, `kg_types`/`kg_types_rev` tables, removal of + `KgNode.kind`/`metadata.rdf_type`, read-side rejection of non-IRI kinds. +- Capability fence `kg-semweb-v1` in `capability.rs` so older binaries fail closed. +- `KG_GENERATION` constant plus `kg_index_meta["kg_generation"]`. The upgrade rebuild runs + automatically on the first writable open and via `atomic query reindex --full`, + regardless of repository size. The marker is written in the same transaction as the final + rebuild batch, so the rebuild happens exactly once per generation (D6). The upgrade path + clears all `kg_*` tables and runs M4's reconcile. +- `kg_neighbors` gains an optional predicate filter and direction; unfiltered behavior kept + for `vault context`/RAG (they now filter by type, not `kind`). + +### M4 — Projector rewrite and unified reconcile (lands with M3/M7) + +- New module `atomic-repository/src/kg/` with one projector per family, all emitting via + the typed API: + - `vault.rs` (from `vault_triples.rs`): intents (canonical `kind` → intent subclass; + lifecycle → `ManagedIntent`), memories, goals, skills, tool results, decomposition + children, `:::ref` edges split by edge name, author edges for intents. + - `vcs.rs` (from `vault_kg_enrich.rs`): views + `childViewOf`, `directlyOnView` from + `VIEW_CHANGES`, changes (full hash), `directlyDependsOnChange`, `modifiesFile`, + `authoredBy`, modules/files. + - `ast.rs`: entities, `defines`/`exports`/`includes`/`calls`, `modifiesEntity` on both + paths. + - Scope-qualified entity IDs (D10): add `Entity.scope` (enclosing containers) to all + seven `atomic-semantic` parsers plus normalized parameter types for overloading languages; + update `query callers` display and `extract_path_from_id` to read node data; print + `path:startLine-endLine` beside every entity ID; accept `path:line` and + `path#kind:scope::name` as CLI entity arguments. + - `session.rs`: move the vocabulary out of `atomic-core/.../session_kg.rs` (keep the + record-time hook in core, but it calls the shared emitter), sessions, turns, todos, + provenance graphs, `ManagedTurn` typing. + - `attestation.rs` (**new**): load `.attest` objects, emit Attestation node and §3.3 edges; + run signature verification (reuse `provenance_gate` verification + MAC provider) and + assert `verificationResult`. + - `findings.rs` (**new**): `MissingHistoricalPlanId`, `MissingProvenance`, + `MissingAttestation`, `UnverifiableAttestation` — never guessed links. +- `reconcile(scope)` pipeline (RFC "Sync and enrichment contract"): ingest vault → discover + VCS/session/provenance/attestation objects → project → verify → infer (M5) → validate + (M6) → publish generation + stats. `vault sync`, `query enrich`, `query reindex`, clone, + pull and import all call it (full or incremental scope). Record-time incremental + enrichment calls the same projectors for the new objects. +- Determinism: projectors sort inputs; output digest = blake3 over sorted canonical N-Triples. + +### M5 — OWL 2 RL inference (additive engine; wired in M4) + +- `atomic_core::kg::reason`: forward-chaining to fixpoint over rules **generated from the + TTL** (so the ontology is data): `rdfs:subClassOf`, `rdfs:subPropertyOf`, + `owl:inverseOf`, `rdfs:domain`/`rdfs:range` typing, 2-step `owl:propertyChainAxiom`. +- Transitive properties (`atomic:dependsOn`, `atomic:ancestorView`) are **not** + materialized; query operators compute closure from an explicit seed set. +- Every inferred edge/type carries a derivation record; inferred edges never overwrite an + asserted edge with the same key (asserted wins, derivation is appended). +- `verificationResult` and any proof-validity term are excluded from rule heads. +- Tests: golden fixtures per rule; idempotence (re-running adds nothing); order-independence. + +### M6 — Validation (SHACL as data + hand-coded checks) + +Per D1: + +- `atomic-core/ontology/shapes/projection-1.0.0.ttl` and `publication-1.0.0.ttl` + (normative). Projection profile: shortcut agreement, referential existence, full-hash IDs, + every generated change has `implements`/`explainedBy`/`attestedBy`. Publication profile: + managed turn has exactly one `hasPlan`; managed intent governs ≥1 turn; turn has exactly + one provenance graph; every generated change is covered by the session attestation; + attestation has signer + verified proof. +- Rust `kg::validate` implements each shape rule (one function per `sh:` rule ID, as in + `atomic-canonical/src/gate.rs`) over a small graph-access trait, and produces a report keyed + by shape/path IRIs. If it grows into a general SHACL evaluator, extract it into a + dedicated `atomic-core`-free crate that CB can consume or that can be published (D1). +- **Publication profile hooks into `provenance_gate.rs`** (the existing boundary for + shared-view insert, triage promotion, push, Git export) behind policy + `[provenance] require_plan = true` (default off). Absent `plan_id` fails publication. +- Projection failure → deterministic repair (re-run projectors for the offending subjects) + → revalidate → if still failing, generation is marked non-conforming and query output says + so. +- `atomic query export --format nquads|turtle [--inferred]` (hand-rolled serializer, no + deps) so the CB workflow can run real SHACL; differential fixture corpus checked into + `atomic-core/ontology/shapes/fixtures/` with expected conforms/violations for both the + Rust checks and CB. + +### M7 — Consumer cutover (lands with M3/M4) + +- `query_plan.rs`: new steps `Relation { predicate, direction }`, `DirectViewMembers`, + `ImplementingChanges`, `DependencyClosure { seeds }`, `TypeFilter`; `Filter` on `kind` + becomes type-based; plan JSON schema version bump. +- CLI `query`: `callers` → `atomic:calls`; graph colour map keyed by IRI; DOT/text output + compacts IRIs to CURIEs via `PREFIXES`; JSON emits absolute IRIs; `ask` prompt lists + canonical predicates and IRI forms. +- Triage (`atomic-repository/src/triage/report.rs` `kind_is`, `repository/triage.rs`), `vault_intent.rs` informedBy checks, + `vault/context.rs` memory filter (type `atomic:Memory`). +- libatomic daemon duplicates (`services_agent.rs`, `services_query.rs`); `rpc.rs`. +- Proto: document `KGEdge.relation` as an absolute IRI; add `types` repeated field to the node + message; bump knowledge service API version. +- No compatibility shim for JSON/RPC consumers; the release is announced as breaking (D7). +- Update ~110 test assertion sites; add a lint test that greps the workspace for legacy + kind literals outside `LEGACY_TERMS`. + +## 5. Acceptance tests for Phase 1 + +Mapped to RFC acceptance criteria 1-4, 7, 8: + +1. **Closure path (AC1):** integration test drives a managed agent turn with `plan_id` → + record → attest; asserts the change reaches intent, provenance graph, attestation, + session and turn via asserted edges, and that `implements`/`explainedBy`/`attestedBy` + shortcuts exist and agree. +2. **Repair (AC2):** delete all `kg_*` contents → `reconcile --full` → graph digest equals + the incrementally built graph's digest. +3. **Inference (AC3):** golden test for inverses, hierarchy and both property chains; + rerun is a no-op. +4. **Validation (AC4):** fixtures for each shape rule (positive + negative); publication + gate refuses push when `require_plan` and a managed turn lacks `plan_id`. +5. **Historical gaps (AC7):** pre-`plan_id` turn yields `MissingHistoricalPlanId` finding and + **no** `implements` edge. +6. **Coverage (AC8):** M1 coverage test + workspace lint for legacy literals. +7. **Membership:** `directlyOnView` for a change inserted into `dev` from `feature` lists it on + both views (both have it in `VIEW_CHANGES`) and never on the working copy's desired view + unless it is a member. + +## 6. Suggested order of work + +1. M1 (ontology + registry + coverage test) — establishes the vocabulary everyone reviews. +2. M2 (project identity + IRIs) — unblocks every projector. +3. M5 (reasoner) in parallel with M3 (storage) — independent. +4. M4 (projectors + reconcile) on top of M2/M3/M5. +5. M6 (validation + export + gate). +6. M7 (consumers), then the end-to-end acceptance suite; merge M3+M4+M6+M7 as one release. + +## 7. Risks + +- **Rebuild cost on large repos** (AST + CALLS fan-out). Mitigation: D6, progress output, + incremental reconcile by changed objects. +- **Edge-key growth** from absolute IRIs (~40-80 bytes per key). Acceptable for redb; can + intern predicate IRIs to `u16` later if measured. +- **Attestation verification on receiving remotes**: MAC attestations are unverifiable + without the key (per `provenance_gate`); they must project as `Unverifiable`, never + `Verified`, and the projection profile must treat that as a reported state, not a repair loop. +- **Out-of-repo clients** break on release day by design (D7); the API version bump makes the failure loud. +- **`atom:` vs `atomic:` confusion** in reviews: write D3 (v1 frozen and read-only, v2 for new objects) into AGENTS.md. + +## 8. Decisions and open questions + +### 8.1 Resolved + +**D1 — SHACL enforcement location: option (a).** Shapes ship as `.ttl` data and are the +normative source. Atomic implements hand-coded checks that mirror them and exports RDF for +the CB workflow. Implementation rule: +- Start with hand-coded checks in `kg::validate`, one function per shape rule (the + `atomic-canonical/src/gate.rs` pattern). +- Write the checks against a small graph-access trait (subjects, objects, types, + literals), not against redb or `KgNode` directly, so they can move later without a + rewrite. +- **Extraction trigger:** if the checks start to look like a real SHACL implementation + (generic `sh:` constraint dispatch, property-path evaluation, `sh:node`/`sh:sparql` + semantics, a shapes loader), move them into a dedicated crate (working name + `atomic-shacl`). That crate depends only on an RDF term model, never on `atomic-core`, so + CB can pull it in, it can be relocated, or it can be published separately. + +**D2 — OWL 2 RL implementation (RFC open question 1): option (a), written in-house.** Options considered: + +| Option | What it is | Cons | +|---|---|---| +| **(a) In-house rule engine** (chosen) | Forward-chaining over rules generated from our TTL: subClassOf/subPropertyOf, inverseOf, domain/range, 2-step property chains | We own correctness and must test every rule. It covers only the RL rules we implement, so the TTL loader must **reject** unsupported axioms (fail closed, the oxirs lesson) rather than silently ignore them | +| (b) `reasonable` crate | OWL 2 RL reasoner (Brick Schema project), Datalog on `datafrog` + `oxrdf` | Latest release is a prerelease (`0.5.0-a1`) from a small research-driven project. It materializes the full RL rule set, which produces many trivial triples (`owl:Thing`, reflexive `sameAs`, …) we'd filter out. It works on its own in-memory graph, so every reconcile copies the KG in and out of redb. It keeps no per-triple derivation records (rule + premises), which the RFC requires | +| (c) Datalog library (`datafrog` 2.0, `ascent` 0.8, `crepe`) + our rules | We write the RL rules; the library evaluates them to a fixpoint | Fixpoint evaluation is the easy part, and we still write and test every rule. `ascent`/`crepe` compile rules from macros, so the ontology stops being data. Derivation tracking is still ours. Worth it only if (a) turns out too slow | +| (d) DL/EL reasoners (`rustdl` OWL 2 DL on `horned-owl`, `whelk-el` OWL 2 EL) | Description-logic classification | Wrong job: they classify the ontology's class hierarchy rather than deriving instance edges, so they don't scale to deriving `implements` across a large KG. The RFC deliberately keeps DL off the conformance path | +| (e) RDF store + rules (`oxigraph` 0.5) | SPARQL store; rules written as `CONSTRUCT` queries looped to a fixpoint | No OWL reasoning built in. A second database next to redb (two copies to keep in sync), a large dependency tree, and we still write the rules by hand | +| (f) External (Jena, RDFox, GraphDB, or inference inside CB like SHACL) | Mature reasoners outside the binary | JVM or service dependency. Local queries need the derived shortcuts (`implements`, `attestedBy`) in the local KG, so offline repos would lose them, and query correctness would depend on an external workflow | + +Decision: (a). The TTL loader rejects any axiom outside the supported rule set. Option (c) +is the fallback only if profiling shows the fixpoint loop is a bottleneck. + +**D3 — Canonical document namespace: option (b), applied narrowly.** +- New signed objects use a **ctx v2** built on the canonical `atomic:` workflow namespace + (`https://atomic.dev/ns/workflow#`), under a bumped object schema version. +- Existing signed objects stay byte-for-byte unchanged. +- ctx v1 is **frozen** as a read-only historical input format. The KG projector translates + it to canonical terms through a fixed, tested mapping, so `atom:` terms never reach KG + storage, queries, APIs, or new objects. +- `https://atomic.dev/ns#` stays published with permanently fixed meaning (option (c) is + ruled out). +- Why (b) over (a): the RFC gives `atomic:` sole ownership of workflow concepts, requires + every producer to emit canonical terms, and rejects `atom:*` variants after migration. + Byte stability protects only *existing* objects, not future ones. Both options read v1 + forever; the difference is that (a) keeps producing the legacy format, while narrow (b) + closes it off. +- Consequences: + - `atomic-canonical` gains ctx v2 plus a schema-version bump for Intent, Memory, + delegation, PROV graph and `ProjectIdentity` objects. + - The gate accepts v1 only to verify existing objects. All new writes are v2, including a + revision of a v1 object (its `previousRevision` points at the v1 hash). + - Work lands in M2 (documents) and M4 (the v1 → canonical mapping in the vault projector). +- **RFC clarification required** (M1 doc task): frozen historical document formats are + read-only inputs, not a compatibility vocabulary. + +**D4 — Content-hash URN encoding: base32, algorithm-qualified, exactly specified.** +`urn:atomic:change:blake3-256:`. The RFC doesn't choose an encoding, so this is +a judgment call. +- Why base32: + - It's already used in `urn:atomic:change:`, so hex would add a second identity break on + top of the full-hash migration. + - It's shorter: 52 characters versus 64 for hex. + - RFC 4648 base32 is a multibase encoding, which fits the RFC's "multihash-compatible" goal. +- Normative encoding (the RFC's acceptance criteria depend on byte-identical IDs across + implementations): + 1. **Alphabet:** standard RFC 4648 §6 (`A–Z2–7`), not base32hex, Crockford, or a custom + variant. + 2. **Case:** uppercase is canonical (confirmed: keep current behavior). This is exactly what `Hash::to_base32` emits today + (`atomic-core/src/types/hash.rs:260`) and matches every existing change ID on disk and + in the CLI. The multibase equivalent is prefix `B`. + 3. **No padding:** 52 characters, and the final 4 bits must be zero. + 4. **Strict parsing:** reject any non-canonical form rather than normalizing it. The + existing `from_base32` is case-insensitive and doesn't check the trailing bits, so URN + parsing gets a new strict parser. Lenient prefix lookup in the CLI is unaffected. + 5. A test proves `Hash::to_base32` output round-trips through the strict parser. +- Scope: applies to every content-addressed URN (change, provenance, attestation, + project identity, file-version) and to `blake3-256:` evidence digests. +- Cost: standard BLAKE3 tools (`b3sum`) print hex, so external verifiers get a documented + conversion step. + +**D5 — Where the signed `ProjectId` lives: option (a), its own pristine node type.** The +first recommendation (a tracked `.vault/project/identity.jsonld` file) assumed snapshot +semantics. In atomic a `.vault/` file is content recorded *as a change* in the graph, so: +- its existence and bytes depend on which view you're on, because of the view filter; +- it can be unrecorded, edited by a later change, or put in conflict; +- it can diverge between views. + +Project identity is a repository-level fact that must be the same on every view and +survive insert/unrecord. It also can't be derived from changes, because changes are global +content-addressed objects: the same change can belong to many projects, and nothing +guarantees a unique genesis change. + +| Option | Cons | +|---|---| +| **(a) Dedicated content-addressed, signed object registered as its own pristine node type** (`node_type::PROJECT = 4`, next to CHANGE/TAG/ATTESTATION/PROVENANCE) (chosen). Bytes in the change store; a singleton pristine table `PROJECT_IDENTITY` holds the current hash plus the rotation chain. It is not in any view's `VIEW_CHANGES`, so no view filters, inserts or unrecords it | Needs a new transport object class. Today the push/pull contract excludes sidecars (`atomic-remote/src/sync.rs:476`), so clone/push/pull must learn to carry it, and the server should bind the identity hash to its `ProjectInfo.id` | +| (b) Metadata on the root view's `ViewState` | Still view-scoped: it travels and changes with one view, and other views can't verify it independently | +| (c) Server-assigned `ProjectInfo.id` (UUID) | Requires a remote, so local-only repos have no identity; it's not signed by the user and can't be verified offline | +| ~~(d) Tracked `.vault/` file~~ | Rejected: view-dependent, as explained above | + +Under (a), fork is an explicit command that mints a new object with `prov:wasDerivedFrom` +pointing at the origin. The issuer DID binds the project to an identity now; org binding +comes later through delegation (`ResourceRef`). + +**D6 — KG rebuild on upgrade: both triggers, regardless of repository size.** The rebuild +runs automatically on the first writable open, and `atomic query reindex --full` runs it +explicitly. It must happen **exactly once** per KG generation: +- A compiled `KG_GENERATION` constant. A rebuild runs only when the stored + `kg_index_meta["kg_generation"]` is lower. +- The generation marker is written in the **same write transaction as the final rebuild + batch**. An interrupted rebuild leaves no marker and restarts cleanly (clear, then + rebuild) on the next writable open. It never leaves a half-marked KG. +- Repositories created by the new binary (`init`, `clone`) are stamped with the current + generation at creation and never trigger an upgrade rebuild. +- Pull, import, incremental reconcile and `reindex --full` never lower or clear the marker; + `--full` re-stamps the same generation. +- `KG_GENERATION` bumps only on breaking ontology or projector changes, not on every + release. Routine upgrades don't rebuild. +- Read-only opens on an old-generation KG return `KgNeedsRebuild`, naming the command to run. +- Tests: a second open after the upgrade performs zero rebuild work; a rebuild killed + midway completes on the next open; a fresh clone never rebuilds. + +**D7 — External JSON/RPC clients:** this release is announced loudly as breaking, so there +is no compatibility shim and no coordination gate on `noname`/`the-hive`. The API version +bump stays, so stale clients fail loudly rather than misreading data. + +**D8 — Attestation → turn mapping: option (b), with (a) as the interim.** Record the turn +number in the attestation format at capture time, because the RFC classes turn identity as +a capture-time fact. Until the format change lands, infer the turn only when exactly one +turn's `change_hashes` ⊆ `changes_covered`. Otherwise emit an `AmbiguousAttestationTurn` +finding and no `attests` edge. + +**D9 — Domain separation: provided by project identity, used later by federation.** +Each atomic repository is its own domain; federation is how domains are kept apart in +cross-project search (e.g. `testing/hello-world` vs `production/hello-world`, and later +scoping by identity/org so a caller never searches someone else's projects). Content +hashes stay global: the same change bytes hash the same in every project, which insert, +pull and clone depend on. +- Phase 1 obligation: a durable, signed project identity (D5) that later phases can read, + plus project-qualified IRIs for mutable resources (§3.2). +- Deferred to Phase 5: the catalog, named project graphs, explicit query scope and + authorization that use that identity to keep projects separate in federated queries. +- No change-format or hash-construction change in Phase 1. + +**D10 — Source-entity identity: option (b), scope-qualified IDs.** Where it comes from: +- The RFC's node model lists Entity as a "project-qualified **stable** entity URN" from + the syntax/semantic index, and goal 6 requires portable identities. +- Today's entity IDs are `entity:{file}:{name}:{line}` (`vault_kg_enrich.rs:587`), so they + contain the line number. Adding one line above a function changes its ID. +- As a result, `modifiesEntity` edges from older changes point at IDs that no longer + exist, one function can end up with several IDs over time, and a full rebuild produces + different IDs than incremental enrichment. That breaks the "full rebuild equals + incremental" acceptance test and blame-style queries. +- The line number is also doing a hidden job: the extractor (`atomic-semantic`, `Entity` + at `entity.rs:123`) emits methods by bare name (`fmt`, `find`) with no enclosing type, so + `impl Display for User { fn fmt }` and `impl Display for Order { fn fmt }` in one file are + told apart only by their lines. + +**Decision: a scope-qualified name instead of the line.** + +```text +urn:atomic:project::entity:#:::[()][~] +``` + +| Part | Meaning | +|---|---| +| `` | file path, percent-encoded (as today) | +| `` | `EntityKind` (`function`, `method`, `class`, `interface`, `type-alias`, `const`, `enum`, `module`, …) | +| `::` | the chain of enclosing containers plus the name, joined by `::` in every language, each segment percent-encoded | +| `()` | only in languages with overloading (Java, C++, Swift, TypeScript overload signatures): normalized parameter types from the signature | +| `~` | last-resort ordinal, by order of appearance, when everything above still collides (e.g. two `#[cfg]` variants of the same `fn`) | + +Line, end line, column and signature become literals on the node (`atomic:startLine`, …). + +The file stays in the ID. The examples below show only the fragment after `#`; the full ID +for the `fmt` example is +`urn:atomic:project::entity:src%2Flib.rs#method:::fmt`. + +**Finding code still works, and line numbers stay visible.** Current consumers: + +| Consumer | Where it gets the line today | Change needed | +|---|---|---| +| `query entities ` | parses the file live with tree-sitter; never reads KG IDs (`query/mod.rs:1292-1342`) | none | +| `query neighbors` / `search` text and JSON | node metadata `line`/`end_line` (`query/mod.rs:585, 648-665`) | none: the line stays on the node | +| `query callers` display | **parses the ID**, stripping the trailing `:line` (`query/mod.rs:1267-1277`) | format `path:line` from node data | +| Search ranking `extract_path_from_id` | **parses the ID** with `rsplitn(3, ':')` (`vault_triples.rs:629-645`) | read the path from node data | +| Agents following the code-intelligence skill | read the line straight out of the copied ID | every result line prints `path:startLine-endLine` beside the ID | + +Display rule: anywhere an entity is shown to a person or an agent, print +`path:startLine-endLine` with the ID. That line is always current, because it is refreshed +on every enrichment, whereas a line embedded in an ID is frozen at the moment the ID was +copied. + +Lookup by location: entity arguments in the CLI also accept `path:line` (for example +`atomic query neighbors src/lib.rs:55`) and the shorthand `path#kind:scope::name`. Both +resolve to the full IRI within the current project, so nobody has to type the URN. + +Examples: + +| Source | Today | Proposed fragment | +|---|---|---| +| `pub fn greet()` in `src/lib.rs` | `entity:src/lib.rs:greet:12` | `#function:greet` | +| `impl User { fn new() }` | `entity:src/lib.rs:new:40` | `#method:User::new` | +| `impl Display for User { fn fmt }` | `entity:src/lib.rs:fmt:55` | `#method:::fmt` | +| `mod tests { fn it_works() }` | `entity:src/lib.rs:it_works:90` | `#function:tests::it_works` | +| Python `class Repo: def get(self)` | `entity:repo.py:get:8` | `#method:Repo::get` | +| Java `void put(int)` / `void put(String)` | two line-based IDs | `#method:Cache::put(int)` / `#method:Cache::put(String)` | +| `#[cfg(unix)] fn open` / `#[cfg(windows)] fn open` | two line-based IDs | `#function:open` / `#function:open~2` | + +How it behaves compared with line-based IDs: + +| Edit | Line-based ID | Scope-qualified ID | +|---|---|---| +| Add or remove lines above, edit the body, reformat | changes | **stable** | +| Rename the function | changes | changes (a real semantic change; the semantic diff's `ChangeType::Renamed` can later link old → new with `atomic:renamedFrom`) | +| Move it to another file, or into/out of a type | changes | changes | +| Change parameter types (overloading languages only) | stable unless lines move | changes | + +The payoff is that `modifiesEntity` edges from every change that ever touched a function +converge on one node, so "history of this function" becomes a direct query, and full +rebuild and incremental enrichment produce identical IDs. + +Cost: each of the seven parsers in `atomic-semantic/src/parsers/` (rust, typescript, +python, go, java, cpp, swift) must record its enclosing scope chain (a new +`Entity.scope: Vec` field) and, for overloading languages, normalized parameter +types. `CALLS` resolution stays name-based as today. + +| Option | Cons | +|---|---| +| (a) Keep line-based IDs | Breaks rebuild determinism and history queries, as described | +| **(b) Scope-qualified IDs, as above** (chosen) | Extractor work in all seven parsers; the ordinal can shift if a new same-named duplicate appears earlier in the file (rare) | +| (c) CRDT `LeafId` / syntax-index stable IDs | Survives renames too, but depends on the semantic layer being finished (AGENTS.md: "in progress"), which would put Phase 1 behind it | + +Work lands in M4 (extractor scope chains, ID minting, `callers` display, search ranking, +`path:line` and shorthand lookup). + +**D11 — `Execution` nodes: option (a).** Declare `atomic:Execution` and `atomic:partOfTurn` +in the ontology only. Project nothing until capture records structured executions +(`ProvActivityInput.used` is always empty today). + +All Phase 1 decisions are resolved; there are no open questions. diff --git a/docs/knowledge-graph-closure-rfc.md b/docs/knowledge-graph-closure-rfc.md new file mode 100644 index 00000000..e4a963f5 --- /dev/null +++ b/docs/knowledge-graph-closure-rfc.md @@ -0,0 +1,1307 @@ +# RFC: End-to-End Intent, Provenance, Attestation, Change, and Resource Closure + +> Status: **draft / proposal.** This RFC extends +> [`provenance-graph-plan.md`](./provenance-graph-plan.md) and +> [`vault-sync-design.md`](./vault-sync-design.md) with an enforceable projection +> contract for the project knowledge graph. + +## Summary + +Atomic SHALL expose a complete, queryable path from the work item that authorized a +change to the resources and executions evidenced by that change: + +```text +Intent → Session → Turn → Provenance → Change → File/Entity + ↘ Attestation ↗ ↘ Execution evidence +``` + +The content-addressed change, session ledger, provenance graph, attestation, and +canonical vault objects remain authoritative. The knowledge graph (KG) is a +deterministic, rebuildable read model over those objects. Recording must capture facts +that cannot be inferred later; enrichment must project every captured fact and repair +missing derived nodes and edges. + +The target user experience includes questions such as: + +- What intent introduced a migration or execution that deletes a database table? +- Which model and session produced changes implementing this bug intent? +- Which approved review covered the exact closure promoted into `main`? +- What did the dependency-closed implementation of this feature cost? +- Which migration or execution evidence has no independently attested review? + +## Motivation + +Atomic already persists most of the required facts, but they are split across several +surfaces: + +- Vault entries describe intents, acceptance criteria, tasks, goals, and memories. +- `SessionTurn` records contain `plan_id`, `goal`, `provenance_hash`, and + `change_hashes`. +- Changes contain authors, dependencies, graph/file operations, and inline AI + provenance such as model, tokens, cost, and session identifiers. +- Provenance sidecars contain the causal decision ledger. +- Attestations identify sessions, agents, covered changes, and cryptographic proof. +- CRDT and syntax layers can describe human-readable code operations. + +Partial enrichment makes these facts individually visible without guaranteeing a +continuous path between them. Generic neighborhood traversal also cannot distinguish +membership from incidental reachability. A complete projection contract is needed +before KG queries can be treated as audit evidence. + +## Goals + +1. Provide a continuous graph path from an intent to every implementing change, affected + resource, and recorded execution. +2. Preserve intermediate session, turn, provenance, attestation, review, and change + nodes rather than replacing them with opaque metadata blobs. +3. Make full enrichment idempotently reconstruct the same KG from canonical objects. +4. Validate projection completeness and consistency with SHACL. +5. Support direct membership and dependency-closure queries without relying on graph + depth as a proxy for semantics. +6. Remain portable: use hashes, URNs, stable vault IDs, and paths—not repository-local + `NodeId` values. + +## Non-goals + +- Treating the KG as the authoritative store. +- Guessing an intent for historical changes that did not record one. +- Persisting domain-specific source operations or inferred effects in the project KG. +- Requiring AI-specific fields for human-authored changes. + +## Authoritative facts versus derived facts + +### Must be captured at record/publication time + +These claims cannot be reconstructed reliably and MUST be persisted before publication: + +- governing intent/plan ID; +- session and turn identity; +- exact generated change hashes; +- provenance graph hash; +- exact attestation coverage and signer; +- reviewed change-set digest and verdict; +- model/provider response metadata when AI participated. + +If policy requires a governing intent, publication MUST fail when `plan_id` is absent. +The system must not infer authorization from a nearby timestamp, matching view name, or +similar commit message. + +### Deterministically derived during enrichment + +The following are rebuildable read-model facts: + +- direct `change → intent` and `change → provenance` convenience edges; +- dependency-closed intent implementation sets; +- file/entity links derived from recorded changes; +- generic execution and artifact-role links explicitly present in provenance; +- aggregate token and cost values; +- stale-review findings derived from reviewed and current closure digests. + +## Canonical node model + +| Node kind | Portable identity | Authority | +|---|---|---| +| Intent | `urn:atomic:project::intent:` | Canonical vault intent | +| Goal | `urn:atomic:project::goal:` | Canonical vault goal | +| Session | `urn:atomic:project::session:` | Session record | +| Session turn | `urn:atomic:project::session::turn:` | Session turn record | +| Provenance graph | `urn:atomic:provenance:blake3-256:` | Content-addressed sidecar | +| Attestation | `urn:atomic:attestation:blake3-256:` | Signed attestation object | +| Review intent | `urn:atomic:project::intent:` | Canonical review intent | +| Change | `urn:atomic:change:blake3-256:` | Content-addressed change | +| File | `urn:atomic:project::file:` | Repository tree/semantic trunk | +| Entity | project-qualified stable entity URN | Syntax/semantic index | +| Execution | `urn:atomic:project::execution:` | Recorded provenance activity | +| View | `urn:atomic:project::view:` | View state | + +Full hashes are canonical. Abbreviated hashes are display labels only and MUST NOT be +used as durable identities. + +## Required edge model + +```text +Intent ──atomic:governsTurn──────────────▶ SessionTurn +Session ──atomic:hasTurn──────────────────▶ SessionTurn +SessionTurn ──atomic:hasPlan──────────────────▶ Intent +SessionTurn ──atomic:explainedBy──────────────▶ ProvenanceGraph +SessionTurn ──atomic:generatedChange──────────▶ Change +Attestation ──atomic:attests──────────────────▶ SessionTurn +Attestation ──atomic:coversChange─────────────▶ Change +Change ──atomic:implements───────────────▶ Intent +Change ──atomic:explainedBy──────────────▶ ProvenanceGraph +Change ──atomic:attestedBy───────────────▶ Attestation +Change ──atomic:directlyDependsOnChange─▶ Change +Change ──atomic:directlyOnView───────────▶ View +Change ──atomic:modifiesFile─────────────▶ File +Change ──atomic:modifiesEntity───────────▶ Entity +Execution ──prov:used───────────────────────▶ File/Entity +Execution ──atomic:partOfTurn────────────────▶ SessionTurn +ReviewIntent ──atomic:reviews──────────────────▶ Intent +ReviewIntent ──atomic:reviewsChange────────────▶ Change +ReviewIntent ──atomic:reviewsClosure───────────▶ ChangeSetDigest +``` + +`atomic:implements`, direct `atomic:explainedBy`, and direct `atomic:attestedBy` are derived convenience +edges. Their source intermediate nodes remain present, and validation ensures the +shortcuts agree with the authoritative path. + +## Membership is not neighborhood + +A graph depth is never a membership predicate. Commands and query plans SHOULD expose +named relations or selectors: + +- direct changes recorded on a view: `change atomic:directlyOnView view`; +- effective changes visible in a view: `atomic:visibleFrom` or a view-closure operator; +- implementation changes for an intent: `change atomic:implements intent`; +- direct dependencies: `atomic:directlyDependsOnChange`; +- dependency closure: transitive `atomic:dependsOn` from an explicitly selected seed set. + +This prevents a depth-two query from counting dependencies, parent-view changes, or +changes connected through a session as if all belonged directly to the selected view. + +## AI provenance and attestation projection + +Each AI contribution SHOULD be represented as a provenance contribution node rather +than only a JSON field: + +```text +Change ──atomic:hasContribution──▶ AIContribution +AIContribution ──atomic:usedModel──▶ Model +AIContribution ──atomic:inSession──▶ Session +``` + +An `AIContribution` carries typed literals for vendor, model version, tool, suggestion +type, input/output/total tokens, cost and currency, request ID, timestamp, agent mode, +finish reason, and step count. Large reasoning and transcript content remains in +content-addressed objects referenced by hash; it is not duplicated into every KG node. + +Change-level `cost` and token totals are materialized aggregates for query convenience. +They MUST equal the sum of linked contributions and MUST not include dependency costs +unless a closure query explicitly requests them. + +Attestations remain distinct from AI contribution metadata. An attestation represents a +signed claim and includes signer DID, proof, session, covered changes, prior attestation, +and verification status. + +## Evidence-grounded semantic queries with `query ask` + +The persistent KG stops at durable, generic facts: an intent governed a turn, the turn +generated a change, the change modified a file/entity, and provenance recorded an +execution that used an artifact. It does not create a node for each SQL statement, API +change, security interpretation, or inferred effect. + +Natural-language domain questions are answered by `atomic vault query ask`. The LLM has +two constrained roles—planning and interpretation—and no graph-write authority. + +### Runtime contract + +For a question such as: + +```text +What intent introduced a migration that deletes a database table? +``` + +`query ask` executes this bounded loop: + +1. **Build a result shape.** The model translates the question into required structural + bindings and semantic fields, including intent, change, file/execution, exact source + version, evidence range, interpretation, and uncertainty. +2. **Build a constrained query plan.** The model selects allow-listed operators such as + intent/change traversal, direct view membership, dependency closure, changed-hunk + retrieval, content search, and execution lookup. It cannot submit arbitrary storage + commands or mutate the graph. +3. **Execute structure deterministically.** Atomic executes the plan against canonical + KG relations and returns bounded candidate changes, files, hunks, and executions. +4. **Retrieve exact evidence.** Atomic loads the content-addressed version associated + with each change—not merely the current working copy—and includes stable byte/line + ranges and whether text was added, removed, or executed. +5. **Interpret bounded evidence.** The LLM fills the result shape using only supplied + candidates. Deterministic parsers may be exposed as optional tools, but are not + required to define persistent ontology terms. +6. **Validate the envelope.** Atomic verifies every returned IRI/hash exists, every file + is linked to the cited change, every range exists in the cited object, every quoted + string matches evidence, and every intent/change/provenance edge is present. +7. **Return ephemeral semantics.** Results are labeled `model_interpretation`, include + model/provider, prompt/query-plan hash, evidence hashes, source graphs, confidence or + abstention, and completeness/partial-scope status. They do not enter OWL inference or + SHACL conformance. + +The model-produced plan is a versioned typed AST validated against a closed JSON Schema +before authorization or execution. Unknown fields, operators, relation IRIs, unbounded +limits, raw commands, raw storage keys, arbitrary SPARQL, and unsupported schema versions +are rejected. Each operator has typed parameters and hard server-side cardinality, byte, +time, token, cost, and recursion limits that model output cannot raise. Authorization is +computed over the validated plan and every expansion. The executor never interpolates +model strings into shell, SQL, RDF-store, or filesystem commands. + +The model may request another bounded retrieval round through the same schema when +evidence is insufficient. The planner reports truncation and denied expansions; it does +not silently claim completeness. + +### Result shape + +A representative result is: + +```json +{ + "status": "model_interpretation", + "verified": false, + "intent": "urn:atomic:project:...:intent:...", + "change": "urn:atomic:change:blake3-256:...", + "file": "urn:atomic:project:...:file:migrations/042.sql", + "evidence_object": "urn:atomic:file-version:blake3-256:...", + "evidence": { + "side": "post-change", + "change_role": "added", + "coordinate_unit": "utf8-byte", + "interval": "half-open", + "start": 142, + "end": 166, + "bytes_digest": "blake3-256:...", + "text": "DROP TABLE public.users;" + }, + "interpretation": { + "operation": "drop-table", + "resource": "public.users" + }, + "model": "...", + "query_plan_hash": "...", + "confidence": 0.94 +} +``` + +Evidence citations identify the canonical evidence-object type and hash, pre-change or +post-change side (or execution-record object), hunk role, UTF-8 byte coordinates using a +half-open `[start, end)` interval, and a domain-separated digest of the exact cited bytes. +Line numbers may be displayed but are derived and non-canonical. Removed and replaced +text cites the pre-change object; added text cites the post-change object. Execution +evidence cites the immutable execution record plus its payload span under the record's +canonical encoding. + +`verified: false` means the semantic interpretation is not a canonical fact. Machine +validation recomputes the cited byte digest and proves the evidence and structural path +exist; it does not prove the model interpreted them correctly. Model confidence never upgrades epistemic status. + +### Virtual result graph + +Structured results may be exposed as a request-scoped virtual graph under the +`atomic-query:` namespace so SPARQL/structured-plan consumers can join them with the +canonical KG. This graph disappears with the query envelope and is excluded from OWL and +SHACL inputs. + +### Explicit promotion + +If an interpretation must become durable, a user or policy explicitly promotes the +result into an attested finding, review observation, remediation intent, or memory. The +durable object references the query-plan hash and exact evidence; it records who accepted +the interpretation. Promotion never rewrites the original answer into an objective code +fact, and there is no automatic promotion based on confidence. + +### Example traversal + +```text +atomic vault query ask "What intent introduced a migration that deletes a table?" + → plan: intent ← implements ← change → modifiesFile → exact changed hunks + → optional: execution → prov:used → file + → LLM: interpret only bounded evidence + → validate citations and structural bindings + → return ephemeral shaped answer +``` + +This preserves provenance's purpose: Atomic supplies the factual chain and exact lines; +the LLM supplies question-specific semantics with explicit uncertainty. + +## Example query: cost of changes directly in a view + +```text +view(name) + → inverse atomic:directlyOnView + → changes + → sum(change.cost) +``` + +For effective visible cost, callers explicitly request dependency and ancestor closure: + +```text +view(name) + → visible-change closure + → distinct changes + → sum(change.cost) +``` + +The query engine MUST deduplicate changes by full hash before aggregation. + +## Review linkage + +Feature and bug intents MAY be governed by a policy requiring an approved review intent +before completion. A review records: + +- reviewed intent; +- exact dependency-closed change-set digest; +- each reviewed change; +- verdict; +- reviewer identity and attestation; +- remediation intents/findings; +- timestamp and policy version. + +Adding or replacing an implementation change changes the closure digest and makes the +review stale. A stale review cannot satisfy the completion shape. + +## OWL ontology and inference + +SHACL alone describes whether a concrete graph satisfies operational requirements. It +does not provide the domain theory needed to classify resources, infer inverse +relationships, or derive a change-to-intent path from authoritative intermediate facts. +Atomic SHALL therefore publish this model as a versioned OWL ontology and use SHACL as +the complementary validation layer. + +### Reasoning profile + +The normative ontology SHOULD remain within **OWL 2 RL** so enrichment and query systems +can implement deterministic rule-based materialization without a heavyweight tableau +reasoner. A complete OWL 2 DL representation MAY also be published for external tools, +but the Atomic conformance path depends only on the declared RL subset. + +The ontology IRI and version IRI are immutable, for example: + +```turtle +atomic:provenance-closure + a owl:Ontology ; + owl:versionIRI atomic:provenance-closure/1.0.0 ; + owl:imports , + . +``` + +A KG projection records the ontology version, projector version, and inference profile +used to produce it. Ontology upgrades create a new projection generation; they do not +change canonical historical objects. + +### Namespace ownership + +The canonical `atomic:` namespace is the **Atomic workflow ontology**. Its permanent base +IRI is `https://atomic.dev/ns/workflow#` and SHALL be dereferenceable with content +negotiation for human-readable documentation and canonical RDF serializations. It owns every concept whose meaning comes from +the Atomic workflow, including Intent, Change, Memory, Attestation, Review, View, Session, +Turn, acceptance criteria, tasks, scopes, constraints, and their workflow-specific +relations. + +PROV-O is the shared crossover vocabulary. Atomic reuses `prov:Activity`, `prov:Entity`, +`prov:Agent`, `prov:used`, `prov:generated`, `prov:wasGeneratedBy`, +`prov:wasDerivedFrom`, `prov:wasInformedBy`, and `prov:wasAssociatedWith` wherever their +standard semantics fit. Atomic does not mint equivalent workflow-prefixed copies of PROV +terms. A narrower Atomic property is introduced only when the workflow relation is more +specific than PROV-O—for example, `atomic:generatedChange rdfs:subPropertyOf +prov:generated`—and its narrower domain/range is explicit. + +This gives cross-project and external tools one shared provenance bridge while keeping +all Atomic-specific lifecycle semantics in one coherent workflow namespace. Other +external vocabularies are reused only when their semantics match exactly; they do not +split Atomic workflow ownership into separate intent, change, memory, or attestation +namespaces. + +### Core classes + +```turtle +atomic:WorkIntent a owl:Class ; rdfs:subClassOf prov:Entity . +atomic:ManagedIntent a owl:Class ; rdfs:subClassOf atomic:WorkIntent . +atomic:FeatureIntent a owl:Class ; rdfs:subClassOf atomic:ManagedIntent . +atomic:BugIntent a owl:Class ; rdfs:subClassOf atomic:ManagedIntent . +atomic:ReviewIntent a owl:Class ; rdfs:subClassOf atomic:ManagedIntent . +atomic:Session a owl:Class ; rdfs:subClassOf prov:Activity . +atomic:SessionTurn a owl:Class ; rdfs:subClassOf prov:Activity . +atomic:ManagedTurn a owl:Class ; rdfs:subClassOf atomic:SessionTurn . +atomic:Goal a owl:Class ; rdfs:subClassOf prov:Entity . +atomic:Memory a owl:Class ; rdfs:subClassOf prov:Entity . +atomic:Identity a owl:Class ; rdfs:subClassOf prov:Agent . +atomic:File a owl:Class ; rdfs:subClassOf prov:Entity . +atomic:Change a owl:Class ; rdfs:subClassOf prov:Entity . +atomic:ProvenanceGraph a owl:Class ; rdfs:subClassOf prov:Entity . +atomic:Attestation a owl:Class ; rdfs:subClassOf prov:Entity . +atomic:AIContribution a owl:Class ; rdfs:subClassOf prov:Entity . +atomic:Execution a owl:Class ; rdfs:subClassOf prov:Activity . +``` + +The ontology SHOULD declare clearly impossible overlaps as disjoint, but only where the +claim is stable across future evolution. For example, `AIContribution` and +`WorkIntent` may be disjoint. `FeatureIntent`, `BugIntent`, and `ReviewIntent` SHOULD +not initially be pairwise disjoint because an imported policy may legitimately assign +multiple roles to one intent. + +### Object properties + +```turtle +atomic:hasPlan a owl:ObjectProperty ; + rdfs:domain atomic:SessionTurn ; + rdfs:range atomic:WorkIntent ; + owl:inverseOf atomic:governsTurn . + +atomic:hasTurn a owl:ObjectProperty ; + rdfs:domain atomic:Session ; + rdfs:range atomic:SessionTurn ; + owl:inverseOf atomic:turnOfSession . + +atomic:ManagedIntent rdfs:subClassOf + [ a owl:Restriction ; + owl:onProperty atomic:governsTurn ; + owl:minQualifiedCardinality "1"^^xsd:nonNegativeInteger ; + owl:onClass atomic:SessionTurn ] . + +atomic:ManagedTurn rdfs:subClassOf + [ a owl:Restriction ; + owl:onProperty atomic:hasPlan ; + owl:qualifiedCardinality "1"^^xsd:nonNegativeInteger ; + owl:onClass atomic:WorkIntent ] . + +atomic:generatedChange a owl:ObjectProperty ; + rdfs:subPropertyOf prov:generated ; + rdfs:domain atomic:SessionTurn ; + rdfs:range atomic:Change ; + owl:inverseOf atomic:generatedBy . + +atomic:implements a owl:ObjectProperty ; + rdfs:domain atomic:Change ; + rdfs:range atomic:WorkIntent ; + owl:inverseOf atomic:implementedBy . + +atomic:modifiesFile a owl:ObjectProperty ; + rdfs:domain atomic:Change ; + rdfs:range atomic:File ; + rdfs:subPropertyOf atomic:modifies . + +atomic:modifiesEntity a owl:ObjectProperty ; + rdfs:domain atomic:Change ; + rdfs:range atomic:SourceEntity ; + rdfs:subPropertyOf atomic:modifies . + +atomic:partOfTurn a owl:ObjectProperty ; + rdfs:domain atomic:Execution ; + rdfs:range atomic:SessionTurn . + +atomic:authoredBy a owl:ObjectProperty ; + rdfs:subPropertyOf prov:wasAttributedTo ; + rdfs:domain prov:Entity ; + rdfs:range atomic:Identity . + +atomic:attestedBy a owl:ObjectProperty ; + rdfs:domain atomic:Change ; + rdfs:range atomic:Attestation ; + owl:inverseOf atomic:coversChange . + +atomic:reviews a owl:ObjectProperty ; + rdfs:domain atomic:ReviewIntent ; + rdfs:range atomic:WorkIntent ; + owl:inverseOf atomic:reviewedBy . +``` + +The workflow cardinality is **one managed Intent → one or more Turns**, and every managed +Turn → exactly one governing Intent. A turn may generate multiple changes. A backlog +intent may have zero turns before work starts; transition to a managed/active lifecycle +state activates the minimum-one-turn SHACL gate. OWL describes this meaning under the +open-world assumption, while SHACL enforces it on concrete lifecycle transitions. + +`dependsOn` MAY be declared transitive for semantic queries, but direct dependency edges +must remain distinguishable. The implementation SHOULD use domain-specific direct properties and an inferred broad +`dependsOn` property rather than materializing transitive closure into the authoritative +edge family: + +```turtle +atomic:directlyDependsOnChange rdfs:subPropertyOf atomic:dependsOn . +atomic:dependsOnIntent rdfs:subPropertyOf atomic:dependsOn . +atomic:dependsOn a owl:TransitiveProperty . +``` + +### Property-chain axioms + +OWL property chains derive query-friendly shortcuts from authoritative paths. Given: + +```text +SessionTurn ──generatedChange──▶ Change +SessionTurn ──hasPlan──────────▶ Intent +``` + +The OWL 2 DL companion can express the inverse traversal directly. The normative RL +projection instead emits the explicit inverse `atomic:generatedBy` and infers +`Change implements Intent` with a regular property chain: + +```turtle +atomic:generatedBy owl:inverseOf atomic:generatedChange . +atomic:implements owl:propertyChainAxiom ( + atomic:generatedBy + atomic:hasPlan +) . +``` + +Attestation and provenance shortcuts MAY be inferred through equivalent chains when the +intermediate relationship is unambiguous. Property chains MUST NOT manufacture +cryptographic validity: `verifiedAttestation` is asserted only by signature +verification, never inferred merely because a proof node exists. + +### Datatype properties + +Token counts use non-negative integers; money uses exact decimal values plus an explicit +currency rather than binary floating point: + +```turtle +atomic:inputTokens a owl:DatatypeProperty ; rdfs:range xsd:nonNegativeInteger . +atomic:outputTokens a owl:DatatypeProperty ; rdfs:range xsd:nonNegativeInteger . +atomic:totalTokens a owl:DatatypeProperty ; rdfs:range xsd:nonNegativeInteger . +atomic:costAmount a owl:DatatypeProperty ; rdfs:range xsd:decimal . +atomic:currency a owl:DatatypeProperty ; rdfs:range xsd:string . +``` + +OWL range declarations infer types; they do not reject malformed literals. SHACL remains +responsible for datatype, bounds, cardinality, and currency constraints. + +### Current KG replacement and coverage matrix + +This matrix is normative for a **breaking Semantic Web-native cutover**. It covers the +terms currently defined in `atomic-core/src/pristine/ontology.rs` and the relations +emitted by vault, VCS, AST, and session-ledger enrichment. Current strings are inventory +inputs, not supported aliases. Every producer, stored edge, query, and consumer SHALL be +rewritten to the canonical RDF/OWL term; old spellings are rejected after migration. + +Atomic intentionally provides no `atomic-legacy` ontology, `owl:equivalentProperty` +bridge, uppercase compatibility API, or dual-write period. Repository format and query +API versions advance atomically with this change. + +#### Existing class coverage + +| Current representation | Observed resources | Required canonical replacement | +|---|---|---| +| `vault:Goal`, `kind=goal` | `goal:*` | Replace with `rdf:type atomic:Goal`; remove class string and kind-as-type semantics | +| `vault:Intent`, `kind=intent` | `intent:*` | Replace with `rdf:type atomic:WorkIntent` | +| `vault:Memory`, `kind=memory` | `memory:*` | Replace with `rdf:type atomic:Memory` | +| `vault:Skill`, `kind=skill` | `skill:*` | Replace with `rdf:type atomic:Skill` | +| `vault:ToolResult`, `kind=tool_result` | `tool:*` | Replace with `rdf:type atomic:ToolResult` | +| `vault:File`, `kind=file` | `file:` | Replace with `rdf:type atomic:File` | +| `vault:Identity`, `kind=identity` | `identity:*` | Replace with `rdf:type atomic:Identity`; DID resources are individuals of this class | +| `vault:Concept`, `kind=concept` | `concept:*` | Replace with `rdf:type skos:Concept`; do not mint a duplicate Atomic class | +| `vault:Session`, `kind=session` | `session:*` | Replace with `rdf:type atomic:Session` (`rdfs:subClassOf prov:Activity`) | +| `vault:SessionTurn`, `kind=session_turn` | `session:*/turn:*` | Replace with `rdf:type atomic:SessionTurn` (`rdfs:subClassOf prov:Activity`) | +| `vault:Todo`, `kind=todo` | session-qualified todo IDs | Replace with `rdf:type atomic:Todo`; reject unqualified `todo:*` identities | +| `vault:ProvenanceGraph`, `kind=provenance` | `provenance:` | Replace with `rdf:type atomic:ProvenanceGraph` (`rdfs:subClassOf prov:Entity`) | +| `vault:Task`, `kind=task` | intent task directives | Replace with `rdf:type atomic:Task` | +| `vault:AcceptanceCriterion`, `kind=acceptance_criterion` | intent AC directives | Replace with `rdf:type atomic:AcceptanceCriterion` | +| `vault:ScopeItem`, `kind=scope` | in/out scope directives | Replace with `rdf:type atomic:ScopeItem`; role follows the containing property | +| `vault:Constraint`, `kind=constraint` | constraint directives | Replace with `rdf:type atomic:Constraint` | +| `kind=change` | `change:` | `atomic:Change`; `kind` is metadata, not class assertion | +| `kind=view` | `view:` | `atomic:View`; `kind` is metadata | +| `kind=module` | `module:` | `atomic:Module`; `kind` is metadata | +| `kind=entity` | parsed source symbols | `atomic:SourceEntity`; language-specific kinds become subclasses or metadata | + +All projectors MUST emit actual `rdf:type` assertions. `KgNode.kind` and +`metadata.rdf_type` cease to represent class membership and SHOULD be removed where they +duplicate RDF. UI indexes derive compact display categories from entailed `rdf:type`. + +#### PROV-O and metadata predicate coverage + +| Existing term | Observed domain → range | Canonical OWL disposition | +|---|---|---| +| `prov:wasAssociatedWith` | session/activity → identity; current tool result → goal misuse | Keep PROV-O only for activity → agent; replace tool-result → goal with `atomic:linkedGoal` | +| `prov:used` | activity → entity | Reuse canonical PROV-O property | +| `prov:wasGeneratedBy` | entity → activity | Reuse canonical; inverse of `prov:generated` | +| `prov:generated` | current session turn → change use | Replace with `atomic:generatedChange`, a subproperty of `prov:generated`; do not narrow the imported PROV property globally | +| `prov:wasInformedBy` | turn → preceding turn | Reuse canonical PROV-O property | +| `prov:wasDerivedFrom` | memory/session/entity → source entity | Reuse canonical PROV-O property | +| `atom:informedBy` | intent → memory/source | Replace with `atomic:informedBy`, `rdfs:subPropertyOf prov:wasDerivedFrom`; reject the old prefix form | +| `prov:hadPlan` | current session/turn → intent misuse | Replace with `atomic:hasPlan`; PROV `hadPlan` is reserved for qualified Association → Plan and is not declared equivalent | +| `prov:hadMember` | current session → turn misuse | Replace with `atomic:hasTurn`; Session remains a `prov:Activity`, not a `prov:Collection`, and no equivalence is declared | +| `dc:type` | resource → descriptive type | Retain Dublin Core metadata; never equivalent to `rdf:type` | +| `dc:creator` | resource → creator value | Replace identity-valued edges with object property `atomic:authoredBy`; do not emit literal creators | +| `dc:date` | resource → date literal | Delete stored edges and reproject `dcterms:created`/`dcterms:modified` from authoritative typed source fields; an orphan edge with no role provenance is a migration error | +| `dc:description` | resource → literal | Replace with `dcterms:description`; derive UI summaries from RDF | +| `rdf:type` | resource → class | Canonical class assertion | + +#### Vault workflow predicate coverage + +| Current namespaced predicate | Current uppercase kind | Observed domain → range | Required canonical replacement | +|---|---|---|---| +| `vault:blocks` | `BLOCKS` | intent → intent | Replace both with `atomic:blocks` | +| `vault:blockedBy` | `BLOCKED_BY` | intent → intent | Replace both with inverse `atomic:blockedBy` | +| `vault:assignedTo` | `ASSIGNED_TO` | intent → identity | Replace both with `atomic:assignedTo` | +| `vault:linkedIntent` | `LINKED_INTENT` | goal/change/entity → intent | Replace with typed relations: `atomic:implements` for change → intent and `atomic:linkedIntent` otherwise | +| `vault:linkedGoal` | `LINKED_GOAL` | intent/entity → goal | Replace both with `atomic:linkedGoal` | +| `vault:references` | `REFERENCES` | entity → entity | Replace both with `atomic:references` | +| `vault:appliesTo` | — | entity → concept/domain | Replace with `atomic:appliesTo` | +| `vault:hasLabel` | `HAS_LABEL` | intent/entity → concept | Replace both with `atomic:hasLabel` | +| `vault:status` | metadata `status` | entity → status literal/concept | `atomic:status`; metadata is application-only serialization | +| `vault:priority` | metadata `priority` | entity → priority literal/concept | `atomic:priority`; metadata is application-only serialization | +| `vault:explainedBy` | — | session turn/change → provenance graph | Replace with `atomic:explainedBy` | +| `vault:hasTodo` | — | session turn/intent → todo | Replace with `atomic:hasTodo` | + +#### Intent decomposition and review edge coverage + +| Uppercase edge kind | Observed domain → range | Canonical OWL property and relation | +|---|---|---| +| `HAS_TASK` | intent → task | Replace with `atomic:hasTask` | +| `HAS_ACCEPTANCE_CRITERION` | intent → acceptance criterion | Replace with `atomic:hasAcceptanceCriterion` | +| `HAS_SCOPE_IN` | intent → scope item | Replace with `atomic:hasScopeIn`, a subproperty of `atomic:hasScopeItem` | +| `HAS_SCOPE_OUT` | intent → scope item | Replace with `atomic:hasScopeOut`, a subproperty of `atomic:hasScopeItem` | +| `HAS_CONSTRAINT` | intent → constraint | Replace with `atomic:hasConstraint` | +| `SATISFIES` | task → acceptance criterion | Replace with `atomic:satisfies`; evidence uses separate narrower properties | +| `TOUCHES` | task → file | Replace with `atomic:touches` | +| `DEPENDS_ON` | intent → intent and change → change | Overloaded current kind; replace by endpoint class with `atomic:directlyDependsOnChange` or `atomic:dependsOnIntent`, both subproperties of `atomic:dependsOn` | +| `REMEDIATES` | intent → intent | Replace with `atomic:remediates` | +| `REVIEWS` | review intent → work intent | Replace with `atomic:reviews`; inverse `atomic:reviewedBy` | +| `SCOPE_OUT_FILE` | intent → file | Replace with `atomic:scopeOutFile`; MAY also be inferred through an out-scope item | + +#### VCS, view, and source-code edge coverage + +| Uppercase edge kind | Observed domain → range | Canonical OWL property and relation | +|---|---|---| +| `MODIFIES` | change → file or source entity | Replace by range with `atomic:modifiesFile` or `atomic:modifiesEntity`, both below `atomic:modifies` | +| `DEFINES` | file → source entity | Replace with `atomic:defines` | +| `ON_VIEW` | change/session → view | Overloaded current kind; replace by subject class with `atomic:directlyOnView` or `atomic:sessionViewContext`, both optionally below broad `atomic:onView` | +| `CHILD_OF` | view → parent view | Replace with `atomic:childViewOf`; derive transitive `atomic:ancestorView` separately | +| `VISIBLE_FROM` | not emitted in scoped projectors; intended change/entity → view | Replace with derived `atomic:visibleFrom`, never direct membership | +| `CALLS` | callable source entity → callable source entity | Replace with `atomic:calls` | +| `IMPORTS` | not emitted; intended file → file | Replace with `atomic:imports`, a subproperty of `atomic:includes` | +| `INCLUDES` | file → file | Replace with `atomic:includes` | +| `EXPORTS` | file → exported source entity | Replace with `atomic:exports`, a subproperty of `atomic:defines` | +| `PART_OF` | file → module; module → parent module | Replace with typed `atomic:filePartOfModule` or `atomic:modulePartOfModule`, below `atomic:partOf` | +| `AUTHORED_BY` | goal/intent/change → identity | Replace with object property `atomic:authoredBy`; do not emit a second creator predicate | + +#### SKOS taxonomy coverage + +| Existing term | Observed/intended domain → range | Canonical disposition | +|---|---|---| +| `skos:broader` | concept → broader concept | Reuse SKOS canonical property | +| `skos:narrower` | concept → narrower concept | Reuse SKOS; inverse of `skos:broader` | +| `skos:subject` | not emitted; intended resource → concept | Do not introduce it; use `dcterms:subject` directly | +| `skos:related` | concept → concept | Reuse SKOS canonical symmetric property | + +#### Breaking normalization rules + +The following require type-aware destructive rewrites rather than aliases: + +1. Current `ToolResult prov:wasAssociatedWith Goal` misuses PROV association. Replace it + with `atomic:linkedGoal`; no PROV equivalence is asserted. +2. `DEPENDS_ON` combines intent planning dependencies and VCS change dependencies. + Existing edges are class-aware migrated directly to narrower properties; the broad + transitive property is inferred and never used to represent directness. +3. `MODIFIES` has a union range of files and source entities. Narrower properties retain + query precision while the old broad relation is removed. +4. `ON_VIEW` combines change membership and session workspace context. It is not + equivalent to either canonical property; migrate by subject class to + `directlyOnView` or `sessionViewContext`. Only direct change membership participates + in view-cost queries. +5. `IMPORTS` is currently defined but import extraction often emits `INCLUDES`. New + projection distinguishes language import from textual include and places both in a + declared hierarchy. +6. Session class identity currently appears in `metadata.rdf_type`, while most classes + appear only as `kind`. Migration emits true `rdf:type` triples and removes semantic + dependence on both fields. +7. Session-global `todo:` identities are replaced by + `session:/todo:`; ambiguous old IDs fail migration rather than receiving an + `owl:sameAs` assertion. + +#### Breaking cutover requirements + +Stored uppercase `KgEdge.kind` strings are not RDF IRIs and SHALL be replaced in storage +with canonical, absolute predicate IRIs. The cutover SHALL: + +- rewrite every persisted edge to its canonical RDF predicate, using endpoint types to + split overloaded relations; +- rewrite every class representation to an actual `rdf:type` assertion; +- update producers, query planners, CLI JSON, tests, and consumers in the same repository + format/API version; +- reject old uppercase kinds, `vault:*` duplicates, `atom:*` prefix variants, and + metadata-only typing after migration; +- fail with an actionable migration error when an overloaded edge cannot be classified; +- carry tests proving every current `predicate`, `entity_type`, and `edge_kind` constant + has exactly one canonical replacement and that no compatibility vocabulary is emitted. + +There is one ontology module family rooted at the canonical Atomic namespace. No legacy +OWL artifact is published or loaded during reasoning. + +### Storage-minimization boundary + +The ontology intentionally stops at durable resource and activity classes. It does not +contain language-, framework-, SQL-, API-, or security-specific operation/effect classes. +Those interpretations belong to virtual query graphs or separately attested findings. +This keeps persistent growth proportional to canonical project objects and provenance, +not to the number of possible analyses over each source span. + +### Open-world boundary + +OWL uses the open-world assumption: absence of `hasPlan`, `attestedBy`, or +`reviews` does not mean the relationship does not exist. OWL cardinality restrictions +therefore describe meaning but cannot serve as publication gates. Atomic MUST NOT use an +OWL consistency result to claim that a record is complete, reviewed, or cryptographically +verified. + +The division of responsibility is normative: + +| Concern | OWL | SHACL | +|---|---:|---:| +| Class and property meaning | Yes | Optional | +| Subclass/subproperty inference | Yes | No | +| Inverse and property-chain inference | Yes | No | +| Detect missing required edges | No | Yes | +| Enforce exact cardinality for publication | No | Yes | +| Validate datatypes and numeric bounds | No | Yes | +| Verify signatures or hashes | No | External verifier + SHACL result | +| Check aggregate equality and review freshness | No | SHACL-SPARQL | + +### Reasoning and validation order + +Reconciliation SHALL execute in this order: + +1. project asserted triples from canonical objects; +2. cryptographically verify hashes and attestations and assert verification results; +3. apply the declared OWL 2 RL inference rules; +4. run SHACL over the asserted-plus-inferred graph; +5. materialize approved convenience edges with provenance identifying the rule and + ontology version; +6. publish the projection generation and conformance report. + +SHACL shapes may rely on OWL-inferred triples only when the validation report records the +reasoning profile. Implementations that validate asserted triples alone must use shapes +that traverse the full authoritative path explicitly. + +## SHACL enforcement + +OWL defines meaning and derives entailments; SHACL decides whether a concrete projection +is operationally complete. Validation has two profiles. + +### Publication profile + +Applied before publishing managed agent work: + +- every managed `SessionTurn` has exactly one `atomic:hasPlan` intent; +- every managed/active intent governs at least one turn through `atomic:governsTurn`; +- no managed turn is governed by more than one intent; +- backlog intents may have zero turns until their transition into managed work; +- every turn has exactly one provenance graph; +- every generated change is covered by the session attestation; +- every attestation has a signer DID and valid proof; +- every required review covers the exact implementation closure. + +Failure blocks publication because these facts cannot be safely inferred later. + +### Projection profile + +Applied after sync/enrichment: + +- every generated change has matching `atomic:implements`, `atomic:explainedBy`, and + `atomic:attestedBy` + shortcuts; +- shortcut targets equal the authoritative intermediate path targets; +- all referenced canonical nodes exist; +- aggregate cost/tokens equal linked contribution sums; +- recorded executions point through `prov:used` to existing canonical artifacts and + through provenance to the responsible turn/change; +- full hashes, not abbreviated labels, are used in durable IDs; +- an approved review's closure digest equals the current implementation closure. + +Projection failure triggers deterministic repair and revalidation. If repair still +fails, the KG is marked non-conforming and must not be presented as complete audit +evidence. + +Repositories are upgraded through a breaking format migration before the new graph is +opened. After cutover, one strict shape family applies to the canonical graph. Historical +objects with genuinely absent facts may carry explicit `atomic:Unknown`/finding nodes, +but they do not retain an alternate vocabulary. + +## Sync and enrichment contract + +`atomic vault sync` and `atomic vault query enrich` currently have different scopes. +The user-facing contract SHOULD become a single reconciliation operation: + +1. ingest/materialize canonical vault objects; +2. discover canonical VCS, session, provenance, and attestation objects; +3. project all node and edge families; +4. index exact resource versions and execution evidence needed by authorized `query ask` + retrieval; +5. apply the versioned OWL 2 RL entailment rules; +6. run projection SHACL validation over the asserted-plus-inferred graph; +7. repair stale derived projections; +8. publish ontology version, reasoning profile, conformance status, and statistics. + +Implementations may retain separate internal phases, but users should not need to know +which subsystem owns a missing edge. + +Incremental record-time enrichment runs the same projectors for newly published objects. +A full rebuild must produce an equivalent graph. + +## Historical migration + +1. Replace abbreviated change-node IDs with full-hash canonical IDs; old IDs cease to + resolve after migration. +2. Replay all stored session turns to restore session/turn/intent/change/provenance + edges. +3. Load attestations and emit signer, coverage, chain, and verification nodes/edges. +4. Project AI contribution nodes and recompute change aggregates. +5. Parse provenance ledgers into typed causal nodes and edges. +6. Index exact content-addressed file versions and recorded execution evidence for + `query ask`; do not derive persistent domain operations or effects. +7. Apply the selected OWL profile and materialize derived relations with rule provenance. +8. Run the strict canonical SHACL profile over the entailed graph. +9. Report unrecoverable gaps rather than inventing links. + +An unrecoverable record receives a finding such as `MissingHistoricalPlanId`; it does +not receive a guessed `atomic:implements` edge. + +## Cross-project resource querying + +OWL terms make separately produced project graphs semantically composable, but +federation MUST NOT flatten repositories into one unscoped graph. Atomic models a query +surface as an RDF Dataset: each project projection is a named graph with its own +identity, authorization boundary, ontology version, projection generation, and SHACL +conformance report. + +### Project identity and named graphs + +Every repository receives a stable `ProjectId` from a signed, content-addressed project +identity object containing an issuer DID, creation time, random 256-bit identifier, +schema version, and proof. Ordinary clones transfer that object unchanged. An explicit +fork operation creates and signs a new identity object with `prov:wasDerivedFrom` the +origin project. Import collision is rejected when the same identifier has a different +verified identity object; lost signing authority is handled by an attested rotation +chain, never silent ID replacement. The identity survives relocation and remote URL +changes. A project graph IRI is derived from that identity and projection generation: + +```text +urn:atomic:project: +urn:atomic:graph::projection: +``` + +Mutable names such as a filesystem path, organization slug, or remote URL are labels and +locators, not identity. A fork may intentionally preserve an origin relationship while +receiving a distinct `ProjectId`: + +```text +project:fork ──prov:wasDerivedFrom──▶ project:origin +``` + +The active dataset separates graph roles explicitly: + +```text +urn:atomic:catalog: project/graph catalog +urn:atomic:graph::asserted: canonical projections +urn:atomic:graph::inferred: OWL entailments +urn:atomic:graph::assertions: reified assertion records +urn:atomic:graph:federation:: authorized cross-project links +``` + +Only authorized project graphs enter the active dataset. Catalog triples map projects to +all graph IRIs and projection manifests; the default graph is empty. Asserted project +triples remain in their source graph. Inferred triples are written to the project +inferred graph and linked through deterministic derivation records to source triple IDs, +rule IDs, ontology version, and asserted generation. These records and graph digests are +covered by the signed projection manifest. + +### Portable resource identity + +Resources fall into two identity classes: + +1. **Globally content-addressed objects** — changes, provenance graphs, attestations, + reviewed closure digests, and immutable vault objects use full cryptographic hashes. + The same verified object hash denotes the same immutable bytes across projects. +2. **Project-scoped resources** — views, paths, modules, source entities, mutable intent + names, and local sequence numbers are qualified by `ProjectId`. + +Canonical examples: + +```text +urn:atomic:change:blake3-256: +urn:atomic:provenance:blake3-256: +urn:atomic:attestation:blake3-256: +urn:atomic:project::view: +urn:atomic:project::intent: +urn:atomic:project::file: +``` + +Each object specification defines canonical bytes, schema version, hash algorithm, and a +distinct domain-separation prefix before an IRI can be treated as global identity. +Multihash-compatible algorithm qualification permits migration without collisions across +algorithms or object families. A human key such as `ATOM::lee::42`, a 12-character +change prefix, or `src/lib.rs` is never globally unique. Query parsers resolve shorthand inside an explicit project scope +before evaluating the federated plan. + +Cross-project identity uses `atomic:verifiedEquivalent`, backed by an identity-proof +assertion and policy-controlled substitution. `owl:sameAs` entailment is disabled across +project authorization boundaries because standard replacement semantics can disclose +private properties even when identity itself is proven. It MAY be enabled only inside an +authorization-equivalent graph set whose policy permits full fact substitution. Similar +paths, matching labels, mirrored intents, equivalent APIs, or copied source files use +weaker relations such as `atomic:correspondsTo`, `prov:wasDerivedFrom`, +`dcterms:hasVersion`, or a domain-specific compatibility relation. + +### Cross-project relations + +The federation graph may contain portable links including: + +```text +Project ──atomic:dependsOnProject────▶ Project +Project ──prov:wasDerivedFrom────────▶ Project +Change ──atomic:patchEquivalent─────▶ Change +Intent ──atomic:relatedIntent───────▶ Intent +Intent ──atomic:remediates──────────▶ Intent +Project ──atomic:usesArtifact────────▶ ContentAddressedArtifact +Execution ──prov:used────────────────▶ ContentAddressedArtifact +``` + +Cross-project assertions are first-class assertion nodes, not unannotated RDF edges: + +```text +CrossProjectAssertion + subject, predicate, object + assertingProject, assertingIdentity + sourceObject, timestamp, trustPolicy + attestation, verificationResult +``` + +The federation graph contains the relation only when its assertion node verifies and +policy authorizes its use; the assertion graph retains the complete evidence. A +canonical N-Quads statement identifier or deterministic skolem IRI identifies every +assertion. Blank nodes are forbidden in signed slices, evidence paths, reports, and +federated join keys. Inferred links are never promoted to asserted links without a +canonical object recording that assertion. + +Project dependencies are not automatically change dependencies. A package manifest may +assert `Project A dependsOnProject Project B`; only lockfile/object evidence can identify +the exact version, view state, or change closure consumed. + +### Federation modes + +The query planner supports three compatible execution modes: + +1. **Local catalog** — query named graphs for projects cloned or indexed locally. +2. **Object-backed federation** — fetch signed project projection manifests and required + graph slices from Atomic object storage. +3. **Remote endpoint federation** — dispatch authorized subplans to remote Atomic query + endpoints and combine signed result envelopes. + +A logical query has the same semantics in every mode. The planner pushes filters and +projection down to source graphs, requests only necessary graph slices, and performs +cross-project joins on portable IRIs. + +Queries MUST declare scope explicitly: + +```text +--project one project +--projects , selected projects +--workspace configured local project set +--organization all authorized organization projects +--federated allow remote subplans +``` + +There is no implicit “all projects visible to the server” mode. This avoids accidental +information disclosure and unstable results as unrelated repositories are indexed. + +### Ontology and reasoning negotiation + +Each project graph advertises: + +- canonical ontology version; +- canonical repository graph-format version; +- OWL reasoning profile; +- available `query ask` model/tool capabilities; +- projection generation; +- SHACL conformance report hash. + +Reasoning is an explicit multi-pass operation. For each authorized project, the reasoner +forms a temporary controlled union of that project's asserted graph plus the exact +versioned canonical ontology graphs declared by its manifest. It +writes entailments only to that project's inferred graph. A second federation pass reads +only authorized asserted/inferred graphs, verified assertion nodes, the scoped federation +graph, and explicit version-bridge axioms selected by the planner; its results go to a +separate federation-inferred graph. These bridges relate released versions of the one +canonical Semantic Web ontology; they never expose removed storage strings, uppercase +predicates, old prefixes, or a compatibility query API. Every inferred statement has a +derivation record. The planner computes the newest mutually compatible ontology contract +and MUST NOT union +projects under an arbitrary latest ontology, because that can change historical meaning +or create invalid identity entailments. + +Cross-project inference is limited to bridge-safe rules declared by the federation +profile. In particular: + +- subclass and subproperty mappings may cross graphs; +- verified content-addressed identity may be shared; +- project-local functional or inverse-functional properties do not become globally + functional; +- no cross-project `owl:sameAs` is inferred unless all participating graphs share an + authorization-equivalent policy and an identity proof explicitly permits full OWL + substitution; +- ephemeral `query ask` interpretations do not participate in persistent OWL entailment + unless separately promoted and attested as canonical findings. + +### Authorization and privacy + +Authorization is evaluated before graph selection, reasoning, joins, inference, +aggregation, and result assembly. Unauthorized triples never enter a controlled union or +intermediate binding table. Predicate- and resource-level labels propagate to derived +triples as the intersection of source permissions; a derived statement is visible only +when every supporting statement is visible to the caller. A caller may have access to a request-scoped `query ask` result, or to a separately +attested durable finding, without access to the underlying reasoning text, prompt, source +file, cost, or private intent. + +Result filtering MUST not reveal hidden projects through counts, inferred edges, +aggregate differences, or “redacted” placeholders unless policy explicitly permits +existence disclosure. Timing resistance has an explicit threat profile: untrusted remote +callers receive allow-listed bounded plans, isolated execution, cached/padded negative +responses, and coarse resource limits. The RFC does not claim timing non-interference for +arbitrary local administrators or unrestricted SPARQL endpoints. + +Remote results are returned in signed envelopes containing signer DID, source project, +projection and graph hashes, canonical query-plan hash and parameters, requested scope, +authorization-policy/version, ontology/reasoning versions, canonical result digest, +execution timestamp, nonce/request ID, expiry, applicable SHACL report, and explicit +complete/partial source lists with failure reasons. Signatures prove authenticity and +integrity; completeness is claimed only relative to the signed requested scope and +source manifest. + +### Aggregation and deduplication + +Federated aggregation declares its counting identity: + +- **Object identity** deduplicates the same full content-addressed object across projects. +- **Contribution identity** deduplicates generation cost by a canonical generation-event + ID, not by change or attestation hash. +- **Project membership identity** counts `(ProjectId, object hash)` once per project. +- **Occurrence identity** counts every selected view or deployment occurrence. + +A generation event has canonical identity, typed cost/token measurements, and explicit +`generatedChange` edges. Attestations independently use `attestsContribution`; one +attestation may cover several contributions and one contribution may have several +attestations. When one generation event produces multiple changes, its cost is counted +once for a closure containing any of them unless an explicit, canonical allocation record +assigns portions to changes. + +Cost defaults to contribution identity because a change hash may identify hashed change +content without uniquely identifying every attached provenance payload. Reusing the same +verified contribution in several projects does not incur generation cost again, while +independently generated equivalent changes may. Deployment-risk and adoption queries +often use project membership identity. The API MUST require or report the selected mode +rather than silently choosing based on graph layout. + +### Federated query examples + +Find every intent, in authorized projects, that introduced a table deletion affecting a +shared database resource: + +```sparql +SELECT DISTINCT ?project ?projectGraph ?projection ?ontology ?shaclReport + ?intent ?change ?file ?execution ?analysis ?attestation +WHERE { + # These graph IRIs come from the verified active-dataset configuration. + VALUES (?catalogGraph ?federationGraph ?assertionGraph) { + ( + + ) + } + VALUES ?project { project:payments project:identity project:analytics } + GRAPH ?catalogGraph { + ?project atomic:hasProjection ?projection . + ?projection atomic:assertedGraph ?projectGraph ; + atomic:ontologyVersion ?ontology ; + atomic:shaclReport ?shaclReport ; + atomic:conforms true . + } + GRAPH ?projectGraph { + ?turn atomic:generatedChange ?change ; + atomic:hasPlan ?intent . + ?change atomic:modifiesFile ?file . + OPTIONAL { + ?execution a atomic:Execution ; + prov:used ?file ; + atomic:partOfTurn ?turn . + } + ?attestation atomic:coversChange ?change ; + atomic:verificationResult atomic:Verified . + } + SERVICE atomic-query:askResults { + ?analysis atomic-query:source ?file ; + atomic-query:status atomic-query:ModelInterpretation ; + atomic-query:operation "drop-table" ; + atomic-query:evidenceDigest ?evidenceDigest . + } +} +``` + +Trace a migration or recorded execution across authorized projects: + +```text +projects(workspace) + → changes → modified migration files / provenance executions + → ephemeral query-ask result(operation = drop-table) + → change → intent → attestation +``` + +The `query ask` result is request-scoped. Only the project graph's durable file, execution, +change, intent, and attestation relations participate in persistent federation. + +Find projects consuming an artifact produced by a particular intent: + +```text +intent(project A, uid) + → implementedBy changes + → produced content-addressed artifacts + → inverse atomic:usesArtifact / lockfile or execution provenance + → consuming projects and pinned artifact versions +``` + +If the caller asks whether that artifact is an API contract or whether use is compatible, +`query ask` interprets the exact artifact/manifests and returns shaped ephemeral evidence. + +Every row includes source project and named graph. Query results that combine conforming +and non-conforming graphs distinguish them explicitly instead of presenting a single +undifferentiated confidence level. + +### Federated SHACL validation + +Project SHACL shapes validate each project graph independently. A custom dataset +validation runner then executes SHACL-SPARQL/SHACL-AF checks over declared named graphs +and invokes external cryptographic and authorization-policy verifiers. SHACL validates +the recorded verifier results; SHACL itself does not verify signatures, hashes, remote +envelopes, or authorization. Dataset checks validate cross-project assertions: + +- both endpoints use portable canonical IRIs; +- the asserting identity is authorized to make the link; +- referenced projection/object hashes exist and verify; +- ontology bridge versions are compatible; +- `verifiedEquivalent` carries the required identity proof and policy result; +- externally computed remote-envelope and attestation verification results are valid; +- a cross-project review names the exact reviewed project and closure digest. + +An unavailable remote graph is **unknown**, not conforming and not violating. Query +results report partial federation separately from SHACL violations. All joinable and +signed resources use canonical or deterministic skolem IRIs; blank nodes are rejected. + +## Query and API requirements + +The query layer SHOULD provide first-class operators for: + +- direct view membership; +- effective view visibility; +- dependency closure; +- implementing changes for an intent; +- provenance and attestation traversal; +- evidence-grounded `query ask` over exact file versions and execution records; +- review freshness; +- distinct aggregate cost and token usage; +- explicit project/workspace/organization federation scope; +- named-graph and source-project provenance; +- object, project-membership, and occurrence deduplication modes; +- ontology-version negotiation and partial-federation reporting. + +Results SHOULD include a projection generation, ontology version, reasoning profile, +model/tool versions and query-plan hash when semantic interpretation is requested, and +SHACL conformance state so callers can distinguish complete +audit answers from incomplete or non-conforming results. + +## Security and privacy + +- Reasoning text, prompts, transcripts, and tool output may contain secrets. Store them + in access-controlled content-addressed objects and expose hashes/approved summaries in + the general KG. +- Cost and token totals can reveal operational information and follow repository access + controls. +- `query ask` interpretations are ephemeral claims; results include model/tool + provenance, query-plan hash, exact source/execution evidence, and uncertainty. +- Attestation validity is computed from signatures, not trusted from a mutable boolean. +- Cross-project queries enforce authorization at every source and do not leak hidden + project existence through joins, inference, or aggregates. +- Remote graph slices and result envelopes are verified before their claims participate + in reasoning or SHACL validation. + +## Rollout + +### Phase 1: Referential closure + +- Versioned OWL 2 RL ontology, optional OWL 2 DL companion, and SHACL shape graph. +- Breaking graph-format migration to canonical RDF IRIs with no compatibility ontology. +- Automated replacement coverage test for every current predicate, entity type, and + edge-kind constant. +- Signed stable `ProjectId` objects, project-qualified mutable-resource IRIs, and + algorithm-qualified content-object IRIs. +- Intent/session/turn/provenance/change/attestation nodes. +- Required direct shortcut edges. +- Backfill and projection SHACL validation. + +### Phase 2: Contribution normalization + +- AI contribution and model nodes. +- Typed token/cost fields and validated aggregates. +- Privacy-aware references to large provenance content. + +### Phase 3: Evidence-grounded `query ask` + +- Constrained natural-language-to-query-plan and result-shape generation. +- Bounded retrieval over exact content-addressed files, changed hunks, and executions. +- Structured LLM interpretation with machine-validated citations and abstention. +- Request-scoped virtual result graphs with no persistent statement/effect nodes. +- Optional deterministic tools callable by the planner without defining stored semantics. + +### Phase 4: Review policy + +- Exact reviewed closure digests. +- Review freshness and independent-attestation shapes. +- Promotion/completion gates. + +### Phase 5: Cross-project federation + +- Federation transport and catalogs over the stable project identities and + project-qualified resource IRIs established in Phase 1. +- Named project graphs and signed projection manifests. +- Local, object-backed, and remote endpoint federation. +- Canonical ontology-version bridge negotiation—with no old vocabulary exposure—and + dataset-level SHACL. +- Authorization-safe joins, result envelopes, and explicit aggregate identity modes. + +## Acceptance criteria + +The RFC is implemented when: + +1. A newly recorded managed change has a continuous graph path to its governing intent, + provenance graph, attestation, session, and turn. +2. Full enrichment repairs all derivable links from existing canonical objects. +3. OWL 2 RL reasoning derives the specified inverse, hierarchy, and provenance + property-chain relations deterministically. +4. SHACL rejects missing or inconsistent required links after the strict cutover. +5. A query over a SQL migration containing `DROP TABLE users`, or equivalent recorded + execution evidence, returns ephemeral `query ask` evidence linked to the responsible + file, change, intent, provenance, attestation, and review without persisting a table or + effect node. +6. Direct-view and effective-view cost queries return distinct, explicitly scoped + change sets. +7. Historical missing facts are reported and never guessed. +8. Every current `predicate`, `entity_type`, and `edge_kind` constant has exactly one + tested canonical replacement; no old term or compatibility ontology remains after the + breaking migration. +9. A query can join authorized named graphs from multiple projects using portable + identities while reporting source graph, ontology version, conformance, completeness, + and explicit deduplication semantics. +10. Cross-project identity and review assertions require verifiable evidence and never + infer `owl:sameAs` from labels, paths, or similarity. + +## Resolved decisions + +- Domain semantics are supplied by evidence-grounded `atomic vault query ask`, not by + persistent semantic-effect relations. +- The LLM builds an allow-listed query plan and result shape; Atomic executes traversal, + retrieves exact evidence, and validates all citations and structural bindings. +- Deterministic parsers are optional tools, not a required ontology or persistence layer. +- Semantic answers are ephemeral and excluded from OWL/SHACL unless explicitly promoted + into a separately attested durable artifact. +- `atomic vault sync` and `atomic vault query enrich` call one shared graph-reconciliation + pipeline internally. +- OWL 2 RL entailments are materialized into versioned inferred named graphs with + derivation records. +- Local, object-backed, and remote query transports sit behind one logical planner. +- One managed intent governs one or more turns; every managed turn has exactly one + governing intent, while backlog intents may have zero turns before work begins. +- `atomic:` is the single Atomic workflow ontology namespace, covering Intent, Change, + Memory, Attestation, and every other workflow-owned concept; PROV-O is the shared + crossover vocabulary and is reused directly. + +## Open questions + +1. Which OWL 2 RL implementation should provide local deterministic entailment? +2. Beyond the settled PROV-O boundary, which exact external terms from SIOC, DOAP, SPDX, + Dublin Core, SKOS, and the Data Integrity vocabulary match without weakening workflow + ownership? +3. Which ontology-version differences are safe for automatic bridge reasoning across + projects? From 23c196566b482cd1c8a993a8ba066e78d41184cc Mon Sep 17 00:00:00 2001 From: Lee Faus Date: Tue, 13 Oct 2026 17:09:48 -0400 Subject: [PATCH 02/19] start with a small phase 1 --- docs/knowledge-graph-closure-phase1-plan.md | 1241 ++++++++----------- 1 file changed, 544 insertions(+), 697 deletions(-) diff --git a/docs/knowledge-graph-closure-phase1-plan.md b/docs/knowledge-graph-closure-phase1-plan.md index b4a21ca1..db0ebf9c 100644 --- a/docs/knowledge-graph-closure-phase1-plan.md +++ b/docs/knowledge-graph-closure-phase1-plan.md @@ -1,727 +1,574 @@ -# Phase 1 Implementation Plan: KG Referential Closure +# Phase 1 Implementation Plan: KG Referential Closure — Quick Win > Status: **draft plan.** Implements Phase 1 ("Referential closure") of > [`knowledge-graph-closure-rfc.md`](./knowledge-graph-closure-rfc.md). -> Branch: `feat/semweb-knowledge-graph`. Survey references verified against `dev` at `5ddcf1b`. - -## 1. Phase 1 scope (from the RFC) - -- Versioned OWL 2 RL ontology, optional OWL 2 DL companion, SHACL shape graph. -- Breaking graph-format cutover to canonical RDF IRIs; no compatibility vocabulary. -- Coverage test: every current predicate / entity type / edge kind has exactly one - canonical replacement. -- Signed stable `ProjectId`, project-qualified mutable-resource IRIs, algorithm-qualified - content-object IRIs. -- Intent / session / turn / provenance / change / attestation nodes. -- Required direct shortcut edges (`implements`, `explainedBy`, `attestedBy`). -- Backfill and projection validation. - -Explicitly **not** Phase 1: `AIContribution` nodes and cost aggregates (Phase 2), -`query ask` evidence envelopes (Phase 3), review closure digests / `reviewsClosure` -(Phase 4), federation and named-graph catalogs (Phase 5), `Execution` nodes (no -structured execution capture exists yet — see D11). - -## 2. Current state (survey findings) - -### 2.1 Storage - -- The KG is a **JSON property graph**, not a triple store: - `KgNode { id, kind, label, summary, source, metadata }` and - `KgEdge { from_id, to_id, kind, metadata }` (`atomic-core/src/pristine/vault.rs:462,516`). -- Tables (`atomic-core/src/pristine/tables.rs:797-830`): `kg_nodes`, `kg_edges` - (key `from\0to\0kind`, `encode_edge_key` L856), `kg_edges_from`, `kg_edges_to`, - `kg_fts`, `kg_fts_by_node`, `kg_index_meta`. No predicate or type index. -- The predicate string is **part of the edge key**, so renaming means rewriting keys. -- `upsert_kg_edge` (`txn/write/triples.rs:380`) accepts any string. `ontology::is_known_predicate` - has no callers. -- `kg_neighbors` (`traits/triples.rs:18-68`) is kind-blind, bidirectional BFS with depth ≤ 2; - it is the main read primitive (query plans, `vault context`, RAG, daemon). - -### 2.2 The KG is entirely derived - -Every node/edge comes from one of three producers, all over authoritative objects: - -| Producer | File | Source tag | Trigger | -|---|---|---|---| -| Vault projection | `atomic-repository/src/repository/vault_triples.rs` | `vault` | every `vault_store` (intent/memory/goal writes, `vault sync`, record w/ vault sync), `query reindex` | -| VCS/AST enrichment | `atomic-repository/src/repository/vault_kg_enrich.rs` | `vcs`, `ast` | clone, pull, git import, record (`kg_enrich_change`), `query enrich` | -| Session ledger | `atomic-core/src/pristine/txn/write/session_kg.rs` | `session_ledger` | turn append/rewrite, session lifecycle, sidecar pull, `session rebuild` | - -**Consequence for the cutover:** nothing in the KG needs in-place rewriting. A storage- -generation bump + full rebuild from canonical objects *is* the migration, and the RFC's -"fail on unclassifiable overloaded edge" rule becomes a projector-level guarantee (each -projector knows its endpoint classes at emit time). - -`vault sync` and `query enrich` share **no** code; they meet only on shared ID strings. - -### 2.3 Vocabulary inventory (`atomic-core/src/pristine/ontology.rs`) - -- 6 prefix labels (no IRIs), 30 `predicate::*` CURIEs, 16 `entity_type::*` (`vault:*`), - 29 `edge_kind::*` SCREAMING_CASE strings. -- Actually emitted: 19 uppercase kinds + `prov:{wasDerivedFrom,used,wasAssociatedWith, - hadPlan,hadMember,generated,wasInformedBy}`, `atom:informedBy`, `vault:{explainedBy,hasTodo}`. -- Defined but never emitted: all `dc:*`, `skos:*`, `rdf:type` (as edge), `vault:{status, - priority,assignedTo,blocks,...}` predicates, `IMPORTS`, `VISIBLE_FROM`, and 12 of 16 - `entity_type` values. -- Typing: `kind` string on every node; `metadata.rdf_type` (prefixed, e.g. `vault:Session`) - only on the 4 session-ledger node kinds. **No real `rdf:type` anywhere.** - -### 2.4 Identity problems - -- Change node IDs are **12-char base32 prefixes** in three places - (`vault_kg_enrich.rs:1089 truncate_hash`, `session_kg.rs:46 change_node_id`, - `vault_triples.rs:1268 rdf_target_to_kg_id`). Full hash survives only in metadata. -- `identity:` IDs are display names (`identity:{author.name}` vs `identity:{fm.developer}`); - intents write `author`/`created_by` which the KG ignores, so intents have no author edge. -- `todo:X` from `urn:atomic:todo:X` collides with the legacy session-global todo form. -- `LINKED_INTENT` targets `intent:{raw}` without uppercasing/manifest resolution. -- **No `ProjectId` exists.** Closest: remote `ProjectInfo.id: Uuid` (server-assigned), - `VaultManifest.intent_prefix` (4-char human code), `repository_identity` DID in - `provenance_gate.rs:76`. - -### 2.5 Overloaded / misused relations (confirmed in code) - -| Current | Uses | Lines | -|---|---|---| -| `DEPENDS_ON` | change→change; intent→intent (fallback for any `:::ref`) | enrich :409,:526; triples :1158 | -| `MODIFIES` | change→file; change→entity **only on record-time path**, not bulk | enrich :436,:554,:624 | -| `ON_VIEW` | change→*working copy's desired view* (not membership!); session→view | enrich :386,:514; session_kg :320 | -| `prov:wasAssociatedWith` | tool_result→goal | triples :1002 | -| `prov:hadPlan` | session→intent **and** turn→intent | session_kg :389-397 | -| `prov:hadMember` | session→turn | session_kg :381 | -| `prov:wasDerivedFrom` | memory→{anything}; session→session (fork) | triples :978; session_kg :456 | - -### 2.6 Facts captured but never projected - -- `Attestation` (`atomic-core/src/change/attestation.rs:153`): `session_id`, - `changes_covered`, `signer` (`did:atomic:…` or `session-mac:{sid}`), `signature`, - `previous_attestation`, `provenance_root`, … Stored as `.attest` files. - **No KG projection.** -- Canonical intent/memory attestations (signed JSON-LD vault entries) — explicitly skipped - by the KG (`vault_triples.rs:53`). -- `SessionTurn` (`atomic-core/src/change/session.rs:418`): `plan_id`, `goal`, - `provenance_hash`, `change_hashes`, `previous_provenance`, `outcome`, boundaries — - partially projected (no outcome/boundaries). -- AI `Provenance` on changes (model/tokens/cost) — Phase 2. - -### 2.7 Consumers that hardcode kind strings - -- `atomic-cli/src/commands/query/mod.rs`: `callers` (`to_uppercase()=="CALLS"` :1231,:1255), - HTML `EDGE_COLORS` (:748-755), `ask` system prompt (:1971-1976), text output. -- `atomic-repository/src/triage/report.rs` (`kind_is` at :35, case-insensitive, ~12 sites), - `atomic-repository/src/repository/triage.rs:231,305`. -- `atomic-repository/src/repository/vault_intent.rs:1770,1776` (`atom:informedBy`). -- `atomic-cli/src/commands/vault/context.rs:470-473` (filters `node.kind == "memory"`). -- libatomic daemon copies: `services_agent.rs:131,3372-3387`, `services_query.rs:1001-1038,1059`. -- Wire: `libatomic/proto/.../knowledge_messages.proto:123` `KGEdge.relation` (free string). -- Out-of-repo: `../noname` (`atomic.ts:700,1928`, `kgHops.ts:202-245`, `BlastTriage.tsx`, - `MemoryView.tsx`), `../the-hive/packages/shared/src/types/kg.ts:40-48`. -- ~110 test assertion sites (largest: `vault_triples.rs` ~40, `ontology.rs` ~31, - `atomic-cli/tests/session_integration_test.rs` ~17). - -### 2.8 Existing Semantic Web pieces and constraints - -- `atomic-canonical` already emits signed JSON-LD with context - `https://atomic.dev/ns/ctx.jsonld` mapping `atom:` → `https://atomic.dev/ns#`, and URNs - `urn:atomic:{intent,memory,change,...}:`. Content hash = `blake3(JCS(node))`, which covers - the `@context` **URL string** but not the term definitions behind it. -- `atomic provenance` projects PROV JSON-LD on demand (`atomic-canonical/src/prov.rs`); - not written to the KG. -- **No RDF/OWL/SHACL crates** in the workspace; no `.ttl` files. -- **Standing decision (2026-07-06, `docs/shacl-oxirs-spike-findings.md`):** no SHACL engine - ships in `atomic`; SHACL runs as a Circuit Breaker (CB) workflow; atomic keeps hand-coded - gates and ships shapes `.ttl` as data. oxirs rejected (~432 crates, silently drops - `sh:node`/`sh:sparql`). `oxttl` (~20 crates) noted as acceptable. -- Schema/upgrade machinery: `SCHEMA_VERSION` only rejects newer DBs (`schema.rs`); - **capability fences** (`capability.rs`, `required-capability/`) make older binaries - fail closed; `kg_index_meta` holds lazy per-index markers (`fts_reverse_schema` = 2 is the - precedent for a KG rebuild-on-write). - -## 3. Target model for Phase 1 - -### 3.1 Namespaces - -| Prefix | IRI | Role | -|---|---|---| -| `atomic:` | `https://atomic.dev/ns/workflow#` | the one workflow ontology | -| `prov:` | `http://www.w3.org/ns/prov#` | reused directly | -| `dcterms:` | `http://purl.org/dc/terms/` | `created`, `modified`, `description`, `subject` | -| `skos:` | `http://www.w3.org/2004/02/skos/core#` | `Concept`, `broader`, `narrower`, `related` | -| `rdf:`/`rdfs:`/`owl:`/`xsd:` | standard | | +> Branch: `feat/semweb-knowledge-graph`. -Ontology IRI `https://atomic.dev/ns/workflow`, version IRI -`https://atomic.dev/ns/workflow/1.0.0`. +## Guiding principle for Phase 1 -### 3.2 Node identity (replaces `kind:local` IDs) +Ship **three artifacts** that every downstream phase depends on: -| Kind | IRI | Notes | -|---|---|---| -| Project | `urn:atomic:project:` | `` from the project identity object (D5) | -| Change | `urn:atomic:change:blake3-256:` | full hash, canonical uppercase base32 per D4; kills `truncate_hash` | -| Provenance graph | `urn:atomic:provenance:blake3-256:` | | -| Attestation | `urn:atomic:attestation:blake3-256:` | hash of `.attest` object | -| Intent (incl. review) | `urn:atomic:project::intent:` | ULID lowercase canonical | -| Goal / Memory / Skill / ToolResult | `urn:atomic:project::{goal,memory,skill,tool-result}:` | | -| AC / Task / ScopeItem / Constraint | `urn:atomic:project::intent::{ac,task,scope,constraint}:` | qualified by parent intent | -| Session / Turn / Todo | `urn:atomic:project::session:[:turn: \| :todo:]` | session-qualified todos only | -| View | `urn:atomic:project::view:` | | -| File / Module | `urn:atomic:project::{file,module}:` | | -| Source entity | `urn:atomic:project::entity:#:[~]` | per D10; file stays in the ID, line becomes a literal shown beside it | -| Identity | `did:atomic:…` when known; else `urn:atomic:project::identity:` + `atomic:Unknown`-style finding | stops display-name identity merging | -| Concept | `urn:atomic:project::concept:` typed `skos:Concept` | | -| Finding | `urn:atomic:project::finding::` | e.g. `MissingHistoricalPlanId` | - -A single `atomic_core::kg::iri` module owns all minting and parsing (percent-encoding per -RFC 3987 path rules); producers never format IDs by hand. The FTS ranking prefix checks -(`txn/write/triples.rs:268-280`, duplicated in `read.rs`) switch to type lookups. - -### 3.3 Edge mapping (complete replacement table) - -Every row below is a test fixture for the coverage test (M1). - -**Classes** — `entity_type::*` and `kind` values → `rdf:type`: - -| Current | Canonical | -|---|---| -| `vault:Goal` / `goal` | `atomic:Goal` | -| `vault:Intent` / `intent` | `atomic:WorkIntent` (+ `FeatureIntent`/`BugIntent`/`ReviewIntent`/… from canonical `kind`; `ManagedIntent` from lifecycle status) | -| `vault:Memory` / `memory` | `atomic:Memory` | -| `vault:Skill` / `skill` | `atomic:Skill` | -| `vault:ToolResult` / `tool_result` | `atomic:ToolResult` | -| `vault:File` / `file` | `atomic:File` | -| `vault:Identity` / `identity` | `atomic:Identity` | -| `vault:Concept` / `concept` | `skos:Concept` | -| `vault:Session` / `session` | `atomic:Session` | -| `vault:SessionTurn` / `session_turn` | `atomic:SessionTurn` (+ `atomic:ManagedTurn` when `plan_id` present) | -| `vault:Todo` / `todo` | `atomic:Todo` | -| `vault:ProvenanceGraph` / `provenance` | `atomic:ProvenanceGraph` | -| `vault:Task` / `task` | `atomic:Task` | -| `vault:AcceptanceCriterion` / `acceptance_criterion` | `atomic:AcceptanceCriterion` | -| `vault:ScopeItem` / `scope` | `atomic:ScopeItem` | -| `vault:Constraint` / `constraint` | `atomic:Constraint` | -| `change` | `atomic:Change` | -| `view` | `atomic:View` | -| `module` | `atomic:Module` | -| `entity` | `atomic:SourceEntity` (language kind stays a literal `atomic:entityKind`) | -| *(new)* | `atomic:Attestation`, `atomic:Finding` | - -**Predicates** — split by endpoint class where overloaded: - -| Current | Endpoints | Canonical | -|---|---|---| -| `prov:hadMember` | session→turn | `atomic:hasTurn` (inverse `atomic:turnOfSession`) | -| `prov:hadPlan` | turn→intent | `atomic:hasPlan` (inverse `atomic:governsTurn`) | -| `prov:hadPlan` | session→intent | **dropped**; derivable as session `hasTurn`/`hasPlan` | -| `prov:generated` | turn→change | `atomic:generatedChange` ⊑ `prov:generated` (inverse `atomic:generatedBy`) | -| `prov:wasInformedBy` | turn→turn | `prov:wasInformedBy` (keep) | -| `prov:wasDerivedFrom` | memory→source; session→session (fork) | `prov:wasDerivedFrom` (keep; fork metadata stays edge-level) | -| `prov:used` | tool_result→file | `prov:used` (keep) | -| `prov:wasAssociatedWith` | tool_result→goal | `atomic:linkedGoal` | -| `atom:informedBy` | intent→memory/source | `atomic:informedBy` ⊑ `prov:wasDerivedFrom` | -| `vault:explainedBy` | turn→provenance | `atomic:explainedBy` | -| `vault:hasTodo` | turn→todo | `atomic:hasTodo` | -| `BLOCKS` / `BLOCKED_BY` | intent↔intent | `atomic:blocks` / `atomic:blockedBy` (inverses; emit one, infer the other) | -| `ASSIGNED_TO` | intent→identity | `atomic:assignedTo` | -| `LINKED_INTENT` | goal→intent | `atomic:linkedIntent` (target now manifest-resolved) | -| `LINKED_GOAL` | intent→goal | `atomic:linkedGoal` | -| `REFERENCES` | entry→memory, entry→file | `atomic:references` | -| `HAS_LABEL` | intent→concept | `atomic:hasLabel` | -| `HAS_TASK` | intent→task | `atomic:hasTask` | -| `HAS_ACCEPTANCE_CRITERION` | intent→AC | `atomic:hasAcceptanceCriterion` | -| `HAS_SCOPE_IN` / `HAS_SCOPE_OUT` | intent→scope | `atomic:hasScopeIn` / `atomic:hasScopeOut` ⊑ `atomic:hasScopeItem` | -| `HAS_CONSTRAINT` | intent→constraint | `atomic:hasConstraint` | -| `SATISFIES` | task→AC | `atomic:satisfies` | -| `TOUCHES` | task→file | `atomic:touches` | -| `SCOPE_OUT_FILE` | intent→file | `atomic:scopeOutFile` | -| `REMEDIATES` | intent→intent | `atomic:remediates` | -| `REVIEWS` | review intent→intent | `atomic:reviews` (inverse `atomic:reviewedBy`) | -| `DEPENDS_ON` | change→change | `atomic:directlyDependsOnChange` ⊑ `atomic:dependsOn` | -| `DEPENDS_ON` | intent→intent | `atomic:dependsOnIntent` ⊑ `atomic:dependsOn` | -| `MODIFIES` | change→file | `atomic:modifiesFile` ⊑ `atomic:modifies` | -| `MODIFIES` | change→entity | `atomic:modifiesEntity` ⊑ `atomic:modifies` (emitted on **both** bulk and record paths) | -| `ON_VIEW` | change→view | `atomic:directlyOnView` — **recomputed from `VIEW_CHANGES[view]` own entries**, not the working copy's desired view | -| `ON_VIEW` | session→view | `atomic:sessionViewContext` | -| `CHILD_OF` | view→view | `atomic:childViewOf` | -| `VISIBLE_FROM` | (never emitted) | `atomic:visibleFrom` — query-time operator only in Phase 1 | -| `DEFINES` | file→entity | `atomic:defines` | -| `EXPORTS` | file→entity | `atomic:exports` ⊑ `atomic:defines` | -| `INCLUDES` | file→file | `atomic:includes` | -| `IMPORTS` | (never emitted) | `atomic:imports` ⊑ `atomic:includes` (declared; extraction split deferred) | -| `CALLS` | entity→entity | `atomic:calls` | -| `PART_OF` | file→module / module→module | `atomic:filePartOfModule` / `atomic:modulePartOfModule` ⊑ `atomic:partOf` | -| `AUTHORED_BY` | change/goal/intent→identity | `atomic:authoredBy` ⊑ `prov:wasAttributedTo` (intents now emit it from `author`/`created_by`) | -| `vault:status`, `vault:priority` | literal | `atomic:status`, `atomic:priority` (datatype properties) | -| `vault:appliesTo` | entity→concept | `atomic:appliesTo` | -| `vault:blocks`… duplicate predicate constants | — | removed (uppercase twins already mapped above) | -| `dc:type` | — | removed (never emitted) | -| `dc:creator` | — | `atomic:authoredBy` | -| `dc:date` | — | `dcterms:created` / `dcterms:modified` from typed source fields | -| `dc:description` | — | `dcterms:description` | -| `skos:broader`/`narrower`/`related` | — | kept | -| `skos:subject` | — | removed; `dcterms:subject` | -| `rdf:type` | — | real type assertions (§3.4) | - -**New Phase 1 edges** (no current equivalent): - -| Edge | Source fact | +1. **`atomic:` namespace** — a canonical workflow ontology (`.ttl` + Rust registry). +2. **`ProjectId`** — a stable, unique project identity minted at `atomic init` and cast + into IRIs so that future cross-project SPARQL queries have a canonical namespace per + repository. +3. **Native SHACL engine** — parse and evaluate SHACL shapes against the property-graph + KG. No hand-coded Rust validators. Enterprise users add custom shapes in + `.atomic/shapes/*.ttl`; the engine evaluates them the same way it evaluates builtin + shapes. + +Everything else — projectors, storage cutover, inference, consumers — comes later. +This keeps Phase 1 reviewable while guaranteeing that IRIs are stable and validation +is real SHACL from day one. + +## What we deliberately skip in Phase 1 + +**No code-block-level mapping.** AST and Tree-Sitter already produce call graphs, +definitions, exports, imports, and entity relationships. Building a parallel system +to emit `atomic:calls`, `atomic:defines`, `atomic:exports`, `atomic:includes`, +`atomic:modifiesEntity`, and scope-qualified source-entity IDs duplicates information +we get from parsers for free. Those edges are deferred until we need them for queries +that the syntax index cannot answer (e.g. federated cross-project search). + +| Skipped | Why | |---|---| -| `Attestation atomic:coversChange Change` | `Attestation.changes_covered` | -| `Attestation atomic:attests SessionTurn` | attestation `session_id` + covered change ∈ turn `change_hashes` (D8: exact once the turn is recorded; interim inference only when unique) | -| `Attestation atomic:signedBy Identity` | `Attestation.signer` (`did:atomic` or MAC signer) | -| `Attestation atomic:previousAttestation Attestation` | `previous_attestation` | -| `Attestation atomic:verificationResult atomic:{Verified,Unverifiable,Failed}` | asserted **only** by the verifier pass (never inferred) | -| `Change atomic:implements Intent` | inferred: `generatedBy ∘ hasPlan` | -| `Change atomic:explainedBy ProvenanceGraph` | inferred: `generatedBy ∘ explainedBy` | -| `Change atomic:attestedBy Attestation` | inferred: inverse of `coversChange` | -| `Finding atomic:aboutResource X`, `atomic:findingCode` | historical gaps (`MissingHistoricalPlanId`, `MissingAttestation`, …) | - -### 3.4 Storage shape - -Keep the property-graph tables (they already serve FTS, neighbors, and UI) but make them -RDF-faithful: - -- `KgEdge.kind` → absolute predicate IRI. The write API takes a typed - `Predicate` (ontology constant), not `&str`; unknown IRIs are unrepresentable on write and - rejected on read. -- New field `KgEdge.graph: Asserted | Inferred` (serde default `Asserted`), with - `metadata.derivation = { rule, ontology_version, premises: [edge keys] }` on inferred edges. -- **Types in a dedicated index**, not as `rdf:type` edges: new multimap tables - `kg_types` (node IRI → class IRI) and `kg_types_rev` (class → node). Reason: storing - classes as edge targets would make every node a 2-hop neighbor of every same-typed node - under the kind-blind `kg_neighbors` BFS. RDF export emits them as `rdf:type` triples. - Entailed superclasses are stored with `graph = Inferred`. -- `KgNode.kind` is **removed**; display category derives from the most specific type. - `metadata.rdf_type` is removed. `KgNode.source` stays (it drives `kg_clear_derived`). -- `kg_index_meta["kg_generation"]` records `{ontology_version, projector_version, - inference_profile, built_at}` per the RFC's "projection generation". -- Register new tables in `schema.rs::visit_tables`; retire nothing (same table names, new - contents after rebuild). +| `atomic:defines`, `atomic:exports`, `atomic:calls`, `atomic:modifiesEntity` | Tree-Sitter / `atomic-semantic` resolves these on demand | +| `atomic:includes`, `atomic:imports` | File inclusion graphs are cheap to compute from source | +| `atomic:modulePartOfModule`, `atomic:filePartOfModule` | Module hierarchy derivable from paths | +| Scope-qualified entity IDs | Deferred to a phase that needs stable entity references | +| Parser changes for scope chains / parameter types | Not needed without entity materialization | +| `query callers` KG-backed rewrite | Live-parsed implementation stays | -## 4. Milestones +We keep **file-level** `atomic:modifiesFile` and `atomic:directlyOnView` because those +are workflow facts, not code-structure facts. -The RFC forbids a dual-write window, so M3, M4 and M7 land together on this branch and -merge as one release. M1, M2, M5 and M6 are additive and can be reviewed independently. - -### M1 — Ontology artifacts and term registry (additive, no behavior change) - -- `atomic-core/ontology/workflow-1.0.0.ttl`: OWL 2 RL ontology — classes (§3.3), object - and datatype properties with domain/range, inverses, subproperty hierarchy, the two - property chains, `atomic:dependsOn` as `owl:TransitiveProperty`, safe disjointness - (`AIContribution`/`WorkIntent` deferred with Phase 2 classes), and `atomic:Execution` / - `atomic:partOfTurn` declared but not projected (D11). -- `atomic-core/ontology/workflow-1.0.0-dl.ttl`: optional DL companion (can slip). -- Rewrite `atomic-core/src/pristine/ontology.rs` as `atomic_core::kg::ontology`: - `Class` and `Predicate` enums with `iri()`, `curie()`, domain/range metadata, and a - `PREFIXES` table for display compaction. Embed the `.ttl` via `include_str!`. -- `LEGACY_TERMS` table (test-only): every old constant → `Replacement::{One(term), - SplitByEndpoint([(domain, range, term)]), Removed(reason)}`. -- Tests: - - coverage: each of the 30 + 16 + 29 legacy constants appears exactly once in - `LEGACY_TERMS`, and every `One`/`Split` target exists in the TTL; - - TTL ↔ Rust consistency: every Rust term is declared in the TTL with matching - domain/range and vice versa (parse with `oxttl`/`oxrdf` as **dev-dependencies only**); - - no emitted string starts with `vault:`, `atom:`, or is SCREAMING_CASE (enforced again in - M4 over a rebuilt fixture repo). -- Doc: update the AGENTS.md table section; amend the RFC to state that frozen historical - document formats are read-only inputs, not a compatibility vocabulary (D3); publish - dereferenceable docs at the namespace (ops task, tracked separately). - -### M2 — Project identity and IRI minting - -- `atomic-canonical` ctx v2 on `https://atomic.dev/ns/workflow#` with an object - schema-version bump; v1 frozen for verification only; all new writes are v2 (D3). -- `atomic-canonical`: new `ProjectIdentity` node (`issuer` DID, `createdAt`, 256-bit random - id, schema version, proof), gate rules, JCS hash. Project ID = base32 of the random id. -- Storage and transport per D5: its own pristine node type plus a - `PROJECT_IDENTITY` table, carried by clone/push/pull as a repository-level object). - `atomic init` mints it; upgrading an existing repo mints it on the first writable open by - a signing identity, otherwise fails with `KgNeedsProjectIdentity` and an actionable message. -- `atomic project identity show|fork` (fork records `prov:wasDerivedFrom` origin). Rotation - chain deferred, but the object schema reserves `previousIdentity`. -- Import collision check: same id + different verified object → reject. -- `atomic_core::kg::iri`: mint/parse for every row of §3.2; property tests for round trip and - percent-encoding of paths with spaces, `#`, `:`, non-ASCII. -- Strict canonical base32 parser for content-addressed URNs (D4): rejects lowercase, - padding, wrong length and nonzero trailing bits; tested against `Hash::to_base32`. - -### M3 — Storage generation 2 (lands with M4/M7) - -- `KgEdge.graph`, typed `Predicate` write API, `kg_types`/`kg_types_rev` tables, removal of - `KgNode.kind`/`metadata.rdf_type`, read-side rejection of non-IRI kinds. -- Capability fence `kg-semweb-v1` in `capability.rs` so older binaries fail closed. -- `KG_GENERATION` constant plus `kg_index_meta["kg_generation"]`. The upgrade rebuild runs - automatically on the first writable open and via `atomic query reindex --full`, - regardless of repository size. The marker is written in the same transaction as the final - rebuild batch, so the rebuild happens exactly once per generation (D6). The upgrade path - clears all `kg_*` tables and runs M4's reconcile. -- `kg_neighbors` gains an optional predicate filter and direction; unfiltered behavior kept - for `vault context`/RAG (they now filter by type, not `kind`). - -### M4 — Projector rewrite and unified reconcile (lands with M3/M7) - -- New module `atomic-repository/src/kg/` with one projector per family, all emitting via - the typed API: - - `vault.rs` (from `vault_triples.rs`): intents (canonical `kind` → intent subclass; - lifecycle → `ManagedIntent`), memories, goals, skills, tool results, decomposition - children, `:::ref` edges split by edge name, author edges for intents. - - `vcs.rs` (from `vault_kg_enrich.rs`): views + `childViewOf`, `directlyOnView` from - `VIEW_CHANGES`, changes (full hash), `directlyDependsOnChange`, `modifiesFile`, - `authoredBy`, modules/files. - - `ast.rs`: entities, `defines`/`exports`/`includes`/`calls`, `modifiesEntity` on both - paths. - - Scope-qualified entity IDs (D10): add `Entity.scope` (enclosing containers) to all - seven `atomic-semantic` parsers plus normalized parameter types for overloading languages; - update `query callers` display and `extract_path_from_id` to read node data; print - `path:startLine-endLine` beside every entity ID; accept `path:line` and - `path#kind:scope::name` as CLI entity arguments. - - `session.rs`: move the vocabulary out of `atomic-core/.../session_kg.rs` (keep the - record-time hook in core, but it calls the shared emitter), sessions, turns, todos, - provenance graphs, `ManagedTurn` typing. - - `attestation.rs` (**new**): load `.attest` objects, emit Attestation node and §3.3 edges; - run signature verification (reuse `provenance_gate` verification + MAC provider) and - assert `verificationResult`. - - `findings.rs` (**new**): `MissingHistoricalPlanId`, `MissingProvenance`, - `MissingAttestation`, `UnverifiableAttestation` — never guessed links. -- `reconcile(scope)` pipeline (RFC "Sync and enrichment contract"): ingest vault → discover - VCS/session/provenance/attestation objects → project → verify → infer (M5) → validate - (M6) → publish generation + stats. `vault sync`, `query enrich`, `query reindex`, clone, - pull and import all call it (full or incremental scope). Record-time incremental - enrichment calls the same projectors for the new objects. -- Determinism: projectors sort inputs; output digest = blake3 over sorted canonical N-Triples. - -### M5 — OWL 2 RL inference (additive engine; wired in M4) - -- `atomic_core::kg::reason`: forward-chaining to fixpoint over rules **generated from the - TTL** (so the ontology is data): `rdfs:subClassOf`, `rdfs:subPropertyOf`, - `owl:inverseOf`, `rdfs:domain`/`rdfs:range` typing, 2-step `owl:propertyChainAxiom`. -- Transitive properties (`atomic:dependsOn`, `atomic:ancestorView`) are **not** - materialized; query operators compute closure from an explicit seed set. -- Every inferred edge/type carries a derivation record; inferred edges never overwrite an - asserted edge with the same key (asserted wins, derivation is appended). -- `verificationResult` and any proof-validity term are excluded from rule heads. -- Tests: golden fixtures per rule; idempotence (re-running adds nothing); order-independence. - -### M6 — Validation (SHACL as data + hand-coded checks) - -Per D1: - -- `atomic-core/ontology/shapes/projection-1.0.0.ttl` and `publication-1.0.0.ttl` - (normative). Projection profile: shortcut agreement, referential existence, full-hash IDs, - every generated change has `implements`/`explainedBy`/`attestedBy`. Publication profile: - managed turn has exactly one `hasPlan`; managed intent governs ≥1 turn; turn has exactly - one provenance graph; every generated change is covered by the session attestation; - attestation has signer + verified proof. -- Rust `kg::validate` implements each shape rule (one function per `sh:` rule ID, as in - `atomic-canonical/src/gate.rs`) over a small graph-access trait, and produces a report keyed - by shape/path IRIs. If it grows into a general SHACL evaluator, extract it into a - dedicated `atomic-core`-free crate that CB can consume or that can be published (D1). -- **Publication profile hooks into `provenance_gate.rs`** (the existing boundary for - shared-view insert, triage promotion, push, Git export) behind policy - `[provenance] require_plan = true` (default off). Absent `plan_id` fails publication. -- Projection failure → deterministic repair (re-run projectors for the offending subjects) - → revalidate → if still failing, generation is marked non-conforming and query output says - so. -- `atomic query export --format nquads|turtle [--inferred]` (hand-rolled serializer, no - deps) so the CB workflow can run real SHACL; differential fixture corpus checked into - `atomic-core/ontology/shapes/fixtures/` with expected conforms/violations for both the - Rust checks and CB. - -### M7 — Consumer cutover (lands with M3/M4) - -- `query_plan.rs`: new steps `Relation { predicate, direction }`, `DirectViewMembers`, - `ImplementingChanges`, `DependencyClosure { seeds }`, `TypeFilter`; `Filter` on `kind` - becomes type-based; plan JSON schema version bump. -- CLI `query`: `callers` → `atomic:calls`; graph colour map keyed by IRI; DOT/text output - compacts IRIs to CURIEs via `PREFIXES`; JSON emits absolute IRIs; `ask` prompt lists - canonical predicates and IRI forms. -- Triage (`atomic-repository/src/triage/report.rs` `kind_is`, `repository/triage.rs`), `vault_intent.rs` informedBy checks, - `vault/context.rs` memory filter (type `atomic:Memory`). -- libatomic daemon duplicates (`services_agent.rs`, `services_query.rs`); `rpc.rs`. -- Proto: document `KGEdge.relation` as an absolute IRI; add `types` repeated field to the node - message; bump knowledge service API version. -- No compatibility shim for JSON/RPC consumers; the release is announced as breaking (D7). -- Update ~110 test assertion sites; add a lint test that greps the workspace for legacy - kind literals outside `LEGACY_TERMS`. - -## 5. Acceptance tests for Phase 1 - -Mapped to RFC acceptance criteria 1-4, 7, 8: - -1. **Closure path (AC1):** integration test drives a managed agent turn with `plan_id` → - record → attest; asserts the change reaches intent, provenance graph, attestation, - session and turn via asserted edges, and that `implements`/`explainedBy`/`attestedBy` - shortcuts exist and agree. -2. **Repair (AC2):** delete all `kg_*` contents → `reconcile --full` → graph digest equals - the incrementally built graph's digest. -3. **Inference (AC3):** golden test for inverses, hierarchy and both property chains; - rerun is a no-op. -4. **Validation (AC4):** fixtures for each shape rule (positive + negative); publication - gate refuses push when `require_plan` and a managed turn lacks `plan_id`. -5. **Historical gaps (AC7):** pre-`plan_id` turn yields `MissingHistoricalPlanId` finding and - **no** `implements` edge. -6. **Coverage (AC8):** M1 coverage test + workspace lint for legacy literals. -7. **Membership:** `directlyOnView` for a change inserted into `dev` from `feature` lists it on - both views (both have it in `VIEW_CHANGES`) and never on the working copy's desired view - unless it is a member. - -## 6. Suggested order of work - -1. M1 (ontology + registry + coverage test) — establishes the vocabulary everyone reviews. -2. M2 (project identity + IRIs) — unblocks every projector. -3. M5 (reasoner) in parallel with M3 (storage) — independent. -4. M4 (projectors + reconcile) on top of M2/M3/M5. -5. M6 (validation + export + gate). -6. M7 (consumers), then the end-to-end acceptance suite; merge M3+M4+M6+M7 as one release. - -## 7. Risks - -- **Rebuild cost on large repos** (AST + CALLS fan-out). Mitigation: D6, progress output, - incremental reconcile by changed objects. -- **Edge-key growth** from absolute IRIs (~40-80 bytes per key). Acceptable for redb; can - intern predicate IRIs to `u16` later if measured. -- **Attestation verification on receiving remotes**: MAC attestations are unverifiable - without the key (per `provenance_gate`); they must project as `Unverifiable`, never - `Verified`, and the projection profile must treat that as a reported state, not a repair loop. -- **Out-of-repo clients** break on release day by design (D7); the API version bump makes the failure loud. -- **`atom:` vs `atomic:` confusion** in reviews: write D3 (v1 frozen and read-only, v2 for new objects) into AGENTS.md. - -## 8. Decisions and open questions - -### 8.1 Resolved - -**D1 — SHACL enforcement location: option (a).** Shapes ship as `.ttl` data and are the -normative source. Atomic implements hand-coded checks that mirror them and exports RDF for -the CB workflow. Implementation rule: -- Start with hand-coded checks in `kg::validate`, one function per shape rule (the - `atomic-canonical/src/gate.rs` pattern). -- Write the checks against a small graph-access trait (subjects, objects, types, - literals), not against redb or `KgNode` directly, so they can move later without a - rewrite. -- **Extraction trigger:** if the checks start to look like a real SHACL implementation - (generic `sh:` constraint dispatch, property-path evaluation, `sh:node`/`sh:sparql` - semantics, a shapes loader), move them into a dedicated crate (working name - `atomic-shacl`). That crate depends only on an RDF term model, never on `atomic-core`, so - CB can pull it in, it can be relocated, or it can be published separately. - -**D2 — OWL 2 RL implementation (RFC open question 1): option (a), written in-house.** Options considered: - -| Option | What it is | Cons | +## 1. Phase 1 scope + +### 1.1 Ontology (`atomic-core/ontology/workflow-1.0.0.ttl`) + +**Namespace:** `atomic:` → `https://atomic.dev/ns/workflow#` + +Ontology IRI: `https://atomic.dev/ns/workflow` +Version IRI: `https://atomic.dev/ns/workflow/1.0.0` + +**Classes** (workflow-level only): + +| Canonical class | Superclass | Note | |---|---|---| -| **(a) In-house rule engine** (chosen) | Forward-chaining over rules generated from our TTL: subClassOf/subPropertyOf, inverseOf, domain/range, 2-step property chains | We own correctness and must test every rule. It covers only the RL rules we implement, so the TTL loader must **reject** unsupported axioms (fail closed, the oxirs lesson) rather than silently ignore them | -| (b) `reasonable` crate | OWL 2 RL reasoner (Brick Schema project), Datalog on `datafrog` + `oxrdf` | Latest release is a prerelease (`0.5.0-a1`) from a small research-driven project. It materializes the full RL rule set, which produces many trivial triples (`owl:Thing`, reflexive `sameAs`, …) we'd filter out. It works on its own in-memory graph, so every reconcile copies the KG in and out of redb. It keeps no per-triple derivation records (rule + premises), which the RFC requires | -| (c) Datalog library (`datafrog` 2.0, `ascent` 0.8, `crepe`) + our rules | We write the RL rules; the library evaluates them to a fixpoint | Fixpoint evaluation is the easy part, and we still write and test every rule. `ascent`/`crepe` compile rules from macros, so the ontology stops being data. Derivation tracking is still ours. Worth it only if (a) turns out too slow | -| (d) DL/EL reasoners (`rustdl` OWL 2 DL on `horned-owl`, `whelk-el` OWL 2 EL) | Description-logic classification | Wrong job: they classify the ontology's class hierarchy rather than deriving instance edges, so they don't scale to deriving `implements` across a large KG. The RFC deliberately keeps DL off the conformance path | -| (e) RDF store + rules (`oxigraph` 0.5) | SPARQL store; rules written as `CONSTRUCT` queries looped to a fixpoint | No OWL reasoning built in. A second database next to redb (two copies to keep in sync), a large dependency tree, and we still write the rules by hand | -| (f) External (Jena, RDFox, GraphDB, or inference inside CB like SHACL) | Mature reasoners outside the binary | JVM or service dependency. Local queries need the derived shortcuts (`implements`, `attestedBy`) in the local KG, so offline repos would lose them, and query correctness would depend on an external workflow | - -Decision: (a). The TTL loader rejects any axiom outside the supported rule set. Option (c) -is the fallback only if profiling shows the fixpoint loop is a bottleneck. - -**D3 — Canonical document namespace: option (b), applied narrowly.** -- New signed objects use a **ctx v2** built on the canonical `atomic:` workflow namespace - (`https://atomic.dev/ns/workflow#`), under a bumped object schema version. -- Existing signed objects stay byte-for-byte unchanged. -- ctx v1 is **frozen** as a read-only historical input format. The KG projector translates - it to canonical terms through a fixed, tested mapping, so `atom:` terms never reach KG - storage, queries, APIs, or new objects. -- `https://atomic.dev/ns#` stays published with permanently fixed meaning (option (c) is - ruled out). -- Why (b) over (a): the RFC gives `atomic:` sole ownership of workflow concepts, requires - every producer to emit canonical terms, and rejects `atom:*` variants after migration. - Byte stability protects only *existing* objects, not future ones. Both options read v1 - forever; the difference is that (a) keeps producing the legacy format, while narrow (b) - closes it off. -- Consequences: - - `atomic-canonical` gains ctx v2 plus a schema-version bump for Intent, Memory, - delegation, PROV graph and `ProjectIdentity` objects. - - The gate accepts v1 only to verify existing objects. All new writes are v2, including a - revision of a v1 object (its `previousRevision` points at the v1 hash). - - Work lands in M2 (documents) and M4 (the v1 → canonical mapping in the vault projector). -- **RFC clarification required** (M1 doc task): frozen historical document formats are - read-only inputs, not a compatibility vocabulary. - -**D4 — Content-hash URN encoding: base32, algorithm-qualified, exactly specified.** -`urn:atomic:change:blake3-256:`. The RFC doesn't choose an encoding, so this is -a judgment call. -- Why base32: - - It's already used in `urn:atomic:change:`, so hex would add a second identity break on - top of the full-hash migration. - - It's shorter: 52 characters versus 64 for hex. - - RFC 4648 base32 is a multibase encoding, which fits the RFC's "multihash-compatible" goal. -- Normative encoding (the RFC's acceptance criteria depend on byte-identical IDs across - implementations): - 1. **Alphabet:** standard RFC 4648 §6 (`A–Z2–7`), not base32hex, Crockford, or a custom - variant. - 2. **Case:** uppercase is canonical (confirmed: keep current behavior). This is exactly what `Hash::to_base32` emits today - (`atomic-core/src/types/hash.rs:260`) and matches every existing change ID on disk and - in the CLI. The multibase equivalent is prefix `B`. - 3. **No padding:** 52 characters, and the final 4 bits must be zero. - 4. **Strict parsing:** reject any non-canonical form rather than normalizing it. The - existing `from_base32` is case-insensitive and doesn't check the trailing bits, so URN - parsing gets a new strict parser. Lenient prefix lookup in the CLI is unaffected. - 5. A test proves `Hash::to_base32` output round-trips through the strict parser. -- Scope: applies to every content-addressed URN (change, provenance, attestation, - project identity, file-version) and to `blake3-256:` evidence digests. -- Cost: standard BLAKE3 tools (`b3sum`) print hex, so external verifiers get a documented - conversion step. - -**D5 — Where the signed `ProjectId` lives: option (a), its own pristine node type.** The -first recommendation (a tracked `.vault/project/identity.jsonld` file) assumed snapshot -semantics. In atomic a `.vault/` file is content recorded *as a change* in the graph, so: -- its existence and bytes depend on which view you're on, because of the view filter; -- it can be unrecorded, edited by a later change, or put in conflict; -- it can diverge between views. - -Project identity is a repository-level fact that must be the same on every view and -survive insert/unrecord. It also can't be derived from changes, because changes are global -content-addressed objects: the same change can belong to many projects, and nothing -guarantees a unique genesis change. - -| Option | Cons | -|---|---| -| **(a) Dedicated content-addressed, signed object registered as its own pristine node type** (`node_type::PROJECT = 4`, next to CHANGE/TAG/ATTESTATION/PROVENANCE) (chosen). Bytes in the change store; a singleton pristine table `PROJECT_IDENTITY` holds the current hash plus the rotation chain. It is not in any view's `VIEW_CHANGES`, so no view filters, inserts or unrecords it | Needs a new transport object class. Today the push/pull contract excludes sidecars (`atomic-remote/src/sync.rs:476`), so clone/push/pull must learn to carry it, and the server should bind the identity hash to its `ProjectInfo.id` | -| (b) Metadata on the root view's `ViewState` | Still view-scoped: it travels and changes with one view, and other views can't verify it independently | -| (c) Server-assigned `ProjectInfo.id` (UUID) | Requires a remote, so local-only repos have no identity; it's not signed by the user and can't be verified offline | -| ~~(d) Tracked `.vault/` file~~ | Rejected: view-dependent, as explained above | - -Under (a), fork is an explicit command that mints a new object with `prov:wasDerivedFrom` -pointing at the origin. The issuer DID binds the project to an identity now; org binding -comes later through delegation (`ResourceRef`). - -**D6 — KG rebuild on upgrade: both triggers, regardless of repository size.** The rebuild -runs automatically on the first writable open, and `atomic query reindex --full` runs it -explicitly. It must happen **exactly once** per KG generation: -- A compiled `KG_GENERATION` constant. A rebuild runs only when the stored - `kg_index_meta["kg_generation"]` is lower. -- The generation marker is written in the **same write transaction as the final rebuild - batch**. An interrupted rebuild leaves no marker and restarts cleanly (clear, then - rebuild) on the next writable open. It never leaves a half-marked KG. -- Repositories created by the new binary (`init`, `clone`) are stamped with the current - generation at creation and never trigger an upgrade rebuild. -- Pull, import, incremental reconcile and `reindex --full` never lower or clear the marker; - `--full` re-stamps the same generation. -- `KG_GENERATION` bumps only on breaking ontology or projector changes, not on every - release. Routine upgrades don't rebuild. -- Read-only opens on an old-generation KG return `KgNeedsRebuild`, naming the command to run. -- Tests: a second open after the upgrade performs zero rebuild work; a rebuild killed - midway completes on the next open; a fresh clone never rebuilds. - -**D7 — External JSON/RPC clients:** this release is announced loudly as breaking, so there -is no compatibility shim and no coordination gate on `noname`/`the-hive`. The API version -bump stays, so stale clients fail loudly rather than misreading data. - -**D8 — Attestation → turn mapping: option (b), with (a) as the interim.** Record the turn -number in the attestation format at capture time, because the RFC classes turn identity as -a capture-time fact. Until the format change lands, infer the turn only when exactly one -turn's `change_hashes` ⊆ `changes_covered`. Otherwise emit an `AmbiguousAttestationTurn` -finding and no `attests` edge. - -**D9 — Domain separation: provided by project identity, used later by federation.** -Each atomic repository is its own domain; federation is how domains are kept apart in -cross-project search (e.g. `testing/hello-world` vs `production/hello-world`, and later -scoping by identity/org so a caller never searches someone else's projects). Content -hashes stay global: the same change bytes hash the same in every project, which insert, -pull and clone depend on. -- Phase 1 obligation: a durable, signed project identity (D5) that later phases can read, - plus project-qualified IRIs for mutable resources (§3.2). -- Deferred to Phase 5: the catalog, named project graphs, explicit query scope and - authorization that use that identity to keep projects separate in federated queries. -- No change-format or hash-construction change in Phase 1. - -**D10 — Source-entity identity: option (b), scope-qualified IDs.** Where it comes from: -- The RFC's node model lists Entity as a "project-qualified **stable** entity URN" from - the syntax/semantic index, and goal 6 requires portable identities. -- Today's entity IDs are `entity:{file}:{name}:{line}` (`vault_kg_enrich.rs:587`), so they - contain the line number. Adding one line above a function changes its ID. -- As a result, `modifiesEntity` edges from older changes point at IDs that no longer - exist, one function can end up with several IDs over time, and a full rebuild produces - different IDs than incremental enrichment. That breaks the "full rebuild equals - incremental" acceptance test and blame-style queries. -- The line number is also doing a hidden job: the extractor (`atomic-semantic`, `Entity` - at `entity.rs:123`) emits methods by bare name (`fmt`, `find`) with no enclosing type, so - `impl Display for User { fn fmt }` and `impl Display for Order { fn fmt }` in one file are - told apart only by their lines. - -**Decision: a scope-qualified name instead of the line.** - -```text -urn:atomic:project::entity:#:::[()][~] +| `atomic:WorkIntent` | `owl:Thing` | Base for all intents | +| `atomic:FeatureIntent` | `atomic:WorkIntent` | From canonical `kind` | +| `atomic:BugIntent` | `atomic:WorkIntent` | From canonical `kind` | +| `atomic:ReviewIntent` | `atomic:WorkIntent` | From canonical `kind` | +| `atomic:ManagedIntent` | `atomic:WorkIntent` | Lifecycle status | +| `atomic:Goal` | `owl:Thing` | | +| `atomic:Memory` | `owl:Thing` | | +| `atomic:Skill` | `owl:Thing` | | +| `atomic:ToolResult` | `owl:Thing` | | +| `atomic:Task` | `owl:Thing` | | +| `atomic:AcceptanceCriterion` | `owl:Thing` | | +| `atomic:ScopeItem` | `owl:Thing` | | +| `atomic:Constraint` | `owl:Thing` | | +| `atomic:Todo` | `owl:Thing` | | +| `atomic:Session` | `owl:Thing` | | +| `atomic:SessionTurn` | `owl:Thing` | | +| `atomic:ManagedTurn` | `atomic:SessionTurn` | When `plan_id` present | +| `atomic:ProvenanceGraph` | `owl:Thing` | | +| `atomic:Attestation` | `owl:Thing` | | +| `atomic:Finding` | `owl:Thing` | Historical-gap findings | +| `atomic:Change` | `owl:Thing` | | +| `atomic:View` | `owl:Thing` | | +| `atomic:File` | `owl:Thing` | Workflow artifact, not code entity | +| `atomic:Identity` | `owl:Thing` | | +| `atomic:Concept` | `skos:Concept` | | + +**Object properties** (workflow edges only): + +| Property | Domain | Range | Inverse | Super | +|---|---|---|---|---| +| `atomic:hasTurn` | `atomic:Session` | `atomic:SessionTurn` | `atomic:turnOfSession` | — | +| `atomic:hasPlan` | `atomic:SessionTurn` | `atomic:WorkIntent` | `atomic:governsTurn` | — | +| `atomic:generatedChange` | `atomic:SessionTurn` | `atomic:Change` | `atomic:generatedBy` | `prov:generated` | +| `atomic:informedBy` | `atomic:WorkIntent` | `atomic:Memory` | — | `prov:wasDerivedFrom` | +| `atomic:explainedBy` | `atomic:SessionTurn` | `atomic:ProvenanceGraph` | — | — | +| `atomic:hasTodo` | `atomic:SessionTurn` | `atomic:Todo` | — | — | +| `atomic:blocks` | `atomic:WorkIntent` | `atomic:WorkIntent` | `atomic:blockedBy` | — | +| `atomic:assignedTo` | `atomic:WorkIntent` | `atomic:Identity` | — | — | +| `atomic:linkedIntent` | `atomic:Goal` | `atomic:WorkIntent` | — | — | +| `atomic:linkedGoal` | `atomic:WorkIntent` | `atomic:Goal` | — | — | +| `atomic:references` | `owl:Thing` | `owl:Thing` | — | — | +| `atomic:hasLabel` | `atomic:WorkIntent` | `skos:Concept` | — | — | +| `atomic:hasTask` | `atomic:WorkIntent` | `atomic:Task` | — | — | +| `atomic:hasAcceptanceCriterion` | `atomic:WorkIntent` | `atomic:AcceptanceCriterion` | — | — | +| `atomic:hasScopeIn` | `atomic:WorkIntent` | `atomic:ScopeItem` | — | `atomic:hasScopeItem` | +| `atomic:hasScopeOut` | `atomic:WorkIntent` | `atomic:ScopeItem` | — | `atomic:hasScopeItem` | +| `atomic:hasConstraint` | `atomic:WorkIntent` | `atomic:Constraint` | — | — | +| `atomic:satisfies` | `atomic:Task` | `atomic:AcceptanceCriterion` | — | — | +| `atomic:touches` | `atomic:Task` | `atomic:File` | — | — | +| `atomic:scopeOutFile` | `atomic:WorkIntent` | `atomic:File` | — | — | +| `atomic:remediates` | `atomic:WorkIntent` | `atomic:WorkIntent` | — | — | +| `atomic:reviews` | `atomic:ReviewIntent` | `atomic:WorkIntent` | `atomic:reviewedBy` | — | +| `atomic:directlyDependsOnChange` | `atomic:Change` | `atomic:Change` | — | `atomic:dependsOn` | +| `atomic:dependsOnIntent` | `atomic:WorkIntent` | `atomic:WorkIntent` | — | `atomic:dependsOn` | +| `atomic:modifiesFile` | `atomic:Change` | `atomic:File` | — | `atomic:modifies` | +| `atomic:directlyOnView` | `atomic:Change` | `atomic:View` | — | — | +| `atomic:sessionViewContext` | `atomic:Session` | `atomic:View` | — | — | +| `atomic:childViewOf` | `atomic:View` | `atomic:View` | — | — | +| `atomic:visibleFrom` | — | — | — | Query-time operator | +| `atomic:authoredBy` | `owl:Thing` | `atomic:Identity` | — | `prov:wasAttributedTo` | +| `atomic:coversChange` | `atomic:Attestation` | `atomic:Change` | — | — | +| `atomic:attests` | `atomic:Attestation` | `atomic:SessionTurn` | — | — | +| `atomic:signedBy` | `atomic:Attestation` | `atomic:Identity` | — | — | +| `atomic:previousAttestation` | `atomic:Attestation` | `atomic:Attestation` | — | — | +| `atomic:implements` | `atomic:Change` | `atomic:WorkIntent` | — | — | +| `atomic:attestedBy` | `atomic:Change` | `atomic:Attestation` | — | — | +| `atomic:aboutResource` | `atomic:Finding` | `owl:Thing` | — | — | + +Note: `atomic:dependsOn` and `atomic:ancestorView` are declared `owl:TransitiveProperty` +but are **not** materialized in Phase 1. + +**Datatype properties:** + +| Property | Domain | Range | +|---|---|---| +| `atomic:status` | `atomic:WorkIntent` | `xsd:string` | +| `atomic:priority` | `atomic:WorkIntent` | `xsd:string` | +| `atomic:findingCode` | `atomic:Finding` | `xsd:string` | + +**Property chains** (declared in TTL, materialized in Phase 2): + +- `atomic:generatedChange ∘ atomic:hasPlan` → `atomic:implements` +- `atomic:generatedChange ∘ atomic:explainedBy` → `atomic:explainedBy` (on Change) + +**What the ontology does NOT contain:** + +`atomic:defines`, `atomic:exports`, `atomic:calls`, `atomic:includes`, +`atomic:imports`, `atomic:modulePartOfModule`, `atomic:filePartOfModule`, +`atomic:modifiesEntity`, and the `atomic:SourceEntity` class are omitted. They +describe code structure inside files; that is the syntax index's job. + +### 1.2 Rust term registry (`atomic_core::kg::ontology`) + +Rewrite `atomic-core/src/pristine/ontology.rs`: + +```rust +pub enum Class { WorkIntent, FeatureIntent, ... } +pub enum Predicate { HasTurn, HasPlan, ... } + +impl Class { + pub fn iri(&self) -> &'static str; + pub fn curie(&self) -> &'static str; + pub fn parents(&self) -> &'static [Class]; +} + +impl Predicate { + pub fn iri(&self) -> &'static str; + pub fn curie(&self) -> &'static str; + pub fn domain(&self) -> Option; + pub fn range(&self) -> Option; + pub fn inverse(&self) -> Option; + pub fn super_predicate(&self) -> Option; +} + +pub static PREFIXES: &[(&str, &str)] = &[ + ("atomic", "https://atomic.dev/ns/workflow#"), + ("prov", "http://www.w3.org/ns/prov#"), + ("dcterms", "http://purl.org/dc/terms/"), + ("skos", "http://www.w3.org/2004/02/skos/core#"), + ("rdf", "http://www.w3.org/1999/02/22-rdf-syntax-ns#"), + ("rdfs", "http://www.w3.org/2000/01/rdf-schema#"), + ("owl", "http://www.w3.org/2002/07/owl#"), + ("xsd", "http://www.w3.org/2001/XMLSchema#"), +]; ``` -| Part | Meaning | -|---|---| -| `` | file path, percent-encoded (as today) | -| `` | `EntityKind` (`function`, `method`, `class`, `interface`, `type-alias`, `const`, `enum`, `module`, …) | -| `::` | the chain of enclosing containers plus the name, joined by `::` in every language, each segment percent-encoded | -| `()` | only in languages with overloading (Java, C++, Swift, TypeScript overload signatures): normalized parameter types from the signature | -| `~` | last-resort ordinal, by order of appearance, when everything above still collides (e.g. two `#[cfg]` variants of the same `fn`) | +Embed the TTL via `include_str!("../../ontology/workflow-1.0.0.ttl")` so the Rust +source and the TTL cannot drift. + +### 1.3 Project identity (`atomic_core::kg::project_id` + `atomic_core::kg::iri`) + +**Design goal:** a single stable identifier per repository that survives views, +inserts, unrecords, and clone/pull/push, and becomes the `` component in +project-qualified IRIs. + +**Format:** +- 256-bit random ID generated at `atomic init` time. +- Canonical encoding: uppercase RFC 4648 base32 (52 characters, no padding), same + alphabet as `Hash::to_base32`. +- Display form: the base32 string. +- URN prefix: `urn:atomic:project:`. + +**Storage:** +- Dedicated table `PROJECT_IDENTITY` in pristine (not `VIEW_CHANGES`, therefore + view-independent and immune to insert/unrecord). +- Schema: `current_hash` → the content-addressed object hash of the signed identity + object; `created_at` → timestamp. A `previous` chain column is reserved for rotation + (deferred), so the schema is forward-compatible. + +**Identity object (content-addressed, signed JSON-LD):** + +```jsonld +{ + "@context": "https://atomic.dev/ns/ctx.jsonld", + "@type": "atomic:ProjectIdentity", + "atomic:projectId": "ABCD...EFGH", // the 52-char base32 + "atomic:createdAt": "2026-01-15T09:00:00Z", + "atomic:schemaVersion": 2, + "atomic:issuer": "did:atomic:..." // optional in Phase 1; may be unknown +} +``` -Line, end line, column and signature become literals on the node (`atomic:startLine`, …). +- Content hash = `blake3(JCS(normalized_bytes))`. +- Signed by the initializing identity if available; unsigned is allowed in Phase 1 + (the object schema reserves the `proof` field). + +**Why not `.vault/` or view metadata?** +A `.vault/` file is a change-tracked artifact. It can be absent on some views, +in conflict, or unrecorded. Project identity must be the same on every view and +survive all graph mutations. The only place that guarantees this is a repository-level +pristine table outside the view system. + +**Transport:** +- Clone / pull / push carry the project identity object as a repository-level + sidecar (not a change). The receiving side stores it in its own `PROJECT_IDENTITY` + table if the table is empty; if already populated, the clone keeps its own identity + (a cloned project is a new project). +- No fork command in Phase 1; fork semantics (new identity with + `prov:wasDerivedFrom` pointing at origin) come later. + +**API:** + +```rust +// atomic-core/src/kg/project_id.rs +#[derive(Clone, Debug, PartialEq, Eq, Hash)] +pub struct ProjectId([u8; 32]); + +impl ProjectId { + pub fn generate() -> Self; // OsRng → 32 bytes + pub fn from_base32(s: &str) -> Result; + pub fn to_base32(&self) -> String; // 52-char uppercase, no padding + pub fn to_urn(&self) -> String; // urn:atomic:project: +} + +// atomic-core/src/kg/iri.rs +pub fn mint_intent_iri(pid: &ProjectId, ulid: &str) -> String; +pub fn mint_goal_iri(pid: &ProjectId, encoded_name: &str) -> String; +pub fn mint_memory_iri(pid: &ProjectId, encoded_name: &str) -> String; +pub fn mint_session_iri(pid: &ProjectId, sid: &str) -> String; +pub fn mint_view_iri(pid: &ProjectId, view_name: &str) -> String; +pub fn mint_file_iri(pid: &ProjectId, path: &str) -> String; +// ... and parser functions for the reverse direction +``` -The file stays in the ID. The examples below show only the fragment after `#`; the full ID -for the `fmt` example is -`urn:atomic:project::entity:src%2Flib.rs#method:::fmt`. +All minting functions percent-encode path/ name segments per RFC 3987 and guarantee +round-trip stability. Property tests cover paths with spaces, `#`, `:`, non-ASCII, +and the empty string. -**Finding code still works, and line numbers stay visible.** Current consumers: +**CLI:** -| Consumer | Where it gets the line today | Change needed | -|---|---|---| -| `query entities ` | parses the file live with tree-sitter; never reads KG IDs (`query/mod.rs:1292-1342`) | none | -| `query neighbors` / `search` text and JSON | node metadata `line`/`end_line` (`query/mod.rs:585, 648-665`) | none: the line stays on the node | -| `query callers` display | **parses the ID**, stripping the trailing `:line` (`query/mod.rs:1267-1277`) | format `path:line` from node data | -| Search ranking `extract_path_from_id` | **parses the ID** with `rsplitn(3, ':')` (`vault_triples.rs:629-645`) | read the path from node data | -| Agents following the code-intelligence skill | read the line straight out of the copied ID | every result line prints `path:startLine-endLine` beside the ID | +```bash +# Show the project identity +atomic project identity show +# → project: ABCDEFGH...12345678 +# urn: urn:atomic:project:ABCDEFGH...12345678 +# created: 2026-01-15T09:00:00Z +``` -Display rule: anywhere an entity is shown to a person or an agent, print -`path:startLine-endLine` with the ID. That line is always current, because it is refreshed -on every enrichment, whereas a line embedded in an ID is frozen at the moment the ID was -copied. +**Phase 1 IRI usage:** -Lookup by location: entity arguments in the CLI also accept `path:line` (for example -`atomic query neighbors src/lib.rs:55`) and the shorthand `path#kind:scope::name`. Both -resolve to the full IRI within the current project, so nobody has to type the URN. +The projectors (Phase 2) will emit IRIs like: -Examples: +``` +urn:atomic:project::intent: +urn:atomic:project::goal: +urn:atomic:project::memory: +urn:atomic:project::session: +urn:atomic:project::view: +urn:atomic:project::file: +``` -| Source | Today | Proposed fragment | -|---|---|---| -| `pub fn greet()` in `src/lib.rs` | `entity:src/lib.rs:greet:12` | `#function:greet` | -| `impl User { fn new() }` | `entity:src/lib.rs:new:40` | `#method:User::new` | -| `impl Display for User { fn fmt }` | `entity:src/lib.rs:fmt:55` | `#method:::fmt` | -| `mod tests { fn it_works() }` | `entity:src/lib.rs:it_works:90` | `#function:tests::it_works` | -| Python `class Repo: def get(self)` | `entity:repo.py:get:8` | `#method:Repo::get` | -| Java `void put(int)` / `void put(String)` | two line-based IDs | `#method:Cache::put(int)` / `#method:Cache::put(String)` | -| `#[cfg(unix)] fn open` / `#[cfg(windows)] fn open` | two line-based IDs | `#function:open` / `#function:open~2` | +In Phase 1 we only mint the `ProjectId` and provide the IRI functions; no +projector actually emits project-qualified IRIs yet. This is deliberate: +projectors come in Phase 2, but the identity must exist first so that Phase 2 +can use it without a second migration. + +### 1.4 SHACL validation engine (`atomic_core::kg::shacl`) + +**Decision:** SHACL is the primary validation mechanism. We do not write hand-coded +Rust checks that mirror SHACL rules. We parse SHACL shapes and evaluate them against +the property-graph KG. + +**Dependencies:** `oxttl` (~20 crates) + `oxrdf` for parsing Turtle / RDF. These are +acceptable per the 2026-07-06 decision in `docs/shacl-oxirs-spike-findings.md`. +`oxirs` (~432 crates, silently drops `sh:node`/`sh:sparql`) is still rejected. -How it behaves compared with line-based IDs: +**What the engine supports (Phase 1):** -| Edit | Line-based ID | Scope-qualified ID | +| SHACL feature | Status | Rationale | |---|---|---| -| Add or remove lines above, edit the body, reformat | changes | **stable** | -| Rename the function | changes | changes (a real semantic change; the semantic diff's `ChangeType::Renamed` can later link old → new with `atomic:renamedFrom`) | -| Move it to another file, or into/out of a type | changes | changes | -| Change parameter types (overloading languages only) | stable unless lines move | changes | +| `sh:NodeShape`, `sh:PropertyShape` | ✓ | Core shape types | +| `sh:targetClass`, `sh:targetSubjectsOf`, `sh:targetObjectsOf` | ✓ | Target declaration | +| `sh:class`, `sh:datatype`, `sh:minCount`, `sh:maxCount` | ✓ | Cardinality and typing | +| `sh:minExclusive`, `sh:minInclusive`, `sh:maxExclusive`, `sh:maxInclusive` | ✓ | Range checks | +| `sh:minLength`, `sh:maxLength`, `sh:pattern` | ✓ | String constraints | +| `sh:hasValue`, `sh:in` | ✓ | Enumerated values | +| `sh:node` | ✗ | Defer; complex nesting | +| `sh:or`, `sh:and`, `sh:not` | ✓ | Boolean logic | +| `sh:sparql` | ✗ | Requires SPARQL; defer to federation phase | +| `sh:property` paths (`sh:path`) | ✓ | Predicate, inverse, sequence (limited) | + +**Builtin shape graphs:** + +- `atomic-core/ontology/shapes/projection-1.0.0.ttl` — projection profile: + - every generated change has `atomic:implements`, `atomic:explainedBy`, and + `atomic:attestedBy` shortcuts; + - shortcut targets equal the authoritative intermediate path targets; + - all referenced canonical nodes exist; + - full hashes (not abbreviated labels) in durable IDs. + +- `atomic-core/ontology/shapes/publication-1.0.0.ttl` — publication profile: + - every managed `SessionTurn` has exactly one `atomic:hasPlan` intent; + - no managed turn is governed by more than one intent; + - every turn has exactly one provenance graph; + - every generated change is covered by the session attestation; + - every attestation has a signer DID and valid proof. + +**Enterprise custom shapes:** + +Repository directory `.atomic/shapes/*.ttl` is loaded automatically on `atomic query +validate` and on every `vault sync`. Failed shapes are reported in the CLI with +`sh:focusNode`, `sh:resultPath`, and `sh:resultMessage`. + +Example enterprise shape: + +```turtle +@prefix atomic: . +@prefix sh: . + +<#SoxChangeShape> + a sh:NodeShape ; + sh:targetClass atomic:Change ; + sh:property [ + sh:path atomic:attestedBy ; + sh:minCount 1 ; + sh:message "SOX: every change must be attested." + ] ; + sh:property [ + sh:path atomic:implements ; + sh:minCount 1 ; + sh:message "SOX: every change must trace to an intent." + ] . +``` + +**API surface:** + +```rust +// atomic-core/src/kg/shacl.rs (new module) +pub struct ShaclValidator { ... } + +impl ShaclValidator { + /// Load a shape graph from a .ttl file. + pub fn load_shapes(&mut self, ttl: &str) -> Result<(), ShaclError>; + + /// Validate the KG (property-graph read interface) against loaded shapes. + pub fn validate(&self, kg: &dyn KgRead) -> ValidationReport; + + /// Validate a single focus node. + pub fn validate_node(&self, kg: &dyn KgRead, node: &NodeId, shape: &ShapeId) + -> Result<(), Vec>; +} + +pub struct ValidationReport { + pub conforms: bool, + pub results: Vec, +} + +pub struct ConstraintViolation { + pub focus_node: String, // IRI + pub result_path: Option, + pub severity: Severity, // sh:Violation, sh:Warning, sh:Info + pub message: String, + pub shape: String, +} +``` -The payoff is that `modifiesEntity` edges from every change that ever touched a function -converge on one node, so "history of this function" becomes a direct query, and full -rebuild and incremental enrichment produce identical IDs. +The validator operates over a small **graph-access trait**, not redb directly: -Cost: each of the seven parsers in `atomic-semantic/src/parsers/` (rust, typescript, -python, go, java, cpp, swift) must record its enclosing scope chain (a new -`Entity.scope: Vec` field) and, for overloading languages, normalized parameter -types. `CALLS` resolution stays name-based as today. +```rust +pub trait KgRead { + fn subjects(&self) -> Box>; + fn types(&self, node: &str) -> Vec<&str>; + fn objects(&self, subject: &str, predicate: &str) -> Vec<&str>; + fn has_property(&self, subject: &str, predicate: &str, object: &str) -> bool; + fn literal(&self, subject: &str, predicate: &str) -> Vec; +} +``` + +This trait is implemented for the property-graph tables (`kg_nodes`, `kg_edges`) +and also for an in-memory `oxrdf::Dataset` during tests. + +**CLI:** + +```bash +# Validate against all loaded shapes (builtin + .atomic/shapes/) +atomic query validate + +# Validate against a specific shape +atomic query validate --shape + +# Validate a specific node +atomic query validate --node + +# Output validation report as JSON or human-readable +atomic query validate --format json|text +``` + +### 1.5 Legacy term coverage + +`LEGACY_TERMS` (test-only, gated by `#[cfg(test)]`): -| Option | Cons | +```rust +pub enum Replacement { + One { class: Class }, + OnePred { predicate: Predicate }, + SplitByEndpoint(&[(Option, Option, Predicate)]), + Removed { reason: &'static str }, +} + +static LEGACY_TERMS: &[(/* old */ &'static str, Replacement)] = &[ ... ]; +``` + +Tests: + +1. Every legacy `predicate::*`, `entity_type::*`, and `edge_kind::*` constant appears + exactly once in `LEGACY_TERMS`. +2. Every `One`/`Split` target exists in the TTL. +3. TTL ↔ Rust consistency: parsed with `oxttl` + `oxrdf` (**dev-dependencies only**). + Every class/property in the TTL has a corresponding Rust enum variant, and vice + versa, with matching domain/range. +4. No emitted string starts with `vault:`, `atom:`, or is SCREAMING_CASE. + +## 2. Out of scope for Phase 1 + +| Item | Deferred to | |---|---| -| (a) Keep line-based IDs | Breaks rebuild determinism and history queries, as described | -| **(b) Scope-qualified IDs, as above** (chosen) | Extractor work in all seven parsers; the ordinal can shift if a new same-named duplicate appears earlier in the file (rare) | -| (c) CRDT `LeafId` / syntax-index stable IDs | Survives renames too, but depends on the semantic layer being finished (AGENTS.md: "in progress"), which would put Phase 1 behind it | +| `KgEdge.graph` (`Asserted`/`Inferred`) | Phase 2 | +| `kg_types` / `kg_types_rev` tables | Phase 2 | +| Storage generation 2 / `KG_GENERATION` | Phase 2 | +| Projector rewrite (`vault.rs`, `vcs.rs`, `session.rs`, `attestation.rs`, `findings.rs`) | Phase 2 | +| `reconcile()` pipeline | Phase 2 | +| OWL 2 RL inference engine (`kg::reason`) | Phase 2 | +| `sh:sparql` and advanced path expressions | Federation phase | +| Consumer cutover (`query`, triage, daemon, proto) | Phase 2 | +| Code-structure edges (`calls`, `defines`, `exports`, `includes`, `modifiesEntity`) | When sourced from AST | +| Scope-qualified entity IDs | Same — when code-structure edges are needed | +| `sh:node` (shape nesting) | Phase 2 | +| Project identity rotation / fork command | Phase 2 | +| DID binding on `ProjectIdentity.issuer` | Phase 2 | + +## 3. Acceptance criteria + +1. `workflow-1.0.0.ttl` is parseable by `oxttl` and contains no axioms outside the + supported RL subset. +2. `atomic_core::kg::ontology::Class::iri()` returns the correct absolute IRI for + every variant. +3. `atomic_core::kg::ontology::Predicate::domain()` and `::range()` match the TTL. +4. `LEGACY_TERMS` coverage test passes: all old constants mapped, no unmapped survivors. +5. Workspace lint: no legacy `vault:*`, `atom:*`, or SCREAMING_CASE kind strings + outside `LEGACY_TERMS`. +6. **Project identity:** `atomic init` creates a `ProjectId`; `atomic project identity show` + displays it; the ID round-trips through base32 encoding; IRIs minted via `kg::iri` + match the canonical format. +7. **SHACL engine:** `projection-1.0.0.ttl` and `publication-1.0.0.ttl` parse and + evaluate against a test property graph without error. +8. **Custom shapes:** placing a `.ttl` in `.atomic/shapes/` causes `atomic query validate` + to load and evaluate it. +9. **Conformance report:** `ValidationReport` produces `sh:focusNode`, `sh:resultPath`, + `sh:resultMessage` for every violation. + +## 4. Milestones + +### M1 — Ontology + registry (one PR) + +- `workflow-1.0.0.ttl` +- `atomic_core::kg::ontology` module +- `LEGACY_TERMS` + coverage tests +- TTL ↔ Rust consistency test (`oxttl`/`oxrdf` as dev-deps) + +### M2 — Project identity + IRI minting (one PR, depends on M1) + +- `atomic_core::kg::project_id` (`ProjectId`, base32, URN) +- `atomic_core::kg::iri` (mint/parse for every node kind) +- `PROJECT_IDENTITY` pristine table +- Identity object (JSON-LD, content-addressed, optionally signed) +- `atomic init` mints on creation; upgrading repos mint on first writable open +- Minimal clone/pull/push transport (repository-level sidecar) +- CLI: `atomic project identity show` +- Property tests: round-trip, percent-encoding, strict base32 parsing + +### M3 — SHACL engine + builtin shapes (one PR, depends on M1) + +- `oxttl` + `oxrdf` dependency +- `atomic_core::kg::shacl` module with `ShaclValidator`, `KgRead` trait, + `ValidationReport`, `ConstraintViolation` +- `projection-1.0.0.ttl` and `publication-1.0.0.ttl` +- In-memory `oxrdf::Dataset` adapter for `KgRead` (testing) +- CLI: `atomic query validate` +- Custom shape loader from `.atomic/shapes/*.ttl` +- Test: a synthetic property graph passes/fails each builtin shape as expected + +## 5. Development notes + +### Why not oxirs? + +`oxirs` (~432 transitive deps) silently drops `sh:node` and `sh:sparql`. We write +the subset we need (~800–1200 lines) against `oxrdf`/`oxttl`, grow it only when a +shape requires it, and keep the graph-access trait small enough to extract later +into a standalone crate if the engine becomes general-purpose. + +### Why `KgRead` trait instead of redb directly? + +The trait abstracts over property-graph tables, `oxrdf::Dataset`, and any future +RDF store. Validation tests run against an in-memory dataset without a database. +If the engine ever needs to move into a standalone crate (e.g. for the Circuit +Breaker workflow), the trait goes with it and redb does not. + +### Enterprise workflow shapes + +The `.atomic/shapes/` convention is intentionally simple: drop a `.ttl` file, run +`atomic query validate`. No registry, no compilation step. A CI pipeline can run +`atomic query validate --format json` and fail on any `sh:Violation` severity. + +### Why view-independent storage for project identity? + +If project identity lived in a `.vault/` file (a tracked change), it would: +- be invisible on views that don't include that change; +- be lost on `unrecord`; +- diverge across views in a merge. + +Cross-project SPARQL assumes every repository in the federation has one stable +`urn:atomic:project:` that never changes. A repository-level pristine table +outside the view system is the only place that guarantees this. + +### Phase 2 expectations + +Phase 2 will: +- rewrite projectors to emit project-qualified IRIs using `kg::iri`; +- implement `kg_types`/`kg_types_rev` for type indexing; +- add the OWL 2 RL inference engine; +- wire SHACL validation into `vault sync` and publication gates; +- cut over consumers to use canonical IRIs. + +Because Phase 1 already minted `ProjectId`, Phase 2 projectors can use it +immediately without a second migration or ID rewrite. -Work lands in M4 (extractor scope chains, ID minting, `callers` display, search ranking, -`path:line` and shorthand lookup). +--- -**D11 — `Execution` nodes: option (a).** Declare `atomic:Execution` and `atomic:partOfTurn` -in the ontology only. Project nothing until capture records structured executions -(`ProvActivityInput.used` is always empty today). +## Appendix: Why we skip code-block mapping now -All Phase 1 decisions are resolved; there are no open questions. +The existing `vault_kg_enrich.rs` pipeline extracts `CALLS`, `DEFINES`, `EXPORTS`, +`INCLUDES`, `PART_OF`, and `MODIFIES` (entity) by walking source files and emitting +KG edges. This duplicates `atomic-semantic`'s syntax index, produces stale line-based +entity IDs, and generates millions of mostly-unqueried edges. The right time to +materialize code-structure edges in the KG is when we need them for federated +queries that the syntax index cannot answer. Until then, `atomic:File` stays as a +workflow artifact and Tree-Sitter handles everything inside it. From 84b05b741b3609cfce2991ee1fb7908650789d64 Mon Sep 17 00:00:00 2001 From: Lee Faus Date: Tue, 13 Oct 2026 17:32:45 -0400 Subject: [PATCH 03/19] update chains --- docs/knowledge-graph-closure-phase1-plan.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/docs/knowledge-graph-closure-phase1-plan.md b/docs/knowledge-graph-closure-phase1-plan.md index db0ebf9c..0d134b03 100644 --- a/docs/knowledge-graph-closure-phase1-plan.md +++ b/docs/knowledge-graph-closure-phase1-plan.md @@ -134,10 +134,14 @@ but are **not** materialized in Phase 1. | `atomic:priority` | `atomic:WorkIntent` | `xsd:string` | | `atomic:findingCode` | `atomic:Finding` | `xsd:string` | -**Property chains** (declared in TTL, materialized in Phase 2): +**Property chains** (declared in TTL as `owl:propertyChainAxiom` for Phase 2 inference): -- `atomic:generatedChange ∘ atomic:hasPlan` → `atomic:implements` -- `atomic:generatedChange ∘ atomic:explainedBy` → `atomic:explainedBy` (on Change) +| Chain | Meaning | +|---|---| +| `atomic:generatedBy ∘ atomic:hasPlan` → `atomic:implements` | A change was generated by a turn that had a plan → the change implements that intent. | +| `atomic:generatedBy ∘ atomic:explainedBy` → `atomic:explainedBy` (on Change) | A change was generated by a turn explained by a provenance graph → the change is explained by that graph. | + +These chains use `atomic:generatedBy` (the inverse of `atomic:generatedChange`), so the intermediate node is consistently `SessionTurn`. In Phase 1 the chains are **declarative only** — the ontology records the intended meaning so that enterprise SHACL shapes and documentation see a complete model. The OWL 2 RL reasoner in Phase 2 will materialize them from the TTL. **What the ontology does NOT contain:** From 92933c614a7494c988faa5518b61c0a5ac12ba71 Mon Sep 17 00:00:00 2001 From: Lee Faus Date: Tue, 13 Oct 2026 17:56:22 -0400 Subject: [PATCH 04/19] doc updates and SHACL tests --- docs/knowledge-graph-closure-phase1-plan.md | 222 +++++++++++--------- docs/shacl-purrdf-spike-findings.md | 87 ++++++++ 2 files changed, 207 insertions(+), 102 deletions(-) create mode 100644 docs/shacl-purrdf-spike-findings.md diff --git a/docs/knowledge-graph-closure-phase1-plan.md b/docs/knowledge-graph-closure-phase1-plan.md index 0d134b03..748b34dc 100644 --- a/docs/knowledge-graph-closure-phase1-plan.md +++ b/docs/knowledge-graph-closure-phase1-plan.md @@ -12,10 +12,10 @@ Ship **three artifacts** that every downstream phase depends on: 2. **`ProjectId`** — a stable, unique project identity minted at `atomic init` and cast into IRIs so that future cross-project SPARQL queries have a canonical namespace per repository. -3. **Native SHACL engine** — parse and evaluate SHACL shapes against the property-graph - KG. No hand-coded Rust validators. Enterprise users add custom shapes in - `.atomic/shapes/*.ttl`; the engine evaluates them the same way it evaluates builtin - shapes. +3. **SHACL integration using PurRDF** — adapt workflow data to RDF and evaluate + builtin and enterprise-authored shapes with an existing engine, provisionally + `purrdf-shapes = 3.0.1`. No hand-rolled validator or constraint evaluator. + Enterprise users add custom shapes in `.atomic/shapes/*.ttl`. Everything else — projectors, storage cutover, inference, consumers — comes later. This keeps Phase 1 reviewable while guaranteeing that IRIs are stable and validation @@ -295,30 +295,42 @@ projector actually emits project-qualified IRIs yet. This is deliberate: projectors come in Phase 2, but the identity must exist first so that Phase 2 can use it without a second migration. -### 1.4 SHACL validation engine (`atomic_core::kg::shacl`) +### 1.4 SHACL integration (PurRDF, existing engine) -**Decision:** SHACL is the primary validation mechanism. We do not write hand-coded -Rust checks that mirror SHACL rules. We parse SHACL shapes and evaluate them against -the property-graph KG. +**Decision:** SHACL shapes are the primary workflow-policy definitions. Use an +existing SHACL engine, not hand-coded checks or an Atomic-owned subset evaluator. +The provisional choice is **`purrdf-shapes = "=3.0.1"`** from +[PurRDF](https://github.com/Blackcat-Informatics/purrdf). -**Dependencies:** `oxttl` (~20 crates) + `oxrdf` for parsing Turtle / RDF. These are -acceptable per the 2026-07-06 decision in `docs/shacl-oxirs-spike-findings.md`. -`oxirs` (~432 crates, silently drops `sh:node`/`sh:sparql`) is still rejected. +The [initial spike](./shacl-purrdf-spike-findings.md) compiled the published crate +without source changes and passed eleven tests. The reproducible standalone package +is [`tools/shacl-purrdf-spike`](../tools/shacl-purrdf-spike/README.md), outside the +Atomic workspace. This is evidence for adoption, not proof of complete conformance. -**What the engine supports (Phase 1):** +**Observed in the spike:** -| SHACL feature | Status | Rationale | -|---|---|---| -| `sh:NodeShape`, `sh:PropertyShape` | ✓ | Core shape types | -| `sh:targetClass`, `sh:targetSubjectsOf`, `sh:targetObjectsOf` | ✓ | Target declaration | -| `sh:class`, `sh:datatype`, `sh:minCount`, `sh:maxCount` | ✓ | Cardinality and typing | -| `sh:minExclusive`, `sh:minInclusive`, `sh:maxExclusive`, `sh:maxInclusive` | ✓ | Range checks | -| `sh:minLength`, `sh:maxLength`, `sh:pattern` | ✓ | String constraints | -| `sh:hasValue`, `sh:in` | ✓ | Enumerated values | -| `sh:node` | ✗ | Defer; complex nesting | -| `sh:or`, `sh:and`, `sh:not` | ✓ | Boolean logic | -| `sh:sparql` | ✗ | Requires SPARQL; defer to federation phase | -| `sh:property` paths (`sh:path`) | ✓ | Predicate, inverse, sequence (limited) | +| SHACL behavior | Result | +|---|---| +| Target class + required/exactly-one property | Positive and negative fixtures pass | +| Nested `sh:node` with enum constraints | Enforced; no deferral to Phase 2 | +| Local `sh:sparql` conditional requirements | Enforced; independent of federation | +| Inverse and sequence paths | Enforced; complex report path retained | +| Class, datatype, and `sh:or` | Positive and negative fixtures pass | +| Report focus/path/shape/component/severity/message | Present; separate paths do not collapse | +| Malformed cardinality parameter / malformed SPARQL query | Shape loading returns an error | +| Unsupported SHACL-JS | Returns an error, even with an empty data graph | +| Valid shape with no applicable focus nodes | Conforms vacuously; target coverage must be reported separately | + +PurRDF documents broader Core/SPARQL support; additional constructs must be covered +by profile regression tests before we depend on them. `oxttl` and `oxrdf` are +parser/data-model utilities, **not** the validation engine. + +**Toolchain/deployment gate:** PurRDF 3.0.1 requires **Rust 1.98**, while Atomic +currently declares **1.90**. The spike uses a separate toolchain and resolves 73 +dependency packages (including target-specific packages). M3 must explicitly choose +an approved MSRV upgrade for embedded use or a separately built PurRDF runner. +Do not silently raise Atomic's MSRV. Either deployment runs the same SHACL shapes; +there is no hand-rolled fallback. **Builtin shape graphs:** @@ -334,15 +346,27 @@ acceptable per the 2026-07-06 decision in `docs/shacl-oxirs-spike-findings.md`. - no managed turn is governed by more than one intent; - every turn has exactly one provenance graph; - every generated change is covered by the session attestation; - - every attestation has a signer DID and valid proof. + - every attestation has the required signer identity and an explicit successful + verification result supplied by a separate cryptographic verifier. + +SHACL cannot establish signature validity by itself. These shapes define the +intended profiles; fixture tests in Phase 1 prove their behavior, while complete +live projection and mandatory publication enforcement land in Phase 2. Running +shapes without applicable focus nodes must not be presented as complete audit evidence. **Enterprise custom shapes:** -Repository directory `.atomic/shapes/*.ttl` is loaded automatically on `atomic query -validate` and on every `vault sync`. Failed shapes are reported in the CLI with -`sh:focusNode`, `sh:resultPath`, and `sh:resultMessage`. +Repository directory `.atomic/shapes/*.ttl` is loaded on `atomic query validate`. +Automatic `vault sync` validation and publication hooks land in Phase 2. Builtin +and enterprise shapes use the same engine; policy changes do not require Rust code. +Use explicit validation profiles so teams can select applicable workflow rules. -Example enterprise shape: +Report `sh:focusNode`, `sh:resultPath`, `sh:sourceShape`, +`sh:sourceConstraintComponent`, `sh:resultSeverity`, and `sh:resultMessage`, retaining +complex path structure and multiple findings. Shape parse/load/execution errors +are distinct from violations and must never yield a successful validation result. + +Example enterprise traceability shape (illustrative, not a complete SOX policy): ```turtle @prefix atomic: . @@ -363,52 +387,26 @@ Example enterprise shape: ] . ``` -**API surface:** - -```rust -// atomic-core/src/kg/shacl.rs (new module) -pub struct ShaclValidator { ... } - -impl ShaclValidator { - /// Load a shape graph from a .ttl file. - pub fn load_shapes(&mut self, ttl: &str) -> Result<(), ShaclError>; - - /// Validate the KG (property-graph read interface) against loaded shapes. - pub fn validate(&self, kg: &dyn KgRead) -> ValidationReport; - - /// Validate a single focus node. - pub fn validate_node(&self, kg: &dyn KgRead, node: &NodeId, shape: &ShapeId) - -> Result<(), Vec>; -} - -pub struct ValidationReport { - pub conforms: bool, - pub results: Vec, -} - -pub struct ConstraintViolation { - pub focus_node: String, // IRI - pub result_path: Option, - pub severity: Severity, // sh:Violation, sh:Warning, sh:Info - pub message: String, - pub shape: String, -} -``` - -The validator operates over a small **graph-access trait**, not redb directly: - -```rust -pub trait KgRead { - fn subjects(&self) -> Box>; - fn types(&self, node: &str) -> Vec<&str>; - fn objects(&self, subject: &str, predicate: &str) -> Vec<&str>; - fn has_property(&self, subject: &str, predicate: &str, object: &str) -> bool; - fn literal(&self, subject: &str, predicate: &str) -> Vec; -} -``` - -This trait is implemented for the property-graph tables (`kg_nodes`, `kg_edges`) -and also for an in-memory `oxrdf::Dataset` during tests. +**Integration boundary — adaptation, not evaluation:** + +1. Adapt a read-only workflow snapshot into RDF with canonical predicates/types, + full resource IRIs, typed literals, and properly scoped blank nodes. Existing + legacy workflow terms are translated at this boundary; AST relationships are + excluded. This does not require a storage cutover or rewriting signed objects. +2. Load builtin and selected enterprise shape bundles with explicit base IRIs and + controlled imports. Do not fetch arbitrary remote shape imports implicitly. +3. Call PurRDF's existing API. The spike uses + `purrdf_shapes::engine::validate_graphs(data_nt, shapes_ttl, shapes_base)`; + assess its prepared-shape/dataset APIs for production snapshot reuse. +4. Preserve its standard validation report; adapt presentation to CLI JSON/text + without losing source shapes, paths, messages, or severity. + +Atomic code owns snapshot adaptation, profile selection, invocation, and report +presentation only. PurRDF owns shape parsing and constraint execution. Missing +projection facts remain missing—no guessed links or bespoke policy evaluator. +Record snapshot/profile identity and target coverage; an empty target set does not +prove workflow completeness. The adapter's fidelity needs integration tests beyond +the synthetic spike. No OWL inference is performed by this boundary in Phase 1. **CLI:** @@ -461,11 +459,10 @@ Tests: | Projector rewrite (`vault.rs`, `vcs.rs`, `session.rs`, `attestation.rs`, `findings.rs`) | Phase 2 | | `reconcile()` pipeline | Phase 2 | | OWL 2 RL inference engine (`kg::reason`) | Phase 2 | -| `sh:sparql` and advanced path expressions | Federation phase | +| Cross-project SPARQL endpoints / federation catalogs | Later federation phase | | Consumer cutover (`query`, triage, daemon, proto) | Phase 2 | | Code-structure edges (`calls`, `defines`, `exports`, `includes`, `modifiesEntity`) | When sourced from AST | | Scope-qualified entity IDs | Same — when code-structure edges are needed | -| `sh:node` (shape nesting) | Phase 2 | | Project identity rotation / fork command | Phase 2 | | DID binding on `ProjectIdentity.issuer` | Phase 2 | @@ -482,12 +479,20 @@ Tests: 6. **Project identity:** `atomic init` creates a `ProjectId`; `atomic project identity show` displays it; the ID round-trips through base32 encoding; IRIs minted via `kg::iri` match the canonical format. -7. **SHACL engine:** `projection-1.0.0.ttl` and `publication-1.0.0.ttl` parse and - evaluate against a test property graph without error. +7. **SHACL integration:** PurRDF evaluates positive and negative fixtures for both + builtin profiles, nested shapes, local SPARQL conditions, and property paths. + No Atomic-owned constraint evaluator. The MSRV/deployment decision is explicit. 8. **Custom shapes:** placing a `.ttl` in `.atomic/shapes/` causes `atomic query validate` - to load and evaluate it. -9. **Conformance report:** `ValidationReport` produces `sh:focusNode`, `sh:resultPath`, - `sh:resultMessage` for every violation. + to load and evaluate it using the same engine as builtin profiles. +9. **Conformance report:** preserve standard focus/path/source/component/severity/ + message fields and complex path structures. Parse/runtime failures are errors, + never conformance. Report the selected profile and target coverage separately. +10. **Adapter fidelity:** real workflow snapshots preserve resource identity, types, + typed literals, and missing facts. Empty or partial projections must not be + described as complete audit evidence. +11. **Regression corpus:** retain the isolated spike, add the existing gate fixtures + and relevant W3C cases, and compare against an independent SHACL implementation + before enabling mandatory production gates. ## 4. Milestones @@ -509,32 +514,45 @@ Tests: - CLI: `atomic project identity show` - Property tests: round-trip, percent-encoding, strict base32 parsing -### M3 — SHACL engine + builtin shapes (one PR, depends on M1) - -- `oxttl` + `oxrdf` dependency -- `atomic_core::kg::shacl` module with `ShaclValidator`, `KgRead` trait, - `ValidationReport`, `ConstraintViolation` -- `projection-1.0.0.ttl` and `publication-1.0.0.ttl` -- In-memory `oxrdf::Dataset` adapter for `KgRead` (testing) -- CLI: `atomic query validate` -- Custom shape loader from `.atomic/shapes/*.ttl` -- Test: a synthetic property graph passes/fails each builtin shape as expected +### M3 — PurRDF SHACL integration + builtin shapes (depends on M1/M2) + +- Retain the standalone `purrdf-shapes = 3.0.1` spike and lockfile. +- Resolve Rust 1.98 versus Atomic's 1.90 MSRV: approved embedded upgrade or separate + validation runner. Select deployment before adding workspace dependencies. +- Implement the read-only workflow-to-RDF adapter and a thin PurRDF invocation layer; + no custom SHACL evaluator, policy-rule compilation, or hand-coded mirror checks. +- `projection-1.0.0.ttl` and `publication-1.0.0.ttl`, including local SPARQL rules + wherever workflow consistency requires them. +- CLI: `atomic query validate`, profile selection, standard reports and error exits. +- Custom shape loader from `.atomic/shapes/*.ttl`, controlled imports, no-target + diagnostics, and snapshot/profile identification. +- Positive/negative profile fixtures, adapter integration tests, existing gate + regressions, and independent-validator comparisons before production enforcement. ## 5. Development notes -### Why not oxirs? +### Engine selection and historical decision + +The [OxiRS spike](./shacl-oxirs-spike-findings.md) tested **0.3.1**, with nested/ +SPARQL constraints silently unenforced, lost result paths, and a large dependency +footprint. Do not generalize those findings to every future OxiRS release. + +The [PurRDF 3.0.1 spike](./shacl-purrdf-spike-findings.md) passed the initial probes +for those failure categories. It is the provisional Phase 1 engine choice, subject +to the MSRV decision and broader regression coverage. A version number alone is +not a conformance or maturity guarantee. -`oxirs` (~432 transitive deps) silently drops `sh:node` and `sh:sparql`. We write -the subset we need (~800–1200 lines) against `oxrdf`/`oxttl`, grow it only when a -shape requires it, and keep the graph-access trait small enough to extract later -into a standalone crate if the engine becomes general-purpose. +This plan replaces the historical direction of hand-coded workflow gates / an +in-house SHACL subset with **SHACL shapes executed by an existing engine**. Existing +cryptographic verification and identity/authorization checks remain separate; +SHACL evaluates their recorded results, not signatures themselves. -### Why `KgRead` trait instead of redb directly? +### Why an RDF snapshot adapter instead of redb inside the engine? -The trait abstracts over property-graph tables, `oxrdf::Dataset`, and any future -RDF store. Validation tests run against an in-memory dataset without a database. -If the engine ever needs to move into a standalone crate (e.g. for the Circuit -Breaker workflow), the trait goes with it and redb does not. +Keep the database and authoritative objects in Atomic. Adapt workflow facts to +PurRDF's RDF input and retain its reports. This allows either embedded validation +or an isolated runner without maintaining a second authoritative store. The engine +integration must not pull source-code relationship extraction into Phase 1. ### Enterprise workflow shapes diff --git a/docs/shacl-purrdf-spike-findings.md b/docs/shacl-purrdf-spike-findings.md new file mode 100644 index 00000000..4bcd759d --- /dev/null +++ b/docs/shacl-purrdf-spike-findings.md @@ -0,0 +1,87 @@ +# PurRDF SHACL engine spike — findings (`purrdf-shapes` 3.0.1) + +> Status: **initial spike passed; provisional Phase 1 engine choice.** +> Reproducer: [`tools/shacl-purrdf-spike`](../tools/shacl-purrdf-spike/README.md). +> No engine has been added to Atomic's workspace or publication path. + +## Question + +Can an existing Rust SHACL engine enforce enterprise-authored Turtle shapes, +including nested constraints and local SPARQL conditions, without an Atomic-owned +constraint evaluator? + +## Result + +**Yes for the tested cases.** The published crate compiled without modification on +Rust 1.98.0 (aarch64 macOS). All eleven tests passed, with positive and negative +fixtures for the operational constraints. Initial debug build took about 26 seconds +(after dependency downloads); the final test runner reported about 0.04 seconds for +the eleven tests. These are smoke-test timings, not performance benchmarks. + +| Probe | Observation | +|---|---| +| `sh:minCount` / `sh:maxCount` | Missing plan fails; exactly one passes; multiple plans fail | +| Nested `sh:node` + `sh:in` | Missing/invalid criterion status fails; valid status passes | +| `sh:sparql` conditional requirement | Open criterion passes; met criterion fails until both verifier and evidence exist | +| Inverse + sequence `sh:path` | Change → generating turn → intent path enforced; complex path structure retained in report | +| `sh:class`, `sh:datatype`, `sh:or` | Wrong type/datatype and unmet alternatives fail; corresponding positive cases pass | +| Violation report | Focus, path, source shape, source component, severity, and message available; standard RDF report serializes | +| Multiple missing properties | Two separate violations retain two distinct property paths | +| Malformed `sh:minCount "one"` | Shape loading returns an error | +| Unsupported SHACL-JS | Shape loading returns an error, including validation against an empty data graph | +| Malformed SPARQL query | Shape loading and validation return an error, even against an empty data graph | +| Valid shape with no applicable focus nodes | Empty graph conforms with no results; confirms the need for separate target-coverage reporting | +| Project-scoped subjects | Same local turn ID in two project namespaces stays distinct; violation identifies the correct project | + +The nested constraint, SPARQL conditional, and result-path probes directly cover +categories that failed in the older OxiRS 0.3.1 spike. This is not a rerun of the +entire old differential corpus and makes no claim about newer OxiRS versions. + +## Dependency and toolchain cost + +- Direct dependency: `purrdf-shapes = "=3.0.1"`; the lockfile resolved its PurRDF + family to 3.0.1 as well. +- **73 dependency packages** in the resolved metadata, excluding the spike itself. + This includes target-specific dependencies; not all were compiled on macOS. +- Native Rust library API; this run needed neither a Python runtime nor a JVM. +- **Rust 1.98 required.** Atomic declares 1.90 and the default compiler on this + machine was 1.97.1. We installed a separate 1.98.0 toolchain for the spike only. +- Before embedding PurRDF, approve an Atomic MSRV upgrade or keep validation in a + separately built runner. Do not silently raise the main workspace requirement. + +## Integration decision + +Use PurRDF as the **provisional existing SHACL engine**, not as a model for writing +our own validator. Atomic owns RDF snapshot adaptation, shape bundle loading, +validation invocation, report presentation, and workflow-boundary integration. +PurRDF owns constraint parsing and evaluation, including SHACL-SPARQL. + +`oxttl` / `oxrdf` can still be useful parser/model utilities elsewhere; they are +not SHACL engines and are not substitutes for PurRDF. + +This changes the direction recorded in +[`shacl-oxirs-spike-findings.md`](./shacl-oxirs-spike-findings.md): SHACL is now the +primary workflow-policy mechanism under evaluation. The historical rejection of +OxiRS 0.3.1 remains a historical finding, not a ban on every embedded engine. + +## Still required before production gates + +1. Run relevant W3C fixtures and the full existing intent/memory gate regression + corpus; compare results with an independent established SHACL implementation. +2. Exercise the actual Atomic RDF adapter, custom shape bundles, and CLI, not only + synthetic N-Triples data. Preserve typed literals, paths, full IDs, and types. +3. Confirm all shapes in each supported profile load and evaluate, including + shortcut agreement, coverage, and recorded verifier-result checks. +4. Define profile scope and target coverage. Valid shapes with zero applicable + focus nodes can conform vacuously; that must not imply complete audit evidence. +5. Test malformed queries, unresolved imports, unsupported terms, and runtime + failures: errors must never be reported as conformance. The spike proves three + specific rejection cases, not universal fail-closed behavior. +6. Bound validation time/memory and shape import access. Test severity handling, + deterministic report comparison, and repeated validation. +7. Resolve the MSRV/deployment choice. If validation runs externally, unavailable + or failed validation must block any boundary configured to require it. + +SHACL validates **recorded cryptographic verification results**; signature and hash +verification remain separate operations. This spike contains no signature verifier, +OWL inference, cross-project SPARQL endpoint, redb adapter, or publication hook. From 671e3d2fd46d2ce2b7666c7b035572f78a49f0bf Mon Sep 17 00:00:00 2001 From: Lee Faus Date: Tue, 13 Oct 2026 17:56:46 -0400 Subject: [PATCH 05/19] spike for SHAQL --- tools/shacl-purrdf-spike/.gitignore | 1 + tools/shacl-purrdf-spike/Cargo.lock | 725 ++++++++++++++++++++++++++++ tools/shacl-purrdf-spike/Cargo.toml | 12 + tools/shacl-purrdf-spike/README.md | 46 ++ tools/shacl-purrdf-spike/src/lib.rs | 295 +++++++++++ 5 files changed, 1079 insertions(+) create mode 100644 tools/shacl-purrdf-spike/.gitignore create mode 100644 tools/shacl-purrdf-spike/Cargo.lock create mode 100644 tools/shacl-purrdf-spike/Cargo.toml create mode 100644 tools/shacl-purrdf-spike/README.md create mode 100644 tools/shacl-purrdf-spike/src/lib.rs diff --git a/tools/shacl-purrdf-spike/.gitignore b/tools/shacl-purrdf-spike/.gitignore new file mode 100644 index 00000000..b83d2226 --- /dev/null +++ b/tools/shacl-purrdf-spike/.gitignore @@ -0,0 +1 @@ +/target/ diff --git a/tools/shacl-purrdf-spike/Cargo.lock b/tools/shacl-purrdf-spike/Cargo.lock new file mode 100644 index 00000000..aaf75a1a --- /dev/null +++ b/tools/shacl-purrdf-spike/Cargo.lock @@ -0,0 +1,725 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "aead" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1973cfbc1a2daf9cf550e74e1f088c28e7f7d8c1e1418fb6c9dc5184b7e84c99" +dependencies = [ + "crypto-common", + "inout", +] + +[[package]] +name = "aes" +version = "0.9.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "35f0f96ce78e38c3dc6d8948aa8163d06385be74000f3c7a95bf1eef35d3ea32" +dependencies = [ + "cipher", + "cpubits", + "cpufeatures", +] + +[[package]] +name = "aes-gcm" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7f2b8006a0c83f52b62ba44a97b58bf76fe2f70a329e588f67f89691d93d498f" +dependencies = [ + "aead", + "aes", + "cipher", + "ctr", + "ctutils", + "ghash", +] + +[[package]] +name = "aho-corasick" +version = "1.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c982642fa9e8606056828ee9a8505737230110bb1099153c79efe865c59d12ba" +dependencies = [ + "memchr", +] + +[[package]] +name = "atomic-purrdf-shacl-spike" +version = "0.1.0" +dependencies = [ + "purrdf-shapes", +] + +[[package]] +name = "block-buffer" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d2f6c7dbe95a6ed67ad9f18e57daf93a2f034c524b99fd2b76d18fdfeb6660aa" +dependencies = [ + "hybrid-array", +] + +[[package]] +name = "bumpalo" +version = "3.20.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649" + +[[package]] +name = "cfg-if" +version = "1.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4e7648175b45a9a48536d676f68d918270699102aa8dab5496df06904c914600" + +[[package]] +name = "cipher" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e8cf2a2c93cd704877c0858356ed03480ff301ee950b43f1cbe4573b088bfa6c" +dependencies = [ + "block-buffer", + "crypto-common", + "inout", +] + +[[package]] +name = "cmov" +version = "0.5.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c9ea0ac24bc397ab3c98583a3c9ba74fa56b09a4449bbe172b9b1ddb016027a" + +[[package]] +name = "const-oid" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a6ef517f0926dd24a1582492c791b6a4818a4d94e789a334894aa15b0d12f55c" + +[[package]] +name = "cpubits" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "15b85f9c39137c3a891689859392b1bd49812121d0d61c9caf00d46ed5ce06ae" + +[[package]] +name = "cpufeatures" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5ca28b0ae3115b884660db4118d803791fd6756b6e88f39c0f3f7859060d7566" +dependencies = [ + "libc", +] + +[[package]] +name = "crossbeam-deque" +version = "0.8.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "622f3fc73690be383c7214310406f28a90e6edeadc3cea882f9d71e495b9711a" +dependencies = [ + "crossbeam-epoch", + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-epoch" +version = "0.9.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc74980687109a3b14c72fd458107bf0baa1da1a1a805e178d15501ba9b86d9d" +dependencies = [ + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-utils" +version = "0.8.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a31eee39dddec8330830986fcd7625edb5a24ec90ea038215273bbc3adb08ac6" + +[[package]] +name = "crypto-common" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ce6e4c961d6cd6c9a86db418387425e8bdeaf05b3c8bc1411e6dca4c252f1453" +dependencies = [ + "hybrid-array", +] + +[[package]] +name = "ctr" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "baaca1c4b237092596f64d571e9db6ce4109c4ef9742e27590f1709594461f21" +dependencies = [ + "cipher", +] + +[[package]] +name = "ctutils" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "03bb0e1cc970d482d121d9a1744999169b69a07470b3d644a7894e53fcaf4574" +dependencies = [ + "cmov", +] + +[[package]] +name = "digest" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2" +dependencies = [ + "block-buffer", + "const-oid", + "crypto-common", +] + +[[package]] +name = "either" +version = "1.19.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0e9c71c2167ca323c882b99918929403426e2373ea17242ff5653e0d5e1058be" + +[[package]] +name = "fastrand" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "da7c62ceae207dd37ea5b845da6a0696c799f85e97da1ab5b7910be3c1c80223" + +[[package]] +name = "getrandom" +version = "0.3.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd" +dependencies = [ + "cfg-if", + "libc", + "r-efi", + "wasip2", +] + +[[package]] +name = "ghash" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2eecf2d5dc9b66b732b97707a0210906b1d30523eb773193ab777c0c84b3e8d5" +dependencies = [ + "polyval", +] + +[[package]] +name = "hashbrown" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a" + +[[package]] +name = "hybrid-array" +version = "0.4.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "27f864f10dfb56725ce5ce5472bc52252c8f93a4ab86327122cebf62c5f59a17" +dependencies = [ + "typenum", +] + +[[package]] +name = "inout" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4250ce6452e92010fdf7268ccc5d14faa80bb12fc741938534c58f16804e03c7" +dependencies = [ + "hybrid-array", +] + +[[package]] +name = "libc" +version = "0.2.190" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ce5d3ddc6d3fa000eb1536d85e147bfe31aacaba692ed6a876f95cb7c855be78" + +[[package]] +name = "memchr" +version = "2.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98" + +[[package]] +name = "once_cell" +version = "1.21.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" + +[[package]] +name = "polyval" +version = "0.7.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0fa31d631f2b2cb2a544d0aa321ce847a94764d701ca2becc411138b93d49cd" +dependencies = [ + "cpubits", + "cpufeatures", + "universal-hash", +] + +[[package]] +name = "proc-macro2" +version = "1.0.107" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "purrdf-cdt" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5f3f6ce304b34bb666d5ffe441587563206cbc7702ab76708e26eb2398f531af" +dependencies = [ + "purrdf-events", + "purrdf-hash", + "purrdf-iri", + "purrdf-lex", + "purrdf-xsd", +] + +[[package]] +name = "purrdf-core" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1d073303e9c754f0244391992392aee7a71129a99817b6b608fc0d24a0d80e8d" +dependencies = [ + "hashbrown", + "purrdf-cdt", + "purrdf-deflate", + "purrdf-events", + "purrdf-hash", + "purrdf-iri", + "purrdf-lex", + "purrdf-xsd", + "regex", + "regex-syntax", + "sha2", +] + +[[package]] +name = "purrdf-datalog" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "438d3d607a0e55794e0b016af3cba4332b954ddf2ec4c04cb864393e022934aa" +dependencies = [ + "hashbrown", + "purrdf-core", + "purrdf-hash", + "purrdf-lex", + "rayon", +] + +[[package]] +name = "purrdf-deflate" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f1322db69953bf995fa856f361e463ebeed7a052d8853dd2d77c59c4001e979" +dependencies = [ + "purrdf-hash", +] + +[[package]] +name = "purrdf-ed25519" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ac199f2e42bd2f48caba9a6573b1161f067d9fbe06c376c28cfc93ed69892912" +dependencies = [ + "purrdf-hash", + "sha2", +] + +[[package]] +name = "purrdf-events" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b5455431280c55a65bbf44f41f5f2d901a0c5923052cce94d392abead530e40c" + +[[package]] +name = "purrdf-gts" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "76a60ea5fc7ca62449fa7a6ea80bd0a8c14983d15d9584d4fa42140a3e1a9e4f" +dependencies = [ + "aes-gcm", + "fastrand", + "purrdf-deflate", + "purrdf-ed25519", + "purrdf-events", + "purrdf-hash", + "purrdf-iri", + "purrdf-lex", + "purrdf-xsd", + "rayon", + "structured-zstd", + "visual-hashing", +] + +[[package]] +name = "purrdf-hash" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bd021886ad0790918ec3501c47f952f98cc60474340fe4dad8df1f4b77471a91" + +[[package]] +name = "purrdf-iri" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "682988bf556aa2d09dbcb30bea74dc07fa356648b7217c0fd95330a168694aaa" +dependencies = [ + "purrdf-hash", + "purrdf-lex", +] + +[[package]] +name = "purrdf-lex" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c1443d0cd967b66b82188958bc92b63dc9673e253f86f559eb93de0319a73486" +dependencies = [ + "purrdf-hash", +] + +[[package]] +name = "purrdf-rdf" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "967240728285800972d323c7bd67dd5facd937572a3d17d65716242e39bb2a10" +dependencies = [ + "hashbrown", + "purrdf-core", + "purrdf-ed25519", + "purrdf-events", + "purrdf-gts", + "purrdf-hash", + "purrdf-iri", + "purrdf-lex", + "purrdf-sparql-algebra", + "purrdf-sparql-eval", + "purrdf-xsd", + "rayon", + "regex", + "sha2", +] + +[[package]] +name = "purrdf-shapes" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a4e6d0720b974870f2b29256e5c5bb5e62c4d40308afb24ee66556525a2bbeb5" +dependencies = [ + "purrdf-core", + "purrdf-datalog", + "purrdf-deflate", + "purrdf-hash", + "purrdf-iri", + "purrdf-lex", + "purrdf-rdf", + "purrdf-slice", + "purrdf-sparql-algebra", + "purrdf-sparql-eval", + "purrdf-xsd", + "rayon", + "regex", +] + +[[package]] +name = "purrdf-slice" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "edbe19c6825da25e00e675f8d8049f503f2a76624c2f41f1add7f55756ea02c0" +dependencies = [ + "purrdf-core", + "purrdf-gts", + "purrdf-hash", + "purrdf-iri", + "purrdf-lex", + "purrdf-rdf", + "purrdf-sparql-algebra", + "rayon", + "regex", + "sha2", +] + +[[package]] +name = "purrdf-sparql-algebra" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "429dfe21a2abbcc515c3cf05ce354f0e29e0c36c9624342f964bcc9cab7cbb24" +dependencies = [ + "purrdf-cdt", + "purrdf-hash", + "purrdf-iri", + "purrdf-lex", + "purrdf-xsd", +] + +[[package]] +name = "purrdf-sparql-eval" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "08a95bb1166c152d784081ac144d10e482e1f2922d6438cdae5e6aa60f1ad69c" +dependencies = [ + "getrandom", + "hashbrown", + "purrdf-cdt", + "purrdf-core", + "purrdf-hash", + "purrdf-iri", + "purrdf-lex", + "purrdf-sparql-algebra", + "purrdf-sparql-results", + "purrdf-stack", + "purrdf-xsd", + "rayon", + "regex", + "sha2", + "wasm-bindgen", +] + +[[package]] +name = "purrdf-sparql-results" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "332b4b18dd46a7841eb776f44813c0d1cdd3f76b504c6ef151332575c916b355" +dependencies = [ + "purrdf-core", + "purrdf-iri", + "purrdf-lex", +] + +[[package]] +name = "purrdf-stack" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b9667b6e28591992199ae8841b5f82611fc92e99b25f1faf3826f22bfd359376" +dependencies = [ + "libc", + "windows-sys", +] + +[[package]] +name = "purrdf-xsd" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6d925ae9a7cac3fd7ded1fca01da0cc7caf895cf2df5ef4400669c39a753a8f8" +dependencies = [ + "purrdf-hash", + "purrdf-lex", +] + +[[package]] +name = "quote" +version = "1.0.47" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "r-efi" +version = "5.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f" + +[[package]] +name = "rayon" +version = "1.12.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fb39b166781f92d482534ef4b4b1b2568f42613b53e5b6c160e24cfbfa30926d" +dependencies = [ + "either", + "rayon-core", +] + +[[package]] +name = "rayon-core" +version = "1.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "22e18b0f0062d30d4230b2e85ff77fdfe4326feb054b9783a3460d8435c8ab91" +dependencies = [ + "crossbeam-deque", + "crossbeam-utils", +] + +[[package]] +name = "regex" +version = "1.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f020237b6c8eed93db2e2cb53c00c60a8e1bc73da7d073199a1180401450218d" +dependencies = [ + "aho-corasick", + "memchr", + "regex-automata", + "regex-syntax", +] + +[[package]] +name = "regex-automata" +version = "0.4.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ad8553b9b26413251cbf30e620595c7a41b3887f03da04579c0e6b0d6a06b4b2" +dependencies = [ + "aho-corasick", + "memchr", + "regex-syntax", +] + +[[package]] +name = "regex-syntax" +version = "0.8.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4" + +[[package]] +name = "rustversion" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f" + +[[package]] +name = "sha2" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "47d7069beb7d6ac7b9acd1039986e73443f24234f41074da099d6f994ac9ad19" +dependencies = [ + "cfg-if", + "cpufeatures", + "digest", +] + +[[package]] +name = "structured-zstd" +version = "0.0.49" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "392eea135985654a313cdfee001e93031bacd96a3693a5dc4b0d13793e40b27a" +dependencies = [ + "fastrand", + "twox-hash", +] + +[[package]] +name = "syn" +version = "2.0.119" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "twox-hash" +version = "2.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "86a801b3cea342a06d468c8710662aa29e5e05e4f5c0d62f00bbb7f2ad7941c2" + +[[package]] +name = "typenum" +version = "1.20.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20" + +[[package]] +name = "unicode-ident" +version = "1.0.27" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a2c754d6c33795a1c324727428e5a7dedb5b06195f9890bdbcba760d3e246563" + +[[package]] +name = "universal-hash" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f4987bdc12753382e0bec4a65c50738ffaabc998b9cdd1f952fb5f39b0048a96" +dependencies = [ + "crypto-common", + "ctutils", +] + +[[package]] +name = "visual-hashing" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5056c65a9a789c9c1156fbfab38df224d7b729089e3b075892136e8f3092166e" + +[[package]] +name = "wasip2" +version = "1.0.4+wasi-0.2.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b67efb37e106e55ce722a510d6b5f9c17f083e5fc79afc2badeb12cc313d9487" +dependencies = [ + "wit-bindgen", +] + +[[package]] +name = "wasm-bindgen" +version = "0.2.125" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8ddb3f79143bced6de84270411622a2699cee572fc0875aeaf1e7867cf9fca1a" +dependencies = [ + "cfg-if", + "once_cell", + "rustversion", + "wasm-bindgen-macro", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-macro" +version = "0.2.125" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4e21a184b13fb19e157296e2c46056aec9092264fab83e4ba59e68c61b323c3d" +dependencies = [ + "quote", + "wasm-bindgen-macro-support", +] + +[[package]] +name = "wasm-bindgen-macro-support" +version = "0.2.125" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fecefd9c35bd935a20fc3fc344b5f29138961e4f47fb03297d88f2587afb5ebd" +dependencies = [ + "bumpalo", + "proc-macro2", + "quote", + "syn", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-shared" +version = "0.2.125" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "23939e44bb9a5d7576fa2b563dc2e136628f1224e88a8deed09e04858b77871f" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] + +[[package]] +name = "wit-bindgen" +version = "0.57.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ebf944e87a7c253233ad6766e082e3cd714b5d03812acc24c318f549614536e" diff --git a/tools/shacl-purrdf-spike/Cargo.toml b/tools/shacl-purrdf-spike/Cargo.toml new file mode 100644 index 00000000..3bca0c17 --- /dev/null +++ b/tools/shacl-purrdf-spike/Cargo.toml @@ -0,0 +1,12 @@ +[package] +name = "atomic-purrdf-shacl-spike" +version = "0.1.0" +edition = "2024" +rust-version = "1.98" +publish = false + +# Keep this experiment outside Atomic's workspace/dependency graph. +[workspace] + +[dependencies] +purrdf-shapes = "=3.0.1" diff --git a/tools/shacl-purrdf-spike/README.md b/tools/shacl-purrdf-spike/README.md new file mode 100644 index 00000000..c488384b --- /dev/null +++ b/tools/shacl-purrdf-spike/README.md @@ -0,0 +1,46 @@ +# PurRDF SHACL integration spike + +Isolated test of the **published `purrdf-shapes = 3.0.1`** crate. This is not an +Atomic validator implementation and is deliberately outside the main workspace. +It does not alter Atomic's dependency graph, Rust requirement, or database. + +## Run + +PurRDF 3.0.1 requires Rust **1.98**; Atomic currently declares **1.90**. +Install the spike toolchain without changing the global default: + +```bash +rustup toolchain install 1.98.0 --profile minimal +rustup run 1.98.0 cargo test --manifest-path tools/shacl-purrdf-spike/Cargo.toml --locked +``` + +If a non-rustup `rustc` on PATH overrides the selected compiler (as with the +Homebrew installation used for this spike), use: + +```bash +RUSTC="$(rustup which --toolchain 1.98.0 rustc)" \ +RUSTDOC="$(rustup which --toolchain 1.98.0 rustdoc)" \ +rustup run 1.98.0 cargo test \ + --manifest-path tools/shacl-purrdf-spike/Cargo.toml --locked -- --nocapture +``` + +`Cargo.lock` records the complete experiment dependency resolution. No source +patches or `--ignore-rust-version` are used. + +## Coverage + +- Required plan cardinality: missing, exactly one, and multiple plans. +- Standard report fields, source shape/component, severity, and message. +- Nested `sh:node` acceptance-criterion enum validation. +- SHACL-SPARQL: met criteria require both reviewer and evidence. +- Inverse/sequence property paths, including complex path report serialization. +- Class, datatype, and boolean (`sh:or`) constraints. +- Separate missing properties retain distinct result paths. +- Malformed constraint parameters and SPARQL queries fail shape loading. +- Unsupported SHACL-JS fails loading, even with an empty data graph. +- Valid shapes with no applicable focus nodes conform vacuously (not audit evidence). +- Two project-scoped subjects with identical local IDs remain distinct. + +See [`docs/shacl-purrdf-spike-findings.md`](../../docs/shacl-purrdf-spike-findings.md) +for results and integration limits. These tests do not establish complete SHACL +conformance or enterprise compliance. diff --git a/tools/shacl-purrdf-spike/src/lib.rs b/tools/shacl-purrdf-spike/src/lib.rs new file mode 100644 index 00000000..b8908e4f --- /dev/null +++ b/tools/shacl-purrdf-spike/src/lib.rs @@ -0,0 +1,295 @@ +//! Isolated integration probe of the published PurRDF SHACL engine. +//! No Atomic database, policy evaluator, or syntax extraction is involved. + +#[cfg(test)] +mod tests { + use purrdf_shapes::engine::{parse_shapes, validate_graphs}; + use purrdf_shapes::report::ValidationReport; + + const PREFIXES: &str = r#" + @prefix atomic: . + @prefix sh: . + @prefix xsd: . + @prefix ex: . + "#; + const RDF_TYPE: &str = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type"; + const NS: &str = "https://atomic.dev/ns/workflow#"; + + fn iri(local: &str) -> String { + format!("") + } + + fn class(subject: &str, name: &str) -> String { + format!("{} <{RDF_TYPE}> <{NS}{name}> .\n", iri(subject)) + } + + fn edge(subject: &str, predicate: &str, object: &str) -> String { + format!("{} <{NS}{predicate}> {} .\n", iri(subject), iri(object)) + } + + fn literal(subject: &str, predicate: &str, value: &str) -> String { + format!("{} <{NS}{predicate}> \"{value}\" .\n", iri(subject)) + } + + fn validate(data: &str, shapes: &str) -> ValidationReport { + validate_graphs(data, &format!("{PREFIXES}\n{shapes}"), None) + .expect("shape loading and validation must succeed, not silently skip rules") + } + + #[test] + fn required_plan_cardinality_and_report_fields() { + let shapes = r#" + ex:TurnShape a sh:NodeShape; sh:targetClass atomic:ManagedTurn; + sh:property ex:PlanProperty . + ex:PlanProperty sh:path atomic:hasPlan; sh:minCount 1; sh:maxCount 1; + sh:message "A managed turn needs exactly one plan." . + "#; + let missing = class("turn:1", "ManagedTurn"); + let report = validate(&missing, shapes); + assert!(!report.conforms); + assert_eq!(report.results.len(), 1); + let result = &report.results[0]; + assert_eq!(result.focus_value(), "urn:atomic:project:TEST:turn:1"); + assert!( + result + .result_path + .as_ref() + .unwrap() + .to_string() + .contains("hasPlan") + ); + assert!(result.source_shape.to_string().contains("PlanProperty")); + assert!( + result + .source_constraint_component + .to_string() + .contains("MinCount") + ); + assert!(result.severity.iri().ends_with("Violation")); + assert_eq!(result.messages.len(), 1); + let nt = report.to_ntriples(); + for field in ["focusNode", "resultPath", "sourceShape", "resultMessage"] { + assert!(nt.contains(field), "missing standard report field: {field}"); + } + let good = format!("{missing}{}", edge("turn:1", "hasPlan", "intent:A")); + assert!(validate(&good, shapes).conforms); + let two = format!("{good}{}", edge("turn:1", "hasPlan", "intent:B")); + assert!(!validate(&two, shapes).conforms); + } + + #[test] + fn nested_acceptance_criterion_shape_is_enforced() { + let shapes = r#" + ex:IntentShape a sh:NodeShape; sh:targetClass atomic:WorkIntent; + sh:property [ sh:path atomic:hasAcceptanceCriterion; sh:node ex:ACShape ] . + ex:ACShape a sh:NodeShape; + sh:property [ sh:path atomic:status; sh:minCount 1; sh:in ("open" "met") ] . + "#; + let base = format!( + "{}{}", + class("intent:A", "WorkIntent"), + edge("intent:A", "hasAcceptanceCriterion", "ac:1") + ); + assert!(!validate(&base, shapes).conforms); + assert!( + !validate( + &format!("{base}{}", literal("ac:1", "status", "bogus")), + shapes + ) + .conforms + ); + assert!( + validate( + &format!("{base}{}", literal("ac:1", "status", "met")), + shapes + ) + .conforms + ); + } + + #[test] + fn sparql_conditional_met_requires_verifier_and_evidence() { + let shapes = r#" + ex:ACShape a sh:NodeShape; sh:targetClass atomic:AcceptanceCriterion; + sh:sparql [ a sh:SPARQLConstraint; + sh:message "Met criteria require a verifier and evidence."; + sh:select """ + PREFIX atomic: + SELECT $this WHERE { + $this atomic:status "met" . + FILTER ( + NOT EXISTS { $this atomic:verifiedBy ?verifier } || + NOT EXISTS { $this atomic:evidence ?evidence } + ) + } + """ ] . + "#; + let base = class("ac:1", "AcceptanceCriterion"); + assert!( + validate( + &format!("{base}{}", literal("ac:1", "status", "open")), + shapes + ) + .conforms + ); + let met = format!("{base}{}", literal("ac:1", "status", "met")); + assert!(!validate(&met, shapes).conforms); + let verified = format!("{met}{}", edge("ac:1", "verifiedBy", "identity:reviewer")); + assert!(!validate(&verified, shapes).conforms); + let complete = format!("{verified}{}", literal("ac:1", "evidence", "test-report")); + assert!(validate(&complete, shapes).conforms); + } + + #[test] + fn inverse_sequence_path_follows_change_to_turn_to_intent() { + let shapes = r#" + ex:ChangeShape a sh:NodeShape; sh:targetClass atomic:Change; + sh:property [ + sh:path ( [ sh:inversePath atomic:generatedChange ] atomic:hasPlan ); + sh:minCount 1 + ] . + "#; + let base = format!( + "{}{}", + class("change:1", "Change"), + edge("turn:1", "generatedChange", "change:1") + ); + let report = validate(&base, shapes); + assert!(!report.conforms); + assert!(report.results[0].result_path.is_some()); + assert!(report.results[0].path_structure.is_some()); + assert!(report.to_ntriples().contains("inversePath")); + assert!( + validate( + &format!("{base}{}", edge("turn:1", "hasPlan", "intent:A")), + shapes + ) + .conforms + ); + } + + #[test] + fn class_datatype_and_boolean_constraints() { + let shapes = r#" + ex:IntentShape a sh:NodeShape; sh:targetClass atomic:WorkIntent; + sh:property [ sh:path atomic:assignedTo; sh:minCount 1; sh:class atomic:Identity ]; + sh:property [ sh:path atomic:priority; sh:datatype xsd:integer ]; + sh:or ( + [ sh:property [ sh:path atomic:status; sh:hasValue "backlog" ] ] + [ sh:property [ sh:path atomic:hasTask; sh:minCount 1 ] ] + ) . + "#; + let base = format!( + "{}{}{}", + class("intent:A", "WorkIntent"), + edge("intent:A", "assignedTo", "identity:reviewer"), + literal("intent:A", "status", "backlog") + ); + assert!(!validate(&base, shapes).conforms); + let typed = format!("{base}{}", class("identity:reviewer", "Identity")); + assert!(validate(&typed, shapes).conforms); + let wrong_datatype = format!("{typed}{}", literal("intent:A", "priority", "high")); + assert!(!validate(&wrong_datatype, shapes).conforms); + let correct_datatype = format!( + "{typed}{} <{NS}priority> \"1\"^^ .\n", + iri("intent:A") + ); + assert!(validate(&correct_datatype, shapes).conforms); + let active = correct_datatype.replace("\"backlog\"", "\"active\""); + assert!(!validate(&active, shapes).conforms); + assert!( + validate( + &format!("{active}{}", edge("intent:A", "hasTask", "task:1")), + shapes + ) + .conforms + ); + } + + #[test] + fn two_missing_properties_produce_distinct_paths() { + let shapes = r#" + ex:ChangeShape a sh:NodeShape; sh:targetClass atomic:Change; + sh:property [ sh:path atomic:implements; sh:minCount 1 ]; + sh:property [ sh:path atomic:attestedBy; sh:minCount 1 ] . + "#; + let report = validate(&class("change:1", "Change"), shapes); + assert!(!report.conforms); + assert_eq!(report.results.len(), 2); + let paths: std::collections::BTreeSet<_> = report + .results + .iter() + .map(|r| r.result_path.as_ref().unwrap().to_string()) + .collect(); + assert_eq!(paths.len(), 2); + } + + #[test] + fn malformed_shape_parameter_is_rejected() { + let shapes = format!( + r#"{PREFIXES} + ex:BadShape a sh:NodeShape; sh:targetClass atomic:Change; + sh:property [ sh:path atomic:implements; sh:minCount "one" ] . + "# + ); + assert!(parse_shapes(&shapes, None).is_err()); + } + + #[test] + fn unsupported_shacl_js_is_rejected_even_with_no_targets() { + let shapes = format!( + r#"{PREFIXES} + ex:Unsupported a sh:NodeShape; sh:targetClass atomic:Change; + sh:js [ a sh:JSConstraint; sh:jsFunctionName "validate" ] . + "# + ); + assert!(parse_shapes(&shapes, None).is_err()); + assert!(validate_graphs("", &shapes, None).is_err()); + } + + #[test] + fn malformed_sparql_query_is_rejected() { + let shapes = format!( + r#"{PREFIXES} + ex:BadQuery a sh:NodeShape; sh:targetClass atomic:Change; + sh:sparql [ a sh:SPARQLConstraint; sh:select "SELECT $this WHERE {{" ] . + "# + ); + assert!(parse_shapes(&shapes, None).is_err()); + assert!(validate_graphs("", &shapes, None).is_err()); + } + + #[test] + fn valid_shapes_with_no_focus_nodes_conform_vacuously() { + let shapes = r#" + ex:ChangeShape a sh:NodeShape; sh:targetClass atomic:Change; + sh:property [ sh:path atomic:attestedBy; sh:minCount 1 ] . + "#; + let report = validate("", shapes); + assert!(report.conforms); + assert!(report.results.is_empty()); + // Atomic must report target coverage separately: this is not audit evidence. + assert!(!validate(&class("change:1", "Change"), shapes).conforms); + } + + #[test] + fn project_scoped_subjects_do_not_merge_in_a_combined_graph() { + let shapes = r#" + ex:TurnShape a sh:NodeShape; sh:targetClass atomic:ManagedTurn; + sh:property [ sh:path atomic:hasPlan; sh:minCount 1 ] . + "#; + let data = format!( + "{}{}{}", + class("turn:1", "ManagedTurn"), + class("turn:1", "ManagedTurn").replace(":TEST:", ":OTHER:"), + edge("turn:1", "hasPlan", "intent:A") + ); + let report = validate(&data, shapes); + assert!(!report.conforms); + assert_eq!(report.results.len(), 1); + assert_eq!( + report.results[0].focus_value(), + "urn:atomic:project:OTHER:turn:1" + ); + } +} From f9a16166018a9bdb531dfb5b436fd9ded6b12ca1 Mon Sep 17 00:00:00 2001 From: Bradley Hilton Date: Wed, 14 Oct 2026 09:52:20 -0500 Subject: [PATCH 06/19] docs: move KG cutover into Phase 1, fix explainedBy domain, keep identity on clone MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Phase 1 now covers clean up plus using the common language: the KG producers are rewritten to emit canonical terms and project-qualified IRIs, and every consumer moves onto them (new §1.6, milestone M3). The storage prerequisites (typed predicate API, kg_types, one-time KG_GENERATION rebuild, capability fence) move with it; SHACL becomes M4. - AST edges leave the KG, so `query callers` and triage blast radius switch to live atomic-semantic parsing (they read KG CALLS/DEFINES edges today). - Split atomic:explainedBy into turnExplainedBy and changeExplainedBy: a SessionTurn domain on the chain-derived property would infer every Change to be a SessionTurn. - Clones keep the origin's identity: only the `atomic init` command mints (clone calls Repository::init before pulling), pulls never change a populated identity, the first push binds the Atomic storage project, and mismatched pushes are rejected. Co-Authored-By: Claude Opus 5.5 --- docs/knowledge-graph-closure-phase1-plan.md | 174 ++++++++++++++++---- 1 file changed, 138 insertions(+), 36 deletions(-) diff --git a/docs/knowledge-graph-closure-phase1-plan.md b/docs/knowledge-graph-closure-phase1-plan.md index 748b34dc..612e4d86 100644 --- a/docs/knowledge-graph-closure-phase1-plan.md +++ b/docs/knowledge-graph-closure-phase1-plan.md @@ -6,7 +6,8 @@ ## Guiding principle for Phase 1 -Ship **three artifacts** that every downstream phase depends on: +Phase 1 is about **clean up, defining a common language, and using it**. It ships four +things that every downstream phase depends on: 1. **`atomic:` namespace** — a canonical workflow ontology (`.ttl` + Rust registry). 2. **`ProjectId`** — a stable, unique project identity minted at `atomic init` and cast @@ -16,10 +17,13 @@ Ship **three artifacts** that every downstream phase depends on: builtin and enterprise-authored shapes with an existing engine, provisionally `purrdf-shapes = 3.0.1`. No hand-rolled validator or constraint evaluator. Enterprise users add custom shapes in `.atomic/shapes/*.ttl`. +4. **KG cutover** — rewrite the KG producers to emit the canonical vocabulary and + project-qualified IRIs, and move every consumer onto it, so the common language is + actually in use by the end of Phase 1 (§1.6). -Everything else — projectors, storage cutover, inference, consumers — comes later. -This keeps Phase 1 reviewable while guaranteeing that IRIs are stable and validation -is real SHACL from day one. +Inference, the unified `reconcile()` pipeline, and mandatory publication gates come +later. By the end of Phase 1 the IRIs are stable, the KG speaks only the canonical +vocabulary, and validation is real SHACL. ## What we deliberately skip in Phase 1 @@ -37,7 +41,7 @@ that the syntax index cannot answer (e.g. federated cross-project search). | `atomic:modulePartOfModule`, `atomic:filePartOfModule` | Module hierarchy derivable from paths | | Scope-qualified entity IDs | Deferred to a phase that needs stable entity references | | Parser changes for scope chains / parameter types | Not needed without entity materialization | -| `query callers` KG-backed rewrite | Live-parsed implementation stays | +| KG-backed code-structure queries | `query callers` and triage blast radius read KG `CALLS`/`DEFINES`/`PART_OF`/`INCLUDES` edges today (`query/mod.rs:1223-1231`, `triage/report.rs:122-201, 683-691`); the Phase 1 cutover stops emitting those edges, so both move to live `atomic-semantic` parsing in Phase 1 (§1.6) | We keep **file-level** `atomic:modifiesFile` and `atomic:directlyOnView` because those are workflow facts, not code-structure facts. @@ -89,7 +93,9 @@ Version IRI: `https://atomic.dev/ns/workflow/1.0.0` | `atomic:hasPlan` | `atomic:SessionTurn` | `atomic:WorkIntent` | `atomic:governsTurn` | — | | `atomic:generatedChange` | `atomic:SessionTurn` | `atomic:Change` | `atomic:generatedBy` | `prov:generated` | | `atomic:informedBy` | `atomic:WorkIntent` | `atomic:Memory` | — | `prov:wasDerivedFrom` | -| `atomic:explainedBy` | `atomic:SessionTurn` | `atomic:ProvenanceGraph` | — | — | +| `atomic:explainedBy` | — | `atomic:ProvenanceGraph` | — | — | +| `atomic:turnExplainedBy` | `atomic:SessionTurn` | `atomic:ProvenanceGraph` | — | `atomic:explainedBy` | +| `atomic:changeExplainedBy` | `atomic:Change` | `atomic:ProvenanceGraph` | — | `atomic:explainedBy` | | `atomic:hasTodo` | `atomic:SessionTurn` | `atomic:Todo` | — | — | | `atomic:blocks` | `atomic:WorkIntent` | `atomic:WorkIntent` | `atomic:blockedBy` | — | | `atomic:assignedTo` | `atomic:WorkIntent` | `atomic:Identity` | — | — | @@ -139,9 +145,11 @@ but are **not** materialized in Phase 1. | Chain | Meaning | |---|---| | `atomic:generatedBy ∘ atomic:hasPlan` → `atomic:implements` | A change was generated by a turn that had a plan → the change implements that intent. | -| `atomic:generatedBy ∘ atomic:explainedBy` → `atomic:explainedBy` (on Change) | A change was generated by a turn explained by a provenance graph → the change is explained by that graph. | +| `atomic:generatedBy ∘ atomic:turnExplainedBy` → `atomic:changeExplainedBy` | A change was generated by a turn explained by a provenance graph → the change is explained by that graph. | -These chains use `atomic:generatedBy` (the inverse of `atomic:generatedChange`), so the intermediate node is consistently `SessionTurn`. In Phase 1 the chains are **declarative only** — the ontology records the intended meaning so that enterprise SHACL shapes and documentation see a complete model. The OWL 2 RL reasoner in Phase 2 will materialize them from the TTL. +These chains use `atomic:generatedBy` (the inverse of `atomic:generatedChange`), so the intermediate node is consistently `SessionTurn`. + +`atomic:explainedBy` has no domain of its own. In OWL a domain is an inference rule, not a check: a `SessionTurn` domain on the property that the chain derives for Changes would infer every Change to be a `SessionTurn`, and the publication shapes that target `SessionTurn` would then run against Changes. The two subproperties carry the domains instead; queries on `atomic:explainedBy` still see both kinds of edge. In Phase 1 the chains are **declarative only** — the ontology records the intended meaning so that enterprise SHACL shapes and documentation see a complete model. The OWL 2 RL reasoner in Phase 2 will materialize them from the TTL. **What the ontology does NOT contain:** @@ -190,12 +198,14 @@ source and the TTL cannot drift. ### 1.3 Project identity (`atomic_core::kg::project_id` + `atomic_core::kg::iri`) -**Design goal:** a single stable identifier per repository that survives views, +**Design goal:** a single stable identifier per project that survives views, inserts, unrecords, and clone/pull/push, and becomes the `` component in -project-qualified IRIs. +project-qualified IRIs. **Every clone of a project carries the origin's identity**; only +an explicit fork (later phase) mints a new one. **Format:** -- 256-bit random ID generated at `atomic init` time. +- 256-bit random ID generated by the `atomic init` command (not by + `Repository::init`; see Transport). - Canonical encoding: uppercase RFC 4648 base32 (52 characters, no padding), same alphabet as `Hash::to_base32`. - Display form: the base32 string. @@ -232,10 +242,34 @@ survive all graph mutations. The only place that guarantees this is a repository pristine table outside the view system. **Transport:** -- Clone / pull / push carry the project identity object as a repository-level - sidecar (not a change). The receiving side stores it in its own `PROJECT_IDENTITY` - table if the table is empty; if already populated, the clone keeps its own identity - (a cloned project is a new project). + +Clones keep the origin's identity, and pulls never change it. Rules: + +- **Where minting happens.** Only the `atomic init` command mints a new identity. + `Repository::init` does not, because `atomic clone` calls it first + (`clone/command.rs:992`) and then pulls; minting there would give every clone a fresh + identity and discard the origin's. +- **Clone** (from Atomic storage or any other remote): the remote supplies its + identity object as a repository-level sidecar (not a change). The new repository + verifies it (hash, and signature when present) and stores it in `PROJECT_IDENTITY`. + The clone is the same project as its origin. +- **Pull**: the local identity never changes. If the remote's identity matches, nothing + happens. If the local table is empty (a repository created before identities + existed), the pulled identity is adopted. If it differs, the pull still succeeds + (changes are global objects and may move between projects) but reports that the + remote belongs to a different project. +- **Push**: the first push to a storage project with no bound identity binds it (the + server records the identity hash against its `ProjectInfo.id`). Later pushes carrying + a different identity are rejected, so two unrelated repositories cannot claim the same + storage project. +- **Upgrading existing repositories:** a repository with no remote mints on its first + writable open. A repository with a remote does **not** auto-mint, since that would + diverge from its origin; it adopts the identity on its next pull. The owner mints once + with `atomic project identity init` and pushes, and every clone picks it up. +- **Atomic storage work:** the remote protocol (`atomic-remote`) and the storage + server gain an identity object transfer and the push-time binding check. The current + push/pull contract excludes sidecars (`atomic-remote/src/sync.rs:476`), so this is a + new object class, not a reuse of an existing path. - No fork command in Phase 1; fork semantics (new identity with `prov:wasDerivedFrom` pointing at origin) come later. @@ -335,7 +369,7 @@ there is no hand-rolled fallback. **Builtin shape graphs:** - `atomic-core/ontology/shapes/projection-1.0.0.ttl` — projection profile: - - every generated change has `atomic:implements`, `atomic:explainedBy`, and + - every generated change has `atomic:implements`, `atomic:changeExplainedBy`, and `atomic:attestedBy` shortcuts; - shortcut targets equal the authoritative intermediate path targets; - all referenced canonical nodes exist; @@ -390,9 +424,10 @@ Example enterprise traceability shape (illustrative, not a complete SOX policy): **Integration boundary — adaptation, not evaluation:** 1. Adapt a read-only workflow snapshot into RDF with canonical predicates/types, - full resource IRIs, typed literals, and properly scoped blank nodes. Existing - legacy workflow terms are translated at this boundary; AST relationships are - excluded. This does not require a storage cutover or rewriting signed objects. + full resource IRIs, typed literals, and properly scoped blank nodes. After the + §1.6 cutover the KG already uses canonical terms and IRIs, so the adapter does no + legacy-term translation; AST relationships are not in the KG. Signed objects are + never rewritten. 2. Load builtin and selected enterprise shape bundles with explicit base IRIs and controlled imports. Do not fetch arbitrary remote shape imports implicitly. 3. Call PurRDF's existing API. The spike uses @@ -449,18 +484,63 @@ Tests: versa, with matching domain/range. 4. No emitted string starts with `vault:`, `atom:`, or is SCREAMING_CASE. +### 1.6 KG cutover: producers and consumers + +The KG is entirely derived from canonical objects (vault entries, changes, views, the +session ledger, provenance sidecars, attestations), so the cutover is a rebuild under a +new generation, not an in-place data migration. + +**Storage prerequisites (minimal):** +- Typed write API: `upsert_kg_edge` takes a `Predicate` from `kg::ontology`, not a + `&str`, so unknown terms cannot be written. Reads reject non-canonical kinds. +- `kg_types` / `kg_types_rev` multimap tables (node IRI ↔ class IRI) for `rdf:type`. + Types are kept out of the edge tables so the kind-blind neighborhood traversal does + not connect every node to every same-typed node; the RDF adapter emits them as + `rdf:type` triples. `KgNode.kind` and `metadata.rdf_type` are removed. +- `KG_GENERATION` constant plus `kg_index_meta["kg_generation"]`. The rebuild runs once + per generation: automatically on the first writable open and via + `atomic query reindex --full`, regardless of repository size. The marker is written in + the same transaction as the final rebuild batch, so an interrupted rebuild restarts + cleanly and a completed one never repeats. Repositories created by the new binary are + stamped at creation. Read-only opens on an old-generation KG return `KgNeedsRebuild`. +- Capability fence `kg-semweb-v1` (`capability.rs`) so older binaries fail closed. + +**Producer rewrite** (canonical predicates, `rdf:type`, project-qualified IRIs via +`kg::iri`, full-hash change IRIs): + +| Producer | Today | Phase 1 | +|---|---|---| +| Vault projection | `vault_triples.rs` | intents (subclass from canonical `kind`), memories, goals, skills, tool results, decomposition children, `:::ref` edges split by edge name (`dependsOnIntent`, `blockedBy`, `remediates`, `reviews`) | +| VCS projection | `vault_kg_enrich.rs` (VCS part) | views and `childViewOf`, changes (full hash), `directlyDependsOnChange`, `modifiesFile`, `authoredBy`, and `directlyOnView` recomputed from `VIEW_CHANGES` (today `ON_VIEW` records the working copy's desired view, not membership) | +| Session ledger | `session_kg.rs` | sessions, turns (`ManagedTurn` when `plan_id` is present), todos, provenance graphs; `hasTurn`, `hasPlan`, `generatedChange`, `turnExplainedBy`, `hasTodo`, `sessionViewContext` | +| Attestations (**new**) | not projected | `Attestation` nodes, `coversChange`, `attests`, `signedBy`, `previousAttestation`, and the verifier's recorded result (never inferred) | +| Findings (**new**) | none | `MissingHistoricalPlanId`, `MissingAttestation`, `AmbiguousAttestationTurn`, `UnverifiableAttestation`; never guessed links | +| AST enrichment | `vault_kg_enrich.rs` (AST part) | **removed** from the KG: no entity nodes, `CALLS`, `DEFINES`, `EXPORTS`, `INCLUDES`, `PART_OF` | + +Existing triggers stay as they are (record, vault sync, clone, pull, import, session +hooks); unifying them into one `reconcile()` pipeline is Phase 2. + +**Consumer cutover:** +- `query_plan.rs`: `Filter` on `kind` becomes type-based; add a predicate filter and + direction to neighborhood steps; bump the plan JSON schema version. +- CLI `query`: graph colour map keyed by IRI; text/DOT output compacts IRIs to CURIEs via + `PREFIXES`; JSON emits absolute IRIs; the `ask` prompt lists canonical predicates. +- `query callers` and the triage report's blast radius / layer ordering + (`triage/report.rs`) switch from KG code edges to live `atomic-semantic` parsing. +- Triage, `vault_intent.rs`, and `vault/context.rs` match canonical predicates and types. +- libatomic daemon duplicates (`services_agent.rs`, `services_query.rs`), `rpc.rs`, and + the knowledge proto (`KGEdge.relation` documented as an absolute IRI; node types added; + API version bumped). The release is announced as breaking; no compatibility shim. +- Update the ~110 test assertion sites. + ## 2. Out of scope for Phase 1 | Item | Deferred to | |---|---| | `KgEdge.graph` (`Asserted`/`Inferred`) | Phase 2 | -| `kg_types` / `kg_types_rev` tables | Phase 2 | -| Storage generation 2 / `KG_GENERATION` | Phase 2 | -| Projector rewrite (`vault.rs`, `vcs.rs`, `session.rs`, `attestation.rs`, `findings.rs`) | Phase 2 | -| `reconcile()` pipeline | Phase 2 | +| Unified `reconcile()` pipeline (one entry point for sync/enrich/reindex) | Phase 2 | | OWL 2 RL inference engine (`kg::reason`) | Phase 2 | | Cross-project SPARQL endpoints / federation catalogs | Later federation phase | -| Consumer cutover (`query`, triage, daemon, proto) | Phase 2 | | Code-structure edges (`calls`, `defines`, `exports`, `includes`, `modifiesEntity`) | When sourced from AST | | Scope-qualified entity IDs | Same — when code-structure edges are needed | | Project identity rotation / fork command | Phase 2 | @@ -479,6 +559,9 @@ Tests: 6. **Project identity:** `atomic init` creates a `ProjectId`; `atomic project identity show` displays it; the ID round-trips through base32 encoding; IRIs minted via `kg::iri` match the canonical format. + - A clone (from Atomic storage and from a local path) has the origin's identity. + - A pull never changes a populated identity; a differing remote identity is reported. + - Pushing a different identity to a bound storage project is rejected. 7. **SHACL integration:** PurRDF evaluates positive and negative fixtures for both builtin profiles, nested shapes, local SPARQL conditions, and property paths. No Atomic-owned constraint evaluator. The MSRV/deployment decision is explicit. @@ -493,6 +576,11 @@ Tests: 11. **Regression corpus:** retain the isolated spike, add the existing gate fixtures and relevant W3C cases, and compare against an independent SHACL implementation before enabling mandatory production gates. +12. **Cutover:** after the one-time rebuild, the KG contains only canonical predicates, + `rdf:type` entries in `kg_types`, project-qualified IRIs, and full-hash change IRIs; + no AST nodes or edges remain. A second writable open performs no rebuild work. +13. **Consumers:** every consumer listed in §1.6 works against the canonical KG; + `query callers` and triage blast radius produce the same results from live parsing. ## 4. Milestones @@ -509,12 +597,27 @@ Tests: - `atomic_core::kg::iri` (mint/parse for every node kind) - `PROJECT_IDENTITY` pristine table - Identity object (JSON-LD, content-addressed, optionally signed) -- `atomic init` mints on creation; upgrading repos mint on first writable open -- Minimal clone/pull/push transport (repository-level sidecar) -- CLI: `atomic project identity show` +- The `atomic init` command mints (not `Repository::init`); upgrading repos without a + remote mint on first writable open; repos with a remote adopt on pull +- Clone/pull/push identity transport, including Atomic storage: clone adopts, pull never + changes a populated identity, first push binds the storage project, mismatched pushes + are rejected +- CLI: `atomic project identity show` and `atomic project identity init` - Property tests: round-trip, percent-encoding, strict base32 parsing -### M3 — PurRDF SHACL integration + builtin shapes (depends on M1/M2) +### M3 — KG cutover (depends on M1/M2) + +- Typed predicate write API, `kg_types`/`kg_types_rev`, `KG_GENERATION` one-time + rebuild, capability fence +- Producer rewrite (vault, VCS, session ledger) plus new attestation and findings + projectors; AST enrichment removed from the KG +- Consumer cutover (query plan, CLI, triage, vault context, daemon, proto); `query + callers` and triage blast radius on live parsing +- Test updates; the lint in acceptance criterion 5 passes + +Lands as one PR (or a stack merged together): the RFC allows no dual-write window. + +### M4 — PurRDF SHACL integration + builtin shapes (depends on M1/M3) - Retain the standalone `purrdf-shapes = 3.0.1` spike and lockfile. - Resolve Rust 1.98 versus Atomic's 1.90 MSRV: approved embedded upgrade or separate @@ -574,14 +677,13 @@ outside the view system is the only place that guarantees this. ### Phase 2 expectations Phase 2 will: -- rewrite projectors to emit project-qualified IRIs using `kg::iri`; -- implement `kg_types`/`kg_types_rev` for type indexing; -- add the OWL 2 RL inference engine; -- wire SHACL validation into `vault sync` and publication gates; -- cut over consumers to use canonical IRIs. - -Because Phase 1 already minted `ProjectId`, Phase 2 projectors can use it -immediately without a second migration or ID rewrite. +- add the OWL 2 RL inference engine and the `KgEdge.graph` asserted/inferred split, so + `implements` and `changeExplainedBy` are materialized from the chains; +- unify sync/enrich/reindex into one `reconcile()` pipeline; +- wire SHACL validation into `vault sync` and publication gates. + +Phase 1 already mints `ProjectId` and moves the KG to canonical IRIs, so Phase 2 needs +no second migration or ID rewrite. --- From 1c4d6daffd66bb553f9012c3f5b359d3ad923a8e Mon Sep 17 00:00:00 2001 From: Bradley Hilton Date: Wed, 14 Oct 2026 12:15:08 -0500 Subject: [PATCH 07/19] docs: make Phase 1 plan self-contained and add proposals for review Plan updates: - Restore a "Current code (starting point)" section with file:line references to the KG storage, producers, vocabulary, identity problems, overloaded relations, unprojected facts, and consumers. - Defer clone/pull/push identity transport until atomic-storage implements it (clone is HTTP-only, so all three depend on the server); Phase 1 covers local identity work only. Acceptance criterion 6 marks the transport checks as deferred. - Add implementation rules: uppercase identifiers with strict URN parsers, old-layout intents resolved via the manifest or a finding, attestation-to-turn mapping only on a unique match, MAC attestations recorded as Unverifiable, and atom: terms translated at projection time without rewriting signed documents. New docs/knowledge-graph-phase1-proposals.md, for approval: the complete LEGACY_TERMS mapping, missing ontology terms (including a status/priority domain bug), the ProjectIdentity context and hash construction, and two open questions (PurRDF's Rust 1.98 requirement, MAC signers vs. the publication profile). Co-Authored-By: Claude Opus 5.5 --- docs/knowledge-graph-closure-phase1-plan.md | 159 +++++++++++++- docs/knowledge-graph-phase1-proposals.md | 217 ++++++++++++++++++++ 2 files changed, 366 insertions(+), 10 deletions(-) create mode 100644 docs/knowledge-graph-phase1-proposals.md diff --git a/docs/knowledge-graph-closure-phase1-plan.md b/docs/knowledge-graph-closure-phase1-plan.md index 612e4d86..5ce4c220 100644 --- a/docs/knowledge-graph-closure-phase1-plan.md +++ b/docs/knowledge-graph-closure-phase1-plan.md @@ -46,6 +46,122 @@ that the syntax index cannot answer (e.g. federated cross-project search). We keep **file-level** `atomic:modifiesFile` and `atomic:directlyOnView` because those are workflow facts, not code-structure facts. +## Current code (starting point) + +Verified against `dev` at `5ddcf1b`. Line numbers drift; search for the named symbol if a +reference is off. + +### C.1 Storage + +- The KG is a **JSON property graph**, not a triple store: + `KgNode { id, kind, label, summary, source, metadata }` and + `KgEdge { from_id, to_id, kind, metadata }` (`atomic-core/src/pristine/vault.rs:462,516`). +- Tables (`atomic-core/src/pristine/tables.rs:797-830`): `kg_nodes`, `kg_edges` + (key `from\0to\0kind`, `encode_edge_key` L856), `kg_edges_from`, `kg_edges_to`, + `kg_fts`, `kg_fts_by_node`, `kg_index_meta`. No predicate or type index. +- The predicate string is **part of the edge key**, so renaming means rewriting keys. +- `upsert_kg_edge` (`txn/write/triples.rs:380`) accepts any string. `ontology::is_known_predicate` + has no callers. +- `kg_neighbors` (`traits/triples.rs:18-68`) is kind-blind, bidirectional BFS with depth ≤ 2; + it is the main read primitive (query plans, `vault context`, RAG, daemon). + +### C.2 The KG is entirely derived + +Every node/edge comes from one of three producers, all over authoritative objects: + +| Producer | File | Source tag | Trigger | +|---|---|---|---| +| Vault projection | `atomic-repository/src/repository/vault_triples.rs` | `vault` | every `vault_store` (intent/memory/goal writes, `vault sync`, record w/ vault sync), `query reindex` | +| VCS/AST enrichment | `atomic-repository/src/repository/vault_kg_enrich.rs` | `vcs`, `ast` | clone, pull, git import, record (`kg_enrich_change`), `query enrich` | +| Session ledger | `atomic-core/src/pristine/txn/write/session_kg.rs` | `session_ledger` | turn append/rewrite, session lifecycle, sidecar pull, `session rebuild` | + +**Consequence for the cutover:** nothing in the KG needs in-place rewriting. A storage- +generation bump + full rebuild from canonical objects *is* the migration, and the RFC's +"fail on unclassifiable overloaded edge" rule becomes a projector-level guarantee (each +projector knows its endpoint classes at emit time). + +`vault sync` and `query enrich` share **no** code; they meet only on shared ID strings. + +### C.3 Vocabulary inventory (`atomic-core/src/pristine/ontology.rs`) + +- 6 prefix labels (no IRIs), 30 `predicate::*` CURIEs, 16 `entity_type::*` (`vault:*`), + 29 `edge_kind::*` SCREAMING_CASE strings. +- Actually emitted: 19 uppercase kinds + `prov:{wasDerivedFrom,used,wasAssociatedWith, + hadPlan,hadMember,generated,wasInformedBy}`, `atom:informedBy`, `vault:{explainedBy,hasTodo}`. +- Defined but never emitted: all `dc:*`, `skos:*`, `rdf:type` (as edge), `vault:{status, + priority,assignedTo,blocks,...}` predicates, `IMPORTS`, `VISIBLE_FROM`, and 12 of 16 + `entity_type` values. +- Typing: `kind` string on every node; `metadata.rdf_type` (prefixed, e.g. `vault:Session`) + only on the 4 session-ledger node kinds. **No real `rdf:type` anywhere.** + +### C.4 Identity problems + +- Change node IDs are **12-char base32 prefixes** in three places + (`vault_kg_enrich.rs:1089 truncate_hash`, `session_kg.rs:46 change_node_id`, + `vault_triples.rs:1268 rdf_target_to_kg_id`). Full hash survives only in metadata. +- `identity:` IDs are display names (`identity:{author.name}` vs `identity:{fm.developer}`); + intents write `author`/`created_by` which the KG ignores, so intents have no author edge. +- `todo:X` from `urn:atomic:todo:X` collides with the legacy session-global todo form. +- `LINKED_INTENT` targets `intent:{raw}` without uppercasing/manifest resolution. +- **No `ProjectId` exists.** Closest: remote `ProjectInfo.id: Uuid` (server-assigned), + `VaultManifest.intent_prefix` (4-char human code), `repository_identity` DID in + `provenance_gate.rs:76`. + +### C.5 Overloaded / misused relations (confirmed in code) + +| Current | Uses | Lines | +|---|---|---| +| `DEPENDS_ON` | change→change; intent→intent (fallback for any `:::ref`) | enrich :409,:526; triples :1158 | +| `MODIFIES` | change→file; change→entity **only on record-time path**, not bulk | enrich :436,:554,:624 | +| `ON_VIEW` | change→*working copy's desired view* (not membership!); session→view | enrich :386,:514; session_kg :320 | +| `prov:wasAssociatedWith` | tool_result→goal | triples :1002 | +| `prov:hadPlan` | session→intent **and** turn→intent | session_kg :389-397 | +| `prov:hadMember` | session→turn | session_kg :381 | +| `prov:wasDerivedFrom` | memory→{anything}; session→session (fork) | triples :978; session_kg :456 | + +### C.6 Facts captured but never projected + +- `Attestation` (`atomic-core/src/change/attestation.rs:153`): `session_id`, + `changes_covered`, `signer` (`did:atomic:…` or `session-mac:{sid}`), `signature`, + `previous_attestation`, `provenance_root`, … Stored as `.attest` files. + **No KG projection.** +- Canonical intent/memory attestations (signed JSON-LD vault entries) — explicitly skipped + by the KG (`vault_triples.rs:53`). +- `SessionTurn` (`atomic-core/src/change/session.rs:418`): `plan_id`, `goal`, + `provenance_hash`, `change_hashes`, `previous_provenance`, `outcome`, boundaries — + partially projected (no outcome/boundaries). +- AI `Provenance` on changes (model/tokens/cost): not part of Phase 1. + +### C.7 Consumers that hardcode kind strings + +- `atomic-cli/src/commands/query/mod.rs`: `callers` (`to_uppercase()=="CALLS"` :1231,:1255), + HTML `EDGE_COLORS` (:748-755), `ask` system prompt (:1971-1976), text output. +- `atomic-repository/src/triage/report.rs` (`kind_is` at :35, case-insensitive, ~12 sites), + `atomic-repository/src/repository/triage.rs:231,305`. +- `atomic-repository/src/repository/vault_intent.rs:1770,1776` (`atom:informedBy`). +- `atomic-cli/src/commands/vault/context.rs:470-473` (filters `node.kind == "memory"`). +- libatomic daemon copies: `services_agent.rs:131,3372-3387`, `services_query.rs:1001-1038,1059`. +- Wire: `libatomic/proto/.../knowledge_messages.proto:123` `KGEdge.relation` (free string). +- Out-of-repo clients (`../noname`, `../the-hive`) also match kind strings; the release is + announced as breaking, so they are not updated as part of this plan. +- ~110 test assertion sites (largest: `vault_triples.rs` ~40, `ontology.rs` ~31, + `atomic-cli/tests/session_integration_test.rs` ~17). + +### C.8 Existing Semantic Web pieces and constraints + +- `atomic-canonical` emits signed JSON-LD with context `https://atomic.dev/ns/ctx.jsonld`, + mapping the old `atom:` prefix to `https://atomic.dev/ns#`. Its content hash is + `blake3(JCS(node))`, which covers the `@context` **URL string** but not the term + definitions behind it. Signed documents are never rewritten. +- `atomic provenance` projects PROV JSON-LD on demand (`atomic-canonical/src/prov.rs`); + it is not written to the KG. +- No RDF/OWL/SHACL crates are in the workspace and there are no `.ttl` files yet. SHACL + engine selection is in §1.4 and `docs/shacl-purrdf-spike-findings.md`. +- Upgrade machinery: `SCHEMA_VERSION` only rejects newer databases (`schema.rs`); + **capability fences** (`capability.rs`, `required-capability/`) make older binaries + fail closed; `kg_index_meta` holds per-index markers (`fts_reverse_schema` = 2 is the + precedent for a KG rebuild). + ## 1. Phase 1 scope ### 1.1 Ontology (`atomic-core/ontology/workflow-1.0.0.ttl`) @@ -243,6 +359,14 @@ pristine table outside the view system. **Transport:** +> **Phase 1 implementation boundary.** `atomic clone` is HTTP-only (plus a Git bootstrap +> path that never contacts storage), so identity transport over clone, pull and push all +> depends on server support in `atomic-storage`, which is owned by another effort. In +> Phase 1, implement only the local pieces: minting in the `atomic init` command (not in +> `Repository::init`), the `PROJECT_IDENTITY` table, the upgrade rule, and the CLI. **Do +> not change push, pull, clone, `atomic-remote`, or the daemon's sync code yet.** The rules +> below are the target behavior that the later transport work implements. + Clones keep the origin's identity, and pulls never change it. Rules: - **Where minting happens.** Only the `atomic init` command mints a new identity. @@ -298,7 +422,19 @@ pub fn mint_file_iri(pid: &ProjectId, path: &str) -> String; ``` All minting functions percent-encode path/ name segments per RFC 3987 and guarantee -round-trip stability. Property tests cover paths with spaces, `#`, `:`, non-ASCII, +round-trip stability. + +**Identifier casing.** Every identifier inside a URN is uppercase: base32 hashes and +project IDs (as `Hash::to_base32` emits) and intent ULIDs (uppercase Crockford, as +`ulid::Ulid::to_string()` emits, `vault_intent.rs:141`). Parsers for URNs are strict: they +reject lowercase, padding, wrong length, and (for base32) nonzero trailing bits. The +existing `Hash::from_base32` is case-insensitive, so URN parsing needs a new strict parser; +lenient prefix lookup in CLI arguments is unaffected. + +**Old-layout intents.** Intents created before the ULID layout live at +`intents////intent.md` (e.g. `ATOM-2` in this repo's vault) and +carry no ULID. The projector resolves their identity through the vault manifest; if none is +recorded, it emits a finding instead of inventing an ID. Property tests cover paths with spaces, `#`, `:`, non-ASCII, and the empty string. **CLI:** @@ -510,10 +646,10 @@ new generation, not an in-place data migration. | Producer | Today | Phase 1 | |---|---|---| -| Vault projection | `vault_triples.rs` | intents (subclass from canonical `kind`), memories, goals, skills, tool results, decomposition children, `:::ref` edges split by edge name (`dependsOnIntent`, `blockedBy`, `remediates`, `reviews`) | +| Vault projection | `vault_triples.rs` | intents (subclass from canonical `kind`), memories, goals, skills, tool results, decomposition children, `:::ref` edges split by edge name (`dependsOnIntent`, `blockedBy`, `remediates`, `reviews`). Signed canonical documents keep their old `atom:` terms; the projector translates them through `LEGACY_TERMS` at projection time and never rewrites signed bytes | | VCS projection | `vault_kg_enrich.rs` (VCS part) | views and `childViewOf`, changes (full hash), `directlyDependsOnChange`, `modifiesFile`, `authoredBy`, and `directlyOnView` recomputed from `VIEW_CHANGES` (today `ON_VIEW` records the working copy's desired view, not membership) | | Session ledger | `session_kg.rs` | sessions, turns (`ManagedTurn` when `plan_id` is present), todos, provenance graphs; `hasTurn`, `hasPlan`, `generatedChange`, `turnExplainedBy`, `hasTodo`, `sessionViewContext` | -| Attestations (**new**) | not projected | `Attestation` nodes, `coversChange`, `attests`, `signedBy`, `previousAttestation`, and the verifier's recorded result (never inferred) | +| Attestations (**new**) | not projected | `Attestation` nodes, `coversChange`, `attests`, `signedBy`, `previousAttestation`, and the verifier's recorded result (never inferred). **Turn mapping:** an attestation records `session_id` and `changes_covered` but no turn number, so emit `attests` only when exactly one turn of that session has `change_hashes` ⊆ `changes_covered`; otherwise emit an `AmbiguousAttestationTurn` finding and no `attests` edge. MAC-signed attestations (`session-mac:{sid}`) can't be verified without the key and record `Unverifiable`, never `Verified` | | Findings (**new**) | none | `MissingHistoricalPlanId`, `MissingAttestation`, `AmbiguousAttestationTurn`, `UnverifiableAttestation`; never guessed links | | AST enrichment | `vault_kg_enrich.rs` (AST part) | **removed** from the KG: no entity nodes, `CALLS`, `DEFINES`, `EXPORTS`, `INCLUDES`, `PART_OF` | @@ -559,9 +695,11 @@ hooks); unifying them into one `reconcile()` pipeline is Phase 2. 6. **Project identity:** `atomic init` creates a `ProjectId`; `atomic project identity show` displays it; the ID round-trips through base32 encoding; IRIs minted via `kg::iri` match the canonical format. - - A clone (from Atomic storage and from a local path) has the origin's identity. - - A pull never changes a populated identity; a differing remote identity is reported. - - Pushing a different identity to a bound storage project is rejected. + - `Repository::init` (as called by `atomic clone`) does not mint an identity. + - **Deferred until `atomic-storage` implements identity transport (owned by another + effort):** a clone has the origin's identity; a pull never changes a populated + identity and reports a differing one; pushing a different identity to a bound + storage project is rejected. Do not implement or test these in Phase 1. 7. **SHACL integration:** PurRDF evaluates positive and negative fixtures for both builtin profiles, nested shapes, local SPARQL conditions, and property paths. No Atomic-owned constraint evaluator. The MSRV/deployment decision is explicit. @@ -598,10 +736,11 @@ hooks); unifying them into one `reconcile()` pipeline is Phase 2. - `PROJECT_IDENTITY` pristine table - Identity object (JSON-LD, content-addressed, optionally signed) - The `atomic init` command mints (not `Repository::init`); upgrading repos without a - remote mint on first writable open; repos with a remote adopt on pull -- Clone/pull/push identity transport, including Atomic storage: clone adopts, pull never - changes a populated identity, first push binds the storage project, mismatched pushes - are rejected + remote mint on first writable open; repos with a remote stay unminted until transport + lands (or the owner runs `atomic project identity init`) +- **Not in Phase 1:** clone/pull/push identity transport. It waits for `atomic-storage` + (see the boundary note in §1.3); leave push, pull, clone, `atomic-remote`, and the daemon's + sync code unchanged - CLI: `atomic project identity show` and `atomic project identity init` - Property tests: round-trip, percent-encoding, strict base32 parsing diff --git a/docs/knowledge-graph-phase1-proposals.md b/docs/knowledge-graph-phase1-proposals.md new file mode 100644 index 00000000..939b7e5a --- /dev/null +++ b/docs/knowledge-graph-phase1-proposals.md @@ -0,0 +1,217 @@ +# Phase 1 Plan: Proposals for Approval + +> Status: **for review.** Companion to +> [`knowledge-graph-closure-phase1-plan.md`](./knowledge-graph-closure-phase1-plan.md). +> Each section below is a proposed addition to the plan. Approved sections get merged +> into the plan; this file is then deleted. + +| # | Proposal | Plan section it fills | +|---|---|---| +| A | The `LEGACY_TERMS` mapping: every old constant → its canonical replacement | §1.5 (the table is currently `&[ ... ]`) | +| B | Ontology terms the plan uses but never declares, plus one domain bug | §1.1 | +| C | `ProjectIdentity` JSON-LD context and hash construction | §1.3 | +| D | Two questions that need a decision (neither blocks M1–M3) | new "Open questions" | + +--- + +## A. `LEGACY_TERMS` mapping + +The code has the old terms (`atomic-core/src/pristine/ontology.rs`: 6 prefixes, +30 `predicate::*`, 16 `entity_type::*`, 29 `edge_kind::*`) but no mapping. The M1 +coverage test requires each old constant to appear exactly once with one of: +`One`/`OnePred` (one replacement), `SplitByEndpoint` (replacement depends on the +endpoint classes), or `Removed` (with a reason). + +Rows marked **(B)** depend on a term proposed in section B. + +### A.1 Classes (`entity_type::*` and node `kind` values) + +| Old | Replacement | +|---|---| +| `vault:Goal` / `goal` | `atomic:Goal` | +| `vault:Intent` / `intent` | `atomic:WorkIntent`, plus `FeatureIntent` / `BugIntent` / `ReviewIntent` from the canonical `kind`, and `ManagedIntent` from lifecycle status | +| `vault:Memory` / `memory` | `atomic:Memory` | +| `vault:Skill` / `skill` | `atomic:Skill` | +| `vault:ToolResult` / `tool_result` | `atomic:ToolResult` | +| `vault:File` / `file` | `atomic:File` | +| `vault:Identity` / `identity` | `atomic:Identity` | +| `vault:Concept` / `concept` | `atomic:Concept` (⊑ `skos:Concept`, per §1.1) | +| `vault:Session` / `session` | `atomic:Session` | +| `vault:SessionTurn` / `session_turn` | `atomic:SessionTurn`, plus `atomic:ManagedTurn` when `plan_id` is present | +| `vault:Todo` / `todo` | `atomic:Todo` | +| `vault:ProvenanceGraph` / `provenance` | `atomic:ProvenanceGraph` | +| `vault:Task` / `task` | `atomic:Task` | +| `vault:AcceptanceCriterion` / `acceptance_criterion` | `atomic:AcceptanceCriterion` | +| `vault:ScopeItem` / `scope` | `atomic:ScopeItem` | +| `vault:Constraint` / `constraint` | `atomic:Constraint` | +| `change` | `atomic:Change` | +| `view` | `atomic:View` | +| `module` | **Removed**: module hierarchy is derivable from paths (plan "What we skip") | +| `entity` | **Removed**: source entities stay in the syntax index (plan "What we skip") | +| *(new)* | `atomic:Attestation`, `atomic:Finding`; `atomic:Project`, `atomic:ProjectIdentity` **(B)** | + +### A.2 Predicates and edge kinds + +| Old | Endpoints | Replacement | +|---|---|---| +| `prov:hadMember` | session → turn | `atomic:hasTurn` | +| `prov:hadPlan` | turn → intent | `atomic:hasPlan` | +| `prov:hadPlan` | session → intent | **Removed**: derivable as session `hasTurn` → turn `hasPlan` | +| `prov:generated` | turn → change | `atomic:generatedChange` (⊑ `prov:generated`) | +| `prov:wasInformedBy` | turn → previous turn | `prov:wasInformedBy`, kept as PROV **(B)** | +| `prov:wasDerivedFrom` | memory → source; forked session → parent | `prov:wasDerivedFrom`, kept as PROV **(B)** | +| `prov:used` | tool result → file | `prov:used`, kept as PROV **(B)** | +| `prov:wasAssociatedWith` | tool result → goal | `atomic:linkedGoal` (PROV association is for activity → agent) | +| `prov:wasGeneratedBy` | never emitted | **Removed** | +| `atom:informedBy` | intent → memory/source | `atomic:informedBy` (⊑ `prov:wasDerivedFrom`) | +| `vault:explainedBy` | turn → provenance graph | `atomic:turnExplainedBy` (⊑ `atomic:explainedBy`) | +| `vault:hasTodo` | turn → todo | `atomic:hasTodo` | +| `BLOCKS` / `BLOCKED_BY` | intent ↔ intent | `atomic:blocks` / `atomic:blockedBy` (inverses) | +| `ASSIGNED_TO` | intent → identity | `atomic:assignedTo` | +| `LINKED_INTENT` | goal → intent | `atomic:linkedIntent` (target now resolved through the manifest) | +| `LINKED_GOAL` | intent → goal | `atomic:linkedGoal` | +| `REFERENCES` | entry → memory, entry → file | `atomic:references` | +| `HAS_LABEL` | intent → concept | `atomic:hasLabel` | +| `HAS_TASK` | intent → task | `atomic:hasTask` | +| `HAS_ACCEPTANCE_CRITERION` | intent → criterion | `atomic:hasAcceptanceCriterion` | +| `HAS_SCOPE_IN` / `HAS_SCOPE_OUT` | intent → scope item | `atomic:hasScopeIn` / `atomic:hasScopeOut` (⊑ `atomic:hasScopeItem`) | +| `HAS_CONSTRAINT` | intent → constraint | `atomic:hasConstraint` | +| `SATISFIES` | task → criterion | `atomic:satisfies` | +| `TOUCHES` | task → file | `atomic:touches` | +| `SCOPE_OUT_FILE` | intent → file | `atomic:scopeOutFile` | +| `REMEDIATES` | intent → intent | `atomic:remediates` | +| `REVIEWS` | review intent → intent | `atomic:reviews` | +| `DEPENDS_ON` | change → change | `atomic:directlyDependsOnChange` (⊑ `atomic:dependsOn`) | +| `DEPENDS_ON` | intent → intent | `atomic:dependsOnIntent` (⊑ `atomic:dependsOn`) | +| `MODIFIES` | change → file | `atomic:modifiesFile` (⊑ `atomic:modifies`) | +| `MODIFIES` | change → entity | **Removed**: code structure (plan "What we skip") | +| `ON_VIEW` | change → view | `atomic:directlyOnView`, recomputed from `VIEW_CHANGES` (today it records the working copy's desired view, not membership) | +| `ON_VIEW` | session → view | `atomic:sessionViewContext` | +| `CHILD_OF` | view → parent view | `atomic:childViewOf` | +| `VISIBLE_FROM` | never emitted | `atomic:visibleFrom`, declared as a query-time operator, never stored | +| `AUTHORED_BY` | change/goal/intent → identity | `atomic:authoredBy` (⊑ `prov:wasAttributedTo`); intents now emit it from `author`/`created_by` | +| `DEFINES`, `EXPORTS`, `INCLUDES`, `IMPORTS`, `CALLS` | code structure | **Removed**: syntax index (plan "What we skip") | +| `PART_OF` | file → module, module → module | **Removed**: derivable from paths | +| `vault:status`, `vault:priority` | literal | `atomic:status`, `atomic:priority` (see the domain fix in B.6) | +| `vault:blocks`, `vault:blockedBy`, `vault:assignedTo`, `vault:linkedIntent`, `vault:linkedGoal`, `vault:references`, `vault:hasLabel` | never emitted; duplicates of the uppercase kinds | **Removed**: the uppercase twins are mapped above | +| `vault:appliesTo` | never emitted | **Removed** | +| `dc:type`, `dc:creator`, `dc:date`, `dc:description` | never emitted | **Removed** (creator → `authoredBy`; dates and descriptions come from typed source fields, see B.5) | +| `skos:broader`, `skos:narrower`, `skos:subject`, `skos:related` | never emitted | **Removed** | +| `rdf:type` (as an edge) | never emitted | **Removed**: types are stored in `kg_types` (§1.6) | + +### A.3 Prefix constants + +| Old | Replacement | +|---|---| +| `prefix::{ATOM, PROV, SKOS, DC, VAULT, RDF}` | **Removed**: replaced by the `PREFIXES` table in `kg::ontology` (§1.2) | + +--- + +## B. Ontology terms to add + +The plan's §1.1 tables reference these, or the producers in §1.6 emit them, but the +ontology doesn't declare them. Without them the TTL ↔ Rust consistency test and the +`LEGACY_TERMS` coverage test (which requires every target to exist in the TTL) would fail. + +### B.1 Parents and inverses that are referenced but not declared + +| Term | Kind | Notes | +|---|---|---| +| `atomic:dependsOn` | object property, `owl:TransitiveProperty` | Parent of `directlyDependsOnChange` / `dependsOnIntent`; never materialized in Phase 1 | +| `atomic:modifies` | object property | Parent of `modifiesFile` | +| `atomic:hasScopeItem` | object property | Parent of `hasScopeIn` / `hasScopeOut` | +| `atomic:ancestorView` | object property, `owl:TransitiveProperty` | Mentioned in §1.1; never materialized | +| `atomic:turnOfSession`, `atomic:governsTurn`, `atomic:generatedBy`, `atomic:blockedBy`, `atomic:reviewedBy` | object properties, `owl:inverseOf` their partners | Named in the "Inverse" column but have no rows; `generatedBy` is used by both property chains | + +### B.2 PROV terms the producers keep + +Declared as reused terms (`rdfs:isDefinedBy `), not redefined: +`prov:wasInformedBy` (turn → previous turn), `prov:wasDerivedFrom` (memory → source, +forked session → parent), `prov:used` (tool result → file), and the two parents the plan +already uses, `prov:generated` and `prov:wasAttributedTo`. + +### B.3 Attestation verification + +| Term | Kind | +|---|---| +| `atomic:verificationResult` | object property, `Attestation` → `atomic:VerificationStatus` | +| `atomic:VerificationStatus` | class | +| `atomic:Verified`, `atomic:Unverifiable`, `atomic:Failed` | individuals of `VerificationStatus` | + +Written only by the cryptographic verifier, never inferred. MAC-signed attestations +record `Unverifiable`. + +### B.4 Project identity + +| Term | Kind | +|---|---| +| `atomic:Project` | class (the `urn:atomic:project:` node) | +| `atomic:ProjectIdentity` | class (the signed identity object) | +| `atomic:hasIdentity` | object property, `Project` → `ProjectIdentity` | +| `atomic:projectId` | datatype property, `xsd:string` (52-character uppercase base32) | +| `atomic:schemaVersion` | datatype property, `xsd:nonNegativeInteger` | +| `atomic:issuer` | object property, `ProjectIdentity` → `Identity` | + +Creation time uses `dcterms:created` (B.5). + +### B.5 Literals the producers carry today + +`rdfs:label` (labels), `dcterms:description` (summaries), `dcterms:created` / +`dcterms:modified` (timestamps), and `atomic:turnNumber` (`xsd:nonNegativeInteger`). + +### B.6 Domain bug: `atomic:status` and `atomic:priority` + +Both are declared with domain `atomic:WorkIntent`, but tasks, acceptance criteria, +todos, and sessions also carry a status (and todos a priority). An OWL domain is an +inference rule, not a check, so every task, criterion, todo, and session would be +inferred to be a `WorkIntent`, and intent shapes would then run against them. This is +the same problem that was fixed for `explainedBy`. + +**Proposal:** remove the domain from both properties. They are literals, so no +subproperties are needed; shapes constrain the allowed values per class. + +--- + +## C. `ProjectIdentity` context and hash + +**Problem.** §1.3's identity object uses `"@context": "https://atomic.dev/ns/ctx.jsonld"`. +That context defines the old `atom:` prefix but not `atomic:`, so `"atomic:ProjectIdentity"` +does not expand to `https://atomic.dev/ns/workflow#ProjectIdentity`; a JSON-LD processor +reads `atomic:` as an unknown URI scheme. The hash, `blake3(JCS(...))`, has no type +prefix, so it isn't separated from other object types. + +**Proposal.** `ProjectIdentity` is new and has no existing signed copies, so fixing it now +costs nothing: + +```jsonld +{ + "@context": { "atomic": "https://atomic.dev/ns/workflow#", + "dcterms": "http://purl.org/dc/terms/" }, + "@type": "atomic:ProjectIdentity", + "atomic:projectId": "ABCD...EFGH", + "dcterms:created": "2026-01-15T09:00:00Z", + "atomic:schemaVersion": 1, + "atomic:issuer": "did:atomic:..." +} +``` + +- The context is inline, so the hashed bytes include the term definitions and their + meaning can never drift. +- Content hash = `blake3("atomic.project-identity.v1" ‖ 0x00 ‖ JCS(object))`. +- Existing object hashes (changes, attestations, provenance, canonical intents and + memories) are unchanged. + +--- + +## D. Open questions (need a decision; neither blocks M1–M3) + +1. **PurRDF needs Rust 1.98; Atomic declares 1.90** (`Cargo.toml:27`). Raise Atomic's + minimum Rust version, or run validation as a separately built binary? This affects + CI, packaging, and every contributor. Blocks M4 only. +2. **MAC-signed attestations vs. the publication profile.** The publication shape + requires a signer identity and a successful verification result. MAC-signed + attestations (`session-mac:{sid}`) can't be verified without the key, so MAC-signed + work would fail it. Require DID signers for publication, or allow MAC signers that + `[git.trust]` explicitly trusts? Phase 1 only records MAC attestations as + `Unverifiable`; the decision is needed before publication gates are enforced in + Phase 2. From 860027798678d155776854d3d7f5d34f8fbd136d Mon Sep 17 00:00:00 2001 From: Bradley Hilton Date: Wed, 14 Oct 2026 12:39:35 -0500 Subject: [PATCH 08/19] docs: record DID-signing decision and Rust 1.98 findings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Publication requires an atomic identity DID signer; MAC-signed attestations never satisfy the publication profile (decided by Lee). Applied to plan §1.4 and the attestation projector row in §1.6. - Rust 1.98: CI's `cargo clippy --workspace -- -D warnings` passes with rust-version = "1.98" on 1.98.1. A workspace-wide bump would break atomic-storage (Docker images pin rust:1.94) and sherpa-server (declares 1.94), since every crate inherits the workspace rust-version. Recommend a scoped bump: PurRDF in its own crate at 1.98 used only by atomic-cli, shared library crates stay at 1.90. Co-Authored-By: Claude Opus 5.5 --- docs/knowledge-graph-closure-phase1-plan.md | 8 ++-- docs/knowledge-graph-phase1-proposals.md | 42 ++++++++++++++++----- 2 files changed, 37 insertions(+), 13 deletions(-) diff --git a/docs/knowledge-graph-closure-phase1-plan.md b/docs/knowledge-graph-closure-phase1-plan.md index 5ce4c220..acc5f48f 100644 --- a/docs/knowledge-graph-closure-phase1-plan.md +++ b/docs/knowledge-graph-closure-phase1-plan.md @@ -516,8 +516,10 @@ there is no hand-rolled fallback. - no managed turn is governed by more than one intent; - every turn has exactly one provenance graph; - every generated change is covered by the session attestation; - - every attestation has the required signer identity and an explicit successful - verification result supplied by a separate cryptographic verifier. + - every attestation is signed by an atomic identity DID (`did:atomic:…`) and has an + explicit successful verification result supplied by a separate cryptographic + verifier. MAC-signed attestations (`session-mac:{sid}`) never satisfy the publication + profile; agents publish with their atomic identity DID (decided by Lee, 2026-10-14). SHACL cannot establish signature validity by itself. These shapes define the intended profiles; fixture tests in Phase 1 prove their behavior, while complete @@ -649,7 +651,7 @@ new generation, not an in-place data migration. | Vault projection | `vault_triples.rs` | intents (subclass from canonical `kind`), memories, goals, skills, tool results, decomposition children, `:::ref` edges split by edge name (`dependsOnIntent`, `blockedBy`, `remediates`, `reviews`). Signed canonical documents keep their old `atom:` terms; the projector translates them through `LEGACY_TERMS` at projection time and never rewrites signed bytes | | VCS projection | `vault_kg_enrich.rs` (VCS part) | views and `childViewOf`, changes (full hash), `directlyDependsOnChange`, `modifiesFile`, `authoredBy`, and `directlyOnView` recomputed from `VIEW_CHANGES` (today `ON_VIEW` records the working copy's desired view, not membership) | | Session ledger | `session_kg.rs` | sessions, turns (`ManagedTurn` when `plan_id` is present), todos, provenance graphs; `hasTurn`, `hasPlan`, `generatedChange`, `turnExplainedBy`, `hasTodo`, `sessionViewContext` | -| Attestations (**new**) | not projected | `Attestation` nodes, `coversChange`, `attests`, `signedBy`, `previousAttestation`, and the verifier's recorded result (never inferred). **Turn mapping:** an attestation records `session_id` and `changes_covered` but no turn number, so emit `attests` only when exactly one turn of that session has `change_hashes` ⊆ `changes_covered`; otherwise emit an `AmbiguousAttestationTurn` finding and no `attests` edge. MAC-signed attestations (`session-mac:{sid}`) can't be verified without the key and record `Unverifiable`, never `Verified` | +| Attestations (**new**) | not projected | `Attestation` nodes, `coversChange`, `attests`, `signedBy`, `previousAttestation`, and the verifier's recorded result (never inferred). **Turn mapping:** an attestation records `session_id` and `changes_covered` but no turn number, so emit `attests` only when exactly one turn of that session has `change_hashes` ⊆ `changes_covered`; otherwise emit an `AmbiguousAttestationTurn` finding and no `attests` edge. MAC-signed attestations (`session-mac:{sid}`) can't be verified without the key and record `Unverifiable`, never `Verified`; they never satisfy the publication profile, which requires an atomic identity DID signer | | Findings (**new**) | none | `MissingHistoricalPlanId`, `MissingAttestation`, `AmbiguousAttestationTurn`, `UnverifiableAttestation`; never guessed links | | AST enrichment | `vault_kg_enrich.rs` (AST part) | **removed** from the KG: no entity nodes, `CALLS`, `DEFINES`, `EXPORTS`, `INCLUDES`, `PART_OF` | diff --git a/docs/knowledge-graph-phase1-proposals.md b/docs/knowledge-graph-phase1-proposals.md index 939b7e5a..7f8b1001 100644 --- a/docs/knowledge-graph-phase1-proposals.md +++ b/docs/knowledge-graph-phase1-proposals.md @@ -205,13 +205,35 @@ costs nothing: ## D. Open questions (need a decision; neither blocks M1–M3) -1. **PurRDF needs Rust 1.98; Atomic declares 1.90** (`Cargo.toml:27`). Raise Atomic's - minimum Rust version, or run validation as a separately built binary? This affects - CI, packaging, and every contributor. Blocks M4 only. -2. **MAC-signed attestations vs. the publication profile.** The publication shape - requires a signer identity and a successful verification result. MAC-signed - attestations (`session-mac:{sid}`) can't be verified without the key, so MAC-signed - work would fail it. Require DID signers for publication, or allow MAC signers that - `[git.trust]` explicitly trusts? Phase 1 only records MAC attestations as - `Unverifiable`; the decision is needed before publication gates are enforced in - Phase 2. +1. **PurRDF needs Rust 1.98; Atomic declares 1.90** (`Cargo.toml:27`). Lee is happy to + upgrade to 1.98 as long as nothing else breaks. Findings (2026-10-14): + + - **This repo is clean at 1.98.** With `rust-version = "1.98"`, CI's + `cargo clippy --workspace -- -D warnings` passes with no warnings on Rust 1.98.1, + including the clippy lints that only switch on above a given MSRV. + - **Where 1.90 is pinned:** `Cargo.toml:27` (`rust-version`), the CI MSRV job + (`.github/workflows/ci.yml:136`), and the **release build** + (`.github/workflows/release.yml:129`, so shipped binaries are compiled with 1.90). + The other CI jobs use `stable`. + - **What a workspace-wide bump would break.** Every crate inherits the workspace + `rust-version`, and Cargo refuses to build a dependency whose `rust-version` is newer + than the compiler. Sibling repos that build this repo's crates by path: + + | Repo | Uses | Pinned Rust | Effect of a workspace-wide 1.98 bump | + |---|---|---|---| + | `atomic-storage` (owned by another effort) | `atomic-core`, `atomic-canonical`, `atomic-identity`, `atomic-objects`, `atomic-repository` | `rust-version = "1.94"`, Docker `rust:1.94-slim-bookworm` (`Dockerfile:14`, `Dockerfile.stress:11`) | **Docker builds fail** until both images and its `rust-version` move to 1.98 | + | `basecamp/sherpa-server` | `atomic-identity` | `rust-version = "1.94"` | No longer meets its declared minimum | + | `atomic-enterprise` | `atomic-agent`, `atomic-core`, `atomic-config`, `atomic-identity`, `atomic-repository`, `atomic-remote` | none | Needs a ≥ 1.98 compiler | + + **Options:** + + | Option | What changes | Breaks others? | + |---|---|---| + | **(a) Scoped bump** (recommended) | PurRDF lives in its own crate (working name `atomic-shacl`) with `rust-version = "1.98"`, depended on only by `atomic-cli` (and libatomic, if the daemon validates). Shared library crates stay at 1.90, and **the SHACL crate must not become a dependency of any of them** (`atomic-core`, `atomic-repository`, `atomic-remote`, `atomic-agent`, …); `atomic query validate` wires it in at the CLI layer. The release workflow moves to 1.98; the CI MSRV job checks the library crates on 1.90 and the CLI on 1.98. This is also the "dedicated package" shape already anticipated for the validator. | **No.** None of the three depends on `atomic-cli` or `libatomic`, directly or through `atomic-repository`/`atomic-remote`/`atomic-agent` (checked) | + | (b) Workspace-wide bump | `Cargo.toml:27`, `ci.yml:136`, `release.yml:129` → 1.98 | **Yes**, until `atomic-storage` updates its Docker images and `rust-version`, and `sherpa-server` its `rust-version` (one-line changes each, owned by others) | + + Blocks M4 only. +2. ~~**MAC-signed attestations vs. the publication profile.**~~ **Decided (Lee, + 2026-10-14):** this is Atomic, so publication requires signing with the atomic + identity DID (`did:atomic:…`). MAC-signed attestations record `Unverifiable` and never + satisfy the publication profile. Applied to plan §1.4 and §1.6. From ae4de005abd2cd9dd12a50a869b4e6a7e6155bb0 Mon Sep 17 00:00:00 2001 From: Bradley Hilton Date: Wed, 14 Oct 2026 16:08:12 -0500 Subject: [PATCH 09/19] docs: merge approved Phase 1 proposals into the plan MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lee approved the proposals; merge them and delete the review doc. - §1.5: the complete LEGACY_TERMS mapping (all 81 old constants, each exactly once); coverage test now includes prefix constants. - §1.1: missing ontology terms (parent and inverse properties, verificationResult and VerificationStatus, Project/ProjectIdentity terms, reused PROV and literal terms, turnNumber) and remove the WorkIntent domain from status/priority. - §1.3: ProjectIdentity uses an inline @context and a type-prefixed hash, blake3("atomic.project-identity.v1" || 0x00 || JCS). - §1.4/M4: scoped Rust 1.98 bump. PurRDF goes in its own atomic-shacl crate used only by atomic-cli; shared library crates stay at 1.90 so sibling repos that build them by path keep working. Co-Authored-By: Claude Opus 5.5 --- docs/knowledge-graph-closure-phase1-plan.md | 182 +++++++++++++-- docs/knowledge-graph-phase1-proposals.md | 239 -------------------- 2 files changed, 161 insertions(+), 260 deletions(-) delete mode 100644 docs/knowledge-graph-phase1-proposals.md diff --git a/docs/knowledge-graph-closure-phase1-plan.md b/docs/knowledge-graph-closure-phase1-plan.md index acc5f48f..d0dfea2b 100644 --- a/docs/knowledge-graph-closure-phase1-plan.md +++ b/docs/knowledge-graph-closure-phase1-plan.md @@ -200,6 +200,9 @@ Version IRI: `https://atomic.dev/ns/workflow/1.0.0` | `atomic:File` | `owl:Thing` | Workflow artifact, not code entity | | `atomic:Identity` | `owl:Thing` | | | `atomic:Concept` | `skos:Concept` | | +| `atomic:Project` | `owl:Thing` | The `urn:atomic:project:` node | +| `atomic:ProjectIdentity` | `owl:Thing` | The signed identity object (§1.3) | +| `atomic:VerificationStatus` | `owl:Thing` | Individuals `atomic:Verified`, `atomic:Unverifiable`, `atomic:Failed` | **Object properties** (workflow edges only): @@ -244,6 +247,23 @@ Version IRI: `https://atomic.dev/ns/workflow/1.0.0` | `atomic:implements` | `atomic:Change` | `atomic:WorkIntent` | — | — | | `atomic:attestedBy` | `atomic:Change` | `atomic:Attestation` | — | — | | `atomic:aboutResource` | `atomic:Finding` | `owl:Thing` | — | — | +| `atomic:dependsOn` | — | — | — | `owl:TransitiveProperty`; parent of `directlyDependsOnChange` / `dependsOnIntent` | +| `atomic:modifies` | `atomic:Change` | — | — | Parent of `modifiesFile` | +| `atomic:hasScopeItem` | `atomic:WorkIntent` | `atomic:ScopeItem` | — | Parent of `hasScopeIn` / `hasScopeOut` | +| `atomic:ancestorView` | `atomic:View` | `atomic:View` | — | `owl:TransitiveProperty` | +| `atomic:turnOfSession` | `atomic:SessionTurn` | `atomic:Session` | `atomic:hasTurn` | — | +| `atomic:governsTurn` | `atomic:WorkIntent` | `atomic:SessionTurn` | `atomic:hasPlan` | — | +| `atomic:generatedBy` | `atomic:Change` | `atomic:SessionTurn` | `atomic:generatedChange` | Used by both property chains | +| `atomic:blockedBy` | `atomic:WorkIntent` | `atomic:WorkIntent` | `atomic:blocks` | — | +| `atomic:reviewedBy` | `atomic:WorkIntent` | `atomic:ReviewIntent` | `atomic:reviews` | — | +| `atomic:verificationResult` | `atomic:Attestation` | `atomic:VerificationStatus` | — | Written only by the cryptographic verifier, never inferred; MAC-signed attestations record `atomic:Unverifiable` | +| `atomic:hasIdentity` | `atomic:Project` | `atomic:ProjectIdentity` | — | — | +| `atomic:issuer` | `atomic:ProjectIdentity` | `atomic:Identity` | — | — | + +**Reused PROV terms** (declared with `rdfs:isDefinedBy `, +not redefined): `prov:wasInformedBy` (turn → previous turn), `prov:wasDerivedFrom` +(memory → source, forked session → parent), `prov:used` (tool result → file), and the +parents `prov:generated` and `prov:wasAttributedTo`. Note: `atomic:dependsOn` and `atomic:ancestorView` are declared `owl:TransitiveProperty` but are **not** materialized in Phase 1. @@ -252,9 +272,20 @@ but are **not** materialized in Phase 1. | Property | Domain | Range | |---|---|---| -| `atomic:status` | `atomic:WorkIntent` | `xsd:string` | -| `atomic:priority` | `atomic:WorkIntent` | `xsd:string` | +| `atomic:status` | — | `xsd:string` | +| `atomic:priority` | — | `xsd:string` | | `atomic:findingCode` | `atomic:Finding` | `xsd:string` | +| `atomic:projectId` | `atomic:ProjectIdentity` | `xsd:string` (52-character uppercase base32) | +| `atomic:schemaVersion` | `atomic:ProjectIdentity` | `xsd:nonNegativeInteger` | +| `atomic:turnNumber` | `atomic:SessionTurn` | `xsd:nonNegativeInteger` | + +`atomic:status` and `atomic:priority` have **no domain**: tasks, acceptance criteria, +todos, and sessions also carry a status (and todos a priority), and an OWL domain is an +inference rule, so a `WorkIntent` domain would infer each of them to be an intent. Shapes +constrain the allowed values per class. + +**Reused literal terms:** `rdfs:label` (labels), `dcterms:description` (summaries), +`dcterms:created` / `dcterms:modified` (timestamps). **Property chains** (declared in TTL as `owl:propertyChainAxiom` for Phase 2 inference): @@ -338,16 +369,25 @@ an explicit fork (later phase) mints a new one. ```jsonld { - "@context": "https://atomic.dev/ns/ctx.jsonld", + "@context": { "atomic": "https://atomic.dev/ns/workflow#", + "dcterms": "http://purl.org/dc/terms/" }, "@type": "atomic:ProjectIdentity", - "atomic:projectId": "ABCD...EFGH", // the 52-char base32 - "atomic:createdAt": "2026-01-15T09:00:00Z", - "atomic:schemaVersion": 2, - "atomic:issuer": "did:atomic:..." // optional in Phase 1; may be unknown + "atomic:projectId": "ABCD...EFGH", + "dcterms:created": "2026-01-15T09:00:00Z", + "atomic:schemaVersion": 1, + "atomic:issuer": "did:atomic:..." } ``` -- Content hash = `blake3(JCS(normalized_bytes))`. +- `atomic:projectId` is the 52-character uppercase base32 ID. `atomic:issuer` is + optional in Phase 1 and may be unknown. +- The `@context` is **inline**, so the hashed bytes include the term definitions and their + meaning can never drift. The existing `https://atomic.dev/ns/ctx.jsonld` context is not + used: it defines the old `atom:` prefix but not `atomic:`, so `atomic:` terms would not + expand to the workflow namespace. +- Content hash = `blake3("atomic.project-identity.v1" ‖ 0x00 ‖ JCS(object))`. The type + prefix keeps identity hashes distinct from every other object type. Existing object + hashes (changes, attestations, provenance, canonical intents and memories) are unchanged. - Signed by the initializing identity if available; unsigned is allowed in Phase 1 (the object schema reserves the `proof` field). @@ -495,12 +535,23 @@ PurRDF documents broader Core/SPARQL support; additional constructs must be cove by profile regression tests before we depend on them. `oxttl` and `oxrdf` are parser/data-model utilities, **not** the validation engine. -**Toolchain/deployment gate:** PurRDF 3.0.1 requires **Rust 1.98**, while Atomic -currently declares **1.90**. The spike uses a separate toolchain and resolves 73 -dependency packages (including target-specific packages). M3 must explicitly choose -an approved MSRV upgrade for embedded use or a separately built PurRDF runner. -Do not silently raise Atomic's MSRV. Either deployment runs the same SHACL shapes; -there is no hand-rolled fallback. +**Toolchain (decided by Lee, 2026-10-14): scoped Rust 1.98 bump.** PurRDF 3.0.1 requires +Rust 1.98; the workspace declares 1.90 (`Cargo.toml:27`). This repo passes CI's +`cargo clippy --workspace -- -D warnings` with `rust-version = "1.98"`, but every crate +inherits the workspace `rust-version`, and sibling repos build these crates by path with +older pins (`atomic-storage`: `rust:1.94` Docker images; `basecamp/sherpa-server`: +`rust-version = "1.94"`). A workspace-wide bump would break them. So: + +- PurRDF lives in its **own crate** (working name `atomic-shacl`) with + `rust-version = "1.98"`, depended on only by `atomic-cli` (and libatomic, if the daemon + validates). +- The shared library crates (`atomic-core`, `atomic-repository`, `atomic-remote`, + `atomic-agent`, `atomic-canonical`, `atomic-identity`, `atomic-objects`, + `atomic-config`, …) stay at 1.90 and **must not depend on the SHACL crate**. + `atomic query validate` wires it in at the CLI layer. +- The release workflow (`.github/workflows/release.yml:129`) moves to 1.98. The CI MSRV + job (`ci.yml:136`) checks the library crates on 1.90 and `atomic-cli` on 1.98. +- Do not raise the workspace `rust-version`. There is no hand-rolled fallback validator. **Builtin shape graphs:** @@ -612,10 +663,97 @@ pub enum Replacement { static LEGACY_TERMS: &[(/* old */ &'static str, Replacement)] = &[ ... ]; ``` +The table below is the complete content of `LEGACY_TERMS` (approved by Lee, +2026-10-14). The code has the old constants in `atomic-core/src/pristine/ontology.rs` +(6 prefixes, 30 `predicate::*`, 16 `entity_type::*`, 29 `edge_kind::*`) but no mapping; +each appears below exactly once. "Removed" rows carry their reason into +`Replacement::Removed`. + +#### Classes (`entity_type::*` and node `kind` values) + +| Old | Replacement | +|---|---| +| `vault:Goal` / `goal` | `atomic:Goal` | +| `vault:Intent` / `intent` | `atomic:WorkIntent`, plus `FeatureIntent` / `BugIntent` / `ReviewIntent` from the canonical `kind`, and `ManagedIntent` from lifecycle status | +| `vault:Memory` / `memory` | `atomic:Memory` | +| `vault:Skill` / `skill` | `atomic:Skill` | +| `vault:ToolResult` / `tool_result` | `atomic:ToolResult` | +| `vault:File` / `file` | `atomic:File` | +| `vault:Identity` / `identity` | `atomic:Identity` | +| `vault:Concept` / `concept` | `atomic:Concept` (⊑ `skos:Concept`, per §1.1) | +| `vault:Session` / `session` | `atomic:Session` | +| `vault:SessionTurn` / `session_turn` | `atomic:SessionTurn`, plus `atomic:ManagedTurn` when `plan_id` is present | +| `vault:Todo` / `todo` | `atomic:Todo` | +| `vault:ProvenanceGraph` / `provenance` | `atomic:ProvenanceGraph` | +| `vault:Task` / `task` | `atomic:Task` | +| `vault:AcceptanceCriterion` / `acceptance_criterion` | `atomic:AcceptanceCriterion` | +| `vault:ScopeItem` / `scope` | `atomic:ScopeItem` | +| `vault:Constraint` / `constraint` | `atomic:Constraint` | +| `change` | `atomic:Change` | +| `view` | `atomic:View` | +| `module` | **Removed**: module hierarchy is derivable from paths (plan "What we skip") | +| `entity` | **Removed**: source entities stay in the syntax index (plan "What we skip") | +| *(new)* | `atomic:Attestation`, `atomic:Finding`; `atomic:Project`, `atomic:ProjectIdentity` | + +#### Predicates and edge kinds + +| Old | Endpoints | Replacement | +|---|---|---| +| `prov:hadMember` | session → turn | `atomic:hasTurn` | +| `prov:hadPlan` | turn → intent | `atomic:hasPlan` | +| `prov:hadPlan` | session → intent | **Removed**: derivable as session `hasTurn` → turn `hasPlan` | +| `prov:generated` | turn → change | `atomic:generatedChange` (⊑ `prov:generated`) | +| `prov:wasInformedBy` | turn → previous turn | `prov:wasInformedBy`, kept as PROV | +| `prov:wasDerivedFrom` | memory → source; forked session → parent | `prov:wasDerivedFrom`, kept as PROV | +| `prov:used` | tool result → file | `prov:used`, kept as PROV | +| `prov:wasAssociatedWith` | tool result → goal | `atomic:linkedGoal` (PROV association is for activity → agent) | +| `prov:wasGeneratedBy` | never emitted | **Removed** | +| `atom:informedBy` | intent → memory/source | `atomic:informedBy` (⊑ `prov:wasDerivedFrom`) | +| `vault:explainedBy` | turn → provenance graph | `atomic:turnExplainedBy` (⊑ `atomic:explainedBy`) | +| `vault:hasTodo` | turn → todo | `atomic:hasTodo` | +| `BLOCKS` / `BLOCKED_BY` | intent ↔ intent | `atomic:blocks` / `atomic:blockedBy` (inverses) | +| `ASSIGNED_TO` | intent → identity | `atomic:assignedTo` | +| `LINKED_INTENT` | goal → intent | `atomic:linkedIntent` (target now resolved through the manifest) | +| `LINKED_GOAL` | intent → goal | `atomic:linkedGoal` | +| `REFERENCES` | entry → memory, entry → file | `atomic:references` | +| `HAS_LABEL` | intent → concept | `atomic:hasLabel` | +| `HAS_TASK` | intent → task | `atomic:hasTask` | +| `HAS_ACCEPTANCE_CRITERION` | intent → criterion | `atomic:hasAcceptanceCriterion` | +| `HAS_SCOPE_IN` / `HAS_SCOPE_OUT` | intent → scope item | `atomic:hasScopeIn` / `atomic:hasScopeOut` (⊑ `atomic:hasScopeItem`) | +| `HAS_CONSTRAINT` | intent → constraint | `atomic:hasConstraint` | +| `SATISFIES` | task → criterion | `atomic:satisfies` | +| `TOUCHES` | task → file | `atomic:touches` | +| `SCOPE_OUT_FILE` | intent → file | `atomic:scopeOutFile` | +| `REMEDIATES` | intent → intent | `atomic:remediates` | +| `REVIEWS` | review intent → intent | `atomic:reviews` | +| `DEPENDS_ON` | change → change | `atomic:directlyDependsOnChange` (⊑ `atomic:dependsOn`) | +| `DEPENDS_ON` | intent → intent | `atomic:dependsOnIntent` (⊑ `atomic:dependsOn`) | +| `MODIFIES` | change → file | `atomic:modifiesFile` (⊑ `atomic:modifies`) | +| `MODIFIES` | change → entity | **Removed**: code structure (plan "What we skip") | +| `ON_VIEW` | change → view | `atomic:directlyOnView`, recomputed from `VIEW_CHANGES` (today it records the working copy's desired view, not membership) | +| `ON_VIEW` | session → view | `atomic:sessionViewContext` | +| `CHILD_OF` | view → parent view | `atomic:childViewOf` | +| `VISIBLE_FROM` | never emitted | `atomic:visibleFrom`, declared as a query-time operator, never stored | +| `AUTHORED_BY` | change/goal/intent → identity | `atomic:authoredBy` (⊑ `prov:wasAttributedTo`); intents now emit it from `author`/`created_by` | +| `DEFINES`, `EXPORTS`, `INCLUDES`, `IMPORTS`, `CALLS` | code structure | **Removed**: syntax index (plan "What we skip") | +| `PART_OF` | file → module, module → module | **Removed**: derivable from paths | +| `vault:status`, `vault:priority` | literal | `atomic:status`, `atomic:priority` (no domain; see §1.1) | +| `vault:blocks`, `vault:blockedBy`, `vault:assignedTo`, `vault:linkedIntent`, `vault:linkedGoal`, `vault:references`, `vault:hasLabel` | never emitted; duplicates of the uppercase kinds | **Removed**: the uppercase twins are mapped above | +| `vault:appliesTo` | never emitted | **Removed** | +| `dc:type`, `dc:creator`, `dc:date`, `dc:description` | never emitted | **Removed** (creator → `authoredBy`; dates and descriptions come from typed source fields, see §1.1 datatype properties) | +| `skos:broader`, `skos:narrower`, `skos:subject`, `skos:related` | never emitted | **Removed** | +| `rdf:type` (as an edge) | never emitted | **Removed**: types are stored in `kg_types` (§1.6) | + +#### Prefix constants + +| Old | Replacement | +|---|---| +| `prefix::{ATOM, PROV, SKOS, DC, VAULT, RDF}` | **Removed**: replaced by the `PREFIXES` table in `kg::ontology` (§1.2) | + Tests: -1. Every legacy `predicate::*`, `entity_type::*`, and `edge_kind::*` constant appears - exactly once in `LEGACY_TERMS`. +1. Every legacy `prefix::*`, `predicate::*`, `entity_type::*`, and `edge_kind::*` constant + appears exactly once in `LEGACY_TERMS`. 2. Every `One`/`Split` target exists in the TTL. 3. TTL ↔ Rust consistency: parsed with `oxttl` + `oxrdf` (**dev-dependencies only**). Every class/property in the TTL has a corresponding Rust enum variant, and vice @@ -704,7 +842,8 @@ hooks); unifying them into one `reconcile()` pipeline is Phase 2. storage project is rejected. Do not implement or test these in Phase 1. 7. **SHACL integration:** PurRDF evaluates positive and negative fixtures for both builtin profiles, nested shapes, local SPARQL conditions, and property paths. - No Atomic-owned constraint evaluator. The MSRV/deployment decision is explicit. + No Atomic-owned constraint evaluator. PurRDF lives in `atomic-shacl` at Rust 1.98; + the shared library crates still build on 1.90 (CI MSRV job). 8. **Custom shapes:** placing a `.ttl` in `.atomic/shapes/` causes `atomic query validate` to load and evaluate it using the same engine as builtin profiles. 9. **Conformance report:** preserve standard focus/path/source/component/severity/ @@ -761,8 +900,9 @@ Lands as one PR (or a stack merged together): the RFC allows no dual-write windo ### M4 — PurRDF SHACL integration + builtin shapes (depends on M1/M3) - Retain the standalone `purrdf-shapes = 3.0.1` spike and lockfile. -- Resolve Rust 1.98 versus Atomic's 1.90 MSRV: approved embedded upgrade or separate - validation runner. Select deployment before adding workspace dependencies. +- New `atomic-shacl` crate at `rust-version = "1.98"`, used only by `atomic-cli`; shared + library crates stay at 1.90 (§1.4 Toolchain). Release workflow to 1.98; CI MSRV job split + into library crates on 1.90 and `atomic-cli` on 1.98. - Implement the read-only workflow-to-RDF adapter and a thin PurRDF invocation layer; no custom SHACL evaluator, policy-rule compilation, or hand-coded mirror checks. - `projection-1.0.0.ttl` and `publication-1.0.0.ttl`, including local SPARQL rules @@ -782,8 +922,8 @@ SPARQL constraints silently unenforced, lost result paths, and a large dependenc footprint. Do not generalize those findings to every future OxiRS release. The [PurRDF 3.0.1 spike](./shacl-purrdf-spike-findings.md) passed the initial probes -for those failure categories. It is the provisional Phase 1 engine choice, subject -to the MSRV decision and broader regression coverage. A version number alone is +for those failure categories. It is the Phase 1 engine choice, built with the scoped +Rust 1.98 bump (§1.4 Toolchain) and subject to broader regression coverage. A version number alone is not a conformance or maturity guarantee. This plan replaces the historical direction of hand-coded workflow gates / an diff --git a/docs/knowledge-graph-phase1-proposals.md b/docs/knowledge-graph-phase1-proposals.md deleted file mode 100644 index 7f8b1001..00000000 --- a/docs/knowledge-graph-phase1-proposals.md +++ /dev/null @@ -1,239 +0,0 @@ -# Phase 1 Plan: Proposals for Approval - -> Status: **for review.** Companion to -> [`knowledge-graph-closure-phase1-plan.md`](./knowledge-graph-closure-phase1-plan.md). -> Each section below is a proposed addition to the plan. Approved sections get merged -> into the plan; this file is then deleted. - -| # | Proposal | Plan section it fills | -|---|---|---| -| A | The `LEGACY_TERMS` mapping: every old constant → its canonical replacement | §1.5 (the table is currently `&[ ... ]`) | -| B | Ontology terms the plan uses but never declares, plus one domain bug | §1.1 | -| C | `ProjectIdentity` JSON-LD context and hash construction | §1.3 | -| D | Two questions that need a decision (neither blocks M1–M3) | new "Open questions" | - ---- - -## A. `LEGACY_TERMS` mapping - -The code has the old terms (`atomic-core/src/pristine/ontology.rs`: 6 prefixes, -30 `predicate::*`, 16 `entity_type::*`, 29 `edge_kind::*`) but no mapping. The M1 -coverage test requires each old constant to appear exactly once with one of: -`One`/`OnePred` (one replacement), `SplitByEndpoint` (replacement depends on the -endpoint classes), or `Removed` (with a reason). - -Rows marked **(B)** depend on a term proposed in section B. - -### A.1 Classes (`entity_type::*` and node `kind` values) - -| Old | Replacement | -|---|---| -| `vault:Goal` / `goal` | `atomic:Goal` | -| `vault:Intent` / `intent` | `atomic:WorkIntent`, plus `FeatureIntent` / `BugIntent` / `ReviewIntent` from the canonical `kind`, and `ManagedIntent` from lifecycle status | -| `vault:Memory` / `memory` | `atomic:Memory` | -| `vault:Skill` / `skill` | `atomic:Skill` | -| `vault:ToolResult` / `tool_result` | `atomic:ToolResult` | -| `vault:File` / `file` | `atomic:File` | -| `vault:Identity` / `identity` | `atomic:Identity` | -| `vault:Concept` / `concept` | `atomic:Concept` (⊑ `skos:Concept`, per §1.1) | -| `vault:Session` / `session` | `atomic:Session` | -| `vault:SessionTurn` / `session_turn` | `atomic:SessionTurn`, plus `atomic:ManagedTurn` when `plan_id` is present | -| `vault:Todo` / `todo` | `atomic:Todo` | -| `vault:ProvenanceGraph` / `provenance` | `atomic:ProvenanceGraph` | -| `vault:Task` / `task` | `atomic:Task` | -| `vault:AcceptanceCriterion` / `acceptance_criterion` | `atomic:AcceptanceCriterion` | -| `vault:ScopeItem` / `scope` | `atomic:ScopeItem` | -| `vault:Constraint` / `constraint` | `atomic:Constraint` | -| `change` | `atomic:Change` | -| `view` | `atomic:View` | -| `module` | **Removed**: module hierarchy is derivable from paths (plan "What we skip") | -| `entity` | **Removed**: source entities stay in the syntax index (plan "What we skip") | -| *(new)* | `atomic:Attestation`, `atomic:Finding`; `atomic:Project`, `atomic:ProjectIdentity` **(B)** | - -### A.2 Predicates and edge kinds - -| Old | Endpoints | Replacement | -|---|---|---| -| `prov:hadMember` | session → turn | `atomic:hasTurn` | -| `prov:hadPlan` | turn → intent | `atomic:hasPlan` | -| `prov:hadPlan` | session → intent | **Removed**: derivable as session `hasTurn` → turn `hasPlan` | -| `prov:generated` | turn → change | `atomic:generatedChange` (⊑ `prov:generated`) | -| `prov:wasInformedBy` | turn → previous turn | `prov:wasInformedBy`, kept as PROV **(B)** | -| `prov:wasDerivedFrom` | memory → source; forked session → parent | `prov:wasDerivedFrom`, kept as PROV **(B)** | -| `prov:used` | tool result → file | `prov:used`, kept as PROV **(B)** | -| `prov:wasAssociatedWith` | tool result → goal | `atomic:linkedGoal` (PROV association is for activity → agent) | -| `prov:wasGeneratedBy` | never emitted | **Removed** | -| `atom:informedBy` | intent → memory/source | `atomic:informedBy` (⊑ `prov:wasDerivedFrom`) | -| `vault:explainedBy` | turn → provenance graph | `atomic:turnExplainedBy` (⊑ `atomic:explainedBy`) | -| `vault:hasTodo` | turn → todo | `atomic:hasTodo` | -| `BLOCKS` / `BLOCKED_BY` | intent ↔ intent | `atomic:blocks` / `atomic:blockedBy` (inverses) | -| `ASSIGNED_TO` | intent → identity | `atomic:assignedTo` | -| `LINKED_INTENT` | goal → intent | `atomic:linkedIntent` (target now resolved through the manifest) | -| `LINKED_GOAL` | intent → goal | `atomic:linkedGoal` | -| `REFERENCES` | entry → memory, entry → file | `atomic:references` | -| `HAS_LABEL` | intent → concept | `atomic:hasLabel` | -| `HAS_TASK` | intent → task | `atomic:hasTask` | -| `HAS_ACCEPTANCE_CRITERION` | intent → criterion | `atomic:hasAcceptanceCriterion` | -| `HAS_SCOPE_IN` / `HAS_SCOPE_OUT` | intent → scope item | `atomic:hasScopeIn` / `atomic:hasScopeOut` (⊑ `atomic:hasScopeItem`) | -| `HAS_CONSTRAINT` | intent → constraint | `atomic:hasConstraint` | -| `SATISFIES` | task → criterion | `atomic:satisfies` | -| `TOUCHES` | task → file | `atomic:touches` | -| `SCOPE_OUT_FILE` | intent → file | `atomic:scopeOutFile` | -| `REMEDIATES` | intent → intent | `atomic:remediates` | -| `REVIEWS` | review intent → intent | `atomic:reviews` | -| `DEPENDS_ON` | change → change | `atomic:directlyDependsOnChange` (⊑ `atomic:dependsOn`) | -| `DEPENDS_ON` | intent → intent | `atomic:dependsOnIntent` (⊑ `atomic:dependsOn`) | -| `MODIFIES` | change → file | `atomic:modifiesFile` (⊑ `atomic:modifies`) | -| `MODIFIES` | change → entity | **Removed**: code structure (plan "What we skip") | -| `ON_VIEW` | change → view | `atomic:directlyOnView`, recomputed from `VIEW_CHANGES` (today it records the working copy's desired view, not membership) | -| `ON_VIEW` | session → view | `atomic:sessionViewContext` | -| `CHILD_OF` | view → parent view | `atomic:childViewOf` | -| `VISIBLE_FROM` | never emitted | `atomic:visibleFrom`, declared as a query-time operator, never stored | -| `AUTHORED_BY` | change/goal/intent → identity | `atomic:authoredBy` (⊑ `prov:wasAttributedTo`); intents now emit it from `author`/`created_by` | -| `DEFINES`, `EXPORTS`, `INCLUDES`, `IMPORTS`, `CALLS` | code structure | **Removed**: syntax index (plan "What we skip") | -| `PART_OF` | file → module, module → module | **Removed**: derivable from paths | -| `vault:status`, `vault:priority` | literal | `atomic:status`, `atomic:priority` (see the domain fix in B.6) | -| `vault:blocks`, `vault:blockedBy`, `vault:assignedTo`, `vault:linkedIntent`, `vault:linkedGoal`, `vault:references`, `vault:hasLabel` | never emitted; duplicates of the uppercase kinds | **Removed**: the uppercase twins are mapped above | -| `vault:appliesTo` | never emitted | **Removed** | -| `dc:type`, `dc:creator`, `dc:date`, `dc:description` | never emitted | **Removed** (creator → `authoredBy`; dates and descriptions come from typed source fields, see B.5) | -| `skos:broader`, `skos:narrower`, `skos:subject`, `skos:related` | never emitted | **Removed** | -| `rdf:type` (as an edge) | never emitted | **Removed**: types are stored in `kg_types` (§1.6) | - -### A.3 Prefix constants - -| Old | Replacement | -|---|---| -| `prefix::{ATOM, PROV, SKOS, DC, VAULT, RDF}` | **Removed**: replaced by the `PREFIXES` table in `kg::ontology` (§1.2) | - ---- - -## B. Ontology terms to add - -The plan's §1.1 tables reference these, or the producers in §1.6 emit them, but the -ontology doesn't declare them. Without them the TTL ↔ Rust consistency test and the -`LEGACY_TERMS` coverage test (which requires every target to exist in the TTL) would fail. - -### B.1 Parents and inverses that are referenced but not declared - -| Term | Kind | Notes | -|---|---|---| -| `atomic:dependsOn` | object property, `owl:TransitiveProperty` | Parent of `directlyDependsOnChange` / `dependsOnIntent`; never materialized in Phase 1 | -| `atomic:modifies` | object property | Parent of `modifiesFile` | -| `atomic:hasScopeItem` | object property | Parent of `hasScopeIn` / `hasScopeOut` | -| `atomic:ancestorView` | object property, `owl:TransitiveProperty` | Mentioned in §1.1; never materialized | -| `atomic:turnOfSession`, `atomic:governsTurn`, `atomic:generatedBy`, `atomic:blockedBy`, `atomic:reviewedBy` | object properties, `owl:inverseOf` their partners | Named in the "Inverse" column but have no rows; `generatedBy` is used by both property chains | - -### B.2 PROV terms the producers keep - -Declared as reused terms (`rdfs:isDefinedBy `), not redefined: -`prov:wasInformedBy` (turn → previous turn), `prov:wasDerivedFrom` (memory → source, -forked session → parent), `prov:used` (tool result → file), and the two parents the plan -already uses, `prov:generated` and `prov:wasAttributedTo`. - -### B.3 Attestation verification - -| Term | Kind | -|---|---| -| `atomic:verificationResult` | object property, `Attestation` → `atomic:VerificationStatus` | -| `atomic:VerificationStatus` | class | -| `atomic:Verified`, `atomic:Unverifiable`, `atomic:Failed` | individuals of `VerificationStatus` | - -Written only by the cryptographic verifier, never inferred. MAC-signed attestations -record `Unverifiable`. - -### B.4 Project identity - -| Term | Kind | -|---|---| -| `atomic:Project` | class (the `urn:atomic:project:` node) | -| `atomic:ProjectIdentity` | class (the signed identity object) | -| `atomic:hasIdentity` | object property, `Project` → `ProjectIdentity` | -| `atomic:projectId` | datatype property, `xsd:string` (52-character uppercase base32) | -| `atomic:schemaVersion` | datatype property, `xsd:nonNegativeInteger` | -| `atomic:issuer` | object property, `ProjectIdentity` → `Identity` | - -Creation time uses `dcterms:created` (B.5). - -### B.5 Literals the producers carry today - -`rdfs:label` (labels), `dcterms:description` (summaries), `dcterms:created` / -`dcterms:modified` (timestamps), and `atomic:turnNumber` (`xsd:nonNegativeInteger`). - -### B.6 Domain bug: `atomic:status` and `atomic:priority` - -Both are declared with domain `atomic:WorkIntent`, but tasks, acceptance criteria, -todos, and sessions also carry a status (and todos a priority). An OWL domain is an -inference rule, not a check, so every task, criterion, todo, and session would be -inferred to be a `WorkIntent`, and intent shapes would then run against them. This is -the same problem that was fixed for `explainedBy`. - -**Proposal:** remove the domain from both properties. They are literals, so no -subproperties are needed; shapes constrain the allowed values per class. - ---- - -## C. `ProjectIdentity` context and hash - -**Problem.** §1.3's identity object uses `"@context": "https://atomic.dev/ns/ctx.jsonld"`. -That context defines the old `atom:` prefix but not `atomic:`, so `"atomic:ProjectIdentity"` -does not expand to `https://atomic.dev/ns/workflow#ProjectIdentity`; a JSON-LD processor -reads `atomic:` as an unknown URI scheme. The hash, `blake3(JCS(...))`, has no type -prefix, so it isn't separated from other object types. - -**Proposal.** `ProjectIdentity` is new and has no existing signed copies, so fixing it now -costs nothing: - -```jsonld -{ - "@context": { "atomic": "https://atomic.dev/ns/workflow#", - "dcterms": "http://purl.org/dc/terms/" }, - "@type": "atomic:ProjectIdentity", - "atomic:projectId": "ABCD...EFGH", - "dcterms:created": "2026-01-15T09:00:00Z", - "atomic:schemaVersion": 1, - "atomic:issuer": "did:atomic:..." -} -``` - -- The context is inline, so the hashed bytes include the term definitions and their - meaning can never drift. -- Content hash = `blake3("atomic.project-identity.v1" ‖ 0x00 ‖ JCS(object))`. -- Existing object hashes (changes, attestations, provenance, canonical intents and - memories) are unchanged. - ---- - -## D. Open questions (need a decision; neither blocks M1–M3) - -1. **PurRDF needs Rust 1.98; Atomic declares 1.90** (`Cargo.toml:27`). Lee is happy to - upgrade to 1.98 as long as nothing else breaks. Findings (2026-10-14): - - - **This repo is clean at 1.98.** With `rust-version = "1.98"`, CI's - `cargo clippy --workspace -- -D warnings` passes with no warnings on Rust 1.98.1, - including the clippy lints that only switch on above a given MSRV. - - **Where 1.90 is pinned:** `Cargo.toml:27` (`rust-version`), the CI MSRV job - (`.github/workflows/ci.yml:136`), and the **release build** - (`.github/workflows/release.yml:129`, so shipped binaries are compiled with 1.90). - The other CI jobs use `stable`. - - **What a workspace-wide bump would break.** Every crate inherits the workspace - `rust-version`, and Cargo refuses to build a dependency whose `rust-version` is newer - than the compiler. Sibling repos that build this repo's crates by path: - - | Repo | Uses | Pinned Rust | Effect of a workspace-wide 1.98 bump | - |---|---|---|---| - | `atomic-storage` (owned by another effort) | `atomic-core`, `atomic-canonical`, `atomic-identity`, `atomic-objects`, `atomic-repository` | `rust-version = "1.94"`, Docker `rust:1.94-slim-bookworm` (`Dockerfile:14`, `Dockerfile.stress:11`) | **Docker builds fail** until both images and its `rust-version` move to 1.98 | - | `basecamp/sherpa-server` | `atomic-identity` | `rust-version = "1.94"` | No longer meets its declared minimum | - | `atomic-enterprise` | `atomic-agent`, `atomic-core`, `atomic-config`, `atomic-identity`, `atomic-repository`, `atomic-remote` | none | Needs a ≥ 1.98 compiler | - - **Options:** - - | Option | What changes | Breaks others? | - |---|---|---| - | **(a) Scoped bump** (recommended) | PurRDF lives in its own crate (working name `atomic-shacl`) with `rust-version = "1.98"`, depended on only by `atomic-cli` (and libatomic, if the daemon validates). Shared library crates stay at 1.90, and **the SHACL crate must not become a dependency of any of them** (`atomic-core`, `atomic-repository`, `atomic-remote`, `atomic-agent`, …); `atomic query validate` wires it in at the CLI layer. The release workflow moves to 1.98; the CI MSRV job checks the library crates on 1.90 and the CLI on 1.98. This is also the "dedicated package" shape already anticipated for the validator. | **No.** None of the three depends on `atomic-cli` or `libatomic`, directly or through `atomic-repository`/`atomic-remote`/`atomic-agent` (checked) | - | (b) Workspace-wide bump | `Cargo.toml:27`, `ci.yml:136`, `release.yml:129` → 1.98 | **Yes**, until `atomic-storage` updates its Docker images and `rust-version`, and `sherpa-server` its `rust-version` (one-line changes each, owned by others) | - - Blocks M4 only. -2. ~~**MAC-signed attestations vs. the publication profile.**~~ **Decided (Lee, - 2026-10-14):** this is Atomic, so publication requires signing with the atomic - identity DID (`did:atomic:…`). MAC-signed attestations record `Unverifiable` and never - satisfy the publication profile. Applied to plan §1.4 and §1.6. From 005b293d5be68c78cdb9cb68317100e6df4ae9d6 Mon Sep 17 00:00:00 2001 From: Bradley Hilton Date: Thu, 15 Oct 2026 07:26:56 -0500 Subject: [PATCH 10/19] docs: close Phase 1 plan gaps found by cold-read reviews Three cold-read reviews by agents without prior context; the third found no blockers. Changes: - Identity: every repository mints (Created for init/git import, Upgrade otherwise, replaceable later by transport) through one ensure_project_identity() called from init/import, full writable opens, and every KG write path (covers clone and git import, which enrich without reopening). IRI schemes for every node kind; identity object storage, signing, and crate placement; CLI renamed to `atomic repo identity`. - Shortcuts: producers assert implements, changeExplainedBy, and attestedBy in Phase 1; per-change findings explain missing ones and the projection shape accepts them. attestedBy is the inverse of coversChange. - Attestations: always link to their session (attestsSession); link to a turn only on a unique non-empty match. Verification looks up the signer's public key by DID from the local identity store, then a signer-key cache filled from Atomic storage; fetched keys must match the DID fingerprint. Notes the storage fix needed: by-key must match did:atomic: fingerprints, not decode them as raw keys. - Ontology: linkedGoal domain and informedBy range removed; term enums cover external terms and datatype ranges; RL allow-list test. - Specs: which opens mint/rebuild, producers skip writes on an old-generation KG, legacy URN resolution, ManagedIntent, encoding set, metadata-to-RDF mapping, query callers argument, whole-view triage parsing, lint scope, lockfile checks, corrected references. Co-Authored-By: Claude Opus 5.5 --- docs/knowledge-graph-closure-phase1-plan.md | 462 +++++++++++++------- 1 file changed, 314 insertions(+), 148 deletions(-) diff --git a/docs/knowledge-graph-closure-phase1-plan.md b/docs/knowledge-graph-closure-phase1-plan.md index d0dfea2b..25ee2d81 100644 --- a/docs/knowledge-graph-closure-phase1-plan.md +++ b/docs/knowledge-graph-closure-phase1-plan.md @@ -10,7 +10,7 @@ Phase 1 is about **clean up, defining a common language, and using it**. It ship things that every downstream phase depends on: 1. **`atomic:` namespace** — a canonical workflow ontology (`.ttl` + Rust registry). -2. **`ProjectId`** — a stable, unique project identity minted at `atomic init` and cast +2. **`ProjectId`** — a stable, unique project identity minted for every repository (§1.3) and cast into IRIs so that future cross-project SPARQL queries have a canonical namespace per repository. 3. **SHACL integration using PurRDF** — adapt workflow data to RDF and evaluate @@ -98,7 +98,7 @@ projector knows its endpoint classes at emit time). - Change node IDs are **12-char base32 prefixes** in three places (`vault_kg_enrich.rs:1089 truncate_hash`, `session_kg.rs:46 change_node_id`, - `vault_triples.rs:1268 rdf_target_to_kg_id`). Full hash survives only in metadata. + `vault_triples.rs:1246 rdf_target_to_kg_id`, the `urn:atomic:change:` branch at :1268). Full hash survives only in metadata. - `identity:` IDs are display names (`identity:{author.name}` vs `identity:{fm.developer}`); intents write `author`/`created_by` which the KG ignores, so intents have no author edge. - `todo:X` from `urn:atomic:todo:X` collides with the legacy session-global todo form. @@ -136,12 +136,18 @@ projector knows its endpoint classes at emit time). - `atomic-cli/src/commands/query/mod.rs`: `callers` (`to_uppercase()=="CALLS"` :1231,:1255), HTML `EDGE_COLORS` (:748-755), `ask` system prompt (:1971-1976), text output. -- `atomic-repository/src/triage/report.rs` (`kind_is` at :35, case-insensitive, ~12 sites), +- `atomic-repository/src/triage/report.rs` (`kind_is` at :35, case-insensitive, ~12 sites; it + also reads `IMPORTS` at :200, which no producer emits), `atomic-repository/src/repository/triage.rs:231,305`. -- `atomic-repository/src/repository/vault_intent.rs:1770,1776` (`atom:informedBy`). +- `atomic-repository/src/repository/vault_intent.rs:1770,1776` match `atom:informedBy`, in + test code only. +- Code that parses the **shape** of KG IDs (`entity:`, `file:`, `change:` prefixes) rather + than kind strings, which breaks once IDs become IRIs: `extract_path_from_id`, + `path_tier`, `kg_label`, `edge_is_owned_by_vault_entry`, `delete_intent_child_nodes` + (`vault_triples.rs:573-1222`), and triage's `file:` joins (`triage/report.rs:162`). - `atomic-cli/src/commands/vault/context.rs:470-473` (filters `node.kind == "memory"`). - libatomic daemon copies: `services_agent.rs:131,3372-3387`, `services_query.rs:1001-1038,1059`. -- Wire: `libatomic/proto/.../knowledge_messages.proto:123` `KGEdge.relation` (free string). +- Wire: `libatomic/proto/.../knowledge_messages.proto:126` `KGEdge.relation` (free string). - Out-of-repo clients (`../noname`, `../the-hive`) also match kind strings; the release is announced as breaking, so they are not updated as part of this plan. - ~110 test assertion sites (largest: `vault_triples.rs` ~40, `ontology.rs` ~31, @@ -211,7 +217,7 @@ Version IRI: `https://atomic.dev/ns/workflow/1.0.0` | `atomic:hasTurn` | `atomic:Session` | `atomic:SessionTurn` | `atomic:turnOfSession` | — | | `atomic:hasPlan` | `atomic:SessionTurn` | `atomic:WorkIntent` | `atomic:governsTurn` | — | | `atomic:generatedChange` | `atomic:SessionTurn` | `atomic:Change` | `atomic:generatedBy` | `prov:generated` | -| `atomic:informedBy` | `atomic:WorkIntent` | `atomic:Memory` | — | `prov:wasDerivedFrom` | +| `atomic:informedBy` | `atomic:WorkIntent` | — | — | `prov:wasDerivedFrom`; no range: intents link to memories and other sources (`vault_triples.rs:963` accepts any `urn:` target) | | `atomic:explainedBy` | — | `atomic:ProvenanceGraph` | — | — | | `atomic:turnExplainedBy` | `atomic:SessionTurn` | `atomic:ProvenanceGraph` | — | `atomic:explainedBy` | | `atomic:changeExplainedBy` | `atomic:Change` | `atomic:ProvenanceGraph` | — | `atomic:explainedBy` | @@ -219,7 +225,7 @@ Version IRI: `https://atomic.dev/ns/workflow/1.0.0` | `atomic:blocks` | `atomic:WorkIntent` | `atomic:WorkIntent` | `atomic:blockedBy` | — | | `atomic:assignedTo` | `atomic:WorkIntent` | `atomic:Identity` | — | — | | `atomic:linkedIntent` | `atomic:Goal` | `atomic:WorkIntent` | — | — | -| `atomic:linkedGoal` | `atomic:WorkIntent` | `atomic:Goal` | — | — | +| `atomic:linkedGoal` | — | `atomic:Goal` | — | No domain: both intents and tool results link to goals (legacy `prov:wasAssociatedWith` maps here) | | `atomic:references` | `owl:Thing` | `owl:Thing` | — | — | | `atomic:hasLabel` | `atomic:WorkIntent` | `skos:Concept` | — | — | | `atomic:hasTask` | `atomic:WorkIntent` | `atomic:Task` | — | — | @@ -240,12 +246,13 @@ Version IRI: `https://atomic.dev/ns/workflow/1.0.0` | `atomic:childViewOf` | `atomic:View` | `atomic:View` | — | — | | `atomic:visibleFrom` | — | — | — | Query-time operator | | `atomic:authoredBy` | `owl:Thing` | `atomic:Identity` | — | `prov:wasAttributedTo` | -| `atomic:coversChange` | `atomic:Attestation` | `atomic:Change` | — | — | -| `atomic:attests` | `atomic:Attestation` | `atomic:SessionTurn` | — | — | +| `atomic:coversChange` | `atomic:Attestation` | `atomic:Change` | `atomic:attestedBy` | — | +| `atomic:attests` | `atomic:Attestation` | `atomic:SessionTurn` | — | Only when the turn mapping is unique (§1.6) | +| `atomic:attestsSession` | `atomic:Attestation` | `atomic:Session` | — | Always, from the attestation's `session_id` | | `atomic:signedBy` | `atomic:Attestation` | `atomic:Identity` | — | — | | `atomic:previousAttestation` | `atomic:Attestation` | `atomic:Attestation` | — | — | -| `atomic:implements` | `atomic:Change` | `atomic:WorkIntent` | — | — | -| `atomic:attestedBy` | `atomic:Change` | `atomic:Attestation` | — | — | +| `atomic:implements` | `atomic:Change` | `atomic:WorkIntent` | — | Asserted by the session-ledger producer in Phase 1 | +| `atomic:attestedBy` | `atomic:Change` | `atomic:Attestation` | `atomic:coversChange` | Asserted by the attestation producer in Phase 1 | | `atomic:aboutResource` | `atomic:Finding` | `owl:Thing` | — | — | | `atomic:dependsOn` | — | — | — | `owl:TransitiveProperty`; parent of `directlyDependsOnChange` / `dependsOnIntent` | | `atomic:modifies` | `atomic:Change` | — | — | Parent of `modifiesFile` | @@ -296,7 +303,9 @@ constrain the allowed values per class. These chains use `atomic:generatedBy` (the inverse of `atomic:generatedChange`), so the intermediate node is consistently `SessionTurn`. -`atomic:explainedBy` has no domain of its own. In OWL a domain is an inference rule, not a check: a `SessionTurn` domain on the property that the chain derives for Changes would infer every Change to be a `SessionTurn`, and the publication shapes that target `SessionTurn` would then run against Changes. The two subproperties carry the domains instead; queries on `atomic:explainedBy` still see both kinds of edge. In Phase 1 the chains are **declarative only** — the ontology records the intended meaning so that enterprise SHACL shapes and documentation see a complete model. The OWL 2 RL reasoner in Phase 2 will materialize them from the TTL. +`atomic:explainedBy` has no domain of its own. In OWL a domain is an inference rule, not a check: a `SessionTurn` domain on the property that the chain derives for Changes would infer every Change to be a `SessionTurn`, and the publication shapes that target `SessionTurn` would then run against Changes. The two subproperties carry the domains instead; queries on `atomic:explainedBy` still see both kinds of edge. + +**Shortcut edges in Phase 1.** The three shortcuts the projection profile requires are **asserted directly by producers** (§1.6), not inferred: the session-ledger producer emits `implements` (from the turn's `plan_id`) and `changeExplainedBy` (from the turn's provenance graph) for each change the turn generated, and the attestation producer emits `attestedBy` alongside `coversChange`. This follows the RFC, which puts the required direct shortcut edges in Phase 1. The chains and the `coversChange`/`attestedBy` inverse document the same meaning; the Phase 2 OWL 2 RL reasoner derives them from the TTL and checks that they agree with the asserted shortcuts. **What the ontology does NOT contain:** @@ -310,20 +319,31 @@ describe code structure inside files; that is the syntax index's job. Rewrite `atomic-core/src/pristine/ontology.rs`: ```rust +// Every class and predicate the KG writes, including reused external terms +// (prov:wasInformedBy, prov:wasDerivedFrom, prov:used, prov:generated, +// prov:wasAttributedTo, rdfs:label, dcterms:description/created/modified). +// External variants are not declared in the atomic: namespace; is_external() is true. pub enum Class { WorkIntent, FeatureIntent, ... } -pub enum Predicate { HasTurn, HasPlan, ... } +pub enum Predicate { HasTurn, HasPlan, ..., ProvWasInformedBy, ProvUsed, ... } + +// A class's parent may be an atomic class or an external one (owl:Thing, skos:Concept). +pub enum ClassRef { Atomic(Class), External(&'static str) } + +// A property's range: an atomic class, an external class IRI, or an XSD datatype. +pub enum Range { Class(Class), External(&'static str), Datatype(&'static str) } impl Class { pub fn iri(&self) -> &'static str; pub fn curie(&self) -> &'static str; - pub fn parents(&self) -> &'static [Class]; + pub fn parents(&self) -> &'static [ClassRef]; } impl Predicate { pub fn iri(&self) -> &'static str; pub fn curie(&self) -> &'static str; + pub fn is_external(&self) -> bool; pub fn domain(&self) -> Option; - pub fn range(&self) -> Option; + pub fn range(&self) -> Option; pub fn inverse(&self) -> Option; pub fn super_predicate(&self) -> Option; } @@ -347,25 +367,61 @@ source and the TTL cannot drift. **Design goal:** a single stable identifier per project that survives views, inserts, unrecords, and clone/pull/push, and becomes the `` component in -project-qualified IRIs. **Every clone of a project carries the origin's identity**; only -an explicit fork (later phase) mints a new one. +project-qualified IRIs. The long-term target is that **every clone of a project carries +the origin's identity** and only an explicit fork (later phase) mints a new one. Phase 1 +gets there in two steps: every repository mints its own identity now, and the later +transport work reconciles clones with their origin (see Transport). **Format:** -- 256-bit random ID generated by the `atomic init` command (not by - `Repository::init`; see Transport). +- 256-bit random ID (`OsRng`). - Canonical encoding: uppercase RFC 4648 base32 (52 characters, no padding), same alphabet as `Hash::to_base32`. - Display form: the base32 string. - URN prefix: `urn:atomic:project:`. +**When an identity is minted (Phase 1):** every repository gets one, because the §1.6 KG +cutover needs a `` for every project-qualified IRI. +- One function, `Repository::ensure_project_identity(reason)` (in `atomic-repository`), + mints when `PROJECT_IDENTITY` is empty and is a no-op otherwise. It is called from: + - the `atomic init` command and `atomic git import` (which creates a new project), right + after the repository is created, with `mint_reason = Created`; + - full writable opens, with `mint_reason = Upgrade` (existing repositories after + upgrade). These are the opens that already run reconcile/recovery: `Repository::open` + and `open_with_budget` with a full budget (`repository/mod.rs:526, 540`). The other + constructors (`open_existing`, `open_for_workspace_transaction`, `open_with_pristine`, + the `*_wait` variants, `MetadataOnly` budgets, sandbox opens, `open_for_native_repair` + at :872, and the read-only opens) neither mint nor rebuild; + - the start of every KG write path (the §1.6 producers, including `kg_enrich_from_vcs`), + with `mint_reason = Upgrade`. This covers `atomic clone` and Git bootstrap + (`clone --into-existing`), which call `Repository::init` and then enrich the KG on the + same handle without a writable reopen (`clone/command.rs:992, 1333`), and the same + pattern in `git import` (`import.rs:832`, enriching at :979 and :1114). Repository + entry points that write session turns (`publish_session_checkpoint`, + `save_provenance_graph`, `rebuild_session_index` in `repository/changes.rs`) call it + before reaching the session-ledger producer, which runs inside an `atomic-core` + transaction and returns a `MissingProjectIdentity` error if the table is still empty. +- On a writable open, minting runs **before** the §1.6 one-time KG rebuild, which needs the + ``. +- `Repository::init` itself does not mint; callers that create a new project call + `ensure_project_identity(Created)` explicitly. `atomic git import` does so only when it + creates the repository; when it opens an existing one (`import.rs:739-747`) the existing + identity is kept. +- Consequence, accepted for Phase 1: until transport lands, each clone of a project has its + own identity. `mint_reason = Upgrade` marks those identities as replaceable by the + later transport work (see Transport). + **Storage:** - Dedicated table `PROJECT_IDENTITY` in pristine (not `VIEW_CHANGES`, therefore - view-independent and immune to insert/unrecord). -- Schema: `current_hash` → the content-addressed object hash of the signed identity - object; `created_at` → timestamp. A `previous` chain column is reserved for rotation - (deferred), so the schema is forward-compatible. + view-independent and immune to insert/unrecord), holding a single current record: + - `object`: the identity object's stored bytes (canonical JSON, including `proof` when + signed); + - `hash`: the identity object's content hash (below); + - `mint_reason`: `Created` or `Upgrade`; + - `created_at`: timestamp; + - `previous`: reserved for rotation (deferred), so the schema is forward-compatible. +- Register the table in `schema.rs::visit_tables` (a test fails if a table is missing there). -**Identity object (content-addressed, signed JSON-LD):** +**Identity object (content-addressed, optionally signed JSON-LD):** ```jsonld { @@ -375,21 +431,32 @@ an explicit fork (later phase) mints a new one. "atomic:projectId": "ABCD...EFGH", "dcterms:created": "2026-01-15T09:00:00Z", "atomic:schemaVersion": 1, - "atomic:issuer": "did:atomic:..." + "atomic:issuer": { "@id": "did:atomic:..." } } ``` -- `atomic:projectId` is the 52-character uppercase base32 ID. `atomic:issuer` is - optional in Phase 1 and may be unknown. +- `atomic:projectId` is the 52-character uppercase base32 ID. `atomic:issuer` is an IRI + (`{"@id": …}`), since it is an object property; it is omitted when no signing identity is + configured. - The `@context` is **inline**, so the hashed bytes include the term definitions and their meaning can never drift. The existing `https://atomic.dev/ns/ctx.jsonld` context is not used: it defines the old `atom:` prefix but not `atomic:`, so `atomic:` terms would not expand to the workflow namespace. -- Content hash = `blake3("atomic.project-identity.v1" ‖ 0x00 ‖ JCS(object))`. The type - prefix keeps identity hashes distinct from every other object type. Existing object - hashes (changes, attestations, provenance, canonical intents and memories) are unchanged. -- Signed by the initializing identity if available; unsigned is allowed in Phase 1 - (the object schema reserves the `proof` field). +- Content hash = `blake3("atomic.project-identity.v1" ‖ 0x00 ‖ JCS(object without proof))`. + The type prefix keeps identity hashes distinct from every other object type. Existing + object hashes (changes, attestations, provenance, canonical intents and memories) are + unchanged. +- **Signing:** when the initializing user has an atomic identity (`atomic-identity`), sign + with the same Data Integrity cryptosuite `atomic-canonical/src/proof.rs` uses + (`eddsa-jcs-2022`): the proof covers the object without `proof`, and is then attached as + `proof`. `atomic:issuer` is that identity's DID. Only `Created` mints (an interactive + `atomic init` / `atomic git import`) are signed; `Upgrade` mints happen during ordinary + opens, hooks, and the daemon, where no key can be unlocked, so they are stored unsigned. + Without a configured identity, `Created` mints are also unsigned (allowed in Phase 1). +- **Crate placement:** `ProjectId`, the IRI functions, and the table live in `atomic-core` + (`atomic_core::kg`). Building, hashing, and signing the identity object lives in + `atomic-repository`, because JCS and the `eddsa-jcs-2022` proof are in + `atomic-canonical`, which `atomic-core` does not depend on. **Why not `.vault/` or view metadata?** A `.vault/` file is a change-tracked artifact. It can be absent on some views, @@ -397,39 +464,28 @@ in conflict, or unrecorded. Project identity must be the same on every view and survive all graph mutations. The only place that guarantees this is a repository-level pristine table outside the view system. -**Transport:** +**Transport (later; not Phase 1):** > **Phase 1 implementation boundary.** `atomic clone` is HTTP-only (plus a Git bootstrap > path that never contacts storage), so identity transport over clone, pull and push all > depends on server support in `atomic-storage`, which is owned by another effort. In -> Phase 1, implement only the local pieces: minting in the `atomic init` command (not in -> `Repository::init`), the `PROJECT_IDENTITY` table, the upgrade rule, and the CLI. **Do -> not change push, pull, clone, `atomic-remote`, or the daemon's sync code yet.** The rules -> below are the target behavior that the later transport work implements. - -Clones keep the origin's identity, and pulls never change it. Rules: - -- **Where minting happens.** Only the `atomic init` command mints a new identity. - `Repository::init` does not, because `atomic clone` calls it first - (`clone/command.rs:992`) and then pulls; minting there would give every clone a fresh - identity and discard the origin's. -- **Clone** (from Atomic storage or any other remote): the remote supplies its - identity object as a repository-level sidecar (not a change). The new repository - verifies it (hash, and signature when present) and stores it in `PROJECT_IDENTITY`. - The clone is the same project as its origin. -- **Pull**: the local identity never changes. If the remote's identity matches, nothing - happens. If the local table is empty (a repository created before identities - existed), the pulled identity is adopted. If it differs, the pull still succeeds - (changes are global objects and may move between projects) but reports that the - remote belongs to a different project. -- **Push**: the first push to a storage project with no bound identity binds it (the +> Phase 1, implement only the local pieces: minting, the `PROJECT_IDENTITY` table, and the +> CLI. **Do not change push, pull, clone, `atomic-remote`, or the daemon's sync code.** The +> rules below record the target behavior for the later transport work. + +- **Clone:** the remote supplies its identity object as a repository-level object (not a + change). The new repository verifies it (hash, and signature when present) and stores it + with `mint_reason = Created`. The clone is the same project as its origin. +- **Reconciling Phase 1 identities:** a local identity with `mint_reason = Upgrade` is + replaced by the remote's identity on the first pull from a remote that has one, and the + KG is rebuilt with the new `` (cheap: the KG is entirely derived). +- **Pull:** an identity with `mint_reason = Created` never changes. If the remote's + differs, the pull still succeeds (changes are global objects and may move between + projects) but reports that the remote belongs to a different project. +- **Push:** the first push to a storage project with no bound identity binds it (the server records the identity hash against its `ProjectInfo.id`). Later pushes carrying a different identity are rejected, so two unrelated repositories cannot claim the same storage project. -- **Upgrading existing repositories:** a repository with no remote mints on its first - writable open. A repository with a remote does **not** auto-mint, since that would - diverge from its origin; it adopts the identity on its next pull. The owner mints once - with `atomic project identity init` and pushes, and every clone picks it up. - **Atomic storage work:** the remote protocol (`atomic-remote`) and the storage server gain an identity object transfer and the push-time binding check. The current push/pull contract excludes sidecars (`atomic-remote/src/sync.rs:476`), so this is a @@ -437,6 +493,32 @@ Clones keep the origin's identity, and pulls never change it. Rules: - No fork command in Phase 1; fork semantics (new identity with `prov:wasDerivedFrom` pointing at origin) come later. +**IRI schemes (every node kind the §1.6 producers emit):** + +| Node kind | IRI | Notes | +|---|---|---| +| Project | `urn:atomic:project:` | | +| Project identity object | `urn:atomic:project-identity:blake3-256:` | Its prefixed content hash | +| Change | `urn:atomic:change:blake3-256:` | **Global, not project-qualified**: the full existing change hash (`Change::hash`). Replaces the 12-character IDs (`truncate_hash`, `change_node_id`, `rdf_target_to_kg_id`). Existing change hashes are used as-is; they are not re-hashed with a type prefix | +| Provenance graph | `urn:atomic:provenance:blake3-256:` | Global; existing sidecar hash | +| Attestation | `urn:atomic:attestation:blake3-256:` | Global; existing `.attest` hash | +| Intent (incl. review) | `urn:atomic:project::intent:` | | +| Acceptance criterion / task / scope item / constraint | `urn:atomic:project::intent::{ac,task,scope,constraint}:` | `` is the child's canonical ID, uppercased (today's `ac:{ID}` etc.) | +| Goal / memory / skill | `urn:atomic:project::{goal,memory,skill}:` | | +| Tool result | `urn:atomic:project::tool-result:` | | +| Concept | `urn:atomic:project::concept:` | | +| Session / turn / todo | `urn:atomic:project::session:`, `…:session::turn:`, `…:session::todo:` | Todos are always session-qualified. A legacy global `todo:` whose session can't be determined gets an `UnresolvedLegacyTodo` finding, not a guessed IRI (fixes the `todo:X` collision) | +| View | `urn:atomic:project::view:` | | +| File | `urn:atomic:project::file:` | | +| Identity | the DID (`did:atomic:…`) when known; otherwise `urn:atomic:project::identity:` | Display-name identities stay project-scoped and are never merged with DIDs | +| Finding | `urn:atomic:project::finding::` | | + +`` segments are UTF-8 percent-encoded with uppercase hex. Only RFC 3986 unreserved +characters (`A–Z a–z 0–9 - . _ ~`) stay literal, plus `/` in file paths; everything else, +including `:`, `#`, `%`, spaces, and non-ASCII, is encoded. `` for goals, +memories, and skills is the entry's vault file stem, as today's `goal:`/`memory:`/`skill:` +IDs use. + **API:** ```rust @@ -446,23 +528,18 @@ pub struct ProjectId([u8; 32]); impl ProjectId { pub fn generate() -> Self; // OsRng → 32 bytes - pub fn from_base32(s: &str) -> Result; + pub fn from_base32(s: &str) -> Result; // strict pub fn to_base32(&self) -> String; // 52-char uppercase, no padding pub fn to_urn(&self) -> String; // urn:atomic:project: } -// atomic-core/src/kg/iri.rs -pub fn mint_intent_iri(pid: &ProjectId, ulid: &str) -> String; -pub fn mint_goal_iri(pid: &ProjectId, encoded_name: &str) -> String; -pub fn mint_memory_iri(pid: &ProjectId, encoded_name: &str) -> String; -pub fn mint_session_iri(pid: &ProjectId, sid: &str) -> String; -pub fn mint_view_iri(pid: &ProjectId, view_name: &str) -> String; -pub fn mint_file_iri(pid: &ProjectId, path: &str) -> String; -// ... and parser functions for the reverse direction +// atomic-core/src/kg/iri.rs — one mint_* and one parse_* function per row of the +// IRI table above (project-qualified kinds take &ProjectId; global kinds take a Hash). ``` -All minting functions percent-encode path/ name segments per RFC 3987 and guarantee -round-trip stability. +All minting functions percent-encode path/name segments per RFC 3987 and guarantee +round-trip stability. Property tests cover paths with spaces, `#`, `:`, non-ASCII, +and the empty string. **Identifier casing.** Every identifier inside a URN is uppercase: base32 hashes and project IDs (as `Hash::to_base32` emits) and intent ULIDs (uppercase Crockford, as @@ -474,36 +551,22 @@ lenient prefix lookup in CLI arguments is unaffected. **Old-layout intents.** Intents created before the ULID layout live at `intents////intent.md` (e.g. `ATOM-2` in this repo's vault) and carry no ULID. The projector resolves their identity through the vault manifest; if none is -recorded, it emits a finding instead of inventing an ID. Property tests cover paths with spaces, `#`, `:`, non-ASCII, -and the empty string. +recorded, it emits a `MissingIntentId` finding instead of inventing an ID. -**CLI:** +**CLI:** `atomic repo identity` (`atomic project` already exists for remote server +projects, so the identity commands live under a new `repo` group). ```bash -# Show the project identity -atomic project identity show +atomic repo identity show # → project: ABCDEFGH...12345678 # urn: urn:atomic:project:ABCDEFGH...12345678 # created: 2026-01-15T09:00:00Z +# minted: created | upgrade +# signed by: did:atomic:... | unsigned ``` -**Phase 1 IRI usage:** - -The projectors (Phase 2) will emit IRIs like: - -``` -urn:atomic:project::intent: -urn:atomic:project::goal: -urn:atomic:project::memory: -urn:atomic:project::session: -urn:atomic:project::view: -urn:atomic:project::file: -``` - -In Phase 1 we only mint the `ProjectId` and provide the IRI functions; no -projector actually emits project-qualified IRIs yet. This is deliberate: -projectors come in Phase 2, but the identity must exist first so that Phase 2 -can use it without a second migration. +`atomic repo identity show --json` emits the same fields. There is no `init` subcommand in +Phase 1: minting is automatic (above). ### 1.4 SHACL integration (PurRDF, existing engine) @@ -543,21 +606,30 @@ older pins (`atomic-storage`: `rust:1.94` Docker images; `basecamp/sherpa-server `rust-version = "1.94"`). A workspace-wide bump would break them. So: - PurRDF lives in its **own crate** (working name `atomic-shacl`) with - `rust-version = "1.98"`, depended on only by `atomic-cli` (and libatomic, if the daemon - validates). + `rust-version = "1.98"`, depended on only by `atomic-cli`. Validation runs in the CLI + process in Phase 1, so libatomic does not depend on it. - The shared library crates (`atomic-core`, `atomic-repository`, `atomic-remote`, `atomic-agent`, `atomic-canonical`, `atomic-identity`, `atomic-objects`, `atomic-config`, …) stay at 1.90 and **must not depend on the SHACL crate**. `atomic query validate` wires it in at the CLI layer. -- The release workflow (`.github/workflows/release.yml:129`) moves to 1.98. The CI MSRV +- The release workflow (`.github/workflows/release.yml:128`) moves to 1.98. The CI MSRV job (`ci.yml:136`) checks the library crates on 1.90 and `atomic-cli` on 1.98. - Do not raise the workspace `rust-version`. There is no hand-rolled fallback validator. **Builtin shape graphs:** - `atomic-core/ontology/shapes/projection-1.0.0.ttl` — projection profile: - - every generated change has `atomic:implements`, `atomic:changeExplainedBy`, and - `atomic:attestedBy` shortcuts; + - every generated change (targeted with `sh:targetObjectsOf atomic:generatedChange`; no + producer emits the inverse `generatedBy`) has `atomic:implements`, + `atomic:changeExplainedBy`, and `atomic:attestedBy` shortcuts, **or** a finding that + explains each missing one (`MissingPlanId` for `implements`, `MissingProvenance` for + `changeExplainedBy`, `MissingAttestation` for `attestedBy`). A change with a matching finding + conforms; one with neither is a violation. Changes with no generating turn (human + changes) are not targeted. **Finding linkage:** all three missing-shortcut findings are + emitted **per change**, with `atomic:aboutResource` pointing at the change and + `atomic:findingCode` naming the missing shortcut; the shape checks each shortcut + with `sh:or` (the shortcut exists, or a finding reached through + `sh:inversePath atomic:aboutResource` has the matching `atomic:findingCode`); - shortcut targets equal the authoritative intermediate path targets; - all referenced canonical nodes exist; - full hashes (not abbreviated labels) in durable IDs. @@ -582,7 +654,11 @@ shapes without applicable focus nodes must not be presented as complete audit ev Repository directory `.atomic/shapes/*.ttl` is loaded on `atomic query validate`. Automatic `vault sync` validation and publication hooks land in Phase 2. Builtin and enterprise shapes use the same engine; policy changes do not require Rust code. -Use explicit validation profiles so teams can select applicable workflow rules. +Validation runs in the CLI process (`atomic-cli` → `atomic-shacl`), not in the daemon, in +Phase 1, so libatomic does not depend on `atomic-shacl` and stays at Rust 1.90. +Use explicit validation profiles so teams can select applicable workflow rules. The +default profile for `atomic query validate` is **projection** plus any shapes in +`.atomic/shapes/`; `--profile publication` selects the publication profile. Report `sh:focusNode`, `sh:resultPath`, `sh:sourceShape`, `sh:sourceConstraintComponent`, `sh:resultSeverity`, and `sh:resultMessage`, retaining @@ -616,11 +692,14 @@ Example enterprise traceability shape (illustrative, not a complete SOX policy): full resource IRIs, typed literals, and properly scoped blank nodes. After the §1.6 cutover the KG already uses canonical terms and IRIs, so the adapter does no legacy-term translation; AST relationships are not in the KG. Signed objects are - never rewritten. + never rewritten. Each node gets its asserted types only (no OWL inference), and the + data graph also includes the ontology's `rdfs:subClassOf` triples, so SHACL's standard + `sh:targetClass` subclass matching finds, for example, `atomic:FeatureIntent` nodes + for a shape targeting `atomic:WorkIntent`. 2. Load builtin and selected enterprise shape bundles with explicit base IRIs and controlled imports. Do not fetch arbitrary remote shape imports implicitly. 3. Call PurRDF's existing API. The spike uses - `purrdf_shapes::engine::validate_graphs(data_nt, shapes_ttl, shapes_base)`; + `purrdf_shapes::engine::validate_graphs(data_nt, shapes_ttl, None)` (no shapes base); assess its prepared-shape/dataset APIs for production snapshot reuse. 4. Preserve its standard validation report; adapt presentation to CLI JSON/text without losing source shapes, paths, messages, or severity. @@ -638,10 +717,10 @@ the synthetic spike. No OWL inference is performed by this boundary in Phase 1. # Validate against all loaded shapes (builtin + .atomic/shapes/) atomic query validate -# Validate against a specific shape +# Validate against one shape from the selected profile (default: projection) atomic query validate --shape -# Validate a specific node +# Validate only one focus node, against the selected profile atomic query validate --node # Output validation report as JSON or human-readable @@ -650,17 +729,37 @@ atomic query validate --format json|text ### 1.5 Legacy term coverage -`LEGACY_TERMS` (test-only, gated by `#[cfg(test)]`): +`LEGACY_TERMS` lives in a production module, `atomic_core::kg::legacy`, not behind +`#[cfg(test)]`: the producers and the lint read it, and the coverage tests check it. It maps +the old **KG vocabulary** (the 81 constants), not the `atom:` terms of signed canonical +JSON-LD documents: the vault projector reads those documents through `atomic-canonical`'s +lifted `CanonicalNode` values (`lift_intent` and friends) and maps their fields to predicates +explicitly (§1.6). ```rust -pub enum Replacement { - One { class: Class }, - OnePred { predicate: Predicate }, - SplitByEndpoint(&[(Option, Option, Predicate)]), +pub enum Target { + Class(Class), + Predicate(Predicate), Removed { reason: &'static str }, } -static LEGACY_TERMS: &[(/* old */ &'static str, Replacement)] = &[ ... ]; +pub struct EndpointRule { + // Legacy node kinds (e.g. "session", "intent", "entity"), not `Class`: some legacy + // endpoints, like "entity", have no canonical class. + pub from_kind: Option<&'static str>, + pub to_kind: Option<&'static str>, + pub target: Target, +} + +pub enum Replacement { + One(Target), + // A base class plus subclasses chosen from source data, e.g. `vault:Intent` → + // `WorkIntent` + `FeatureIntent`/`BugIntent`/… from the canonical `kind`. + ClassWithRefinement { base: Class, refinements: &'static [Class], rule: &'static str }, + SplitByEndpoint(&'static [EndpointRule]), +} + +pub static LEGACY_TERMS: &[(/* old */ &'static str, Replacement)] = &[ ... ]; ``` The table below is the complete content of `LEGACY_TERMS` (approved by Lee, @@ -754,7 +853,8 @@ Tests: 1. Every legacy `prefix::*`, `predicate::*`, `entity_type::*`, and `edge_kind::*` constant appears exactly once in `LEGACY_TERMS`. -2. Every `One`/`Split` target exists in the TTL. +2. Every target class and predicate (in `One`, `ClassWithRefinement`, and `SplitByEndpoint`) + exists in the TTL. 3. TTL ↔ Rust consistency: parsed with `oxttl` + `oxrdf` (**dev-dependencies only**). Every class/property in the TTL has a corresponding Rust enum variant, and vice versa, with matching domain/range. @@ -778,7 +878,16 @@ new generation, not an in-place data migration. `atomic query reindex --full`, regardless of repository size. The marker is written in the same transaction as the final rebuild batch, so an interrupted rebuild restarts cleanly and a completed one never repeats. Repositories created by the new binary are - stamped at creation. Read-only opens on an old-generation KG return `KgNeedsRebuild`. + stamped at creation. Most commands use the writable `Repository::open`, so in practice + the first command after upgrading performs the rebuild (accepted: it happens once). + Opens that must stay light skip it (`ReconcileEffectBudget::MetadataOnly` opens, sandbox + opens detected in `open_once` at `repository/mod.rs:549`, and `open_for_native_repair` at + :872; full list in §1.3). While the stored generation is old, **producers skip KG writes** + (the pending rebuild regenerates everything), e.g. a `git watch` → `MetadataOnly` import + that calls `kg_enrich_from_vcs` (`import.rs:741, 979, 1114`). Read-only opens on an + old-generation KG fail only KG reads (`query …`, `vault context`, triage) with + `KgNeedsRebuild`; their other reads work normally. On first writable open, project + identity minting (§1.3) runs **before** the rebuild. - Capability fence `kg-semweb-v1` (`capability.rs`) so older binaries fail closed. **Producer rewrite** (canonical predicates, `rdf:type`, project-qualified IRIs via @@ -786,27 +895,51 @@ new generation, not an in-place data migration. | Producer | Today | Phase 1 | |---|---|---| -| Vault projection | `vault_triples.rs` | intents (subclass from canonical `kind`), memories, goals, skills, tool results, decomposition children, `:::ref` edges split by edge name (`dependsOnIntent`, `blockedBy`, `remediates`, `reviews`). Signed canonical documents keep their old `atom:` terms; the projector translates them through `LEGACY_TERMS` at projection time and never rewrites signed bytes | -| VCS projection | `vault_kg_enrich.rs` (VCS part) | views and `childViewOf`, changes (full hash), `directlyDependsOnChange`, `modifiesFile`, `authoredBy`, and `directlyOnView` recomputed from `VIEW_CHANGES` (today `ON_VIEW` records the working copy's desired view, not membership) | -| Session ledger | `session_kg.rs` | sessions, turns (`ManagedTurn` when `plan_id` is present), todos, provenance graphs; `hasTurn`, `hasPlan`, `generatedChange`, `turnExplainedBy`, `hasTodo`, `sessionViewContext` | -| Attestations (**new**) | not projected | `Attestation` nodes, `coversChange`, `attests`, `signedBy`, `previousAttestation`, and the verifier's recorded result (never inferred). **Turn mapping:** an attestation records `session_id` and `changes_covered` but no turn number, so emit `attests` only when exactly one turn of that session has `change_hashes` ⊆ `changes_covered`; otherwise emit an `AmbiguousAttestationTurn` finding and no `attests` edge. MAC-signed attestations (`session-mac:{sid}`) can't be verified without the key and record `Unverifiable`, never `Verified`; they never satisfy the publication profile, which requires an atomic identity DID signer | -| Findings (**new**) | none | `MissingHistoricalPlanId`, `MissingAttestation`, `AmbiguousAttestationTurn`, `UnverifiableAttestation`; never guessed links | +| Vault projection | `vault_triples.rs` | intents (subclass from canonical `kind`), memories, goals, skills, tool results, decomposition children, `:::ref` edges split by edge name (`dependsOnIntent`, `blockedBy`, `remediates`, `reviews`). Signed canonical documents are read as lifted structs (`lift_intent` etc.) and never rewritten. **Legacy URNs** inside them map to IRIs: `urn:atomic:change:` resolves to the full change IRI, using the same rule as `Repository::find_change_by_prefix` (`repository/changes.rs:1160`) but against a prefix index built once per projection run (that function scans every change per call); an ambiguous prefix is unresolved; `urn:atomic:intent:` and `urn:atomic:memory:` resolve through the vault manifest (case-insensitively). Anything that doesn't resolve gets an `UnresolvedReference` finding, not a guessed IRI. **`ManagedIntent`:** an intent whose status is `in_progress`, `needs-review`, or `done` (not `backlog`, `todo`, or `icebox`); `chore` and `remediation` kinds get only `WorkIntent` | +| VCS projection | `vault_kg_enrich.rs` (VCS part) | views and `childViewOf`, changes (full hash), `directlyDependsOnChange`, `modifiesFile`, `authoredBy`, and `directlyOnView` recomputed from `VIEW_CHANGES` (today `ON_VIEW` records the working copy's desired view, not membership). Also emits the `Project` node, its `ProjectIdentity` node, and `hasIdentity` from `PROJECT_IDENTITY` | +| Session ledger | `session_kg.rs` | sessions, turns (`ManagedTurn` when `plan_id` is present), todos, provenance graphs; `hasTurn`, `hasPlan`, `generatedChange`, `turnExplainedBy`, `hasTodo`, `sessionViewContext`, `prov:wasInformedBy` (previous turn), `prov:wasDerivedFrom` (forked session → parent). **Shortcuts:** for each change a turn generated, also emit `implements` → the turn's plan intent (or a `MissingPlanId` finding if the turn has no `plan_id`) and `changeExplainedBy` → the turn's provenance graph, only when that graph is registered locally (provenance sidecars can be absent, e.g. not transferred); otherwise a `MissingProvenance` finding and no edge, so every referenced node exists | +| Attestations (**new**) | not projected | `Attestation` nodes, `coversChange` plus the `attestedBy` shortcut on each covered change, `attests`, `signedBy`, `previousAttestation`, and `verificationResult`. **Verification:** the projector runs the existing verifier (`Attestation::verify_with_did`, `atomic-core/src/change/attestation.rs`) with the signer's public key, found from the attestation's `signer` DID: first the local identity store (`atomic-identity`), then a local **signer-key cache**. The cache is a new pristine table (`SIGNER_KEYS`: DID → base32 Ed25519 public key) filled from Atomic storage, which already serves public keys (`atomic-remote/src/storage.rs`: `resolve_identity_by_email`/`by_name` → `/identities/resolve`, authenticated; `resolve_identity_by_public_key` → `/identities/by-key/{key}`; agent delegation certificates also carry the agent key as `delegateKey`). Keys are fetched outside the producer, when a remote is configured and authenticated (after `pull`, and on `atomic query reindex`), never inside a KG write, so projection stays offline and fast. Before caching or using any fetched key, the client checks `atomic_canonical::did::did_matches_public_key(signer_did, key)` (a `did:atomic:` DID is `base32(blake3(key))`), so a wrong or malicious server answer can't produce `Verified`. **Storage dependency to confirm:** the attestation identifies its signer only by this fingerprint DID, but the current endpoints look up by email/name or by the raw key, and `/identities/by-key` decodes a `did:atomic:` suffix as if it were the raw key (`atomic-storage` `identities/handlers.rs:211`). For atomic's DIDs that suffix decodes to `blake3(key)`, a one-way fingerprint (`atomic-identity/src/identity.rs:50-58`, `atomic-canonical/src/did.rs:7-9`), so the key cannot be recovered from it and the lookup finds nothing. The needed storage fix is small: when `by-key` receives a `did:atomic:` value, match it against `blake3(stored_public_key)` (or an indexed `did` column) instead of the raw key. That belongs to the `atomic-storage` effort; until it lands, the client can still fill the cache for agent signers from `/identities/agents` (each agent's `did`) plus their published delegation certificates (`delegateKey`), checking the DID match as above. A verified signature records `Verified` and a failed one `Failed`; a signer whose key is neither local nor cached, or a MAC signature, records `Unverifiable`, and the attestation is re-projected when its key arrives in the cache. The result is never inferred. A generated change with no covering attestation gets a `MissingAttestation` finding. **Turn mapping:** an attestation records `session_id` and `changes_covered` but no turn number, so always emit `attestsSession` (attestation → its session), and emit `attests` (attestation → turn) only when exactly one **non-empty** turn of that session has `change_hashes` ⊆ `changes_covered` (turns with no changes are ignored); otherwise emit an `AmbiguousAttestationTurn` finding and no `attests` edge. Since attestations usually cover a whole session, `attestsSession` is the normal case. **Placement and triggers:** the producer lives in `atomic-repository` (verification needs `atomic-identity`) and runs after `save_attestation` (`repository/changes.rs:344`), after sidecar pull, on `session rebuild`, and in the full rebuild. Findings are derived: each run re-projects the session and deletes stale findings for its changes, so a `MissingAttestation` disappears once the session's attestation arrives. MAC-signed attestations (`session-mac:{sid}`) can't be verified without the key and record `Unverifiable`, never `Verified`; they never satisfy the publication profile, which requires an atomic identity DID signer | +| Findings (**new**) | none | `MissingPlanId` (any turn without a `plan_id`, so its changes have no `implements`), `MissingProvenance`, `MissingAttestation`, `AmbiguousAttestationTurn` (no `attests` edge; does not affect `attestedBy`), `MissingIntentId` (old-layout intent with no recorded ID), `UnresolvedLegacyTodo` (legacy `todo:` with no known session), `UnresolvedReference` (a legacy URN that doesn't resolve). Verification outcomes are recorded only as `verificationResult`, not as findings. Never guessed links | | AST enrichment | `vault_kg_enrich.rs` (AST part) | **removed** from the KG: no entity nodes, `CALLS`, `DEFINES`, `EXPORTS`, `INCLUDES`, `PART_OF` | Existing triggers stay as they are (record, vault sync, clone, pull, import, session -hooks); unifying them into one `reconcile()` pipeline is Phase 2. +hooks). Add triggers for view-membership changes so `directlyOnView` and `childViewOf` stay +current: `insert` and `unrecord` re-emit `directlyOnView` for the affected changes, and view +create/delete re-emit `childViewOf` and drop the deleted view's edges. Unifying all +triggers into one `reconcile()` pipeline is Phase 2. + +**KG metadata → RDF:** the RDF adapter exports these node fields as literals: `label` → +`rdfs:label`, `summary` → `dcterms:description`, timestamps → `dcterms:created` / +`dcterms:modified`, `status` / `priority` → `atomic:status` / `atomic:priority`, +`turn_number` → `atomic:turnNumber`. Other `metadata` keys stay application-only JSON and +are not exported. The `VerificationStatus` individuals (`atomic:Verified`, …) are IRIs +defined in the ontology, not `kg_nodes` rows. **Consumer cutover:** - `query_plan.rs`: `Filter` on `kind` becomes type-based; add a predicate filter and - direction to neighborhood steps; bump the plan JSON schema version. + direction to neighborhood steps; add a schema version field to the plan JSON (none + exists today). - CLI `query`: graph colour map keyed by IRI; text/DOT output compacts IRIs to CURIEs via `PREFIXES`; JSON emits absolute IRIs; the `ask` prompt lists canonical predicates. -- `query callers` and the triage report's blast radius / layer ordering - (`triage/report.rs`) switch from KG code edges to live `atomic-semantic` parsing. -- Triage, `vault_intent.rs`, and `vault/context.rs` match canonical predicates and types. -- libatomic daemon duplicates (`services_agent.rs`, `services_query.rs`), `rpc.rs`, and - the knowledge proto (`KGEdge.relation` documented as an absolute IRI; node types added; - API version bumped). The release is announced as breaking; no compatibility shim. +- `query callers`, the daemon's copy of it (`services_agent.rs:3372-3387`), and the triage + report's blast radius / layer ordering (`triage/report.rs`) switch from KG code edges to + live `atomic-semantic` parsing (`ParserRegistry::extract(file_path, source)` and + `extract_references(file_path, source)`; reference extraction exists for 7 languages and + returns nothing for the rest, as today). `query callers` parses the + working copy; triage parses each file's content as of the view being triaged (the + repository's view-aware content retrieval in `repository/content.rs`). +- Triage, `vault/context.rs`, and the ID-shape parsers listed in C.7 match canonical + predicates, types, and IRIs (read paths and labels from node data, not by splitting IDs). +- `query callers` takes `#` or a bare symbol name instead of an `entity:…` ID, + and finds callers with `atomic-semantic`'s reference extraction (`extract_references`) + across the working copy, resolving by name. Triage parses every tracked file in the view + being triaged, once per triage run, which matches today's coverage (`kg_enrich_calls` + also parses every tracked file); parse results are cached for the run. +- `atomic query reindex` gains a `--full` flag (it has no flags today). +- The knowledge proto's single `KGNode.node_type` string becomes a repeated `types` field. +- libatomic daemon duplicates (`services_agent.rs`, `services_query.rs`), + `atomic-cli/src/commands/rpc.rs`, and the knowledge proto (`KGEdge.relation` documented + as an absolute IRI; node types added; add an API version field, since none exists). The release is announced as breaking; no compatibility shim. - Update the ~110 test assertion sites. ## 2. Out of scope for Phase 1 @@ -820,22 +953,33 @@ hooks); unifying them into one `reconcile()` pipeline is Phase 2. | Code-structure edges (`calls`, `defines`, `exports`, `includes`, `modifiesEntity`) | When sourced from AST | | Scope-qualified entity IDs | Same — when code-structure edges are needed | | Project identity rotation / fork command | Phase 2 | -| DID binding on `ProjectIdentity.issuer` | Phase 2 | +| Requiring a signed `ProjectIdentity` (unsigned is allowed in Phase 1) | Phase 2 | ## 3. Acceptance criteria -1. `workflow-1.0.0.ttl` is parseable by `oxttl` and contains no axioms outside the - supported RL subset. +1. `workflow-1.0.0.ttl` is parseable by `oxttl`, and an M1 test checks every axiom against + an allow-list of the RL constructs this plan uses (`rdfs:subClassOf`, + `rdfs:subPropertyOf`, `rdfs:domain`, `rdfs:range`, `owl:inverseOf`, + `owl:TransitiveProperty`, `owl:propertyChainAxiom`, class/property/individual + declarations, `rdfs:isDefinedBy`, plus ontology metadata: `owl:Ontology`, + `owl:versionIRI`, `rdfs:label`, `rdfs:comment`); anything else fails the test. 2. `atomic_core::kg::ontology::Class::iri()` returns the correct absolute IRI for every variant. 3. `atomic_core::kg::ontology::Predicate::domain()` and `::range()` match the TTL. 4. `LEGACY_TERMS` coverage test passes: all old constants mapped, no unmapped survivors. -5. Workspace lint: no legacy `vault:*`, `atom:*`, or SCREAMING_CASE kind strings - outside `LEGACY_TERMS`. -6. **Project identity:** `atomic init` creates a `ProjectId`; `atomic project identity show` +5. Workspace lint (a unit test that scans `*.rs` sources): no string literal outside + `atomic_core::kg::legacy` equals a retired legacy **predicate, entity type, or edge + kind** value, i.e. the `predicate::*`, `entity_type::*`, and `edge_kind::*` values that + §1.5 maps to something other than themselves (e.g. `"MODIFIES"`, `"prov:hadMember"`, + `"vault:Session"`, `"atom:informedBy"`). Exempt: kept terms (`prov:wasDerivedFrom`, + `prov:used`, `prov:wasInformedBy`, `prov:generated`), the prefix labels, and the bare + node-kind words from the class table (`"memory"`, `"intent"`, `"file"`, …), which are + ordinary strings elsewhere in the code. +6. **Project identity:** `atomic init` and `atomic git import` mint a `ProjectId` + (`mint_reason = Created`); any other repository without one mints on first writable + open (`mint_reason = Upgrade`), before the KG rebuild; `atomic repo identity show` displays it; the ID round-trips through base32 encoding; IRIs minted via `kg::iri` - match the canonical format. - - `Repository::init` (as called by `atomic clone`) does not mint an identity. + match the §1.3 table; the identity object hash and (when signed) proof verify. - **Deferred until `atomic-storage` implements identity transport (owned by another effort):** a clone has the origin's identity; a pull never changes a populated identity and reports a differing one; pushing a different identity to a bound @@ -852,14 +996,22 @@ hooks); unifying them into one `reconcile()` pipeline is Phase 2. 10. **Adapter fidelity:** real workflow snapshots preserve resource identity, types, typed literals, and missing facts. Empty or partial projections must not be described as complete audit evidence. -11. **Regression corpus:** retain the isolated spike, add the existing gate fixtures - and relevant W3C cases, and compare against an independent SHACL implementation - before enabling mandatory production gates. +11. **Regression corpus:** retain the isolated spike and add the existing gate fixtures + and relevant W3C cases in Phase 1. The comparison against an independent SHACL + implementation gates the Phase 2 mandatory publication gates, not Phase 1. 12. **Cutover:** after the one-time rebuild, the KG contains only canonical predicates, `rdf:type` entries in `kg_types`, project-qualified IRIs, and full-hash change IRIs; no AST nodes or edges remain. A second writable open performs no rebuild work. -13. **Consumers:** every consumer listed in §1.6 works against the canonical KG; - `query callers` and triage blast radius produce the same results from live parsing. +13. **Consumers:** every consumer listed in §1.6 works against the canonical KG. Before + M3 changes anything, capture golden outputs of `query callers` and the triage report + on fixture repositories, recorded as result sets (callers per function; files per + triage layer), not CLI text, since the `query callers` argument format changes; after + M3, live parsing produces the same sets, except for documented intended differences + (live parsing has no 10-callee fan-out cap). +14. **Shortcuts (M3):** every generated change has `implements`, `changeExplainedBy`, and + `attestedBy`, or the matching finding, checked by a KG-level test in M3. The projection + profile conforming on a fixture repository with complete and incomplete history is + checked in M4, once SHACL exists. ## 4. Milestones @@ -867,22 +1019,25 @@ hooks); unifying them into one `reconcile()` pipeline is Phase 2. - `workflow-1.0.0.ttl` - `atomic_core::kg::ontology` module -- `LEGACY_TERMS` + coverage tests -- TTL ↔ Rust consistency test (`oxttl`/`oxrdf` as dev-deps) +- `atomic_core::kg::legacy` (`LEGACY_TERMS`) + coverage tests and the RL allow-list test +- TTL ↔ Rust consistency test (`oxttl`/`oxrdf` as dev-deps). Dev-dependencies still go in + the shared `Cargo.lock`, so confirm the CI MSRV job (Rust 1.90) still passes; pin + versions if it doesn't (see the M4 lockfile note) ### M2 — Project identity + IRI minting (one PR, depends on M1) - `atomic_core::kg::project_id` (`ProjectId`, base32, URN) -- `atomic_core::kg::iri` (mint/parse for every node kind) -- `PROJECT_IDENTITY` pristine table -- Identity object (JSON-LD, content-addressed, optionally signed) -- The `atomic init` command mints (not `Repository::init`); upgrading repos without a - remote mint on first writable open; repos with a remote stay unminted until transport - lands (or the owner runs `atomic project identity init`) +- `atomic_core::kg::iri` (mint/parse for every row of the §1.3 IRI table) +- `PROJECT_IDENTITY` pristine table (`object`, `hash`, `mint_reason`, `created_at`, + reserved `previous`) +- Identity object (inline-context JSON-LD, type-prefixed hash, signed with the user's + atomic identity when available) +- Minting: `atomic init` and `atomic git import` at creation; every other repository + without an identity on first writable open - **Not in Phase 1:** clone/pull/push identity transport. It waits for `atomic-storage` (see the boundary note in §1.3); leave push, pull, clone, `atomic-remote`, and the daemon's sync code unchanged -- CLI: `atomic project identity show` and `atomic project identity init` +- CLI: `atomic repo identity show` (`--json`) - Property tests: round-trip, percent-encoding, strict base32 parsing ### M3 — KG cutover (depends on M1/M2) @@ -890,7 +1045,9 @@ hooks); unifying them into one `reconcile()` pipeline is Phase 2. - Typed predicate write API, `kg_types`/`kg_types_rev`, `KG_GENERATION` one-time rebuild, capability fence - Producer rewrite (vault, VCS, session ledger) plus new attestation and findings - projectors; AST enrichment removed from the KG + projectors, including the asserted shortcut edges and attestation verification; + AST enrichment removed from the KG +- Golden outputs for `query callers` and triage captured before the cutover (AC13) - Consumer cutover (query plan, CLI, triage, vault context, daemon, proto); `query callers` and triage blast radius on live parsing - Test updates; the lint in acceptance criterion 5 passes @@ -910,8 +1067,14 @@ Lands as one PR (or a stack merged together): the RFC allows no dual-write windo - CLI: `atomic query validate`, profile selection, standard reports and error exits. - Custom shape loader from `.atomic/shapes/*.ttl`, controlled imports, no-target diagnostics, and snapshot/profile identification. -- Positive/negative profile fixtures, adapter integration tests, existing gate - regressions, and independent-validator comparisons before production enforcement. +- Positive/negative profile fixtures, adapter integration tests, and existing gate + regressions; the projection-profile half of AC14. +- **Lockfile check:** the workspace uses `resolver = "2"`, which is not MSRV-aware, so adding + `atomic-shacl` to the shared `Cargo.lock` can upgrade transitive dependencies of the + library crates beyond Rust 1.90. The CI MSRV job (library crates on 1.90, same lockfile) + catches this. Fix by pinning the affected versions (`cargo update --precise`) or enabling + the MSRV-aware resolver; if neither works, move `atomic-shacl` to its own workspace with + its own lockfile. ## 5. Development notes @@ -959,7 +1122,10 @@ outside the view system is the only place that guarantees this. Phase 2 will: - add the OWL 2 RL inference engine and the `KgEdge.graph` asserted/inferred split, so - `implements` and `changeExplainedBy` are materialized from the chains; + `implements`, `changeExplainedBy`, and `attestedBy` are derived from the chains and the + inverse and checked against the shortcuts Phase 1 producers assert; +- implement identity transport once `atomic-storage` supports it, including reconciling + `mint_reason = Upgrade` identities with their origin (§1.3); - unify sync/enrich/reindex into one `reconcile()` pipeline; - wire SHACL validation into `vault sync` and publication gates. From 7398106d1787025dc175bb8e49f7d9865ed5d5bf Mon Sep 17 00:00:00 2001 From: Bradley Hilton Date: Fri, 16 Oct 2026 11:11:53 -0500 Subject: [PATCH 11/19] feat: implement phase 1 knowledge graph closure and SHACL validation --- .github/workflows/ci.yml | 9 +- .github/workflows/release.yml | 2 +- .gitignore | 3 + Cargo.lock | 550 ++- Cargo.toml | 2 + atomic-cli/Cargo.toml | 3 +- atomic-cli/src/commands/git/import.rs | 7 +- atomic-cli/src/commands/init.rs | 2 + atomic-cli/src/commands/mod.rs | 2 + atomic-cli/src/commands/pull/command.rs | 20 +- atomic-cli/src/commands/query/mod.rs | 327 +- atomic-cli/src/commands/query/validate.rs | 100 + atomic-cli/src/commands/repo.rs | 69 + atomic-cli/src/commands/rpc.rs | 108 +- atomic-cli/src/commands/triage/output.rs | 21 +- atomic-cli/src/commands/vault/context.rs | 119 +- atomic-cli/src/error.rs | 6 + atomic-cli/src/main.rs | 5 + atomic-cli/src/service/mod.rs | 11 +- atomic-cli/tests/session_integration_test.rs | 219 +- atomic-core/Cargo.toml | 4 + .../ontology/shapes/projection-1.0.0.ttl | 63 + .../ontology/shapes/publication-1.0.0.ttl | 33 + atomic-core/ontology/workflow-1.0.0.ttl | 115 + atomic-core/src/kg/iri.rs | 257 ++ atomic-core/src/kg/legacy.rs | 256 ++ atomic-core/src/kg/mod.rs | 18 + atomic-core/src/kg/ontology.rs | 267 ++ atomic-core/src/kg/project_id.rs | 84 + atomic-core/src/lib.rs | 2 +- atomic-core/src/pristine/capability.rs | 9 +- atomic-core/src/pristine/error.rs | 4 + atomic-core/src/pristine/mod.rs | 1 - atomic-core/src/pristine/ontology.rs | 354 -- atomic-core/src/pristine/schema.rs | 4 + atomic-core/src/pristine/tables.rs | 17 +- atomic-core/src/pristine/traits/triples.rs | 23 +- atomic-core/src/pristine/txn/kg_identity.rs | 197 ++ atomic-core/src/pristine/txn/mod.rs | 1 + atomic-core/src/pristine/txn/pristine.rs | 54 +- atomic-core/src/pristine/txn/read.rs | 40 +- atomic-core/src/pristine/txn/write/mod.rs | 2 + .../src/pristine/txn/write/session_kg.rs | 725 ++-- atomic-core/src/pristine/txn/write/tests.rs | 18 + atomic-core/src/pristine/txn/write/triples.rs | 158 +- atomic-core/src/pristine/vault.rs | 88 +- atomic-core/tests/kg_iri.rs | 112 + atomic-core/tests/kg_ontology.rs | 197 ++ atomic-core/tests/kg_vocabulary_lint.rs | 76 + atomic-repository/src/ai/mod.rs | 67 +- atomic-repository/src/ai/tools.rs | 2 +- atomic-repository/src/query_plan.rs | 144 +- atomic-repository/src/repository/changes.rs | 27 +- atomic-repository/src/repository/history.rs | 9 + atomic-repository/src/repository/insert.rs | 4 + .../src/repository/kg_attestation.rs | 192 ++ .../src/repository/kg_rebuild.rs | 95 + atomic-repository/src/repository/mod.rs | 14 + atomic-repository/src/repository/operation.rs | 8 + .../src/repository/project_identity.rs | 144 + .../src/repository/syntax_index.rs | 187 + .../src/repository/tests/capability_tests.rs | 37 +- atomic-repository/src/repository/triage.rs | 32 +- atomic-repository/src/repository/vault.rs | 40 +- .../src/repository/vault_intent.rs | 18 +- .../src/repository/vault_kg_enrich.rs | 1517 ++------ .../src/repository/vault_projection.rs | 622 ++++ .../src/repository/vault_triples.rs | 3037 ++--------------- atomic-repository/src/repository/views.rs | 28 + atomic-repository/src/triage/report.rs | 147 +- atomic-repository/tests/kg_projection.rs | 146 + atomic-repository/tests/project_identity.rs | 33 + atomic-shacl/Cargo.toml | 25 + atomic-shacl/src/adapter.rs | 109 + atomic-shacl/src/lib.rs | 283 ++ atomic-shacl/tests/fixtures/w3c-class-001.ttl | 15 + atomic-shacl/tests/profiles.rs | 237 ++ atomic-shacl/tests/w3c_core.rs | 51 + atomic-shacl/tests/workflow_snapshot.rs | 297 ++ docs/knowledge-graph-closure-phase1-plan.md | 19 +- docs/knowledge-graph-phase1-upgrade.md | 63 + libatomic/Cargo.toml | 1 + .../knowledge/knowledge_messages.proto | 14 +- libatomic/src/daemon/services_agent.rs | 230 +- libatomic/src/daemon/services_query.rs | 18 +- libatomic/src/daemon/services_sync_remote.rs | 14 + libatomic/src/lib.rs | 1 + libatomic/src/signer_refresh.rs | 50 + libatomic/tests/signer_refresh.rs | 112 + tests/harness/14_vault_kg.sh | 6 +- tests/harness/46_knowledge_graph_closure.sh | 12 + .../harness/fixtures/kg-closure-baseline.json | 9 + tests/harness/kg_closure_baseline.py | 76 + tests/harness/kg_closure_e2e.py | 123 + 94 files changed, 7340 insertions(+), 5739 deletions(-) create mode 100644 atomic-cli/src/commands/query/validate.rs create mode 100644 atomic-cli/src/commands/repo.rs create mode 100644 atomic-core/ontology/shapes/projection-1.0.0.ttl create mode 100644 atomic-core/ontology/shapes/publication-1.0.0.ttl create mode 100644 atomic-core/ontology/workflow-1.0.0.ttl create mode 100644 atomic-core/src/kg/iri.rs create mode 100644 atomic-core/src/kg/legacy.rs create mode 100644 atomic-core/src/kg/mod.rs create mode 100644 atomic-core/src/kg/ontology.rs create mode 100644 atomic-core/src/kg/project_id.rs delete mode 100644 atomic-core/src/pristine/ontology.rs create mode 100644 atomic-core/src/pristine/txn/kg_identity.rs create mode 100644 atomic-core/tests/kg_iri.rs create mode 100644 atomic-core/tests/kg_ontology.rs create mode 100644 atomic-core/tests/kg_vocabulary_lint.rs create mode 100644 atomic-repository/src/repository/kg_attestation.rs create mode 100644 atomic-repository/src/repository/kg_rebuild.rs create mode 100644 atomic-repository/src/repository/project_identity.rs create mode 100644 atomic-repository/src/repository/syntax_index.rs create mode 100644 atomic-repository/src/repository/vault_projection.rs create mode 100644 atomic-repository/tests/kg_projection.rs create mode 100644 atomic-repository/tests/project_identity.rs create mode 100644 atomic-shacl/Cargo.toml create mode 100644 atomic-shacl/src/adapter.rs create mode 100644 atomic-shacl/src/lib.rs create mode 100644 atomic-shacl/tests/fixtures/w3c-class-001.ttl create mode 100644 atomic-shacl/tests/profiles.rs create mode 100644 atomic-shacl/tests/w3c_core.rs create mode 100644 atomic-shacl/tests/workflow_snapshot.rs create mode 100644 docs/knowledge-graph-phase1-upgrade.md create mode 100644 libatomic/src/signer_refresh.rs create mode 100644 libatomic/tests/signer_refresh.rs create mode 100644 tests/harness/46_knowledge_graph_closure.sh create mode 100644 tests/harness/fixtures/kg-closure-baseline.json create mode 100644 tests/harness/kg_closure_baseline.py create mode 100644 tests/harness/kg_closure_e2e.py diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 742332e2..336358e0 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -153,5 +153,10 @@ jobs: restore-keys: | ${{ runner.os }}-cargo-msrv- - - name: cargo check (MSRV) - run: cargo check --workspace + - name: cargo check (shared library MSRV) + run: cargo check --workspace --exclude atomic-cli --exclude atomic-shacl --locked + + - uses: dtolnay/rust-toolchain@1.98 + + - name: cargo check (CLI and SHACL MSRV) + run: cargo check -p atomic-cli -p atomic-shacl --locked diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 2f7da682..ad3c5724 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -143,7 +143,7 @@ jobs: run: node --test libatomic/scripts/package-runtime.test.mjs - name: Install Rust toolchain - uses: dtolnay/rust-toolchain@1.90 + uses: dtolnay/rust-toolchain@1.98 with: targets: ${{ matrix.target }} diff --git a/.gitignore b/.gitignore index 8c462998..d4c77ffe 100644 --- a/.gitignore +++ b/.gitignore @@ -140,6 +140,9 @@ secrets/ # Atomic repository data (for testing) .atomic/ +# Local vault state (existing tracked project content remains tracked) +/.vault/ + # Benchmark results criterion/ benches/target/ diff --git a/Cargo.lock b/Cargo.lock index ad39a4aa..5bfcf366 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -8,6 +8,41 @@ version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa" +[[package]] +name = "aead" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1973cfbc1a2daf9cf550e74e1f088c28e7f7d8c1e1418fb6c9dc5184b7e84c99" +dependencies = [ + "crypto-common 0.2.2", + "inout", +] + +[[package]] +name = "aes" +version = "0.9.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "35f0f96ce78e38c3dc6d8948aa8163d06385be74000f3c7a95bf1eef35d3ea32" +dependencies = [ + "cipher", + "cpubits", + "cpufeatures 0.3.1", +] + +[[package]] +name = "aes-gcm" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7f2b8006a0c83f52b62ba44a97b58bf76fe2f70a329e588f67f89691d93d498f" +dependencies = [ + "aead", + "aes", + "cipher", + "ctr", + "ctutils", + "ghash", +] + [[package]] name = "ahash" version = "0.8.12" @@ -176,7 +211,7 @@ dependencies = [ "serde", "serde_json", "serial_test", - "sha2", + "sha2 0.10.9", "tempfile", "thiserror 1.0.69", "tokio", @@ -222,6 +257,7 @@ dependencies = [ "atomic-remote", "atomic-repository", "atomic-semantic", + "atomic-shacl", "atomic-teams", "blake3", "bytes", @@ -301,8 +337,11 @@ dependencies = [ "ed25519-dalek", "fastcdc", "log", + "oxrdf", + "oxttl", "parking_lot", "postcard", + "proc-macro2", "quickcheck", "quickcheck_macros", "rand 0.8.8", @@ -312,6 +351,7 @@ dependencies = [ "serde", "serde_json", "similar", + "syn 2.0.119", "tempfile", "thiserror 1.0.69", "ulid", @@ -416,7 +456,7 @@ dependencies = [ "serde", "serde_json", "sha1", - "sha2", + "sha2 0.10.9", "syntext", "tar", "tempfile", @@ -448,6 +488,26 @@ dependencies = [ "tree-sitter-typescript", ] +[[package]] +name = "atomic-shacl" +version = "0.20.0" +dependencies = [ + "atomic-canonical", + "atomic-core", + "atomic-identity", + "atomic-repository", + "blake3", + "oxrdf", + "oxttl", + "purrdf-shapes", + "redb 4.4.0", + "serde", + "serde_json", + "tempfile", + "thiserror 1.0.69", + "url", +] + [[package]] name = "atomic-teams" version = "0.20.0" @@ -596,6 +656,15 @@ dependencies = [ "generic-array", ] +[[package]] +name = "block-buffer" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d2f6c7dbe95a6ed67ad9f18e57daf93a2f034c524b99fd2b76d18fdfeb6660aa" +dependencies = [ + "hybrid-array", +] + [[package]] name = "bs58" version = "0.5.1" @@ -691,6 +760,17 @@ dependencies = [ "windows-link", ] +[[package]] +name = "cipher" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e8cf2a2c93cd704877c0858356ed03480ff301ee950b43f1cbe4573b088bfa6c" +dependencies = [ + "block-buffer 0.12.1", + "crypto-common 0.2.2", + "inout", +] + [[package]] name = "clap" version = "4.6.7" @@ -744,6 +824,12 @@ version = "1.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1c133bc6a41be0d194c306b5506d15e6feeea7b1d6604bd3f8310dfb2ca96486" +[[package]] +name = "cmov" +version = "0.5.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c9ea0ac24bc397ab3c98583a3c9ba74fa56b09a4449bbe172b9b1ddb016027a" + [[package]] name = "cobs" version = "0.3.0" @@ -795,6 +881,12 @@ version = "0.9.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8" +[[package]] +name = "const-oid" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a6ef517f0926dd24a1582492c791b6a4818a4d94e789a334894aa15b0d12f55c" + [[package]] name = "constant_time_eq" version = "0.4.2" @@ -842,6 +934,12 @@ version = "1.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7f8f80099a98041a3d1622845c271458a2d73e688351bf3cb999266764b81d48" +[[package]] +name = "cpubits" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "15b85f9c39137c3a891689859392b1bd49812121d0d61c9caf00d46ed5ce06ae" + [[package]] name = "cpufeatures" version = "0.2.17" @@ -919,6 +1017,15 @@ dependencies = [ "typenum", ] +[[package]] +name = "crypto-common" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ce6e4c961d6cd6c9a86db418387425e8bdeaf05b3c8bc1411e6dca4c252f1453" +dependencies = [ + "hybrid-array", +] + [[package]] name = "ctor" version = "0.2.9" @@ -929,6 +1036,24 @@ dependencies = [ "syn 2.0.119", ] +[[package]] +name = "ctr" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "baaca1c4b237092596f64d571e9db6ce4109c4ef9742e27590f1709594461f21" +dependencies = [ + "cipher", +] + +[[package]] +name = "ctutils" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "03bb0e1cc970d482d121d9a1744999169b69a07470b3d644a7894e53fcaf4574" +dependencies = [ + "cmov", +] + [[package]] name = "curve25519-dalek" version = "4.1.3" @@ -938,7 +1063,7 @@ dependencies = [ "cfg-if", "cpufeatures 0.2.17", "curve25519-dalek-derive", - "digest", + "digest 0.10.7", "fiat-crypto", "rustc_version", "subtle", @@ -999,7 +1124,7 @@ version = "0.7.10" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb" dependencies = [ - "const-oid", + "const-oid 0.9.6", "zeroize", ] @@ -1028,8 +1153,19 @@ version = "0.10.7" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" dependencies = [ - "block-buffer", - "crypto-common", + "block-buffer 0.10.4", + "crypto-common 0.1.7", +] + +[[package]] +name = "digest" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2" +dependencies = [ + "block-buffer 0.12.1", + "const-oid 0.10.2", + "crypto-common 0.2.2", ] [[package]] @@ -1084,7 +1220,7 @@ dependencies = [ "curve25519-dalek", "ed25519", "serde", - "sha2", + "sha2 0.10.9", "subtle", "zeroize", ] @@ -1420,6 +1556,15 @@ dependencies = [ "wasm-bindgen", ] +[[package]] +name = "ghash" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2eecf2d5dc9b66b732b97707a0210906b1d30523eb773193ab777c0c84b3e8d5" +dependencies = [ + "polyval", +] + [[package]] name = "git2" version = "0.19.0" @@ -1571,6 +1716,15 @@ version = "1.0.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "df3b46402a9d5adb4c86a0cf463f42e19994e3ee891101b1841f30a545cb49a9" +[[package]] +name = "hybrid-array" +version = "0.4.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "27f864f10dfb56725ce5ce5472bc52252c8f93a4ab86327122cebf62c5f59a17" +dependencies = [ + "typenum", +] + [[package]] name = "hyper" version = "1.12.0" @@ -1841,6 +1995,15 @@ dependencies = [ "web-time", ] +[[package]] +name = "inout" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d8c0d789294fb457b791f7797a04478704788f30b407e5d0b01deaa846572d9f" +dependencies = [ + "hybrid-array", +] + [[package]] name = "ipnet" version = "2.12.2" @@ -1955,9 +2118,9 @@ dependencies = [ [[package]] name = "js-sys" -version = "0.3.106" +version = "0.3.102" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7883d941dae510fb2d978fc3fe018c71c9e2892fd38854de3e8b92c2e5ad9cc5" +checksum = "03d04c30968dffe80775bd4d7fb676131cd04a1fb46d2686dbffbaec2d9dfd31" dependencies = [ "cfg-if", "futures-util", @@ -2006,6 +2169,7 @@ dependencies = [ "tonic-build", "tracing", "ulid", + "url", "uuid", ] @@ -2431,6 +2595,46 @@ version = "0.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "04744f49eae99ab78e0d5c0b603ab218f515ea8cfe5a456d7629ad883a3b6e7d" +[[package]] +name = "oxilangtag" +version = "0.1.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5d3b4eb570abd4a1dcb062c31fd37b832264d9dc7292c3e69acfe926c87b063f" +dependencies = [ + "serde", +] + +[[package]] +name = "oxiri" +version = "0.2.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "54b4ed3a7192fa19f5f48f99871f2755047fabefd7f222f12a1df1773796a102" + +[[package]] +name = "oxrdf" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a04761319ef84de1f59782f189d072cbfc3a9a40c4e8bded8667202fbd35b02a" +dependencies = [ + "oxilangtag", + "oxiri", + "rand 0.8.8", + "thiserror 1.0.69", +] + +[[package]] +name = "oxttl" +version = "0.1.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0d385f1776d7cace455ef6b7c54407838eff902ca897303d06eb12a26f4cf8a0" +dependencies = [ + "memchr", + "oxilangtag", + "oxiri", + "oxrdf", + "thiserror 1.0.69", +] + [[package]] name = "parking_lot" version = "0.12.5" @@ -2512,6 +2716,17 @@ version = "0.3.34" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f6b464fbc74e149a392436b17d523f769e057cb6877f6a5c4618bc6f11800548" +[[package]] +name = "polyval" +version = "0.7.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0fa31d631f2b2cb2a544d0aa321ce847a94764d701ca2becc411138b93d49cd" +dependencies = [ + "cpubits", + "cpufeatures 0.3.1", + "universal-hash", +] + [[package]] name = "portable-atomic" version = "1.15.0" @@ -2675,6 +2890,249 @@ dependencies = [ "thiserror 1.0.69", ] +[[package]] +name = "purrdf-cdt" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5f3f6ce304b34bb666d5ffe441587563206cbc7702ab76708e26eb2398f531af" +dependencies = [ + "purrdf-events", + "purrdf-hash", + "purrdf-iri", + "purrdf-lex", + "purrdf-xsd", +] + +[[package]] +name = "purrdf-core" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1d073303e9c754f0244391992392aee7a71129a99817b6b608fc0d24a0d80e8d" +dependencies = [ + "hashbrown 0.17.1", + "purrdf-cdt", + "purrdf-deflate", + "purrdf-events", + "purrdf-hash", + "purrdf-iri", + "purrdf-lex", + "purrdf-xsd", + "regex", + "regex-syntax", + "sha2 0.11.1", +] + +[[package]] +name = "purrdf-datalog" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "438d3d607a0e55794e0b016af3cba4332b954ddf2ec4c04cb864393e022934aa" +dependencies = [ + "hashbrown 0.17.1", + "purrdf-core", + "purrdf-hash", + "purrdf-lex", + "rayon", +] + +[[package]] +name = "purrdf-deflate" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f1322db69953bf995fa856f361e463ebeed7a052d8853dd2d77c59c4001e979" +dependencies = [ + "purrdf-hash", +] + +[[package]] +name = "purrdf-ed25519" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ac199f2e42bd2f48caba9a6573b1161f067d9fbe06c376c28cfc93ed69892912" +dependencies = [ + "purrdf-hash", + "sha2 0.11.1", +] + +[[package]] +name = "purrdf-events" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b5455431280c55a65bbf44f41f5f2d901a0c5923052cce94d392abead530e40c" + +[[package]] +name = "purrdf-gts" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "76a60ea5fc7ca62449fa7a6ea80bd0a8c14983d15d9584d4fa42140a3e1a9e4f" +dependencies = [ + "aes-gcm", + "fastrand", + "purrdf-deflate", + "purrdf-ed25519", + "purrdf-events", + "purrdf-hash", + "purrdf-iri", + "purrdf-lex", + "purrdf-xsd", + "rayon", + "structured-zstd", + "visual-hashing", +] + +[[package]] +name = "purrdf-hash" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bd021886ad0790918ec3501c47f952f98cc60474340fe4dad8df1f4b77471a91" + +[[package]] +name = "purrdf-iri" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "682988bf556aa2d09dbcb30bea74dc07fa356648b7217c0fd95330a168694aaa" +dependencies = [ + "purrdf-hash", + "purrdf-lex", +] + +[[package]] +name = "purrdf-lex" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c1443d0cd967b66b82188958bc92b63dc9673e253f86f559eb93de0319a73486" +dependencies = [ + "purrdf-hash", +] + +[[package]] +name = "purrdf-rdf" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "967240728285800972d323c7bd67dd5facd937572a3d17d65716242e39bb2a10" +dependencies = [ + "hashbrown 0.17.1", + "purrdf-core", + "purrdf-ed25519", + "purrdf-events", + "purrdf-gts", + "purrdf-hash", + "purrdf-iri", + "purrdf-lex", + "purrdf-sparql-algebra", + "purrdf-sparql-eval", + "purrdf-xsd", + "rayon", + "regex", + "sha2 0.11.1", +] + +[[package]] +name = "purrdf-shapes" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a4e6d0720b974870f2b29256e5c5bb5e62c4d40308afb24ee66556525a2bbeb5" +dependencies = [ + "purrdf-core", + "purrdf-datalog", + "purrdf-deflate", + "purrdf-hash", + "purrdf-iri", + "purrdf-lex", + "purrdf-rdf", + "purrdf-slice", + "purrdf-sparql-algebra", + "purrdf-sparql-eval", + "purrdf-xsd", + "rayon", + "regex", +] + +[[package]] +name = "purrdf-slice" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "edbe19c6825da25e00e675f8d8049f503f2a76624c2f41f1add7f55756ea02c0" +dependencies = [ + "purrdf-core", + "purrdf-gts", + "purrdf-hash", + "purrdf-iri", + "purrdf-lex", + "purrdf-rdf", + "purrdf-sparql-algebra", + "rayon", + "regex", + "sha2 0.11.1", +] + +[[package]] +name = "purrdf-sparql-algebra" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "429dfe21a2abbcc515c3cf05ce354f0e29e0c36c9624342f964bcc9cab7cbb24" +dependencies = [ + "purrdf-cdt", + "purrdf-hash", + "purrdf-iri", + "purrdf-lex", + "purrdf-xsd", +] + +[[package]] +name = "purrdf-sparql-eval" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "08a95bb1166c152d784081ac144d10e482e1f2922d6438cdae5e6aa60f1ad69c" +dependencies = [ + "getrandom 0.3.4", + "hashbrown 0.17.1", + "purrdf-cdt", + "purrdf-core", + "purrdf-hash", + "purrdf-iri", + "purrdf-lex", + "purrdf-sparql-algebra", + "purrdf-sparql-results", + "purrdf-stack", + "purrdf-xsd", + "rayon", + "regex", + "sha2 0.11.1", + "wasm-bindgen", +] + +[[package]] +name = "purrdf-sparql-results" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "332b4b18dd46a7841eb776f44813c0d1cdd3f76b504c6ef151332575c916b355" +dependencies = [ + "purrdf-core", + "purrdf-iri", + "purrdf-lex", +] + +[[package]] +name = "purrdf-stack" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b9667b6e28591992199ae8841b5f82611fc92e99b25f1faf3826f22bfd359376" +dependencies = [ + "libc", + "windows-sys 0.61.2", +] + +[[package]] +name = "purrdf-xsd" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6d925ae9a7cac3fd7ded1fca01da0cc7caf895cf2df5ef4400669c39a753a8f8" +dependencies = [ + "purrdf-hash", + "purrdf-lex", +] + [[package]] name = "quickcheck" version = "1.1.0" @@ -3311,7 +3769,7 @@ checksum = "a978451301f4db1d02937a4ab3ccce137717b81826e79b7d49ffe3244a13c3b8" dependencies = [ "cfg-if", "cpufeatures 0.2.17", - "digest", + "digest 0.10.7", ] [[package]] @@ -3322,7 +3780,18 @@ checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283" dependencies = [ "cfg-if", "cpufeatures 0.2.17", - "digest", + "digest 0.10.7", +] + +[[package]] +name = "sha2" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "47d7069beb7d6ac7b9acd1039986e73443f24234f41074da099d6f994ac9ad19" +dependencies = [ + "cfg-if", + "cpufeatures 0.3.1", + "digest 0.11.3", ] [[package]] @@ -3452,6 +3921,16 @@ version = "0.11.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f" +[[package]] +name = "structured-zstd" +version = "0.0.49" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "392eea135985654a313cdfee001e93031bacd96a3693a5dc4b0d13793e40b27a" +dependencies = [ + "fastrand", + "twox-hash", +] + [[package]] name = "subtle" version = "2.6.1" @@ -4101,6 +4580,12 @@ version = "0.2.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b" +[[package]] +name = "twox-hash" +version = "2.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "86a801b3cea342a06d468c8710662aa29e5e05e4f5c0d62f00bbb7f2ad7941c2" + [[package]] name = "typenum" version = "1.20.1" @@ -4150,6 +4635,16 @@ version = "0.2.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b4ac048d71ede7ee76d585517add45da530660ef4390e49b098733c6e897f254" +[[package]] +name = "universal-hash" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f4987bdc12753382e0bec4a65c50738ffaabc998b9cdd1f952fb5f39b0048a96" +dependencies = [ + "crypto-common 0.2.2", + "ctutils", +] + [[package]] name = "untrusted" version = "0.9.0" @@ -4216,6 +4711,12 @@ version = "0.9.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" +[[package]] +name = "visual-hashing" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5056c65a9a789c9c1156fbfab38df224d7b729089e3b075892136e8f3092166e" + [[package]] name = "walkdir" version = "2.5.0" @@ -4252,9 +4753,9 @@ dependencies = [ [[package]] name = "wasm-bindgen" -version = "0.2.129" +version = "0.2.125" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9bb54f33acc68fd454578d9820b0bde1a1a3d17aa17bb7b6595806d02886d409" +checksum = "8ddb3f79143bced6de84270411622a2699cee572fc0875aeaf1e7867cf9fca1a" dependencies = [ "cfg-if", "once_cell", @@ -4265,20 +4766,19 @@ dependencies = [ [[package]] name = "wasm-bindgen-futures" -version = "0.4.79" +version = "0.4.75" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3cbab34de2d982e9b48e18d216d04c4a6f641066ff19ffb699980f591ee3610e" +checksum = "503b14d284f2c8dac03b819967e155ea753f573586193b2b2c95990cb5d69280" dependencies = [ "js-sys", - "tokio", "wasm-bindgen", ] [[package]] name = "wasm-bindgen-macro" -version = "0.2.129" +version = "0.2.125" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2e29d0c35b16e224a7eeb5cd2d25e3e1968fbd65604117b44d3b789d00ee8535" +checksum = "4e21a184b13fb19e157296e2c46056aec9092264fab83e4ba59e68c61b323c3d" dependencies = [ "quote", "wasm-bindgen-macro-support", @@ -4286,22 +4786,22 @@ dependencies = [ [[package]] name = "wasm-bindgen-macro-support" -version = "0.2.129" +version = "0.2.125" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6f501a8bc3719dba86ef8ae4728879c08001bea749eb1333ac5b91e040e2a6b7" +checksum = "fecefd9c35bd935a20fc3fc344b5f29138961e4f47fb03297d88f2587afb5ebd" dependencies = [ "bumpalo", "proc-macro2", "quote", - "syn 3.0.9", + "syn 2.0.119", "wasm-bindgen-shared", ] [[package]] name = "wasm-bindgen-shared" -version = "0.2.129" +version = "0.2.125" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "23f0c9c52aa7cd7d77769a4cfe2a9adb1b331f489a41d912ce14513d5ab995c6" +checksum = "23939e44bb9a5d7576fa2b563dc2e136628f1224e88a8deed09e04858b77871f" dependencies = [ "unicode-ident", ] @@ -4339,9 +4839,9 @@ dependencies = [ [[package]] name = "web-sys" -version = "0.3.106" +version = "0.3.102" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "88261b9deccee56594c11a3460c462c41f58d148598fe70ad77070126a68aba4" +checksum = "a6430a72df5eb332242960fe84b3002a241163998241eb596d4f739b9757061d" dependencies = [ "js-sys", "wasm-bindgen", diff --git a/Cargo.toml b/Cargo.toml index fbabc9e0..f6d43f7d 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -12,6 +12,7 @@ members = [ "atomic-remote", "atomic-repository", "atomic-semantic", + "atomic-shacl", "atomic-teams", "atomic-wasm", "libatomic", @@ -38,6 +39,7 @@ atomic-objects = { path = "atomic-objects" } atomic-remote = { path = "atomic-remote" } atomic-repository = { path = "atomic-repository" } atomic-semantic = { path = "atomic-semantic" } +atomic-shacl = { path = "atomic-shacl" } atomic-teams = { path = "atomic-teams" } libatomic = { path = "libatomic" } diff --git a/atomic-cli/Cargo.toml b/atomic-cli/Cargo.toml index 3947a7c2..f8c667aa 100644 --- a/atomic-cli/Cargo.toml +++ b/atomic-cli/Cargo.toml @@ -14,7 +14,7 @@ edition.workspace = true authors.workspace = true license.workspace = true repository.workspace = true -rust-version.workspace = true +rust-version = "1.98" description = "A mathematically sound distributed version control system" readme = "README.md" keywords = ["vcs", "version-control", "distributed", "patch-theory"] @@ -39,6 +39,7 @@ atomic-client = { workspace = true } tonic = { workspace = true } prost-types = { workspace = true } atomic-semantic = { workspace = true } +atomic-shacl = { workspace = true } atomic-teams = { workspace = true, optional = true } # CLI diff --git a/atomic-cli/src/commands/git/import.rs b/atomic-cli/src/commands/git/import.rs index 9ac35228..bc393183 100644 --- a/atomic-cli/src/commands/git/import.rs +++ b/atomic-cli/src/commands/git/import.rs @@ -849,8 +849,11 @@ impl Command for Import { )); // Seed the default view with Git's default branch name so Atomic's // shared view matches the project's Git default branch. - Repository::init_with_view(workdir, &default_branch) - .map_err(|e| CliError::Internal(e.into()))? + let repo = Repository::init_with_view(workdir, &default_branch) + .map_err(|e| CliError::Internal(e.into()))?; + repo.ensure_project_identity(atomic_core::kg::project_id::MintReason::Created) + .map_err(CliError::Repository)?; + repo }; // Record the detected vault policy for a freshly created repository diff --git a/atomic-cli/src/commands/init.rs b/atomic-cli/src/commands/init.rs index 82898660..d0951169 100644 --- a/atomic-cli/src/commands/init.rs +++ b/atomic-cli/src/commands/init.rs @@ -543,6 +543,8 @@ impl Command for Init { } other => CliError::Repository(other), })?; + repo.ensure_project_identity(atomic_core::kg::project_id::MintReason::Created) + .map_err(CliError::Repository)?; let working_copy = repo .require_working_copy_id() .map_err(CliError::Repository)?; diff --git a/atomic-cli/src/commands/mod.rs b/atomic-cli/src/commands/mod.rs index b35f49b4..8b2dd558 100644 --- a/atomic-cli/src/commands/mod.rs +++ b/atomic-cli/src/commands/mod.rs @@ -87,6 +87,7 @@ pub mod op; pub mod provenance; pub mod record; pub mod remove; +pub mod repo; pub mod restore; pub mod revise; pub(crate) mod rpc; @@ -168,6 +169,7 @@ pub use query::Query; pub use record::Record; pub use remote::Remote; pub use remove::Remove; +pub use repo::Repo; pub use restore::Restore; pub use revise::Revise; pub use sandbox::Sandbox; diff --git a/atomic-cli/src/commands/pull/command.rs b/atomic-cli/src/commands/pull/command.rs index b2ec1bfc..1369ff6b 100644 --- a/atomic-cli/src/commands/pull/command.rs +++ b/atomic-cli/src/commands/pull/command.rs @@ -530,6 +530,7 @@ impl Pull { let config = self .build_remote_config(&remote_url, identity_hint.as_deref()) .await; + let signer_client = libatomic::signer_refresh::client_for_remote(&remote_url, &config); let remote = HttpRemote::with_config(&remote_url, config).map_err(|e| { finish_error(&spinner, "Failed to connect"); convert_remote_error(e, &remote_url) @@ -767,6 +768,11 @@ impl Pull { crate::commands::sidecars::report_sidecars(sidecar_stats); } } + if let Some(client) = &signer_client { + if let Err(error) = libatomic::signer_refresh::refresh(&repo, client).await { + log::warn!("signer key refresh after download failed: {error}"); + } + } print_blank(); print_success(&format!( "Downloaded {} (not inserted - use 'atomic insert' to insert)", @@ -960,16 +966,10 @@ impl Pull { } } - // Enrich the knowledge graph from pulled VCS data (views, files, changes). - // This runs for every pull that applies changes, not just vaulted repos. - if stats.changes_applied > 0 { - // Ensure KG tables exist (idempotent — no-op if already created) - if let Err(e) = repo.init_kg() { - log::warn!("KG table init failed: {}", e); - } - match repo.kg_enrich_from_vcs(working_copy) { - Ok(kg_stats) => log::info!("KG enriched after pull: {}", kg_stats), - Err(e) => log::warn!("KG enrichment after pull failed: {}", e), + // Ingestion/membership APIs project regardless of download/apply counts. + if let Some(client) = &signer_client { + if let Err(error) = libatomic::signer_refresh::refresh(&repo, client).await { + log::warn!("signer key refresh after pull failed: {error}"); } } diff --git a/atomic-cli/src/commands/query/mod.rs b/atomic-cli/src/commands/query/mod.rs index a7eeb315..a66b8bca 100644 --- a/atomic-cli/src/commands/query/mod.rs +++ b/atomic-cli/src/commands/query/mod.rs @@ -4,6 +4,7 @@ use std::collections::{HashMap, HashSet}; use clap::{Parser, Subcommand}; +use atomic_core::kg::ontology::{compact, Class}; use atomic_core::pristine::vault::{KgEdge, KgNode}; use atomic_core::types::{Base32, Hash}; use atomic_repository::Repository; @@ -11,10 +12,13 @@ use atomic_repository::Repository; use crate::commands::{find_repository_root, Command}; use crate::error::{CliError, CliResult}; use crate::output::{create_spinner, finish_error, finish_success}; +mod validate; /// Subcommands for vault knowledge graph queries. #[derive(Subcommand, Debug)] pub enum QueryCommands { + /// Evaluate builtin and local enterprise SHACL shapes over a workflow snapshot. + Validate(validate::Validate), /// Search the knowledge graph by keywords. /// /// Performs full-text search over KG node IDs, labels, and summaries, @@ -205,6 +209,7 @@ pub struct Query { impl Command for Query { fn run(&self) -> CliResult<()> { match &self.command { + QueryCommands::Validate(cmd) => cmd.run(), QueryCommands::Search(cmd) => cmd.run(), QueryCommands::Neighbors(cmd) => cmd.run(), QueryCommands::Callers(cmd) => cmd.run(), @@ -294,19 +299,18 @@ impl Command for QueryGraph { } else { self.kinds .split(',') - .map(|s| s.trim().to_lowercase()) + .map(|s| s.trim().to_owned()) .filter(|s| !s.is_empty()) .collect() }; - let kind_allowed = - |kind: &str| -> bool { allow_all || allowed_kinds.contains(&kind.to_lowercase()) }; + let kind_allowed = |kind: &str| -> bool { allow_all || allowed_kinds.contains(kind) }; // 1. Seed: search the KG, filtered to allowed kinds let seed_nodes: Vec = repo .vault_kg_search(&self.query, self.limit * 3, None) // fetch extra, then filter .map_err(CliError::Repository)? .into_iter() - .filter(|n| kind_allowed(&n.kind)) + .filter(|n| n.types.iter().any(|class| kind_allowed(class))) .take(self.limit) .collect(); @@ -340,11 +344,11 @@ impl Command for QueryGraph { let mut changes_for_seed: usize = 0; for node in subgraph.nodes { - if !kind_allowed(&node.kind) { + if !node.types.iter().any(|class| kind_allowed(class)) { continue; } // Cap change nodes per seed - if node.kind == "change" && cap_changes { + if node.has_type(Class::Change) && cap_changes { if node_map.contains_key(&node.id) { continue; // already added by another seed, don't count } @@ -357,19 +361,17 @@ impl Command for QueryGraph { } for edge in subgraph.edges { - let from_kind_ok = allow_all || { - let prefix = edge.from_id.split(':').next().unwrap_or(""); - kind_allowed(prefix) - }; - let to_kind_ok = allow_all || { - let prefix = edge.to_id.split(':').next().unwrap_or(""); - kind_allowed(prefix) - }; + let from_kind_ok = node_map.contains_key(&edge.from_id); + let to_kind_ok = node_map.contains_key(&edge.to_id); if from_kind_ok && to_kind_ok { // Only add edge if both endpoints made it into node_map // (respects the per-seed change cap) if node_map.contains_key(&edge.from_id) || node_map.contains_key(&edge.to_id) { - let key = (edge.from_id.clone(), edge.to_id.clone(), edge.kind.clone()); + let key = ( + edge.from_id.clone(), + edge.to_id.clone(), + edge.kind.to_string(), + ); if edge_set.insert(key) { all_edges.push(edge); } @@ -511,7 +513,7 @@ pub(crate) fn emit_dot(nodes: &[KgNode], edges: &[KgEdge]) -> String { // Nodes for node in nodes { - let (fillcolor, shape) = node_style(&node.kind); + let (fillcolor, shape) = node_style(node.primary_type()); let label = node_label(node); out.push_str(&format!( " \"{}\" [label=\"{}\", fillcolor=\"{}\", shape={}];\n", @@ -530,7 +532,7 @@ pub(crate) fn emit_dot(nodes: &[KgNode], edges: &[KgEdge]) -> String { " \"{}\" -> \"{}\" [label=\"{}\"];\n", dot_escape_id(&edge.from_id), dot_escape_id(&edge.to_id), - dot_escape_label(&edge.kind), + dot_escape_label(&compact(edge.kind.iri())), )); } @@ -540,23 +542,25 @@ pub(crate) fn emit_dot(nodes: &[KgNode], edges: &[KgEdge]) -> String { /// Returns `(fillcolor, shape)` for a given node kind. fn node_style(kind: &str) -> (&'static str, &'static str) { - match kind.to_lowercase().as_str() { - "module" => ("#d4e6f1", "folder"), - "file" => ("#d5f5e3", "box"), - "entity" => ("#fdebd0", "ellipse"), - "change" => ("#fadbd8", "note"), - "view" => ("#e8daef", "pentagon"), - "identity" => ("#fef9e7", "box"), - "goal" => ("#d1f2eb", "hexagon"), - "intent" => ("#d1f2eb", "hexagon"), - "memory" => ("#fef9e7", "box"), + match Class::from_iri(kind) { + Some(Class::File) => ("#d5f5e3", "box"), + Some(Class::Change) => ("#fadbd8", "note"), + Some(Class::View) => ("#e8daef", "pentagon"), + Some( + Class::Goal + | Class::WorkIntent + | Class::FeatureIntent + | Class::BugIntent + | Class::ReviewIntent, + ) => ("#d1f2eb", "hexagon"), + Some(Class::Memory | Class::Identity) => ("#fef9e7", "box"), _ => ("#f2f3f4", "box"), } } /// Build a human-readable label for a node in DOT output. fn node_label(node: &KgNode) -> String { - let kind = node.kind.to_lowercase(); + let kind = compact(node.primary_type()); match kind.as_str() { "module" => { // Last path component + trailing slash @@ -567,7 +571,7 @@ fn node_label(node: &KgNode) -> String { .unwrap_or(&node.label); format!("{}/", last) } - "file" => { + "atomic:File" => { // Filename, optionally with match count let filename = node.label.rsplit('/').next().unwrap_or(&node.label); if let Some(ref meta) = node.metadata { @@ -594,7 +598,7 @@ fn node_label(node: &KgNode) -> String { } label } - "change" => { + "atomic:Change" => { // Short hash + optional summary excerpt let mut label = node.label.clone(); if let Some(ref summary) = node.summary { @@ -656,7 +660,8 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri .unwrap_or(0); serde_json::json!({ "id": n.id, - "kind": n.kind, + "kind": n.primary_type(), + "types": n.types, "label": n.label, "summary": n.summary.as_deref().unwrap_or(""), "content_matches": content_matches, @@ -735,33 +740,26 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri var graphData = {data_json}; var COLORS = {{ - module: '#3498db', - file: '#2ecc71', - entity: '#e67e22', - change: '#e74c3c', - view: '#9b59b6', - identity: '#f1c40f', - goal: '#1abc9c', - intent: '#1abc9c', - memory: '#f39c12', + 'https://atomic.dev/ns/workflow#File': '#2ecc71', + 'https://atomic.dev/ns/workflow#Change': '#e74c3c', + 'https://atomic.dev/ns/workflow#View': '#9b59b6', + 'https://atomic.dev/ns/workflow#Identity': '#f1c40f', + 'https://atomic.dev/ns/workflow#Goal': '#1abc9c', + 'https://atomic.dev/ns/workflow#WorkIntent': '#1abc9c', + 'https://atomic.dev/ns/workflow#Memory': '#f39c12', }}; var EDGE_COLORS = {{ - DEFINES: '#e67e22', - MODIFIES: '#e74c3c', - PART_OF: '#3498db', - INCLUDES: '#2ecc71', - AUTHORED_BY: '#f1c40f', - DEPENDS_ON: '#e74c3c', - ON_VIEW: '#9b59b6', - REFERENCES: '#95a5a6', + 'https://atomic.dev/ns/workflow#modifiesFile': '#e74c3c', + 'https://atomic.dev/ns/workflow#authoredBy': '#f1c40f', + 'https://atomic.dev/ns/workflow#directlyDependsOnChange': '#e74c3c', + 'https://atomic.dev/ns/workflow#directlyOnView': '#9b59b6', + 'https://atomic.dev/ns/workflow#references': '#95a5a6', }}; var DEFAULT_COLOR = '#7f8c8d'; function nodeRadius(d) {{ - if (d.kind === 'module') return 14 + Math.min(d.content_matches, 50) * 0.3; - if (d.kind === 'file') return 5 + Math.min(d.content_matches, 100) * 0.12; - if (d.kind === 'entity') return 4; - if (d.kind === 'change') return 4; + if (d.kind === 'https://atomic.dev/ns/workflow#File') return 5 + Math.min(d.content_matches, 100) * 0.12; + if (d.kind === 'https://atomic.dev/ns/workflow#Change') return 4; return 5; }} @@ -802,10 +800,10 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri var kindsPresent = new Set(nodes.map(function(n) {{ return n.kind; }})); var legendEl = document.getElementById('legend'); var legendHTML = '
Node Types
'; - var kindOrder = ['module', 'file', 'entity', 'change', 'view', 'identity', 'goal', 'intent', 'memory']; + var kindOrder = Array.from(kindsPresent).sort(); kindOrder.forEach(function(k) {{ if (kindsPresent.has(k)) {{ - legendHTML += '
' + k + '
'; + legendHTML += '
' + k.replace('https://atomic.dev/ns/workflow#', 'atomic:') + '
'; }} }}); legendEl.innerHTML = legendHTML; @@ -1105,10 +1103,10 @@ impl Command for QueryNodes { .map_err(CliError::Repository)?; let nodes: Vec<_> = if let Some(ref kind_filter) = self.kind { - let k = kind_filter.to_lowercase(); + let k = kind_filter.to_owned(); all_nodes .into_iter() - .filter(|n| n.kind.to_lowercase() == k) + .filter(|n| n.types.contains(&k)) .take(self.limit) .collect() } else { @@ -1127,7 +1125,12 @@ impl Command for QueryNodes { } else { format!(" {}", truncate_display(summary, 60)) }; - println!(" [{}] {}{}", node.kind, node.id, summary_display); + println!( + " [{}] {}{}", + compact(node.primary_type()), + node.id, + summary_display + ); } println!("\n{} result(s).", nodes.len()); } @@ -1177,7 +1180,12 @@ impl Command for QueryNeighbors { } else { format!(" {}", truncate_display(summary, 50)) }; - println!(" [{}] {}{}", node.kind, node.id, summary_display); + println!( + " [{}] {}{}", + compact(node.primary_type()), + node.id, + summary_display + ); } println!("\nEdges ({}):", subgraph.edges.len()); for edge in &subgraph.edges { @@ -1199,10 +1207,7 @@ impl Command for QueryNeighbors { /// Find all functions that call a given entity node. #[derive(Parser, Debug)] pub struct QueryCallers { - /// Full entity node ID, e.g. `entity:src/auth.rs:verify:42`. - /// - /// Use `atomic vault query entities ` to list entity IDs for a file, - /// then pass one here. + /// Symbol selector: `#` or a bare symbol name. pub entity_id: String, /// Output as JSON. @@ -1220,70 +1225,42 @@ impl Command for QueryCallers { let root = find_repository_root()?; let repo = Repository::open(&root).map_err(CliError::Repository)?; - let subgraph = repo - .vault_kg_neighbors(&self.entity_id, 1) + let callers = repo + .query_callers(&self.entity_id) .map_err(CliError::Repository)?; + render_callers(&self.entity_id, &callers, self.json) + } +} - // Incoming CALLS edges: from_id is the caller, to_id is our entity. - let caller_edges: Vec<&KgEdge> = subgraph - .edges - .iter() - .filter(|e| e.kind.to_uppercase() == "CALLS" && e.to_id == self.entity_id) - .collect(); - - if self.json { - let out: Vec = caller_edges - .iter() - .map(|e| { - let node = subgraph.nodes.iter().find(|n| n.id == e.from_id); - serde_json::json!({ - "caller_id": e.from_id, - "summary": node.and_then(|n| n.summary.as_deref()), - }) - }) - .collect(); - println!("{}", serde_json::to_string_pretty(&out).unwrap()); - return Ok(()); - } - - if caller_edges.is_empty() { - println!("No callers found for '{}'.", self.entity_id); - // Distinguish "no callers" from "CALLS edges not yet enriched" - let has_any_calls = subgraph - .edges - .iter() - .any(|e| e.kind.to_uppercase() == "CALLS"); - if !has_any_calls { - println!( - "Hint: run `atomic vault query enrich` to populate CALLS edges, \ - then retry." - ); - } else { - println!("This function is an entry point — nothing in the repo calls it."); - } - } else { - println!("{} caller(s) of {}:\n", caller_edges.len(), self.entity_id); - for edge in &caller_edges { - // Parse entity:{file}:{name}:{line} for a readable display - let display = if let Some(rest) = edge.from_id.strip_prefix("entity:") { - let last = rest.rfind(':').unwrap_or(rest.len()); - let inner = &rest[..last]; // {file}:{name} - let sig = subgraph - .nodes - .iter() - .find(|n| n.id == edge.from_id) - .and_then(|n| n.summary.as_deref()) - .map(|s| format!(" [{}]", s.lines().next().unwrap_or(s))); - format!(" {}{}", inner, sig.unwrap_or_default()) - } else { - format!(" {}", edge.from_id) - }; - println!("{display}"); - } +pub(crate) fn render_callers( + target: &str, + callers: &[atomic_repository::repository::Caller], + json: bool, +) -> CliResult<()> { + if json { + println!( + "{}", + serde_json::to_string_pretty(callers).map_err(|e| CliError::Internal(e.into()))? + ); + } else if callers.is_empty() { + println!("No callers found for '{target}'."); + } else { + println!("{} caller(s) of {target}:\n", callers.len()); + for caller in callers { + println!( + " {}#{}:{}{}", + caller.path, + caller.name, + caller.line, + caller + .summary + .as_ref() + .map(|summary| format!(" [{summary}]")) + .unwrap_or_default() + ); } - - Ok(()) } + Ok(()) } // --------------------------------------------------------------------------- @@ -1590,7 +1567,7 @@ mod enrich_tests { ) .unwrap(); let full_hash = outcome.hash().to_base32(); - let node_id = format!("change:{}", &full_hash[..12]); + let node_id = atomic_core::kg::iri::mint_change(outcome.hash()); assert!(repo.vault_kg_node(&node_id).unwrap().is_none()); drop(repo); @@ -1639,14 +1616,10 @@ impl Command for QueryEnrich { return Ok(()); } - // Clean rebuild: drop stale VCS/AST-derived nodes so files that became - // ignored or were deleted don't linger. Vault/session nodes are kept. + // Shared file/view resources also carry vault/session relationships; + // regenerate the whole read model rather than cascade-delete those joins. if self.rebuild { - let spinner = create_spinner("Clearing stale KG entries..."); - match repo.kg_clear_derived() { - Ok(n) => finish_success(&spinner, &format!("{} stale node(s) removed", n)), - Err(e) => finish_error(&spinner, &format!("clear failed: {}", e)), - } + repo.rebuild_kg(true).map_err(CliError::Repository)?; } let mut total = atomic_repository::KgEnrichStats::default(); @@ -1671,16 +1644,6 @@ impl Command for QueryEnrich { Err(e) => finish_error(&spinner, &format!("files failed: {}", e)), } - // Phase 3: Modules + PART_OF edges - let spinner = create_spinner("Enriching modules..."); - match repo.kg_enrich_modules(working_copy) { - Ok(n) => { - total.modules = n; - finish_success(&spinner, &format!("{} module(s)", n)); - } - Err(e) => finish_error(&spinner, &format!("modules failed: {}", e)), - } - // Phase 4: Changes let spinner = create_spinner("Enriching changes..."); match repo.kg_enrich_changes(working_copy) { @@ -1691,36 +1654,6 @@ impl Command for QueryEnrich { Err(e) => finish_error(&spinner, &format!("changes failed: {}", e)), } - // Phase 5: AST entities - let spinner = create_spinner("Extracting entities (tree-sitter)..."); - match repo.kg_enrich_entities(working_copy) { - Ok(n) => { - total.entities = n; - finish_success(&spinner, &format!("{} entit(ies)", n)); - } - Err(e) => finish_error(&spinner, &format!("entities failed: {}", e)), - } - - // Phase 6: INCLUDES edges - let spinner = create_spinner("Resolving includes..."); - match repo.kg_enrich_includes(working_copy) { - Ok(n) => { - total.includes = n; - finish_success(&spinner, &format!("{} include(s)", n)); - } - Err(e) => finish_error(&spinner, &format!("includes failed: {}", e)), - } - - // Phase 7: CALLS edges (caller entity → callee entity) - let spinner = create_spinner("Resolving call sites..."); - match repo.kg_enrich_calls(working_copy) { - Ok(n) => { - total.calls = n; - finish_success(&spinner, &format!("{} call(s)", n)); - } - Err(e) => finish_error(&spinner, &format!("calls failed: {}", e)), - } - // Phase 8: Content search index (syntext) let spinner = create_spinner("Building content index..."); match atomic_repository::build_content_index(&root) { @@ -1749,18 +1682,53 @@ impl Command for QueryEnrich { /// Rebuild the KG index from all vault entries. #[derive(Parser, Debug)] -pub struct Reindex; +pub struct Reindex { + /// Rebuild the complete workflow graph from canonical objects. + #[arg(long)] + pub full: bool, +} impl Command for Reindex { fn run(&self) -> CliResult<()> { + let root = find_repository_root()?; + // Key discovery precedes both local and daemon reindex execution. A + // missing remote or authentication leaves attestations Unverifiable. + { + let repo = Repository::open(&root).map_err(CliError::Repository)?; + if let Ok((_, remote)) = repo.get_default_remote() { + let runtime = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .map_err(|error| CliError::Internal(error.into()))?; + runtime.block_on(async { + let config = crate::commands::auth::attach_identity( + atomic_remote::HttpRemoteConfig::new(), + &remote.url, + None, + ) + .await; + if let Some(client) = + libatomic::signer_refresh::client_for_remote(&remote.url, &config) + { + if let Err(error) = libatomic::signer_refresh::refresh(&repo, &client).await + { + log::warn!("signer refresh: {error}"); + } + } + }); + } + } // Route every form through the service layer (the handler runs // the same vault_reindex_kg domain call). - if crate::commands::rpc::query_reindex()? { + if crate::commands::rpc::query_reindex(self.full)? { return Ok(()); } let root = find_repository_root()?; let repo = Repository::open(&root).map_err(CliError::Repository)?; + if self.full { + repo.rebuild_kg(true).map_err(CliError::Repository)?; + } let count = repo.vault_reindex_kg().map_err(CliError::Repository)?; println!("Indexed {} nodes + edges.", count); @@ -1831,7 +1799,7 @@ impl Command for QueryEmbed { /// Use `--json` for machine-readable output. /// /// Example plan: -/// {"steps":[{"type":"kg_search","query":"auth","limit":5,"bind":"results"}]} +/// {"schema_version":1,"steps":[{"type":"kg_search","query":"auth","limit":5,"bind":"results"}]} #[derive(Parser, Debug)] pub struct PlanExec { /// Output as JSON. @@ -1883,7 +1851,7 @@ impl Command for PlanExec { if !result.nodes.is_empty() { println!("\nNodes ({}):", result.nodes.len()); for node in &result.nodes { - print!(" [{}] {}", node.kind, node.label); + print!(" [{}] {}", compact(node.primary_type()), node.label); if let Some(ref s) = node.summary { // Truncate safely for Unicode text. let short: String = s.chars().take(60).collect(); @@ -1968,12 +1936,11 @@ Strategy: 5. Answer concisely using what you found. Cite file paths and line numbers. The knowledge graph contains: -- file nodes (file:path) — tracked files -- entity nodes (entity:file:name:line) — functions, structs, classes from tree-sitter -- change nodes (change:hash) — commit history with dates and messages -- view nodes (view:name) — like branches -- goal/intent/memory nodes — development sessions and work items -- edges: MODIFIES, DEFINES, AUTHORED_BY, DEPENDS_ON, REFERENCES, ON_VIEW, etc."; +- nodes with asserted class IRIs in https://atomic.dev/ns/workflow# (File, Change, View, WorkIntent, Goal, Memory) +- project-qualified resource IRIs and global full-hash change IRIs; copy IDs from search results +- predicates: atomic:modifiesFile, atomic:authoredBy, atomic:directlyDependsOnChange, atomic:references, atomic:directlyOnView, atomic:hasPlan, atomic:implements +- views filter membership in one common graph; directlyOnView is own membership, not inherited visibility +- source entities and callers are parsed live with list_entities/code_search, never stored in the workflow graph."; impl Command for QueryAsk { fn run(&self) -> CliResult<()> { diff --git a/atomic-cli/src/commands/query/validate.rs b/atomic-cli/src/commands/query/validate.rs new file mode 100644 index 00000000..9b8e9a9b --- /dev/null +++ b/atomic-cli/src/commands/query/validate.rs @@ -0,0 +1,100 @@ +use super::*; +use clap::ValueEnum; + +#[derive(Debug, Clone, Copy, Default, ValueEnum)] +pub enum Profile { + #[default] + Projection, + Publication, +} +#[derive(Debug, Clone, Copy, Default, ValueEnum)] +pub enum Format { + #[default] + Text, + Json, +} + +#[derive(Debug, Parser)] +pub struct Validate { + #[arg(long, value_enum, default_value = "projection")] + pub profile: Profile, + #[arg(long)] + pub shape: Option, + #[arg(long)] + pub node: Option, + #[arg(long, value_enum, default_value = "text")] + pub format: Format, +} + +impl Command for Validate { + fn run(&self) -> CliResult<()> { + // Validation deliberately runs in the CLI process so neither the + // daemon nor shared libraries acquire the SHACL engine's MSRV. + let repo = Repository::open(find_repository_root()?).map_err(CliError::Repository)?; + let snapshot = repo.workflow_snapshot().map_err(CliError::Repository)?; + let error = |error: atomic_shacl::ValidationError| CliError::ShaclError(error.to_string()); + let data = + atomic_shacl::snapshot_ntriples(&snapshot.nodes, &snapshot.edges).map_err(error)?; + let request = atomic_shacl::ValidationRequest { + profile: match self.profile { + Profile::Projection => atomic_shacl::Profile::Projection, + Profile::Publication => atomic_shacl::Profile::Publication, + }, + shape: self.shape.clone(), + node: self.node.clone(), + custom: atomic_shacl::load_custom_shapes(&repo.dot_dir().join("shapes")) + .map_err(error)?, + }; + let report = + atomic_shacl::validate_ntriples(&data, &snapshot.identity, &request).map_err(error)?; + match self.format { + Format::Json => println!( + "{}", + serde_json::to_string_pretty(&report) + .map_err(|error| CliError::Internal(error.into()))? + ), + Format::Text => { + println!( + "SHACL {:?}: {}", + report.profile, + if report.conforms { + "conforms" + } else { + "violations" + } + ); + println!( + "Snapshot: {}\nProfile: {}", + report.snapshot_identity, report.profile_identity + ); + for coverage in &report.target_coverage { + println!( + " {}: {} evaluated / {} applicable", + coverage.shape, coverage.evaluated, coverage.applicable + ); + } + for result in &report.results { + println!( + " {} {}: {} (path {}, shape {}, component {})", + result["result_severity"], + result["focus_node"], + result["result_messages"], + result["result_path"], + result["source_shape"], + result["source_constraint_component"] + ); + } + for diagnostic in &report.diagnostics { + println!(" {diagnostic}"); + } + } + } + if report.conforms { + Ok(()) + } else { + Err(CliError::ShaclViolation { + count: report.results.len(), + }) + } + } +} diff --git a/atomic-cli/src/commands/repo.rs b/atomic-cli/src/commands/repo.rs new file mode 100644 index 00000000..dba7e8bc --- /dev/null +++ b/atomic-cli/src/commands/repo.rs @@ -0,0 +1,69 @@ +//! Local repository metadata, independent of remote storage project management. + +use super::{find_repository_root, Command}; +use crate::error::{CliError, CliResult}; +use atomic_core::types::Base32; +use atomic_repository::Repository; +use clap::{Parser, Subcommand}; + +#[derive(Debug, Parser)] +pub struct Repo { + #[command(subcommand)] + command: RepoCommand, +} +#[derive(Debug, Subcommand)] +enum RepoCommand { + /// Inspect the stable local project identity. + Identity { + #[command(subcommand)] + command: IdentityCommand, + }, +} +#[derive(Debug, Subcommand)] +enum IdentityCommand { + Show { + #[arg(long)] + json: bool, + }, +} + +impl Command for Repo { + fn run(&self) -> CliResult<()> { + let repo = Repository::open(find_repository_root()?).map_err(CliError::Repository)?; + let record = repo + .project_identity() + .map_err(CliError::Repository)? + .ok_or_else(|| CliError::Internal(anyhow::anyhow!("MissingProjectIdentity")))?; + let project = record + .project_id() + .map_err(|e| CliError::Internal(e.into()))?; + let object: serde_json::Value = + serde_json::from_slice(&record.object).map_err(|e| CliError::Internal(e.into()))?; + let signed_by = object + .get("atomic:issuer") + .and_then(|v| v.get("@id")) + .and_then(|v| v.as_str()); + let minted = serde_json::to_value(record.mint_reason).expect("enum serialization"); + let json = matches!( + self.command, + RepoCommand::Identity { + command: IdentityCommand::Show { json: true } + } + ); + if json { + println!("{}", serde_json::to_string_pretty(&serde_json::json!({ + "project": project.to_base32(), "urn": project.to_urn(), "created": record.created_at, + "minted": minted, "signed_by": signed_by, "hash": record.hash.to_base32(), + })).expect("JSON serialization")); + } else { + println!( + "project: {project}\nurn: {}\ncreated: {}\nminted: {}\nsigned by: {}", + project.to_urn(), + record.created_at, + minted.as_str().unwrap_or_default(), + signed_by.unwrap_or("unsigned") + ); + } + Ok(()) + } +} diff --git a/atomic-cli/src/commands/rpc.rs b/atomic-cli/src/commands/rpc.rs index 4b05b89a..caee8184 100644 --- a/atomic-cli/src/commands/rpc.rs +++ b/atomic-cli/src/commands/rpc.rs @@ -145,7 +145,7 @@ fn memory_sources(bundle: &EntryBundle) -> crate::commands::memory::bridge::Atte fn kg_node_domain(node: &pb::KgNode) -> atomic_core::pristine::vault::KgNode { atomic_core::pristine::vault::KgNode { id: node.id.clone(), - kind: node.node_type.clone(), + types: node.types.clone(), label: node.name.clone().unwrap_or_default(), summary: node .data @@ -160,16 +160,21 @@ fn kg_node_domain(node: &pb::KgNode) -> atomic_core::pristine::vault::KgNode { } /// Wire KGEdge → the domain edge. -fn kg_edge_domain(edge: &pb::KgEdge) -> atomic_core::pristine::vault::KgEdge { - atomic_core::pristine::vault::KgEdge { +fn kg_edge_domain(edge: &pb::KgEdge) -> CliResult { + Ok(atomic_core::pristine::vault::KgEdge { from_id: edge.source.clone(), to_id: edge.target.clone(), - kind: edge.relation.clone(), + kind: atomic_core::kg::ontology::Predicate::from_iri(&edge.relation).ok_or_else(|| { + CliError::Internal(anyhow::anyhow!( + "unknown canonical predicate from service: {}", + edge.relation + )) + })?, metadata: edge .metadata .as_ref() .and_then(|metadata| serde_json::from_slice(metadata).ok()), - } + }) } pub(crate) fn request_meta() -> Option { @@ -1225,7 +1230,12 @@ pub fn query_search(args: &super::query::QueryNodes) -> CliResult { } else { format!(" {}", super::query::truncate_display(summary, 60)) }; - println!(" [{}] {}{}", node.kind, node.id, summary_display); + println!( + " [{}] {}{}", + atomic_core::kg::ontology::compact(node.primary_type()), + node.id, + summary_display + ); } println!("\n{} result(s).", nodes.len()); Ok(true) @@ -2650,7 +2660,7 @@ pub fn query_neighbors(args: &super::query::QueryNeighbors) -> CliResult { .edges .iter() .map(kg_edge_domain) - .collect::>(); + .collect::>>()?; if args.json { let subgraph = atomic_core::pristine::vault::KgSubgraph { nodes, edges }; println!("{}", serde_json::to_string_pretty(&subgraph).unwrap()); @@ -2668,7 +2678,12 @@ pub fn query_neighbors(args: &super::query::QueryNeighbors) -> CliResult { } else { format!(" {}", super::query::truncate_display(summary, 50)) }; - println!(" [{}] {}{}", node.kind, node.id, summary_display); + println!( + " [{}] {}{}", + atomic_core::kg::ontology::compact(node.primary_type()), + node.id, + summary_display + ); } println!("\nEdges ({}):", edges.len()); for edge in &edges { @@ -6103,7 +6118,7 @@ pub fn query_graph(args: &super::query::QueryGraph) -> CliResult { .edges .iter() .map(kg_edge_domain) - .collect::>(); + .collect::>>()?; if nodes.is_empty() { if args.json { println!("{{\"nodes\":[],\"edges\":[]}}"); @@ -6197,60 +6212,17 @@ pub fn query_callers(args: &super::query::QueryCallers) -> CliResult { view: None, }; let response = session.query_graph(request)?; - let caller_nodes = response - .nodes - .iter() - .map(kg_node_domain) - .collect::>(); - let caller_edges = response - .edges - .iter() - .map(kg_edge_domain) - .collect::>(); - - if args.json { - let out: Vec = caller_edges - .iter() - .map(|edge| { - let node = caller_nodes.iter().find(|node| node.id == edge.from_id); - serde_json::json!({ - "caller_id": edge.from_id, - "summary": node.and_then(|node| node.summary.as_deref()), - }) - }) - .collect(); - println!("{}", serde_json::to_string_pretty(&out).unwrap()); - return Ok(true); - } - - if caller_edges.is_empty() { - println!("No callers found for '{}'.", args.entity_id); - if !response.calls_present { - println!( - "Hint: run `atomic vault query enrich` to populate CALLS edges, \ - then retry." - ); - } else { - println!("This function is an entry point — nothing in the repo calls it."); - } - } else { - println!("{} caller(s) of {}:\n", caller_edges.len(), args.entity_id); - for edge in &caller_edges { - let display = if let Some(rest) = edge.from_id.strip_prefix("entity:") { - let last = rest.rfind(':').unwrap_or(rest.len()); - let inner = &rest[..last]; - let sig = caller_nodes - .iter() - .find(|node| node.id == edge.from_id) - .and_then(|node| node.summary.as_deref()) - .map(|summary| format!(" [{}]", summary)); - format!(" {}{}", inner, sig.unwrap_or_default()) - } else { - format!(" {}", edge.from_id) - }; - println!("{display}"); - } - } + let callers: Vec<_> = response + .callers + .into_iter() + .map(|caller| atomic_repository::repository::Caller { + path: caller.path, + name: caller.name, + line: caller.line, + summary: caller.summary, + }) + .collect(); + super::query::render_callers(&args.entity_id, &callers, args.json)?; Ok(true) } @@ -6305,7 +6277,11 @@ pub fn query_plan(args: &super::query::PlanExec) -> CliResult { if !result.nodes.is_empty() { println!("\nNodes ({}):", result.nodes.len()); for node in &result.nodes { - print!(" [{}] {}", node.kind, node.label); + print!( + " [{}] {}", + atomic_core::kg::ontology::compact(node.primary_type()), + node.label + ); if let Some(summary) = &node.summary { let short: String = summary.chars().take(60).collect(); print!(": {}", short); @@ -6439,7 +6415,7 @@ pub fn query_embed(args: &super::query::QueryEmbed) -> CliResult { /// `atomic query reindex` over MaintainKnowledgeGraph (REINDEX) — the /// handler runs vault_reindex_kg; the CLI prints the local report. -pub fn query_reindex() -> CliResult { +pub fn query_reindex(full: bool) -> CliResult { let Some(session) = Service::open()? else { return Ok(false); }; @@ -6447,7 +6423,7 @@ pub fn query_reindex() -> CliResult { repository: Some(session.reference.clone()), meta: request_meta(), action: pb::KnowledgeMaintainAction::Reindex as i32, - scope: None, + scope: full.then(|| "full".to_owned()), changes: Vec::new(), }; let response = session.maintain_knowledge_graph(request)?; diff --git a/atomic-cli/src/commands/triage/output.rs b/atomic-cli/src/commands/triage/output.rs index 0b7f9ce0..85159f89 100644 --- a/atomic-cli/src/commands/triage/output.rs +++ b/atomic-cli/src/commands/triage/output.rs @@ -211,7 +211,11 @@ pub fn walkthrough_text(report: &TriageReport) -> String { let files: Vec = layer .files .iter() - .map(|f| f.strip_prefix("file:").unwrap_or(f).to_string()) + .map(|f| { + atomic_core::kg::iri::parse_file(f) + .map(|(_, path)| path) + .unwrap_or_else(|_| f.clone()) + }) .collect(); out.push_str(&format!(" files: {}\n", capped_list(&files))); if !layer.criteria.is_empty() { @@ -695,10 +699,14 @@ fn render_walkthrough(report: &TriageReport) -> String { // The layer's code: per-file diffs scoped to this chapter's files, // pulled from the contributing changes' canonical diffs. - let layer_paths: std::collections::HashSet<&str> = layer + let layer_paths: std::collections::HashSet = layer .files .iter() - .map(|f| f.strip_prefix("file:").unwrap_or(f)) + .map(|f| { + atomic_core::kg::iri::parse_file(f) + .map(|(_, path)| path) + .unwrap_or_else(|_| f.clone()) + }) .collect(); for ch_id in &layer.changes { let Some(change) = report.changes.iter().find(|c| &c.id == ch_id) else { @@ -850,6 +858,7 @@ mod tests { /// A report fixture whose finding message carries HTML/JS metacharacters, /// so escaping can be asserted. fn sample_report() -> TriageReport { + let project = atomic_core::kg::project_id::ProjectId::from_base32(&"A".repeat(52)).unwrap(); TriageReport { reference: "urn:atomic:triage:abc123".to_string(), verdict: Verdict::Blocked, @@ -888,7 +897,7 @@ mod tests { changes: vec![ChangeReport { id: "HASH1".to_string(), message: "fix & taxes".to_string(), - modifies: vec!["file:src/.rs".to_string()], + modifies: vec![atomic_core::kg::iri::mint_file(&project, "src/.rs")], coverage: "uncovered".to_string(), files: vec![FileChange { symbol: "~".to_string(), @@ -932,7 +941,7 @@ mod tests { id: "layer:src".to_string(), title: "src".to_string(), rationale: "reads & foremost".to_string(), - files: vec!["file:src/.rs".to_string()], + files: vec![atomic_core::kg::iri::mint_file(&project, "src/.rs")], tasks: vec!["urn:atomic:task:T-1".to_string()], criteria: vec!["urn:atomic:ac:DEMO-A-1".to_string()], changes: vec!["HASH1".to_string()], @@ -942,7 +951,7 @@ mod tests { id: "layer:cli".to_string(), title: "cli".to_string(), rationale: "the entry point".to_string(), - files: vec!["file:cli/main.rs".to_string()], + files: vec![atomic_core::kg::iri::mint_file(&project, "cli/main.rs")], tasks: vec![], criteria: vec![], changes: vec!["HASH1".to_string()], diff --git a/atomic-cli/src/commands/vault/context.rs b/atomic-cli/src/commands/vault/context.rs index 5bf9c1b5..7e755d56 100644 --- a/atomic-cli/src/commands/vault/context.rs +++ b/atomic-cli/src/commands/vault/context.rs @@ -48,6 +48,7 @@ use std::collections::{HashMap, HashSet}; use clap::Parser; +use atomic_core::kg::{iri, project_id::ProjectId}; use atomic_core::pristine::tables::tokenize_for_fts; use atomic_core::pristine::vault::{KgNode, VaultEntry, VaultEntryType}; use atomic_core::types::{Base32, Hash}; @@ -248,7 +249,10 @@ impl Context { } for file in &self.files { - let node_id = format!("file:{}", file.trim_start_matches("./")); + let node_id = iri::mint_file( + &repo.project_id().map_err(CliError::Repository)?, + file.trim_start_matches("./"), + ); add_memory_neighbors(repo, &node_id, 1, &mut candidates)?; } @@ -256,7 +260,11 @@ impl Context { if !seed_query.trim().is_empty() { let pool = limit.saturating_mul(SEARCH_POOL_MULTIPLIER); let nodes = repo - .vault_kg_search_by_kind(&seed_query, pool, "memory") + .vault_kg_search_by_kind( + &seed_query, + pool, + atomic_core::kg::ontology::Class::Memory, + ) .map_err(CliError::Repository)?; for (rank, node) in nodes.iter().enumerate() { let base = rank_score(rank); @@ -287,7 +295,10 @@ impl Context { if !is_eligible_memory(&entry) { continue; } - if let Some(node_id) = memory_path_to_node_id(&meta.path) { + if let Some(node_id) = memory_path_to_node_id( + &repo.project_id().map_err(CliError::Repository)?, + &meta.path, + ) { merge_candidate(&mut candidates, &node_id, Some(meta.path), 0.0, false); } if candidates.len() >= limit { @@ -319,7 +330,10 @@ impl Context { .vault_list("memory/", Some(VaultEntryType::Memory)) .map_err(CliError::Repository)?; for meta in metas { - let Some(node_id) = memory_path_to_node_id(&meta.path) else { + let Some(node_id) = memory_path_to_node_id( + &repo.project_id().map_err(CliError::Repository)?, + &meta.path, + ) else { continue; }; let Some(entry) = repo @@ -386,9 +400,13 @@ impl Context { } } - let node_id = intent_path_to_node_id(&intent_path); - // Intent -> referenced file/domain -> memory is commonly two hops. - add_memory_neighbors(repo, &node_id, 2, candidates)?; + if let Ok(node_id) = iri::mint_intent( + &repo.project_id().map_err(CliError::Repository)?, + &summary.uid.to_uppercase(), + ) { + // Identity comes from the manifest, including old-layout entries. + add_memory_neighbors(repo, &node_id, 2, candidates)?; + } Ok(()) } @@ -470,7 +488,7 @@ fn add_memory_neighbors( .vault_kg_neighbors(node_id, depth) .map_err(CliError::Repository)?; for node in subgraph.nodes { - if node.kind == "memory" && node.id != node_id { + if node.has_type(atomic_core::kg::ontology::Class::Memory) && node.id != node_id { merge_candidate( candidates, &node.id, @@ -497,7 +515,7 @@ fn resolve_and_rank( let Some(path) = candidate .path .clone() - .or_else(|| legacy_memory_node_id_to_path(&node_id)) + .or_else(|| memory_node_id_to_path(&node_id)) else { continue; }; @@ -516,11 +534,11 @@ fn resolve_and_rank( let memory_id = frontmatter_memory_id(&entry.frontmatter_json).unwrap_or_else(|| node_id.clone()); let name = frontmatter_name(&entry.frontmatter_json).unwrap_or_else(|| { - node_id - .rsplit_once(':') - .map(|(_, n)| n) - .unwrap_or(&node_id) - .to_string() + std::path::Path::new(&path) + .file_stem() + .unwrap_or_default() + .to_string_lossy() + .into_owned() }); let recency = recency_score(&entry.updated_at, now); items.push(MemoryItem { @@ -641,12 +659,11 @@ fn memory_path_from_node(node: &KgNode) -> Option { .and_then(|path| path.as_str()) .filter(|path| path.starts_with("memory/") && path.ends_with(".md")) .map(String::from) - .or_else(|| legacy_memory_node_id_to_path(&node.id)) + .or_else(|| memory_node_id_to_path(&node.id)) } -/// Legacy fallback: `memory:architecture` -> `memory/architecture.md`. -fn legacy_memory_node_id_to_path(node_id: &str) -> Option { - let name = node_id.strip_prefix("memory:")?; +fn memory_node_id_to_path(node_id: &str) -> Option { + let (_, name) = iri::parse_memory(node_id).ok()?; if name.is_empty() { return None; } @@ -654,23 +671,12 @@ fn legacy_memory_node_id_to_path(node_id: &str) -> Option { } /// `memory/architecture.md` -> `memory:architecture`. -fn memory_path_to_node_id(path: &str) -> Option { +fn memory_path_to_node_id(project: &ProjectId, path: &str) -> Option { let name = path.strip_prefix("memory/")?.strip_suffix(".md")?; if name.is_empty() { return None; } - Some(format!("memory:{}", name)) -} - -/// `intents/pimo-1/intent.md` -> `intent:PIMO-1` (nested paths keep -/// their inner segments, uppercased, matching the KG indexer). -fn intent_path_to_node_id(path: &str) -> String { - let id = path - .strip_prefix("intents/") - .and_then(|s| s.strip_suffix("/intent.md")) - .unwrap_or(path) - .to_uppercase(); - format!("intent:{}", id) + Some(iri::mint_memory(project, name)) } // Frontmatter helpers @@ -1061,20 +1067,24 @@ mod tests { } #[test] - fn test_legacy_memory_node_id_to_path() { + fn test_memory_node_id_to_path() { + let project = ProjectId::generate(); assert_eq!( - legacy_memory_node_id_to_path("memory:architecture"), + memory_node_id_to_path(&iri::mint_memory(&project, "architecture")), Some("memory/architecture.md".to_string()) ); - assert_eq!(legacy_memory_node_id_to_path("memory:"), None); - assert_eq!(legacy_memory_node_id_to_path("file:src/main.rs"), None); + assert_eq!(memory_node_id_to_path("memory:"), None); + assert_eq!( + memory_node_id_to_path(&iri::mint_file(&project, "src/main.rs")), + None + ); } #[test] fn test_memory_path_from_node_prefers_metadata() { let node = KgNode::new( "urn:atomic:memory:01J8ZC4R8T", - "memory", + atomic_core::kg::ontology::Class::Memory, "architecture", "vault", ) @@ -1087,28 +1097,14 @@ mod tests { #[test] fn test_memory_path_to_node_id() { + let project = ProjectId::generate(); assert_eq!( - memory_path_to_node_id("memory/architecture.md"), - Some("memory:architecture".to_string()) - ); - assert_eq!(memory_path_to_node_id("intents/x/intent.md"), None); - } - - #[test] - fn test_intent_path_to_node_id_flat() { - assert_eq!( - intent_path_to_node_id("intents/pimo-1/intent.md"), - "intent:PIMO-1" + memory_path_to_node_id(&project, "memory/architecture.md"), + Some(iri::mint_memory(&project, "architecture")) ); - } - - #[test] - fn test_intent_path_to_node_id_nested() { - // Hook-created intents nest view/session/turn segments; the KG - // indexer keeps them (uppercased) in the node id. assert_eq!( - intent_path_to_node_id("intents/tight-storm/019e/1/intent.md"), - "intent:TIGHT-STORM/019E/1" + memory_path_to_node_id(&project, "intents/x/intent.md"), + None ); } @@ -1623,7 +1619,10 @@ mod tests { let items = resolve_and_rank(&repo, candidates, 5, true).unwrap(); assert_eq!(items.len(), 1); assert_eq!(items[0].memory_id, "urn:atomic:memory:auth-decision"); - assert_eq!(items[0].kg_node_id, "memory:auth-decision"); + assert_eq!( + items[0].kg_node_id, + iri::mint_memory(&repo.project_id().unwrap(), "auth-decision") + ); assert!(items[0].body.contains("RS256")); } @@ -1663,9 +1662,8 @@ mod tests { }; let candidates = request.gather_candidates(&repo, 5).unwrap(); let items = resolve_and_rank(&repo, candidates, 5, true).unwrap(); - assert!(items - .iter() - .any(|item| item.kg_node_id == "memory:auth-decision")); + assert!(items.iter().any(|item| item.kg_node_id + == iri::mint_memory(&repo.project_id().unwrap(), "auth-decision"))); } #[test] @@ -1695,7 +1693,8 @@ mod tests { let items = resolve_and_rank(&repo, candidates, 5, true).unwrap(); assert!(items .iter() - .any(|item| item.kg_node_id == "memory:auth-file")); + .any(|item| item.kg_node_id + == iri::mint_memory(&repo.project_id().unwrap(), "auth-file"))); } #[test] diff --git a/atomic-cli/src/error.rs b/atomic-cli/src/error.rs index ac3dc41e..7edbc9b6 100644 --- a/atomic-cli/src/error.rs +++ b/atomic-cli/src/error.rs @@ -55,6 +55,10 @@ pub type CliResult = Result; /// - **Internal errors**: Unexpected failures (bugs) #[derive(Debug, Error)] pub enum CliError { + #[error("SHACL validation found {count} result(s) that prevent conformance")] + ShaclViolation { count: usize }, + #[error("SHACL validation error: {0}")] + ShaclError(String), // Repository Errors /// No Atomic repository found in the current directory or any parent. /// @@ -680,6 +684,8 @@ impl CliError { /// The exit code to use when terminating due to this error. pub fn exit_code(&self) -> i32 { match self { + Self::ShaclViolation { .. } => 1, + Self::ShaclError(_) => 3, // User-fixable, general errors Self::NothingToRecord | Self::Cancelled diff --git a/atomic-cli/src/main.rs b/atomic-cli/src/main.rs index 4c600d6d..2b0acd73 100644 --- a/atomic-cli/src/main.rs +++ b/atomic-cli/src/main.rs @@ -94,6 +94,7 @@ use commands::{ Record, Remote, Remove, + Repo, Restore, Revise, Sandbox, @@ -659,6 +660,9 @@ enum Commands { /// ``` Project(ProjectCmd), + /// Inspect local repository identity and metadata. + Repo(Repo), + /// Manage organizations and members. /// /// Organizations group teams, workspaces, and projects. Each user has @@ -1061,6 +1065,7 @@ fn main() { Commands::Workspace(workspace) => workspace.run(), Commands::Project(project) => project.run(), + Commands::Repo(repo) => repo.run(), #[cfg(feature = "teams")] Commands::Org(org) => org.run(), diff --git a/atomic-cli/src/service/mod.rs b/atomic-cli/src/service/mod.rs index 934722ae..9be82e6a 100644 --- a/atomic-cli/src/service/mod.rs +++ b/atomic-cli/src/service/mod.rs @@ -1065,7 +1065,7 @@ impl Service { impl Service { pub fn query_graph(&self, request: pb::QueryGraphRequest) -> CliResult { - self.call(move |backend| async move { + let response = self.call(move |backend| async move { match backend { Backend::Local(state) => services_agent::KnowledgeImpl { state } .query_graph(Request::new(request)) @@ -1080,7 +1080,14 @@ impl Service { .map(|response| response.into_inner()) } } - }) + })?; + if response.api_version != 1 { + return Err(crate::error::CliError::Internal(anyhow::anyhow!( + "unsupported knowledge graph API version {}; expected 1", + response.api_version + ))); + } + Ok(response) } pub fn maintain_knowledge_graph( diff --git a/atomic-cli/tests/session_integration_test.rs b/atomic-cli/tests/session_integration_test.rs index 4d844b75..865644d7 100644 --- a/atomic-cli/tests/session_integration_test.rs +++ b/atomic-cli/tests/session_integration_test.rs @@ -16,7 +16,10 @@ use std::process::{Command, Output}; use atomic_core::change::provenance_graph::{ProvenanceNode, ProvenanceNodeKind}; use atomic_core::change::session::SessionTodo; use atomic_core::change::ProvenanceGraph; -use atomic_core::pristine::ontology::{edge_kind, entity_type, predicate}; +use atomic_core::kg::{ + iri, + ontology::{Class as C, Predicate as P}, +}; use atomic_core::pristine::vault::{KgEdge, KgNode}; use atomic_core::pristine::{KgMutTxnT, KgTxnT}; use atomic_core::types::{Base32, Hash}; @@ -138,35 +141,22 @@ fn session_order_and_manifest_do_not_depend_on_ingestion_order() { repo_a.save_provenance_graph(&child).unwrap(); let repo_b = Repository::open(b_tmp.path()).unwrap(); + let project = repo_b.project_id().unwrap(); + let incoming = iri::mint_memory(&project, "incoming-link"); + let outgoing = iri::mint_memory(&project, "outgoing-link"); + let turn0 = iri::mint_turn(&project, "ordered-session", 0); + let turn1 = iri::mint_turn(&project, "ordered-session", 1); repo_b.save_provenance_graph(&child).unwrap(); { let mut txn = repo_b.pristine().write_txn().unwrap(); - txn.upsert_kg_node(&KgNode::new( - "memory:incoming-link", - "memory", - "incoming", - "vault", - )) - .unwrap(); - txn.upsert_kg_node(&KgNode::new( - "memory:outgoing-link", - "memory", - "outgoing", - "vault", - )) - .unwrap(); - txn.upsert_kg_edge(&KgEdge::new( - "memory:incoming-link", - "session:ordered-session/turn:0", - "extension:references", - )) - .unwrap(); - txn.upsert_kg_edge(&KgEdge::new( - "session:ordered-session/turn:0", - "memory:outgoing-link", - "extension:produced", - )) - .unwrap(); + txn.upsert_kg_node(&KgNode::new(&incoming, C::Memory, "incoming", "vault")) + .unwrap(); + txn.upsert_kg_node(&KgNode::new(&outgoing, C::Memory, "outgoing", "vault")) + .unwrap(); + txn.upsert_kg_edge(&KgEdge::new(&incoming, &turn0, P::References)) + .unwrap(); + txn.upsert_kg_edge(&KgEdge::new(&turn0, &outgoing, P::References)) + .unwrap(); txn.commit().unwrap(); } repo_b.save_provenance_graph(&parent).unwrap(); @@ -183,30 +173,22 @@ fn session_order_and_manifest_do_not_depend_on_ingestion_order() { assert_eq!(turns_b[1].provenance_hash, child_hash); let txn = repo_b.pristine().read_txn().unwrap(); - let child_edges = txn - .get_kg_edges_from("session:ordered-session/turn:1") - .unwrap(); - assert!(child_edges.iter().any(|edge| { - edge.kind == predicate::WAS_INFORMED_BY && edge.to_id == "session:ordered-session/turn:0" - })); - let parent_edges = txn - .get_kg_edges_from("session:ordered-session/turn:0") - .unwrap(); + let child_edges = txn.get_kg_edges_from(&turn1).unwrap(); + assert!(child_edges + .iter() + .any(|edge| { edge.kind == P::ProvWasInformedBy && edge.to_id == turn0 })); + let parent_edges = txn.get_kg_edges_from(&turn0).unwrap(); assert!(!parent_edges .iter() - .any(|edge| edge.kind == predicate::WAS_INFORMED_BY)); + .any(|edge| edge.kind == P::ProvWasInformedBy)); assert!(child_edges .iter() - .any(|edge| { edge.kind == "extension:produced" && edge.to_id == "memory:outgoing-link" })); - let child_incoming = txn - .get_kg_edges_to("session:ordered-session/turn:1") - .unwrap(); - assert!(child_incoming.iter().any(|edge| { - edge.kind == "extension:references" && edge.from_id == "memory:incoming-link" - })); - assert!(!parent_edges + .any(|edge| { edge.kind == P::References && edge.to_id == outgoing })); + let child_incoming = txn.get_kg_edges_to(&turn1).unwrap(); + assert!(child_incoming .iter() - .any(|edge| edge.kind.starts_with("extension:"))); + .any(|edge| { edge.kind == P::References && edge.from_id == incoming })); + assert!(!parent_edges.iter().any(|edge| edge.kind == P::References)); } #[test] @@ -298,24 +280,21 @@ fn session_rebuild_is_idempotent_and_tolerates_corruption() { assert_eq!(turns.len(), 2, "rebuild must not duplicate turns"); let mut txn = repo.pristine().write_txn().unwrap(); - txn.del_kg_edge( - "session:rebuild/turn:1", - "session:rebuild/turn:0", - predicate::WAS_INFORMED_BY, - ) - .unwrap(); + let project = repo.project_id().unwrap(); + let turn0 = iri::mint_turn(&project, "rebuild", 0); + let turn1 = iri::mint_turn(&project, "rebuild", 1); + txn.del_kg_edge(&turn1, &turn0, P::ProvWasInformedBy.iri()) + .unwrap(); txn.commit().unwrap(); let (indexed, skipped, corrupt) = repo.rebuild_session_index().expect("repair rebuild"); assert_eq!((indexed, skipped, corrupt), (0, 2, 0)); let txn = repo.pristine().read_txn().unwrap(); assert!(txn - .get_kg_edges_from("session:rebuild/turn:1") + .get_kg_edges_from(&turn1) .unwrap() .iter() - .any(|edge| { - edge.kind == predicate::WAS_INFORMED_BY && edge.to_id == "session:rebuild/turn:0" - })); + .any(|edge| { edge.kind == P::ProvWasInformedBy && edge.to_id == turn0 })); } // A corrupt provenance file is reported, not fatal. @@ -495,7 +474,13 @@ fn session_rebuild_restores_a_fork_from_its_manifest() { assert_eq!(empty_record.turn_count, 0); assert!(empty_turns.is_empty()); let txn = repo_b.pristine().read_txn().unwrap(); - assert!(txn.get_kg_node("session:empty-child").unwrap().is_some()); + assert!(txn + .get_kg_node(&iri::mint_session( + &repo_b.project_id().unwrap(), + "empty-child" + )) + .unwrap() + .is_some()); drop(txn); let empty_head = repo_b.get_session_head("empty-child").unwrap().unwrap(); let empty_rebuilt = repo_b.get_session_manifest(&empty_head).unwrap().unwrap(); @@ -604,7 +589,14 @@ fn session_ledger_emits_rdf_taxonomy() { let home_dir = home_tmp.path(); init_repo(repo_dir, home_dir); - let repo = Repository::open(repo_dir).expect("open repo"); + let mut repo = Repository::open(repo_dir).expect("open repo"); + let parent = repo.current_view().to_owned(); + repo.create_draft_view("agent-kg-session", &parent).unwrap(); + let project = repo.project_id().unwrap(); + let session_id = iri::mint_session(&project, "kg-session"); + let turn0_id = iri::mint_turn(&project, "kg-session", 0); + let turn1_id = iri::mint_turn(&project, "kg-session", 1); + let todo_id = iri::mint_todo(&project, "kg-session", "todo-rdf"); let change0 = Hash::of(&[1u8]); let p0 = repo .save_provenance_graph( @@ -631,7 +623,7 @@ fn session_ledger_emits_rdf_taxonomy() { .expect("save contextual provenance"); let _p1 = save_turn_graph(&repo, "kg-session", "kg turn one", 2, Some(p0)); - // Lifecycle: view metadata + status ride the session node, ON_VIEW edge. + // Session view context is distinct from change membership. repo.upsert_session_lifecycle( "kg-session", Some("agent-kg-session".into()), @@ -644,69 +636,64 @@ fn session_ledger_emits_rdf_taxonomy() { // Session node: typed, lifecycle metadata. let session = txn - .get_kg_node("session:kg-session") + .get_kg_node(&session_id) .expect("kg read") .expect("session node"); let meta = session.metadata.as_ref().expect("session metadata"); - assert_eq!(meta["rdf_type"], entity_type::SESSION); + assert!(session.has_type(C::Session)); + assert!(meta.get("rdf_type").is_none()); assert_eq!(meta["view"], "agent-kg-session"); assert_eq!(meta["status"], "active"); // Turn + provenance nodes typed; the taxonomy chains present. let turn1 = txn - .get_kg_node("session:kg-session/turn:1") + .get_kg_node(&turn1_id) .expect("kg read") .expect("turn node"); - assert_eq!( - turn1.metadata.as_ref().unwrap()["rdf_type"], - entity_type::SESSION_TURN - ); + assert!(turn1.has_type(C::SessionTurn)); - let session_edges = txn - .get_kg_edges_from("session:kg-session") - .expect("kg read"); - assert!(session_edges - .iter() - .any(|e| e.kind == predicate::HAD_MEMBER && e.to_id == "session:kg-session/turn:0")); + let session_edges = txn.get_kg_edges_from(&session_id).expect("kg read"); assert!(session_edges .iter() - .any(|e| e.kind == edge_kind::ON_VIEW && e.to_id == "view:agent-kg-session")); - assert!(session_edges + .any(|e| e.kind == P::HasTurn && e.to_id == turn0_id)); + assert!(session_edges.iter().any(|e| e.kind == P::SessionViewContext + && e.to_id == iri::mint_view(&project, "agent-kg-session"))); + assert!(!session_edges .iter() - .any(|e| e.kind == predicate::HAD_PLAN && e.to_id == "intent:ATOM-20")); + .any(|e| matches!(e.kind, P::HasPlan | P::DirectlyOnView))); - let turn0_edges = txn - .get_kg_edges_from("session:kg-session/turn:0") - .expect("kg read"); - assert!(turn0_edges.iter().any(|e| e.kind == predicate::EXPLAINED_BY - && e.to_id == format!("provenance:{}", p0.to_base32()))); - let change0_short = &change0.to_base32()[..12]; + let turn0_edges = txn.get_kg_edges_from(&turn0_id).expect("kg read"); assert!(turn0_edges .iter() - .any(|e| e.kind == predicate::GENERATED && e.to_id == format!("change:{}", change0_short))); + .any(|e| e.kind == P::TurnExplainedBy && e.to_id == iri::mint_provenance(&p0))); + assert!(turn0_edges + .iter() + .any(|e| e.kind == P::GeneratedChange && e.to_id == iri::mint_change(&change0))); + // An unresolvable historical plan is a gap, never a guessed intent IRI. + assert!(!turn0_edges.iter().any(|e| e.kind == P::HasPlan)); + assert!(txn + .get_kg_node(&iri::mint_finding( + &project, + "UnresolvedReference", + &turn0_id + )) + .unwrap() + .is_some()); assert!(turn0_edges .iter() - .any(|e| e.kind == predicate::HAD_PLAN && e.to_id == "intent:ATOM-20")); - assert!(turn0_edges.iter().any(|e| { - e.kind == predicate::HAS_TODO && e.to_id == "session:kg-session/todo:todo-rdf" - })); + .any(|e| { e.kind == P::HasTodo && e.to_id == todo_id })); let todo = txn - .get_kg_node("session:kg-session/todo:todo-rdf") + .get_kg_node(&todo_id) .expect("kg read") .expect("todo node"); - assert_eq!( - todo.metadata.as_ref().unwrap()["rdf_type"], - entity_type::TODO - ); + assert!(todo.has_type(C::Todo)); assert_eq!(todo.metadata.as_ref().unwrap()["status"], "completed"); // Turn 1 informed by turn 0. - let turn1_edges = txn - .get_kg_edges_from("session:kg-session/turn:1") - .expect("kg read"); + let turn1_edges = txn.get_kg_edges_from(&turn1_id).expect("kg read"); assert!(turn1_edges .iter() - .any(|e| e.kind == predicate::WAS_INFORMED_BY && e.to_id == "session:kg-session/turn:0")); + .any(|e| e.kind == P::ProvWasInformedBy && e.to_id == turn0_id)); drop(txn); @@ -715,10 +702,12 @@ fn session_ledger_emits_rdf_taxonomy() { .fork_session("kg-session", 1, "kg-child") .expect("fork"); let txn = repo.pristine().read_txn().expect("read txn"); - let child_edges = txn.get_kg_edges_from("session:kg-child").expect("kg read"); + let child_edges = txn + .get_kg_edges_from(&iri::mint_session(&project, "kg-child")) + .expect("kg read"); let fork_edge = child_edges .iter() - .find(|e| e.kind == predicate::WAS_DERIVED_FROM && e.to_id == "session:kg-session") + .find(|e| e.kind == P::ProvWasDerivedFrom && e.to_id == session_id) .expect("fork edge"); let fork_meta = fork_edge.metadata.as_ref().expect("fork metadata"); assert_eq!(fork_meta["fork_turn"], 1); @@ -729,30 +718,20 @@ fn session_ledger_emits_rdf_taxonomy() { // Inherited prefix emits under the child session too. assert!(txn - .get_kg_node("session:kg-child/turn:1") + .get_kg_node(&iri::mint_turn(&project, "kg-child", 1)) .expect("kg read") .is_some()); drop(txn); - // Simulate the old global todo representation on an inherited-only child. + // Repair a missing derived todo on an inherited-only child, preserving + // an unrelated resource owned by a different producer. let mut txn = repo.pristine().write_txn().expect("write txn"); - txn.del_kg_node("session:kg-child/todo:todo-rdf").unwrap(); - txn.upsert_kg_node(&KgNode::new( - "todo:todo-rdf", - "todo", - "Emit RDF plan and todo links", - "session_ledger", - )) - .unwrap(); - txn.upsert_kg_edge(&KgEdge::new( - "session:kg-child/turn:0", - "todo:todo-rdf", - predicate::HAS_TODO, - )) - .unwrap(); + txn.del_kg_node(&iri::mint_todo(&project, "kg-child", "todo-rdf")) + .unwrap(); + let unrelated_id = iri::mint_todo(&project, "external-session", "external-collision"); txn.upsert_kg_node(&KgNode::new( - "todo:external-collision", - "todo", + &unrelated_id, + C::Todo, "Owned by another index", "external_index", )) @@ -761,13 +740,12 @@ fn session_ledger_emits_rdf_taxonomy() { repo.rebuild_session_index().expect("rebuild"); let txn = repo.pristine().read_txn().expect("read txn"); - assert!(txn.get_kg_node("todo:todo-rdf").unwrap().is_none()); assert!(txn - .get_kg_node("session:kg-child/todo:todo-rdf") + .get_kg_node(&iri::mint_todo(&project, "kg-child", "todo-rdf")) .unwrap() .is_some()); let unrelated = txn - .get_kg_node("todo:external-collision") + .get_kg_node(&unrelated_id) .unwrap() .expect("unrelated colliding node must survive"); assert_eq!(unrelated.source, "external_index"); @@ -801,12 +779,13 @@ fn todo_ids_are_scoped_to_their_session() { } let txn = repo.pristine().read_txn().unwrap(); + let project = repo.project_id().unwrap(); let first = txn - .get_kg_node("session:todo-a/todo:t-1") + .get_kg_node(&iri::mint_todo(&project, "todo-a", "t-1")) .unwrap() .expect("first session todo"); let second = txn - .get_kg_node("session:todo-b/todo:t-1") + .get_kg_node(&iri::mint_todo(&project, "todo-b", "t-1")) .unwrap() .expect("second session todo"); assert_eq!(first.label, "First session"); diff --git a/atomic-core/Cargo.toml b/atomic-core/Cargo.toml index dd2dc671..11778709 100644 --- a/atomic-core/Cargo.toml +++ b/atomic-core/Cargo.toml @@ -58,6 +58,10 @@ similar.workspace = true quickcheck.workspace = true quickcheck_macros.workspace = true tempfile.workspace = true +oxrdf = "=0.2.4" +oxttl = "=0.1.8" +proc-macro2 = "1" +syn = "2" [features] default = [] diff --git a/atomic-core/ontology/shapes/projection-1.0.0.ttl b/atomic-core/ontology/shapes/projection-1.0.0.ttl new file mode 100644 index 00000000..f0aa5bc1 --- /dev/null +++ b/atomic-core/ontology/shapes/projection-1.0.0.ttl @@ -0,0 +1,63 @@ +@prefix atomic: . +@prefix shape: . +@prefix sh: . +@prefix rdf: . +@prefix prov: . + +shape:GeneratedChange a sh:NodeShape ; + sh:targetObjectsOf atomic:generatedChange ; + sh:class atomic:Change ; + sh:and ( + [ sh:or ( + [ sh:property [ sh:path atomic:implements ; sh:minCount 1 ] ] + [ sh:property [ sh:path ( [ sh:inversePath atomic:aboutResource ] atomic:findingCode ) ; sh:hasValue "MissingPlanId" ] ] + ) ] + [ sh:or ( + [ sh:property [ sh:path atomic:changeExplainedBy ; sh:minCount 1 ] ] + [ sh:property [ sh:path ( [ sh:inversePath atomic:aboutResource ] atomic:findingCode ) ; sh:hasValue "MissingProvenance" ] ] + ) ] + [ sh:or ( + [ sh:property [ sh:path atomic:attestedBy ; sh:minCount 1 ] ] + [ sh:property [ sh:path ( [ sh:inversePath atomic:aboutResource ] atomic:findingCode ) ; sh:hasValue "MissingAttestation" ] ] + ) ] + ) ; + sh:sparql [ + sh:message "Change shortcuts must equal the authoritative turn/attestation paths." ; + sh:select """ + PREFIX atomic: + SELECT $this WHERE { + { $this atomic:implements ?intent . FILTER NOT EXISTS { ?turn atomic:generatedChange $this ; atomic:hasPlan ?intent } } + UNION { ?turn atomic:generatedChange $this ; atomic:hasPlan ?intent . FILTER NOT EXISTS { $this atomic:implements ?intent } } + UNION { $this atomic:changeExplainedBy ?graph . FILTER NOT EXISTS { ?turn atomic:generatedChange $this ; atomic:turnExplainedBy ?graph } } + UNION { ?turn atomic:generatedChange $this ; atomic:turnExplainedBy ?graph . FILTER NOT EXISTS { $this atomic:changeExplainedBy ?graph } } + UNION { $this atomic:attestedBy ?attestation . FILTER NOT EXISTS { ?attestation atomic:coversChange $this } } + UNION { ?attestation atomic:coversChange $this . FILTER NOT EXISTS { $this atomic:attestedBy ?attestation } } + } + """ + ] . + +shape:ReferencedResource a sh:NodeShape ; + sh:targetObjectsOf atomic:hasTurn, atomic:hasPlan, atomic:generatedChange, + atomic:informedBy, atomic:turnExplainedBy, atomic:changeExplainedBy, + atomic:hasTodo, atomic:blocks, atomic:blockedBy, atomic:assignedTo, + atomic:linkedIntent, atomic:linkedGoal, atomic:references, atomic:hasLabel, + atomic:hasTask, atomic:hasAcceptanceCriterion, atomic:hasScopeIn, + atomic:hasScopeOut, atomic:hasConstraint, atomic:satisfies, atomic:touches, + atomic:scopeOutFile, atomic:remediates, atomic:reviews, + atomic:directlyDependsOnChange, atomic:dependsOnIntent, atomic:modifiesFile, + atomic:directlyOnView, atomic:sessionViewContext, atomic:childViewOf, + atomic:authoredBy, atomic:coversChange, atomic:attests, atomic:attestsSession, + atomic:signedBy, atomic:previousAttestation, atomic:implements, + atomic:attestedBy, atomic:aboutResource, atomic:verificationResult, + atomic:hasIdentity, atomic:issuer, atomic:explainedBy, atomic:dependsOn, + atomic:modifies, atomic:hasScopeItem, atomic:ancestorView, + atomic:turnOfSession, atomic:governsTurn, atomic:generatedBy, atomic:reviewedBy, + prov:wasDerivedFrom, prov:used, prov:wasInformedBy, prov:generated, prov:wasAttributedTo ; + sh:nodeKind sh:IRI ; + sh:property [ sh:path rdf:type ; sh:minCount 1 ; sh:message "Every referenced canonical resource must exist with an asserted type." ] . + +shape:FullHashIdentity a sh:NodeShape ; + sh:targetClass atomic:Change, atomic:ProvenanceGraph, atomic:Attestation, atomic:ProjectIdentity ; + sh:nodeKind sh:IRI ; + sh:pattern "^urn:atomic:(change|provenance|attestation|project-identity):blake3-256:[A-Z2-7]{51}[AQ]$" ; + sh:message "Durable content identities require the full canonical Blake3 hash." . diff --git a/atomic-core/ontology/shapes/publication-1.0.0.ttl b/atomic-core/ontology/shapes/publication-1.0.0.ttl new file mode 100644 index 00000000..8d6ca01c --- /dev/null +++ b/atomic-core/ontology/shapes/publication-1.0.0.ttl @@ -0,0 +1,33 @@ +@prefix atomic: . +@prefix shape: . +@prefix sh: . + +shape:ManagedTurn a sh:NodeShape ; + sh:targetClass atomic:ManagedTurn ; + sh:property [ sh:path atomic:hasPlan ; sh:minCount 1 ; sh:maxCount 1 ; sh:class atomic:WorkIntent ] . + +shape:ManagedIntent a sh:NodeShape ; + sh:targetClass atomic:ManagedIntent ; + sh:property [ sh:path [ sh:inversePath atomic:hasPlan ] ; sh:minCount 1 ; sh:class atomic:SessionTurn ] . + +shape:TurnProvenance a sh:NodeShape ; + sh:targetClass atomic:SessionTurn ; + sh:property [ sh:path atomic:turnExplainedBy ; sh:minCount 1 ; sh:maxCount 1 ; sh:class atomic:ProvenanceGraph ] . + +shape:SessionCoverage a sh:NodeShape ; + sh:targetObjectsOf atomic:generatedChange ; + sh:sparql [ + sh:message "Every generated change must be covered by an attestation for its generating session." ; + sh:select """ + PREFIX atomic: + SELECT $this WHERE { + ?turn atomic:generatedChange $this . ?session atomic:hasTurn ?turn . + FILTER NOT EXISTS { ?attestation atomic:attestsSession ?session ; atomic:coversChange $this } + } + """ + ] . + +shape:VerifiedAttestation a sh:NodeShape ; + sh:targetClass atomic:Attestation ; + sh:property [ sh:path atomic:signedBy ; sh:minCount 1 ; sh:maxCount 1 ; sh:nodeKind sh:IRI ; sh:pattern "^did:atomic:[A-Z2-7]{51}[AQ]$" ] ; + sh:property [ sh:path atomic:verificationResult ; sh:hasValue atomic:Verified ; sh:maxCount 1 ] . diff --git a/atomic-core/ontology/workflow-1.0.0.ttl b/atomic-core/ontology/workflow-1.0.0.ttl new file mode 100644 index 00000000..7be0a639 --- /dev/null +++ b/atomic-core/ontology/workflow-1.0.0.ttl @@ -0,0 +1,115 @@ +@prefix atomic: . +@prefix prov: . +@prefix skos: . +@prefix dcterms: . +@prefix rdf: . +@prefix rdfs: . +@prefix owl: . +@prefix xsd: . + + a owl:Ontology ; + owl:versionIRI ; + rdfs:label "Atomic workflow ontology" ; + rdfs:comment "Workflow facts over Atomic's common causal graph and view membership filters. Version 1.0.0." . + +atomic:WorkIntent a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:FeatureIntent a owl:Class ; rdfs:subClassOf atomic:WorkIntent . +atomic:BugIntent a owl:Class ; rdfs:subClassOf atomic:WorkIntent . +atomic:ReviewIntent a owl:Class ; rdfs:subClassOf atomic:WorkIntent . +atomic:ManagedIntent a owl:Class ; rdfs:subClassOf atomic:WorkIntent . +atomic:Goal a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:Memory a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:Skill a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:ToolResult a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:Task a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:AcceptanceCriterion a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:ScopeItem a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:Constraint a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:Todo a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:Session a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:SessionTurn a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:ManagedTurn a owl:Class ; rdfs:subClassOf atomic:SessionTurn . +atomic:ProvenanceGraph a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:Attestation a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:Finding a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:Change a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:View a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:File a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:Identity a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:Concept a owl:Class ; rdfs:subClassOf skos:Concept . +atomic:Project a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:ProjectIdentity a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:VerificationStatus a owl:Class ; rdfs:subClassOf owl:Thing . +atomic:Verified a owl:NamedIndividual, atomic:VerificationStatus . +atomic:Unverifiable a owl:NamedIndividual, atomic:VerificationStatus . +atomic:Failed a owl:NamedIndividual, atomic:VerificationStatus . + +atomic:hasTurn a owl:ObjectProperty ; rdfs:domain atomic:Session ; rdfs:range atomic:SessionTurn ; owl:inverseOf atomic:turnOfSession . +atomic:hasPlan a owl:ObjectProperty ; rdfs:domain atomic:SessionTurn ; rdfs:range atomic:WorkIntent ; owl:inverseOf atomic:governsTurn . +atomic:generatedChange a owl:ObjectProperty ; rdfs:domain atomic:SessionTurn ; rdfs:range atomic:Change ; owl:inverseOf atomic:generatedBy ; rdfs:subPropertyOf prov:generated . +atomic:informedBy a owl:ObjectProperty ; rdfs:domain atomic:WorkIntent ; rdfs:subPropertyOf prov:wasDerivedFrom . +atomic:explainedBy a owl:ObjectProperty ; rdfs:range atomic:ProvenanceGraph . +atomic:turnExplainedBy a owl:ObjectProperty ; rdfs:domain atomic:SessionTurn ; rdfs:range atomic:ProvenanceGraph ; rdfs:subPropertyOf atomic:explainedBy . +atomic:changeExplainedBy a owl:ObjectProperty ; rdfs:domain atomic:Change ; rdfs:range atomic:ProvenanceGraph ; rdfs:subPropertyOf atomic:explainedBy ; owl:propertyChainAxiom (atomic:generatedBy atomic:turnExplainedBy) . +atomic:hasTodo a owl:ObjectProperty ; rdfs:domain atomic:SessionTurn ; rdfs:range atomic:Todo . +atomic:blocks a owl:ObjectProperty ; rdfs:domain atomic:WorkIntent ; rdfs:range atomic:WorkIntent ; owl:inverseOf atomic:blockedBy . +atomic:assignedTo a owl:ObjectProperty ; rdfs:domain atomic:WorkIntent ; rdfs:range atomic:Identity . +atomic:linkedIntent a owl:ObjectProperty ; rdfs:domain atomic:Goal ; rdfs:range atomic:WorkIntent . +atomic:linkedGoal a owl:ObjectProperty ; rdfs:range atomic:Goal . +atomic:references a owl:ObjectProperty ; rdfs:domain owl:Thing ; rdfs:range owl:Thing . +atomic:hasLabel a owl:ObjectProperty ; rdfs:domain atomic:WorkIntent ; rdfs:range skos:Concept . +atomic:hasTask a owl:ObjectProperty ; rdfs:domain atomic:WorkIntent ; rdfs:range atomic:Task . +atomic:hasAcceptanceCriterion a owl:ObjectProperty ; rdfs:domain atomic:WorkIntent ; rdfs:range atomic:AcceptanceCriterion . +atomic:hasScopeIn a owl:ObjectProperty ; rdfs:domain atomic:WorkIntent ; rdfs:range atomic:ScopeItem ; rdfs:subPropertyOf atomic:hasScopeItem . +atomic:hasScopeOut a owl:ObjectProperty ; rdfs:domain atomic:WorkIntent ; rdfs:range atomic:ScopeItem ; rdfs:subPropertyOf atomic:hasScopeItem . +atomic:hasConstraint a owl:ObjectProperty ; rdfs:domain atomic:WorkIntent ; rdfs:range atomic:Constraint . +atomic:satisfies a owl:ObjectProperty ; rdfs:domain atomic:Task ; rdfs:range atomic:AcceptanceCriterion . +atomic:touches a owl:ObjectProperty ; rdfs:domain atomic:Task ; rdfs:range atomic:File . +atomic:scopeOutFile a owl:ObjectProperty ; rdfs:domain atomic:WorkIntent ; rdfs:range atomic:File . +atomic:remediates a owl:ObjectProperty ; rdfs:domain atomic:WorkIntent ; rdfs:range atomic:WorkIntent . +atomic:reviews a owl:ObjectProperty ; rdfs:domain atomic:ReviewIntent ; rdfs:range atomic:WorkIntent ; owl:inverseOf atomic:reviewedBy . +atomic:directlyDependsOnChange a owl:ObjectProperty ; rdfs:domain atomic:Change ; rdfs:range atomic:Change ; rdfs:subPropertyOf atomic:dependsOn . +atomic:dependsOnIntent a owl:ObjectProperty ; rdfs:domain atomic:WorkIntent ; rdfs:range atomic:WorkIntent ; rdfs:subPropertyOf atomic:dependsOn . +atomic:modifiesFile a owl:ObjectProperty ; rdfs:domain atomic:Change ; rdfs:range atomic:File ; rdfs:subPropertyOf atomic:modifies . +atomic:directlyOnView a owl:ObjectProperty ; rdfs:domain atomic:Change ; rdfs:range atomic:View . +atomic:sessionViewContext a owl:ObjectProperty ; rdfs:domain atomic:Session ; rdfs:range atomic:View . +atomic:childViewOf a owl:ObjectProperty ; rdfs:domain atomic:View ; rdfs:range atomic:View . +atomic:visibleFrom a owl:ObjectProperty ; rdfs:comment "Query-time visibility operator; never stored as direct membership." . +atomic:authoredBy a owl:ObjectProperty ; rdfs:domain owl:Thing ; rdfs:range atomic:Identity ; rdfs:subPropertyOf prov:wasAttributedTo . +atomic:coversChange a owl:ObjectProperty ; rdfs:domain atomic:Attestation ; rdfs:range atomic:Change ; owl:inverseOf atomic:attestedBy . +atomic:attests a owl:ObjectProperty ; rdfs:domain atomic:Attestation ; rdfs:range atomic:SessionTurn . +atomic:attestsSession a owl:ObjectProperty ; rdfs:domain atomic:Attestation ; rdfs:range atomic:Session . +atomic:signedBy a owl:ObjectProperty ; rdfs:domain atomic:Attestation ; rdfs:range atomic:Identity . +atomic:previousAttestation a owl:ObjectProperty ; rdfs:domain atomic:Attestation ; rdfs:range atomic:Attestation . +atomic:implements a owl:ObjectProperty ; rdfs:domain atomic:Change ; rdfs:range atomic:WorkIntent ; owl:propertyChainAxiom (atomic:generatedBy atomic:hasPlan) . +atomic:attestedBy a owl:ObjectProperty ; rdfs:domain atomic:Change ; rdfs:range atomic:Attestation ; owl:inverseOf atomic:coversChange . +atomic:aboutResource a owl:ObjectProperty ; rdfs:domain atomic:Finding ; rdfs:range owl:Thing . +atomic:dependsOn a owl:ObjectProperty, owl:TransitiveProperty . +atomic:modifies a owl:ObjectProperty ; rdfs:domain atomic:Change . +atomic:hasScopeItem a owl:ObjectProperty ; rdfs:domain atomic:WorkIntent ; rdfs:range atomic:ScopeItem . +atomic:ancestorView a owl:ObjectProperty, owl:TransitiveProperty ; rdfs:domain atomic:View ; rdfs:range atomic:View . +atomic:turnOfSession a owl:ObjectProperty ; rdfs:domain atomic:SessionTurn ; rdfs:range atomic:Session ; owl:inverseOf atomic:hasTurn . +atomic:governsTurn a owl:ObjectProperty ; rdfs:domain atomic:WorkIntent ; rdfs:range atomic:SessionTurn ; owl:inverseOf atomic:hasPlan . +atomic:generatedBy a owl:ObjectProperty ; rdfs:domain atomic:Change ; rdfs:range atomic:SessionTurn ; owl:inverseOf atomic:generatedChange . +atomic:blockedBy a owl:ObjectProperty ; rdfs:domain atomic:WorkIntent ; rdfs:range atomic:WorkIntent ; owl:inverseOf atomic:blocks . +atomic:reviewedBy a owl:ObjectProperty ; rdfs:domain atomic:WorkIntent ; rdfs:range atomic:ReviewIntent ; owl:inverseOf atomic:reviews . +atomic:verificationResult a owl:ObjectProperty ; rdfs:domain atomic:Attestation ; rdfs:range atomic:VerificationStatus ; rdfs:comment "Asserted only by the cryptographic verifier; never inferred." . +atomic:hasIdentity a owl:ObjectProperty ; rdfs:domain atomic:Project ; rdfs:range atomic:ProjectIdentity . +atomic:issuer a owl:ObjectProperty ; rdfs:domain atomic:ProjectIdentity ; rdfs:range atomic:Identity . + +atomic:status a owl:DatatypeProperty ; rdfs:range xsd:string . +atomic:priority a owl:DatatypeProperty ; rdfs:range xsd:string . +atomic:findingCode a owl:DatatypeProperty ; rdfs:domain atomic:Finding ; rdfs:range xsd:string . +atomic:projectId a owl:DatatypeProperty ; rdfs:domain atomic:ProjectIdentity ; rdfs:range xsd:string . +atomic:schemaVersion a owl:DatatypeProperty ; rdfs:domain atomic:ProjectIdentity ; rdfs:range xsd:nonNegativeInteger . +atomic:turnNumber a owl:DatatypeProperty ; rdfs:domain atomic:SessionTurn ; rdfs:range xsd:nonNegativeInteger . + +prov:wasInformedBy rdfs:isDefinedBy . +prov:wasDerivedFrom rdfs:isDefinedBy . +prov:used rdfs:isDefinedBy . +prov:generated rdfs:isDefinedBy . +prov:wasAttributedTo rdfs:isDefinedBy . +rdfs:label rdfs:isDefinedBy . +dcterms:description rdfs:isDefinedBy . +dcterms:created rdfs:isDefinedBy . +dcterms:modified rdfs:isDefinedBy . diff --git a/atomic-core/src/kg/iri.rs b/atomic-core/src/kg/iri.rs new file mode 100644 index 00000000..1b0b41e6 --- /dev/null +++ b/atomic-core/src/kg/iri.rs @@ -0,0 +1,257 @@ +//! Canonical resource IRIs. Hash parsers intentionally do not use the CLI's +//! lenient prefix parser. Percent encoding is canonical and round-trip checked. + +use super::project_id::{strict_base32, ProjectId}; +use crate::types::{Base32, Hash}; + +#[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)] +#[error("non-canonical Atomic resource IRI: {0}")] +pub struct IriError(pub String); + +fn encode(value: &str, path: bool) -> String { + let mut encoded = String::new(); + for byte in value.bytes() { + if byte.is_ascii_alphanumeric() || b"-._~".contains(&byte) || (path && byte == b'/') { + encoded.push(byte as char); + } else { + use std::fmt::Write; + write!(encoded, "%{byte:02X}").expect("write to string"); + } + } + encoded +} + +fn decode(value: &str, path: bool) -> Result { + let mut decoded = Vec::new(); + let bytes = value.as_bytes(); + let mut i = 0; + while i < bytes.len() { + if bytes[i] == b'%' { + let hex = value + .get(i + 1..i + 3) + .ok_or_else(|| IriError(value.into()))?; + decoded.push(u8::from_str_radix(hex, 16).map_err(|_| IriError(value.into()))?); + i += 3; + } else { + decoded.push(bytes[i]); + i += 1; + } + } + let decoded = String::from_utf8(decoded).map_err(|_| IriError(value.into()))?; + if encode(&decoded, path) != value { + return Err(IriError(value.into())); + } + Ok(decoded) +} + +pub fn mint_project(project: &ProjectId) -> String { + project.to_urn() +} +pub fn parse_project(iri: &str) -> Result { + ProjectId::from_base32( + iri.strip_prefix("urn:atomic:project:") + .ok_or_else(|| IriError(iri.into()))?, + ) + .map_err(|_| IriError(iri.into())) +} + +macro_rules! global { + ($mint:ident, $parse:ident, $kind:literal) => { + pub fn $mint(hash: &Hash) -> String { + format!( + concat!("urn:atomic:", $kind, ":blake3-256:{}"), + hash.to_base32() + ) + } + pub fn $parse(iri: &str) -> Result { + let hash = iri + .strip_prefix(concat!("urn:atomic:", $kind, ":blake3-256:")) + .ok_or_else(|| IriError(iri.into()))?; + strict_base32(hash) + .map(crate::types::Merkle) + .map_err(|_| IriError(iri.into())) + } + }; +} +global!(mint_change, parse_change, "change"); +global!(mint_provenance, parse_provenance, "provenance"); +global!(mint_attestation, parse_attestation, "attestation"); +global!( + mint_project_identity, + parse_project_identity, + "project-identity" +); + +fn qualified(project: &ProjectId, kind: &str, value: &str, path: bool) -> String { + format!("{}:{kind}:{}", project.to_urn(), encode(value, path)) +} + +fn parse_qualified<'a>(iri: &'a str, kind: &str) -> Result<(ProjectId, &'a str), IriError> { + let rest = iri + .strip_prefix("urn:atomic:project:") + .ok_or_else(|| IriError(iri.into()))?; + let (project, rest) = rest.split_once(':').ok_or_else(|| IriError(iri.into()))?; + let project = ProjectId::from_base32(project).map_err(|_| IriError(iri.into()))?; + let rest = rest + .strip_prefix(kind) + .and_then(|s| s.strip_prefix(':')) + .ok_or_else(|| IriError(iri.into()))?; + Ok((project, rest)) +} + +macro_rules! named { + ($mint:ident, $parse:ident, $kind:literal, $path:literal) => { + pub fn $mint(project: &ProjectId, name: &str) -> String { + qualified(project, $kind, name, $path) + } + pub fn $parse(iri: &str) -> Result<(ProjectId, String), IriError> { + let (project, value) = parse_qualified(iri, $kind)?; + Ok((project, decode(value, $path)?)) + } + }; +} +named!(mint_goal, parse_goal, "goal", false); +named!(mint_memory, parse_memory, "memory", false); +named!(mint_skill, parse_skill, "skill", false); +named!(mint_tool_result, parse_tool_result, "tool-result", false); +named!(mint_concept, parse_concept, "concept", false); +named!(mint_view, parse_view, "view", false); +named!(mint_file, parse_file, "file", true); +named!(mint_session, parse_session, "session", false); +named!(mint_named_identity, parse_named_identity, "identity", false); + +pub fn mint_identity(project: &ProjectId, name_or_did: &str) -> String { + if name_or_did.starts_with("did:atomic:") { + name_or_did.to_owned() + } else { + mint_named_identity(project, name_or_did) + } +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum IdentityIri { + Did(String), + Named(ProjectId, String), +} + +pub fn parse_identity(iri: &str) -> Result { + if let Some(fingerprint) = iri.strip_prefix("did:atomic:") { + strict_base32(fingerprint).map_err(|_| IriError(iri.into()))?; + Ok(IdentityIri::Did(iri.to_owned())) + } else { + let (project, name) = parse_named_identity(iri)?; + Ok(IdentityIri::Named(project, name)) + } +} + +fn canonical_uid(uid: &str) -> Result<(), IriError> { + let parsed = ulid::Ulid::from_string(uid).map_err(|_| IriError(uid.into()))?; + if parsed.to_string() != uid { + return Err(IriError(uid.into())); + } + Ok(()) +} + +pub fn mint_intent(project: &ProjectId, uid: &str) -> Result { + canonical_uid(uid)?; + Ok(qualified(project, "intent", uid, false)) +} +pub fn parse_intent(iri: &str) -> Result<(ProjectId, String), IriError> { + let (project, uid) = parse_qualified(iri, "intent")?; + canonical_uid(uid)?; + Ok((project, uid.to_owned())) +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum ChildKind { + AcceptanceCriterion, + Task, + Scope, + Constraint, +} +impl ChildKind { + pub const fn segment(self) -> &'static str { + match self { + Self::AcceptanceCriterion => "ac", + Self::Task => "task", + Self::Scope => "scope", + Self::Constraint => "constraint", + } + } +} +pub fn mint_child( + project: &ProjectId, + uid: &str, + kind: ChildKind, + child: &str, +) -> Result { + Ok(format!( + "{}:{}:{}", + mint_intent(project, uid)?, + kind.segment(), + encode(&child.to_uppercase(), false) + )) +} +pub fn parse_child(iri: &str) -> Result<(ProjectId, String, ChildKind, String), IriError> { + let (project, rest) = parse_qualified(iri, "intent")?; + let mut parts = rest.split(':'); + let uid = parts.next().ok_or_else(|| IriError(iri.into()))?; + canonical_uid(uid)?; + let kind = match parts.next() { + Some("ac") => ChildKind::AcceptanceCriterion, + Some("task") => ChildKind::Task, + Some("scope") => ChildKind::Scope, + Some("constraint") => ChildKind::Constraint, + _ => return Err(IriError(iri.into())), + }; + let child = decode(parts.next().ok_or_else(|| IriError(iri.into()))?, false)?; + if parts.next().is_some() || child != child.to_uppercase() { + return Err(IriError(iri.into())); + } + Ok((project, uid.to_owned(), kind, child)) +} + +pub fn mint_turn(project: &ProjectId, session: &str, number: u32) -> String { + format!("{}:turn:{number}", mint_session(project, session)) +} +pub fn parse_turn(iri: &str) -> Result<(ProjectId, String, u32), IriError> { + let (session, number) = iri + .rsplit_once(":turn:") + .ok_or_else(|| IriError(iri.into()))?; + let (project, session) = parse_session(session)?; + let n: u32 = number.parse().map_err(|_| IriError(iri.into()))?; + if n.to_string() != number { + return Err(IriError(iri.into())); + } + Ok((project, session, n)) +} +pub fn mint_todo(project: &ProjectId, session: &str, id: &str) -> String { + format!( + "{}:todo:{}", + mint_session(project, session), + encode(id, false) + ) +} +pub fn parse_todo(iri: &str) -> Result<(ProjectId, String, String), IriError> { + let (session, id) = iri + .rsplit_once(":todo:") + .ok_or_else(|| IriError(iri.into()))?; + let (project, session) = parse_session(session)?; + Ok((project, session, decode(id, false)?)) +} +pub fn mint_finding(project: &ProjectId, code: &str, subject: &str) -> String { + format!( + "{}:finding:{}:{}", + project.to_urn(), + encode(code, false), + Hash::of(subject.as_bytes()).to_base32() + ) +} +pub fn parse_finding(iri: &str) -> Result<(ProjectId, String, Hash), IriError> { + let (project, rest) = parse_qualified(iri, "finding")?; + let (code, hash) = rest.split_once(':').ok_or_else(|| IriError(iri.into()))?; + let hash = strict_base32(hash) + .map(crate::types::Merkle) + .map_err(|_| IriError(iri.into()))?; + Ok((project, decode(code, false)?, hash)) +} diff --git a/atomic-core/src/kg/legacy.rs b/atomic-core/src/kg/legacy.rs new file mode 100644 index 00000000..0f7b225d --- /dev/null +++ b/atomic-core/src/kg/legacy.rs @@ -0,0 +1,256 @@ +//! Frozen pre-cutover KG vocabulary inventory. These are dispositions, not aliases. +//! Projectors classify authoritative data; legacy strings are never accepted by +//! the canonical write API. + +use super::ontology::{Class, Predicate}; + +#[derive(Clone, Copy, Debug)] +pub enum Target { + Class(Class), + Predicate(Predicate), + Removed { reason: &'static str }, +} + +#[derive(Clone, Copy, Debug)] +pub struct EndpointRule { + pub from_kind: Option<&'static str>, + pub to_kind: Option<&'static str>, + pub target: Target, +} + +#[derive(Clone, Copy, Debug)] +pub enum Replacement { + One(Target), + ClassWithRefinement { + base: Class, + refinements: &'static [Class], + rule: &'static str, + }, + SplitByEndpoint(&'static [EndpointRule]), +} + +use Class as C; +use Predicate as P; +use Replacement::{ClassWithRefinement, One, SplitByEndpoint}; +const fn class(class: Class) -> Replacement { + One(Target::Class(class)) +} +const fn predicate(predicate: Predicate) -> Replacement { + One(Target::Predicate(predicate)) +} +const fn removed(reason: &'static str) -> Replacement { + One(Target::Removed { reason }) +} +const fn endpoint(from: &'static str, to: &'static str, predicate: Predicate) -> EndpointRule { + EndpointRule { + from_kind: Some(from), + to_kind: Some(to), + target: Target::Predicate(predicate), + } +} +const INTENT: Replacement = ClassWithRefinement { + base: C::WorkIntent, + refinements: &[ + C::FeatureIntent, + C::BugIntent, + C::ReviewIntent, + C::ManagedIntent, + ], + rule: "Canonical kind chooses feature/bug/review; active lifecycle chooses ManagedIntent", +}; +const TURN: Replacement = ClassWithRefinement { + base: C::SessionTurn, + refinements: &[C::ManagedTurn], + rule: "ManagedTurn iff plan_id is present", +}; + +pub static LEGACY_TERMS: &[(&str, Replacement)] = &[ + ( + "atom", + removed("Prefix replaced by the absolute PREFIXES registry"), + ), + ( + "prov", + removed("Prefix replaced by the absolute PREFIXES registry"), + ), + ( + "skos", + removed("Prefix replaced by the absolute PREFIXES registry"), + ), + ("dc", removed("Use dcterms for source metadata")), + ("vault", removed("Workflow terms use the atomic namespace")), + ( + "rdf", + removed("Prefix replaced by the absolute PREFIXES registry"), + ), + ("prov:wasAssociatedWith", predicate(P::LinkedGoal)), + ("prov:used", predicate(P::ProvUsed)), + ("prov:wasGeneratedBy", removed("Never emitted")), + ("prov:wasInformedBy", predicate(P::ProvWasInformedBy)), + ("prov:wasDerivedFrom", predicate(P::ProvWasDerivedFrom)), + ("atom:informedBy", predicate(P::InformedBy)), + ( + "prov:hadPlan", + SplitByEndpoint(&[ + endpoint("session_turn", "intent", P::HasPlan), + EndpointRule { + from_kind: Some("session"), + to_kind: Some("intent"), + target: Target::Removed { + reason: "Use session hasTurn / turn hasPlan", + }, + }, + ]), + ), + ("prov:hadMember", predicate(P::HasTurn)), + ("prov:generated", predicate(P::GeneratedChange)), + ("skos:broader", removed("Never emitted")), + ("skos:narrower", removed("Never emitted")), + ("skos:subject", removed("Never emitted")), + ("skos:related", removed("Never emitted")), + ("dc:type", removed("Never emitted")), + ( + "dc:creator", + removed("Derive authoredBy from typed source fields"), + ), + ( + "dc:date", + removed("Derive dcterms timestamps from typed source fields"), + ), + ( + "dc:description", + removed("Derive dcterms description from typed source fields"), + ), + ( + "rdf:type", + removed("Class assertions use kg_types, not neighborhood edges"), + ), + ( + "vault:blocks", + removed("Unused duplicate of uppercase edge"), + ), + ( + "vault:blockedBy", + removed("Unused duplicate of uppercase edge"), + ), + ("vault:status", predicate(P::Status)), + ( + "vault:assignedTo", + removed("Unused duplicate of uppercase edge"), + ), + ( + "vault:linkedIntent", + removed("Unused duplicate of uppercase edge"), + ), + ( + "vault:linkedGoal", + removed("Unused duplicate of uppercase edge"), + ), + ( + "vault:references", + removed("Unused duplicate of uppercase edge"), + ), + ("vault:appliesTo", removed("Never emitted")), + ( + "vault:hasLabel", + removed("Unused duplicate of uppercase edge"), + ), + ("vault:priority", predicate(P::Priority)), + ("vault:explainedBy", predicate(P::TurnExplainedBy)), + ("vault:hasTodo", predicate(P::HasTodo)), + ("vault:Goal", class(C::Goal)), + ("vault:Intent", INTENT), + ("vault:Memory", class(C::Memory)), + ("vault:Skill", class(C::Skill)), + ("vault:ToolResult", class(C::ToolResult)), + ("vault:File", class(C::File)), + ("vault:Identity", class(C::Identity)), + ("vault:Concept", class(C::Concept)), + ("vault:Session", class(C::Session)), + ("vault:SessionTurn", TURN), + ("vault:Todo", class(C::Todo)), + ("vault:ProvenanceGraph", class(C::ProvenanceGraph)), + ("vault:Task", class(C::Task)), + ("vault:AcceptanceCriterion", class(C::AcceptanceCriterion)), + ("vault:ScopeItem", class(C::ScopeItem)), + ("vault:Constraint", class(C::Constraint)), + ( + "MODIFIES", + SplitByEndpoint(&[ + endpoint("change", "file", P::ModifiesFile), + EndpointRule { + from_kind: Some("change"), + to_kind: Some("entity"), + target: Target::Removed { + reason: "Source entities stay in the syntax index", + }, + }, + ]), + ), + ("DEFINES", removed("Code structure is parsed live")), + ( + "DEPENDS_ON", + SplitByEndpoint(&[ + endpoint("change", "change", P::DirectlyDependsOnChange), + endpoint("intent", "intent", P::DependsOnIntent), + ]), + ), + ( + "ON_VIEW", + SplitByEndpoint(&[ + endpoint("change", "view", P::DirectlyOnView), + endpoint("session", "view", P::SessionViewContext), + ]), + ), + ("CHILD_OF", predicate(P::ChildViewOf)), + ("VISIBLE_FROM", predicate(P::VisibleFrom)), + ("CALLS", removed("Code structure is parsed live")), + ("IMPORTS", removed("Code structure is parsed live")), + ("EXPORTS", removed("Code structure is parsed live")), + ("REFERENCES", predicate(P::References)), + ("LINKED_INTENT", predicate(P::LinkedIntent)), + ("LINKED_GOAL", predicate(P::LinkedGoal)), + ("ASSIGNED_TO", predicate(P::AssignedTo)), + ("BLOCKS", predicate(P::Blocks)), + ("BLOCKED_BY", predicate(P::BlockedBy)), + ("AUTHORED_BY", predicate(P::AuthoredBy)), + ("HAS_LABEL", predicate(P::HasLabel)), + ("PART_OF", removed("Module hierarchy is derived from paths")), + ("INCLUDES", removed("Code structure is parsed live")), + ("HAS_TASK", predicate(P::HasTask)), + ( + "HAS_ACCEPTANCE_CRITERION", + predicate(P::HasAcceptanceCriterion), + ), + ("HAS_SCOPE_IN", predicate(P::HasScopeIn)), + ("HAS_SCOPE_OUT", predicate(P::HasScopeOut)), + ("HAS_CONSTRAINT", predicate(P::HasConstraint)), + ("SATISFIES", predicate(P::Satisfies)), + ("TOUCHES", predicate(P::Touches)), + ("REMEDIATES", predicate(P::Remediates)), + ("REVIEWS", predicate(P::Reviews)), + ("SCOPE_OUT_FILE", predicate(P::ScopeOutFile)), + ("goal", class(C::Goal)), + ("intent", INTENT), + ("memory", class(C::Memory)), + ("skill", class(C::Skill)), + ("tool_result", class(C::ToolResult)), + ("file", class(C::File)), + ("identity", class(C::Identity)), + ("concept", class(C::Concept)), + ("session", class(C::Session)), + ("session_turn", TURN), + ("todo", class(C::Todo)), + ("provenance", class(C::ProvenanceGraph)), + ("task", class(C::Task)), + ("acceptance_criterion", class(C::AcceptanceCriterion)), + ("scope", class(C::ScopeItem)), + ("constraint", class(C::Constraint)), + ("change", class(C::Change)), + ("view", class(C::View)), + ("module", removed("Module hierarchy is derived from paths")), + ( + "entity", + removed("Source entities stay in the syntax index"), + ), +]; diff --git a/atomic-core/src/kg/mod.rs b/atomic-core/src/kg/mod.rs new file mode 100644 index 00000000..60bde518 --- /dev/null +++ b/atomic-core/src/kg/mod.rs @@ -0,0 +1,18 @@ +//! Canonical workflow vocabulary and portable resource identity. + +pub mod iri; +pub mod legacy; +pub mod ontology; +pub mod project_id; + +/// Storage generation of the derived canonical workflow graph. +pub const KG_GENERATION: u32 = 1; + +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum Direction { + Outgoing, + Incoming, + #[default] + Both, +} diff --git a/atomic-core/src/kg/ontology.rs b/atomic-core/src/kg/ontology.rs new file mode 100644 index 00000000..2c023552 --- /dev/null +++ b/atomic-core/src/kg/ontology.rs @@ -0,0 +1,267 @@ +//! The closed vocabulary of the versioned Atomic workflow ontology. +//! +//! Domain/range are RDF axioms, not write-time policy checks. SHACL owns +//! conformance. External terms retain their original namespace and meaning. + +use serde::{Deserialize, Serialize}; + +pub const NAMESPACE: &str = "https://atomic.dev/ns/workflow#"; +pub const VERSION_IRI: &str = "https://atomic.dev/ns/workflow/1.0.0"; +pub const TTL: &str = include_str!("../../ontology/workflow-1.0.0.ttl"); +pub const PREFIXES: &[(&str, &str)] = &[ + ("atomic", NAMESPACE), + ("prov", "http://www.w3.org/ns/prov#"), + ("dcterms", "http://purl.org/dc/terms/"), + ("skos", "http://www.w3.org/2004/02/skos/core#"), + ("rdf", "http://www.w3.org/1999/02/22-rdf-syntax-ns#"), + ("rdfs", "http://www.w3.org/2000/01/rdf-schema#"), + ("owl", "http://www.w3.org/2002/07/owl#"), + ("xsd", "http://www.w3.org/2001/XMLSchema#"), +]; + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum ExternalClass { + OwlThing, + SkosConcept, +} + +impl ExternalClass { + pub const fn iri(self) -> &'static str { + match self { + Self::OwlThing => "http://www.w3.org/2002/07/owl#Thing", + Self::SkosConcept => "http://www.w3.org/2004/02/skos/core#Concept", + } + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum ClassRef { + Atomic(Class), + External(ExternalClass), +} + +impl ClassRef { + pub const fn iri(self) -> &'static str { + match self { + Self::Atomic(class) => class.iri(), + Self::External(class) => class.iri(), + } + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Datatype { + String, + NonNegativeInteger, + DateTime, +} + +impl Datatype { + pub const fn iri(self) -> &'static str { + match self { + Self::String => "http://www.w3.org/2001/XMLSchema#string", + Self::NonNegativeInteger => "http://www.w3.org/2001/XMLSchema#nonNegativeInteger", + Self::DateTime => "http://www.w3.org/2001/XMLSchema#dateTime", + } + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Range { + Class(ClassRef), + Datatype(Datatype), +} + +impl Range { + pub const fn iri(self) -> &'static str { + match self { + Self::Class(class) => class.iri(), + Self::Datatype(datatype) => datatype.iri(), + } + } +} + +macro_rules! classes { + ($($name:ident => $parent:expr),+ $(,)?) => { + #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)] + pub enum Class { $($name),+ } + impl Class { + pub const ALL: &'static [Self] = &[$(Self::$name),+]; + pub const fn iri(self) -> &'static str { match self { $(Self::$name => concat!("https://atomic.dev/ns/workflow#", stringify!($name))),+ } } + pub const fn curie(self) -> &'static str { match self { $(Self::$name => concat!("atomic:", stringify!($name))),+ } } + pub const fn parents(self) -> &'static [ClassRef] { match self { $(Self::$name => &[$parent]),+ } } + pub fn from_iri(iri: &str) -> Option { Self::ALL.iter().copied().find(|class| class.iri() == iri) } + } + }; +} + +use ClassRef::{Atomic as A, External as E}; +use ExternalClass::{OwlThing, SkosConcept}; +classes! { + WorkIntent => E(OwlThing), FeatureIntent => A(Class::WorkIntent), + BugIntent => A(Class::WorkIntent), ReviewIntent => A(Class::WorkIntent), + ManagedIntent => A(Class::WorkIntent), Goal => E(OwlThing), Memory => E(OwlThing), + Skill => E(OwlThing), ToolResult => E(OwlThing), Task => E(OwlThing), + AcceptanceCriterion => E(OwlThing), ScopeItem => E(OwlThing), Constraint => E(OwlThing), + Todo => E(OwlThing), Session => E(OwlThing), SessionTurn => E(OwlThing), + ManagedTurn => A(Class::SessionTurn), ProvenanceGraph => E(OwlThing), + Attestation => E(OwlThing), Finding => E(OwlThing), Change => E(OwlThing), + View => E(OwlThing), File => E(OwlThing), Identity => E(OwlThing), + Concept => E(SkosConcept), Project => E(OwlThing), ProjectIdentity => E(OwlThing), + VerificationStatus => E(OwlThing), +} + +macro_rules! predicates { + ($($name:ident => ($iri:literal, $curie:literal, $domain:expr, $range:expr, $inverse:expr, $super:expr)),+ $(,)?) => { + #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)] + pub enum Predicate { $($name),+ } + impl Predicate { + pub const ALL: &'static [Self] = &[$(Self::$name),+]; + pub const fn iri(self) -> &'static str { match self { $(Self::$name => $iri),+ } } + pub const fn curie(self) -> &'static str { match self { $(Self::$name => $curie),+ } } + pub fn is_external(self) -> bool { !self.iri().starts_with(NAMESPACE) } + pub const fn domain(self) -> Option { match self { $(Self::$name => $domain),+ } } + pub const fn range(self) -> Option { match self { $(Self::$name => $range),+ } } + pub const fn inverse(self) -> Option { match self { $(Self::$name => $inverse),+ } } + pub const fn super_predicate(self) -> Option { match self { $(Self::$name => $super),+ } } + pub fn from_iri(iri: &str) -> Option { Self::ALL.iter().copied().find(|predicate| predicate.iri() == iri) } + } + }; +} + +const fn domain(class: Class) -> Option { + Some(A(class)) +} +const fn range(class: Class) -> Option { + Some(Range::Class(A(class))) +} +use Class::*; +use Predicate::*; +predicates! { + HasTurn => ("https://atomic.dev/ns/workflow#hasTurn", "atomic:hasTurn", domain(Session), range(SessionTurn), Some(TurnOfSession), None), + HasPlan => ("https://atomic.dev/ns/workflow#hasPlan", "atomic:hasPlan", domain(SessionTurn), range(WorkIntent), Some(GovernsTurn), None), + GeneratedChange => ("https://atomic.dev/ns/workflow#generatedChange", "atomic:generatedChange", domain(SessionTurn), range(Change), Some(GeneratedBy), Some(ProvGenerated)), + InformedBy => ("https://atomic.dev/ns/workflow#informedBy", "atomic:informedBy", domain(WorkIntent), None, None, Some(ProvWasDerivedFrom)), + ExplainedBy => ("https://atomic.dev/ns/workflow#explainedBy", "atomic:explainedBy", None, range(ProvenanceGraph), None, None), + TurnExplainedBy => ("https://atomic.dev/ns/workflow#turnExplainedBy", "atomic:turnExplainedBy", domain(SessionTurn), range(ProvenanceGraph), None, Some(ExplainedBy)), + ChangeExplainedBy => ("https://atomic.dev/ns/workflow#changeExplainedBy", "atomic:changeExplainedBy", domain(Change), range(ProvenanceGraph), None, Some(ExplainedBy)), + HasTodo => ("https://atomic.dev/ns/workflow#hasTodo", "atomic:hasTodo", domain(SessionTurn), range(Todo), None, None), + Blocks => ("https://atomic.dev/ns/workflow#blocks", "atomic:blocks", domain(WorkIntent), range(WorkIntent), Some(BlockedBy), None), + AssignedTo => ("https://atomic.dev/ns/workflow#assignedTo", "atomic:assignedTo", domain(WorkIntent), range(Identity), None, None), + LinkedIntent => ("https://atomic.dev/ns/workflow#linkedIntent", "atomic:linkedIntent", domain(Goal), range(WorkIntent), None, None), + LinkedGoal => ("https://atomic.dev/ns/workflow#linkedGoal", "atomic:linkedGoal", None, range(Goal), None, None), + References => ("https://atomic.dev/ns/workflow#references", "atomic:references", Some(E(OwlThing)), Some(Range::Class(E(OwlThing))), None, None), + HasLabel => ("https://atomic.dev/ns/workflow#hasLabel", "atomic:hasLabel", domain(WorkIntent), Some(Range::Class(E(SkosConcept))), None, None), + HasTask => ("https://atomic.dev/ns/workflow#hasTask", "atomic:hasTask", domain(WorkIntent), range(Task), None, None), + HasAcceptanceCriterion => ("https://atomic.dev/ns/workflow#hasAcceptanceCriterion", "atomic:hasAcceptanceCriterion", domain(WorkIntent), range(AcceptanceCriterion), None, None), + HasScopeIn => ("https://atomic.dev/ns/workflow#hasScopeIn", "atomic:hasScopeIn", domain(WorkIntent), range(ScopeItem), None, Some(HasScopeItem)), + HasScopeOut => ("https://atomic.dev/ns/workflow#hasScopeOut", "atomic:hasScopeOut", domain(WorkIntent), range(ScopeItem), None, Some(HasScopeItem)), + HasConstraint => ("https://atomic.dev/ns/workflow#hasConstraint", "atomic:hasConstraint", domain(WorkIntent), range(Constraint), None, None), + Satisfies => ("https://atomic.dev/ns/workflow#satisfies", "atomic:satisfies", domain(Task), range(AcceptanceCriterion), None, None), + Touches => ("https://atomic.dev/ns/workflow#touches", "atomic:touches", domain(Task), range(File), None, None), + ScopeOutFile => ("https://atomic.dev/ns/workflow#scopeOutFile", "atomic:scopeOutFile", domain(WorkIntent), range(File), None, None), + Remediates => ("https://atomic.dev/ns/workflow#remediates", "atomic:remediates", domain(WorkIntent), range(WorkIntent), None, None), + Reviews => ("https://atomic.dev/ns/workflow#reviews", "atomic:reviews", domain(ReviewIntent), range(WorkIntent), Some(ReviewedBy), None), + DirectlyDependsOnChange => ("https://atomic.dev/ns/workflow#directlyDependsOnChange", "atomic:directlyDependsOnChange", domain(Change), range(Change), None, Some(DependsOn)), + DependsOnIntent => ("https://atomic.dev/ns/workflow#dependsOnIntent", "atomic:dependsOnIntent", domain(WorkIntent), range(WorkIntent), None, Some(DependsOn)), + ModifiesFile => ("https://atomic.dev/ns/workflow#modifiesFile", "atomic:modifiesFile", domain(Change), range(File), None, Some(Modifies)), + DirectlyOnView => ("https://atomic.dev/ns/workflow#directlyOnView", "atomic:directlyOnView", domain(Change), range(View), None, None), + SessionViewContext => ("https://atomic.dev/ns/workflow#sessionViewContext", "atomic:sessionViewContext", domain(Session), range(View), None, None), + ChildViewOf => ("https://atomic.dev/ns/workflow#childViewOf", "atomic:childViewOf", domain(View), range(View), None, None), + VisibleFrom => ("https://atomic.dev/ns/workflow#visibleFrom", "atomic:visibleFrom", None, None, None, None), + AuthoredBy => ("https://atomic.dev/ns/workflow#authoredBy", "atomic:authoredBy", Some(E(OwlThing)), range(Identity), None, Some(ProvWasAttributedTo)), + CoversChange => ("https://atomic.dev/ns/workflow#coversChange", "atomic:coversChange", domain(Attestation), range(Change), Some(AttestedBy), None), + Attests => ("https://atomic.dev/ns/workflow#attests", "atomic:attests", domain(Attestation), range(SessionTurn), None, None), + AttestsSession => ("https://atomic.dev/ns/workflow#attestsSession", "atomic:attestsSession", domain(Attestation), range(Session), None, None), + SignedBy => ("https://atomic.dev/ns/workflow#signedBy", "atomic:signedBy", domain(Attestation), range(Identity), None, None), + PreviousAttestation => ("https://atomic.dev/ns/workflow#previousAttestation", "atomic:previousAttestation", domain(Attestation), range(Attestation), None, None), + Implements => ("https://atomic.dev/ns/workflow#implements", "atomic:implements", domain(Change), range(WorkIntent), None, None), + AttestedBy => ("https://atomic.dev/ns/workflow#attestedBy", "atomic:attestedBy", domain(Change), range(Attestation), Some(CoversChange), None), + AboutResource => ("https://atomic.dev/ns/workflow#aboutResource", "atomic:aboutResource", domain(Finding), Some(Range::Class(E(OwlThing))), None, None), + DependsOn => ("https://atomic.dev/ns/workflow#dependsOn", "atomic:dependsOn", None, None, None, None), + Modifies => ("https://atomic.dev/ns/workflow#modifies", "atomic:modifies", domain(Change), None, None, None), + HasScopeItem => ("https://atomic.dev/ns/workflow#hasScopeItem", "atomic:hasScopeItem", domain(WorkIntent), range(ScopeItem), None, None), + AncestorView => ("https://atomic.dev/ns/workflow#ancestorView", "atomic:ancestorView", domain(View), range(View), None, None), + TurnOfSession => ("https://atomic.dev/ns/workflow#turnOfSession", "atomic:turnOfSession", domain(SessionTurn), range(Session), Some(HasTurn), None), + GovernsTurn => ("https://atomic.dev/ns/workflow#governsTurn", "atomic:governsTurn", domain(WorkIntent), range(SessionTurn), Some(HasPlan), None), + GeneratedBy => ("https://atomic.dev/ns/workflow#generatedBy", "atomic:generatedBy", domain(Change), range(SessionTurn), Some(GeneratedChange), None), + BlockedBy => ("https://atomic.dev/ns/workflow#blockedBy", "atomic:blockedBy", domain(WorkIntent), range(WorkIntent), Some(Blocks), None), + ReviewedBy => ("https://atomic.dev/ns/workflow#reviewedBy", "atomic:reviewedBy", domain(WorkIntent), range(ReviewIntent), Some(Reviews), None), + VerificationResult => ("https://atomic.dev/ns/workflow#verificationResult", "atomic:verificationResult", domain(Attestation), range(VerificationStatus), None, None), + HasIdentity => ("https://atomic.dev/ns/workflow#hasIdentity", "atomic:hasIdentity", domain(Project), range(ProjectIdentity), None, None), + Issuer => ("https://atomic.dev/ns/workflow#issuer", "atomic:issuer", domain(ProjectIdentity), range(Identity), None, None), + Status => ("https://atomic.dev/ns/workflow#status", "atomic:status", None, Some(Range::Datatype(Datatype::String)), None, None), + Priority => ("https://atomic.dev/ns/workflow#priority", "atomic:priority", None, Some(Range::Datatype(Datatype::String)), None, None), + FindingCode => ("https://atomic.dev/ns/workflow#findingCode", "atomic:findingCode", domain(Finding), Some(Range::Datatype(Datatype::String)), None, None), + ProjectId => ("https://atomic.dev/ns/workflow#projectId", "atomic:projectId", domain(ProjectIdentity), Some(Range::Datatype(Datatype::String)), None, None), + SchemaVersion => ("https://atomic.dev/ns/workflow#schemaVersion", "atomic:schemaVersion", domain(ProjectIdentity), Some(Range::Datatype(Datatype::NonNegativeInteger)), None, None), + TurnNumber => ("https://atomic.dev/ns/workflow#turnNumber", "atomic:turnNumber", domain(SessionTurn), Some(Range::Datatype(Datatype::NonNegativeInteger)), None, None), + ProvWasInformedBy => ("http://www.w3.org/ns/prov#wasInformedBy", "prov:wasInformedBy", None, None, None, None), + ProvWasDerivedFrom => ("http://www.w3.org/ns/prov#wasDerivedFrom", "prov:wasDerivedFrom", None, None, None, None), + ProvUsed => ("http://www.w3.org/ns/prov#used", "prov:used", None, None, None, None), + ProvGenerated => ("http://www.w3.org/ns/prov#generated", "prov:generated", None, None, None, None), + ProvWasAttributedTo => ("http://www.w3.org/ns/prov#wasAttributedTo", "prov:wasAttributedTo", None, None, None, None), + RdfsLabel => ("http://www.w3.org/2000/01/rdf-schema#label", "rdfs:label", None, None, None, None), + DctermsDescription => ("http://purl.org/dc/terms/description", "dcterms:description", None, None, None, None), + DctermsCreated => ("http://purl.org/dc/terms/created", "dcterms:created", None, None, None, None), + DctermsModified => ("http://purl.org/dc/terms/modified", "dcterms:modified", None, None, None, None), +} + +impl Predicate { + pub const fn as_str(self) -> &'static str { + self.iri() + } + pub const fn property_chain(self) -> &'static [Self] { + match self { + Self::Implements => &[Self::GeneratedBy, Self::HasPlan], + Self::ChangeExplainedBy => &[Self::GeneratedBy, Self::TurnExplainedBy], + _ => &[], + } + } + pub const fn is_transitive(self) -> bool { + matches!(self, Self::DependsOn | Self::AncestorView) + } +} + +impl std::ops::Deref for Predicate { + type Target = str; + fn deref(&self) -> &str { + self.iri() + } +} +impl std::fmt::Display for Predicate { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(self.iri()) + } +} +impl PartialEq<&str> for Predicate { + fn eq(&self, other: &&str) -> bool { + self.iri() == *other + } +} +impl PartialEq for Predicate { + fn eq(&self, other: &str) -> bool { + self.iri() == other + } +} +impl Serialize for Predicate { + fn serialize(&self, serializer: S) -> Result { + serializer.serialize_str(self.iri()) + } +} +impl<'de> Deserialize<'de> for Predicate { + fn deserialize>(deserializer: D) -> Result { + let iri = String::deserialize(deserializer)?; + Self::from_iri(&iri) + .ok_or_else(|| serde::de::Error::custom(format!("unknown canonical predicate: {iri}"))) + } +} + +pub fn compact(iri: &str) -> String { + for (prefix, namespace) in PREFIXES { + if let Some(local) = iri.strip_prefix(namespace) { + return format!("{prefix}:{local}"); + } + } + iri.to_owned() +} diff --git a/atomic-core/src/kg/project_id.rs b/atomic-core/src/kg/project_id.rs new file mode 100644 index 00000000..0b9e1131 --- /dev/null +++ b/atomic-core/src/kg/project_id.rs @@ -0,0 +1,84 @@ +//! Stable repository identity, independent of working copies and view membership. + +use rand::RngCore; +use serde::{Deserialize, Serialize}; +use std::fmt; + +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)] +pub struct ProjectId([u8; 32]); + +#[derive(Debug, thiserror::Error)] +#[error("expected 52-character uppercase unpadded RFC 4648 base32 with zero trailing bits")] +pub struct ProjectIdError; + +pub(crate) fn strict_base32(s: &str) -> Result<[u8; 32], ProjectIdError> { + if s.len() != 52 + || !s + .bytes() + .all(|b| b.is_ascii_uppercase() || (b'2'..=b'7').contains(&b)) + { + return Err(ProjectIdError); + } + let bytes = data_encoding::BASE32_NOPAD + .decode(s.as_bytes()) + .map_err(|_| ProjectIdError)?; + let bytes: [u8; 32] = bytes.try_into().map_err(|_| ProjectIdError)?; + if data_encoding::BASE32_NOPAD.encode(&bytes) != s { + return Err(ProjectIdError); + } + Ok(bytes) +} + +impl ProjectId { + pub fn generate() -> Self { + let mut bytes = [0; 32]; + rand::rngs::OsRng.fill_bytes(&mut bytes); + Self(bytes) + } + pub fn from_base32(s: &str) -> Result { + strict_base32(s).map(Self) + } + pub fn to_base32(self) -> String { + data_encoding::BASE32_NOPAD.encode(&self.0) + } + pub fn to_urn(self) -> String { + format!("urn:atomic:project:{}", self.to_base32()) + } +} + +impl fmt::Display for ProjectId { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str(&self.to_base32()) + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum MintReason { + Created, + Upgrade, +} + +/// Singleton persisted outside the change/view system. `object` is canonical +/// JSON bytes; hash authority belongs to its domain-separated proof-free view. +#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] +pub struct ProjectIdentityRecord { + pub object: Vec, + pub hash: crate::types::Hash, + pub mint_reason: MintReason, + pub created_at: String, + pub previous: Option, +} + +impl ProjectIdentityRecord { + pub fn project_id(&self) -> Result { + let object: serde_json::Value = + serde_json::from_slice(&self.object).map_err(|_| ProjectIdError)?; + ProjectId::from_base32( + object + .get("atomic:projectId") + .and_then(serde_json::Value::as_str) + .ok_or(ProjectIdError)?, + ) + } +} diff --git a/atomic-core/src/lib.rs b/atomic-core/src/lib.rs index 4133648f..2edeb0f1 100644 --- a/atomic-core/src/lib.rs +++ b/atomic-core/src/lib.rs @@ -31,6 +31,7 @@ pub mod change; pub mod crdt; pub mod diff; +pub mod kg; pub mod operation; pub mod pristine; pub mod types; @@ -72,7 +73,6 @@ pub use change::{ }; // Re-export pristine types -pub use pristine::ontology; pub use pristine::{EmbeddingRecord, KgEdge, KgNode, KgQueryResponse, KgSubgraph, SearchResult}; pub use pristine::{ EmbeddingsMutTxnT, EmbeddingsTxnT, GraphTxnT, KgMutTxnT, KgTxnT, MutTxnT, OperationMutTxnT, diff --git a/atomic-core/src/pristine/capability.rs b/atomic-core/src/pristine/capability.rs index 7aed7420..e103fe33 100644 --- a/atomic-core/src/pristine/capability.rs +++ b/atomic-core/src/pristine/capability.rs @@ -18,9 +18,14 @@ pub const CHANGE_FORMAT_VNEXT_CAPABILITY: RepositoryCapability = pub const SHIM_CUTOVER_CAPABILITY: RepositoryCapability = RepositoryCapability::new("git-bridge-cutover", 1); +pub const KG_SEMWEB_CAPABILITY: RepositoryCapability = RepositoryCapability::new("kg-semweb-v1", 1); + /// Capabilities supported by this Atomic build. -pub const SUPPORTED_REPOSITORY_CAPABILITIES: &[RepositoryCapability] = - &[CHANGE_FORMAT_VNEXT_CAPABILITY, SHIM_CUTOVER_CAPABILITY]; +pub const SUPPORTED_REPOSITORY_CAPABILITIES: &[RepositoryCapability] = &[ + CHANGE_FORMAT_VNEXT_CAPABILITY, + SHIM_CUTOVER_CAPABILITY, + KG_SEMWEB_CAPABILITY, +]; /// A repository capability and the minimum version required to use it. #[derive(Debug, Clone, Copy, PartialEq, Eq)] diff --git a/atomic-core/src/pristine/error.rs b/atomic-core/src/pristine/error.rs index 2590fe71..3ca9ff75 100644 --- a/atomic-core/src/pristine/error.rs +++ b/atomic-core/src/pristine/error.rs @@ -70,6 +70,8 @@ use super::UnsupportedRepositoryCapability; /// ``` #[derive(Debug)] pub enum PristineError { + MissingProjectIdentity, + KgNeedsRebuild, // Database Errors (from redb) /// Error opening or creating the database /// @@ -531,6 +533,8 @@ impl fmt::Display for PristineError { // Serialization errors Self::Serialization { message } => write!(f, "serialization error: {}", message), + Self::MissingProjectIdentity => f.write_str("MissingProjectIdentity: project identity is required before KG projection"), + Self::KgNeedsRebuild => f.write_str("KgNeedsRebuild: open the repository writable or run atomic query reindex --full"), // Schema errors Self::UnsupportedSchema { found, supported } => write!( diff --git a/atomic-core/src/pristine/mod.rs b/atomic-core/src/pristine/mod.rs index 1d9e7735..2ecc824f 100644 --- a/atomic-core/src/pristine/mod.rs +++ b/atomic-core/src/pristine/mod.rs @@ -156,7 +156,6 @@ mod capability; mod error; mod inode_graph; pub mod merge; -pub mod ontology; mod path_claim; mod ref_mapping; pub mod schema; diff --git a/atomic-core/src/pristine/ontology.rs b/atomic-core/src/pristine/ontology.rs deleted file mode 100644 index fc47139a..00000000 --- a/atomic-core/src/pristine/ontology.rs +++ /dev/null @@ -1,354 +0,0 @@ -//! Predicate vocabulary for the vault knowledge graph. -//! -//! Defines standard ontology prefixes and predicates used in the triple store. -//! Based on W3C PROV-O, SKOS, Dublin Core, and vault-specific relations. - -/// Ontology prefix constants. -pub mod prefix { - /// Atomic's canonical domain ontology. - pub const ATOM: &str = "atom"; - /// W3C Provenance Ontology — who did what, when, using what. - pub const PROV: &str = "prov"; - /// Simple Knowledge Organization System — concepts, hierarchies. - pub const SKOS: &str = "skos"; - /// Dublin Core — document metadata. - pub const DC: &str = "dc"; - /// Vault-specific relations. - pub const VAULT: &str = "vault"; - /// RDF type. - pub const RDF: &str = "rdf"; -} - -/// Standard predicates used in the vault triple store. -pub mod predicate { - // ── PROV-O (Provenance) ────────────────────────────────────── - /// An entity was associated with an agent (e.g., session → developer). - pub const WAS_ASSOCIATED_WITH: &str = "prov:wasAssociatedWith"; - /// An entity used another entity (e.g., session → file). - pub const USED: &str = "prov:used"; - /// An entity was generated by an activity (e.g., file → session). - pub const WAS_GENERATED_BY: &str = "prov:wasGeneratedBy"; - /// An activity was informed by another activity. - pub const WAS_INFORMED_BY: &str = "prov:wasInformedBy"; - /// An entity was derived from another entity. - pub const WAS_DERIVED_FROM: &str = "prov:wasDerivedFrom"; - /// An Atomic Intent was informed by a Memory. - pub const INFORMED_BY: &str = "atom:informedBy"; - /// An activity executed a plan (e.g., session turn → vault intent). - pub const HAD_PLAN: &str = "prov:hadPlan"; - /// A collection contains a member (e.g., session → session turn). - pub const HAD_MEMBER: &str = "prov:hadMember"; - /// An activity generated an entity (e.g., session turn → change). - pub const GENERATED: &str = "prov:generated"; - - // ── SKOS (Knowledge Organization) ──────────────────────────── - /// Broader concept (e.g., "authentication" → "security"). - pub const BROADER: &str = "skos:broader"; - /// Narrower concept (e.g., "security" → "authentication"). - pub const NARROWER: &str = "skos:narrower"; - /// Subject/topic (e.g., intent → concept). - pub const SUBJECT: &str = "skos:subject"; - /// Related concept (non-hierarchical). - pub const RELATED: &str = "skos:related"; - - // ── Dublin Core (Metadata) ─────────────────────────────────── - /// Resource type (e.g., "goal", "intent", "memory"). - pub const TYPE: &str = "dc:type"; - /// Creator of the resource. - pub const CREATOR: &str = "dc:creator"; - /// Date of creation/modification. - pub const DATE: &str = "dc:date"; - /// Human-readable description. - pub const DESCRIPTION: &str = "dc:description"; - - // ── RDF ────────────────────────────────────────────────────── - /// Type assertion (e.g., session:abc rdf:type vault:Goal). - pub const RDF_TYPE: &str = "rdf:type"; - - // ── Vault-specific ────────────────────────────────────────── - /// Intent blocks another intent. - pub const BLOCKS: &str = "vault:blocks"; - /// Intent is blocked by another intent. - pub const BLOCKED_BY: &str = "vault:blockedBy"; - /// Current status of an entity. - pub const STATUS: &str = "vault:status"; - /// Entity is assigned to a person/agent. - pub const ASSIGNED_TO: &str = "vault:assignedTo"; - /// Entity links to an intent. - pub const LINKED_INTENT: &str = "vault:linkedIntent"; - /// Entity links to a goal. - pub const LINKED_GOAL: &str = "vault:linkedGoal"; - /// Entity references another entity (wiki-link style). - pub const REFERENCES: &str = "vault:references"; - /// Entity applies to a concept/domain. - pub const APPLIES_TO: &str = "vault:appliesTo"; - /// Entity has a label/tag. - pub const HAS_LABEL: &str = "vault:hasLabel"; - /// Entity has priority. - pub const PRIORITY: &str = "vault:priority"; - /// A session turn is explained by a provenance graph. - pub const EXPLAINED_BY: &str = "vault:explainedBy"; - /// A session turn produced a todo snapshot (status in edge metadata). - pub const HAS_TODO: &str = "vault:hasTodo"; -} - -/// Vault entity type constants (used as objects with `rdf:type`). -pub mod entity_type { - pub const GOAL: &str = "vault:Goal"; - pub const INTENT: &str = "vault:Intent"; - pub const MEMORY: &str = "vault:Memory"; - pub const SKILL: &str = "vault:Skill"; - pub const TOOL_RESULT: &str = "vault:ToolResult"; - pub const FILE: &str = "vault:File"; - pub const IDENTITY: &str = "vault:Identity"; - pub const CONCEPT: &str = "vault:Concept"; - /// An agent session (one ledger). - pub const SESSION: &str = "vault:Session"; - /// One turn row in a session ledger. - pub const SESSION_TURN: &str = "vault:SessionTurn"; - /// A todo item tracked by an intent or snapshotted by a turn. - pub const TODO: &str = "vault:Todo"; - /// A content-addressed provenance graph explaining turn(s)/change(s). - pub const PROVENANCE: &str = "vault:ProvenanceGraph"; - /// A decomposed task lifted from an intent's directive body. - pub const TASK: &str = "vault:Task"; - /// An acceptance criterion lifted from an intent's directive body. - pub const ACCEPTANCE_CRITERION: &str = "vault:AcceptanceCriterion"; - /// A scope boundary item lifted from an intent's directive body. - pub const SCOPE_ITEM: &str = "vault:ScopeItem"; - /// A constraint lifted from an intent's directive body. - pub const CONSTRAINT: &str = "vault:Constraint"; -} - -/// Edge kinds used in the knowledge graph. -/// -/// These match the atomic-enterprise KG model for compatibility. -pub mod edge_kind { - /// Change modifies a file. - pub const MODIFIES: &str = "MODIFIES"; - /// Entity defines a symbol (function, class, etc.). - pub const DEFINES: &str = "DEFINES"; - /// Change depends on another change. - pub const DEPENDS_ON: &str = "DEPENDS_ON"; - /// Entity/change belongs to a view. - pub const ON_VIEW: &str = "ON_VIEW"; - /// View is a child of another view (parent chain). - pub const CHILD_OF: &str = "CHILD_OF"; - /// Change is visible from a view. - pub const VISIBLE_FROM: &str = "VISIBLE_FROM"; - /// Entity calls another entity. - pub const CALLS: &str = "CALLS"; - /// File imports from another file. - pub const IMPORTS: &str = "IMPORTS"; - /// File exports a symbol. - pub const EXPORTS: &str = "EXPORTS"; - /// Entity references another entity. - pub const REFERENCES: &str = "REFERENCES"; - /// Goal is linked to an intent. - pub const LINKED_INTENT: &str = "LINKED_INTENT"; - /// Goal is linked to another goal. - pub const LINKED_GOAL: &str = "LINKED_GOAL"; - /// Entity is assigned to an identity. - pub const ASSIGNED_TO: &str = "ASSIGNED_TO"; - /// Intent blocks another intent. - pub const BLOCKS: &str = "BLOCKS"; - /// Intent is blocked by another intent. - pub const BLOCKED_BY: &str = "BLOCKED_BY"; - /// Goal/intent was authored by an identity. - pub const AUTHORED_BY: &str = "AUTHORED_BY"; - /// Entity has a tag/label. - pub const HAS_LABEL: &str = "HAS_LABEL"; - /// File belongs to a module (directory). - pub const PART_OF: &str = "PART_OF"; - /// File includes/imports another file (via #include or import statement). - pub const INCLUDES: &str = "INCLUDES"; - - // ── Intent decomposition (lifted from the directive body) ──── - /// Intent has a task (decomposed work item). - pub const HAS_TASK: &str = "HAS_TASK"; - /// Intent has an acceptance criterion. - pub const HAS_ACCEPTANCE_CRITERION: &str = "HAS_ACCEPTANCE_CRITERION"; - /// Intent declares an in-scope boundary item. - pub const HAS_SCOPE_IN: &str = "HAS_SCOPE_IN"; - /// Intent declares an out-of-scope boundary item. - pub const HAS_SCOPE_OUT: &str = "HAS_SCOPE_OUT"; - /// Intent declares a constraint the implementation must respect. - pub const HAS_CONSTRAINT: &str = "HAS_CONSTRAINT"; - /// Task fulfills an acceptance criterion. - pub const SATISFIES: &str = "SATISFIES"; - /// Task touches a file. - pub const TOUCHES: &str = "TOUCHES"; - /// Intent remediates a flaw delivered by another intent. - pub const REMEDIATES: &str = "REMEDIATES"; - /// Review intent reviews another intent. - pub const REVIEWS: &str = "REVIEWS"; - /// Intent declares this file out of scope. - pub const SCOPE_OUT_FILE: &str = "SCOPE_OUT_FILE"; -} - -/// Check if a predicate string is a known predicate. -pub fn is_known_predicate(pred: &str) -> bool { - matches!( - pred, - predicate::WAS_ASSOCIATED_WITH - | predicate::USED - | predicate::WAS_GENERATED_BY - | predicate::WAS_INFORMED_BY - | predicate::WAS_DERIVED_FROM - | predicate::INFORMED_BY - | predicate::BROADER - | predicate::NARROWER - | predicate::SUBJECT - | predicate::RELATED - | predicate::TYPE - | predicate::CREATOR - | predicate::DATE - | predicate::DESCRIPTION - | predicate::RDF_TYPE - | predicate::BLOCKS - | predicate::BLOCKED_BY - | predicate::STATUS - | predicate::ASSIGNED_TO - | predicate::LINKED_INTENT - | predicate::LINKED_GOAL - | predicate::REFERENCES - | predicate::APPLIES_TO - | predicate::HAS_LABEL - | predicate::PRIORITY - | predicate::HAD_PLAN - | predicate::HAD_MEMBER - | predicate::GENERATED - | predicate::EXPLAINED_BY - | predicate::HAS_TODO - ) -} - -/// List all known predicates. -pub fn all_predicates() -> Vec<&'static str> { - vec![ - predicate::WAS_ASSOCIATED_WITH, - predicate::USED, - predicate::WAS_GENERATED_BY, - predicate::WAS_INFORMED_BY, - predicate::WAS_DERIVED_FROM, - predicate::INFORMED_BY, - predicate::BROADER, - predicate::NARROWER, - predicate::SUBJECT, - predicate::RELATED, - predicate::TYPE, - predicate::CREATOR, - predicate::DATE, - predicate::DESCRIPTION, - predicate::RDF_TYPE, - predicate::BLOCKS, - predicate::BLOCKED_BY, - predicate::STATUS, - predicate::ASSIGNED_TO, - predicate::LINKED_INTENT, - predicate::LINKED_GOAL, - predicate::REFERENCES, - predicate::APPLIES_TO, - predicate::HAS_LABEL, - predicate::PRIORITY, - predicate::HAD_PLAN, - predicate::HAD_MEMBER, - predicate::GENERATED, - predicate::EXPLAINED_BY, - predicate::HAS_TODO, - ] -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn all_predicates_are_known() { - for pred in all_predicates() { - assert!( - is_known_predicate(pred), - "Predicate '{}' not recognized", - pred - ); - } - } - - #[test] - fn predicate_count() { - assert_eq!(all_predicates().len(), 30); - } - - #[test] - fn session_ledger_terms_registered() { - // The ledger taxonomy vocabulary: session/turn typing plus the - // execution and explanation relations. - for pred in [ - predicate::HAD_PLAN, - predicate::HAD_MEMBER, - predicate::GENERATED, - predicate::EXPLAINED_BY, - predicate::HAS_TODO, - ] { - assert!(is_known_predicate(pred)); - } - for ty in [ - entity_type::SESSION, - entity_type::SESSION_TURN, - entity_type::TODO, - entity_type::PROVENANCE, - ] { - assert!(ty.starts_with("vault:")); - } - } - - #[test] - fn predicates_have_prefix() { - for pred in all_predicates() { - assert!(pred.contains(':'), "Predicate '{}' missing prefix", pred); - } - } - - #[test] - fn unknown_predicate_rejected() { - assert!(!is_known_predicate("foo:bar")); - assert!(!is_known_predicate("")); - } - - #[test] - fn edge_kinds_are_uppercase() { - // Convention: edge kinds are SCREAMING_SNAKE_CASE - let kinds = [ - edge_kind::MODIFIES, - edge_kind::DEFINES, - edge_kind::DEPENDS_ON, - edge_kind::ON_VIEW, - edge_kind::CHILD_OF, - edge_kind::VISIBLE_FROM, - edge_kind::CALLS, - edge_kind::IMPORTS, - edge_kind::EXPORTS, - edge_kind::REFERENCES, - edge_kind::LINKED_INTENT, - edge_kind::LINKED_GOAL, - edge_kind::ASSIGNED_TO, - edge_kind::BLOCKS, - edge_kind::BLOCKED_BY, - edge_kind::AUTHORED_BY, - edge_kind::HAS_LABEL, - edge_kind::PART_OF, - edge_kind::INCLUDES, - edge_kind::REMEDIATES, - edge_kind::REVIEWS, - edge_kind::SCOPE_OUT_FILE, - ]; - for kind in &kinds { - assert_eq!( - *kind, - kind.to_uppercase(), - "Edge kind '{}' should be uppercase", - kind - ); - } - } -} diff --git a/atomic-core/src/pristine/schema.rs b/atomic-core/src/pristine/schema.rs index 384f87df..8eda1d3c 100644 --- a/atomic-core/src/pristine/schema.rs +++ b/atomic-core/src/pristine/schema.rs @@ -151,6 +151,10 @@ pub(crate) fn visit_tables(visitor: &mut impl TableVisitor) -> PristineResult<() visitor.multimap(KG_FTS)?; visitor.multimap(KG_FTS_BY_NODE)?; visitor.table(KG_INDEX_META)?; + visitor.table(PROJECT_IDENTITY)?; + visitor.table(SIGNER_KEYS)?; + visitor.multimap(KG_TYPES)?; + visitor.multimap(KG_TYPES_REV)?; visitor.table(EMBEDDINGS)?; // Semantic CRDT diff --git a/atomic-core/src/pristine/tables.rs b/atomic-core/src/pristine/tables.rs index b8deb5d9..92d5861c 100644 --- a/atomic-core/src/pristine/tables.rs +++ b/atomic-core/src/pristine/tables.rs @@ -834,6 +834,15 @@ pub const KG_FTS_BY_NODE: MultimapTableDefinition<&str, &str> = /// KG index metadata used for transactional index migrations. pub const KG_INDEX_META: TableDefinition<&str, u32> = TableDefinition::new("kg_index_meta"); +/// View-independent project identity singleton (key 0). +pub const PROJECT_IDENTITY: TableDefinition = TableDefinition::new("project_identity"); +/// DID-bound Ed25519 public keys, used by offline attestation projection. +pub const SIGNER_KEYS: TableDefinition<&str, &str> = TableDefinition::new("signer_keys"); +/// Asserted class membership; excluded from kind-blind neighborhoods. +pub const KG_TYPES: MultimapTableDefinition<&str, &str> = MultimapTableDefinition::new("kg_types"); +pub const KG_TYPES_REV: MultimapTableDefinition<&str, &str> = + MultimapTableDefinition::new("kg_types_rev"); + // Embedding Tables (vector similarity search) /// Vector embeddings: vault_path → EmbeddingRecord (serialized) @@ -1349,11 +1358,15 @@ mod tests { #[test] fn test_edge_key_roundtrip() { - let key = encode_edge_key("change:abc", "file:auth.rs", "MODIFIES"); + let key = encode_edge_key( + "change:abc", + "file:auth.rs", + crate::kg::ontology::Predicate::ModifiesFile.iri(), + ); let (from, to, kind) = decode_edge_key(&key).unwrap(); assert_eq!(from, "change:abc"); assert_eq!(to, "file:auth.rs"); - assert_eq!(kind, "MODIFIES"); + assert_eq!(kind, crate::kg::ontology::Predicate::ModifiesFile.iri()); } #[test] diff --git a/atomic-core/src/pristine/traits/triples.rs b/atomic-core/src/pristine/traits/triples.rs index 41ebc1e9..34c480f3 100644 --- a/atomic-core/src/pristine/traits/triples.rs +++ b/atomic-core/src/pristine/traits/triples.rs @@ -16,6 +16,16 @@ pub trait KgTxnT { /// Get the 1-hop neighborhood of a node (outgoing + incoming edges + connected nodes). fn kg_neighbors(&self, node_id: &str, depth: u8) -> Result { + self.kg_neighbors_filtered(node_id, depth, None, crate::kg::Direction::Both) + } + + fn kg_neighbors_filtered( + &self, + node_id: &str, + depth: u8, + predicate: Option, + direction: crate::kg::Direction, + ) -> Result { let depth = depth.min(2); let mut visited = std::collections::HashSet::new(); let mut result_edges = Vec::new(); @@ -30,12 +40,22 @@ pub trait KgTxnT { for current_id in &frontier { for edge in self.get_kg_edges_from(current_id)? { + if direction == crate::kg::Direction::Incoming + || predicate.is_some_and(|p| p != edge.kind) + { + continue; + } if visited.insert(edge.to_id.clone()) { next_frontier.push(edge.to_id.clone()); } result_edges.push(edge); } for edge in self.get_kg_edges_to(current_id)? { + if direction == crate::kg::Direction::Outgoing + || predicate.is_some_and(|p| p != edge.kind) + { + continue; + } if visited.insert(edge.from_id.clone()) { next_frontier.push(edge.from_id.clone()); } @@ -49,8 +69,7 @@ pub trait KgTxnT { // Deduplicate edges { let mut seen = std::collections::HashSet::new(); - result_edges - .retain(|e| seen.insert((e.from_id.clone(), e.to_id.clone(), e.kind.clone()))); + result_edges.retain(|e| seen.insert((e.from_id.clone(), e.to_id.clone(), e.kind))); } // Collect nodes diff --git a/atomic-core/src/pristine/txn/kg_identity.rs b/atomic-core/src/pristine/txn/kg_identity.rs new file mode 100644 index 00000000..bffc48c0 --- /dev/null +++ b/atomic-core/src/pristine/txn/kg_identity.rs @@ -0,0 +1,197 @@ +use super::{ReadTxn, WriteTxn}; +use crate::kg::project_id::ProjectIdentityRecord; +use crate::pristine::tables::*; +use crate::pristine::{ + tables::{PROJECT_IDENTITY, SIGNER_KEYS}, + PristineError, PristineResult, +}; +use redb::{ReadableMultimapTable, ReadableTable}; + +fn serialization(error: impl std::fmt::Display) -> PristineError { + PristineError::Serialization { + message: error.to_string(), + } +} + +macro_rules! reads { + () => { + pub fn all_kg_nodes(&self) -> PristineResult> { + self.require_current_kg()?; + let table = match self.txn.open_table(KG_NODES) { + Ok(table) => table, + Err(redb::TableError::TableDoesNotExist(_)) => return Ok(Vec::new()), + Err(error) => return Err(error.into()), + }; + table + .iter()? + .map(|row| serde_json::from_slice(row?.1.value()).map_err(serialization)) + .collect() + } + pub fn all_kg_edges(&self) -> PristineResult> { + self.require_current_kg()?; + let table = match self.txn.open_table(KG_EDGES) { + Ok(table) => table, + Err(redb::TableError::TableDoesNotExist(_)) => return Ok(Vec::new()), + Err(error) => return Err(error.into()), + }; + table + .iter()? + .map(|row| serde_json::from_slice(row?.1.value()).map_err(serialization)) + .collect() + } + pub fn kg_node_types(&self, node: &str) -> PristineResult> { + self.require_current_kg()?; + let table = self.txn.open_multimap_table(KG_TYPES)?; + let result = table + .get(node)? + .map(|row| { + row.map(|row| row.value().to_owned()) + .map_err(PristineError::from) + }) + .collect(); + result + } + pub fn kg_nodes_of_type( + &self, + class: crate::kg::ontology::Class, + ) -> PristineResult> { + self.require_current_kg()?; + let table = self.txn.open_multimap_table(KG_TYPES_REV)?; + let result = table + .get(class.iri())? + .map(|row| { + row.map(|row| row.value().to_owned()) + .map_err(PristineError::from) + }) + .collect(); + result + } + pub fn kg_generation(&self) -> PristineResult { + let table = match self.txn.open_table(KG_INDEX_META) { + Ok(table) => table, + Err(redb::TableError::TableDoesNotExist(_)) => return Ok(0), + Err(error) => return Err(error.into()), + }; + let row = table.get("kg_generation")?; + Ok(row.map(|row| row.value()).unwrap_or(0)) + } + pub fn project_identity(&self) -> PristineResult> { + let table = match self.txn.open_table(PROJECT_IDENTITY) { + Ok(table) => table, + Err(redb::TableError::TableDoesNotExist(_)) => return Ok(None), + Err(error) => return Err(error.into()), + }; + let row = table.get(0)?; + row.map(|row| serde_json::from_slice(row.value()).map_err(serialization)) + .transpose() + } + pub fn signer_key(&self, did: &str) -> PristineResult> { + let table = match self.txn.open_table(SIGNER_KEYS) { + Ok(table) => table, + Err(redb::TableError::TableDoesNotExist(_)) => return Ok(None), + Err(error) => return Err(error.into()), + }; + let row = table.get(did)?; + Ok(row.map(|row| row.value().to_owned())) + } + }; +} +impl ReadTxn { + reads!(); + pub fn require_current_kg(&self) -> PristineResult<()> { + if self.kg_generation()? != crate::kg::KG_GENERATION { + return Err(PristineError::KgNeedsRebuild); + } + Ok(()) + } +} +impl WriteTxn<'_> { + reads!(); + + pub fn kg_session_rows( + &self, + ) -> PristineResult< + Vec<( + crate::change::session::SessionRecord, + Vec, + )>, + > { + use crate::change::session::{SessionRecord, SessionTurn}; + let mut turns = std::collections::BTreeMap::>::new(); + for row in self.txn.open_table(SESSION_TURNS)?.iter()? { + let turn = SessionTurn::from_bytes(row?.1.value()).map_err(serialization)?; + turns.entry(turn.session_id.clone()).or_default().push(turn); + } + let mut result = Vec::new(); + for row in self.txn.open_table(SESSIONS)?.iter()? { + let record = SessionRecord::from_bytes(row?.1.value()).map_err(serialization)?; + let mut rows = turns.remove(&record.session_id).unwrap_or_default(); + rows.sort_by_key(|row| row.turn_number); + result.push((record, rows)); + } + Ok(result) + } + + pub fn require_current_kg(&self) -> PristineResult<()> { + if !self.kg_rebuilding && self.kg_generation()? != crate::kg::KG_GENERATION { + return Err(PristineError::KgNeedsRebuild); + } + Ok(()) + } + + pub fn kg_writes_enabled(&self) -> PristineResult { + Ok(self.kg_rebuilding || self.kg_generation()? == crate::kg::KG_GENERATION) + } + + /// All derived tables are replaced atomically; abort leaves the old graph + /// and generation intact. No partially rebuilt graph becomes readable. + pub fn begin_kg_rebuild(&mut self) -> PristineResult<()> { + self.kg_rebuilding = true; + self.txn.delete_table(KG_NODES)?; + self.txn.delete_table(KG_EDGES)?; + self.txn.delete_multimap_table(KG_EDGES_FROM)?; + self.txn.delete_multimap_table(KG_EDGES_TO)?; + self.txn.delete_multimap_table(KG_FTS)?; + self.txn.delete_multimap_table(KG_FTS_BY_NODE)?; + self.txn.delete_multimap_table(KG_TYPES)?; + self.txn.delete_multimap_table(KG_TYPES_REV)?; + self.txn + .open_table(KG_INDEX_META)? + .remove("fts_reverse_schema")?; + crate::pristine::KgMutTxnT::init_kg(self)?; + Ok(()) + } + + pub fn finish_kg_rebuild(&mut self) -> PristineResult<()> { + self.txn + .open_table(KG_INDEX_META)? + .insert("kg_generation", crate::kg::KG_GENERATION)?; + self.txn + .open_table(PRISTINE_META)? + .insert("required-capability/kg-semweb-v1", 1)?; + self.kg_rebuilding = false; + Ok(()) + } + + /// Compare-and-insert; concurrent minting keeps the first committed identity. + pub fn put_project_identity_if_absent( + &mut self, + record: &ProjectIdentityRecord, + ) -> PristineResult { + if let Some(existing) = self.project_identity()? { + return Ok(existing); + } + record.project_id().map_err(serialization)?; + let bytes = serde_json::to_vec(record).map_err(serialization)?; + self.txn + .open_table(PROJECT_IDENTITY)? + .insert(0, bytes.as_slice())?; + Ok(record.clone()) + } + + /// Callers must check DID/key correspondence before caching key material. + pub fn put_signer_key(&mut self, did: &str, public_key: &str) -> PristineResult<()> { + self.txn.open_table(SIGNER_KEYS)?.insert(did, public_key)?; + Ok(()) + } +} diff --git a/atomic-core/src/pristine/txn/mod.rs b/atomic-core/src/pristine/txn/mod.rs index aac9bf8b..804a70e9 100644 --- a/atomic-core/src/pristine/txn/mod.rs +++ b/atomic-core/src/pristine/txn/mod.rs @@ -9,6 +9,7 @@ //! - `helpers` - Serialization and utility functions pub(super) mod helpers; +mod kg_identity; mod pristine; mod read; mod write; diff --git a/atomic-core/src/pristine/txn/pristine.rs b/atomic-core/src/pristine/txn/pristine.rs index a641db7f..3a2ec074 100644 --- a/atomic-core/src/pristine/txn/pristine.rs +++ b/atomic-core/src/pristine/txn/pristine.rs @@ -376,6 +376,18 @@ impl Pristine { write_txn.open_table(SESSION_PROVENANCE)?; write_txn.open_table(SESSION_MANIFESTS)?; write_txn.open_table(SESSION_HEADS)?; + write_txn.open_table(PROJECT_IDENTITY)?; + write_txn.open_table(SIGNER_KEYS)?; + write_txn.open_multimap_table(KG_TYPES)?; + write_txn.open_multimap_table(KG_TYPES_REV)?; + if is_new_database { + write_txn + .open_table(KG_INDEX_META)? + .insert("kg_generation", crate::kg::KG_GENERATION)?; + write_txn + .open_table(PRISTINE_META)? + .insert("required-capability/kg-semweb-v1", 1)?; + } stamp_schema_version(&write_txn)?; } @@ -783,20 +795,32 @@ mod tests { .unwrap(); assert_eq!( pristine.required_repository_capabilities().unwrap(), - vec![RequiredRepositoryCapability { - id: CHANGE_FORMAT_VNEXT_CAPABILITY.id().to_string(), - minimum_version: CHANGE_FORMAT_VNEXT_CAPABILITY.minimum_version(), - }] + vec![ + RequiredRepositoryCapability { + id: CHANGE_FORMAT_VNEXT_CAPABILITY.id().to_string(), + minimum_version: CHANGE_FORMAT_VNEXT_CAPABILITY.minimum_version(), + }, + RequiredRepositoryCapability { + id: "kg-semweb-v1".into(), + minimum_version: 1 + } + ] ); drop(pristine); let reopened = Pristine::open_readonly(&db_path).unwrap(); assert_eq!( reopened.required_repository_capabilities().unwrap(), - vec![RequiredRepositoryCapability { - id: "change-format-vnext".to_string(), - minimum_version: 1, - }] + vec![ + RequiredRepositoryCapability { + id: "change-format-vnext".to_string(), + minimum_version: 1, + }, + RequiredRepositoryCapability { + id: "kg-semweb-v1".into(), + minimum_version: 1 + } + ] ); } @@ -819,10 +843,16 @@ mod tests { .unwrap(); assert_eq!( pristine.required_repository_capabilities().unwrap(), - vec![RequiredRepositoryCapability { - id: "change-format-vnext".to_string(), - minimum_version: 1, - }] + vec![ + RequiredRepositoryCapability { + id: "change-format-vnext".to_string(), + minimum_version: 1, + }, + RequiredRepositoryCapability { + id: "kg-semweb-v1".into(), + minimum_version: 1 + } + ] ); } diff --git a/atomic-core/src/pristine/txn/read.rs b/atomic-core/src/pristine/txn/read.rs index f80763c1..01d12b1f 100644 --- a/atomic-core/src/pristine/txn/read.rs +++ b/atomic-core/src/pristine/txn/read.rs @@ -1371,6 +1371,7 @@ impl VaultTxnT for ReadTxn { impl KgTxnT for ReadTxn { fn get_kg_node(&self, id: &str) -> PristineResult> { + self.require_current_kg()?; let table = match self.txn.open_table(KG_NODES) { Ok(t) => t, Err(redb::TableError::TableDoesNotExist(_)) => return Ok(None), @@ -1390,6 +1391,7 @@ impl KgTxnT for ReadTxn { } fn get_kg_edges_from(&self, node_id: &str) -> PristineResult> { + self.require_current_kg()?; let from_table = match self.txn.open_multimap_table(KG_EDGES_FROM) { Ok(t) => t, Err(redb::TableError::TableDoesNotExist(_)) => return Ok(Vec::new()), @@ -1419,6 +1421,7 @@ impl KgTxnT for ReadTxn { } fn get_kg_edges_to(&self, node_id: &str) -> PristineResult> { + self.require_current_kg()?; let to_table = match self.txn.open_multimap_table(KG_EDGES_TO) { Ok(t) => t, Err(redb::TableError::TableDoesNotExist(_)) => return Ok(Vec::new()), @@ -1448,6 +1451,7 @@ impl KgTxnT for ReadTxn { } fn kg_fts_search(&self, query: &str, limit: usize) -> PristineResult> { + self.require_current_kg()?; let fts_table = match self.txn.open_multimap_table(KG_FTS) { Ok(t) => t, Err(redb::TableError::TableDoesNotExist(_)) => return Ok(Vec::new()), @@ -1474,40 +1478,11 @@ impl KgTxnT for ReadTxn { } } - // Sort by relevance: boost entity nodes (3x) and file nodes (2x) - // over change nodes (1x). Entities and files are more useful for - // code exploration than individual change records. - let mut ranked: Vec<(String, usize)> = hit_counts.into_iter().collect(); - ranked.sort_by(|a, b| { - let boost_a = if a.0.starts_with("entity:") { - a.1 * 3 - } else if a.0.starts_with("file:") { - a.1 * 2 - } else { - a.1 - }; - let boost_b = if b.0.starts_with("entity:") { - b.1 * 3 - } else if b.0.starts_with("file:") { - b.1 * 2 - } else { - b.1 - }; - boost_b.cmp(&boost_a) - }); - ranked.truncate(limit); - - // Fetch full nodes - let mut nodes = Vec::new(); - for (id, _) in &ranked { - if let Some(node) = self.get_kg_node(id)? { - nodes.push(node); - } - } - Ok(nodes) + crate::pristine::vault::rank_kg_hits(self, hit_counts, limit) } fn kg_fts_match_ids(&self, query: &str) -> PristineResult> { + self.require_current_kg()?; let fts_table = match self.txn.open_multimap_table(KG_FTS) { Ok(t) => t, Err(redb::TableError::TableDoesNotExist(_)) => return Ok(Vec::new()), @@ -1537,6 +1512,7 @@ impl KgTxnT for ReadTxn { } fn count_kg_nodes(&self) -> PristineResult { + self.require_current_kg()?; let table = match self.txn.open_table(KG_NODES) { Ok(t) => t, Err(redb::TableError::TableDoesNotExist(_)) => return Ok(0), @@ -1546,6 +1522,7 @@ impl KgTxnT for ReadTxn { } fn kg_node_ids_by_source(&self, sources: &[&str]) -> PristineResult> { + self.require_current_kg()?; let table = match self.txn.open_table(KG_NODES) { Ok(t) => t, Err(redb::TableError::TableDoesNotExist(_)) => return Ok(Vec::new()), @@ -1564,6 +1541,7 @@ impl KgTxnT for ReadTxn { } fn count_kg_edges(&self) -> PristineResult { + self.require_current_kg()?; let table = match self.txn.open_table(KG_EDGES) { Ok(t) => t, Err(redb::TableError::TableDoesNotExist(_)) => return Ok(0), diff --git a/atomic-core/src/pristine/txn/write/mod.rs b/atomic-core/src/pristine/txn/write/mod.rs index 9444a149..de59c823 100644 --- a/atomic-core/src/pristine/txn/write/mod.rs +++ b/atomic-core/src/pristine/txn/write/mod.rs @@ -95,6 +95,7 @@ fn allocate_shared_inode(next_inode: &AtomicU64) -> PristineResult { /// Provides read and write access to the pristine database. Only one write /// transaction can be active at a time. pub struct WriteTxn<'a> { + pub(crate) kg_rebuilding: bool, pub(crate) txn: WriteTransaction, pub(crate) next_node_id: &'a AtomicU64, pub(crate) next_view_id: &'a AtomicU64, @@ -111,6 +112,7 @@ impl<'a> WriteTxn<'a> { next_inode: &'a AtomicU64, ) -> Self { Self { + kg_rebuilding: false, txn, next_node_id, next_view_id, diff --git a/atomic-core/src/pristine/txn/write/session_kg.rs b/atomic-core/src/pristine/txn/write/session_kg.rs index ac74b711..d01299f2 100644 --- a/atomic-core/src/pristine/txn/write/session_kg.rs +++ b/atomic-core/src/pristine/txn/write/session_kg.rs @@ -1,468 +1,453 @@ -//! Session-ledger taxonomy — the single place where session-ledger rows are -//! arranged into the knowledge graph. -//! -//! Model (ATOM-16): the ontology (TBox, [`crate::pristine::ontology`]) types -//! the ledger's columns; this module is the ABox organization — the node-ID -//! minting, `rdf:type` assertions, and containment/derivation chains that -//! arrange session-ledger instances into the KG. The VCS graph stays -//! canonical and untouched: these triples classify over it, never alter it. -//! -//! Taxonomy rules (all of them, nowhere else): -//! -//! - Node IDs: `session:{id}`, `session:{id}/turn:{n}`, `provenance:{hash}`, -//! `change:{12-char-truncated}` — the last joins the enrichment convention; -//! the full hash rides in edge metadata for precision. -//! - Every relation uses an ontology predicate — no ad-hoc edge kinds. -//! - `rdf:type` is asserted in node metadata (`rdf_type` key, full URI); -//! `KgNode.kind` stays a short searchable category like the rest of the KG. -//! - Emission is idempotent (upsert semantics) and rides the caller's write -//! transaction; it never opens a second one. +//! Offline projection of authoritative session-ledger rows. Class assertions +//! live outside neighborhood edges; view context never asserts membership. +use super::WriteTxn; use crate::change::session::{SessionRecord, SessionTurn}; -use crate::pristine::error::PristineResult; -use crate::pristine::ontology::{edge_kind, entity_type, predicate}; -use crate::pristine::traits::{KgMutTxnT, KgTxnT}; +use crate::kg::{ + iri, + ontology::{Class, Predicate}, + project_id::ProjectId, +}; use crate::pristine::vault::{KgEdge, KgNode}; +use crate::pristine::{ + GraphTxnT, KgMutTxnT, KgTxnT, PristineError, PristineResult, VaultTxnT, ViewTxnT, +}; use crate::types::{Base32, Hash}; +use redb::ReadableTable; -use super::WriteTxn; - -/// Node id for a session ledger. -pub(crate) fn session_node_id(session_id: &str) -> String { - format!("session:{}", session_id) -} - -/// Node id for one turn row in a session ledger. -pub(crate) fn turn_node_id(session_id: &str, turn_number: u32) -> String { - format!("session:{}/turn:{}", session_id, turn_number) -} - -/// Node id for a provenance graph. -pub(crate) fn provenance_node_id(hash: &Hash) -> String { - format!("provenance:{}", hash.to_base32()) -} - -/// Node id for a change, using the enrichment convention (12-char truncated). -pub(crate) fn change_node_id(hash: &Hash) -> String { - let b32 = hash.to_base32(); - format!("change:{}", &b32[..12.min(b32.len())]) -} - -/// Node id for a todo. Agent-provided IDs are only stable within their -/// session, so qualify them with that session. Generated turn-local IDs are -/// already fully qualified and pass through unchanged. -pub(crate) fn todo_node_id(session_id: &str, todo_id: &str) -> String { - if todo_id.starts_with(&format!("session:{session_id}/")) { - todo_id.to_string() - } else { - format!("session:{session_id}/todo:{todo_id}") +impl WriteTxn<'_> { + pub fn kg_project(&self) -> PristineResult { + self.project_identity()? + .ok_or(PristineError::MissingProjectIdentity)? + .project_id() + .map_err(|e| PristineError::Serialization { + message: e.to_string(), + }) } -} -fn legacy_todo_node_id(todo_id: &str) -> String { - if todo_id.starts_with("session:") { - todo_id.to_string() - } else { - format!("todo:{todo_id}") + /// Resolve a plan through authoritative manifest identity; a raw label is + /// insufficient evidence to invent a canonical intent resource. + pub fn kg_intent_id(&self, raw: &str) -> PristineResult> { + let raw = raw.strip_prefix("urn:atomic:intent:").unwrap_or(raw); + let manifest = self.get_vault_manifest()?; + let summary = manifest.intents.iter().find(|(key, summary)| { + key.eq_ignore_ascii_case(raw) || summary.uid.eq_ignore_ascii_case(raw) + }); + Ok(summary.and_then(|(_, summary)| { + iri::mint_intent(&self.kg_project().ok()?, &summary.uid.to_uppercase()).ok() + })) } -} -fn is_session_owned_turn_edge(edge: &KgEdge, turn: &SessionTurn) -> bool { - let turn_id = turn_node_id(&turn.session_id, turn.turn_number); - let session_turn_prefix = format!("session:{}/turn:", turn.session_id); - if edge.from_id == turn_id { - return (edge.kind == predicate::EXPLAINED_BY - && edge.to_id == provenance_node_id(&turn.provenance_hash)) - || (edge.kind == predicate::HAD_PLAN - && turn.plan_id.as_ref().is_some_and(|plan_id| { - edge.to_id == format!("intent:{}", plan_id.to_uppercase()) - })) - || (edge.kind == predicate::HAS_TODO - && turn.todos.iter().any(|todo| { - edge.to_id == todo_node_id(&turn.session_id, &todo.id) - || edge.to_id == legacy_todo_node_id(&todo.id) - })) - || (edge.kind == predicate::GENERATED - && turn - .change_hashes - .iter() - .any(|hash| edge.to_id == change_node_id(hash))) - || (edge.kind == predicate::WAS_INFORMED_BY - && edge.to_id.starts_with(&session_turn_prefix)); + pub fn kg_finding(&mut self, code: &str, subject: &str, message: &str) -> PristineResult<()> { + let id = iri::mint_finding(&self.kg_project()?, code, subject); + self.upsert_kg_node( + &KgNode::new(&id, Class::Finding, code, "findings") + .with_summary(message) + .with_metadata(serde_json::json!({"finding_code": code})), + )?; + self.upsert_kg_edge(&KgEdge::new(id, subject, Predicate::AboutResource)) } - edge.to_id == turn_id - && ((edge.kind == predicate::HAD_MEMBER - && edge.from_id == session_node_id(&turn.session_id)) - || (edge.kind == predicate::WAS_INFORMED_BY - && edge.from_id.starts_with(&session_turn_prefix))) -} + pub fn kg_clear_finding(&mut self, code: &str, subject: &str) -> PristineResult<()> { + self.del_kg_node(&iri::mint_finding(&self.kg_project()?, code, subject))?; + Ok(()) + } -impl WriteTxn<'_> { - /// Remove derived turn/todo KG state before a session ledger is - /// re-numbered. Edges not owned by the session index are returned with - /// their turn endpoints remapped so callers can restore them afterward. pub(crate) fn clear_session_turn_kg( &mut self, - old_turns: &[SessionTurn], - new_turns: &[SessionTurn], + old: &[SessionTurn], + new: &[SessionTurn], ) -> PristineResult> { - let mut todo_ids = std::collections::BTreeSet::new(); - let old_turn_ids: std::collections::BTreeSet = old_turns - .iter() - .map(|turn| turn_node_id(&turn.session_id, turn.turn_number)) - .collect(); - let new_turn_ids: std::collections::HashMap = new_turns + if !self.kg_writes_enabled()? { + return Ok(Vec::new()); + } + let project = self.kg_project()?; + let replacements: std::collections::HashMap = new .iter() .map(|turn| { ( turn.provenance_hash, - turn_node_id(&turn.session_id, turn.turn_number), + iri::mint_turn(&project, &turn.session_id, turn.turn_number), ) }) .collect(); - let mut remapped_turn_ids = std::collections::HashMap::new(); + let mut remap = std::collections::HashMap::new(); let mut preserved = std::collections::BTreeMap::new(); - - for turn in old_turns { - let old_turn_id = turn_node_id(&turn.session_id, turn.turn_number); - if let Some(new_turn_id) = new_turn_ids.get(&turn.provenance_hash) { - remapped_turn_ids.insert(old_turn_id.clone(), new_turn_id.clone()); - } - for todo in &turn.todos { - // Current namespaced form. - todo_ids.insert(todo_node_id(&turn.session_id, &todo.id)); - // Pre-fix global form, for safe migration during a rewrite. - todo_ids.insert(legacy_todo_node_id(&todo.id)); + for turn in old { + let id = iri::mint_turn(&project, &turn.session_id, turn.turn_number); + if let Some(new_id) = replacements.get(&turn.provenance_hash) { + remap.insert(id.clone(), new_id.clone()); } - - let edges = self - .get_kg_edges_from(&old_turn_id)? + for edge in self + .get_kg_edges_from(&id)? .into_iter() - .chain(self.get_kg_edges_to(&old_turn_id)?); - for edge in edges { - if !is_session_owned_turn_edge(&edge, turn) { - preserved.insert( - (edge.from_id.clone(), edge.to_id.clone(), edge.kind.clone()), - edge, - ); + .chain(self.get_kg_edges_to(&id)?) + { + if edge + .metadata + .as_ref() + .and_then(|m| m.get("projector")) + .and_then(|m| m.as_str()) + != Some("session_ledger") + { + preserved.insert((edge.from_id.clone(), edge.to_id.clone(), edge.kind), edge); } } - } - - for turn_id in &old_turn_ids { - self.del_kg_node(turn_id)?; - } - - for todo_id in todo_ids { - let is_session_todo = self - .get_kg_node(&todo_id)? - .is_some_and(|node| node.kind == "todo" && node.source == "session_ledger"); - if is_session_todo - && self.get_kg_edges_from(&todo_id)?.is_empty() - && self.get_kg_edges_to(&todo_id)?.is_empty() - { - self.del_kg_node(&todo_id)?; + self.del_kg_node(&id)?; + for hash in &turn.change_hashes { + let change = iri::mint_change(hash); + for edge in self.get_kg_edges_from(&change)? { + if edge + .metadata + .as_ref() + .and_then(|m| m.get("turn_provenance")) + .and_then(|m| m.as_str()) + == Some(turn.provenance_hash.to_base32().as_str()) + { + self.del_kg_edge(&edge.from_id, &edge.to_id, &edge.kind)?; + } + } } } - Ok(preserved .into_values() .map(|mut edge| { - if let Some(new_id) = remapped_turn_ids.get(&edge.from_id) { - edge.from_id = new_id.clone(); + if let Some(id) = remap.get(&edge.from_id) { + edge.from_id = id.clone(); } - if let Some(new_id) = remapped_turn_ids.get(&edge.to_id) { - edge.to_id = new_id.clone(); + if let Some(id) = remap.get(&edge.to_id) { + edge.to_id = id.clone(); } edge }) .collect()) } - /// Check the derived nodes and edges covered by the session-ledger schema - /// marker before an idempotent rebuild skips a full rewrite. pub(crate) fn session_turn_kg_is_current( &self, - session_id: &str, + session: &str, turns: &[SessionTurn], ) -> PristineResult { - let session_node = session_node_id(session_id); - let is_current_session = self - .get_kg_node(&session_node)? - .is_some_and(|node| node.kind == "session" && node.source == "session_ledger"); - if !is_current_session { + if !self.kg_writes_enabled()? { + return Ok(true); + } + let project = self.kg_project()?; + if self + .get_kg_node(&iri::mint_session(&project, session))? + .is_none() + { return Ok(false); } - - let session_edges = self.get_kg_edges_from(&session_node)?; - let turn_numbers: std::collections::HashMap = turns - .iter() - .map(|turn| (turn.provenance_hash, turn.turn_number)) - .collect(); - let turn_prefix = format!("session:{session_id}/turn:"); - for turn in turns { - if turn.session_id != session_id { + let id = iri::mint_turn(&project, session, turn.turn_number); + let Some(node) = self.get_kg_node(&id)? else { return Ok(false); - } - - let turn_id = turn_node_id(&turn.session_id, turn.turn_number); - let is_current_turn = self - .get_kg_node(&turn_id)? - .is_some_and(|node| node.kind == "session_turn" && node.source == "session_ledger"); - if !is_current_turn - || !session_edges - .iter() - .any(|edge| edge.kind == predicate::HAD_MEMBER && edge.to_id == turn_id) + }; + if !node.has_type(Class::SessionTurn) + || node.has_type(Class::ManagedTurn) != turn.plan_id.is_some() { return Ok(false); } - - let turn_edges = self.get_kg_edges_from(&turn_id)?; - if !turn_edges.iter().any(|edge| { - edge.kind == predicate::EXPLAINED_BY - && edge.to_id == provenance_node_id(&turn.provenance_hash) - }) { - return Ok(false); - } - - let expected_previous = turn.previous_provenance.and_then(|hash| { - turn_numbers - .get(&hash) - .map(|number| turn_node_id(session_id, *number)) - }); - let actual_previous: Vec<&str> = turn_edges - .iter() - .filter(|edge| { - edge.kind == predicate::WAS_INFORMED_BY && edge.to_id.starts_with(&turn_prefix) - }) - .map(|edge| edge.to_id.as_str()) - .collect(); - match expected_previous { - Some(expected) - if actual_previous.len() == 1 && actual_previous[0] == expected.as_str() => {} - None if actual_previous.is_empty() => {} - _ => return Ok(false), - } - - if let Some(plan_id) = &turn.plan_id { - let intent_id = format!("intent:{}", plan_id.to_uppercase()); - if !turn_edges - .iter() - .any(|edge| edge.kind == predicate::HAD_PLAN && edge.to_id == intent_id) - || !session_edges - .iter() - .any(|edge| edge.kind == predicate::HAD_PLAN && edge.to_id == intent_id) - { - return Ok(false); - } - } - + let edges = self.get_kg_edges_from(&id)?; for todo in &turn.todos { - let current_id = todo_node_id(&turn.session_id, &todo.id); - let is_current_todo = self - .get_kg_node(¤t_id)? - .is_some_and(|node| node.kind == "todo" && node.source == "session_ledger"); - if !is_current_todo - || !turn_edges + let todo = iri::mint_todo(&project, session, &todo.id); + if self.get_kg_node(&todo)?.is_none() + || !edges .iter() - .any(|edge| edge.kind == predicate::HAS_TODO && edge.to_id == current_id) + .any(|edge| edge.kind == Predicate::HasTodo && edge.to_id == todo) { return Ok(false); } - - let legacy_id = legacy_todo_node_id(&todo.id); - if legacy_id != current_id - && self - .get_kg_node(&legacy_id)? - .is_some_and(|node| node.kind == "todo" && node.source == "session_ledger") + } + if let Some(previous) = turn.previous_provenance.and_then(|hash| { + turns + .iter() + .find(|candidate| candidate.provenance_hash == hash) + }) { + let previous = iri::mint_turn(&project, session, previous.turn_number); + if !edges + .iter() + .any(|edge| edge.kind == Predicate::ProvWasInformedBy && edge.to_id == previous) { return Ok(false); } } - - for change_hash in &turn.change_hashes { - let change_id = change_node_id(change_hash); - if !turn_edges - .iter() - .any(|edge| edge.kind == predicate::GENERATED && edge.to_id == change_id) - { + for hash in &turn.change_hashes { + if !edges.iter().any(|e| { + e.kind == Predicate::GeneratedChange && e.to_id == iri::mint_change(hash) + }) { return Ok(false); } } } - Ok(true) } - /// Upsert the session node with its current lifecycle metadata. pub(crate) fn emit_session_node(&mut self, record: &SessionRecord) -> PristineResult<()> { - let node = KgNode::new( - session_node_id(&record.session_id), - "session", - &record.session_id, - "session_ledger", - ) - .with_metadata(serde_json::json!({ - "rdf_type": entity_type::SESSION, - "view": record.view_name, - "parent_view": record.parent_view, - "status": record.status.label(), - "incomplete": record.status.incomplete(), - "turn_count": record.turn_count, - "json_path": record.json_path, - })); - self.upsert_kg_node(&node)?; - - // session ON_VIEW view — shares the enrichment's view vocabulary so - // `neighbors view:X` surfaces sessions alongside changes and files. + if !self.kg_writes_enabled()? { + return Ok(()); + } + let project = self.kg_project()?; + let id = iri::mint_session(&project, &record.session_id); + self.upsert_kg_node(&KgNode::new(&id, Class::Session, &record.session_id, "session_ledger") + .with_metadata(serde_json::json!({"view": record.view_name, "parent_view": record.parent_view, + "status": record.status.label(), "incomplete": record.status.incomplete(), "turn_count": record.turn_count, + "json_path": record.json_path})))?; + for edge in self.get_kg_edges_from(&id)? { + if edge.kind == Predicate::SessionViewContext { + self.del_kg_edge(&edge.from_id, &edge.to_id, &edge.kind)?; + } + } if let Some(view) = &record.view_name { - self.upsert_kg_edge(&KgEdge::new( - session_node_id(&record.session_id), - format!("view:{}", view), - edge_kind::ON_VIEW, - ))?; + let view_id = iri::mint_view(&project, view); + if self.get_view(view)?.is_some() { + if self.get_kg_node(&view_id)?.is_none() { + self.upsert_kg_node(&KgNode::new(&view_id, Class::View, view, "vcs"))?; + } + self.upsert_kg_edge(&KgEdge::new(&id, view_id, Predicate::SessionViewContext))?; + } } Ok(()) } - /// Emit one turn row's nodes and edges. - /// - /// Triples (all in the caller's transaction): - /// - `session:{id}` (upserted with latest lifecycle metadata) - /// - `provenance:{hash}` typed `vault:ProvenanceGraph` - /// - `session:{id}/turn:{n}` typed `vault:SessionTurn` - /// - session `prov:hadMember` turn - /// - turn `vault:explainedBy` provenance - /// - turn `prov:generated` change (per explained change; full hash in edge metadata) - /// - turn `prov:wasInformedBy` the turn named by `previous_provenance` pub(crate) fn emit_turn_kg( &mut self, record: &SessionRecord, turn: &SessionTurn, - previous_turn_number: Option, + previous: Option, ) -> PristineResult<()> { + if !self.kg_writes_enabled()? { + return Ok(()); + } + let project = self.kg_project()?; self.emit_session_node(record)?; - - let prov_id = provenance_node_id(&turn.provenance_hash); - let prov_node = KgNode::new( - &prov_id, - "provenance", - &turn.provenance_hash.to_base32()[..12], - "session_ledger", - ) - .with_metadata(serde_json::json!({ - "rdf_type": entity_type::PROVENANCE, - "session_id": turn.session_id, - "timestamp": turn.timestamp, - })); - self.upsert_kg_node(&prov_node)?; - - let turn_id = turn_node_id(&turn.session_id, turn.turn_number); - let turn_node = KgNode::new( - &turn_id, - "session_turn", - format!("turn {}", turn.turn_number), - "session_ledger", - ) - .with_metadata(serde_json::json!({ - "rdf_type": entity_type::SESSION_TURN, - "turn_number": turn.turn_number, - "timestamp": turn.timestamp, - "goal": turn.goal, - })); - let turn_node = match &turn.goal { - Some(goal) => turn_node.with_summary(goal), - None => turn_node, + let id = iri::mint_turn(&project, &turn.session_id, turn.turn_number); + // Replace only this projector's facts. Independent workflow references + // survive replay, while removed plans/todos/shortcuts cannot linger. + for old in self.get_kg_edges_from(&id)? { + if old + .metadata + .as_ref() + .and_then(|metadata| metadata.get("projector")) + .and_then(|value| value.as_str()) + == Some("session_ledger") + { + self.del_kg_edge(&old.from_id, &old.to_id, &old.kind)?; + } + } + for hash in &turn.change_hashes { + for old in self.get_kg_edges_from(&iri::mint_change(hash))? { + if old + .metadata + .as_ref() + .and_then(|metadata| metadata.get("turn_provenance")) + .and_then(|value| value.as_str()) + == Some(turn.provenance_hash.to_base32().as_str()) + { + self.del_kg_edge(&old.from_id, &old.to_id, &old.kind)?; + } + } + } + self.kg_clear_finding("UnresolvedReference", &id)?; + let created_at = chrono::DateTime::from_timestamp(turn.timestamp, 0) + .ok_or_else(|| PristineError::Serialization { + message: "session turn timestamp is outside the RDF dateTime range".into(), + })? + .to_rfc3339(); + let mut node = KgNode::new(&id, Class::SessionTurn, format!("turn {}", turn.turn_number), "session_ledger") + .with_metadata(serde_json::json!({"turn_number": turn.turn_number, "created_at": created_at, "goal": turn.goal, + "plan_id": turn.plan_id, "outcome": turn.outcome, "boundary_start": turn.boundary_start, "boundary_end": turn.boundary_end})); + if turn.plan_id.is_some() { + node = node.with_type(Class::ManagedTurn); + } + if let Some(goal) = &turn.goal { + node = node.with_summary(goal); + } + self.upsert_kg_node(&node)?; + let edge = |from: &str, to: &str, predicate| { + KgEdge::new(from, to, predicate) + .with_metadata(serde_json::json!({"projector": "session_ledger", "turn_provenance": turn.provenance_hash.to_base32()})) }; - self.upsert_kg_node(&turn_node)?; - - self.upsert_kg_edge(&KgEdge::new( - session_node_id(&turn.session_id), - &turn_id, - predicate::HAD_MEMBER, + self.upsert_kg_edge(&edge( + &iri::mint_session(&project, &turn.session_id), + &id, + Predicate::HasTurn, ))?; - - self.upsert_kg_edge(&KgEdge::new(&turn_id, &prov_id, predicate::EXPLAINED_BY))?; - - if let Some(plan_id) = &turn.plan_id { - let intent_id = format!("intent:{}", plan_id.to_uppercase()); - // The managed plan governs both the session and this concrete turn. - self.upsert_kg_edge(&KgEdge::new( - session_node_id(&turn.session_id), - &intent_id, - predicate::HAD_PLAN, + let provenance = iri::mint_provenance(&turn.provenance_hash); + let has_provenance = self.get_internal(&turn.provenance_hash)?.is_some(); + if has_provenance { + self.upsert_kg_node(&KgNode::new( + &provenance, + Class::ProvenanceGraph, + &turn.provenance_hash.to_base32()[..12], + "session_ledger", ))?; - self.upsert_kg_edge(&KgEdge::new(&turn_id, &intent_id, predicate::HAD_PLAN))?; + self.upsert_kg_edge(&edge(&id, &provenance, Predicate::TurnExplainedBy))?; } - - for todo in &turn.todos { - let todo_id = todo_node_id(&turn.session_id, &todo.id); - let todo_node = KgNode::new(&todo_id, "todo", &todo.content, "session_ledger") - .with_summary(&todo.content) - .with_metadata(serde_json::json!({ - "rdf_type": entity_type::TODO, - "todo_id": todo.id, - "status": todo.status, - "priority": todo.priority, - "session_id": turn.session_id, - "turn_number": turn.turn_number, - })); - self.upsert_kg_node(&todo_node)?; - self.upsert_kg_edge( - &KgEdge::new(&turn_id, &todo_id, predicate::HAS_TODO).with_metadata( - serde_json::json!({ - "status": todo.status, - "priority": todo.priority, - }), - ), + let plan = match turn.plan_id.as_deref() { + Some(raw) => self.kg_intent_id(raw)?, + None => None, + }; + if let Some(plan) = &plan { + if self.get_kg_node(plan)?.is_some() { + self.upsert_kg_edge(&edge(&id, plan, Predicate::HasPlan))?; + } else { + self.kg_finding( + "UnresolvedReference", + &id, + "The governing intent is not projected locally", + )?; + } + } else if turn.plan_id.is_some() { + self.kg_finding( + "UnresolvedReference", + &id, + "The governing intent could not be resolved through the vault manifest", )?; } + for todo in &turn.todos { + let todo_id = iri::mint_todo(&project, &turn.session_id, &todo.id); + self.upsert_kg_node(&KgNode::new(&todo_id, Class::Todo, &todo.content, "session_ledger") + .with_summary(&todo.content).with_metadata(serde_json::json!({"todo_id": todo.id, "status": todo.status, + "priority": todo.priority, "session_id": turn.session_id, "turn_number": turn.turn_number})))?; + self.upsert_kg_edge(&edge(&id, &todo_id, Predicate::HasTodo))?; + } + for hash in &turn.change_hashes { + let change = iri::mint_change(hash); + if self.get_kg_node(&change)?.is_none() { + self.upsert_kg_node( + &KgNode::new( + &change, + Class::Change, + &hash.to_base32()[..12], + "session_ledger", + ) + .with_metadata(serde_json::json!({"hash": hash.to_base32()})), + )?; + } + self.upsert_kg_edge(&edge(&id, &change, Predicate::GeneratedChange))?; + self.kg_clear_finding("MissingPlanId", &change)?; + if let Some(plan) = plan + .as_ref() + .filter(|plan| self.get_kg_node(plan).ok().flatten().is_some()) + { + self.upsert_kg_edge(&edge(&change, plan, Predicate::Implements))?; + } else { + self.kg_finding( + "MissingPlanId", + &change, + "The generating turn has no locally resolved governing intent", + )?; + } + self.kg_clear_finding("MissingProvenance", &change)?; + if has_provenance { + self.upsert_kg_edge(&edge(&change, &provenance, Predicate::ChangeExplainedBy))?; + } else { + self.kg_finding( + "MissingProvenance", + &change, + "The generating turn's provenance sidecar is absent", + )?; + } + if !self + .get_kg_edges_from(&change)? + .iter() + .any(|edge| edge.kind == Predicate::AttestedBy) + { + self.kg_finding( + "MissingAttestation", + &change, + "No covering attestation is registered locally", + )?; + } + } + if let Some(previous) = previous { + let previous = iri::mint_turn(&project, &turn.session_id, previous); + if self.get_kg_node(&previous)?.is_some() { + self.upsert_kg_edge(&edge(&id, &previous, Predicate::ProvWasInformedBy))?; + } + } + Ok(()) + } - for change_hash in &turn.change_hashes { - self.upsert_kg_edge( - &KgEdge::new(&turn_id, change_node_id(change_hash), predicate::GENERATED) - .with_metadata(serde_json::json!({ - "change_hash": change_hash.to_base32(), - })), + pub fn project_session_kg( + &mut self, + record: &SessionRecord, + turns: &[SessionTurn], + ) -> PristineResult<()> { + if !self.kg_writes_enabled()? { + return Ok(()); + } + self.emit_session_node(record)?; + let numbers: std::collections::HashMap = turns + .iter() + .map(|turn| (turn.provenance_hash, turn.turn_number)) + .collect(); + for turn in turns { + self.emit_turn_kg( + record, + turn, + turn.previous_provenance + .and_then(|hash| numbers.get(&hash).copied()), )?; } - - if let Some(previous_turn_number) = previous_turn_number { - self.upsert_kg_edge(&KgEdge::new( - &turn_id, - turn_node_id(&turn.session_id, previous_turn_number), - predicate::WAS_INFORMED_BY, - ))?; + let lineage = { + use crate::change::session::SessionManifest; + use crate::pristine::tables::{SESSION_HEADS, SESSION_MANIFESTS}; + let heads = self.txn.open_table(SESSION_HEADS)?; + let manifests = self.txn.open_table(SESSION_MANIFESTS)?; + let decode = |bytes: &[u8]| { + SessionManifest::from_bytes(bytes).map_err(|error| PristineError::Serialization { + message: error.to_string(), + }) + }; + let mut lineage = None; + if let Some(head) = heads.get(record.session_id.as_str())? { + if let Some(bytes) = manifests.get(head.value())? { + let manifest = decode(bytes.value())?; + if let (Some(parent), Some(turn)) = + (manifest.parent_session, manifest.fork_turn) + { + if let Some(bytes) = manifests.get(parent.as_bytes())? { + lineage = Some((decode(bytes.value())?.session_id, turn, parent)); + } + } + } + } + lineage + }; + if let Some((parent, turn, manifest)) = lineage { + let parent_id = iri::mint_session(&self.kg_project()?, &parent); + if self.get_kg_node(&parent_id)?.is_none() { + self.upsert_kg_node(&KgNode::new( + parent_id, + Class::Session, + &parent, + "session_ledger", + ))?; + } + self.emit_session_fork_kg(&record.session_id, &parent, turn, &manifest)?; } - Ok(()) } - /// Emit the fork-derivation triple: child session `prov:wasDerivedFrom` - /// parent session, with the fork boundary in edge metadata. - /// - /// Public because the fork boundary is decided at the repository layer - /// (`Repository::fork_session`); the turn/session emission stays internal - /// to the index paths. pub fn emit_session_fork_kg( &mut self, - child_session_id: &str, - parent_session_id: &str, + child: &str, + parent: &str, fork_turn: u32, parent_manifest: &Hash, ) -> PristineResult<()> { - self.upsert_kg_edge( - &KgEdge::new( - session_node_id(child_session_id), - session_node_id(parent_session_id), - predicate::WAS_DERIVED_FROM, - ) - .with_metadata(serde_json::json!({ - "fork_turn": fork_turn, - "parent_manifest": parent_manifest.to_base32(), - })), - ) + if !self.kg_writes_enabled()? { + return Ok(()); + } + let project = self.kg_project()?; + let child_id = iri::mint_session(&project, child); + let parent_id = iri::mint_session(&project, parent); + if self.get_kg_node(&child_id)?.is_some() && self.get_kg_node(&parent_id)?.is_some() { + self.upsert_kg_edge(&KgEdge::new(child_id, parent_id, Predicate::ProvWasDerivedFrom) + .with_metadata(serde_json::json!({"fork_turn": fork_turn, "parent_manifest": parent_manifest.to_base32()})))?; + } + Ok(()) } } diff --git a/atomic-core/src/pristine/txn/write/tests.rs b/atomic-core/src/pristine/txn/write/tests.rs index 04fb6df4..ecb6a4e9 100644 --- a/atomic-core/src/pristine/txn/write/tests.rs +++ b/atomic-core/src/pristine/txn/write/tests.rs @@ -8,6 +8,22 @@ mod tests { use crate::pristine::Pristine; use tempfile::tempdir; + fn seed_project_identity(txn: &mut WriteTxn<'_>) { + use crate::kg::project_id::{MintReason, ProjectId, ProjectIdentityRecord}; + let object = serde_json::to_vec( + &serde_json::json!({"atomic:projectId": ProjectId::generate().to_base32()}), + ) + .unwrap(); + txn.put_project_identity_if_absent(&ProjectIdentityRecord { + hash: Hash::of(&object), + object, + mint_reason: MintReason::Upgrade, + created_at: "2026-10-15T00:00:00Z".into(), + previous: None, + }) + .unwrap(); + } + #[test] fn test_register_change() { let dir = tempdir().unwrap(); @@ -1173,6 +1189,7 @@ mod tests { ); let mut txn = pristine.write_txn().unwrap(); + seed_project_identity(&mut txn); let persisted = txn .mark_session_incomplete( "sess-incomplete", @@ -1249,6 +1266,7 @@ mod tests { }; let mut txn = pristine.write_txn().unwrap(); + seed_project_identity(&mut txn); txn.index_session_turn( "sess-b", ".atomic/sessions/sess-b.json", diff --git a/atomic-core/src/pristine/txn/write/triples.rs b/atomic-core/src/pristine/txn/write/triples.rs index 3ce9cb18..b9bac524 100644 --- a/atomic-core/src/pristine/txn/write/triples.rs +++ b/atomic-core/src/pristine/txn/write/triples.rs @@ -155,6 +155,7 @@ impl WriteTxn<'_> { impl<'a> KgTxnT for WriteTxn<'a> { fn get_kg_node(&self, id: &str) -> PristineResult> { + self.require_current_kg()?; let table = match self.txn.open_table(KG_NODES) { Ok(t) => t, Err(redb::TableError::TableDoesNotExist(_)) => return Ok(None), @@ -176,6 +177,7 @@ impl<'a> KgTxnT for WriteTxn<'a> { } fn get_kg_edges_from(&self, node_id: &str) -> PristineResult> { + self.require_current_kg()?; let from_table = match self.txn.open_multimap_table(KG_EDGES_FROM) { Ok(t) => t, Err(redb::TableError::TableDoesNotExist(_)) => return Ok(Vec::new()), @@ -205,6 +207,7 @@ impl<'a> KgTxnT for WriteTxn<'a> { } fn get_kg_edges_to(&self, node_id: &str) -> PristineResult> { + self.require_current_kg()?; let to_table = match self.txn.open_multimap_table(KG_EDGES_TO) { Ok(t) => t, Err(redb::TableError::TableDoesNotExist(_)) => return Ok(Vec::new()), @@ -234,6 +237,7 @@ impl<'a> KgTxnT for WriteTxn<'a> { } fn kg_fts_search(&self, query: &str, limit: usize) -> PristineResult> { + self.require_current_kg()?; let fts_table = match self.txn.open_multimap_table(KG_FTS) { Ok(t) => t, Err(redb::TableError::TableDoesNotExist(_)) => return Ok(Vec::new()), @@ -260,40 +264,11 @@ impl<'a> KgTxnT for WriteTxn<'a> { } } - // Sort by relevance: boost entity nodes (3x) and file nodes (2x) - // over change nodes (1x). Entities and files are more useful for - // code exploration than individual change records. - let mut ranked: Vec<(String, usize)> = hit_counts.into_iter().collect(); - ranked.sort_by(|a, b| { - let boost_a = if a.0.starts_with("entity:") { - a.1 * 3 - } else if a.0.starts_with("file:") { - a.1 * 2 - } else { - a.1 - }; - let boost_b = if b.0.starts_with("entity:") { - b.1 * 3 - } else if b.0.starts_with("file:") { - b.1 * 2 - } else { - b.1 - }; - boost_b.cmp(&boost_a) - }); - ranked.truncate(limit); - - // Fetch full nodes - let mut nodes = Vec::new(); - for (id, _) in &ranked { - if let Some(node) = self.get_kg_node(id)? { - nodes.push(node); - } - } - Ok(nodes) + crate::pristine::vault::rank_kg_hits(self, hit_counts, limit) } fn kg_fts_match_ids(&self, query: &str) -> PristineResult> { + self.require_current_kg()?; let fts_table = match self.txn.open_multimap_table(KG_FTS) { Ok(t) => t, Err(redb::TableError::TableDoesNotExist(_)) => return Ok(Vec::new()), @@ -323,6 +298,7 @@ impl<'a> KgTxnT for WriteTxn<'a> { } fn count_kg_nodes(&self) -> PristineResult { + self.require_current_kg()?; let table = match self.txn.open_table(KG_NODES) { Ok(t) => t, Err(redb::TableError::TableDoesNotExist(_)) => return Ok(0), @@ -332,6 +308,7 @@ impl<'a> KgTxnT for WriteTxn<'a> { } fn kg_node_ids_by_source(&self, sources: &[&str]) -> PristineResult> { + self.require_current_kg()?; let table = match self.txn.open_table(KG_NODES) { Ok(t) => t, Err(redb::TableError::TableDoesNotExist(_)) => return Ok(Vec::new()), @@ -350,6 +327,7 @@ impl<'a> KgTxnT for WriteTxn<'a> { } fn count_kg_edges(&self) -> PristineResult { + self.require_current_kg()?; let table = match self.txn.open_table(KG_EDGES) { Ok(t) => t, Err(redb::TableError::TableDoesNotExist(_)) => return Ok(0), @@ -361,6 +339,38 @@ impl<'a> KgTxnT for WriteTxn<'a> { impl<'a> KgMutTxnT for WriteTxn<'a> { fn upsert_kg_node(&mut self, node: &KgNode) -> PristineResult<()> { + if !self.kg_writes_enabled()? { + return Ok(()); + } + if node.types.is_empty() + || node + .types + .iter() + .any(|iri| crate::kg::ontology::Class::from_iri(iri).is_none()) + { + return Err(PristineError::Serialization { + message: "node must have canonical asserted classes".into(), + }); + } + let old_types: Vec = self + .txn + .open_multimap_table(KG_TYPES)? + .remove_all(node.id.as_str())? + .map(|row| row.map(|row| row.value().to_owned())) + .collect::>()?; + for class in old_types { + self.txn + .open_multimap_table(KG_TYPES_REV)? + .remove(class.as_str(), node.id.as_str())?; + } + for class in &node.types { + self.txn + .open_multimap_table(KG_TYPES)? + .insert(node.id.as_str(), class.as_str())?; + self.txn + .open_multimap_table(KG_TYPES_REV)? + .insert(class.as_str(), node.id.as_str())?; + } // Replacement semantics are important: old ID/label/summary tokens // must disappear before the same node ID is re-indexed. self.remove_kg_fts_entries(&node.id)?; @@ -378,6 +388,9 @@ impl<'a> KgMutTxnT for WriteTxn<'a> { } fn upsert_kg_edge(&mut self, edge: &KgEdge) -> PristineResult<()> { + if !self.kg_writes_enabled()? { + return Ok(()); + } let edge_key = encode_edge_key(&edge.from_id, &edge.to_id, &edge.kind); let mut edges_table = self.txn.open_table(KG_EDGES)?; @@ -399,6 +412,20 @@ impl<'a> KgMutTxnT for WriteTxn<'a> { } fn del_kg_node(&mut self, id: &str) -> PristineResult { + if !self.kg_writes_enabled()? { + return Ok(false); + } + let classes: Vec = self + .txn + .open_multimap_table(KG_TYPES)? + .remove_all(id)? + .map(|row| row.map(|row| row.value().to_owned())) + .collect::>()?; + for class in classes { + self.txn + .open_multimap_table(KG_TYPES_REV)? + .remove(class.as_str(), id)?; + } // First delete all edges for this node self.del_kg_edges_for_node(id)?; @@ -419,6 +446,9 @@ impl<'a> KgMutTxnT for WriteTxn<'a> { } fn del_kg_edge(&mut self, from_id: &str, to_id: &str, kind: &str) -> PristineResult { + if !self.kg_writes_enabled()? { + return Ok(false); + } let edge_key = encode_edge_key(from_id, to_id, kind); let mut edges_table = match self.txn.open_table(KG_EDGES) { @@ -444,6 +474,9 @@ impl<'a> KgMutTxnT for WriteTxn<'a> { } fn del_kg_edges_for_node(&mut self, node_id: &str) -> PristineResult { + if !self.kg_writes_enabled()? { + return Ok(0); + } // Collect all edge keys for this node (both directions) let outgoing = self.get_kg_edges_from(node_id)?; let incoming = self.get_kg_edges_to(node_id)?; @@ -458,6 +491,11 @@ impl<'a> KgMutTxnT for WriteTxn<'a> { } fn init_kg(&mut self) -> PristineResult<()> { + if !self.kg_writes_enabled()? { + return Ok(()); + } + self.txn.open_multimap_table(KG_TYPES)?; + self.txn.open_multimap_table(KG_TYPES_REV)?; self.txn.open_table(KG_NODES)?; self.txn.open_table(KG_EDGES)?; self.txn.open_multimap_table(KG_EDGES_FROM)?; @@ -470,6 +508,7 @@ impl<'a> KgMutTxnT for WriteTxn<'a> { #[cfg(test)] mod tests { use super::*; + use crate::kg::ontology::{Class as C, Predicate as P}; use crate::pristine::traits::MutTxnT; use crate::pristine::Pristine; use tempfile::tempdir; @@ -481,13 +520,13 @@ mod tests { let mut txn = pristine.write_txn().unwrap(); txn.init_kg().unwrap(); - let node = KgNode::new("change:abc123", "change", "abc123", "change_graph") + let node = KgNode::new("change:abc123", C::Change, "abc123", "change_graph") .with_summary("Fix auth bug"); txn.upsert_kg_node(&node).unwrap(); let retrieved = txn.get_kg_node("change:abc123").unwrap().unwrap(); assert_eq!(retrieved.id, "change:abc123"); - assert_eq!(retrieved.kind, "change"); + assert!(retrieved.has_type(C::Change)); assert_eq!(retrieved.summary, Some("Fix auth bug".to_string())); assert!(txn.get_kg_node("nonexistent").unwrap().is_none()); @@ -506,17 +545,17 @@ mod tests { txn.init_kg().unwrap(); // Create nodes - txn.upsert_kg_node(&KgNode::new("change:abc", "change", "abc", "graph")) + txn.upsert_kg_node(&KgNode::new("change:abc", C::Change, "abc", "graph")) .unwrap(); - txn.upsert_kg_node(&KgNode::new("file:auth.rs", "file", "auth.rs", "graph")) + txn.upsert_kg_node(&KgNode::new("file:auth.rs", C::File, "auth.rs", "graph")) .unwrap(); - txn.upsert_kg_node(&KgNode::new("file:main.rs", "file", "main.rs", "graph")) + txn.upsert_kg_node(&KgNode::new("file:main.rs", C::File, "main.rs", "graph")) .unwrap(); // Create edges - txn.upsert_kg_edge(&KgEdge::new("change:abc", "file:auth.rs", "MODIFIES")) + txn.upsert_kg_edge(&KgEdge::new("change:abc", "file:auth.rs", P::ModifiesFile)) .unwrap(); - txn.upsert_kg_edge(&KgEdge::new("change:abc", "file:main.rs", "MODIFIES")) + txn.upsert_kg_edge(&KgEdge::new("change:abc", "file:main.rs", P::ModifiesFile)) .unwrap(); // Query outgoing @@ -530,7 +569,7 @@ mod tests { // Delete edge assert!(txn - .del_kg_edge("change:abc", "file:auth.rs", "MODIFIES") + .del_kg_edge("change:abc", "file:auth.rs", P::ModifiesFile.iri()) .unwrap()); assert_eq!(txn.get_kg_edges_from("change:abc").unwrap().len(), 1); @@ -548,14 +587,16 @@ mod tests { let mut txn = pristine.write_txn().unwrap(); txn.init_kg().unwrap(); - txn.upsert_kg_node(&KgNode::new("a", "x", "A", "test")) + txn.upsert_kg_node(&KgNode::new("a", C::Memory, "A", "test")) .unwrap(); - txn.upsert_kg_node(&KgNode::new("b", "x", "B", "test")) + txn.upsert_kg_node(&KgNode::new("b", C::Memory, "B", "test")) .unwrap(); - txn.upsert_kg_node(&KgNode::new("c", "x", "C", "test")) + txn.upsert_kg_node(&KgNode::new("c", C::Memory, "C", "test")) + .unwrap(); + txn.upsert_kg_edge(&KgEdge::new("a", "b", P::References)) + .unwrap(); + txn.upsert_kg_edge(&KgEdge::new("b", "c", P::References)) .unwrap(); - txn.upsert_kg_edge(&KgEdge::new("a", "b", "LINK")).unwrap(); - txn.upsert_kg_edge(&KgEdge::new("b", "c", "LINK")).unwrap(); // Depth 1 from "a" let sg = txn.kg_neighbors("a", 1).unwrap(); @@ -578,12 +619,12 @@ mod tests { txn.init_kg().unwrap(); txn.upsert_kg_node( - &KgNode::new("change:abc", "change", "abc123", "graph") + &KgNode::new("change:abc", C::Change, "abc123", "graph") .with_summary("Fix authentication bug"), ) .unwrap(); txn.upsert_kg_node( - &KgNode::new("change:def", "change", "def456", "graph").with_summary("Add logging"), + &KgNode::new("change:def", C::Change, "def456", "graph").with_summary("Add logging"), ) .unwrap(); @@ -612,12 +653,12 @@ mod tests { let mut txn = pristine.write_txn().unwrap(); txn.init_kg().unwrap(); - let old = KgNode::new("memory:build", "memory", "Build", "vault") + let old = KgNode::new("memory:build", C::Memory, "Build", "vault") .with_summary("obsoletecapybara summary"); txn.upsert_kg_node(&old).unwrap(); assert_eq!(txn.kg_fts_match_ids("obsoletecapybara").unwrap().len(), 1); - let updated = KgNode::new("memory:build", "memory", "Build", "vault") + let updated = KgNode::new("memory:build", C::Memory, "Build", "vault") .with_summary("currentwombat summary"); txn.upsert_kg_node(&updated).unwrap(); @@ -634,12 +675,12 @@ mod tests { let mut txn = pristine.write_txn().unwrap(); txn.init_kg().unwrap(); - let node = KgNode::new("memory:reused", "memory", "Original", "vault") + let node = KgNode::new("memory:reused", C::Memory, "Original", "vault") .with_summary("ghostplatypus"); txn.upsert_kg_node(&node).unwrap(); assert!(txn.del_kg_node(&node.id).unwrap()); - let replacement = KgNode::new("memory:reused", "memory", "Replacement", "vault"); + let replacement = KgNode::new("memory:reused", C::Memory, "Replacement", "vault"); txn.upsert_kg_node(&replacement).unwrap(); assert!(txn.kg_fts_match_ids("ghostplatypus").unwrap().is_empty()); @@ -655,7 +696,7 @@ mod tests { // Simulate an index written before replacement-safe indexing existed, // including obsolete metadata and a deleted node's orphan posting. { - let node = KgNode::new("memory:legacy", "memory", "Legacy", "vault") + let node = KgNode::new("memory:legacy", C::Memory, "Legacy", "vault") .with_summary("legacyechidna"); let bytes = serde_json::to_vec(&node).unwrap(); let mut nodes = txn.txn.open_table(KG_NODES).unwrap(); @@ -689,12 +730,14 @@ mod tests { let mut txn = pristine.write_txn().unwrap(); txn.init_kg().unwrap(); - txn.upsert_kg_node(&KgNode::new("a", "x", "A", "t")) + txn.upsert_kg_node(&KgNode::new("a", C::Memory, "A", "t")) + .unwrap(); + txn.upsert_kg_node(&KgNode::new("b", C::Memory, "B", "t")) .unwrap(); - txn.upsert_kg_node(&KgNode::new("b", "x", "B", "t")) + txn.upsert_kg_edge(&KgEdge::new("a", "b", P::References)) + .unwrap(); + txn.upsert_kg_edge(&KgEdge::new("b", "a", P::References)) .unwrap(); - txn.upsert_kg_edge(&KgEdge::new("a", "b", "LINK")).unwrap(); - txn.upsert_kg_edge(&KgEdge::new("b", "a", "BACK")).unwrap(); txn.del_kg_node("a").unwrap(); @@ -711,9 +754,10 @@ mod tests { { let mut txn = pristine.write_txn().unwrap(); txn.init_kg().unwrap(); - txn.upsert_kg_node(&KgNode::new("x", "t", "X", "s")) + txn.upsert_kg_node(&KgNode::new("x", C::Memory, "X", "s")) + .unwrap(); + txn.upsert_kg_edge(&KgEdge::new("x", "y", P::References)) .unwrap(); - txn.upsert_kg_edge(&KgEdge::new("x", "y", "LINK")).unwrap(); txn.commit().unwrap(); } { diff --git a/atomic-core/src/pristine/vault.rs b/atomic-core/src/pristine/vault.rs index 85a0847b..e83d45be 100644 --- a/atomic-core/src/pristine/vault.rs +++ b/atomic-core/src/pristine/vault.rs @@ -7,6 +7,31 @@ use serde::{Deserialize, Serialize}; use std::collections::HashMap; +pub(crate) fn rank_kg_hits( + txn: &impl crate::pristine::KgTxnT, + hits: HashMap, + limit: usize, +) -> Result, crate::pristine::PristineError> { + let mut ranked = Vec::new(); + for (id, hits) in hits { + if let Some(node) = txn.get_kg_node(&id)? { + let score = hits + * if node.has_type(crate::kg::ontology::Class::File) { + 2 + } else { + 1 + }; + ranked.push((score, node)); + } + } + ranked.sort_by(|a, b| b.0.cmp(&a.0).then_with(|| a.1.id.cmp(&b.1.id))); + Ok(ranked + .into_iter() + .take(limit) + .map(|(_, node)| node) + .collect()) +} + /// Type of vault entry. #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)] #[serde(rename_all = "snake_case")] @@ -463,8 +488,8 @@ pub fn parse_intent_reference( pub struct KgNode { /// Structured identifier (e.g., `"change:{hash}"`, `"file:{path}"`). pub id: String, - /// Node category: `"change"`, `"file"`, `"entity"`, `"view"`, `"goal"`, `"intent"`, `"memory"`. - pub kind: String, + /// Asserted absolute class IRIs (also indexed in KG_TYPES). + pub types: Vec, /// Human-readable label (short hash, filename, symbol name). pub label: String, /// Optional longer description (commit message, entity signature, etc.). @@ -478,16 +503,35 @@ pub struct KgNode { } impl KgNode { + pub fn has_type(&self, class: crate::kg::ontology::Class) -> bool { + self.types.iter().any(|iri| iri == class.iri()) + } + + pub fn primary_type(&self) -> &str { + self.types.first().map(String::as_str).unwrap_or_default() + } + + pub fn with_type(mut self, class: crate::kg::ontology::Class) -> Self { + if !self.has_type(class) { + self.types.push(class.iri().to_owned()); + self.types.sort(); + } + self + } + + pub fn path(&self) -> Option<&str> { + self.metadata.as_ref()?.get("path")?.as_str() + } /// Create a new KG node. pub fn new( id: impl Into, - kind: impl Into, + class: crate::kg::ontology::Class, label: impl Into, source: impl Into, ) -> Self { Self { id: id.into(), - kind: kind.into(), + types: vec![class.iri().to_owned()], label: label.into(), summary: None, source: source.into(), @@ -519,7 +563,7 @@ pub struct KgEdge { /// Target node ID. pub to_id: String, /// Relationship type (e.g., `"MODIFIES"`, `"DEFINES"`, `"DEPENDS_ON"`). - pub kind: String, + pub kind: crate::kg::ontology::Predicate, /// Optional edge metadata. #[serde(skip_serializing_if = "Option::is_none")] pub metadata: Option, @@ -529,12 +573,12 @@ impl KgEdge { pub fn new( from_id: impl Into, to_id: impl Into, - kind: impl Into, + predicate: crate::kg::ontology::Predicate, ) -> Self { Self { from_id: from_id.into(), to_id: to_id.into(), - kind: kind.into(), + kind: predicate, metadata: None, } } @@ -948,21 +992,30 @@ mod tests { #[test] fn kg_node_new_and_builder() { - let node = KgNode::new("change:abc123", "change", "abc123", "change_graph") - .with_summary("Fix auth bug") - .with_metadata(serde_json::json!({"view": "main"})); + let node = KgNode::new( + "change:abc123", + crate::kg::ontology::Class::Change, + "abc123", + "change_graph", + ) + .with_summary("Fix auth bug") + .with_metadata(serde_json::json!({"view": "main"})); assert_eq!(node.id, "change:abc123"); - assert_eq!(node.kind, "change"); + assert!(node.has_type(crate::kg::ontology::Class::Change)); assert_eq!(node.summary, Some("Fix auth bug".to_string())); assert!(node.metadata.is_some()); } #[test] fn kg_edge_new() { - let edge = KgEdge::new("change:abc", "file:src/auth.rs", "MODIFIES"); + let edge = KgEdge::new( + "change:abc", + "file:src/auth.rs", + crate::kg::ontology::Predicate::ModifiesFile, + ); assert_eq!(edge.from_id, "change:abc"); assert_eq!(edge.to_id, "file:src/auth.rs"); - assert_eq!(edge.kind, "MODIFIES"); + assert_eq!(edge.kind, crate::kg::ontology::Predicate::ModifiesFile); assert!(edge.metadata.is_none()); } @@ -974,7 +1027,12 @@ mod tests { #[test] fn kg_node_json_roundtrip() { - let node = KgNode::new("file:src/main.rs", "file", "src/main.rs", "change_graph"); + let node = KgNode::new( + "file:src/main.rs", + crate::kg::ontology::Class::File, + "src/main.rs", + "change_graph", + ); let json = serde_json::to_string(&node).unwrap(); let decoded: KgNode = serde_json::from_str(&json).unwrap(); assert_eq!(node, decoded); @@ -982,7 +1040,7 @@ mod tests { #[test] fn kg_node_skips_none_fields() { - let node = KgNode::new("x", "y", "z", "w"); + let node = KgNode::new("x", crate::kg::ontology::Class::Memory, "z", "w"); let json = serde_json::to_string(&node).unwrap(); assert!(!json.contains("summary")); assert!(!json.contains("metadata")); diff --git a/atomic-core/tests/kg_iri.rs b/atomic-core/tests/kg_iri.rs new file mode 100644 index 00000000..90ec3685 --- /dev/null +++ b/atomic-core/tests/kg_iri.rs @@ -0,0 +1,112 @@ +use atomic_core::kg::{iri::*, project_id::ProjectId}; +use atomic_core::types::Hash; +use quickcheck_macros::quickcheck; + +#[quickcheck] +fn names_and_paths_round_trip(value: String) -> bool { + let project = ProjectId::generate(); + let pairs = [ + (mint_file(&project, &value), parse_file as fn(&str) -> _), + (mint_memory(&project, &value), parse_memory), + (mint_goal(&project, &value), parse_goal), + (mint_skill(&project, &value), parse_skill), + (mint_tool_result(&project, &value), parse_tool_result), + (mint_concept(&project, &value), parse_concept), + (mint_view(&project, &value), parse_view), + (mint_session(&project, &value), parse_session), + (mint_named_identity(&project, &value), parse_named_identity), + ]; + pairs + .into_iter() + .all(|(iri, parse)| parse(&iri).unwrap() == (project, value.clone())) + && parse_todo(&mint_todo(&project, &value, &value)).unwrap() + == (project, value.clone(), value) +} + +#[test] +fn all_global_ids_intents_findings_and_identity_forms_round_trip() { + let project = ProjectId::generate(); + let hash = Hash::of(b"global object"); + assert_eq!(parse_project(&mint_project(&project)).unwrap(), project); + for (iri, parse) in [ + (mint_change(&hash), parse_change as fn(&str) -> _), + (mint_provenance(&hash), parse_provenance), + (mint_attestation(&hash), parse_attestation), + (mint_project_identity(&hash), parse_project_identity), + ] { + assert_eq!(parse(&iri).unwrap(), hash); + } + let uid = ulid::Ulid::new().to_string(); + assert_eq!( + parse_intent(&mint_intent(&project, &uid).unwrap()).unwrap(), + (project, uid.clone()) + ); + for kind in [ + ChildKind::AcceptanceCriterion, + ChildKind::Task, + ChildKind::Scope, + ChildKind::Constraint, + ] { + assert_eq!( + parse_child(&mint_child(&project, &uid, kind, "child:#").unwrap()).unwrap(), + (project, uid.clone(), kind, "CHILD:#".into()) + ); + } + let subject = mint_change(&hash); + assert_eq!( + parse_finding(&mint_finding(&project, "MissingPlanId", &subject)).unwrap(), + ( + project, + "MissingPlanId".into(), + Hash::of(subject.as_bytes()) + ) + ); + let did = format!("did:atomic:{}", project.to_base32()); + assert_eq!( + parse_identity(&mint_identity(&project, &did)).unwrap(), + IdentityIri::Did(did) + ); + assert_eq!( + parse_identity(&mint_identity(&project, "Alice Smith")).unwrap(), + IdentityIri::Named(project, "Alice Smith".into()) + ); + assert!(parse_identity("did:atomic:not-a-fingerprint").is_err()); +} + +#[test] +fn strict_identifiers_and_scoped_resources() { + let project = ProjectId::generate(); + assert_eq!( + ProjectId::from_base32(&project.to_base32()).unwrap(), + project + ); + assert!(ProjectId::from_base32(&project.to_base32().to_lowercase()).is_err()); + let hash = Hash::of(b"change"); + let change = mint_change(&hash); + assert_eq!(parse_change(&change).unwrap(), hash); + assert!(parse_change(&change.to_lowercase()).is_err()); + assert!(parse_change(&(change.clone() + "=")).is_err()); + assert!(parse_change(&change[..change.len() - 1]).is_err()); + let mut nonzero = mint_change(&Hash::of(b"trailing bits")); + let last = nonzero.pop().unwrap(); + let alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZ234567"; + nonzero.push(alphabet.as_bytes()[alphabet.find(last).unwrap() + 1] as char); + assert!(parse_change(&nonzero).is_err()); + let value = "a b/#:%/日本語"; + let file = mint_file(&project, value); + assert!(file.ends_with("a%20b/%23%3A%25/%E6%97%A5%E6%9C%AC%E8%AA%9E")); + assert!(parse_file(&file.replace("%3A", "%3a")).is_err()); + assert!(parse_file(&mint_file(&project, "A").replace(":file:A", ":file:%41")).is_err()); + assert_ne!( + mint_todo(&project, "one", "t1"), + mint_todo(&project, "two", "t1") + ); + assert_eq!( + parse_turn(&mint_turn(&project, "session:id", 42)).unwrap(), + (project, "session:id".into(), 42) + ); + let uid = ulid::Ulid::new().to_string(); + let child = mint_child(&project, &uid, ChildKind::Task, "task-a").unwrap(); + assert_eq!(parse_child(&child).unwrap().3, "TASK-A"); + assert!(mint_intent(&project, &uid.to_lowercase()).is_err()); +} diff --git a/atomic-core/tests/kg_ontology.rs b/atomic-core/tests/kg_ontology.rs new file mode 100644 index 00000000..8b377575 --- /dev/null +++ b/atomic-core/tests/kg_ontology.rs @@ -0,0 +1,197 @@ +use atomic_core::kg::{ + legacy::{Replacement, Target, LEGACY_TERMS}, + ontology::*, +}; +use oxrdf::{Subject, Term, Triple}; +use oxttl::TurtleParser; +use std::collections::BTreeSet; + +fn graph() -> Vec { + TurtleParser::new() + .for_reader(TTL.as_bytes()) + .collect::>() + .unwrap() +} + +fn objects<'a>(graph: &'a [Triple], subject: &str, predicate: &str) -> BTreeSet<&'a str> { + graph + .iter() + .filter_map(|triple| match (&triple.subject, &triple.object) { + (Subject::NamedNode(s), Term::NamedNode(o)) + if s.as_str() == subject && triple.predicate.as_str() == predicate => + { + Some(o.as_str()) + } + _ => None, + }) + .collect() +} + +#[test] +fn registry_matches_the_complete_versioned_ontology() { + let graph = graph(); + let rdfs = "http://www.w3.org/2000/01/rdf-schema#"; + let owl = "http://www.w3.org/2002/07/owl#"; + let rdf_type = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type"; + for class in Class::ALL { + assert!(objects(&graph, class.iri(), rdf_type).contains(format!("{owl}Class").as_str())); + assert_eq!( + objects(&graph, class.iri(), &format!("{rdfs}subClassOf")), + class.parents().iter().map(|p| p.iri()).collect() + ); + } + for predicate in Predicate::ALL { + if !predicate.is_external() { + let property_type = if matches!(predicate.range(), Some(Range::Datatype(_))) { + "DatatypeProperty" + } else { + "ObjectProperty" + }; + let declarations = objects(&graph, predicate.iri(), rdf_type); + assert!( + declarations.contains(format!("{owl}{property_type}").as_str()), + "missing property declaration: {predicate:?}" + ); + assert_eq!( + declarations.contains(format!("{owl}TransitiveProperty").as_str()), + predicate.is_transitive() + ); + } + assert_eq!( + objects(&graph, predicate.iri(), &format!("{rdfs}domain")), + predicate.domain().map(ClassRef::iri).into_iter().collect(), + "domain: {predicate:?}" + ); + assert_eq!( + objects(&graph, predicate.iri(), &format!("{rdfs}range")), + predicate.range().map(Range::iri).into_iter().collect(), + "range: {predicate:?}" + ); + assert_eq!( + objects(&graph, predicate.iri(), &format!("{owl}inverseOf")), + predicate + .inverse() + .map(Predicate::iri) + .into_iter() + .collect() + ); + assert_eq!( + objects(&graph, predicate.iri(), &format!("{rdfs}subPropertyOf")), + predicate + .super_predicate() + .map(Predicate::iri) + .into_iter() + .collect() + ); + let mut lists = graph.iter().filter(|t| { + t.subject.to_string() == format!("<{}>", predicate.iri()) + && t.predicate.as_str() == format!("{owl}propertyChainAxiom") + }); + let mut chain = Vec::new(); + if let Some(triple) = lists.next() { + let mut cursor = triple.object.clone(); + while cursor.to_string() != "" { + let row = |property: &str| { + graph + .iter() + .find(|t| { + t.subject.to_string() == cursor.to_string() + && t.predicate.as_str() == property + }) + .unwrap() + .object + .clone() + }; + chain.push(row("http://www.w3.org/1999/02/22-rdf-syntax-ns#first").to_string()); + cursor = row("http://www.w3.org/1999/02/22-rdf-syntax-ns#rest"); + } + } + assert!(lists.next().is_none()); + assert_eq!( + chain, + predicate + .property_chain() + .iter() + .map(|p| format!("<{}>", p.iri())) + .collect::>() + ); + } + for triple in &graph { + if let Subject::NamedNode(subject) = &triple.subject { + if subject.as_str().starts_with(NAMESPACE) + && triple.predicate.as_str().ends_with("#type") + { + match triple.object.to_string().as_str() { + "" => { + assert!(Class::from_iri(subject.as_str()).is_some()) + } + "" + | "" + | "" => { + assert!(Predicate::from_iri(subject.as_str()).is_some()) + } + _ => {} + } + } + } + } +} + +#[test] +fn ontology_uses_only_the_declared_rl_constructs() { + let allowed = [ + "rdf:first", + "rdf:rest", + "rdfs:subClassOf", + "rdfs:subPropertyOf", + "rdfs:domain", + "rdfs:range", + "owl:inverseOf", + "owl:propertyChainAxiom", + "rdfs:isDefinedBy", + "owl:versionIRI", + "rdfs:label", + "rdfs:comment", + ]; + for triple in graph() { + assert!( + triple.predicate.as_str() == "http://www.w3.org/1999/02/22-rdf-syntax-ns#type" + || allowed.contains(&compact(triple.predicate.as_str()).as_str()), + "unexpected axiom: {triple}" + ); + } +} + +#[test] +fn legacy_inventory_is_unique_and_all_targets_exist() { + let mut old = BTreeSet::new(); + let target = |target: Target| match target { + Target::Class(c) => assert!(Class::ALL.contains(&c)), + Target::Predicate(p) => assert!(Predicate::ALL.contains(&p)), + Target::Removed { reason } => assert!(!reason.is_empty()), + }; + for (term, replacement) in LEGACY_TERMS { + assert!(old.insert(*term), "duplicate legacy term: {term}"); + match replacement { + Replacement::One(t) => target(*t), + Replacement::ClassWithRefinement { + base, + refinements, + rule, + } => { + target(Target::Class(*base)); + assert!(!rule.is_empty()); + for c in *refinements { + target(Target::Class(*c)); + } + } + Replacement::SplitByEndpoint(rules) => { + for rule in *rules { + target(rule.target); + } + } + } + } + // 81 former constants, plus 20 bare node kinds. + assert_eq!(old.len(), 101); +} diff --git a/atomic-core/tests/kg_vocabulary_lint.rs b/atomic-core/tests/kg_vocabulary_lint.rs new file mode 100644 index 00000000..73b50d33 --- /dev/null +++ b/atomic-core/tests/kg_vocabulary_lint.rs @@ -0,0 +1,76 @@ +//! A token-level workspace lint: comments and substrings are not vocabulary +//! uses. Retired exact string literals belong only in the migration inventory. +use atomic_core::kg::legacy::LEGACY_TERMS; +use proc_macro2::{TokenStream, TokenTree}; +use std::{collections::BTreeSet, path::Path}; + +fn literals( + tokens: TokenStream, + forbidden: &BTreeSet<&str>, + path: &Path, + failures: &mut Vec, +) { + for token in tokens { + match token { + TokenTree::Group(group) => literals(group.stream(), forbidden, path, failures), + TokenTree::Literal(literal) => { + if let Ok(value) = syn::parse_str::(&literal.to_string()) { + if forbidden.contains(value.value().as_str()) { + failures.push(format!("{}: {:?}", path.display(), value.value())); + } + } + } + _ => {} + } + } +} + +fn scan(path: &Path, forbidden: &BTreeSet<&str>, failures: &mut Vec) { + if path.is_dir() { + for entry in std::fs::read_dir(path).unwrap() { + scan(&entry.unwrap().path(), forbidden, failures); + } + } else if path.extension().is_some_and(|extension| extension == "rs") + && !path.ends_with("src/kg/legacy.rs") + { + let source = std::fs::read_to_string(path).unwrap(); + let tokens = source + .parse() + .unwrap_or_else(|error| panic!("{}: {error}", path.display())); + literals(tokens, forbidden, path, failures); + } +} + +#[test] +fn workspace_has_no_retired_kg_vocabulary_literals() { + let forbidden = LEGACY_TERMS + .iter() + .map(|(term, _)| *term) + .filter(|term| { + (term.contains(':') || term.chars().all(|c| c.is_ascii_uppercase() || c == '_')) + && ![ + "prov:wasDerivedFrom", + "prov:used", + "prov:wasInformedBy", + "prov:generated", + ] + .contains(term) + && !term.starts_with("prefix::") + }) + .collect(); + let root = Path::new(env!("CARGO_MANIFEST_DIR")).parent().unwrap(); + let mut failures = Vec::new(); + for entry in std::fs::read_dir(root).unwrap() { + let path = entry.unwrap().path(); + let name = path.file_name().unwrap().to_string_lossy(); + if path.is_dir() && (name.starts_with("atomic-") || name == "libatomic") { + scan(&path.join("src"), &forbidden, &mut failures); + scan(&path.join("tests"), &forbidden, &mut failures); + } + } + assert!( + failures.is_empty(), + "retired vocabulary:\n{}", + failures.join("\n") + ); +} diff --git a/atomic-repository/src/ai/mod.rs b/atomic-repository/src/ai/mod.rs index e342e5c0..2c71d3f3 100644 --- a/atomic-repository/src/ai/mod.rs +++ b/atomic-repository/src/ai/mod.rs @@ -445,18 +445,18 @@ pub fn build_context_string(nodes: &[KgNode], edges: &[KgEdge], query: &str) -> let mut groups: std::collections::BTreeMap<&str, Vec<&KgNode>> = std::collections::BTreeMap::new(); for node in nodes { - groups.entry(node.kind.as_str()).or_default().push(node); + groups.entry(node.primary_type()).or_default().push(node); } for (kind, group_nodes) in &groups { - buf.push_str(&format!("{}:\n", capitalize(kind))); + buf.push_str(&format!("{}:\n", atomic_core::kg::ontology::compact(kind))); for node in group_nodes { buf.push_str(&format!("- {}", node.label)); if let Some(ref summary) = node.summary { buf.push_str(&format!(": {}", summary)); } // Render metadata for change nodes (timestamp, author, sequence) - if node.kind == "change" { + if node.has_type(atomic_core::kg::ontology::Class::Change) { if let Some(ref meta) = node.metadata { let mut details = Vec::new(); if let Some(ts) = meta.get("timestamp").and_then(|v| v.as_str()) { @@ -471,7 +471,7 @@ pub fn build_context_string(nodes: &[KgNode], edges: &[KgEdge], query: &str) -> } } // Render metadata for entity nodes (kind, file, line range) - if node.kind == "entity" { + if node.has_type(atomic_core::kg::ontology::Class::File) { if let Some(ref meta) = node.metadata { let mut details = Vec::new(); if let Some(k) = meta.get("kind").and_then(|v| v.as_str()) { @@ -505,7 +505,9 @@ pub fn build_context_string(nodes: &[KgNode], edges: &[KgEdge], query: &str) -> for edge in edges { buf.push_str(&format!( "- {} -[{}]-> {}\n", - edge.from_id, edge.kind, edge.to_id + edge.from_id, + edge.kind.curie(), + edge.to_id )); if buf.len() >= MAX_CHARS { break; @@ -517,6 +519,7 @@ pub fn build_context_string(nodes: &[KgNode], edges: &[KgEdge], query: &str) -> buf } +#[cfg(test)] fn capitalize(s: &str) -> String { let mut c = s.chars(); match c.next() { @@ -601,15 +604,30 @@ mod tests { #[test] fn test_build_context_string() { let nodes = vec![ - KgNode::new("change:abc", "change", "abc123", "graph").with_summary("Fix auth bug"), - KgNode::new("file:src/auth.rs", "file", "src/auth.rs", "graph"), + KgNode::new( + "change:abc", + atomic_core::kg::ontology::Class::Change, + "abc123", + "graph", + ) + .with_summary("Fix auth bug"), + KgNode::new( + "file:src/auth.rs", + atomic_core::kg::ontology::Class::File, + "src/auth.rs", + "graph", + ), ]; - let edges = vec![KgEdge::new("change:abc", "file:src/auth.rs", "MODIFIES")]; + let edges = vec![KgEdge::new( + "change:abc", + "file:src/auth.rs", + atomic_core::kg::ontology::Predicate::ModifiesFile, + )]; let context = build_context_string(&nodes, &edges, "who fixed auth?"); assert!(context.contains("abc123")); assert!(context.contains("Fix auth bug")); - assert!(context.contains("MODIFIES")); + assert!(context.contains("atomic:modifiesFile")); assert!(context.contains("who fixed auth?")); } @@ -626,7 +644,7 @@ mod tests { .map(|i| { KgNode::new( format!("entity:{}", i), - "entity", + atomic_core::kg::ontology::Class::File, format!("entity-{}", i), "graph", ) @@ -640,14 +658,33 @@ mod tests { #[test] fn test_build_context_string_groups_by_kind() { let nodes = vec![ - KgNode::new("file:a.rs", "file", "a.rs", "graph"), - KgNode::new("change:x", "change", "x", "graph"), - KgNode::new("file:b.rs", "file", "b.rs", "graph"), + KgNode::new( + "file:a.rs", + atomic_core::kg::ontology::Class::File, + "a.rs", + "graph", + ), + KgNode::new( + "change:x", + atomic_core::kg::ontology::Class::Change, + "x", + "graph", + ), + KgNode::new( + "file:b.rs", + atomic_core::kg::ontology::Class::File, + "b.rs", + "graph", + ), ]; let context = build_context_string(&nodes, &[], "test"); // BTreeMap orders keys: "change" < "file" - let change_pos = context.find("Change:").expect("should contain Change:"); - let file_pos = context.find("File:").expect("should contain File:"); + let change_pos = context + .find("atomic:Change:") + .expect("should contain atomic:Change:"); + let file_pos = context + .find("atomic:File:") + .expect("should contain atomic:File:"); assert!( change_pos < file_pos, "Change group should appear before File group (BTreeMap order)" diff --git a/atomic-repository/src/ai/tools.rs b/atomic-repository/src/ai/tools.rs index 2c766957..9af5b47c 100644 --- a/atomic-repository/src/ai/tools.rs +++ b/atomic-repository/src/ai/tools.rs @@ -734,7 +734,7 @@ impl<'a> ToolExecutor for RepoToolExecutor<'a> { "properties": { "node_id": { "type": "string", - "description": "Node ID (e.g. \"file:src/main.rs\", \"change:abc123\")" + "description": "Canonical resource IRI copied from a search result; files are project-qualified and changes use their full content hash" }, "depth": { "type": "integer", diff --git a/atomic-repository/src/query_plan.rs b/atomic-repository/src/query_plan.rs index 4cc4d350..430ef8cb 100644 --- a/atomic-repository/src/query_plan.rs +++ b/atomic-repository/src/query_plan.rs @@ -14,20 +14,27 @@ use serde::{Deserialize, Serialize}; use crate::repository::Repository; use crate::RepositoryError; +use atomic_core::kg::{ + iri, + ontology::{Class, Predicate}, + Direction, +}; use atomic_core::pristine::vault::{KgEdge, KgNode}; // ── Query Plan Schema ─────────────────────────────────────────── /// A query plan is a sequence of steps that the executor runs in order. #[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] pub struct QueryPlan { + pub schema_version: u32, /// Ordered list of steps to execute. pub steps: Vec, } /// A single step in the query plan. #[derive(Debug, Clone, Serialize, Deserialize)] -#[serde(tag = "type", rename_all = "snake_case")] +#[serde(tag = "type", rename_all = "snake_case", deny_unknown_fields)] pub enum QueryStep { /// Full-text keyword search over KG nodes. KgSearch { @@ -48,6 +55,10 @@ pub enum QueryStep { /// Traversal depth (1 or 2). #[serde(default = "default_depth")] depth: u8, + #[serde(default)] + predicate: Option, + #[serde(default)] + direction: Direction, /// Variable name to bind results to. #[serde(default)] bind: Option, @@ -141,6 +152,11 @@ pub struct StepStat { /// Execute a query plan against a repository. pub fn execute_plan(repo: &Repository, plan: &QueryPlan) -> Result { + if plan.schema_version != 1 { + return Err(RepositoryError::Serialization( + "unsupported query plan schema version".into(), + )); + } let overall_start = std::time::Instant::now(); let mut bindings: HashMap> = HashMap::new(); @@ -172,6 +188,8 @@ pub fn execute_plan(repo: &Repository, plan: &QueryPlan) -> Result { // Resolve $variable references @@ -181,15 +199,9 @@ pub fn execute_plan(repo: &Repository, plan: &QueryPlan) -> Result { - step_nodes.extend(sg.nodes); - step_edges.extend(sg.edges); - } - Err(e) => { - log::warn!("KG neighbors failed for {}: {}", id, e); - } - } + let sg = repo.workflow_neighbors(id, *depth, *predicate, *direction)?; + step_nodes.extend(sg.nodes); + step_edges.extend(sg.edges); } let count = step_nodes.len(); @@ -222,8 +234,11 @@ pub fn execute_plan(repo: &Repository, plan: &QueryPlan) -> Result Result Result = input_nodes .into_iter() .filter(|node| { + if field == "type" { + return equals + .as_ref() + .is_none_or(|value| node.types.contains(value)) + && not_equals + .as_ref() + .is_none_or(|value| !node.types.contains(value)); + } let value = get_node_field(node, field); if let Some(ref eq) = equals { if value.as_deref() != Some(eq.as_str()) { @@ -360,7 +386,7 @@ pub fn execute_plan(repo: &Repository, plan: &QueryPlan) -> Result>) -> Vec Option { match field { - "kind" => Some(node.kind.clone()), + "type" => Some(node.primary_type().to_owned()), "label" => Some(node.label.clone()), "source" => Some(node.source.clone()), "summary" => node.summary.clone(), @@ -450,8 +476,14 @@ fn get_node_field(node: &KgNode, field: &str) -> Option { /// Parse a query plan from JSON. pub fn parse_plan(json: &str) -> Result { - serde_json::from_str(json) - .map_err(|e| RepositoryError::Serialization(format!("Invalid query plan: {}", e))) + let plan: QueryPlan = serde_json::from_str(json) + .map_err(|e| RepositoryError::Serialization(format!("Invalid query plan: {}", e)))?; + if plan.schema_version != 1 { + return Err(RepositoryError::Serialization( + "unsupported query plan schema version".into(), + )); + } + Ok(plan) } #[cfg(test)] @@ -461,6 +493,7 @@ mod tests { #[test] fn test_parse_simple_plan() { let json = r#"{ + "schema_version": 1, "steps": [ {"type": "kg_search", "query": "authentication", "limit": 5, "bind": "auth_nodes"} ] @@ -480,9 +513,10 @@ mod tests { #[test] fn test_parse_multi_step_plan() { let json = r#"{ + "schema_version": 1, "steps": [ {"type": "kg_search", "query": "deploy blocked", "bind": "blockers"}, - {"type": "filter", "input": "blockers", "field": "kind", "equals": "intent", "bind": "blocking_intents"}, + {"type": "filter", "input": "blockers", "field": "type", "equals": "https://atomic.dev/ns/workflow#WorkIntent", "bind": "blocking_intents"}, {"type": "kg_neighbors", "node_id": "$blocking_intents", "depth": 1, "bind": "context"}, {"type": "read_content", "sources": ["$blocking_intents"], "max_chars": 1000} ] @@ -494,6 +528,7 @@ mod tests { #[test] fn test_parse_vector_search() { let json = r#"{ + "schema_version": 1, "steps": [ {"type": "vector_search", "query": "caching performance", "top_k": 3} ] @@ -505,6 +540,7 @@ mod tests { #[test] fn test_parse_defaults() { let json = r#"{ + "schema_version": 1, "steps": [ {"type": "kg_search", "query": "test"}, {"type": "kg_neighbors", "node_id": "intent:X"}, @@ -555,13 +591,13 @@ mod tests { #[test] fn test_parse_empty_steps_ok() { // An empty steps array is valid JSON for our schema (no minimum). - let plan = parse_plan(r#"{"steps": []}"#).unwrap(); + let plan = parse_plan(r#"{"schema_version":1,"steps": []}"#).unwrap(); assert!(plan.steps.is_empty()); } #[test] fn test_parse_unknown_step_type_errors() { - let json = r#"{"steps": [{"type": "unknown_step", "query": "x"}]}"#; + let json = r#"{"schema_version":1,"steps": [{"type": "unknown_step", "query": "x"}]}"#; assert!(parse_plan(json).is_err()); } @@ -578,8 +614,8 @@ mod tests { bindings.insert( "results".to_string(), vec![ - KgNode::new("intent:PIMO-1", "intent", "PIMO-1", "test"), - KgNode::new("intent:PIMO-2", "intent", "PIMO-2", "test"), + KgNode::new("intent:PIMO-1", Class::WorkIntent, "PIMO-1", "test"), + KgNode::new("intent:PIMO-2", Class::WorkIntent, "PIMO-2", "test"), ], ); let ids = resolve_node_ids("$results", &bindings); @@ -603,7 +639,7 @@ mod tests { #[test] fn test_resolve_paths_variable_with_metadata() { let mut bindings = HashMap::new(); - let node = KgNode::new("embedding:x:0", "embedding", "some-label", "vector_search") + let node = KgNode::new("embedding:x:0", Class::File, "some-label", "vector_search") .with_metadata(serde_json::json!({"path": "memory/auth.md"})); bindings.insert("results".to_string(), vec![node]); @@ -614,7 +650,7 @@ mod tests { #[test] fn test_resolve_paths_variable_fallback_to_label() { let mut bindings = HashMap::new(); - let node = KgNode::new("x", "memory", "memory/arch.md", "vault"); + let node = KgNode::new("x", Class::Memory, "memory/arch.md", "vault"); bindings.insert("results".to_string(), vec![node]); let paths = resolve_paths("$results", &bindings); @@ -624,7 +660,7 @@ mod tests { #[test] fn test_resolve_paths_variable_skips_non_path_labels() { let mut bindings = HashMap::new(); - let node = KgNode::new("x", "intent", "PIMO-1", "vault"); + let node = KgNode::new("x", Class::WorkIntent, "PIMO-1", "vault"); bindings.insert("results".to_string(), vec![node]); let paths = resolve_paths("$results", &bindings); @@ -640,11 +676,14 @@ mod tests { #[test] fn test_get_node_field() { - let node = KgNode::new("x", "intent", "PIMO-1", "vault") + let node = KgNode::new("x", Class::WorkIntent, "PIMO-1", "vault") .with_summary("Fix auth") .with_metadata(serde_json::json!({"status": "in-progress", "priority": "high"})); - assert_eq!(get_node_field(&node, "kind"), Some("intent".to_string())); + assert_eq!( + get_node_field(&node, "type"), + Some(Class::WorkIntent.iri().to_string()) + ); assert_eq!(get_node_field(&node, "label"), Some("PIMO-1".to_string())); assert_eq!( get_node_field(&node, "summary"), @@ -662,11 +701,12 @@ mod tests { #[test] fn test_get_node_field_metadata_types() { - let node = KgNode::new("x", "test", "test", "test").with_metadata(serde_json::json!({ - "count": 42, - "active": true, - "tags": ["a", "b"], - })); + let node = + KgNode::new("x", Class::Memory, "test", "test").with_metadata(serde_json::json!({ + "count": 42, + "active": true, + "tags": ["a", "b"], + })); assert_eq!(get_node_field(&node, "count"), Some("42".to_string())); assert_eq!(get_node_field(&node, "active"), Some("true".to_string())); @@ -676,30 +716,30 @@ mod tests { #[test] fn test_get_node_field_no_metadata() { - let node = KgNode::new("x", "test", "test", "test"); + let node = KgNode::new("x", Class::Memory, "test", "test"); assert_eq!(get_node_field(&node, "unknown"), None); } #[test] fn test_get_node_field_no_summary() { - let node = KgNode::new("x", "test", "test", "test"); + let node = KgNode::new("x", Class::Memory, "test", "test"); assert_eq!(get_node_field(&node, "summary"), None); } #[test] fn test_filter_nodes() { let nodes = [ - KgNode::new("a", "intent", "A", "test") + KgNode::new("a", Class::WorkIntent, "A", "test") .with_metadata(serde_json::json!({"status": "done"})), - KgNode::new("b", "intent", "B", "test") + KgNode::new("b", Class::WorkIntent, "B", "test") .with_metadata(serde_json::json!({"status": "in-progress"})), - KgNode::new("c", "change", "C", "test"), + KgNode::new("c", Class::Change, "C", "test"), ]; - // Filter: kind == "intent" + // Filter on an asserted class. let filtered: Vec<&KgNode> = nodes .iter() - .filter(|n| get_node_field(n, "kind").as_deref() == Some("intent")) + .filter(|n| n.has_type(Class::WorkIntent)) .collect(); assert_eq!(filtered.len(), 2); @@ -714,8 +754,8 @@ mod tests { #[test] fn test_plan_result_serialization() { let result = PlanResult { - nodes: vec![KgNode::new("a", "intent", "A", "test")], - edges: vec![KgEdge::new("a", "b", "DEPENDS_ON")], + nodes: vec![KgNode::new("a", Class::WorkIntent, "A", "test")], + edges: vec![KgEdge::new("a", "b", Predicate::DependsOnIntent)], content: HashMap::new(), step_stats: vec![StepStat { step_type: "kg_search".to_string(), @@ -726,7 +766,7 @@ mod tests { }; let json = serde_json::to_string(&result).unwrap(); assert!(json.contains("kg_search")); - assert!(json.contains("intent")); + assert!(json.contains(Class::WorkIntent.iri())); // Empty content map should be skipped assert!(!json.contains("content")); } @@ -750,6 +790,7 @@ mod tests { #[test] fn test_query_plan_json_roundtrip() { let plan = QueryPlan { + schema_version: 1, steps: vec![ QueryStep::KgSearch { query: "auth".to_string(), @@ -758,14 +799,16 @@ mod tests { }, QueryStep::Filter { input: "results".to_string(), - field: "kind".to_string(), - equals: Some("intent".to_string()), + field: "type".to_string(), + equals: Some(Class::WorkIntent.iri().to_string()), not_equals: None, bind: Some("intents".to_string()), }, QueryStep::KgNeighbors { node_id: "$intents".to_string(), depth: 2, + predicate: Some(Predicate::DependsOnIntent), + direction: Direction::Outgoing, bind: None, }, ], @@ -796,6 +839,7 @@ mod tests { // Execute a simple search plan let plan = parse_plan( r#"{ + "schema_version": 1, "steps": [ {"type": "kg_search", "query": "auth", "limit": 5, "bind": "results"} ] @@ -818,7 +862,7 @@ mod tests { let repo = Repository::init(dir.path()).unwrap(); repo.init_vault().unwrap(); - let plan = parse_plan(r#"{"steps": []}"#).unwrap(); + let plan = parse_plan(r#"{"schema_version":1,"steps": []}"#).unwrap(); let result = execute_plan(&repo, &plan).unwrap(); assert!(result.nodes.is_empty()); assert!(result.edges.is_empty()); @@ -844,6 +888,7 @@ mod tests { let plan = parse_plan( r#"{ + "schema_version": 1, "steps": [ {"type": "read_content", "sources": ["memory/arch.md"], "max_chars": 500} ] @@ -875,6 +920,7 @@ mod tests { let plan = parse_plan( r#"{ + "schema_version": 1, "steps": [ {"type": "read_content", "sources": ["memory/big.md"], "max_chars": 100} ] @@ -897,6 +943,7 @@ mod tests { let plan = parse_plan( r#"{ + "schema_version": 1, "steps": [ {"type": "read_content", "sources": ["memory/nonexistent.md"]} ] @@ -939,9 +986,10 @@ mod tests { // Search then filter to just memory nodes let plan = parse_plan( r#"{ + "schema_version": 1, "steps": [ {"type": "kg_search", "query": "test memory goal", "limit": 10, "bind": "all"}, - {"type": "filter", "input": "all", "field": "kind", "equals": "memory", "bind": "only_memory"} + {"type": "filter", "input": "all", "field": "type", "equals": "https://atomic.dev/ns/workflow#Memory", "bind": "only_memory"} ] }"#, ) @@ -971,6 +1019,7 @@ mod tests { // Two search steps that will find the same nodes let plan = parse_plan( r#"{ + "schema_version": 1, "steps": [ {"type": "kg_search", "query": "dup", "limit": 10}, {"type": "kg_search", "query": "duplicate", "limit": 10} @@ -1008,6 +1057,7 @@ mod tests { // Search then explore neighbors of results let plan = parse_plan( r#"{ + "schema_version": 1, "steps": [ {"type": "kg_search", "query": "neighbor", "limit": 5, "bind": "found"}, {"type": "kg_neighbors", "node_id": "$found", "depth": 1, "bind": "neighbors"} diff --git a/atomic-repository/src/repository/changes.rs b/atomic-repository/src/repository/changes.rs index d0dcd12a..c597d4b6 100644 --- a/atomic-repository/src/repository/changes.rs +++ b/atomic-repository/src/repository/changes.rs @@ -140,7 +140,16 @@ impl Repository { computed.to_base32() ))); } - self.save_change_bytes(hash, bytes, &change) + let stored = self.save_change_bytes(hash, bytes, &change)?; + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; + if !change.kind().is_snapshot() { + let mut txn = self.pristine.write_txn()?; + if txn.kg_writes_enabled()? { + self.project_change_kg(&mut txn, hash, &change)?; + } + txn.commit()?; + } + Ok(stored) } /// Save a change using pre-serialized ATOM bytes (hash-stable). @@ -345,6 +354,7 @@ impl Repository { &self, attestation: &atomic_core::change::Attestation, ) -> Result { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; use atomic_core::pristine::MutTxnT; // Save to disk @@ -386,6 +396,7 @@ impl Repository { txn.commit() .map_err(|e| RepositoryError::Database(e.to_string()))?; + self.refresh_session_kg()?; Ok(hash) } @@ -472,6 +483,7 @@ impl Repository { &self, manifest: &atomic_core::change::session::SessionManifest, ) -> Result { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; let mut txn = self .pristine .write_txn() @@ -491,6 +503,7 @@ impl Repository { /// files are reported in the result rather than aborting the rebuild. /// Returns `(indexed_count, skipped_existing, corrupt_count)`. pub fn rebuild_session_index(&self) -> Result<(usize, usize, usize), RepositoryError> { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; let mut indexed = 0usize; let mut skipped = 0usize; let mut corrupt = 0usize; @@ -665,6 +678,7 @@ impl Repository { } } + self.refresh_session_kg()?; Ok((indexed, skipped, corrupt)) } @@ -682,6 +696,7 @@ impl Repository { parent_view: Option, ended_at: Option, ) -> Result<(), RepositoryError> { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; let json_path = self .dot_dir .join("sessions") @@ -710,6 +725,7 @@ impl Repository { parent_view: Option, incomplete: &atomic_core::change::session::IncompleteSession, ) -> Result { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; let json_path = self .dot_dir .join("sessions") @@ -743,6 +759,7 @@ impl Repository { boundary_end: &atomic_core::change::session::TurnBoundary, outcome: &atomic_core::change::session::ManagedTurnOutcome, ) -> Result, RepositoryError> { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; let mut txn = self .pristine .write_txn() @@ -758,6 +775,9 @@ impl Repository { .map_err(|e| RepositoryError::Database(e.to_string()))?; txn.commit() .map_err(|e| RepositoryError::Database(e.to_string()))?; + if attached { + self.refresh_session_kg()?; + } Ok(Some(attached)) } @@ -774,6 +794,7 @@ impl Repository { fork_turn: u32, child_session_id: &str, ) -> Result<(Hash, Hash), RepositoryError> { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; // Resolve the parent ledger and manifest. let (parent_record, parent_turns) = self.get_session_ledger(parent_session_id)?.ok_or_else(|| { @@ -1208,6 +1229,7 @@ impl Repository { graph: &atomic_core::change::ProvenanceGraph, mut turn: atomic_core::change::session::SessionTurn, ) -> Result { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; use atomic_core::pristine::MutTxnT; let existing_turns = self @@ -1270,6 +1292,7 @@ impl Repository { .map_err(|error| RepositoryError::Database(error.to_string()))?; txn.commit() .map_err(|error| RepositoryError::Database(error.to_string()))?; + self.refresh_session_kg()?; Ok(publication) } @@ -1286,6 +1309,7 @@ impl Repository { &self, graph: &atomic_core::change::ProvenanceGraph, ) -> Result { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; use atomic_core::pristine::MutTxnT; // Save to disk @@ -1357,6 +1381,7 @@ impl Repository { ); } + self.refresh_session_kg()?; Ok(hash) } diff --git a/atomic-repository/src/repository/history.rs b/atomic-repository/src/repository/history.rs index 3efe10a9..042615b3 100644 --- a/atomic-repository/src/repository/history.rs +++ b/atomic-repository/src/repository/history.rs @@ -320,6 +320,7 @@ impl Repository { return crate::unrecord::preview_unrecord(&txn, &view, &[*hash], &options) .map_err(|error| RepositoryError::Unrecord(error.to_string())); } + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; let working_copy = self.require_working_copy_id()?; let operation_lock = self.try_lock_operation(working_copy)?; if let OperationHeadState::Diverged(heads) = @@ -459,6 +460,10 @@ impl Repository { .map_err(|e| RepositoryError::Database(e.to_string()))?; let affected_tree_paths = self.realign_tree_projection_in_txn(&mut txn, &view_name)?; + if txn.kg_writes_enabled()? { + self.project_view_kg(&mut txn)?; + } + // Commit the transaction txn.commit() .map_err(|e| RepositoryError::Database(e.to_string()))?; @@ -666,6 +671,7 @@ impl Repository { hash: &Hash, at_sequence: Option, ) -> Result<(Merkle, u64), RepositoryError> { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; let change = self.load_change(hash)?; // Get write transaction @@ -701,6 +707,9 @@ impl Repository { txn.update_view(&view) .map_err(|e| RepositoryError::Database(e.to_string()))?; let affected_tree_paths = self.realign_tree_projection_in_txn(&mut txn, view_name)?; + if txn.kg_writes_enabled()? { + self.project_view_kg(&mut txn)?; + } // Commit the transaction txn.commit() diff --git a/atomic-repository/src/repository/insert.rs b/atomic-repository/src/repository/insert.rs index 5ebf5d72..30dedf24 100644 --- a/atomic-repository/src/repository/insert.rs +++ b/atomic-repository/src/repository/insert.rs @@ -1758,6 +1758,7 @@ impl Repository { hash: &Hash, options: InsertOptions, ) -> Result { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; let trace_insert = std::env::var_os("ATOMIC_TRACE_INSERT").is_some(); let t0 = std::time::Instant::now(); @@ -2065,6 +2066,9 @@ impl Repository { // Commit the transaction log::debug!("insert_change: committing transaction..."); + if txn.kg_writes_enabled()? { + self.project_view_kg(&mut txn)?; + } let commit_start = std::time::Instant::now(); txn.commit() .map_err(|e| RepositoryError::Database(e.to_string()))?; diff --git a/atomic-repository/src/repository/kg_attestation.rs b/atomic-repository/src/repository/kg_attestation.rs new file mode 100644 index 00000000..b1ecc838 --- /dev/null +++ b/atomic-repository/src/repository/kg_attestation.rs @@ -0,0 +1,192 @@ +//! Cryptographic verification is offline; network lookup only fills the +//! DID-checked key cache through the repository boundary. + +use super::*; +use atomic_core::pristine::{KgMutTxnT, KgTxnT}; +use atomic_core::{ + kg::{ + iri, + ontology::{Class as C, Predicate as P}, + project_id::MintReason, + }, + KgEdge, KgNode, +}; +use atomic_identity::keypair::PublicKey; + +impl Repository { + pub fn cache_signer_key(&self, did: &str, key: &PublicKey) -> Result<(), RepositoryError> { + if !did.starts_with("did:atomic:") + || !atomic_canonical::did::did_matches_public_key(did, key) + { + return Err(RepositoryError::InvalidOperation { + message: "signer DID does not match the supplied public key".into(), + }); + } + self.ensure_project_identity(MintReason::Upgrade)?; + let mut txn = self.pristine.write_txn()?; + txn.put_signer_key(did, &key.to_base32())?; + if txn.kg_writes_enabled()? { + self.project_attestations_kg(&mut txn)?; + } + txn.commit()?; + Ok(()) + } + + pub fn unresolved_attestation_signers(&self) -> Result, RepositoryError> { + let txn = self.pristine.read_txn()?; + let mut result = std::collections::BTreeSet::new(); + for hash in self.change_store.iter_attestations() { + let attestation = self + .load_attestation(&hash.map_err(|e| RepositoryError::Database(e.to_string()))?)?; + if let Some(did) = attestation + .signer + .as_deref() + .filter(|did| did.starts_with("did:atomic:")) + { + if txn.signer_key(did)?.is_none() { + result.insert(did.to_owned()); + } + } + } + Ok(result.into_iter().collect()) + } + + pub(super) fn project_attestations_kg( + &self, + txn: &mut atomic_core::WriteTxn<'_>, + ) -> Result<(), RepositoryError> { + let project = txn.kg_project()?; + let local = atomic_identity::IdentityStore::open_default() + .ok() + .and_then(|store| store.list().ok()) + .unwrap_or_default(); + let sessions = txn.kg_session_rows()?; + for hash in self.change_store.iter_attestations() { + let hash = hash.map_err(|e| RepositoryError::Database(e.to_string()))?; + let attestation = self.load_attestation(&hash)?; + let id = iri::mint_attestation(&hash); + txn.upsert_kg_node(&KgNode::new( + &id, + C::Attestation, + &hash.to_base32()[..12], + "attestation", + ))?; + for edge in txn.get_kg_edges_from(&id)? { + txn.del_kg_edge(&edge.from_id, &edge.to_id, &edge.kind)?; + } + let session = iri::mint_session(&project, &attestation.session_id); + if txn.get_kg_node(&session)?.is_none() { + txn.upsert_kg_node(&KgNode::new( + &session, + C::Session, + &attestation.session_id, + "attestation", + ))?; + } + txn.upsert_kg_edge(&KgEdge::new(&id, session, P::AttestsSession))?; + for hash in &attestation.changes_covered { + let change = iri::mint_change(hash); + if txn.get_kg_node(&change)?.is_none() { + txn.upsert_kg_node(&KgNode::new( + &change, + C::Change, + &hash.to_base32()[..12], + "attestation", + ))?; + } + txn.upsert_kg_edge(&KgEdge::new(&id, &change, P::CoversChange))?; + txn.upsert_kg_edge(&KgEdge::new(&change, &id, P::AttestedBy))?; + txn.kg_clear_finding("MissingAttestation", &change)?; + } + let mut verification = "Unverifiable"; + if let Some(signer) = attestation.signer.as_deref() { + let signer_id = if signer.starts_with("did:atomic:") { + signer.to_owned() + } else { + iri::mint_named_identity(&project, signer) + }; + txn.upsert_kg_node(&KgNode::new(&signer_id, C::Identity, signer, "attestation"))?; + txn.upsert_kg_edge(&KgEdge::new(&id, signer_id, P::SignedBy))?; + if signer.starts_with("did:atomic:") { + let key = local + .iter() + .find(|identity| { + atomic_canonical::did::did_matches_public_key( + signer, + &identity.public_key, + ) + }) + .map(|identity| identity.public_key.clone()) + .or_else(|| { + txn.signer_key(signer) + .ok() + .flatten() + .and_then(|key| PublicKey::from_base32(&key).ok()) + }); + if let Some(key) = + key.filter(|key| atomic_canonical::did::did_matches_public_key(signer, key)) + { + verification = if attestation.verify_with_did(key.as_bytes()) { + "Verified" + } else { + "Failed" + }; + } + } + } + txn.upsert_kg_edge(&KgEdge::new( + &id, + format!("https://atomic.dev/ns/workflow#{verification}"), + P::VerificationResult, + ))?; + txn.kg_clear_finding("UnresolvedReference", &id)?; + if let Some(previous) = attestation.previous_attestation { + if self.has_attestation(&previous) { + let previous = iri::mint_attestation(&previous); + if txn.get_kg_node(&previous)?.is_none() { + txn.upsert_kg_node(&KgNode::new( + &previous, + C::Attestation, + "previous attestation", + "attestation", + ))?; + } + txn.upsert_kg_edge(&KgEdge::new(&id, previous, P::PreviousAttestation))?; + } else { + txn.kg_finding( + "UnresolvedReference", + &id, + "The previous attestation sidecar is absent", + )?; + } + } + let matches: Vec<_> = sessions + .iter() + .filter(|(record, _)| record.session_id == attestation.session_id) + .flat_map(|(_, turns)| turns) + .filter(|turn| { + !turn.change_hashes.is_empty() + && turn + .change_hashes + .iter() + .all(|hash| attestation.changes_covered.contains(hash)) + }) + .collect(); + txn.kg_clear_finding("AmbiguousAttestationTurn", &id)?; + if matches.len() == 1 { + let turn = matches[0]; + let turn = iri::mint_turn(&project, &turn.session_id, turn.turn_number); + if txn.get_kg_node(&turn)?.is_some() { + txn.upsert_kg_edge(&KgEdge::new(&id, turn, P::Attests))?; + } + } else { + txn.kg_finding( + "AmbiguousAttestationTurn", + &id, + "Attestation coverage does not identify exactly one non-empty session turn", + )?; + } + } + Ok(()) + } +} diff --git a/atomic-repository/src/repository/kg_rebuild.rs b/atomic-repository/src/repository/kg_rebuild.rs new file mode 100644 index 00000000..cad5d589 --- /dev/null +++ b/atomic-repository/src/repository/kg_rebuild.rs @@ -0,0 +1,95 @@ +use super::*; +use atomic_core::{ + kg::{project_id::MintReason, KG_GENERATION}, + pristine::{KgMutTxnT, KgTxnT, VaultTxnT}, +}; + +impl Repository { + /// Rebuild the derived read model in one immediate transaction. Canonical + /// database rows cannot change under the write transaction, and objects + /// loaded from the content store are immutable. An interruption publishes + /// neither partial tables nor a completed generation marker. + pub fn rebuild_kg(&self, force: bool) -> Result { + self.ensure_project_identity(MintReason::Upgrade)?; + if !force && self.pristine.read_txn()?.kg_generation()? == KG_GENERATION { + return Ok(false); + } + let working_copy = self.require_working_copy_id()?; + let _lock = self.try_lock_operation(working_copy)?; + let mut txn = self.pristine.write_txn_immediate()?; + if !force && txn.kg_generation()? == KG_GENERATION { + return Ok(false); + } + txn.begin_kg_rebuild()?; + self.project_vcs_kg(&mut txn)?; + let hashes = self + .change_store + .iter_changes() + .collect::, _>>() + .map_err(|e| RepositoryError::Database(e.to_string()))?; + let context = + super::vault_projection::VaultProjection::new(&txn, txn.kg_project()?, hashes)?; + let mut graphs = Vec::new(); + for meta in txn.list_vault_entries("", None)? { + if let Some(entry) = txn.get_vault_entry(&meta.path)? { + graphs.push(context.extract(&meta.path, &entry)?); + } + } + // Nodes before relationships: references can point to later vault rows. + for graph in &graphs { + for node in &graph.nodes { + if node + .metadata + .as_ref() + .is_some_and(|metadata| metadata.get("derived_from_vault_path").is_some()) + || txn.get_kg_node(&node.id)?.is_none() + { + txn.upsert_kg_node(node)?; + } + } + } + for graph in &graphs { + for edge in &graph.edges { + txn.upsert_kg_edge(edge)?; + } + } + for (record, turns) in txn.kg_session_rows()? { + txn.project_session_kg(&record, &turns)?; + } + self.project_attestations_kg(&mut txn)?; + // No canonical endpoint may silently dangle; ontology individuals are + // intentionally resources defined by the embedded ontology, not rows. + for edge in txn.all_kg_edges()? { + if txn.get_kg_node(&edge.from_id)?.is_none() + || (txn.get_kg_node(&edge.to_id)?.is_none() + && !matches!( + edge.to_id.as_str(), + "https://atomic.dev/ns/workflow#Verified" + | "https://atomic.dev/ns/workflow#Failed" + | "https://atomic.dev/ns/workflow#Unverifiable" + )) + { + return Err(RepositoryError::InvalidOperation { + message: format!("unclosed KG edge: {} -> {}", edge.from_id, edge.to_id), + }); + } + } + txn.finish_kg_rebuild()?; + txn.commit()?; + Ok(true) + } + + pub(super) fn refresh_session_kg(&self) -> Result<(), RepositoryError> { + self.ensure_project_identity(MintReason::Upgrade)?; + let mut txn = self.pristine.write_txn()?; + if !txn.kg_writes_enabled()? { + return Ok(()); + } + for (record, turns) in txn.kg_session_rows()? { + txn.project_session_kg(&record, &turns)?; + } + self.project_attestations_kg(&mut txn)?; + txn.commit()?; + Ok(()) + } +} diff --git a/atomic-repository/src/repository/mod.rs b/atomic-repository/src/repository/mod.rs index 143618b4..1c23d09a 100644 --- a/atomic-repository/src/repository/mod.rs +++ b/atomic-repository/src/repository/mod.rs @@ -128,7 +128,9 @@ mod split; mod staging; mod stash; mod switch; +mod syntax_index; mod synthesis; +pub use syntax_index::{Caller, SyntaxIndex}; mod tag_projection; mod views; mod working_copy; @@ -243,8 +245,14 @@ mod changes; mod content; mod history; mod insert; +mod kg_attestation; +mod kg_rebuild; +mod project_identity; pub mod provenance_gate; mod provenance_summary; +pub use project_identity::{ + build_project_identity, project_identity_hash, verify_project_identity, +}; mod record; mod remotes; mod status; @@ -253,6 +261,8 @@ mod tracking; mod triage; mod vault; mod vault_defaults; +mod vault_projection; +pub use vault_triples::WorkflowSnapshot; mod vault_embeddings; mod vault_goal; mod vault_identity; @@ -611,6 +621,10 @@ default = "{}" is_sandbox: false, }; repository.fence_recovery_for_budget(working_copy_id, budget)?; + if !budget.is_metadata_only() { + repository.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; + repository.rebuild_kg(false)?; + } // Deferred TREE or operation recovery may rewrite the compatibility // pointer. Reassert the persistent record as the sole authority. diff --git a/atomic-repository/src/repository/operation.rs b/atomic-repository/src/repository/operation.rs index 6693b869..5bcac233 100644 --- a/atomic-repository/src/repository/operation.rs +++ b/atomic-repository/src/repository/operation.rs @@ -1822,6 +1822,14 @@ impl Repository { let mut affected_views = BTreeSet::new(); for (transition, _) in pending { apply_metadata_value_in_write(&mut txn, &transition.target, &transition.expected_new)?; + if matches!( + transition.target, + MetadataTarget::View { .. } | MetadataTarget::ViewChange { .. } + ) && txn.kg_writes_enabled()? + && txn.project_identity()?.is_some() + { + self.project_view_kg(&mut txn)?; + } if let Some(view) = metadata_target_view(&transition.target) { affected_views.insert(view.to_string()); } diff --git a/atomic-repository/src/repository/project_identity.rs b/atomic-repository/src/repository/project_identity.rs new file mode 100644 index 00000000..54fa2f5c --- /dev/null +++ b/atomic-repository/src/repository/project_identity.rs @@ -0,0 +1,144 @@ +use super::*; +use atomic_core::kg::project_id::{MintReason, ProjectId, ProjectIdentityRecord}; +use atomic_identity::{keypair::PublicKey, KeyPair, Signer}; + +fn invalid(error: impl std::fmt::Display) -> RepositoryError { + RepositoryError::InvalidOperation { + message: error.to_string(), + } +} + +/// Hash authority excludes proof and includes the inline namespace definitions. +pub fn project_identity_hash(object: &serde_json::Value) -> Hash { + let value = atomic_canonical::proof::signing_view(object); + let mut bytes = b"atomic.project-identity.v1\0".to_vec(); + bytes.extend_from_slice(atomic_canonical::jcs::canonicalize(&value).as_bytes()); + Hash::of(&bytes) +} + +pub fn build_project_identity( + reason: MintReason, + keypair: Option<&KeyPair>, +) -> ProjectIdentityRecord { + let id = ProjectId::generate(); + let created_at = chrono::Utc::now().to_rfc3339(); + let mut object = serde_json::json!({ + "@context": {"atomic": "https://atomic.dev/ns/workflow#", "dcterms": "http://purl.org/dc/terms/"}, + "@type": "atomic:ProjectIdentity", "atomic:projectId": id.to_base32(), + "atomic:schemaVersion": 1, "dcterms:created": created_at, + }); + if let Some(keypair) = keypair.filter(|_| reason == MintReason::Created) { + object["atomic:issuer"] = + serde_json::json!({"@id": atomic_canonical::did::did_for_public_key(&keypair.public)}); + let bytes = atomic_canonical::jcs::canonicalize(&object); + let signature = Signer::new(keypair).sign(bytes.as_bytes()); + object = atomic_canonical::proof::attach_proof(object, &keypair.public, &signature); + } + ProjectIdentityRecord { + hash: project_identity_hash(&object), + object: atomic_canonical::jcs::canonicalize(&object).into_bytes(), + mint_reason: reason, + created_at, + previous: None, + } +} + +/// Verify the prefixed content hash and, when present, the existing Data Integrity suite. +pub fn verify_project_identity( + record: &ProjectIdentityRecord, + public_key: Option<&PublicKey>, +) -> Result<(), RepositoryError> { + record.project_id().map_err(invalid)?; + let value: serde_json::Value = serde_json::from_slice(&record.object).map_err(invalid)?; + if project_identity_hash(&value) != record.hash { + return Err(invalid("project identity hash mismatch")); + } + if value.get("proof").is_some() { + let key = public_key.ok_or_else(|| invalid("project identity signer key unavailable"))?; + let proof: atomic_canonical::node::Proof = + serde_json::from_value(value["proof"].clone()).map_err(invalid)?; + let did = value["atomic:issuer"]["@id"] + .as_str() + .ok_or_else(|| invalid("project identity has no issuer"))?; + if !atomic_canonical::did::did_matches_public_key(did, key) + || proof.type_ != atomic_canonical::proof::PROOF_TYPE + || proof.cryptosuite != atomic_canonical::proof::CRYPTOSUITE + || proof.proof_purpose != atomic_canonical::proof::PROOF_PURPOSE + || proof.verification_method != atomic_canonical::did::verification_method(did) + { + return Err(invalid("project identity proof metadata mismatch")); + } + let signature = atomic_canonical::proof::proof_signature(&value).map_err(invalid)?; + let unsigned = atomic_canonical::proof::signing_view(&value); + signature + .verify( + atomic_canonical::jcs::canonicalize(&unsigned).as_bytes(), + key, + ) + .map_err(invalid)?; + } + Ok(()) +} + +impl Repository { + pub fn project_identity(&self) -> Result, RepositoryError> { + Ok(self.pristine.read_txn()?.project_identity()?) + } + + /// Mint once, before projecting project-qualified resources. Upgrade opens + /// never load a private key. Repository creation may sign with the default identity. + pub fn ensure_project_identity( + &self, + reason: MintReason, + ) -> Result { + if let Some(record) = self.project_identity()? { + return Ok(record); + } + let keypair = if reason == MintReason::Created { + if let Ok(store) = atomic_identity::IdentityStore::open_default() { + match store.get_default().map_err(invalid)? { + Some(identity) => { + Some(store.load_keypair(&identity.id, None).map_err(invalid)?) + } + None => None, + } + } else { + None + } + } else { + None + }; + let record = build_project_identity(reason, keypair.as_ref()); + let mut txn = self.pristine.write_txn()?; + let record = txn.put_project_identity_if_absent(&record)?; + txn.commit()?; + Ok(record) + } + + pub fn project_id(&self) -> Result { + self.project_identity()? + .ok_or_else(|| invalid("MissingProjectIdentity"))? + .project_id() + .map_err(invalid) + } +} + +#[cfg(test)] +mod tests { + use super::*; + #[test] + fn signed_identity_hash_and_proof_are_independently_verified() { + let pair = KeyPair::generate(); + let record = build_project_identity(MintReason::Created, Some(&pair)); + verify_project_identity(&record, Some(&pair.public)).unwrap(); + let mut forged = record.clone(); + let mut value: serde_json::Value = serde_json::from_slice(&record.object).unwrap(); + value["atomic:schemaVersion"] = serde_json::json!(2); + forged.hash = project_identity_hash(&value); + forged.object = atomic_canonical::jcs::canonicalize(&value).into_bytes(); + assert!(verify_project_identity(&forged, Some(&pair.public)).is_err()); + let upgrade = build_project_identity(MintReason::Upgrade, Some(&pair)); + verify_project_identity(&upgrade, None).unwrap(); + assert!(!String::from_utf8(upgrade.object).unwrap().contains("proof")); + } +} diff --git a/atomic-repository/src/repository/syntax_index.rs b/atomic-repository/src/repository/syntax_index.rs new file mode 100644 index 00000000..4a186c6b --- /dev/null +++ b/atomic-repository/src/repository/syntax_index.rs @@ -0,0 +1,187 @@ +//! Request-scoped source relationships. Nothing here is persisted in the KG. + +use super::*; +use atomic_semantic::{Entity, EntityKind, ParserRegistry}; +use std::collections::{BTreeMap, BTreeSet}; + +#[derive(Clone, Debug, serde::Serialize)] +pub struct Caller { + pub path: String, + pub name: String, + pub line: u32, + pub summary: Option, +} + +#[derive(Default)] +pub struct SyntaxIndex { + pub entities: Vec, + pub calls: BTreeSet<(String, String)>, + pub file_deps: BTreeSet<(String, String)>, + pub file_modules: BTreeMap, +} + +fn selector(entity: &Entity) -> String { + format!("{}#{}", entity.file, entity.name) +} +fn callable(entity: &Entity) -> bool { + matches!(entity.kind, EntityKind::Function | EntityKind::Method) +} + +impl SyntaxIndex { + pub fn from_sources(sources: &[(String, String)]) -> Self { + let mut parser = ParserRegistry::new(); + let mut index = Self::default(); + let mut parsed = Vec::new(); + let mut names = BTreeMap::>::new(); + for (path, source) in sources { + let entities = parser.extract(path, source); + for entity in entities.iter().filter(|entity| callable(entity)) { + names + .entry(entity.name.clone()) + .or_default() + .insert(selector(entity)); + } + index.file_modules.insert( + path.clone(), + Path::new(path) + .parent() + .and_then(|p| p.to_str()) + .unwrap_or_default() + .to_owned(), + ); + index.entities.extend(entities.clone()); + parsed.push((path, source, entities)); + } + for (path, source, entities) in &parsed { + for reference in parser.extract_references(path, source) { + let Some(caller) = entities + .iter() + .filter(|entity| { + callable(entity) + && reference.line >= entity.line + && reference.line <= entity.end_line + }) + .min_by_key(|entity| entity.end_line - entity.line) + else { + continue; + }; + let caller = selector(caller); + if let Some(targets) = names.get(&reference.symbol) { + for target in targets { + if target != &caller { + index.calls.insert((target.clone(), caller.clone())); + } + } + } + } + for import in entities + .iter() + .filter(|entity| entity.kind == EntityKind::Import) + { + let tokens: BTreeSet<_> = import + .name + .split(|c: char| !c.is_alphanumeric() && c != '_') + .filter(|s| !s.is_empty()) + .collect(); + for (target, _) in sources { + let stem = Path::new(target) + .file_stem() + .and_then(|s| s.to_str()) + .unwrap_or_default(); + if target != *path && tokens.contains(stem) { + index.file_deps.insert(((*path).clone(), target.clone())); + } + } + } + } + index + } + + pub fn callers(&self, target: &str) -> Vec { + let (path, name) = target + .rsplit_once('#') + .map(|(path, name)| (Some(path), name)) + .unwrap_or((None, target)); + let targets: BTreeSet<_> = self + .entities + .iter() + .filter(|entity| { + callable(entity) + && entity.name == name + && path.is_none_or(|path| entity.file == path) + }) + .map(selector) + .collect(); + let callers: BTreeSet<_> = self + .calls + .iter() + .filter(|(callee, _)| targets.contains(callee)) + .map(|(_, caller)| caller.as_str()) + .collect(); + let mut result: Vec<_> = self + .entities + .iter() + .filter(|entity| callable(entity) && callers.contains(selector(entity).as_str())) + .map(|entity| Caller { + path: entity.file.clone(), + name: entity.name.clone(), + line: entity.line, + summary: entity.signature.clone(), + }) + .collect(); + result.sort_by(|a, b| (&a.path, &a.name, a.line).cmp(&(&b.path, &b.name, b.line))); + result.dedup_by(|a, b| a.path == b.path && a.name == b.name && a.line == b.line); + result + } +} + +impl Repository { + /// Parse each file once. View reads render graph bytes through the selected + /// filter; working-copy queries read that physical directory's tracked files. + pub fn syntax_index(&self, view: Option<&str>) -> Result { + let paths = if let Some(view) = view { + let txn = self.pristine.read_txn()?; + let view = txn + .get_view(view)? + .ok_or_else(|| RepositoryError::ViewNotFound { + name: view.to_owned(), + })?; + let visibility = graph_visibility_closure(&txn, &view)?; + self.project_tree_for_visibility(&txn, &visibility)? + .present + .into_values() + .filter(|entry| !entry.is_directory) + .map(|entry| entry.path) + .collect::>() + } else { + self.list_tracked_files()? + .into_iter() + .filter(|file| !file.is_directory) + .map(|file| file.path.to_string_lossy().into_owned()) + .collect() + }; + let mut sources = Vec::new(); + for path in paths { + if !atomic_semantic::is_supported(&path) { + continue; + } + let bytes = if let Some(view) = view { + self.get_file_content_on_view(&path, view)? + } else { + match std::fs::read(self.root().join(&path)) { + Ok(bytes) => Some(bytes), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => None, + Err(error) => return Err(error.into()), + } + }; + if let Some(source) = bytes.and_then(|bytes| String::from_utf8(bytes).ok()) { + sources.push((path, source)); + } + } + Ok(SyntaxIndex::from_sources(&sources)) + } + + pub fn query_callers(&self, target: &str) -> Result, RepositoryError> { + Ok(self.syntax_index(None)?.callers(target)) + } +} diff --git a/atomic-repository/src/repository/tests/capability_tests.rs b/atomic-repository/src/repository/tests/capability_tests.rs index 39a13a0c..0a6726fa 100644 --- a/atomic-repository/src/repository/tests/capability_tests.rs +++ b/atomic-repository/src/repository/tests/capability_tests.rs @@ -139,17 +139,24 @@ fn save_change_declares_vnext_capability() { .pristine .required_repository_capabilities() .unwrap() - .is_empty()); + .iter() + .all(|capability| capability.id != "change-format-vnext")); repo.save_change(&create_test_change("capability fence")) .unwrap(); assert_eq!( repo.pristine.required_repository_capabilities().unwrap(), - vec![RequiredRepositoryCapability { - id: "change-format-vnext".to_string(), - minimum_version: 1, - }] + vec![ + RequiredRepositoryCapability { + id: "change-format-vnext".to_string(), + minimum_version: 1, + }, + RequiredRepositoryCapability { + id: "kg-semweb-v1".into(), + minimum_version: 1 + } + ] ); } @@ -166,7 +173,8 @@ fn v1_pre_serialized_save_does_not_declare_vnext() { .pristine .required_repository_capabilities() .unwrap() - .is_empty()); + .iter() + .all(|capability| capability.id != "change-format-vnext")); assert!(repo.change_store.change_path(&hash).is_file()); } @@ -185,10 +193,16 @@ fn v2_pre_serialized_save_marks_before_object_persistence() { assert_eq!( repo.pristine.required_repository_capabilities().unwrap(), - vec![RequiredRepositoryCapability { - id: "change-format-vnext".to_string(), - minimum_version: 1, - }] + vec![ + RequiredRepositoryCapability { + id: "change-format-vnext".to_string(), + minimum_version: 1, + }, + RequiredRepositoryCapability { + id: "kg-semweb-v1".into(), + minimum_version: 1 + } + ] ); assert!(!repo.change_store.change_path(&hash).exists()); } @@ -214,7 +228,8 @@ fn malformed_or_unsupported_pre_serialized_bytes_do_not_mutate() { .pristine .required_repository_capabilities() .unwrap() - .is_empty()); + .iter() + .all(|capability| capability.id != "change-format-vnext")); assert!(!repo.change_store.change_path(&hash).exists()); drop(repo); drop(temp); diff --git a/atomic-repository/src/repository/triage.rs b/atomic-repository/src/repository/triage.rs index ad37bfce..6162c917 100644 --- a/atomic-repository/src/repository/triage.rs +++ b/atomic-repository/src/repository/triage.rs @@ -21,7 +21,7 @@ use super::*; -use atomic_core::pristine::ontology::edge_kind; +use atomic_core::kg::{iri, ontology::Class, ontology::Predicate as P}; use serde::{Deserialize, Serialize}; /// Whether a closure addition is covered by an intent in the knowledge graph. @@ -212,7 +212,7 @@ impl Repository { let modifies: Vec = self .change_modified_paths(hash)? .into_iter() - .map(|p| format!("file:{p}")) + .map(|p| iri::mint_file(&self.project_id().expect("project identity"), &p)) .collect(); if modifies.is_empty() { @@ -228,9 +228,12 @@ impl Repository { for file_id in &modifies { let neighborhood = self.vault_kg_neighbors(file_id, 1)?; if neighborhood.edges.iter().any(|e| { - e.kind == edge_kind::TOUCHES + e.kind == P::Touches && e.to_id == *file_id - && e.from_id.starts_with("task:") + && neighborhood + .nodes + .iter() + .any(|node| node.id == e.from_id && node.has_type(Class::Task)) }) { covered = true; break; @@ -253,10 +256,9 @@ mod tests { use crate::tracking::TrackingOptions; use crate::Repository; use atomic_core::change::ChangeHeader; - use atomic_core::pristine::ontology::edge_kind; + use atomic_core::kg::{iri, ontology::Predicate as P}; use atomic_core::pristine::VaultEntryType; use atomic_core::types::Hash; - use atomic_core::Base32; use tempfile::TempDir; fn record_all(repo: &Repository, message: &str) -> Hash { @@ -299,10 +301,10 @@ mod tests { let hash = record_all(&repo, "add foo"); // Precondition: the KG has NO MODIFIES edge for this change (unenriched). - let node_id = format!("change:{}", &hash.to_base32()[..12]); + let node_id = iri::mint_change(&hash); let sub = repo.vault_kg_neighbors(&node_id, 2).unwrap(); assert!( - !sub.edges.iter().any(|e| e.kind == edge_kind::MODIFIES), + !sub.edges.iter().any(|e| e.kind == P::ModifiesFile), "precondition: no MODIFIES edge should exist without enrichment" ); @@ -314,7 +316,7 @@ mod tests { // An intent whose task touches src/foo.rs projects a TOUCHES edge on // store (via project_intent_semantics) — the only KG-side dependency. - let fm = r#"{"id":"COV-1","title":"Cover foo","status":"in-progress"}"#; + let fm = r#"{"id":"COV-1","uid":"01ARZ3NDEKTSV4RRFFQ69G5FAV","title":"Cover foo","status":"in_progress"}"#; let body = "\ :::task{#cov-1-1 status=open}\nWork on foo.\n::file-ref{path=src/foo.rs}\n:::"; repo.vault_store( @@ -327,7 +329,10 @@ mod tests { // Covered — resolved from file_ops + the task's TOUCHES, no MODIFIES. let (modifies, coverage) = repo.change_coverage(&hash).unwrap(); - assert_eq!(modifies, vec!["file:src/foo.rs".to_string()]); + assert_eq!( + modifies, + vec![iri::mint_file(&repo.project_id().unwrap(), "src/foo.rs")] + ); assert_eq!(coverage, Coverage::Covered); } @@ -352,7 +357,7 @@ mod tests { let hash = record_all(&repo, "add bar"); // An intent whose task touches a DIFFERENT file. - let fm = r#"{"id":"COV-2","title":"Cover foo","status":"in-progress"}"#; + let fm = r#"{"id":"COV-2","uid":"01ARZ3NDEKTSV4RRFFQ69G5FAW","title":"Cover foo","status":"in_progress"}"#; let body = "\ :::task{#cov-2-1 status=open}\nWork on foo.\n::file-ref{path=src/foo.rs}\n:::"; repo.vault_store( @@ -364,7 +369,10 @@ mod tests { .unwrap(); let (modifies, coverage) = repo.change_coverage(&hash).unwrap(); - assert_eq!(modifies, vec!["file:src/bar.rs".to_string()]); + assert_eq!( + modifies, + vec![iri::mint_file(&repo.project_id().unwrap(), "src/bar.rs")] + ); assert_eq!(coverage, Coverage::Uncovered); } diff --git a/atomic-repository/src/repository/vault.rs b/atomic-repository/src/repository/vault.rs index ef29ca1b..29af909e 100644 --- a/atomic-repository/src/repository/vault.rs +++ b/atomic-repository/src/repository/vault.rs @@ -185,6 +185,7 @@ impl Repository { content: Vec, frontmatter_json: String, ) -> Result { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; let now = chrono::Utc::now().to_rfc3339(); let entry = VaultEntry::new(entry_type, content, frontmatter_json, now.clone()); let content_hash = Hash::from_bytes(entry.content_hash); @@ -204,18 +205,14 @@ impl Repository { } _ => Vec::new(), }; - if let Some(previous_entry_type) = previous_entry + if let Some(_previous_entry_type) = previous_entry .as_ref() .map(|previous| previous.entry_type) .filter(|previous| *previous != entry_type) { txn.init_kg() .map_err(|e| RepositoryError::Database(e.to_string()))?; - txn.del_kg_node(&super::vault_triples::entry_subject( - path, - previous_entry_type, - )) - .map_err(|e| RepositoryError::Database(e.to_string()))?; + super::vault_triples::delete_vault_projection(&mut txn, path)?; } txn.put_vault_entry(path, &entry) @@ -258,6 +255,7 @@ impl Repository { path: &str, entry: &VaultEntry, ) -> Result { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; let content_hash = Hash::from_bytes(entry.content_hash); let now = chrono::Utc::now().to_rfc3339(); @@ -276,18 +274,14 @@ impl Repository { } _ => Vec::new(), }; - if let Some(previous_entry_type) = previous_entry + if let Some(_previous_entry_type) = previous_entry .as_ref() .map(|previous| previous.entry_type) .filter(|previous| *previous != entry.entry_type) { txn.init_kg() .map_err(|e| RepositoryError::Database(e.to_string()))?; - txn.del_kg_node(&super::vault_triples::entry_subject( - path, - previous_entry_type, - )) - .map_err(|e| RepositoryError::Database(e.to_string()))?; + super::vault_triples::delete_vault_projection(&mut txn, path)?; } txn.put_vault_entry(path, entry) @@ -324,17 +318,16 @@ impl Repository { /// Called as a side effect of `vault_store` and `vault_store_entry`. /// Best-effort — logs on failure but never fails the parent operation. fn vault_auto_index(&self, path: &str) { - // Signed attestation blobs must not pollute semantic search, and they - // contribute no KG triples (vault_extract_kg early-returns empty for - // Attestation). Skip embedding entirely for them. + // Signed documents contribute typed workflow links, but their serialized + // JSON/proof bytes do not belong in semantic content embeddings. let is_attestation = matches!( self.vault_retrieve(path), Ok(Some(e)) if e.entry_type == VaultEntryType::Attestation ); - // KG indexing — safe: extract_kg returns empty for Attestation, so - // vault_store_kg deletes any stale rows and stores nothing. - if let Err(e) = self.vault_index_kg(path) { + // Re-resolve references as a set: adding/removing a target can repair + // another entry's historical gap without changing that source entry. + if let Err(e) = self.vault_reindex_kg() { log::debug!("Auto KG index for {}: {}", path, e); } @@ -391,6 +384,7 @@ impl Repository { /// Delete a vault entry by path. Also updates the manifest. pub fn vault_delete(&self, path: &str) -> Result { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; let mut txn = self .pristine .write_txn() @@ -406,10 +400,6 @@ impl Repository { } else { Vec::new() }; - let entry_type = previous_entry - .as_ref() - .map(|entry| entry.entry_type) - .unwrap_or_else(|| infer_entry_type(path)); txn.init_kg() .map_err(|e| RepositoryError::Database(e.to_string()))?; @@ -420,13 +410,10 @@ impl Repository { // KG and embeddings are derived from the Vault entry. Clean both even // if the source entry is already missing so a retry repairs stale data. - let node_id = super::vault_triples::entry_subject(path, entry_type); // Remove projected child nodes (tasks, acceptance criteria, scope, // constraints) before the subject so an intent deletion does not orphan // them or their SATISFIES/TOUCHES edges. No-op for non-intent subjects. - super::vault_triples::delete_intent_child_nodes(&mut txn, &node_id)?; - txn.del_kg_node(&node_id) - .map_err(|e| RepositoryError::Database(e.to_string()))?; + super::vault_triples::delete_vault_projection(&mut txn, path)?; txn.del_embeddings(path) .map_err(|e| RepositoryError::Database(e.to_string()))?; @@ -442,6 +429,7 @@ impl Repository { txn.commit() .map_err(|e| RepositoryError::Database(e.to_string()))?; + self.vault_reindex_kg()?; for update in cascade_updates { self.vault_auto_index(&update.path); if let Err(error) = self.vault_materialize(&update.path) { diff --git a/atomic-repository/src/repository/vault_intent.rs b/atomic-repository/src/repository/vault_intent.rs index 47a1bf9c..dde369d2 100644 --- a/atomic-repository/src/repository/vault_intent.rs +++ b/atomic-repository/src/repository/vault_intent.rs @@ -1765,16 +1765,26 @@ mod tests { serde_json::json!(["memory:auth-decision", "urn:atomic:memory:retry-policy"]) ); + for name in ["auth-decision", "retry-policy"] { + repo.vault_store( + &format!("memory/{name}.md"), + VaultEntryType::Memory, + name.as_bytes().to_vec(), + "{}".into(), + ) + .unwrap(); + } + let project = repo.project_id().unwrap(); let (_, edges) = repo.vault_extract_kg(&result.intent_file, &entry).unwrap(); assert!(edges.iter().any(|edge| { - edge.kind == atomic_core::pristine::ontology::predicate::INFORMED_BY - && edge.to_id == "memory:auth-decision" + edge.kind == atomic_core::kg::ontology::Predicate::InformedBy + && edge.to_id == atomic_core::kg::iri::mint_memory(&project, "auth-decision") })); let retry_policy = edges .iter() .find(|edge| { - edge.kind == atomic_core::pristine::ontology::predicate::INFORMED_BY - && edge.to_id == "memory:retry-policy" + edge.kind == atomic_core::kg::ontology::Predicate::InformedBy + && edge.to_id == atomic_core::kg::iri::mint_memory(&project, "retry-policy") }) .expect("missing retry-policy lineage edge"); let metadata = retry_policy.metadata.as_ref().unwrap(); diff --git a/atomic-repository/src/repository/vault_kg_enrich.rs b/atomic-repository/src/repository/vault_kg_enrich.rs index c5e1df6c..e6c80bcf 100644 --- a/atomic-repository/src/repository/vault_kg_enrich.rs +++ b/atomic-repository/src/repository/vault_kg_enrich.rs @@ -1,1393 +1,274 @@ -//! VCS → KG enrichment pipeline. -//! -//! Populates the knowledge graph from repository data: -//! - Changes → `change:{hash}` nodes + `AUTHORED_BY`, `MODIFIES` edges -//! - Files → `file:{path}` nodes -//! - Modules → `module:{dir}` nodes + `PART_OF` edges -//! - Views → `view:{name}` nodes + `CHILD_OF`, `ON_VIEW` edges -//! - Dependencies → `DEPENDS_ON` edges between change nodes -//! - Includes → `INCLUDES` edges between files (from import entities) - -use std::collections::{HashMap, HashSet}; +//! Workflow projection over immutable changes and actual view memberships. +//! Source structure is parsed by atomic-semantic at query time. use super::*; -use atomic_core::pristine::ontology::edge_kind; -use atomic_core::pristine::vault::{KgEdge, KgNode}; use atomic_core::pristine::{KgMutTxnT, KgTxnT}; -use atomic_core::types::Base32; - -/// KG node `source` values produced by the VCS/tree-sitter enrichment -/// pipeline. Nodes with these sources are fully regenerated by -/// [`Repository::kg_enrich_from_vcs`] and its phases, so a rebuild can safely -/// drop and re-derive them. Vault- and session-sourced nodes use other -/// `source` values and are never touched by enrichment. -const DERIVED_KG_SOURCES: &[&str] = &["vcs", "ast"]; - -/// True if `path` must be excluded from KG enrichment: an Atomic-internal -/// path (see [`crate::ignore::is_enrichment_internal`]) or one matched by the -/// enrichment ignore rules (`.atomicignore`, `.gitignore`, `.ignore`, global). -/// -/// `rules` is a preloaded [`IgnoreRules::load_for_enrichment`] matcher so -/// callers can reuse it across many paths. Paths are treated as files; -/// directory-only ignore patterns (e.g. `node_modules/`) still match via the -/// crate's parent-directory matching. -fn is_path_ignored_for_enrichment(path: &str, rules: &IgnoreRules) -> bool { - let p = std::path::Path::new(path); - crate::ignore::is_enrichment_internal(p) || rules.is_ignored(p, false) +use atomic_core::{ + kg::{ + iri, + ontology::{Class as C, Predicate as P}, + project_id::MintReason, + }, + KgEdge, KgNode, +}; + +#[derive(Clone, Debug, Default)] +pub struct KgEnrichStats { + pub views: usize, + pub files: usize, + pub changes: usize, +} +impl KgEnrichStats { + pub fn total(&self) -> usize { + self.views + self.files + self.changes + } +} +impl std::fmt::Display for KgEnrichStats { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!( + f, + "{} views, {} files, {} changes", + self.views, self.files, self.changes + ) + } } impl Repository { - /// Enrich the KG from all VCS data (views, changes, files, deps). - /// - /// This is the main entry point for populating the KG from the - /// repository's version control data. Call this after `init_kg()`. - /// - /// Returns statistics about the number of nodes and edges created. pub fn kg_enrich_from_vcs( &self, working_copy: WorkingCopyId, ) -> Result { self.validate_working_copy(working_copy)?; - // Phase 1: Views → nodes + parent edges - let views = self.kg_enrich_views()?; - - // Phase 2: Tracked files → nodes - let files = self.kg_enrich_files(working_copy)?; - - // Phase 2b: Module nodes + PART_OF edges - let modules = self.kg_enrich_modules(working_copy)?; - - // Phase 3: Changes → nodes + edges (modifies, authored_by, on_view, depends_on) - let changes = self.kg_enrich_changes(working_copy)?; - - let mut stats = KgEnrichStats { - views, - files, - modules, - changes, - ..Default::default() - }; - - // Phase 4: AST entities → nodes + DEFINES edges - stats.entities = self.kg_enrich_entities(working_copy)?; - - // Phase 4b: INCLUDES edges (from import entities) - stats.includes = self.kg_enrich_includes(working_copy)?; - - // Phase 4c: CALLS edges (caller entity → callee entity) - stats.calls = self.kg_enrich_calls(working_copy)?; - - Ok(stats) - } - - /// List tracked files eligible for knowledge-graph enrichment. - /// - /// Filters [`list_tracked_files`](Self::list_tracked_files) to exclude: - /// - Atomic/Git internal directories (`.atomic`, `.git`), and - /// - paths matched by `.atomicignore`, `.gitignore`, `.ignore`, or the - /// global ignore file. - /// - /// This keeps build artifacts and dependencies (e.g. `node_modules/`, - /// `dist/`, `target/`) out of the KG so `atomic query enrich` indexes - /// only meaningful source files. - fn list_enrichable_files( - &self, - working_copy: WorkingCopyId, - ) -> Result, RepositoryError> { - self.validate_working_copy(working_copy)?; - let files = self.list_tracked_files()?; - let rules = IgnoreRules::load_for_enrichment(&self.root); - - Ok(files - .into_iter() - .filter(|file| { - !crate::ignore::is_enrichment_internal(&file.path) - && !rules.is_ignored(&file.path, file.is_directory) - }) - .collect()) - } - - /// Remove all knowledge-graph nodes (and their edges) derived from VCS - /// data and tree-sitter parsing. - /// - /// Clears `file:`, `module:`, `view:`, `change:`, `identity:`, and - /// `entity:` nodes (those whose `source` is in `DERIVED_KG_SOURCES`) so - /// a subsequent full enrich rebuilds the graph from scratch, dropping - /// entries that no longer apply — e.g. files that became ignored or were - /// deleted. Vault- and session-sourced nodes (intents, memories, goals, - /// sessions, todos) are left untouched. - /// - /// Returns the number of nodes removed. - pub fn kg_clear_derived(&self) -> Result { - let mut txn = self - .pristine - .write_txn() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - txn.init_kg() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - let ids = txn - .kg_node_ids_by_source(DERIVED_KG_SOURCES) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - for id in &ids { - txn.del_kg_node(id) - .map_err(|e| RepositoryError::Database(e.to_string()))?; + self.ensure_project_identity(MintReason::Upgrade)?; + let mut txn = self.pristine.write_txn()?; + if !txn.kg_writes_enabled()? { + return Ok(KgEnrichStats::default()); } - - txn.commit() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - Ok(ids.len()) + let stats = self.project_vcs_kg(&mut txn)?; + txn.commit()?; + Ok(stats) } - /// Enrich the KG with view hierarchy data. - /// - /// For each view in the repository, creates a `view:{name}` node and - /// a `CHILD_OF` edge to its parent view (if any). pub fn kg_enrich_views(&self) -> Result { - let view_names = self - .list_views() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - let mut txn = self - .pristine - .write_txn() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - let mut count = 0; - for name in &view_names { - // Resolve full ViewInfo for this view. If resolution fails - // (shouldn't happen — the name just came from list_views), - // skip rather than abort the entire enrichment. - let info = match self.get_view_info(name) { - Ok(info) => info, - Err(e) => { - log::warn!("kg_enrich_views: skipping view '{}': {}", name, e); - continue; - } - }; - - let node = KgNode::new(format!("view:{}", info.name), "view", &info.name, "vcs") - .with_summary(format!( - "{} view, {} changes", - info.kind_label(), - info.change_count - )) - .with_metadata(serde_json::json!({ - "scope": info.kind_label(), - "change_count": info.change_count, - "state": info.state_short(), - })); - - txn.upsert_kg_node(&node) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - count += 1; - - // Parent edge - if let Some(ref parent_name) = info.parent_name { - let edge = KgEdge::new( - format!("view:{}", info.name), - format!("view:{}", parent_name), - edge_kind::CHILD_OF, - ); - txn.upsert_kg_edge(&edge) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - } - } - - txn.commit() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - Ok(count) - } - - /// Enrich the KG with module (directory) nodes and `PART_OF` edges. - /// - /// For each directory that directly contains at least one tracked file, - /// creates a `module:{dir}` node. Then creates `PART_OF` edges from - /// file → module and module → parent module (only between modules that - /// were created, not every intermediate directory). - pub fn kg_enrich_modules(&self, working_copy: WorkingCopyId) -> Result { - let files = self.list_enrichable_files(working_copy)?; - - // Collect directories that directly contain at least one file. - let mut dirs_with_files: HashSet = HashSet::new(); - // Map each file path to its parent directory. - let mut file_to_dir: Vec<(String, String)> = Vec::new(); - - for file in &files { - if file.is_directory { - continue; - } - let path = file.path.to_string_lossy().to_string(); - if let Some(dir) = path.rsplit_once('/').map(|(d, _)| d.to_string()) { - dirs_with_files.insert(dir.clone()); - file_to_dir.push((path, dir)); - } - // Files at the root level (no '/') don't belong to a module. - } - - if dirs_with_files.is_empty() { + self.ensure_project_identity(MintReason::Upgrade)?; + let mut txn = self.pristine.write_txn()?; + if !txn.kg_writes_enabled()? { return Ok(0); } - - let mut txn = self - .pristine - .write_txn() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - let mut count = 0; - - // Create module nodes for each directory that has direct files. - for dir in &dirs_with_files { - let label = dir.rsplit('/').next().unwrap_or(dir); - let node = KgNode::new(format!("module:{}", dir), "module", label, "vcs") - .with_summary(dir.as_str()); - - txn.upsert_kg_node(&node) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - count += 1; - } - - // PART_OF edges: file → module - for (file_path, dir) in &file_to_dir { - let edge = KgEdge::new( - format!("file:{}", file_path), - format!("module:{}", dir), - edge_kind::PART_OF, - ); - txn.upsert_kg_edge(&edge) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - } - - // PART_OF edges: child module → parent module (only between existing modules). - for dir in &dirs_with_files { - if let Some((parent, _)) = dir.rsplit_once('/') { - if dirs_with_files.contains(parent) { - let edge = KgEdge::new( - format!("module:{}", dir), - format!("module:{}", parent), - edge_kind::PART_OF, - ); - txn.upsert_kg_edge(&edge) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - } - } - } - - txn.commit() - .map_err(|e| RepositoryError::Database(e.to_string()))?; + let count = self.project_view_kg(&mut txn)?; + txn.commit()?; Ok(count) } - /// Enrich the KG with tracked file nodes. - /// - /// Creates a `file:{path}` node for each file tracked in the repository. pub fn kg_enrich_files(&self, working_copy: WorkingCopyId) -> Result { - let files = self.list_enrichable_files(working_copy)?; - - let mut txn = self - .pristine - .write_txn() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - let mut count = 0; - for file in &files { - let path = file.path.to_string_lossy(); - let label = path.rsplit('/').next().unwrap_or(&path); - - let node = KgNode::new(format!("file:{}", path), "file", label, "vcs"); - - txn.upsert_kg_node(&node) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - count += 1; - } - - txn.commit() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - Ok(count) + Ok(self.kg_enrich_from_vcs(working_copy)?.files) } - /// Enrich the KG with change data from the current view's history. - /// - /// For each change: - /// - Creates a `change:{short_hash}` node with message and timestamp - /// - Creates `AUTHORED_BY` edges to author identity nodes - /// - Creates `ON_VIEW` edge to the current view - /// - Creates `DEPENDS_ON` edges to dependency changes - /// - Creates `MODIFIES` edges to files (via `FileOps` in the change) pub fn kg_enrich_changes(&self, working_copy: WorkingCopyId) -> Result { - use crate::history::HistoryOptions; - - self.validate_working_copy(working_copy)?; - let view_name = self.desired_view_name(working_copy)?; - let history = self - .log(HistoryOptions::with_headers().view(&view_name)) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - if history.is_empty() { - return Ok(0); - } - - let ignore_rules = IgnoreRules::load_for_enrichment(&self.root); - - let mut txn = self - .pristine - .write_txn() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - let mut count = 0; - for entry in &history { - let hash_str = entry.hash.to_base32(); - let short_hash = truncate_hash(&hash_str); - - let mut node = KgNode::new( - format!("change:{}", short_hash), - "change", - short_hash, - "vcs", - ); - - // Add header info if available - if let Some(ref header) = entry.header { - node = node.with_summary(&header.message); - node = node.with_metadata(serde_json::json!({ - "sequence": entry.sequence, - "timestamp": header.timestamp.to_rfc3339(), - "full_hash": hash_str, - })); - - // Author edges - for author in &header.authors { - let author_id = format!("identity:{}", author.name); - // Ensure identity node exists - let identity_node = KgNode::new(&author_id, "identity", &author.name, "vcs"); - txn.upsert_kg_node(&identity_node) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - let edge = KgEdge::new( - format!("change:{}", short_hash), - &author_id, - edge_kind::AUTHORED_BY, - ); - txn.upsert_kg_edge(&edge) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - } - } else { - node = node.with_metadata(serde_json::json!({ - "sequence": entry.sequence, - "full_hash": hash_str, - })); - } - - txn.upsert_kg_node(&node) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - // ON_VIEW edge - let view_edge = KgEdge::new( - format!("change:{}", short_hash), - format!("view:{}", view_name), - edge_kind::ON_VIEW, - ); - txn.upsert_kg_edge(&view_edge) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - count += 1; - } - - // Dependencies — load each change and extract from hashed.dependencies - for entry in &history { - let hash_str = entry.hash.to_base32(); - let short_hash = truncate_hash(&hash_str); - - // Try to load the change to get dependency hashes directly - match self.load_change(&entry.hash) { - Ok(change) => { - for dep_hash in change.dependencies() { - let dep_hash_str = dep_hash.to_base32(); - let dep_short = truncate_hash(&dep_hash_str); - - let dep_edge = KgEdge::new( - format!("change:{}", short_hash), - format!("change:{}", dep_short), - edge_kind::DEPENDS_ON, - ); - txn.upsert_kg_edge(&dep_edge) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - } - - // File modification edges from FileOps - for file_op in change.file_ops() { - let path = file_op.path(); - if path.is_empty() { - continue; - } - // Skip ignored paths so build artifacts and - // dependencies recorded in a change do not reappear - // as `file:` nodes. - if is_path_ignored_for_enrichment(path, &ignore_rules) { - continue; - } - let file_label = path.rsplit('/').next().unwrap_or(path); - let file_node = - KgNode::new(format!("file:{}", path), "file", file_label, "vcs"); - txn.upsert_kg_node(&file_node) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - let mod_edge = KgEdge::new( - format!("change:{}", short_hash), - format!("file:{}", path), - edge_kind::MODIFIES, - ); - txn.upsert_kg_edge(&mod_edge) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - } - } - Err(e) => { - log::warn!( - "kg_enrich_changes: could not load change {}: {}", - short_hash, - e - ); - } - } - } - - txn.commit() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - Ok(count) + Ok(self.kg_enrich_from_vcs(working_copy)?.changes) } - /// Enrich the KG for a single newly-recorded change. - /// - /// Call this after `record()` to immediately add the new change - /// to the knowledge graph. Lighter than `kg_enrich_from_vcs()`. pub fn kg_enrich_change( &self, working_copy: WorkingCopyId, hash: &Hash, ) -> Result<(), RepositoryError> { self.validate_working_copy(working_copy)?; - let view_name = self.desired_view_name(working_copy)?; - let hash_str = hash.to_base32(); - let short_hash = truncate_hash(&hash_str); - + self.ensure_project_identity(MintReason::Upgrade)?; let change = self.load_change(hash)?; - let header = &change.hashed.header; - - let mut txn = self - .pristine - .write_txn() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - // Create change node - let node = KgNode::new( - format!("change:{}", short_hash), - "change", - short_hash, - "vcs", - ) - .with_summary(&header.message) - .with_metadata(serde_json::json!({ - "timestamp": header.timestamp.to_rfc3339(), - "full_hash": hash_str, - })); - - txn.upsert_kg_node(&node) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - // Author edges - for author in &header.authors { - let author_id = format!("identity:{}", author.name); - let identity_node = KgNode::new(&author_id, "identity", &author.name, "vcs"); - txn.upsert_kg_node(&identity_node) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - txn.upsert_kg_edge(&KgEdge::new( - format!("change:{}", short_hash), - &author_id, - edge_kind::AUTHORED_BY, - )) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - } - - // ON_VIEW edge to the validated working copy's desired view - txn.upsert_kg_edge(&KgEdge::new( - format!("change:{}", short_hash), - format!("view:{}", view_name), - edge_kind::ON_VIEW, - )) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - // Dependency edges - for dep_hash in change.dependencies() { - let dep_hash_str = dep_hash.to_base32(); - let dep_short = truncate_hash(&dep_hash_str); - - txn.upsert_kg_edge(&KgEdge::new( - format!("change:{}", short_hash), - format!("change:{}", dep_short), - edge_kind::DEPENDS_ON, - )) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - } - - // File modification edges from FileOps - let ignore_rules = IgnoreRules::load_for_enrichment(&self.root); - let mut file_paths = Vec::new(); - for file_op in change.file_ops() { - let path = file_op.path(); - if path.is_empty() { - continue; - } - // Skip ignored paths so build artifacts and dependencies recorded - // in the change do not reappear as `file:` nodes (or get parsed - // for AST entities below). - if is_path_ignored_for_enrichment(path, &ignore_rules) { - continue; - } - file_paths.push(path.to_string()); - let file_label = path.rsplit('/').next().unwrap_or(path); - let file_node = KgNode::new(format!("file:{}", path), "file", file_label, "vcs"); - txn.upsert_kg_node(&file_node) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - txn.upsert_kg_edge(&KgEdge::new( - format!("change:{}", short_hash), - format!("file:{}", path), - edge_kind::MODIFIES, - )) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - } - - // AST entity extraction for changed files - { - let mut parser_registry = atomic_semantic::ParserRegistry::new(); - - for path in &file_paths { - // Only parse supported language files - if !atomic_semantic::is_supported(path) { - continue; - } - - // Read the file content from the working copy - let abs_path = self.root().join(path); - let source = match std::fs::read_to_string(&abs_path) { - Ok(s) => s, - Err(e) => { - log::debug!("Can't read {} for AST extraction: {}", path, e); - continue; - } - }; - - // Extract entities - let entities = parser_registry.extract(path, &source); - - for entity in &entities { - let entity_id = - format!("entity:{}:{}:{}", entity.file, entity.name, entity.line); - - let mut node = KgNode::new(&entity_id, "entity", &entity.name, "ast") - .with_metadata(serde_json::json!({ - "kind": entity.kind.as_str(), - "file": entity.file, - "line": entity.line, - "end_line": entity.end_line, - "exported": entity.exported, - })); - - if let Some(ref sig) = entity.signature { - node = node.with_summary(sig); - } - - txn.upsert_kg_node(&node) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - // DEFINES edge: file → entity - txn.upsert_kg_edge(&KgEdge::new( - format!("file:{}", entity.file), - &entity_id, - edge_kind::DEFINES, - )) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - // EXPORTS edge: only for publicly visible entities - if entity.exported { - txn.upsert_kg_edge(&KgEdge::new( - format!("file:{}", entity.file), - &entity_id, - edge_kind::EXPORTS, - )) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - } - - // Change MODIFIES entity (more precise than file-level) - txn.upsert_kg_edge(&KgEdge::new( - format!("change:{}", short_hash), - &entity_id, - edge_kind::MODIFIES, - )) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - } - } + let mut txn = self.pristine.write_txn()?; + if !txn.kg_writes_enabled()? { + return Ok(()); } - - txn.commit() - .map_err(|e| RepositoryError::Database(e.to_string()))?; + self.project_change_kg(&mut txn, hash, &change)?; + self.project_view_kg(&mut txn)?; + txn.commit()?; Ok(()) } - /// Update the enrich database for a single recorded change. - /// - /// This is the incremental "keep the enrich database current" entry point - /// used after a change is recorded (by `record()` when `enrich_kg` is set, - /// and by `atomic query enrich --change `): - /// - /// 1. Adds the change's KG nodes and edges via - /// [`kg_enrich_change`](Self::kg_enrich_change). - /// 2. Incrementally refreshes the content search index for the files the - /// change touched (added/modified files re-indexed, deleted files - /// removed), skipping ignored and Atomic-internal paths. - /// - /// The content-index step is best-effort and a no-op when no content index - /// exists yet — building it is the job of `atomic query enrich`/`index`; - /// this only maintains an index that already exists. pub fn enrich_change( &self, working_copy: WorkingCopyId, hash: &Hash, ) -> Result<(), RepositoryError> { self.kg_enrich_change(working_copy, hash)?; - - // Collect the paths this change touched for an incremental content - // index refresh. Best-effort: a load failure just skips the content - // step (the KG is already updated above). - let paths: Vec = match self.load_change(hash) { - Ok(change) => change - .file_ops() - .iter() - .map(|op| op.path().to_string()) - .filter(|path| !path.is_empty()) - .collect(), - Err(e) => { - log::debug!( - "enrich_change: could not load change for content index: {}", - e - ); - Vec::new() - } - }; - + let paths: Vec = self + .load_change(hash)? + .file_ops() + .iter() + .map(|op| op.path().to_owned()) + .filter(|p| !p.is_empty()) + .collect(); if !paths.is_empty() { - if let Err(e) = crate::content_search::update_content_index_paths(self.root(), &paths) { - log::debug!("enrich_change: content index update failed: {}", e); + if let Err(error) = + crate::content_search::update_content_index_paths(self.root(), &paths) + { + log::debug!("content index refresh: {error}"); } } - Ok(()) } - /// Enrich the KG with AST entities from tracked files. - /// - /// Enrich the KG with `INCLUDES` edges derived from import entities. - /// - /// Re-parses tracked source files for import statements and attempts to - /// resolve each import path to an existing `file:{path}` node in the KG. - /// If a match is found, creates an `INCLUDES` edge from the source file - /// to the included file. Best-effort — unresolvable imports are skipped. - pub fn kg_enrich_includes( + pub(super) fn project_vcs_kg( &self, - working_copy: WorkingCopyId, - ) -> Result { - let files = self.list_enrichable_files(working_copy)?; - - // Build a lookup set of tracked file paths for resolution. - let tracked_paths: HashSet = files - .iter() - .filter(|f| !f.is_directory) - .map(|f| f.path.to_string_lossy().to_string()) - .collect(); - - // Also build a map from filename → full paths for fuzzy resolution. - let mut basename_to_paths: HashMap> = HashMap::new(); - for path in &tracked_paths { - if let Some(basename) = path.rsplit('/').next() { - basename_to_paths - .entry(basename.to_string()) - .or_default() - .push(path.clone()); - } - } - - let mut parser_registry = atomic_semantic::ParserRegistry::new(); - let mut count = 0; - - let mut txn = self - .pristine - .write_txn() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - for file in &files { - if file.is_directory { - continue; - } - - let path_str = file.path.to_string_lossy().to_string(); - - if !atomic_semantic::is_supported(&path_str) { + txn: &mut atomic_core::WriteTxn<'_>, + ) -> Result { + let mut stats = KgEnrichStats::default(); + for hash in self.change_store.iter_changes() { + let hash = hash.map_err(|e| RepositoryError::Database(e.to_string()))?; + let change = self.load_change(&hash)?; + if change.kind().is_snapshot() { continue; } - - let abs_path = self.root().join(&file.path); - let source = match std::fs::read_to_string(&abs_path) { - Ok(s) => s, - Err(_) => continue, - }; - - let entities = parser_registry.extract(&path_str, &source); - - for entity in &entities { - if entity.kind != atomic_semantic::EntityKind::Import { - continue; - } - - // Try to resolve the import name to a tracked file. - let import_name = &entity.name; - - // Strategy 1: treat as a relative path from the source file's directory. - let resolved = if let Some((src_dir, _)) = path_str.rsplit_once('/') { - let candidate = format!("{}/{}", src_dir, import_name); - if tracked_paths.contains(&candidate) { - Some(candidate) - } else { - None - } - } else { - None - }; - - // Strategy 2: treat as an absolute project path. - let resolved = resolved.or_else(|| { - if tracked_paths.contains(import_name) { - Some(import_name.clone()) - } else { - None - } - }); - - // Strategy 3: match by basename (e.g., "header.h" → "src/header.h"). - let resolved = resolved.or_else(|| { - let basename = import_name.rsplit('/').next().unwrap_or(import_name); - if let Some(candidates) = basename_to_paths.get(basename) { - if candidates.len() == 1 { - Some(candidates[0].clone()) - } else { - // Ambiguous — try to pick the candidate whose path ends with the import. - candidates - .iter() - .find(|c| c.ends_with(import_name)) - .cloned() - } - } else { - None - } - }); - - if let Some(target_path) = resolved { - if target_path == path_str { - continue; // skip self-includes - } - let edge = KgEdge::new( - format!("file:{}", path_str), - format!("file:{}", target_path), - edge_kind::INCLUDES, - ); - txn.upsert_kg_edge(&edge) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - count += 1; - } - } + stats.files += self.project_change_kg(txn, &hash, &change)?; + stats.changes += 1; } - - txn.commit() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - Ok(count) + stats.views = self.project_view_kg(txn)?; + Ok(stats) } - /// Parses all tracked source files and writes `CALLS` edges between entity nodes. - /// - /// For every tracked file, this function: - /// 1. Extracts function/method entities (to build a name → entity-ID index). - /// 2. Extracts call sites via tree-sitter reference extraction. - /// 3. For each call site, finds the innermost enclosing function (the caller entity) - /// and resolves the callee name to known entity nodes in the graph. - /// 4. Writes `KgEdge { from: caller_entity, to: callee_entity, kind: CALLS }`. - /// - /// Callee resolution is name-based: a call to `foo()` matches any entity named `foo` - /// across all tracked files. Calls to external crates / stdlib are silently skipped. - pub fn kg_enrich_calls(&self, working_copy: WorkingCopyId) -> Result { - let files = self.list_enrichable_files(working_copy)?; - - let mut parser_registry = atomic_semantic::ParserRegistry::new(); - - // ── Pass 1: Parse every file to build the name → entity-ID index. - struct FileData { - path: String, - source: String, - entities: Vec, + pub(super) fn project_change_kg( + &self, + txn: &mut atomic_core::WriteTxn<'_>, + hash: &Hash, + change: &Change, + ) -> Result { + let project = txn.kg_project()?; + let id = iri::mint_change(hash); + let header = &change.hashed.header; + txn.upsert_kg_node(&KgNode::new(&id, C::Change, &hash.to_base32()[..12], "vcs") + .with_summary(&header.message).with_metadata(serde_json::json!({"hash": hash.to_base32(), "created_at": header.timestamp.to_rfc3339()})))?; + for author in &header.authors { + let author_id = iri::mint_identity(&project, &author.name); + txn.upsert_kg_node(&KgNode::new(&author_id, C::Identity, &author.name, "vcs"))?; + txn.upsert_kg_edge(&KgEdge::new(&id, author_id, P::AuthoredBy))?; } - - let mut all_file_data: Vec = Vec::new(); - // function/method name → list of entity IDs (names can appear in multiple files) - let mut name_to_entity_ids: HashMap> = HashMap::new(); - - for file in &files { - if file.is_directory { - continue; - } - let path_str = file.path.to_string_lossy().to_string(); - if !atomic_semantic::is_supported(&path_str) { - continue; - } - let abs_path = self.root().join(&file.path); - let source = match std::fs::read_to_string(&abs_path) { - Ok(s) => s, - Err(_) => continue, - }; - - let entities = parser_registry.extract(&path_str, &source); - - for entity in &entities { - match entity.kind { - atomic_semantic::EntityKind::Function | atomic_semantic::EntityKind::Method => { - let entity_id = - format!("entity:{}:{}:{}", entity.file, entity.name, entity.line); - name_to_entity_ids - .entry(entity.name.clone()) - .or_default() - .push(entity_id); - } - _ => {} + for hash in change.dependencies() { + if self.has_change(hash) { + let target = iri::mint_change(hash); + if txn.get_kg_node(&target)?.is_none() { + txn.upsert_kg_node(&KgNode::new( + &target, + C::Change, + &hash.to_base32()[..12], + "vcs", + ))?; } + txn.upsert_kg_edge(&KgEdge::new(&id, target, P::DirectlyDependsOnChange))?; + } else { + txn.kg_finding( + "UnresolvedReference", + &id, + "A change dependency is not stored locally", + )?; } - - all_file_data.push(FileData { - path: path_str, - source, - entities, - }); } - - // ── Pass 2: Extract call sites and write CALLS edges. - let mut txn = self - .pristine - .write_txn() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - let mut count = 0; - - for file_data in &all_file_data { - let refs = parser_registry.extract_references(&file_data.path, &file_data.source); - if refs.is_empty() { + let rules = IgnoreRules::load_for_enrichment(self.root()); + let mut paths = std::collections::BTreeSet::new(); + for file in change.file_ops() { + let path = file.path(); + if path.is_empty() || rules.is_ignored(std::path::Path::new(path), false) { continue; } - - // Build a (start_line, end_line, entity_id) list for this file's callable - // entities so we can find which function encloses each call site by line. - let mut callable_entities: Vec<(u32, u32, String)> = file_data - .entities - .iter() - .filter(|e| { - matches!( - e.kind, - atomic_semantic::EntityKind::Function | atomic_semantic::EntityKind::Method - ) - }) - .map(|e| { - ( - e.line, - e.end_line, - format!("entity:{}:{}:{}", e.file, e.name, e.line), - ) - }) - .collect(); - callable_entities.sort_by_key(|(start, _, _)| *start); - - for r in &refs { - // Find the innermost (smallest range) enclosing function. - let caller_id = callable_entities - .iter() - .filter(|(start, end, _)| r.line >= *start && r.line <= *end) - .min_by_key(|(start, end, _)| end - start) - .map(|(_, _, id)| id.as_str()); - - let caller_id = match caller_id { - Some(id) => id, - None => continue, // call outside any function (top-level, macro, etc.) - }; - - // Skip names that resolve to too many definitions — they are almost - // certainly common trait-method names (new, clone, fmt, from, into, - // push, len, …) where name-only matching produces pure noise. - // A call to `Vec::new()` should not write edges to every `fn new()` - // across all 600+ files. - const MAX_CALLEE_FANOUT: usize = 10; - - let callee_ids = match name_to_entity_ids.get(&r.symbol) { - Some(ids) if ids.len() <= MAX_CALLEE_FANOUT => ids, - Some(_) => continue, // too ambiguous — common trait method, skip - None => continue, // callee not in repo (stdlib / external dep) - }; - - for callee_id in callee_ids { - if callee_id == caller_id { - continue; // skip direct recursion - } - txn.upsert_kg_edge(&KgEdge::new(caller_id, callee_id, edge_kind::CALLS)) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - count += 1; - } - } + paths.insert(path); + let file_id = iri::mint_file(&project, path); + txn.upsert_kg_node( + &KgNode::new( + &file_id, + C::File, + path.rsplit('/').next().unwrap_or(path), + "vcs", + ) + .with_metadata(serde_json::json!({"path": path})), + )?; + txn.upsert_kg_edge(&KgEdge::new(&id, file_id, P::ModifiesFile))?; } - - txn.commit() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - Ok(count) + Ok(paths.len()) } - /// Parses all supported source files in the working copy and creates - /// `entity:{file}:{name}:{line}` nodes with `DEFINES` edges from - /// the corresponding file nodes. Runs during bulk enrichment - /// (`atomic vault query enrich`, `atomic git import`). - pub fn kg_enrich_entities( + pub(super) fn project_view_kg( &self, - working_copy: WorkingCopyId, + txn: &mut atomic_core::WriteTxn<'_>, ) -> Result { - let files = self.list_enrichable_files(working_copy)?; - - let mut parser_registry = atomic_semantic::ParserRegistry::new(); - let mut count = 0; - - let mut txn = self - .pristine - .write_txn() + let project = txn.kg_project()?; + let record = txn + .project_identity()? + .ok_or(atomic_core::PristineError::MissingProjectIdentity)?; + let project_iri = iri::mint_project(&project); + let identity = iri::mint_project_identity(&record.hash); + txn.upsert_kg_node(&KgNode::new( + &project_iri, + C::Project, + project.to_base32(), + "vcs", + ))?; + txn.upsert_kg_node(&KgNode::new(&identity, C::ProjectIdentity, project.to_base32(), "vcs") + .with_metadata(serde_json::json!({"created_at": record.created_at, "project_id": project.to_base32(), "schema_version": 1})))?; + txn.upsert_kg_edge(&KgEdge::new(&project_iri, &identity, P::HasIdentity))?; + let object: serde_json::Value = serde_json::from_slice(&record.object) .map_err(|e| RepositoryError::Database(e.to_string()))?; - - for file in &files { - if file.is_directory { - continue; + if let Some(issuer) = object + .get("atomic:issuer") + .and_then(|v| v.get("@id")) + .and_then(|v| v.as_str()) + { + txn.upsert_kg_node(&KgNode::new(issuer, C::Identity, issuer, "vcs"))?; + txn.upsert_kg_edge(&KgEdge::new(&identity, issuer, P::Issuer))?; + } + let views = txn + .list_views()? + .iter() + .map(|name| txn.get_view(name)) + .collect::, _>>()? + .into_iter() + .flatten() + .collect::>(); + let mut live = std::collections::HashSet::new(); + for view in &views { + let id = iri::mint_view(&project, &view.name); + live.insert(id.clone()); + txn.upsert_kg_node(&KgNode::new(&id, C::View, &view.name, "vcs").with_metadata( + serde_json::json!({"name": view.name, "scope": format!("{:?}", view.kind)}), + ))?; + } + for edge in txn.all_kg_edges()? { + if matches!(edge.kind, P::DirectlyOnView | P::ChildViewOf) { + txn.del_kg_edge(&edge.from_id, &edge.to_id, &edge.kind)?; } - - let path_str = file.path.to_string_lossy().to_string(); - - if !atomic_semantic::is_supported(&path_str) { - continue; + } + for node in txn.all_kg_nodes()? { + if node.has_type(C::View) && !live.contains(&node.id) { + txn.del_kg_node(&node.id)?; } - - let abs_path = self.root().join(&file.path); - let source = match std::fs::read_to_string(&abs_path) { - Ok(s) => s, - Err(_) => continue, - }; - - let entities = parser_registry.extract(&path_str, &source); - - for entity in &entities { - let entity_id = format!("entity:{}:{}:{}", entity.file, entity.name, entity.line); - - let mut node = KgNode::new(&entity_id, "entity", &entity.name, "ast") - .with_metadata(serde_json::json!({ - "kind": entity.kind.as_str(), - "file": entity.file, - "line": entity.line, - "end_line": entity.end_line, - "exported": entity.exported, - })); - - if let Some(ref sig) = entity.signature { - node = node.with_summary(sig); - } - - txn.upsert_kg_node(&node) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - + } + for view in &views { + let id = iri::mint_view(&project, &view.name); + if let Some(parent) = view + .parent + .and_then(|parent| views.iter().find(|view| view.id == parent)) + { txn.upsert_kg_edge(&KgEdge::new( - format!("file:{}", entity.file), - &entity_id, - edge_kind::DEFINES, - )) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - // EXPORTS edge: only for publicly visible entities - if entity.exported { - txn.upsert_kg_edge(&KgEdge::new( - format!("file:{}", entity.file), - &entity_id, - edge_kind::EXPORTS, - )) - .map_err(|e| RepositoryError::Database(e.to_string()))?; + &id, + iri::mint_view(&project, &parent.name), + P::ChildViewOf, + ))?; + } + let memberships = txn.iter_changes(view, 0)?.collect::, _>>()?; + for (_, change, _) in memberships { + if let Some(hash) = txn.get_external(change)? { + let change = iri::mint_change(&hash); + if txn.get_kg_node(&change)?.is_none() { + txn.upsert_kg_node(&KgNode::new( + &change, + C::Change, + &hash.to_base32()[..12], + "vcs", + ))?; + } + txn.upsert_kg_edge(&KgEdge::new(change, &id, P::DirectlyOnView))?; } - - count += 1; } } - - txn.commit() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - Ok(count) - } -} - -/// Statistics from a VCS enrichment run. -#[derive(Debug, Clone, Default)] -pub struct KgEnrichStats { - /// Number of view nodes created. - pub views: usize, - /// Number of file nodes created. - pub files: usize, - /// Number of module nodes created. - pub modules: usize, - /// Number of change nodes created. - pub changes: usize, - /// Number of AST entity nodes created. - pub entities: usize, - /// Number of INCLUDES edges created. - pub includes: usize, - /// Number of CALLS edges created. - pub calls: usize, -} - -impl KgEnrichStats { - /// Total number of nodes created across all phases. - pub fn total(&self) -> usize { - self.views - + self.files - + self.modules - + self.changes - + self.entities - + self.includes - + self.calls - } -} - -impl std::fmt::Display for KgEnrichStats { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - write!( - f, - "{} views, {} files, {} modules, {} changes, {} entities, {} includes, {} calls", - self.views, - self.files, - self.modules, - self.changes, - self.entities, - self.includes, - self.calls - ) - } -} - -/// Truncate a base32 hash string to a short prefix for use as an identifier. -fn truncate_hash(hash_str: &str) -> &str { - if hash_str.len() > 12 { - &hash_str[..12] - } else { - hash_str - } -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn test_kg_enrich_stats_display() { - let stats = KgEnrichStats { - views: 3, - files: 10, - modules: 2, - changes: 5, - entities: 8, - includes: 4, - calls: 0, - }; - assert_eq!( - stats.to_string(), - "3 views, 10 files, 2 modules, 5 changes, 8 entities, 4 includes, 0 calls" - ); - } - - #[test] - fn test_kg_enrich_stats_total() { - let stats = KgEnrichStats { - views: 2, - files: 7, - modules: 3, - changes: 3, - entities: 4, - includes: 1, - calls: 5, - }; - assert_eq!(stats.total(), 25); - } - - #[test] - fn test_kg_enrich_stats_default() { - let stats = KgEnrichStats::default(); - assert_eq!(stats.views, 0); - assert_eq!(stats.files, 0); - assert_eq!(stats.modules, 0); - assert_eq!(stats.changes, 0); - assert_eq!(stats.entities, 0); - assert_eq!(stats.includes, 0); - assert_eq!(stats.calls, 0); - assert_eq!(stats.total(), 0); - assert_eq!( - stats.to_string(), - "0 views, 0 files, 0 modules, 0 changes, 0 entities, 0 includes, 0 calls" - ); - } - - #[test] - fn test_truncate_hash() { - assert_eq!(truncate_hash("abcdefghijklmnop"), "abcdefghijkl"); - assert_eq!(truncate_hash("short"), "short"); - assert_eq!(truncate_hash("exactly12chr"), "exactly12chr"); - assert_eq!(truncate_hash(""), ""); - } - - #[test] - fn test_kg_enrich_from_vcs_empty_repo() { - let dir = tempfile::tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_kg().unwrap(); - - let stats = repo - .kg_enrich_from_vcs(repo.require_working_copy_id().unwrap()) - .unwrap(); - // Should have at least the default view (e.g. "dev") - assert!( - stats.views >= 1, - "expected at least 1 view, got {}", - stats.views - ); - assert_eq!(stats.files, 0); - assert_eq!(stats.changes, 0); - } - - #[test] - fn test_kg_enrich_views_creates_nodes() { - let dir = tempfile::tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_kg().unwrap(); - - let count = repo.kg_enrich_views().unwrap(); - assert!(count >= 1); - - // Verify the default view node exists - let view_name = repo.current_view().to_string(); - let node = repo.vault_kg_node(&format!("view:{}", view_name)).unwrap(); - assert!(node.is_some(), "expected view node to exist"); - - let node = node.unwrap(); - assert_eq!(node.kind, "view"); - assert_eq!(node.source, "vcs"); - } - - #[test] - fn test_kg_clear_derived_removes_vcs_and_ast_but_keeps_vault() { - use atomic_core::pristine::vault::KgNode; - use atomic_core::pristine::KgMutTxnT; - - let dir = tempfile::tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_kg().unwrap(); - - { - let mut txn = repo.pristine.write_txn().unwrap(); - txn.init_kg().unwrap(); - txn.upsert_kg_node(&KgNode::new("file:src/a.ts", "file", "a.ts", "vcs")) - .unwrap(); - txn.upsert_kg_node(&KgNode::new( - "entity:src/a.ts:foo:1", - "entity", - "foo", - "ast", - )) - .unwrap(); - txn.upsert_kg_node(&KgNode::new("memory:keep", "memory", "keep", "vault")) - .unwrap(); - txn.commit().unwrap(); - } - - let removed = repo.kg_clear_derived().unwrap(); - assert_eq!(removed, 2, "both the vcs and ast nodes should be removed"); - - assert!(repo.vault_kg_node("file:src/a.ts").unwrap().is_none()); - assert!(repo - .vault_kg_node("entity:src/a.ts:foo:1") - .unwrap() - .is_none()); - assert!( - repo.vault_kg_node("memory:keep").unwrap().is_some(), - "vault-sourced nodes must survive a derived-node clear" - ); - } - - #[test] - fn test_kg_enrich_files_skips_ignored_paths() { - use crate::tracking::TrackingOptions; - - let dir = tempfile::tempdir().unwrap(); - let root = dir.path(); - - // Source file (should be enriched) plus build artifacts, a dependency - // directory, and a `.vault` entry that must all be excluded. - std::fs::create_dir_all(root.join("src")).unwrap(); - std::fs::create_dir_all(root.join("node_modules/typescript")).unwrap(); - std::fs::create_dir_all(root.join("dist")).unwrap(); - std::fs::create_dir_all(root.join(".vault/intents/x")).unwrap(); - std::fs::write(root.join("src/index.ts"), b"export const x = 1;\n").unwrap(); - std::fs::write( - root.join("node_modules/typescript/lib.d.ts"), - b"declare var y;\n", - ) - .unwrap(); - std::fs::write(root.join("dist/index.js"), b"var x = 1;\n").unwrap(); - std::fs::write(root.join(".vault/intents/x/intent.md"), b"# intent\n").unwrap(); - - // Ignore files: node_modules via .gitignore, dist via .atomicignore. - std::fs::write(root.join(".gitignore"), b"node_modules/\n").unwrap(); - std::fs::write(root.join(".atomicignore"), b"dist/\n").unwrap(); - - let repo = Repository::init(root).unwrap(); - repo.init_kg().unwrap(); - - // Track everything on disk. `add` itself only consults `.atomicignore`, - // so node_modules still lands in the tree — enrichment must be the - // layer that filters it out. - repo.add( - repo.require_working_copy_id().unwrap(), - ".", - TrackingOptions::default(), - ) - .unwrap(); - - repo.kg_enrich_files(repo.require_working_copy_id().unwrap()) - .unwrap(); - - assert!( - repo.vault_kg_node("file:src/index.ts").unwrap().is_some(), - "source file should be enriched" - ); - assert!( - repo.vault_kg_node("file:node_modules/typescript/lib.d.ts") - .unwrap() - .is_none(), - ".gitignore'd dependency file should not be enriched" - ); - assert!( - repo.vault_kg_node("file:dist/index.js").unwrap().is_none(), - ".atomicignore'd build artifact should not be enriched" - ); - assert!( - repo.vault_kg_node("file:.vault/intents/x/intent.md") - .unwrap() - .is_none(), - ".vault entries must not become file nodes (they have semantic nodes)" - ); - } - - #[test] - fn test_kg_enrich_changes_skips_ignored_paths() { - use crate::record::RecordOptions; - use crate::tracking::TrackingOptions; - use atomic_core::change::ChangeHeader; - - let dir = tempfile::tempdir().unwrap(); - let root = dir.path(); - - std::fs::create_dir_all(root.join("src")).unwrap(); - std::fs::create_dir_all(root.join("node_modules")).unwrap(); - std::fs::write(root.join("src/index.ts"), b"export const x = 1;\n").unwrap(); - std::fs::write(root.join("node_modules/dep.js"), b"var d = 1;\n").unwrap(); - std::fs::write(root.join(".gitignore"), b"node_modules/\n").unwrap(); - - let repo = Repository::init(root).unwrap(); - repo.init_kg().unwrap(); - let working_copy = repo.require_working_copy_id().unwrap(); - repo.add(working_copy, ".", TrackingOptions::default()) - .unwrap(); - repo.record( - working_copy, - ChangeHeader::builder().message("init").build(), - RecordOptions::new().enrich_kg(false), - ) - .unwrap(); - - // Change enrichment builds `file:` nodes from the change's file_ops. - // Ignored paths recorded in the change must still be filtered out. - repo.kg_enrich_changes(working_copy).unwrap(); - - assert!( - repo.vault_kg_node("file:src/index.ts").unwrap().is_some(), - "source file modified by the change should be enriched" - ); - assert!( - repo.vault_kg_node("file:node_modules/dep.js") - .unwrap() - .is_none(), - ".gitignore'd file recorded in a change must not reappear as a file node" - ); - } - - #[test] - fn test_record_updates_content_index_incrementally() { - use crate::record::RecordOptions; - use crate::tracking::TrackingOptions; - use atomic_core::change::ChangeHeader; - - let dir = tempfile::tempdir().unwrap(); - let root = dir.path(); - std::fs::create_dir_all(root.join("src")).unwrap(); - std::fs::write(root.join("src/a.ts"), b"const alpha = 1;\n").unwrap(); - - let repo = Repository::init(root).unwrap(); - repo.init_kg().unwrap(); - let working_copy = repo.require_working_copy_id().unwrap(); - repo.add(working_copy, ".", TrackingOptions::default()) - .unwrap(); - repo.record( - working_copy, - ChangeHeader::builder().message("init").build(), - RecordOptions::new().with_all(true), - ) - .unwrap(); - - // Build the content index over the current working copy (indexes a.ts). - crate::content_search::build_content_index(root).unwrap(); - - // A newly recorded file must become searchable WITHOUT an explicit - // rebuild: record() (enrich_kg defaults on) refreshes the index for the - // change's paths. - std::fs::write(root.join("src/b.ts"), b"const betamarker = 2;\n").unwrap(); - repo.add(working_copy, ".", TrackingOptions::default()) - .unwrap(); - repo.record( - working_copy, - ChangeHeader::builder().message("add b").build(), - RecordOptions::new().with_all(true), - ) - .unwrap(); - - let result = crate::content_search::search_content( - root, - "betamarker", - crate::content_search::ContentSearchOptions::default(), - ) - .unwrap(); - let paths: Vec<&str> = result.matches.iter().map(|m| m.path.as_str()).collect(); - assert!( - paths.contains(&"src/b.ts"), - "newly recorded file should be searchable without a manual rebuild, got: {paths:?}" - ); + Ok(views.len()) } } diff --git a/atomic-repository/src/repository/vault_projection.rs b/atomic-repository/src/repository/vault_projection.rs new file mode 100644 index 00000000..d2ca7218 --- /dev/null +++ b/atomic-repository/src/repository/vault_projection.rs @@ -0,0 +1,622 @@ +//! Lift canonical vault data into workflow facts without rewriting signed objects. + +use super::*; +use atomic_core::pristine::{VaultEntry, VaultEntryType, VaultTxnT}; +use atomic_core::{ + kg::{ + iri::{self, ChildKind}, + ontology::{Class as C, Predicate as P}, + project_id::ProjectId, + }, + KgEdge, KgNode, KgSubgraph, +}; +use std::collections::BTreeMap; + +pub(super) struct VaultProjection { + project: ProjectId, + subjects: BTreeMap, + paths: BTreeMap, + hashes: BTreeMap, +} + +fn stem(path: &str, prefix: &str) -> String { + path.strip_prefix(prefix) + .unwrap_or(path) + .strip_suffix(".md") + .unwrap_or(path.strip_prefix(prefix).unwrap_or(path)) + .to_owned() +} + +impl VaultProjection { + pub(super) fn new( + txn: &impl VaultTxnT, + project: ProjectId, + hashes: Vec, + ) -> Result { + let hashes = hashes + .into_iter() + .map(|hash| (hash.to_base32(), hash)) + .collect(); + let mut context = Self { + project, + subjects: BTreeMap::new(), + paths: BTreeMap::new(), + hashes, + }; + let manifest = txn.get_vault_manifest()?; + for meta in txn.list_vault_entries("", None)? { + let Some(entry) = txn.get_vault_entry(&meta.path)? else { + continue; + }; + let fm: serde_json::Value = + serde_json::from_str(&entry.frontmatter_json).unwrap_or_default(); + let (subject, class, local_kind, name) = match entry.entry_type { + VaultEntryType::Intent => { + let uid = fm + .get("uid") + .and_then(|v| v.as_str()) + .map(str::to_uppercase) + .or_else(|| { + manifest + .intents + .values() + .find(|summary| { + summary.vault_path == meta.path && !summary.uid.is_empty() + }) + .map(|s| s.uid.to_uppercase()) + }); + let Some(uid) = uid else { + continue; + }; + let Ok(id) = iri::mint_intent(&project, &uid) else { + continue; + }; + (id, C::WorkIntent, "intent", uid) + } + VaultEntryType::Memory => { + let name = stem(&meta.path, "memory/"); + (iri::mint_memory(&project, &name), C::Memory, "memory", name) + } + VaultEntryType::Skill => { + let name = stem(&meta.path, "skills/"); + (iri::mint_skill(&project, &name), C::Skill, "skill", name) + } + VaultEntryType::Session => { + let name = meta + .path + .strip_prefix("goals/") + .and_then(|s| s.strip_suffix("/_goal.md")) + .unwrap_or(&meta.path) + .to_owned(); + (iri::mint_goal(&project, &name), C::Goal, "goal", name) + } + VaultEntryType::ToolResult => ( + iri::mint_tool_result(&project, &meta.path), + C::ToolResult, + "tool-result", + meta.path.clone(), + ), + _ => continue, + }; + context.paths.insert(meta.path.clone(), subject.clone()); + context + .subjects + .insert(subject.to_lowercase(), (subject.clone(), class)); + context.subjects.insert( + format!("urn:atomic:{local_kind}:{name}").to_lowercase(), + (subject.clone(), class), + ); + if entry.entry_type == VaultEntryType::Intent { + for (key, summary) in &manifest.intents { + if summary.vault_path == meta.path { + context.subjects.insert( + format!("urn:atomic:intent:{key}").to_lowercase(), + (subject.clone(), class), + ); + } + } + if let Some(id) = fm.get("id").and_then(|v| v.as_str()) { + context.subjects.insert( + format!("urn:atomic:intent:{id}").to_lowercase(), + (subject.clone(), class), + ); + } + if let Some(map) = fm.as_object() { + if let Ok(node) = atomic_canonical::lift::lift_intent( + map, + &String::from_utf8_lossy(&entry.content_bytes), + ) { + let mut children = Vec::new(); + children.extend( + node.has_task + .iter() + .map(|c| (&c.id, ChildKind::Task, C::Task)), + ); + children.extend(node.has_acceptance_criterion.iter().map(|c| { + ( + &c.id, + ChildKind::AcceptanceCriterion, + C::AcceptanceCriterion, + ) + })); + children.extend( + node.has_scope_in + .iter() + .chain(&node.has_scope_out) + .map(|c| (&c.id, ChildKind::Scope, C::ScopeItem)), + ); + children.extend( + node.has_constraint + .iter() + .map(|c| (&c.id, ChildKind::Constraint, C::Constraint)), + ); + for (raw, kind, class) in children { + let local = raw.rsplit(':').next().unwrap_or(raw); + let id = + iri::mint_child(&project, &name, kind, local).map_err(|e| { + RepositoryError::InvalidOperation { + message: e.to_string(), + } + })?; + context + .subjects + .insert(raw.to_lowercase(), (id.clone(), class)); + context.subjects.insert(id.to_lowercase(), (id, class)); + } + } + } + } + } + Ok(context) + } + + fn resolve(&self, raw: &str) -> Option<(String, C)> { + if raw.starts_with("memory:") || raw.starts_with("intent:") { + return self.resolve(&format!("urn:atomic:{raw}")); + } + if let Some(target) = self.subjects.get(&raw.to_lowercase()) { + return Some(target.clone()); + } + if let Ok(hash) = iri::parse_change(raw) { + return self + .hashes + .contains_key(&hash.to_base32()) + .then(|| (iri::mint_change(&hash), C::Change)); + } + if let Some(prefix) = raw.strip_prefix("urn:atomic:change:") { + let prefix = prefix.to_uppercase(); + if prefix.is_empty() { + return None; + } + let mut matches = self + .hashes + .range(prefix.clone()..) + .take_while(|(key, _)| key.starts_with(&prefix)); + let (_, hash) = matches.next()?; + return matches + .next() + .is_none() + .then(|| (iri::mint_change(hash), C::Change)); + } + None + } + + fn finding( + &self, + graph: &mut KgSubgraph, + subject: &str, + code: &str, + message: &str, + path: &str, + ) { + let id = iri::mint_finding(&self.project, code, subject); + graph.nodes.push( + KgNode::new(&id, C::Finding, code, "vault") + .with_summary(message) + .with_metadata( + serde_json::json!({"finding_code": code, "derived_from_vault_path": path}), + ), + ); + graph.edges.push( + KgEdge::new(id, subject, P::AboutResource) + .with_metadata(serde_json::json!({"derived_from_vault_path": path})), + ); + } + + fn link(&self, graph: &mut KgSubgraph, subject: &str, raw: &str, predicate: P, path: &str) { + if let Some((target, class)) = self.resolve(raw) { + graph + .nodes + .push(KgNode::new(&target, class, raw, "vault_reference")); + graph.edges.push( + KgEdge::new(subject, target, predicate) + .with_metadata(serde_json::json!({"rdf_target": raw})), + ); + } else { + self.finding( + graph, + subject, + if raw.starts_with("urn:atomic:todo:") { + "UnresolvedLegacyTodo" + } else { + "UnresolvedReference" + }, + &format!("Unresolved reference: {raw}"), + path, + ); + } + } + + pub(super) fn extract( + &self, + path: &str, + entry: &VaultEntry, + ) -> Result { + if entry.entry_type == VaultEntryType::Attestation { + return self.extract_signed_document(path, entry); + } + let mut graph = KgSubgraph::default(); + let Some(subject) = self.paths.get(path) else { + if entry.entry_type == VaultEntryType::Intent { + let project = iri::mint_project(&self.project); + graph.nodes.push(KgNode::new( + &project, + C::Project, + self.project.to_base32(), + "vcs", + )); + self.finding( + &mut graph, + &project, + "MissingIntentId", + &format!("Intent has no canonical ULID: {path}"), + path, + ); + } + return Ok(graph); + }; + let fm: serde_json::Value = + serde_json::from_str(&entry.frontmatter_json).unwrap_or_default(); + let class = self + .subjects + .get(&subject.to_lowercase()) + .expect("indexed subject") + .1; + let label = fm + .get("title") + .or_else(|| fm.get("name")) + .and_then(|v| v.as_str()) + .unwrap_or(path); + let mut node = KgNode::new(subject, class, label, "vault") + .with_metadata(serde_json::json!({"vault_path": path, "created_at": entry.created_at, "updated_at": entry.updated_at, + "status": fm.get("status"), "priority": fm.get("priority"), "human_key": fm.get("id"), "uid": fm.get("uid")})); + if let Some(description) = fm + .get("description") + .or_else(|| fm.get("title")) + .and_then(|v| v.as_str()) + { + node = node.with_summary(description); + } + if class == C::WorkIntent { + match fm.get("kind").and_then(|v| v.as_str()).unwrap_or("feature") { + "feature" => node = node.with_type(C::FeatureIntent), + "bug" => node = node.with_type(C::BugIntent), + "review" => node = node.with_type(C::ReviewIntent), + _ => {} + } + if matches!( + fm.get("status").and_then(|v| v.as_str()), + Some("in_progress" | "needs-review" | "done") + ) { + node = node.with_type(C::ManagedIntent); + } + } + for field in [ + "attributedTo", + "author", + "created_by", + "developer", + "creator", + ] { + if let Some(author) = fm + .get(field) + .and_then(|v| v.as_str()) + .filter(|s| !s.is_empty()) + { + let id = iri::mint_identity(&self.project, author); + graph + .nodes + .push(KgNode::new(&id, C::Identity, author, "vault")); + graph.edges.push(KgEdge::new(subject, id, P::AuthoredBy)); + break; + } + } + if let Some(assignee) = fm.get("assignee").and_then(|v| v.as_str()) { + let id = iri::mint_identity(&self.project, assignee); + graph + .nodes + .push(KgNode::new(&id, C::Identity, assignee, "vault")); + graph.edges.push(KgEdge::new(subject, id, P::AssignedTo)); + } + for label in links(&fm, "labels") { + let id = iri::mint_concept(&self.project, label); + graph + .nodes + .push(KgNode::new(&id, C::Concept, label, "vault")); + graph.edges.push(KgEdge::new(subject, id, P::HasLabel)); + } + for (field, kind, predicate) in [ + ("informedBy", "", P::InformedBy), + ("derivedFrom", "", P::ProvWasDerivedFrom), + ("blocked_by", "intent", P::BlockedBy), + ("goals", "goal", P::LinkedGoal), + ("intent", "intent", P::LinkedIntent), + ] { + for raw in links(&fm, field) { + let target = if kind.is_empty() || raw.starts_with("urn:") { + raw.to_owned() + } else { + format!("urn:atomic:{kind}:{raw}") + }; + self.link(&mut graph, subject, &target, predicate, path); + if predicate == P::BlockedBy { + if let Some((target, _)) = self.resolve(&target) { + graph.edges.push(KgEdge::new(target, subject, P::Blocks)); + } + } + } + } + if entry.entry_type == VaultEntryType::ToolResult { + if let Some(goal) = path + .strip_prefix("goals/") + .and_then(|p| p.split_once('/')) + .map(|p| p.0) + { + self.link( + &mut graph, + subject, + &format!("urn:atomic:goal:{goal}"), + P::LinkedGoal, + path, + ); + } + } + let content = String::from_utf8_lossy(&entry.content_bytes); + if class == C::WorkIntent { + if let Some(map) = fm.as_object() { + if let Ok(intent) = atomic_canonical::lift::lift_intent(map, &content) { + for ac in &intent.has_acceptance_criterion { + if let Some((id, class)) = self.resolve(&ac.id) { + graph.nodes.push(KgNode::new(&id, class, &ac.text, "vault").with_summary(&ac.text) + .with_metadata(serde_json::json!({"status": ac.ac_status, "verifiedBy": ac.verified_by, "evidence": ac.evidence}))); + graph + .edges + .push(KgEdge::new(subject, id, P::HasAcceptanceCriterion)); + } + } + for task in &intent.has_task { + if let Some((id, class)) = self.resolve(&task.id) { + graph.nodes.push( + KgNode::new(&id, class, &task.text, "vault") + .with_summary(&task.text) + .with_metadata(serde_json::json!({"status": task.task_status})), + ); + graph.edges.push(KgEdge::new(subject, &id, P::HasTask)); + for ac in task.satisfies.as_slice() { + self.link(&mut graph, &id, ac, P::Satisfies, path); + } + for file in &task.touches_file { + self.file_link(&mut graph, &id, file, P::Touches); + } + } + } + for (items, predicate) in [ + (&intent.has_scope_in, P::HasScopeIn), + (&intent.has_scope_out, P::HasScopeOut), + ] { + for item in items { + if let Some((id, class)) = self.resolve(&item.id) { + graph.nodes.push( + KgNode::new(&id, class, &item.text, "vault") + .with_summary(&item.text), + ); + graph.edges.push(KgEdge::new(subject, id, predicate)); + } + if predicate == P::HasScopeOut { + for file in &item.files { + self.file_link(&mut graph, subject, file, P::ScopeOutFile); + } + } + } + } + for constraint in &intent.has_constraint { + if let Some((id, class)) = self.resolve(&constraint.id) { + graph.nodes.push( + KgNode::new(&id, class, &constraint.text, "vault") + .with_summary(&constraint.text), + ); + graph.edges.push(KgEdge::new(subject, id, P::HasConstraint)); + } + } + for reference in &intent.depends_on { + let predicate = match reference.edge.as_str() { + "dependsOn" => P::DependsOnIntent, + "blockedBy" => P::BlockedBy, + "remediates" => P::Remediates, + "reviews" => P::Reviews, + _ => P::References, + }; + let raw = if reference.to.starts_with("urn:") { + reference.to.clone() + } else { + format!("urn:atomic:intent:{}", reference.to) + }; + self.link(&mut graph, subject, &raw, predicate, path); + } + } + } + } + for part in content.split("[[").skip(1) { + if let Some((name, _)) = part.split_once("]]") { + self.link( + &mut graph, + subject, + &format!("urn:atomic:memory:{name}"), + P::References, + path, + ); + } + } + for word in content.split_whitespace() { + let file = word + .trim_matches([ + '`', '"', '\'', '(', ')', '[', ']', '{', '}', '<', '>', ',', ';', + ]) + .trim_end_matches('.'); + if file.len() < 201 + && file.contains('/') + && [".rs", ".py", ".ts", ".js", ".go", ".md", ".toml", ".json"] + .iter() + .any(|extension| file.ends_with(extension)) + { + self.file_link( + &mut graph, + subject, + file, + if class == C::ToolResult { + P::ProvUsed + } else { + P::References + }, + ); + } + } + graph.nodes.push(node); + for node in &mut graph.nodes { + if node.id == *subject + || node.has_type(C::Task) + || node.has_type(C::AcceptanceCriterion) + || node.has_type(C::ScopeItem) + || node.has_type(C::Constraint) + || node.has_type(C::Finding) + { + let metadata = node.metadata.get_or_insert_with(|| serde_json::json!({})); + metadata["derived_from_vault_path"] = serde_json::json!(path); + } + } + for edge in &mut graph.edges { + let metadata = edge.metadata.get_or_insert_with(|| serde_json::json!({})); + metadata["derived_from_vault_path"] = serde_json::json!(path); + } + Ok(graph) + } + + fn extract_signed_document( + &self, + path: &str, + entry: &VaultEntry, + ) -> Result { + // Signed JSON-LD is an immutable representation of an intent/memory, + // not a mutable source entry and not a session `.attest` object. Read + // typed fields without expanding or rewriting its historical context. + let mut graph = KgSubgraph::default(); + let document = if let Ok(node) = + serde_json::from_slice::(&entry.content_bytes) + { + let mut references: Vec<_> = node + .informed_by + .into_iter() + .map(|target| (target, P::InformedBy)) + .collect(); + for reference in node.depends_on { + let predicate = match reference.edge.as_str() { + "dependsOn" => P::DependsOnIntent, + "blockedBy" => P::BlockedBy, + "remediates" => P::Remediates, + "reviews" => P::Reviews, + _ => P::References, + }; + references.push((reference.to, predicate)); + } + Some((node.id, node.attributed_to, references)) + } else if let Ok(node) = + serde_json::from_slice::(&entry.content_bytes) + { + Some(( + node.id, + node.attributed_to, + node.derived_from + .into_iter() + .map(|target| (target, P::ProvWasDerivedFrom)) + .collect(), + )) + } else { + None + }; + if let Some((id, author, references)) = document { + if let Some((subject, class)) = self.resolve(&id) { + graph + .nodes + .push(KgNode::new(&subject, class, &id, "vault_reference")); + if let Some(author) = author { + let identity = iri::mint_identity(&self.project, &author); + graph.nodes.push(KgNode::new( + &identity, + C::Identity, + author, + "vault_reference", + )); + graph + .edges + .push(KgEdge::new(&subject, identity, P::AuthoredBy)); + } + for (target, predicate) in references { + self.link(&mut graph, &subject, &target, predicate, path); + } + } else { + let project = iri::mint_project(&self.project); + graph.nodes.push(KgNode::new( + &project, + C::Project, + self.project.to_base32(), + "vcs", + )); + self.finding( + &mut graph, + &project, + "UnresolvedReference", + &format!("Signed document has no locally resolved subject: {id}"), + path, + ); + } + } + for edge in &mut graph.edges { + edge.metadata.get_or_insert_with(|| serde_json::json!({}))["derived_from_vault_path"] = + serde_json::json!(path); + } + Ok(graph) + } + + fn file_link(&self, graph: &mut KgSubgraph, subject: &str, path: &str, predicate: P) { + let file = iri::mint_file(&self.project, path); + graph.nodes.push( + KgNode::new(&file, C::File, path, "vault") + .with_metadata(serde_json::json!({"path": path})), + ); + graph.edges.push(KgEdge::new(subject, file, predicate)); + } +} + +fn links<'a>(fm: &'a serde_json::Value, field: &str) -> Vec<&'a str> { + match fm.get(field) { + Some(serde_json::Value::Array(values)) => { + values.iter().filter_map(|v| v.as_str()).collect() + } + Some(serde_json::Value::String(value)) => value + .split(',') + .map(str::trim) + .filter(|s| !s.is_empty()) + .collect(), + _ => Vec::new(), + } +} diff --git a/atomic-repository/src/repository/vault_triples.rs b/atomic-repository/src/repository/vault_triples.rs index d2c1a7b4..61f3b387 100644 --- a/atomic-repository/src/repository/vault_triples.rs +++ b/atomic-repository/src/repository/vault_triples.rs @@ -1,2851 +1,364 @@ -//! Knowledge graph extraction and query operations for the vault. -//! -//! Extracts `KgNode`s and `KgEdge`s from vault entries (goals, intents, -//! memory, skills) and stores them in the pristine KG tables. +//! Canonical vault projection persistence and workflow graph queries. +use super::vault_projection::VaultProjection; use super::*; -use crate::content_search::{has_content_index, search_content, ContentSearchOptions}; -use atomic_canonical::lift::lift_intent; -use atomic_canonical::CanonicalNode; -use atomic_core::pristine::ontology::{edge_kind, predicate}; use atomic_core::pristine::tables::tokenize_for_fts; -use atomic_core::pristine::vault::{ - KgEdge, KgNode, KgSubgraph, VaultEntry, VaultEntryType, VaultManifest, +use atomic_core::pristine::{KgMutTxnT, KgTxnT, VaultEntry, VaultTxnT}; +use atomic_core::{ + kg::{ + iri, + ontology::{Class, VERSION_IRI}, + project_id::MintReason, + }, + KgEdge, KgNode, KgSubgraph, }; -use atomic_core::pristine::{KgMutTxnT, KgTxnT, VaultTxnT}; +use std::collections::{BTreeMap, HashMap, HashSet}; + +fn owned(metadata: Option<&serde_json::Value>, path: &str) -> bool { + metadata + .and_then(|m| m.get("derived_from_vault_path")) + .and_then(|m| m.as_str()) + == Some(path) +} + +pub(super) fn delete_vault_projection( + txn: &mut atomic_core::WriteTxn<'_>, + path: &str, +) -> Result<(), RepositoryError> { + if !txn.kg_writes_enabled()? { + return Ok(()); + } + for node in txn.all_kg_nodes()? { + if owned(node.metadata.as_ref(), path) { + txn.del_kg_node(&node.id)?; + } + } + for edge in txn.all_kg_edges()? { + if owned(edge.metadata.as_ref(), path) { + txn.del_kg_edge(&edge.from_id, &edge.to_id, &edge.kind)?; + } + } + Ok(()) +} + +/// Immutable read snapshot for RDF adaptation; taking one transaction preserves +/// consistency between resources, types, predicates, and the generation marker. +#[derive(Clone, Debug, serde::Serialize)] +pub struct WorkflowSnapshot { + pub generation: u32, + pub ontology_version: &'static str, + pub identity: String, + pub nodes: Vec, + pub edges: Vec, +} impl Repository { - /// Initialize the knowledge graph tables. pub fn init_kg(&self) -> Result<(), RepositoryError> { - let mut txn = self - .pristine - .write_txn() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - txn.init_kg() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - txn.commit() - .map_err(|e| RepositoryError::Database(e.to_string()))?; + self.ensure_project_identity(MintReason::Upgrade)?; + let mut txn = self.pristine.write_txn()?; + if txn.kg_writes_enabled()? { + txn.init_kg()?; + } + txn.commit()?; Ok(()) } - /// Extract KG nodes and edges from a vault entry. - /// - /// Returns `(Vec, Vec)`. Does **not** store them — - /// call `vault_store_kg` for persistence. + fn vault_projection(&self) -> Result { + self.ensure_project_identity(MintReason::Upgrade)?; + let hashes = self + .change_store + .iter_changes() + .collect::, _>>() + .map_err(|e| RepositoryError::Database(e.to_string()))?; + VaultProjection::new(&self.pristine.read_txn()?, self.project_id()?, hashes) + } + pub fn vault_extract_kg( &self, path: &str, entry: &VaultEntry, ) -> Result<(Vec, Vec), RepositoryError> { - let mut nodes: Vec = Vec::new(); - let mut edges: Vec = Vec::new(); - - // Determine the node ID and kind from the entry type and path - let subject = entry_subject(path, entry.entry_type); - - let kind = match entry.entry_type { - VaultEntryType::Session => "goal", - VaultEntryType::Intent => "intent", - VaultEntryType::Memory => "memory", - VaultEntryType::Skill => "skill", - VaultEntryType::ToolResult => "tool_result", - VaultEntryType::Scratch => return Ok((nodes, edges)), // no KG data for scratch - VaultEntryType::Attestation => return Ok((nodes, edges)), // signed blob: no KG data - }; - - // Derive a label from the subject (the part after the colon) - let label = subject.split_once(':').map(|(_, l)| l).unwrap_or(&subject); - - // Keep storage location separate from node identity. Consumers can - // resolve a stable/canonical node ID back to its materialized vault - // entry without parsing the ID or assuming it matches the filename. - let mut node = KgNode::new(&subject, kind, label, "vault") - .with_metadata(serde_json::json!({ "vault_path": path })); - - // Canonical RDF ids and today's path-keyed KG ids are not always the - // same. In particular, a canonical `urn:atomic:intent:atom-1` may live - // at `intents/manual//1/intent.md`. Resolve lineage targets - // through the manifest so edges reach the actual indexed node while - // their metadata retains the exact RDF target. Computed up front so the - // frontmatter extraction below resolves intent links in a single pass. - let manifest = self.vault_manifest().ok(); - - // Extract from frontmatter - if let Ok(fm) = serde_json::from_str::>( - &entry.frontmatter_json, - ) { - extract_frontmatter_kg( - &subject, - entry.entry_type, - &fm, - manifest.as_ref(), - &mut node, - &mut nodes, - &mut edges, - ); - } - - // Extract from content - let content = String::from_utf8_lossy(&entry.content_bytes); - extract_content_edges(&subject, entry.entry_type, &content, &mut edges); - - // Intent bodies carry a directive-based decomposition (acceptance - // criteria, tasks, scope, constraints, and typed dependency refs). Lift - // it and project the structure into the graph so the semantic web is - // queryable: intent→task→criterion, task→file, etc. A body that does not - // lift (legacy prose intents, malformed directives, or a missing `id`) - // is skipped rather than failing KG extraction. - if entry.entry_type == VaultEntryType::Intent { - if let Ok(fm) = serde_json::from_str::>( - &entry.frontmatter_json, - ) { - if let Ok(lifted) = lift_intent(&fm, content.as_ref()) { - project_intent_semantics( - &subject, - &lifted, - manifest.as_ref(), - &mut nodes, - &mut edges, - ); - } - } - } - - // Edge direction is not enough to determine ownership. In particular, - // an Intent's symmetric `BLOCKS` edge points *into* the Intent node. - // Persist the source Vault path so an update replaces only edges - // derived from that entry while preserving RDF links owned elsewhere. - for edge in &mut edges { - let rdf_target = edge - .metadata - .as_ref() - .and_then(|metadata| metadata.get("rdf_target")) - .and_then(serde_json::Value::as_str) - .map(str::to_string); - if let Some(rdf_target) = rdf_target { - edge.to_id = rdf_target_to_kg_id(&rdf_target, manifest.as_ref()); - } - - let metadata = edge.metadata.get_or_insert_with(|| serde_json::json!({})); - if let Some(object) = metadata.as_object_mut() { - object.insert( - "derived_from_vault_path".to_string(), - serde_json::json!(path), - ); - } - } - - nodes.push(node); - Ok((nodes, edges)) + let graph = self.vault_projection()?.extract(path, entry)?; + Ok((graph.nodes, graph.edges)) } - /// Store KG nodes and edges previously returned by [`Self::vault_extract_kg`], - /// replacing existing derived data for that path. - /// - /// The supplied triples must still match the current Vault entry. Prefer - /// [`Self::vault_index_kg`] when the caller does not need the two-step API. pub fn vault_store_kg( &self, path: &str, nodes: &[KgNode], edges: &[KgEdge], ) -> Result { - let current_entry = - self.vault_retrieve(path)? - .ok_or_else(|| RepositoryError::FileNotFound { - path: std::path::PathBuf::from(path), - })?; - let (current_nodes, current_edges) = self.vault_extract_kg(path, ¤t_entry)?; - if nodes != current_nodes || edges != current_edges { + let entry = self + .vault_retrieve(path)? + .ok_or_else(|| RepositoryError::FileNotFound { path: path.into() })?; + let expected = self.vault_extract_kg(path, &entry)?; + if nodes != expected.0 || edges != expected.1 { return Err(RepositoryError::InvalidOperation { - message: format!( - "KG data for '{path}' no longer matches the current Vault entry; re-extract or call vault_index_kg" - ), + message: "stale vault projection; re-extract before storing".into(), }); } - self.vault_store_kg_inner(path, nodes, edges, Some(¤t_entry)) - } - - fn vault_store_kg_inner( - &self, - path: &str, - nodes: &[KgNode], - edges: &[KgEdge], - expected_entry: Option<&VaultEntry>, - ) -> Result { - // Today vault KG identities are path-derived. Prefer the primary node - // identified by its vault_path metadata so ToolResult updates remove - // their real `tool:` node rather than a guessed `vault:` node. - let subject = nodes - .iter() - .find(|node| { - node.metadata - .as_ref() - .and_then(|metadata| metadata.get("vault_path")) - .and_then(serde_json::Value::as_str) - == Some(path) - }) - .or_else(|| nodes.iter().find(|node| node.id == path_to_subject(path))) - .or_else(|| (nodes.len() == 1).then(|| &nodes[0])) - .map(|node| node.id.clone()) - .unwrap_or_else(|| path_to_subject(path)); - - let mut txn = self - .pristine - .write_txn() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - // Extraction happens before the write transaction. If the source was - // updated or deleted meanwhile, this result is stale and must not - // recreate derived KG data after the newer source operation commits. - if let Some(expected) = expected_entry { - let current = txn - .get_vault_entry(path) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - let still_current = current.is_some_and(|entry| { - entry.entry_type == expected.entry_type - && entry.content_hash == expected.content_hash - && entry.frontmatter_json == expected.frontmatter_json - }); - if !still_current { - return Ok(0); - } - } - - txn.init_kg() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - // Remove any child nodes this intent previously projected (tasks, - // acceptance criteria, scope, constraints) before re-projecting, so a - // removed task/criterion does not orphan its node and stale - // SATISFIES/TOUCHES edges. No-op for non-intent subjects. - delete_intent_child_nodes(&mut txn, &subject)?; - - if nodes.iter().any(|node| node.id == subject) { - // This is an in-place replacement of the same resource. Remove - // relationships derived from this Vault entry, but preserve edges - // from other resources that point at it (for example an Intent's - // RDF/PROV link to a source Memory). `upsert_kg_node` below replaces - // the node metadata and FTS terms transactionally. - let outgoing = txn - .get_kg_edges_from(&subject) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - let incoming = txn - .get_kg_edges_to(&subject) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - for edge in outgoing.iter().chain(&incoming) { - if !edge_is_owned_by_vault_entry(edge, path, &subject) { - continue; - } - txn.del_kg_edge(&edge.from_id, &edge.to_id, &edge.kind) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - } - } else { - // No replacement node was extracted (for example an entry that no - // longer participates in the KG), so remove the resource fully. - txn.del_kg_node(&subject) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - } - - // Store new nodes - for node in nodes { - txn.upsert_kg_node(node) - .map_err(|e| RepositoryError::Database(e.to_string()))?; + let mut txn = self.pristine.write_txn()?; + if !txn.kg_writes_enabled()? { + return Ok(0); } - - // Store new edges - for edge in edges { - txn.upsert_kg_edge(edge) - .map_err(|e| RepositoryError::Database(e.to_string()))?; + if txn.get_vault_entry(path)?.as_ref().is_none_or(|current| { + current.content_hash != entry.content_hash + || current.frontmatter_json != entry.frontmatter_json + || current.entry_type != entry.entry_type + }) { + return Ok(0); } - - txn.commit() - .map_err(|e| RepositoryError::Database(e.to_string()))?; + store_projection(&mut txn, path, nodes, edges)?; + txn.commit()?; Ok(nodes.len() + edges.len()) } - /// Extract and store KG data for a vault entry in one step. pub fn vault_index_kg(&self, path: &str) -> Result { + self.ensure_project_identity(MintReason::Upgrade)?; + if self.pristine.read_txn()?.kg_generation()? != atomic_core::kg::KG_GENERATION { + return Ok(0); + } let entry = self .vault_retrieve(path)? - .ok_or_else(|| RepositoryError::FileNotFound { - path: std::path::PathBuf::from(path), - })?; + .ok_or_else(|| RepositoryError::FileNotFound { path: path.into() })?; let (nodes, edges) = self.vault_extract_kg(path, &entry)?; - self.vault_store_kg_inner(path, &nodes, &edges, Some(&entry)) + self.vault_store_kg(path, &nodes, &edges) } - /// Re-index all vault entries into the knowledge graph. pub fn vault_reindex_kg(&self) -> Result { - let entries = self.vault_list("", None)?; - let mut total = 0; - for meta in &entries { - match self.vault_index_kg(&meta.path) { - Ok(count) => total += count, - Err(e) => { - log::warn!("Failed to index KG for {}: {}", meta.path, e); - } - } - } - Ok(total) - } - - /// Full-text search over knowledge graph nodes. - /// - /// `pool` controls the candidate heap size — how many nodes are scored - /// before the top `limit` are selected. A larger pool lets lower-scored - /// node kinds (e.g. changes) survive alongside higher-scored ones - /// (e.g. files with content matches). When `None`, defaults to `limit`. - pub fn vault_kg_search( - &self, - query: &str, - limit: usize, - pool: Option, - ) -> Result, RepositoryError> { - use std::cmp::Reverse; - use std::collections::{BinaryHeap, HashMap, HashSet}; - - let pool_size = pool.unwrap_or(limit).max(limit); - - let txn = self - .pristine - .read_txn() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - // ── Phase 1: Collect and validate matching node IDs ───────────────── - - // Legacy KG FTS only appended postings, and a read-only repository - // cannot run the reverse-index migration. Treat the index as a - // candidate generator, then recompute hits from the current KG node. - // This prevents stale label/summary terms from being served before the - // next write-time migration and also protects against mixed-version - // writers that may have appended an obsolete posting. - let query_tokens: HashSet = tokenize_for_fts(query).into_iter().collect(); - let indexed_matches = txn - .kg_fts_match_ids(query) + self.ensure_project_identity(MintReason::Upgrade)?; + let hashes = self + .change_store + .iter_changes() + .collect::, _>>() .map_err(|e| RepositoryError::Database(e.to_string()))?; - let mut kg_matches: HashMap = HashMap::new(); - for (node_id, _) in indexed_matches { - let Some(node) = txn - .get_kg_node(&node_id) - .map_err(|e| RepositoryError::Database(e.to_string()))? - else { - continue; - }; - let current_hits = kg_node_fts_hit_count(&node, &query_tokens); - if current_hits > 0 { - kg_matches.insert(node_id, current_hits); - } - } - - // Content search: file-level matches from syntext, grouped by path. - let mut content_counts: HashMap = HashMap::new(); - if has_content_index(self.root()) { - let opts = ContentSearchOptions { - max_results: Some(2000), - ..Default::default() - }; - if let Ok(content_results) = search_content(self.root(), query, opts) { - for m in &content_results.matches { - *content_counts.entry(m.path.clone()).or_insert(0) += 1; - } - } - } - - // ── Phase 2: Stream candidates through a bounded min-heap ────────── - // - // As each candidate arrives, score it and bubble it into the heap. - // The heap holds at most `pool_size` items — weakest score at the top. - // If a new candidate beats the weakest, it replaces it. - // We trim to `limit` in Phase 3. - - // Min-heap: Reverse so BinaryHeap (max-heap) gives us min-score at top. - // Tuple: (score, node_id, content_match_count) - let mut heap: BinaryHeap> = - BinaryHeap::with_capacity(pool_size + 1); - let mut seen: HashSet = HashSet::new(); - - // Helper: push a candidate into the bounded heap - let push_candidate = - |id: String, - kg_hits: usize, - content_matches: usize, - heap: &mut BinaryHeap>| { - let score = id_rank_score(&id, kg_hits, content_matches); - if heap.len() < pool_size { - heap.push(Reverse((score, id, content_matches))); - } else if let Some(&Reverse((min_score, _, _))) = heap.peek() { - if score > min_score { - heap.pop(); - heap.push(Reverse((score, id, content_matches))); - } - } - }; - - // Stream KG matches through the heap - for (id, kg_hits) in &kg_matches { - let cm = id - .strip_prefix("file:") - .and_then(|p| content_counts.get(p)) - .copied() - .unwrap_or(0); - seen.insert(id.clone()); - push_candidate(id.clone(), *kg_hits, cm, &mut heap); - } - - // Stream content-only files (not already seen from KG) - for (path, &count) in &content_counts { - let file_id = format!("file:{}", path); - if seen.contains(&file_id) { - continue; - } - push_candidate(file_id, 0, count, &mut heap); - } - - // ── Phase 3: Diversity-aware selection ───────────────────────────── - // - // When pool_size > limit the heap holds more candidates than we - // need. Pure top-N by score would return only the dominant kind - // (usually files with content matches). Instead, reserve a - // minimum number of slots per node kind, then fill the rest by - // overall score. This guarantees that changes, entities, and - // other kinds appear in results when they match the query. - - let mut all_candidates: Vec<(u64, String, usize)> = - heap.into_iter().map(|Reverse(entry)| entry).collect(); - all_candidates.sort_by_key(|entry| std::cmp::Reverse(entry.0)); - - let ranked = if pool_size > limit { - // Diversity mode: reserve slots per kind. - let min_per_kind: usize = 2; - - // Group by kind (inferred from ID prefix) - let mut by_kind: HashMap> = HashMap::new(); - for entry in all_candidates { - let kind = entry.1.split(':').next().unwrap_or("other").to_string(); - by_kind.entry(kind).or_default().push(entry); - } - - // First pass: take up to min_per_kind from each kind (already - // sorted by score within each group since all_candidates was sorted). - let mut selected: Vec<(u64, String, usize)> = Vec::with_capacity(limit); - let mut remaining: Vec<(u64, String, usize)> = Vec::new(); - for (_kind, mut entries) in by_kind { - let reserved: Vec<_> = entries.drain(..entries.len().min(min_per_kind)).collect(); - selected.extend(reserved); - remaining.extend(entries); - } - - // Second pass: fill remaining slots from leftover candidates by score - if selected.len() < limit { - remaining.sort_by_key(|e| std::cmp::Reverse(e.0)); - selected.extend(remaining.into_iter().take(limit - selected.len())); - } - - // Final sort by score for consistent output order - selected.sort_by_key(|e| std::cmp::Reverse(e.0)); - selected.truncate(limit); - selected - } else { - // No diversity — simple top-N by score (original behavior) - all_candidates.truncate(limit); - all_candidates - }; - - // ── Phase 4: Fetch full nodes only for the top N ─────────────────── - - let mut results: Vec = Vec::with_capacity(ranked.len()); - for (_score, id, content_match_count) in &ranked { - let node = match txn.get_kg_node(id) { - Ok(Some(mut n)) => { - if *content_match_count > 0 { - let md = n.metadata.get_or_insert_with(|| serde_json::json!({})); - if let Some(obj) = md.as_object_mut() { - obj.insert( - "content_matches".to_string(), - serde_json::json!(content_match_count), - ); - } - } - n - } - _ => { - // Node not in KG — create a stub (content-only file) - let label = id - .strip_prefix("file:") - .and_then(|p| p.rsplit_once('/').map(|(_, name)| name)) - .unwrap_or(id); - let mut n = KgNode::new(id, "file", label, "content_search"); - if *content_match_count > 0 { - n = n.with_metadata( - serde_json::json!({"content_matches": content_match_count}), - ); - } - n - } - }; - results.push(node); + let mut txn = self.pristine.write_txn()?; + if !txn.kg_writes_enabled()? { + return Ok(0); } - - Ok(results) - } - - /// Full-text search restricted to one KG node kind before top-N selection. - /// - /// Typed callers must not fetch a mixed global top-N and filter afterward: - /// higher-weight file/module results can otherwise crowd every matching - /// memory out of a small result window. This path intentionally searches - /// KG node metadata only; source-file content search is not relevant to - /// memory-only retrieval. - pub fn vault_kg_search_by_kind( - &self, - query: &str, - limit: usize, - kind: &str, - ) -> Result, RepositoryError> { - use std::collections::HashSet; - - if limit == 0 { - return Ok(Vec::new()); - } - let query_tokens: HashSet = tokenize_for_fts(query).into_iter().collect(); - if query_tokens.is_empty() { - return Ok(Vec::new()); - } - - let txn = self - .pristine - .read_txn() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - let matches = txn - .kg_fts_match_ids(query) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - let mut ranked = Vec::new(); - for (node_id, _) in matches { - let Some(node) = txn - .get_kg_node(&node_id) - .map_err(|e| RepositoryError::Database(e.to_string()))? - else { - continue; - }; - if node.kind != kind { - continue; - } - - // Legacy KG FTS only appended postings. Re-check the current node - // text so obsolete postings cannot make a renamed node match. - let hits = kg_node_fts_hit_count(&node, &query_tokens); - if hits > 0 { - ranked.push((hits, node)); - } - } - - ranked.sort_by(|(hits_a, node_a), (hits_b, node_b)| { - hits_b.cmp(hits_a).then_with(|| node_a.id.cmp(&node_b.id)) - }); - ranked.truncate(limit); - Ok(ranked.into_iter().map(|(_, node)| node).collect()) - } -} - -fn kg_node_fts_hit_count(node: &KgNode, query_tokens: &std::collections::HashSet) -> usize { - let mut text = String::with_capacity(node.id.len() + node.label.len() + 64); - text.push_str(&node.id); - text.push(' '); - text.push_str(&node.label); - if let Some(summary) = &node.summary { - text.push(' '); - text.push_str(summary); - } - let node_tokens: std::collections::HashSet = - tokenize_for_fts(&text).into_iter().collect(); - query_tokens.intersection(&node_tokens).count() -} - -/// Whether an existing edge should be replaced when `path` is re-indexed. -/// -/// New edges carry explicit ownership. The direction/kind fallbacks clean up -/// legacy edges created before ownership metadata existed: -/// - outgoing edges were derived from the subject entry; -/// - incoming `BLOCKS` edges were the symmetric half of that Intent's -/// `blocked_by` relationship. -fn edge_is_owned_by_vault_entry(edge: &KgEdge, path: &str, subject: &str) -> bool { - if let Some(owner) = edge - .metadata - .as_ref() - .and_then(|metadata| metadata.get("derived_from_vault_path")) - .and_then(serde_json::Value::as_str) - { - return owner == path; - } - - edge.from_id == subject || (edge.to_id == subject && edge.kind == edge_kind::BLOCKS) -} - -/// Compute a ranking score from a node ID and match counts. -/// -/// Higher score = more relevant. Works on IDs only (no node fetching needed). -/// Combines: node kind weight, path tier, KG hit count, content match count. -fn id_rank_score(id: &str, kg_hits: usize, content_matches: usize) -> u64 { - // Base score by node kind (inferred from ID prefix). - // - // Changes score the same as files: a change whose commit message - // matches the query is equally relevant as a file whose name does. - // Content matches (syntext hits) still boost files above changes - // when the query appears heavily inside file bodies. - let kind_score: u64 = if id.starts_with("module:") { - 500 - } else if id.starts_with("file:") { - 400 - } else if id.starts_with("entity:") { - 300 - } else if id.starts_with("change:") { - 400 - } else { - 200 - }; - - // Extract path from the ID for tier ranking - let path = extract_path_from_id(id); - let tier_penalty: u64 = match path_tier(path) { - 0 => 0, // src/ — no penalty - 1 => 20, // other code files - 2 => 80, // test files - 3 => 120, // docs - 4 => 200, // config/build/CI - _ => 150, - }; - - // KG hit bonus: multi-token matches are more relevant - let kg_bonus: u64 = (kg_hits as u64).saturating_sub(1) * 50; - - // Content match bonus: files with many content hits are more relevant - let content_bonus: u64 = (content_matches as u64).min(100); - - kind_score.saturating_sub(tier_penalty) + kg_bonus + content_bonus -} - -/// Extract a file path from a node ID for ranking purposes. -fn extract_path_from_id(id: &str) -> &str { - if let Some(path) = id.strip_prefix("file:") { - return path; - } - if let Some(path) = id.strip_prefix("module:") { - return path; - } - // entity:file:name:line — extract the file portion - if let Some(rest) = id.strip_prefix("entity:") { - // e.g., "src/mongo/db/repl/repl.cpp:ReplicationCoordinator:42" - // Find the second-to-last colon to get the file path - let parts: Vec<&str> = rest.rsplitn(3, ':').collect(); - if parts.len() == 3 { - return parts[2]; - } - } - id -} - -/// Assign a ranking tier to a file path. -/// -/// Lower tier = higher priority (less penalty). -fn path_tier(path: &str) -> u8 { - // Tier 0: primary source directories - let source_prefixes = ["src/", "lib/", "pkg/", "internal/", "cmd/", "app/"]; - for prefix in &source_prefixes { - if path.starts_with(prefix) { - if is_test_path(path) { - return 2; + let context = VaultProjection::new(&txn, txn.kg_project()?, hashes)?; + let entries = txn.list_vault_entries("", None)?; + let mut total = 0; + for meta in entries { + if let Some(entry) = txn.get_vault_entry(&meta.path)? { + let graph = context.extract(&meta.path, &entry)?; + store_projection(&mut txn, &meta.path, &graph.nodes, &graph.edges)?; + total += graph.nodes.len() + graph.edges.len(); } - return 0; } + txn.commit()?; + self.refresh_session_kg()?; + Ok(total) } - // Tier 1: other code files - let code_extensions = [ - ".rs", ".go", ".py", ".ts", ".js", ".cpp", ".cc", ".c", ".h", ".hpp", ".java", ".kt", - ".swift", ".rb", ".cs", - ]; - if code_extensions.iter().any(|ext| path.ends_with(ext)) { - if is_test_path(path) { - return 2; - } - return 1; - } - - // Tier 2: test paths - if is_test_path(path) { - return 2; - } - - // Tier 3: docs - if path.ends_with(".md") || path.starts_with("docs/") || path.starts_with("doc/") { - return 3; + pub fn workflow_snapshot(&self) -> Result { + let txn = self.pristine.read_txn()?; + let mut nodes = txn.all_kg_nodes()?; + let mut edges = txn.all_kg_edges()?; + nodes.sort_by(|a, b| a.id.cmp(&b.id)); + edges.sort_by(|a, b| (&a.from_id, &a.to_id, a.kind).cmp(&(&b.from_id, &b.to_id, b.kind))); + let bytes = + serde_json::to_vec(&(atomic_core::kg::KG_GENERATION, VERSION_IRI, &nodes, &edges)) + .expect("snapshot serialization"); + Ok(WorkflowSnapshot { + generation: atomic_core::kg::KG_GENERATION, + ontology_version: VERSION_IRI, + identity: Hash::of(&bytes).to_base32(), + nodes, + edges, + }) } - // Tier 4: build scripts, config, CI, generated files - let low_priority = [ - "buildscripts/", - "build/", - ".github/", - "ci/", - "scripts/", - "debian/", - "rpm/", - "packaging/", - "vendor/", - "third_party/", - "node_modules/", - "target/", - ]; - if low_priority.iter().any(|p| path.starts_with(p)) { - return 4; + pub fn vault_kg_neighbors(&self, id: &str, depth: u8) -> Result { + Ok(self.pristine.read_txn()?.kg_neighbors(id, depth)?) } - if path.ends_with(".yml") - || path.ends_with(".yaml") - || path.ends_with(".toml") - || path.ends_with(".json") - || path.ends_with(".xml") - || path.ends_with(".cfg") - { - return 4; - } - - 3 -} - -/// Heuristic: is this path a test file? -fn is_test_path(path: &str) -> bool { - let lower = path.to_lowercase(); - lower.contains("/test/") - || lower.contains("/tests/") - || lower.contains("_test.") - || lower.contains("_test_") - || lower.contains(".test.") - || lower.starts_with("test/") - || lower.starts_with("tests/") - || lower.starts_with("jstests/") - || lower.starts_with("testdata/") -} - -impl Repository { - /// Get the neighborhood subgraph around a node. - pub fn vault_kg_neighbors( + pub fn workflow_neighbors( &self, - node_id: &str, + id: &str, depth: u8, + predicate: Option, + direction: atomic_core::kg::Direction, ) -> Result { - let txn = self + Ok(self .pristine - .read_txn() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - txn.kg_neighbors(node_id, depth) - .map_err(|e| RepositoryError::Database(e.to_string())) + .read_txn()? + .kg_neighbors_filtered(id, depth, predicate, direction)?) } - - /// Get a single KG node by ID. pub fn vault_kg_node(&self, id: &str) -> Result, RepositoryError> { - let txn = self - .pristine - .read_txn() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - txn.get_kg_node(id) - .map_err(|e| RepositoryError::Database(e.to_string())) + Ok(self.pristine.read_txn()?.get_kg_node(id)?) } - - /// Count total KG nodes. pub fn vault_kg_node_count(&self) -> Result { - let txn = self - .pristine - .read_txn() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - txn.count_kg_nodes() - .map_err(|e| RepositoryError::Database(e.to_string())) + Ok(self.pristine.read_txn()?.count_kg_nodes()?) } - - /// Count total KG edges. pub fn vault_kg_edge_count(&self) -> Result { - let txn = self - .pristine - .read_txn() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - txn.count_kg_edges() - .map_err(|e| RepositoryError::Database(e.to_string())) - } -} - -// ── Extraction helpers ────────────────────────────────────────── - -/// Derive a subject URI from a vault path and entry type. -pub(super) fn entry_subject(path: &str, entry_type: VaultEntryType) -> String { - match entry_type { - VaultEntryType::Session => { - // goals/swift-meadow-a3f2/_goal.md -> goal:swift-meadow-a3f2 - let name = path - .strip_prefix("goals/") - .and_then(|s| s.strip_suffix("/_goal.md")) - .unwrap_or(path); - format!("goal:{}", name) - } - VaultEntryType::Intent => { - // intents/pimo-1/intent.md -> intent:PIMO-1 - let id = path - .strip_prefix("intents/") - .and_then(|s| s.strip_suffix("/intent.md")) - .unwrap_or(path) - .to_uppercase(); - format!("intent:{}", id) - } - VaultEntryType::Memory => { - // memory/architecture.md -> memory:architecture - let name = path - .strip_prefix("memory/") - .and_then(|s| s.strip_suffix(".md")) - .unwrap_or(path); - format!("memory:{}", name) - } - VaultEntryType::Skill => { - let name = path - .strip_prefix("skills/") - .and_then(|s| s.strip_suffix(".md")) - .unwrap_or(path); - format!("skill:{}", name) - } - VaultEntryType::ToolResult => { - format!("tool:{}", path.replace('/', ":")) - } - VaultEntryType::Scratch => { - format!("scratch:{}", path.replace('/', ":")) - } - VaultEntryType::Attestation => { - // attestations//attested.md -> attestation: - let name = path - .strip_prefix("attestations/") - .and_then(|s| s.strip_suffix("/attested.md")) - .unwrap_or(path); - format!("attestation:{}", name) - } - } -} - -/// Convert a vault path to a subject URI (simplified). -fn path_to_subject(path: &str) -> String { - if path.starts_with("goals/") { - entry_subject(path, VaultEntryType::Session) - } else if path.starts_with("intents/") { - entry_subject(path, VaultEntryType::Intent) - } else if path.starts_with("memory/") { - entry_subject(path, VaultEntryType::Memory) - } else if path.starts_with("skills/") { - entry_subject(path, VaultEntryType::Skill) - } else { - format!("vault:{}", path.replace('/', ":")) - } -} - -/// Extract KG edges (and populate node summary) from frontmatter fields. -fn extract_frontmatter_kg( - subject: &str, - entry_type: VaultEntryType, - fm: &serde_json::Map, - manifest: Option<&VaultManifest>, - node: &mut KgNode, - nodes: &mut Vec, - edges: &mut Vec, -) { - // Common fields → edges - if let Some(s) = fm.get("developer").and_then(|v| v.as_str()) { - edges.push(KgEdge::new( - subject, - format!("identity:{}", s), - edge_kind::AUTHORED_BY, - )); - } - if let Some(s) = fm.get("creator").and_then(|v| v.as_str()) { - edges.push(KgEdge::new( - subject, - format!("identity:{}", s), - edge_kind::AUTHORED_BY, - )); - } - - // description / title → node summary - if let Some(s) = fm.get("description").and_then(|v| v.as_str()) { - node.summary = Some(s.to_string()); - } - if let Some(s) = fm.get("title").and_then(|v| v.as_str()) { - if node.summary.is_none() { - node.summary = Some(s.to_string()); - } + Ok(self.pristine.read_txn()?.count_kg_edges()?) } - // Entry-type-specific fields - match entry_type { - VaultEntryType::Session => { - if let Some(s) = fm.get("intent").and_then(|v| v.as_str()) { - edges.push(KgEdge::new( - subject, - format!("intent:{}", s), - edge_kind::LINKED_INTENT, - )); - } - if let Some(s) = fm.get("status").and_then(|v| v.as_str()) { - let md = node.metadata.get_or_insert_with(|| serde_json::json!({})); - if let Some(obj) = md.as_object_mut() { - obj.insert("status".to_string(), serde_json::json!(s)); - } - } - } - VaultEntryType::Intent => { - if let Some(s) = fm.get("assignee").and_then(|v| v.as_str()) { - edges.push(KgEdge::new( - subject, - format!("identity:{}", s), - edge_kind::ASSIGNED_TO, - )); - } - if let Some(s) = fm.get("status").and_then(|v| v.as_str()) { - let md = node.metadata.get_or_insert_with(|| serde_json::json!({})); - if let Some(obj) = md.as_object_mut() { - obj.insert("status".to_string(), serde_json::json!(s)); - } - } - if let Some(s) = fm.get("priority").and_then(|v| v.as_str()) { - let md = node.metadata.get_or_insert_with(|| serde_json::json!({})); - if let Some(obj) = md.as_object_mut() { - obj.insert("priority".to_string(), serde_json::json!(s)); - } - } - // blocked_by array — write both directions for symmetric traversal - if let Some(arr) = fm.get("blocked_by").and_then(|v| v.as_array()) { - for item in arr { - if let Some(s) = item.as_str() { - edges.push(KgEdge::new( - subject, - format!("intent:{}", s), - edge_kind::BLOCKED_BY, - )); - // Symmetric: the blocker also BLOCKS this intent - edges.push(KgEdge::new( - format!("intent:{}", s), - subject, - edge_kind::BLOCKS, - )); - } - } - } - // labels array -> HAS_LABEL edges + concept nodes - if let Some(arr) = fm.get("labels").and_then(|v| v.as_array()) { - for item in arr { - if let Some(s) = item.as_str() { - edges.push(KgEdge::new( - subject, - format!("concept:{}", s), - edge_kind::HAS_LABEL, - )); - // Ensure the concept node exists so it is searchable - nodes.push(KgNode::new(format!("concept:{}", s), "concept", s, "vault")); - } - } - } - // goals/sessions array - if let Some(arr) = fm.get("goals").and_then(|v| v.as_array()) { - for item in arr { - if let Some(s) = item.as_str() { - edges.push(KgEdge::new( - subject, - format!("goal:{}", s), - edge_kind::LINKED_GOAL, - )); + pub fn vault_kg_search_by_kind( + &self, + query: &str, + limit: usize, + class: Class, + ) -> Result, RepositoryError> { + let txn = self.pristine.read_txn()?; + let mut rows = Vec::new(); + for (id, _) in txn.kg_fts_match_ids(query)? { + if let Some(node) = txn.get_kg_node(&id)? { + if node.has_type(class) { + let hits = fts_hits(&node, query); + if hits > 0 { + rows.push((hits, node)); } } } - for source in frontmatter_links(fm, "informedBy") { - edges.push(rdf_link_edge( - subject, - source, - predicate::INFORMED_BY, - manifest, - )); - } } - VaultEntryType::Memory => { - if let Some(s) = fm.get("name").and_then(|v| v.as_str()) { - if node.summary.is_none() { - node.summary = Some(s.to_string()); + rows.sort_by(|a, b| b.0.cmp(&a.0).then_with(|| a.1.id.cmp(&b.1.id))); + Ok(rows.into_iter().take(limit).map(|(_, node)| node).collect()) + } + + pub fn vault_kg_search( + &self, + query: &str, + limit: usize, + pool: Option, + ) -> Result, RepositoryError> { + let txn = self.pristine.read_txn()?; + let mut rows = BTreeMap::new(); + let mut content = HashMap::::new(); + if crate::content_search::has_content_index(self.root()) { + if let Ok(result) = crate::content_search::search_content( + self.root(), + query, + crate::content_search::ContentSearchOptions { + max_results: Some(2000), + ..Default::default() + }, + ) { + for result in result.matches { + *content.entry(result.path).or_default() += 1; } } - for source in frontmatter_links(fm, "derivedFrom") { - edges.push(rdf_link_edge( - subject, - source, - predicate::WAS_DERIVED_FROM, - manifest, - )); - } } - VaultEntryType::Skill => { - if let Some(s) = fm.get("name").and_then(|v| v.as_str()) { - if node.summary.is_none() { - node.summary = Some(s.to_string()); + for (id, _) in txn.kg_fts_match_ids(query)? { + if let Some(node) = txn.get_kg_node(&id)? { + let hits = fts_hits(&node, query); + if hits > 0 { + rows.insert(id, (node, hits)); } } } - VaultEntryType::ToolResult => { - // ToolResult files always live at goals/{goal-name}/toolu_*.md. - // Derive the parent goal from the subject: "tool:goals:{goal}:{file}" - if let Some(rest) = subject.strip_prefix("tool:goals:") { - if let Some((goal_name, _)) = rest.split_once(':') { - edges.push(KgEdge::new( - subject, - format!("goal:{}", goal_name), - predicate::WAS_ASSOCIATED_WITH, - )); + let project = self.project_id()?; + for path in content.keys() { + let id = iri::mint_file(&project, path); + rows.entry(id.clone()).or_insert_with(|| { + ( + KgNode::new(&id, Class::File, path, "content_search") + .with_metadata(serde_json::json!({"path": path})), + 0, + ) + }); + } + let mut ranked = Vec::new(); + for (_, (mut node, hits)) in rows { + let count = node + .path() + .and_then(|path| content.get(path)) + .copied() + .unwrap_or(0); + let weight = if node.has_type(Class::File) || node.has_type(Class::Change) { + 400u64 + } else { + 200 + }; + let penalty = node.path().map(path_penalty).unwrap_or(0); + let score = weight.saturating_sub(penalty) + + hits.saturating_sub(1) as u64 * 50 + + count.min(100) as u64; + if count > 0 { + node.metadata.get_or_insert_with(|| serde_json::json!({}))["content_matches"] = + serde_json::json!(count); + } + ranked.push((score, node)); + } + ranked.sort_by(|a, b| b.0.cmp(&a.0).then_with(|| a.1.id.cmp(&b.1.id))); + ranked.truncate(pool.unwrap_or(limit).max(limit)); + if pool.is_some_and(|pool| pool > limit) { + let mut per_type = HashMap::::new(); + let mut selected = Vec::new(); + let mut remaining = Vec::new(); + for row in ranked { + let count = per_type.entry(row.1.primary_type().to_owned()).or_default(); + if *count < 2 { + selected.push(row); + *count += 1; + } else { + remaining.push(row); } } + selected.extend( + remaining + .into_iter() + .take(limit.saturating_sub(selected.len())), + ); + selected.sort_by(|a, b| b.0.cmp(&a.0).then_with(|| a.1.id.cmp(&b.1.id))); + ranked = selected; } - _ => {} + Ok(ranked + .into_iter() + .take(limit) + .map(|(_, node)| node) + .collect()) } } -/// Read an RDF-link field from frontmatter. Canonical writers use arrays, but -/// accepting a comma-delimited scalar keeps old hand-authored entries usable. -fn frontmatter_links<'a>( - fm: &'a serde_json::Map, - key: &str, -) -> Vec<&'a str> { - match fm.get(key) { - Some(serde_json::Value::Array(items)) => items - .iter() - .filter_map(serde_json::Value::as_str) - .map(str::trim) - .filter(|value| !value.is_empty()) - .collect(), - Some(serde_json::Value::String(value)) => value - .split(',') - .map(str::trim) - .filter(|value| !value.is_empty()) - .collect(), - _ => Vec::new(), - } -} - -/// Project a canonical RDF id onto the path-keyed KG id used by today's -/// derived index. The original RDF target remains in edge metadata, so the KG -/// can traverse current nodes without losing the exact lineage identifier. -fn rdf_link_edge( - subject: &str, - rdf_target: &str, - predicate: &str, - manifest: Option<&VaultManifest>, -) -> KgEdge { - KgEdge::new( - subject, - rdf_target_to_kg_id(rdf_target, manifest), - predicate, - ) - .with_metadata(serde_json::json!({ "rdf_target": rdf_target })) -} - -/// Project the lifted directive structure of an Intent into KG nodes and edges. -/// -/// Emits a node per acceptance criterion, task, scope item, and constraint, -/// plus the edges that link them: `intent -HAS_TASK-> task`, -/// `intent -HAS_ACCEPTANCE_CRITERION-> ac`, `task -SATISFIES-> ac`, -/// `task -TOUCHES-> file:`, and the body's `:::ref` dependency edges. -/// File targets reuse the shared `file:` node convention so intent tasks -/// link into the same file nodes that changes reference via `MODIFIES`. -fn project_intent_semantics( - intent_subject: &str, - node: &CanonicalNode, - manifest: Option<&VaultManifest>, - nodes: &mut Vec, - edges: &mut Vec, -) { - // Acceptance criteria. - for ac in &node.has_acceptance_criterion { - let id = child_kg_id(&ac.id); - let mut meta = serde_json::json!({ "status": ac.ac_status }); - if let Some(obj) = meta.as_object_mut() { - if let Some(v) = &ac.verified_by { - obj.insert("verifiedBy".to_string(), serde_json::json!(v)); - } - if let Some(e) = &ac.evidence { - obj.insert("evidence".to_string(), serde_json::json!(e)); - } +pub(super) fn store_projection( + txn: &mut atomic_core::WriteTxn<'_>, + path: &str, + nodes: &[KgNode], + edges: &[KgEdge], +) -> Result<(), RepositoryError> { + for edge in txn.all_kg_edges()? { + if owned(edge.metadata.as_ref(), path) { + txn.del_kg_edge(&edge.from_id, &edge.to_id, &edge.kind)?; } - nodes.push( - KgNode::new(&id, "acceptance_criterion", kg_label(&id), "vault") - .with_summary(ac.text.clone()) - .with_metadata(meta), - ); - edges.push(KgEdge::new( - intent_subject, - &id, - edge_kind::HAS_ACCEPTANCE_CRITERION, - )); } - - // Tasks, with their satisfies (task→criterion) and touches (task→file) edges. - for t in &node.has_task { - let id = child_kg_id(&t.id); - nodes.push( - KgNode::new(&id, "task", kg_label(&id), "vault") - .with_summary(t.text.clone()) - .with_metadata(serde_json::json!({ "status": t.task_status })), - ); - edges.push(KgEdge::new(intent_subject, &id, edge_kind::HAS_TASK)); - // `as_slice` so a legacy scalar `satisfies` still produces its SATISFIES - // edge — otherwise pre-widening intents would silently lose the - // task→criterion links in the knowledge graph. - for ac_urn in t.satisfies.as_slice() { - edges.push(KgEdge::new(&id, child_kg_id(ac_urn), edge_kind::SATISFIES)); - } - for file in &t.touches_file { - edges.push(KgEdge::new(&id, format!("file:{file}"), edge_kind::TOUCHES)); + let retained: HashSet<&str> = nodes.iter().map(|node| node.id.as_str()).collect(); + for old in txn.all_kg_nodes()? { + if owned(old.metadata.as_ref(), path) && !retained.contains(old.id.as_str()) { + txn.del_kg_node(&old.id)?; } } - - // Scope boundaries. - for s in &node.has_scope_in { - let id = child_kg_id(&s.id); - nodes.push(KgNode::new(&id, "scope", kg_label(&id), "vault").with_summary(s.text.clone())); - edges.push(KgEdge::new(intent_subject, &id, edge_kind::HAS_SCOPE_IN)); - } - for s in &node.has_scope_out { - let id = child_kg_id(&s.id); - nodes.push(KgNode::new(&id, "scope", kg_label(&id), "vault").with_summary(s.text.clone())); - edges.push(KgEdge::new(intent_subject, &id, edge_kind::HAS_SCOPE_OUT)); - // Additive to the prose scope node above: a scope-out `::file-ref` - // projects an owned outgoing edge onto the SHARED `file:` node - // (the same convention MODIFIES/TOUCHES use), so a change's MODIFIES - // target and this out-of-scope declaration meet on one node and a breach - // is a single graph lookup. Owned outgoing from the intent subject, so - // the existing re-index/delete GC cleans it like DEPENDS_ON/REMEDIATES. - for file in &s.files { - edges.push(KgEdge::new( - intent_subject, - format!("file:{file}"), - edge_kind::SCOPE_OUT_FILE, - )); + for node in nodes { + if owned(node.metadata.as_ref(), path) || txn.get_kg_node(&node.id)?.is_none() { + txn.upsert_kg_node(node)?; } } - - // Constraints. - for c in &node.has_constraint { - let id = child_kg_id(&c.id); - nodes.push( - KgNode::new(&id, "constraint", kg_label(&id), "vault").with_summary(c.text.clone()), - ); - edges.push(KgEdge::new(intent_subject, &id, edge_kind::HAS_CONSTRAINT)); - } - - // Typed reference edges declared in the body (`:::ref{to= edge=}`). These - // are intent-owned outgoing edges: BLOCKED_BY/DEPENDS_ON on the dependency - // chain, plus the forward intent→intent `REMEDIATES` link (a remediation - // intent B pointing at the flawed intent A it fixes: B --REMEDIATES--> A) and - // the `REVIEWS` link (a review intent pointing at the intent it reviews: - // review --REVIEWS--> target). All share the same ownership/GC lifecycle - // (outgoing from the intent subject, cleaned on re-index/delete), so no - // extra bookkeeping is needed. - for r in &node.depends_on { - let to = canonical_ref_target(&r.to); - let target = rdf_target_to_kg_id(&to, manifest); - let kind = match r.edge.as_str() { - "blockedBy" => edge_kind::BLOCKED_BY, - "remediates" => edge_kind::REMEDIATES, - "reviews" => edge_kind::REVIEWS, - _ => edge_kind::DEPENDS_ON, - }; - edges.push(KgEdge::new(intent_subject, target, kind)); - } -} - -/// The structural edges that connect an intent to the child nodes it projects. -pub(super) const INTENT_CHILD_EDGE_KINDS: &[&str] = &[ - edge_kind::HAS_TASK, - edge_kind::HAS_ACCEPTANCE_CRITERION, - edge_kind::HAS_SCOPE_IN, - edge_kind::HAS_SCOPE_OUT, - edge_kind::HAS_CONSTRAINT, -]; - -/// Delete the child nodes an intent projected (tasks, acceptance criteria, -/// scope items, constraints), reachable from `subject` via the structural -/// `HAS_*` edges. Deleting each child node also removes its own edges -/// (`SATISFIES`, `TOUCHES`) and full-text terms. Shared `file:` nodes are never -/// children, so a task's `TOUCHES` target survives. -/// -/// A no-op for non-intent subjects (they have no `HAS_*` edges), so callers may -/// invoke it unconditionally before deleting or re-projecting a subject. This -/// keeps the graph from orphaning child nodes when an intent is edited (a task -/// removed) or deleted. -pub(super) fn delete_intent_child_nodes( - txn: &mut atomic_core::pristine::WriteTxn<'_>, - subject: &str, -) -> Result<(), RepositoryError> { - let outgoing = txn - .get_kg_edges_from(subject) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - for edge in outgoing { - if INTENT_CHILD_EDGE_KINDS.contains(&edge.kind.as_str()) { - txn.del_kg_node(&edge.to_id) - .map_err(|e| RepositoryError::Database(e.to_string()))?; - } + for edge in edges { + txn.upsert_kg_edge(edge)?; } Ok(()) } -/// Map a canonical `urn:atomic::` id to the KG `:` -/// node-id convention. Non-URN inputs pass through unchanged. -fn urn_to_kg_id(urn: &str) -> String { - urn.strip_prefix("urn:atomic:").unwrap_or(urn).to_string() -} - -/// Canonical KG id for an intent-decomposition child (`ac`, `task`, `scope`, -/// `constraint`). The kind stays lowercase; the local part is UPPERCASED so -/// children always share the intent node's case (intents are uppercase ULIDs). -/// Applied identically on the mint side (`project_intent_semantics`) and the -/// resolve side (`rdf_target_to_kg_id`), so a `derivedFrom` link lands on the -/// real node regardless of the case the id was copied in. -fn child_kg_id(urn: &str) -> String { - match urn.strip_prefix("urn:atomic:") { - Some(rest) => match rest.split_once(':') { - Some((kind, local)) => format!("{kind}:{}", local.to_uppercase()), - None => rest.to_uppercase(), - }, - None => urn.to_string(), - } +fn fts_hits(node: &KgNode, query: &str) -> usize { + let query: HashSet<_> = tokenize_for_fts(query).into_iter().collect(); + let tokens: HashSet<_> = tokenize_for_fts(&format!( + "{} {} {}", + node.id, + node.label, + node.summary.as_deref().unwrap_or_default() + )) + .into_iter() + .collect(); + query.intersection(&tokens).count() } - -/// The label for a `:` KG id is the part after the first colon. -fn kg_label(id: &str) -> &str { - id.split_once(':').map(|(_, l)| l).unwrap_or(id) -} - -/// Canonicalize a `:::ref{to=}` target to an intent URN. -/// -/// Every `:::ref` edge kind is intent→intent (BLOCKED_BY, DEPENDS_ON, -/// REMEDIATES, REVIEWS), so a target written without a `urn:` scheme is an -/// intent named bare — a ULID or a human key. -/// -/// Without this a bare target fell through [`rdf_target_to_kg_id`] -/// untouched and the edge landed on `01ABC…` while every reader looks for -/// `intent:01ABC…`. The edge existed and matched nothing, so triage's -/// `UNREVIEWED_CHANGE` could never clear — and the bare form is exactly -/// what `atomic intent new --review ` scaffolds. -fn canonical_ref_target(target: &str) -> String { - let t = target.trim(); - if t.starts_with("urn:") { - t.to_string() +fn path_penalty(path: &str) -> u64 { + if path.contains("test") { + 80 + } else if path.starts_with("src/") || path.starts_with("lib/") { + 0 + } else if path.ends_with(".md") { + 120 } else { - format!("urn:atomic:intent:{t}") - } -} - -fn rdf_target_to_kg_id(target: &str, manifest: Option<&VaultManifest>) -> String { - if let Some(id) = target.strip_prefix("urn:atomic:memory:") { - return format!("memory:{id}"); - } - if let Some(id) = target.strip_prefix("urn:atomic:intent:") { - if let Some(manifest) = manifest { - // The canonical intent URN carries the ULID `uid`; match on it - // first, then fall back to a direct/legacy human-key match. - let summary = manifest - .intents - .values() - .find(|s| s.uid.eq_ignore_ascii_case(id)) - .or_else(|| manifest.intents.get(id)) - .or_else(|| manifest.intents.get(&id.to_uppercase())); - if let Some(summary) = summary { - if !summary.vault_path.is_empty() { - return entry_subject(&summary.vault_path, VaultEntryType::Intent); - } - } - } - return format!("intent:{}", id.to_uppercase()); - } - if let Some(hash) = target.strip_prefix("urn:atomic:change:") { - return format!("change:{}", hash.chars().take(12).collect::()); - } - // Intent-decomposition children (`ac`, `task`, `scope`, `constraint`) are - // ULID-derived and canonicalized to an UPPERCASE local so they always share - // the intent node's case (intents are uppercase ULIDs). This makes a - // memory's `derivedFrom` link land on the real node regardless of the case - // the id was copied in — `project_intent_semantics` mints the same way. - for kind in ["ac", "task", "scope", "constraint"] { - if target.starts_with(&format!("urn:atomic:{kind}:")) { - return child_kg_id(target); - } - } - // Other canonical urns (e.g. `todo`, whose ids are author-chosen, not - // ULID-derived) keep their authored case. Non-urn targets pass through - // unchanged (`urn_to_kg_id` only strips the `urn:atomic:` prefix). - urn_to_kg_id(target) -} - -/// Extract edges from content text (wiki-links, file paths). -fn extract_content_edges( - subject: &str, - entry_type: VaultEntryType, - content: &str, - edges: &mut Vec, -) { - // Extract [[wiki-links]] — always REFERENCES regardless of entry type - let mut pos = 0; - while let Some(start) = content[pos..].find("[[") { - let abs_start = pos + start + 2; - if let Some(end) = content[abs_start..].find("]]") { - let link = &content[abs_start..abs_start + end]; - if !link.is_empty() && !link.contains('\n') { - edges.push(KgEdge::new( - subject, - format!("memory:{}", link), - edge_kind::REFERENCES, - )); - } - pos = abs_start + end + 2; - } else { - break; - } - } - - // File paths: ToolResult content represents files the agent actually read/used, - // so emit USED edges. All other entry types emit REFERENCES edges. - let file_edge_kind = if entry_type == VaultEntryType::ToolResult { - predicate::USED - } else { - edge_kind::REFERENCES - }; - - for word in content.split_whitespace() { - let clean = word - .trim_start_matches(['`', '"', '\'', '(', '[', '{', '<']) - .trim_end_matches([ - '`', '"', '\'', ')', ',', ']', '}', '>', '.', ';', ':', '!', '?', - ]); - if looks_like_file_path(clean) { - edges.push(KgEdge::new( - subject, - format!("file:{}", clean), - file_edge_kind, - )); - } - } -} - -/// Heuristic: does this string look like a file path? -fn looks_like_file_path(s: &str) -> bool { - if s.len() < 4 || s.len() > 200 { - return false; - } - // Must contain a slash and a dot (extension) - if !s.contains('/') || !s.contains('.') { - return false; - } - // Common code file extensions - let extensions = [ - ".rs", ".ts", ".js", ".py", ".go", ".java", ".md", ".toml", ".yaml", ".yml", ".json", - ".html", ".css", - ]; - extensions.iter().any(|ext| s.ends_with(ext)) -} - -#[cfg(test)] -mod tests { - use super::*; - use tempfile::tempdir; - - #[test] - fn rdf_target_canonicalizes_child_urns_to_uppercase_kg_ids() { - // Intent-decomposition children are uppercased on the local part, so a - // `derivedFrom` link connects to the minted node regardless of the case - // the id was copied in (the uid may be UPPER, as ULIDs are, or lower). - assert_eq!( - rdf_target_to_kg_id("urn:atomic:ac:demo-2-ac-1", None), - "ac:DEMO-2-AC-1" - ); - assert_eq!( - rdf_target_to_kg_id("urn:atomic:ac:DEMO-2-ac-1", None), - "ac:DEMO-2-AC-1" - ); - assert_eq!( - rdf_target_to_kg_id("urn:atomic:task:demo-2-1", None), - "task:DEMO-2-1" - ); - assert_eq!( - rdf_target_to_kg_id("urn:atomic:scope:demo-2-scope-in-1", None), - "scope:DEMO-2-SCOPE-IN-1" - ); - // Todos are author-chosen ids, not ULID-derived — case is preserved. - assert_eq!(rdf_target_to_kg_id("urn:atomic:todo:t2", None), "todo:t2"); - // Existing special cases are unchanged. - assert_eq!( - rdf_target_to_kg_id("urn:atomic:memory:abc", None), - "memory:abc" - ); - assert_eq!( - rdf_target_to_kg_id("urn:atomic:change:0123456789abcdef", None), - "change:0123456789ab", - ); - // Non-urn targets pass through untouched. - assert_eq!( - rdf_target_to_kg_id("file:src/main.rs", None), - "file:src/main.rs" - ); - } - - #[test] - fn memory_derived_from_intent_link_resolves_to_intent_kg_subject() { - // A decision memory that links to an intent by canonical urn must, once - // extracted, carry a `wasDerivedFrom` edge to the intent's real - // path-keyed KG subject (resolved through the manifest) — not the bare - // `intent:` fallback — so the lineage edge actually connects. - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - let created = repo - .vault_intent_create(crate::IntentCreateOptions { - title: "Linkable intent".to_string(), - priority: None, - assignee: None, - labels: Vec::new(), - session_id: None, - turn_id: None, - kind: None, - }) - .unwrap(); - let intent_subject = entry_subject(&created.intent_file, VaultEntryType::Intent); - - let mem_path = "memory/decisions/dtest.md"; - let fm = serde_json::json!({ - "type": "decision", - "derivedFrom": [format!("urn:atomic:intent:{}", created.uid)], - }) - .to_string(); - repo.vault_store( - mem_path, - VaultEntryType::Memory, - b"# Decision\nChose X over Y.\n".to_vec(), - fm, - ) - .unwrap(); - let entry = repo.vault_retrieve(mem_path).unwrap().unwrap(); - - let (_nodes, edges) = repo.vault_extract_kg(mem_path, &entry).unwrap(); - let derived = edges - .iter() - .find(|e| e.kind == predicate::WAS_DERIVED_FROM) - .expect("derivedFrom edge present"); - - // The lineage edge lands on the intent's real path-keyed KG subject - // (resolved through the manifest), so `criterion/intent → decision` - // traversal connects. In the uid-based layout this coincides with the - // bare fallback; in the `manual//` layout the manifest - // resolution is what makes them agree. - assert_eq!(derived.to_id, intent_subject); - assert_eq!( - derived - .metadata - .as_ref() - .and_then(|m| m.get("rdf_target")) - .and_then(|v| v.as_str()), - Some(format!("urn:atomic:intent:{}", created.uid).as_str()) - ); - } - - #[test] - fn test_extract_kg_from_goal() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - let entry = VaultEntry::new( - VaultEntryType::Session, - b"# Goal\nLooking at src/auth.rs\n".to_vec(), - r#"{"developer":"alice","intent":"PIMO-1","status":"active"}"#.to_string(), - "2025-01-01T00:00:00Z".to_string(), - ); - - let (nodes, edges) = repo - .vault_extract_kg("goals/test-goal/_goal.md", &entry) - .unwrap(); - - // Should have a goal node - assert_eq!(nodes.len(), 1); - assert_eq!(nodes[0].kind, "goal"); - assert_eq!(nodes[0].id, "goal:test-goal"); - assert_eq!(nodes[0].source, "vault"); - - // Should have edges for developer, intent, file reference - assert!( - edges - .iter() - .any(|e| e.kind == edge_kind::AUTHORED_BY && e.to_id == "identity:alice"), - "Missing AUTHORED_BY edge" - ); - assert!( - edges - .iter() - .any(|e| e.kind == edge_kind::LINKED_INTENT && e.to_id == "intent:PIMO-1"), - "Missing LINKED_INTENT edge" - ); - assert!( - edges - .iter() - .any(|e| e.kind == edge_kind::REFERENCES && e.to_id.contains("auth.rs")), - "Missing REFERENCES edge for file path" - ); - } - - #[test] - fn test_extract_kg_from_intent() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - let entry = VaultEntry::new( - VaultEntryType::Intent, - b"# Fix auth\n\nSee [[architecture]]\n".to_vec(), - r#"{"title":"Fix auth","status":"in-progress","priority":"high","assignee":"bob","labels":["auth","security"],"informedBy":["urn:atomic:memory:architecture"]}"#.to_string(), - "2025-01-01T00:00:00Z".to_string(), - ); - - let (nodes, edges) = repo - .vault_extract_kg("intents/pimo-1/intent.md", &entry) - .unwrap(); - - // 1 intent node + 1 concept node per label ("auth", "security") - assert_eq!(nodes.len(), 3); - let intent_node = nodes.iter().find(|n| n.kind == "intent").unwrap(); - assert_eq!(intent_node.id, "intent:PIMO-1"); - assert_eq!(intent_node.summary.as_deref(), Some("Fix auth")); - assert!(nodes - .iter() - .any(|n| n.kind == "concept" && n.id == "concept:auth")); - assert!(nodes - .iter() - .any(|n| n.kind == "concept" && n.id == "concept:security")); - - assert!( - edges - .iter() - .any(|e| e.kind == edge_kind::ASSIGNED_TO && e.to_id == "identity:bob"), - "Missing ASSIGNED_TO edge" - ); - assert!( - edges - .iter() - .any(|e| e.kind == edge_kind::HAS_LABEL && e.to_id == "concept:auth"), - "Missing HAS_LABEL edge for auth" - ); - assert!( - edges - .iter() - .any(|e| e.kind == edge_kind::HAS_LABEL && e.to_id == "concept:security"), - "Missing HAS_LABEL edge for security" - ); - // Wiki-link → REFERENCES edge - assert!( - edges - .iter() - .any(|e| e.kind == edge_kind::REFERENCES && e.to_id == "memory:architecture"), - "Missing REFERENCES edge for wiki-link" - ); - let informed = edges - .iter() - .find(|e| e.kind == predicate::INFORMED_BY) - .expect("Missing atom:informedBy edge"); - assert_eq!(informed.to_id, "memory:architecture"); - assert_eq!( - informed.metadata.as_ref().unwrap()["rdf_target"], - "urn:atomic:memory:architecture" - ); - } - - #[test] - fn test_extract_kg_projects_directive_body_semantics() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - // A directive-based intent body: two acceptance criteria, one task that - // satisfies both and touches two files, plus scope and a constraint. - let body = "\ -:::why -Because it matters. -::: - -:::acceptance-criterion{#demo-2-ac-1 status=unmet} -Build succeeds. -::: - -:::acceptance-criterion{#demo-2-ac-2 status=unmet} -Types are strict. -::: - -:::task{#demo-2-1 status=unmet criteria=demo-2-ac-1,demo-2-ac-2} -Do the work. -::file-ref{path=package.json} -::file-ref{path=src/index.ts} -::: - -:::scope-in -The entry point. -::: - -:::scope-out -The build system. -::: - -:::constraint -No new dependencies. -:::"; - let entry = VaultEntry::new( - VaultEntryType::Intent, - body.as_bytes().to_vec(), - r#"{"id":"DEMO-2","title":"Interactive CLI","status":"in-progress"}"#.to_string(), - "2025-01-01T00:00:00Z".to_string(), - ); - - let (nodes, edges) = repo - .vault_extract_kg("intents/demo-2/intent.md", &entry) - .unwrap(); - - let has_node = |id: &str, kind: &str| nodes.iter().any(|n| n.id == id && n.kind == kind); - let has_edge = |from: &str, to: &str, kind: &str| { - edges - .iter() - .any(|e| e.from_id == from && e.to_id == to && e.kind == kind) - }; - - // Acceptance criteria and tasks are their own nodes (uppercase local, - // matching the intent node's case). - assert!(has_node("ac:DEMO-2-AC-1", "acceptance_criterion")); - assert!(has_node("ac:DEMO-2-AC-2", "acceptance_criterion")); - assert!(has_node("task:DEMO-2-1", "task")); - - // Intent → task / criterion structure. - assert!(has_edge( - "intent:DEMO-2", - "task:DEMO-2-1", - edge_kind::HAS_TASK - )); - assert!(has_edge( - "intent:DEMO-2", - "ac:DEMO-2-AC-1", - edge_kind::HAS_ACCEPTANCE_CRITERION - )); - - // The task satisfies BOTH criteria (the multi-criteria fix) … - assert!(has_edge( - "task:DEMO-2-1", - "ac:DEMO-2-AC-1", - edge_kind::SATISFIES - )); - assert!(has_edge( - "task:DEMO-2-1", - "ac:DEMO-2-AC-2", - edge_kind::SATISFIES - )); - - // … and touches both files, reusing the shared file: node id - // (file ids are paths, not ULID-derived, so they stay case-preserving). - assert!(has_edge( - "task:DEMO-2-1", - "file:package.json", - edge_kind::TOUCHES - )); - assert!(has_edge( - "task:DEMO-2-1", - "file:src/index.ts", - edge_kind::TOUCHES - )); - - // Scope and constraint project too. - assert!(edges - .iter() - .any(|e| e.from_id == "intent:DEMO-2" && e.kind == edge_kind::HAS_SCOPE_IN)); - assert!(edges - .iter() - .any(|e| e.from_id == "intent:DEMO-2" && e.kind == edge_kind::HAS_SCOPE_OUT)); - assert!(edges - .iter() - .any(|e| e.from_id == "intent:DEMO-2" && e.kind == edge_kind::HAS_CONSTRAINT)); - - // Every projected edge carries the source path for replace-on-update. - for e in &edges { - assert_eq!( - e.metadata.as_ref().unwrap()["derived_from_vault_path"], - "intents/demo-2/intent.md" - ); - } - } - - #[test] - fn test_intent_child_nodes_are_cleaned_on_edit_and_delete() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - let fm = r#"{"id":"DEMO-2","title":"Demo","status":"in-progress"}"#; - let path = "intents/demo-2/intent.md"; - - let body_v1 = "\ -:::why -r -::: - -:::acceptance-criterion{#demo-2-ac-1 status=unmet} -crit -::: - -:::task{#demo-2-1 status=unmet criteria=demo-2-ac-1} -first -::: - -:::task{#demo-2-2 status=unmet criteria=demo-2-ac-1} -second -:::"; - repo.vault_store( - path, - VaultEntryType::Intent, - body_v1.as_bytes().to_vec(), - fm.to_string(), - ) - .unwrap(); - - let node_exists = |id: &str| -> bool { - let txn = repo.pristine().read_txn().unwrap(); - txn.get_kg_node(id).unwrap().is_some() - }; - - // Both tasks and the criterion are projected (uppercase local). - assert!(node_exists("task:DEMO-2-1")); - assert!(node_exists("task:DEMO-2-2")); - assert!(node_exists("ac:DEMO-2-AC-1")); - - // Edit: drop the second task. Its node must be garbage-collected. - let body_v2 = "\ -:::why -r -::: - -:::acceptance-criterion{#demo-2-ac-1 status=unmet} -crit -::: - -:::task{#demo-2-1 status=unmet criteria=demo-2-ac-1} -first -:::"; - repo.vault_store( - path, - VaultEntryType::Intent, - body_v2.as_bytes().to_vec(), - fm.to_string(), - ) - .unwrap(); - - assert!(node_exists("task:DEMO-2-1"), "surviving task must remain"); - assert!( - !node_exists("task:DEMO-2-2"), - "removed task must be cleaned up, not orphaned" - ); - assert!(node_exists("ac:DEMO-2-AC-1")); - - // Delete: every projected child node must be gone. - repo.vault_delete(path).unwrap(); - assert!(!node_exists("intent:DEMO-2")); - assert!(!node_exists("task:DEMO-2-1")); - assert!(!node_exists("ac:DEMO-2-AC-1")); - } - - #[test] - fn test_remediates_ref_projects_reverse_queryable_edge() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - // Intent B remediates intent A. The link is authored with the same - // `:::ref` leaf `blockedBy` uses, but `edge=remediates`. - let fm_b = r#"{"id":"DEMO-B","title":"Fix the escaped defect","status":"in-progress"}"#; - let body_b = "\ -:::why\n\ -Remediates a defect delivered by DEMO-A.\n\ -:::\n\n\ -:::ref{to=urn:atomic:intent:demo-a edge=remediates}\n\ -:::"; - let path_b = "intents/demo-b/intent.md"; - repo.vault_store( - path_b, - VaultEntryType::Intent, - body_b.as_bytes().to_vec(), - fm_b.to_string(), - ) - .unwrap(); - - // The projected edge is B --REMEDIATES--> A. With no manifest entry for - // A, the target resolves to the uppercased `intent:DEMO-A` convention - // (same as any unresolved intent ref). - let edge_exists = |from: &str, to: &str| -> bool { - let txn = repo.pristine().read_txn().unwrap(); - txn.get_kg_edges_from(from) - .unwrap() - .iter() - .any(|e| e.to_id == to && e.kind == edge_kind::REMEDIATES) - }; - assert!( - edge_exists("intent:DEMO-B", "intent:DEMO-A"), - "B --REMEDIATES--> A must project from the :::ref body directive" - ); - - // Reverse-queryable: neighbors of the remediated intent A surface the - // incoming REMEDIATES edge (kg_neighbors returns incoming + outgoing). - let sub = repo.vault_kg_neighbors("intent:DEMO-A", 1).unwrap(); - assert!( - sub.edges.iter().any(|e| e.from_id == "intent:DEMO-B" - && e.to_id == "intent:DEMO-A" - && e.kind == edge_kind::REMEDIATES), - "REMEDIATES must be reverse-queryable from the remediated intent" - ); - - // Re-index B: the edge survives (owned outgoing edge, replaced in place). - repo.vault_store( - path_b, - VaultEntryType::Intent, - body_b.as_bytes().to_vec(), - fm_b.to_string(), - ) - .unwrap(); - assert!( - edge_exists("intent:DEMO-B", "intent:DEMO-A"), - "edge must survive a re-index, not be orphaned or duplicated" - ); - - // Delete B: its outgoing REMEDIATES edge is cleaned up with the node. - repo.vault_delete(path_b).unwrap(); - assert!( - !edge_exists("intent:DEMO-B", "intent:DEMO-A"), - "edge must be removed when the remediation intent is deleted" - ); - } - - /// A `:::ref` target named BARE — no `urn:` scheme — must still land on - /// the intent node. - /// - /// This is the shape `atomic intent new --review ` scaffolds and - /// the shape triage's own remediation hint used to tell people to type. - /// It projected an edge to `01ABC…` while every reader looks for - /// `intent:01ABC…`, so the edge existed, matched nothing, and triage's - /// `UNREVIEWED_CHANGE` could never be cleared — the review gate was - /// unsatisfiable by the documented command. - #[test] - fn test_bare_ref_target_resolves_to_the_intent_node() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - let fm_r = - r#"{"id":"DEMO-R","title":"Review the work","status":"in-progress","kind":"review"}"#; - // Bare target: no `urn:atomic:intent:` prefix. - let body_r = "\ -:::why\n\ -Reviews DEMO-T.\n\ -:::\n\n\ -:::ref{to=DEMO-T edge=reviews}\n\ -:::"; - repo.vault_store( - "intents/demo-r/intent.md", - VaultEntryType::Intent, - body_r.as_bytes().to_vec(), - fm_r.to_string(), - ) - .unwrap(); - - let txn = repo.pristine().read_txn().unwrap(); - let edges = txn.get_kg_edges_from("intent:DEMO-R").unwrap(); - let reviews: Vec<&str> = edges - .iter() - .filter(|e| e.kind == edge_kind::REVIEWS) - .map(|e| e.to_id.as_str()) - .collect(); - assert_eq!( - reviews, - vec!["intent:DEMO-T"], - "a bare target must be namespaced, not passed through raw" - ); - } - - /// Slice-3 (b): a review intent's `:::ref{edge=reviews}` projects a - /// reverse-queryable `REVIEWS` KG edge (mirroring the remediates test). - #[test] - fn test_reviews_ref_projects_reverse_queryable_edge() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - // Review intent R reviews intent T, authored via `:::ref{edge=reviews}`. - let fm_r = - r#"{"id":"DEMO-R","title":"Review the work","status":"in-progress","kind":"review"}"#; - let body_r = "\ -:::why\n\ -Reviews DEMO-T.\n\ -:::\n\n\ -:::ref{to=urn:atomic:intent:demo-t edge=reviews}\n\ -:::"; - let path_r = "intents/demo-r/intent.md"; - repo.vault_store( - path_r, - VaultEntryType::Intent, - body_r.as_bytes().to_vec(), - fm_r.to_string(), - ) - .unwrap(); - - // The projected edge is R --REVIEWS--> T (T unresolved → intent:DEMO-T). - let edge_exists = |from: &str, to: &str| -> bool { - let txn = repo.pristine().read_txn().unwrap(); - txn.get_kg_edges_from(from) - .unwrap() - .iter() - .any(|e| e.to_id == to && e.kind == edge_kind::REVIEWS) - }; - assert!( - edge_exists("intent:DEMO-R", "intent:DEMO-T"), - "R --REVIEWS--> T must project from the :::ref body directive" - ); - - // Reverse-queryable: neighbors of the reviewed intent T surface it. - let sub = repo.vault_kg_neighbors("intent:DEMO-T", 1).unwrap(); - assert!( - sub.edges.iter().any(|e| e.from_id == "intent:DEMO-R" - && e.to_id == "intent:DEMO-T" - && e.kind == edge_kind::REVIEWS), - "REVIEWS must be reverse-queryable from the reviewed intent" - ); - - // Re-index R: the edge survives (owned outgoing edge, replaced in place). - repo.vault_store( - path_r, - VaultEntryType::Intent, - body_r.as_bytes().to_vec(), - fm_r.to_string(), - ) - .unwrap(); - assert!( - edge_exists("intent:DEMO-R", "intent:DEMO-T"), - "edge must survive a re-index" - ); - - // Delete R: its outgoing REVIEWS edge is cleaned up with the node. - repo.vault_delete(path_r).unwrap(); - assert!( - !edge_exists("intent:DEMO-R", "intent:DEMO-T"), - "edge must be removed when the review intent is deleted" - ); - } - - #[test] - fn test_scope_out_file_ref_projects_reverse_queryable_edge() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - // A `:::scope-out` naming a concrete file via `::file-ref` (the same leaf - // a `:::task` uses for TOUCHES). - let fm = r#"{"id":"DEMO-S","title":"Scope guard","status":"in-progress"}"#; - let body = "\ -:::why\n\ -Keep billing untouched.\n\ -:::\n\n\ -:::scope-out\n\ -The billing subsystem is off limits.\n\ -::file-ref{path=src/billing.rs}\n\ -:::"; - let path = "intents/demo-s/intent.md"; - repo.vault_store( - path, - VaultEntryType::Intent, - body.as_bytes().to_vec(), - fm.to_string(), - ) - .unwrap(); - - // intent --SCOPE_OUT_FILE--> file:src/billing.rs (the SHARED file node, - // same convention MODIFIES/TOUCHES use, so a change's MODIFIES target - // meets this out-of-scope declaration on one node). - let edge_exists = |from: &str, to: &str| -> bool { - let txn = repo.pristine().read_txn().unwrap(); - txn.get_kg_edges_from(from) - .unwrap() - .iter() - .any(|e| e.to_id == to && e.kind == edge_kind::SCOPE_OUT_FILE) - }; - assert!( - edge_exists("intent:DEMO-S", "file:src/billing.rs"), - "SCOPE_OUT_FILE edge must project from the scope-out file-ref" - ); - - // Additive: the prose HAS_SCOPE_OUT edge is still present alongside it. - { - let txn = repo.pristine().read_txn().unwrap(); - assert!( - txn.get_kg_edges_from("intent:DEMO-S") - .unwrap() - .iter() - .any(|e| e.kind == edge_kind::HAS_SCOPE_OUT), - "the prose HAS_SCOPE_OUT edge must remain (additive, not replaced)" - ); - } - - // Reverse-queryable: neighbors of the file node surface the incoming edge. - let sub = repo.vault_kg_neighbors("file:src/billing.rs", 1).unwrap(); - assert!( - sub.edges.iter().any(|e| e.from_id == "intent:DEMO-S" - && e.to_id == "file:src/billing.rs" - && e.kind == edge_kind::SCOPE_OUT_FILE), - "SCOPE_OUT_FILE must be reverse-queryable from the file node" - ); - - // Re-index: owned outgoing edge is replaced in place, not orphaned. - repo.vault_store( - path, - VaultEntryType::Intent, - body.as_bytes().to_vec(), - fm.to_string(), - ) - .unwrap(); - assert!( - edge_exists("intent:DEMO-S", "file:src/billing.rs"), - "edge must survive a re-index" - ); - - // Delete: the intent and its owned SCOPE_OUT_FILE edge are cleaned up. - repo.vault_delete(path).unwrap(); - assert!( - !edge_exists("intent:DEMO-S", "file:src/billing.rs"), - "edge must be removed when the intent is deleted" - ); - } - - #[test] - fn test_extract_kg_legacy_prose_intent_skips_projection() { - // A non-directive (legacy) body must not fail extraction; it simply - // produces no task/criterion nodes. - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - let entry = VaultEntry::new( - VaultEntryType::Intent, - b"# Legacy\n\n## Acceptance Criteria\n- [ ] a thing\n".to_vec(), - r#"{"id":"DEMO-3","title":"Legacy","status":"backlog"}"#.to_string(), - "2025-01-01T00:00:00Z".to_string(), - ); - - let (nodes, _edges) = repo - .vault_extract_kg("intents/demo-3/intent.md", &entry) - .unwrap(); - - assert!(nodes - .iter() - .all(|n| n.kind != "task" && n.kind != "acceptance_criterion")); - } - - #[test] - fn test_extract_learning_memory_lineage() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - let entry = VaultEntry::new( - VaultEntryType::Memory, - b"A reusable lesson learned from the accepted implementation.\n".to_vec(), - r#"{"uid":"learning-1","memoryKind":"lesson","status":"active","derivedFrom":["urn:atomic:intent:temp-2","urn:atomic:change:ABCDEFGHIJKLMNOPQRSTUVWXYZ","urn:atomic:memory:source-1"]}"#.to_string(), - "2026-07-16T00:00:00Z".to_string(), - ); - - let (_, edges) = repo - .vault_extract_kg("memory/learning-1.md", &entry) - .unwrap(); - let lineage: Vec<_> = edges - .iter() - .filter(|edge| edge.kind == predicate::WAS_DERIVED_FROM) - .map(|edge| edge.to_id.as_str()) - .collect(); - assert_eq!( - lineage, - vec!["intent:TEMP-2", "change:ABCDEFGHIJKL", "memory:source-1"] - ); - } - - #[test] - fn test_learning_intent_rdf_id_resolves_to_path_keyed_kg_node() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - let intent = repo - .vault_intent_create(IntentCreateOptions { - title: "Demo lineage".to_string(), - priority: None, - assignee: None, - labels: Vec::new(), - session_id: None, - turn_id: None, - kind: None, - }) - .unwrap(); - // The canonical intent URN is keyed by the ULID uid, not the human key. - let canonical_intent = format!("urn:atomic:intent:{}", intent.uid.to_lowercase()); - let frontmatter = serde_json::json!({ - "uid": "learning-path-resolution", - "memoryKind": "lesson", - "status": "active", - "derivedFrom": [canonical_intent.clone()] - }); - let entry = VaultEntry::new( - VaultEntryType::Memory, - b"A learned fact.\n".to_vec(), - frontmatter.to_string(), - "2026-07-16T00:00:00Z".to_string(), - ); - - let (_, edges) = repo - .vault_extract_kg("memory/learning-path-resolution.md", &entry) - .unwrap(); - let derived = edges - .iter() - .find(|edge| edge.kind == predicate::WAS_DERIVED_FROM) - .expect("learning should link to its Intent"); - assert_eq!( - derived.to_id, - entry_subject(&intent.intent_file, VaultEntryType::Intent) - ); - assert_eq!( - derived.metadata.as_ref().unwrap()["rdf_target"], - canonical_intent - ); - } - - #[test] - fn test_kg_search() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - // Store and index - repo.vault_store( - "memory/architecture.md", - VaultEntryType::Memory, - b"# Architecture\n".to_vec(), - r#"{"name":"architecture"}"#.to_string(), - ) - .unwrap(); - repo.vault_index_kg("memory/architecture.md").unwrap(); - - // FTS search - let results = repo.vault_kg_search("architecture", 10, None).unwrap(); - assert!(!results.is_empty(), "FTS search should return results"); - assert_eq!(results[0].kind, "memory"); - } - - #[test] - fn test_kg_neighbors() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - // Store a goal with edges - repo.vault_store( - "goals/abc/_goal.md", - VaultEntryType::Session, - b"# Goal\n".to_vec(), - r#"{"developer":"alice","intent":"PIMO-1"}"#.to_string(), - ) - .unwrap(); - repo.vault_index_kg("goals/abc/_goal.md").unwrap(); - - let sg = repo.vault_kg_neighbors("goal:abc", 1).unwrap(); - assert!(!sg.is_empty(), "Neighborhood should not be empty"); - } - - #[test] - fn test_kg_counts() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - repo.vault_store( - "memory/test.md", - VaultEntryType::Memory, - b"# Test\n".to_vec(), - r#"{"name":"test"}"#.to_string(), - ) - .unwrap(); - repo.vault_index_kg("memory/test.md").unwrap(); - - assert!(repo.vault_kg_node_count().unwrap() > 0); - } - - #[test] - fn test_kg_store_replaces_old_data() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - let path = "goals/abc/_goal.md"; - repo.vault_store( - path, - VaultEntryType::Session, - b"Used src/main.rs".to_vec(), - r#"{"developer":"alice"}"#.to_string(), - ) - .unwrap(); - let first_entry = repo.vault_retrieve(path).unwrap().unwrap(); - let (first_nodes, first_edges) = repo.vault_extract_kg(path, &first_entry).unwrap(); - assert!( - repo.vault_store_kg(path, &first_nodes, &first_edges) - .unwrap() - > 0 - ); - - repo.vault_store( - path, - VaultEntryType::Session, - b"Used src/lib.rs".to_vec(), - r#"{"developer":"alice"}"#.to_string(), - ) - .unwrap(); - - // A delayed caller cannot write triples extracted from the old source. - assert!(repo - .vault_store_kg(path, &first_nodes, &first_edges) - .is_err()); - - let current_entry = repo.vault_retrieve(path).unwrap().unwrap(); - let (current_nodes, current_edges) = repo.vault_extract_kg(path, ¤t_entry).unwrap(); - repo.vault_store_kg(path, ¤t_nodes, ¤t_edges) - .unwrap(); - - let subgraph = repo.vault_kg_neighbors("goal:abc", 1).unwrap(); - assert!(subgraph - .edges - .iter() - .any(|edge| edge.to_id == "file:src/lib.rs")); - assert!(!subgraph - .edges - .iter() - .any(|edge| edge.to_id == "file:src/main.rs")); - } - - #[test] - fn test_tool_result_update_and_delete_clean_derived_data() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - let path = "goals/demo/toolu_123.md"; - let node_id = entry_subject(path, VaultEntryType::ToolResult); - let previous_node_id = entry_subject(path, VaultEntryType::Memory); - - // Reclassifying an existing path must remove the identity derived from - // the previous entry type before the new node is indexed. - repo.vault_store( - path, - VaultEntryType::Memory, - b"temporary memory".to_vec(), - r#"{"name":"temporary"}"#.to_string(), - ) - .unwrap(); - assert!(repo.vault_kg_node(&previous_node_id).unwrap().is_some()); - - repo.vault_store( - path, - VaultEntryType::ToolResult, - b"read src/auth.rs".to_vec(), - r#"{"description":"staletooltoken"}"#.to_string(), - ) - .unwrap(); - - assert!(repo.vault_kg_node(&previous_node_id).unwrap().is_none()); - assert!(repo.vault_kg_node(&node_id).unwrap().is_some()); - assert!(repo - .vault_kg_search("staletooltoken", 10, None) - .unwrap() - .iter() - .any(|node| node.id == node_id)); - let stale_entry = repo.vault_retrieve(path).unwrap().unwrap(); - let (stale_nodes, stale_edges) = repo.vault_extract_kg(path, &stale_entry).unwrap(); - - repo.vault_store( - path, - VaultEntryType::ToolResult, - b"read src/main.rs".to_vec(), - r#"{"description":"currenttooltoken"}"#.to_string(), - ) - .unwrap(); - - // A slow indexer that extracted the previous entry must not overwrite - // the newer KG state after the source update has committed. - assert_eq!( - repo.vault_store_kg_inner(path, &stale_nodes, &stale_edges, Some(&stale_entry),) - .unwrap(), - 0 - ); - - assert!(repo - .vault_kg_search("staletooltoken", 10, None) - .unwrap() - .is_empty()); - assert!(repo - .vault_kg_search("currenttooltoken", 10, None) - .unwrap() - .iter() - .any(|node| node.id == node_id)); - let graph = repo.vault_kg_neighbors(&node_id, 1).unwrap(); - assert!(!graph - .edges - .iter() - .any(|edge| edge.to_id == "file:src/auth.rs")); - assert!(graph - .edges - .iter() - .any(|edge| edge.to_id == "file:src/main.rs")); - - assert!(repo.vault_delete(path).unwrap()); - assert!(repo.vault_kg_node(&node_id).unwrap().is_none()); - assert!(repo - .vault_kg_search("currenttooltoken", 10, None) - .unwrap() - .is_empty()); - } - - #[test] - fn test_vault_node_update_preserves_incoming_relationships() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - repo.vault_store( - "memory/target.md", - VaultEntryType::Memory, - b"Original target knowledge".to_vec(), - r#"{"name":"target"}"#.to_string(), - ) - .unwrap(); - repo.vault_store( - "memory/source.md", - VaultEntryType::Memory, - b"This decision uses [[target]]".to_vec(), - r#"{"name":"source"}"#.to_string(), - ) - .unwrap(); - - let incoming_exists = |repo: &Repository| { - repo.vault_kg_neighbors("memory:source", 1) - .unwrap() - .edges - .iter() - .any(|edge| { - edge.from_id == "memory:source" - && edge.to_id == "memory:target" - && edge.kind == edge_kind::REFERENCES - }) - }; - assert!(incoming_exists(&repo)); - - // Replacing the target's metadata/body must not erase another Vault - // entry's RDF/KG relationship to that same resource. - repo.vault_store( - "memory/target.md", - VaultEntryType::Memory, - b"Updated target knowledge".to_vec(), - r#"{"name":"target","description":"current target"}"#.to_string(), - ) - .unwrap(); - assert!(incoming_exists(&repo)); - - // A real delete still cascades the now-dangling relationship. - assert!(repo.vault_delete("memory/target.md").unwrap()); - assert!(!incoming_exists(&repo)); - } - - #[test] - fn test_vault_node_update_removes_owned_reverse_relationships() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - let path = "intents/demo/intent.md"; - - repo.vault_store( - path, - VaultEntryType::Intent, - b"# Demo".to_vec(), - r#"{"title":"Demo","blocked_by":["BLOCKER"]}"#.to_string(), - ) - .unwrap(); - let has_reverse_block = |repo: &Repository| { - repo.vault_kg_neighbors("intent:DEMO", 1) - .unwrap() - .edges - .iter() - .any(|edge| { - edge.from_id == "intent:BLOCKER" - && edge.to_id == "intent:DEMO" - && edge.kind == edge_kind::BLOCKS - }) - }; - assert!(has_reverse_block(&repo)); - - repo.vault_store( - path, - VaultEntryType::Intent, - b"# Demo".to_vec(), - r#"{"title":"Demo","blocked_by":[]}"#.to_string(), - ) - .unwrap(); - assert!(!has_reverse_block(&repo)); - } - - #[test] - fn test_current_node_validation_rejects_obsolete_fts_term() { - let node = KgNode::new("memory:payment-decision", "memory", "payments", "vault") - .with_summary("Use payment service v2"); - let old_query: std::collections::HashSet = - tokenize_for_fts("authentication").into_iter().collect(); - let current_query: std::collections::HashSet = - tokenize_for_fts("payments").into_iter().collect(); - - assert_eq!(kg_node_fts_hit_count(&node, &old_query), 0); - assert_eq!(kg_node_fts_hit_count(&node, ¤t_query), 1); - } - - #[test] - fn test_looks_like_file_path() { - assert!(looks_like_file_path("src/auth.rs")); - assert!(looks_like_file_path("crates/pi-agent/src/lib.rs")); - assert!(!looks_like_file_path("hello")); - assert!(!looks_like_file_path("http://example.com")); - assert!(!looks_like_file_path("a")); - } - - #[test] - fn test_entry_subject() { - assert_eq!( - entry_subject("goals/abc/_goal.md", VaultEntryType::Session), - "goal:abc" - ); - assert_eq!( - entry_subject("intents/pimo-1/intent.md", VaultEntryType::Intent), - "intent:PIMO-1" - ); - assert_eq!( - entry_subject("memory/architecture.md", VaultEntryType::Memory), - "memory:architecture" - ); - assert_eq!( - entry_subject("skills/run-tests.md", VaultEntryType::Skill), - "skill:run-tests" - ); - assert_eq!( - entry_subject( - "attestations/PIMO-1/attested.md", - VaultEntryType::Attestation - ), - "attestation:PIMO-1" - ); - } - - #[test] - fn test_path_to_subject() { - assert_eq!(path_to_subject("goals/abc/_goal.md"), "goal:abc"); - assert_eq!(path_to_subject("intents/pimo-1/intent.md"), "intent:PIMO-1"); - assert_eq!( - path_to_subject("memory/architecture.md"), - "memory:architecture" - ); - assert_eq!(path_to_subject("skills/run-tests.md"), "skill:run-tests"); - assert_eq!( - path_to_subject("scratch/notes.md"), - "vault:scratch:notes.md" - ); - } - - #[test] - fn test_extract_content_edges_wiki_links() { - let mut edges = Vec::new(); - extract_content_edges( - "goal:test", - VaultEntryType::Session, - "Check [[architecture]] and [[auth-design]] for details.", - &mut edges, - ); - assert!( - edges - .iter() - .any(|e| e.kind == edge_kind::REFERENCES && e.to_id == "memory:architecture"), - "Missing wiki-link edge for architecture" - ); - assert!( - edges - .iter() - .any(|e| e.kind == edge_kind::REFERENCES && e.to_id == "memory:auth-design"), - "Missing wiki-link edge for auth-design" - ); - } - - #[test] - fn test_extract_content_edges_file_paths() { - let mut edges = Vec::new(); - extract_content_edges( - "goal:test", - VaultEntryType::Session, - "Modified `src/main.rs` and crates/core/src/lib.rs today.", - &mut edges, - ); - assert!( - edges - .iter() - .any(|e| e.kind == edge_kind::REFERENCES && e.to_id == "file:src/main.rs"), - "Missing file path edge for src/main.rs" - ); - assert!( - edges.iter().any( - |e| e.kind == edge_kind::REFERENCES && e.to_id == "file:crates/core/src/lib.rs" - ), - "Missing file path edge for crates/core/src/lib.rs" - ); - } - - #[test] - fn test_extract_content_edges_trims_sentence_punctuation() { - let mut edges = Vec::new(); - extract_content_edges( - "memory:auth", - VaultEntryType::Memory, - "See src/auth.rs. Also check [crates/core/src/lib.rs].", - &mut edges, - ); - assert!(edges.iter().any(|edge| edge.to_id == "file:src/auth.rs")); - assert!(edges - .iter() - .any(|edge| edge.to_id == "file:crates/core/src/lib.rs")); - } - - #[test] - fn test_extract_scratch_returns_empty() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - let entry = VaultEntry::new( - VaultEntryType::Scratch, - b"just some scratch notes".to_vec(), - "{}".to_string(), - "2025-01-01T00:00:00Z".to_string(), - ); - - let (nodes, edges) = repo.vault_extract_kg("scratch/temp.md", &entry).unwrap(); - assert!(nodes.is_empty()); - assert!(edges.is_empty()); - } - - #[test] - fn test_extract_attestation_returns_empty() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - // A signed attestation blob contributes ZERO KG nodes/edges (the kind - // match early-returns empty, mirroring Scratch). - let entry = VaultEntry::new( - VaultEntryType::Attestation, - b"{\"@id\":\"atomic:intent:PIMO-1\"}\n".to_vec(), - r#"{"intentId":"PIMO-1"}"#.to_string(), - "2025-01-01T00:00:00Z".to_string(), - ); - - let (nodes, edges) = repo - .vault_extract_kg("attestations/PIMO-1/attested.md", &entry) - .unwrap(); - assert!(nodes.is_empty()); - assert!(edges.is_empty()); - } - - #[test] - fn test_frontmatter_intent_blocked_by() { - let mut node = KgNode::new("intent:PIMO-1", "intent", "PIMO-1", "vault"); - let mut edges = Vec::new(); - let mut fm = serde_json::Map::new(); - fm.insert( - "blocked_by".to_string(), - serde_json::json!(["PIMO-2", "PIMO-3"]), - ); - let mut nodes = Vec::new(); - extract_frontmatter_kg( - "intent:PIMO-1", - VaultEntryType::Intent, - &fm, - None, - &mut node, - &mut nodes, - &mut edges, - ); - assert!( - edges - .iter() - .any(|e| e.kind == edge_kind::BLOCKED_BY && e.to_id == "intent:PIMO-2"), - "Missing BLOCKED_BY edge for PIMO-2" - ); - assert!( - edges - .iter() - .any(|e| e.kind == edge_kind::BLOCKED_BY && e.to_id == "intent:PIMO-3"), - "Missing BLOCKED_BY edge for PIMO-3" - ); - // Reverse BLOCKS edges should also be present - assert!( - edges.iter().any(|e| e.kind == edge_kind::BLOCKS - && e.from_id == "intent:PIMO-2" - && e.to_id == "intent:PIMO-1"), - "Missing reverse BLOCKS edge from PIMO-2" - ); - } - - #[test] - fn test_frontmatter_intent_goals() { - let mut node = KgNode::new("intent:PIMO-1", "intent", "PIMO-1", "vault"); - let mut edges = Vec::new(); - let mut fm = serde_json::Map::new(); - fm.insert( - "goals".to_string(), - serde_json::json!(["swift-meadow-a3f2"]), - ); - let mut nodes = Vec::new(); - extract_frontmatter_kg( - "intent:PIMO-1", - VaultEntryType::Intent, - &fm, - None, - &mut node, - &mut nodes, - &mut edges, - ); - assert!( - edges - .iter() - .any(|e| e.kind == edge_kind::LINKED_GOAL && e.to_id == "goal:swift-meadow-a3f2"), - "Missing LINKED_GOAL edge" - ); - } - - #[test] - fn test_wiki_link_with_newline_ignored() { - let mut edges = Vec::new(); - extract_content_edges( - "goal:test", - VaultEntryType::Session, - "See [[bad\nlink]] here.", - &mut edges, - ); - assert!( - edges.iter().all(|e| e.kind != edge_kind::REFERENCES), - "Should not extract wiki-link with newline" - ); - } - - #[test] - fn test_unclosed_wiki_link_ignored() { - let mut edges = Vec::new(); - extract_content_edges( - "goal:test", - VaultEntryType::Session, - "See [[unclosed link here.", - &mut edges, - ); - assert!( - edges.iter().all(|e| e.kind != edge_kind::REFERENCES), - "Should not extract unclosed wiki-link" - ); - } - - #[test] - fn test_vault_index_kg() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - repo.vault_store( - "memory/architecture.md", - VaultEntryType::Memory, - b"# Architecture\n".to_vec(), - r#"{"name":"architecture","type":"project"}"#.to_string(), - ) - .unwrap(); - - let count = repo.vault_index_kg("memory/architecture.md").unwrap(); - assert!(count > 0, "Should have indexed at least one node"); - - let nc = repo.vault_kg_node_count().unwrap(); - assert!(nc > 0, "Should have at least one node in KG"); - } - - #[test] - fn test_memory_node_summary_from_name() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - let entry = VaultEntry::new( - VaultEntryType::Memory, - b"# Architecture decisions\n".to_vec(), - r#"{"name":"architecture"}"#.to_string(), - "2025-01-01T00:00:00Z".to_string(), - ); - - let (nodes, _edges) = repo - .vault_extract_kg("memory/architecture.md", &entry) - .unwrap(); - assert_eq!(nodes.len(), 1); - assert_eq!(nodes[0].kind, "memory"); - assert_eq!(nodes[0].summary.as_deref(), Some("architecture")); - assert_eq!( - nodes[0] - .metadata - .as_ref() - .and_then(|metadata| metadata.get("vault_path")) - .and_then(|path| path.as_str()), - Some("memory/architecture.md") - ); - } - - #[test] - fn test_kg_search_by_kind_filters_before_top_n() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - let files: Vec = (0..20) - .map(|index| { - KgNode::new( - format!("file:src/authentication-{index}.rs"), - "file", - format!("authentication-{index}.rs"), - "test", - ) - }) - .collect(); - let mut txn = repo.pristine.write_txn().unwrap(); - txn.init_kg().unwrap(); - for node in &files { - txn.upsert_kg_node(node).unwrap(); - } - txn.commit().unwrap(); - repo.vault_store( - "memory/authentication.md", - VaultEntryType::Memory, - b"Current authentication decision".to_vec(), - r#"{"name":"authentication","status":"active"}"#.to_string(), - ) - .unwrap(); - - let mixed = repo.vault_kg_search("authentication", 5, None).unwrap(); - assert!(mixed.iter().all(|node| node.kind == "file")); - - let memories = repo - .vault_kg_search_by_kind("authentication", 5, "memory") - .unwrap(); - assert_eq!(memories.len(), 1); - assert_eq!(memories[0].id, "memory:authentication"); - } - - #[test] - fn test_kg_search_generous_pool_surfaces_mixed_kinds() { - // Regression test: with a generous candidate pool the diversity-aware - // selection must surface lower-scored vault kinds (e.g. memory) - // alongside high-scored files, instead of returning only files. This - // mirrors the CLI default search path, which passes a large pool so an - // enriched repo does not collapse to file-only results. - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - let files: Vec = (0..20) - .map(|index| { - KgNode::new( - format!("file:src/authentication-{index}.rs"), - "file", - format!("authentication-{index}.rs"), - "test", - ) - }) - .collect(); - let mut txn = repo.pristine.write_txn().unwrap(); - txn.init_kg().unwrap(); - for node in &files { - txn.upsert_kg_node(node).unwrap(); - } - txn.commit().unwrap(); - repo.vault_store( - "memory/authentication.md", - VaultEntryType::Memory, - b"Current authentication decision".to_vec(), - r#"{"name":"authentication","status":"active"}"#.to_string(), - ) - .unwrap(); - - // pool > limit engages diversity-aware selection. - let mixed = repo - .vault_kg_search("authentication", 5, Some(5000)) - .unwrap(); - assert!( - mixed.iter().any(|node| node.kind == "memory"), - "a generous pool should surface the memory node alongside files, got: {:?}", - mixed.iter().map(|n| &n.id).collect::>() - ); - assert!( - mixed.iter().any(|node| node.kind == "file"), - "file nodes should still appear in the mixed results" - ); - } - - #[test] - fn test_kg_search_by_kind_rejects_legacy_stale_postings() { - let dir = tempdir().unwrap(); - let repo = Repository::init(dir.path()).unwrap(); - repo.init_vault().unwrap(); - - let mut txn = repo.pristine.write_txn().unwrap(); - txn.init_kg().unwrap(); - txn.upsert_kg_node(&KgNode::new( - "memory:decision", - "memory", - "authentication", - "test", - )) - .unwrap(); - txn.commit().unwrap(); - - // Legacy writers appended the new terms without removing the old - // "authentication" posting for the same node ID. - let mut txn = repo.pristine.write_txn().unwrap(); - txn.upsert_kg_node(&KgNode::new( - "memory:decision", - "memory", - "payments", - "test", - )) - .unwrap(); - txn.commit().unwrap(); - - assert!(repo - .vault_kg_search_by_kind("authentication", 5, "memory") - .unwrap() - .is_empty()); - let current = repo - .vault_kg_search_by_kind("payments", 5, "memory") - .unwrap(); - assert_eq!(current.len(), 1); - assert_eq!(current[0].id, "memory:decision"); + 20 } } diff --git a/atomic-repository/src/repository/views.rs b/atomic-repository/src/repository/views.rs index 774b9362..2380a303 100644 --- a/atomic-repository/src/repository/views.rs +++ b/atomic-repository/src/repository/views.rs @@ -115,6 +115,7 @@ impl Repository { parent_name: Option<&str>, seed_from: Option<&str>, ) -> Result<(), RepositoryError> { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; // The overlay chain is anchored on the nearest Shared ancestor of the // current view unless a parent is given. Using the nearest SHARED // ancestor (instead of the current view directly) prevents sibling @@ -195,6 +196,10 @@ impl Repository { txn.update_view(&new_view) .map_err(|e| RepositoryError::Database(e.to_string()))?; + if txn.kg_writes_enabled()? { + self.project_view_kg(&mut txn)?; + } + txn.commit() .map_err(|e| RepositoryError::Database(e.to_string()))?; @@ -210,6 +215,7 @@ impl Repository { /// /// Returns `ViewAlreadyExists` if the view already exists. pub fn create_shared_view(&mut self, name: &str) -> Result<(), RepositoryError> { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; ensure_workspace_dir(&self.dot_dir, name)?; let mut txn = self @@ -230,6 +236,10 @@ impl Repository { txn.create_view(name, ViewScope::Shared, None) .map_err(|e| RepositoryError::Database(e.to_string()))?; + if txn.kg_writes_enabled()? { + self.project_view_kg(&mut txn)?; + } + txn.commit() .map_err(|e| RepositoryError::Database(e.to_string()))?; @@ -241,6 +251,7 @@ impl Repository { /// This is useful after `git import` which creates Draft views by /// default. pub fn set_view_scope(&self, name: &str, scope: ViewScope) -> Result<(), RepositoryError> { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; let mut txn = self .pristine .write_txn() @@ -265,6 +276,10 @@ impl Repository { txn.update_view(&view) .map_err(|e| RepositoryError::Database(e.to_string()))?; + if txn.kg_writes_enabled()? { + self.project_view_kg(&mut txn)?; + } + txn.commit() .map_err(|e| RepositoryError::Database(e.to_string()))?; @@ -289,6 +304,7 @@ impl Repository { scope: ViewScope, parent_name: Option<&str>, ) -> Result<(), RepositoryError> { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; let mut txn = self .pristine .write_txn() @@ -323,6 +339,10 @@ impl Repository { txn.update_view(&view) .map_err(|e| RepositoryError::Database(e.to_string()))?; + if txn.kg_writes_enabled()? { + self.project_view_kg(&mut txn)?; + } + txn.commit() .map_err(|e| RepositoryError::Database(e.to_string()))?; @@ -569,6 +589,7 @@ impl Repository { /// - The view is the current view (cannot delete current view) /// - The database operation fails pub fn delete_view(&mut self, name: &str) -> Result<(), RepositoryError> { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; // Cannot delete the current view if name == self.current_view { return Err(RepositoryError::CannotDeleteCurrentView { @@ -621,6 +642,8 @@ impl Repository { txn.commit() .map_err(|e| RepositoryError::Database(e.to_string()))?; + self.kg_enrich_views()?; + // Only clean up legacy shelved artifacts after deletion is durable. // A refused deletion or failed commit must preserve the user's files. // Cleanup remains best-effort because the view is already deleted. @@ -759,6 +782,7 @@ impl Repository { scope: ViewScope, parent_name: Option<&str>, ) -> Result<(), RepositoryError> { + self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; ensure_workspace_dir(&self.dot_dir, name)?; let mut txn = self @@ -791,6 +815,10 @@ impl Repository { txn.create_view(name, scope, parent_id) .map_err(|e| RepositoryError::Database(e.to_string()))?; + if txn.kg_writes_enabled()? { + self.project_view_kg(&mut txn)?; + } + txn.commit() .map_err(|e| RepositoryError::Database(e.to_string()))?; diff --git a/atomic-repository/src/triage/report.rs b/atomic-repository/src/triage/report.rs index 5e7abf6d..2e130576 100644 --- a/atomic-repository/src/triage/report.rs +++ b/atomic-repository/src/triage/report.rs @@ -21,7 +21,7 @@ use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet}; use atomic_canonical::node::CanonicalNode; use atomic_canonical::{intent_substance_hash, triage_reference, validate_intent, TriagePins}; use atomic_core::change::{Change, GraphOp, ProvenanceGraph}; -use atomic_core::pristine::ontology::edge_kind; +use atomic_core::kg::{iri, ontology::Predicate as P}; use atomic_core::types::{Base32, Hash}; use crate::diff_engine::{change_file_diffs, DiffFormat, DiffOutputConfig, FileChangeStatus}; @@ -32,11 +32,10 @@ use crate::Repository; use super::model::*; /// Case-insensitive edge-kind comparison, mirroring `QueryCallers`. -fn kind_is(kind: &str, expected: &str) -> bool { - kind.eq_ignore_ascii_case(expected) +fn kind_is(kind: &P, expected: P) -> bool { + *kind == expected } -/// First 12 base32 chars — the short id used for `change:` KG nodes. /// True for paths the vault owns — intent files, attestations, audit /// entries. Changes confined to these are lifecycle bookkeeping, not work /// a task is expected to have declared. @@ -49,18 +48,6 @@ fn short12(hash_b32: &str) -> String { hash_b32.chars().take(12).collect() } -/// Map a canonical `urn:atomic::` id to its KG child id -/// (`:`), matching `vault_triples::child_kg_id`. -fn child_kg_id(urn: &str) -> String { - match urn.strip_prefix("urn:atomic:") { - Some(rest) => match rest.split_once(':') { - Some((kind, local)) => format!("{kind}:{}", local.to_uppercase()), - None => rest.to_uppercase(), - }, - None => urn.to_string(), - } -} - /// If any acceptance criterion on `node` carries a view-scoped verification /// whose LATEST view-scoped record failed, return that record's `kind` (for the /// finding message). `None` means the view baseline is not known to be red. @@ -89,6 +76,8 @@ pub fn build_report( ) -> Result { // 1. The candidate set (T0): only-in-feature, closure additions, baggage. let set = repo.triage_candidate_set(feature, target)?; + let project = repo.project_id()?; + let syntax = repo.syntax_index(Some(feature))?; // 2. The pinned view Merkle (the materialized state the report is about). let view_merkle = repo.get_view_info(feature)?.state_base32(); @@ -121,8 +110,8 @@ pub fn build_report( // Walkthrough facts, collected in the same KG passes (no extra queries): // path → module (PART_OF) and file → file dependency edges (IMPORTS/ // INCLUDES). Best-effort — empty on an unenriched KG. - let mut file_module: BTreeMap = BTreeMap::new(); - let mut file_deps: BTreeSet<(String, String)> = BTreeSet::new(); + let file_module = syntax.file_modules.clone(); + let file_deps = syntax.file_deps.clone(); // Task facts from reached intents (id, text, touches, satisfies), in // intent-id order — the walkthrough's narrative inputs. let mut task_facts: Vec = Vec::new(); @@ -161,7 +150,7 @@ pub fn build_report( // The shared `file:` node ids that MODIFIES/TOUCHES/DEFINES/ // SCOPE_OUT_FILE all meet on. - let mut modifies: Vec = paths.iter().map(|p| format!("file:{p}")).collect(); + let mut modifies: Vec = paths.iter().map(|p| iri::mint_file(&project, p)).collect(); modifies.sort(); modifies.dedup(); for f in &modifies { @@ -173,37 +162,22 @@ pub fn build_report( // entities) seeds the blast walk. Both sit at depth 1 from the file // node. Absent enrichment these are simply empty (best-effort). let mut touching_tasks: BTreeSet = BTreeSet::new(); - let mut entities: Vec = Vec::new(); + let mut entities: Vec = syntax + .entities + .iter() + .filter(|entity| paths.contains(&entity.file)) + .map(|entity| format!("{}#{}", entity.file, entity.name)) + .collect(); let mut covered = false; for file_id in &modifies { let Ok(fsub) = repo.vault_kg_neighbors(file_id, 1) else { continue; }; - let path = file_id.strip_prefix("file:").unwrap_or(file_id); for e in &fsub.edges { - if e.to_id == *file_id && kind_is(&e.kind, edge_kind::TOUCHES) { + if e.to_id == *file_id && kind_is(&e.kind, P::Touches) { covered = true; touching_tasks.insert(e.from_id.clone()); } - if e.from_id == *file_id && kind_is(&e.kind, edge_kind::DEFINES) { - entities.push(e.to_id.clone()); - } - // Walkthrough facts on the same depth-1 neighborhood: the - // file's module (PART_OF) and its outgoing file dependencies - // (IMPORTS / INCLUDES) — the layer-ordering signal. - if e.from_id == *file_id && kind_is(&e.kind, edge_kind::PART_OF) { - if let Some(module) = e.to_id.strip_prefix("module:") { - file_module.insert(path.to_string(), module.to_string()); - } - } - if e.from_id == *file_id - && (kind_is(&e.kind, edge_kind::IMPORTS) - || kind_is(&e.kind, edge_kind::INCLUDES)) - { - if let Some(target) = e.to_id.strip_prefix("file:") { - file_deps.insert((path.to_string(), target.to_string())); - } - } } } @@ -222,14 +196,14 @@ pub fn build_report( for task in &touching_tasks { let tsub = repo.vault_kg_neighbors(task, 1)?; for e in &tsub.edges { - if e.to_id == *task && kind_is(&e.kind, edge_kind::HAS_TASK) { + if e.to_id == *task && kind_is(&e.kind, P::HasTask) { reached = true; intent_to_changes .entry(e.from_id.clone()) .or_default() .insert(full.clone()); } - if e.from_id == *task && kind_is(&e.kind, edge_kind::SATISFIES) { + if e.from_id == *task && kind_is(&e.kind, P::Satisfies) { ac_to_changes .entry(e.to_id.clone()) .or_default() @@ -262,8 +236,11 @@ pub fn build_report( let mut intent_substance_hashes: BTreeMap = BTreeMap::new(); for intent_kg_id in intent_to_changes.keys() { - let bare = intent_kg_id.strip_prefix("intent:").unwrap_or(intent_kg_id); - match load_intent_node(repo, bare) { + let (_, bare) = + iri::parse_intent(intent_kg_id).map_err(|e| RepositoryError::InvalidOperation { + message: e.to_string(), + })?; + match load_intent_node(repo, &bare) { Ok(node) => { let report = validate_intent(&node); let current_substance = intent_substance_hash(&node); @@ -276,7 +253,7 @@ pub fn build_report( // The pin lives in the intent's frontmatter (not the lifted // node), so read it via the bridge. if node.status == "done" { - if let Some(pin) = load_intent_done_pin(repo, bare) { + if let Some(pin) = load_intent_done_pin(repo, &bare) { if pin != current_substance { findings.push( Finding::new( @@ -307,15 +284,17 @@ pub fn build_report( // never blocking. if let Ok(sub) = repo.vault_kg_neighbors(intent_kg_id, 1) { for e in &sub.edges { - if e.to_id != *intent_kg_id || !kind_is(&e.kind, edge_kind::REMEDIATES) { + if e.to_id != *intent_kg_id || !kind_is(&e.kind, P::Remediates) { continue; } let b_id = &e.from_id; - let b_bare = b_id.strip_prefix("intent:").unwrap_or(b_id); + let b_bare = iri::parse_intent(b_id) + .map(|(_, uid)| uid) + .unwrap_or_default(); // In flight = B is not `done`. If B loads and is done, // the remediation is resolved → skip. If B fails to load // (or is any non-done status), treat it as in flight. - let b_done = load_intent_node(repo, b_bare) + let b_done = load_intent_node(repo, &b_bare) .map(|n| n.status == "done") .unwrap_or(false); if b_done { @@ -352,12 +331,14 @@ pub fn build_report( if let Ok(sub) = repo.vault_kg_neighbors(intent_kg_id, 1) { for e in &sub.edges { // review --REVIEWS--> work (incoming to the work intent). - if e.to_id != *intent_kg_id || !kind_is(&e.kind, edge_kind::REVIEWS) { + if e.to_id != *intent_kg_id || !kind_is(&e.kind, P::Reviews) { continue; } review_exists = true; - let r_bare = e.from_id.strip_prefix("intent:").unwrap_or(&e.from_id); - if let Ok(rnode) = load_intent_node(repo, r_bare) { + let r_bare = iri::parse_intent(&e.from_id) + .map(|(_, uid)| uid) + .unwrap_or_default(); + if let Ok(rnode) = load_intent_node(repo, &r_bare) { let review_author = rnode.attributed_to.clone().unwrap_or_default(); let independent = !work_author.is_empty() && !review_author.is_empty() @@ -425,7 +406,15 @@ pub fn build_report( let mut criteria = Vec::new(); for ac in &node.has_acceptance_criterion { - let ac_kg = child_kg_id(&ac.id); + let ac_kg = iri::mint_child( + &project, + &bare, + iri::ChildKind::AcceptanceCriterion, + ac.id.rsplit(':').next().unwrap_or(&ac.id), + ) + .map_err(|e| RepositoryError::InvalidOperation { + message: e.to_string(), + })?; let satisfied_by: Vec = ac_to_changes .get(&ac_kg) .map(|s| s.iter().cloned().collect()) @@ -638,7 +627,7 @@ pub fn build_report( let scope_out: HashSet = isub .edges .iter() - .filter(|e| e.from_id == *intent_kg_id && kind_is(&e.kind, edge_kind::SCOPE_OUT_FILE)) + .filter(|e| e.from_id == *intent_kg_id && kind_is(&e.kind, P::ScopeOutFile)) .map(|e| e.to_id.clone()) .collect(); if scope_out.is_empty() { @@ -676,29 +665,9 @@ pub fn build_report( for (i, entities) in change_entities.iter().enumerate() { let mut callers: BTreeSet = BTreeSet::new(); for entity in entities { - let Ok(esub) = repo.vault_kg_neighbors(entity, 2) else { - continue; - }; - for e in &esub.edges { - if e.to_id != *entity || !kind_is(&e.kind, edge_kind::CALLS) { - continue; - } - let caller = &e.from_id; - // The caller's defining file(s): incoming DEFINES to the caller. - let caller_files: Vec<&String> = esub - .edges - .iter() - .filter(|d| d.to_id == *caller && kind_is(&d.kind, edge_kind::DEFINES)) - .map(|d| &d.from_id) - .collect(); - // Only flag when we can see a defining file and none of them are - // in the change-set (avoids noise on partially-enriched graphs). - if caller_files.is_empty() { - continue; - } - let in_change_set = caller_files.iter().any(|f| all_modified_files.contains(*f)); - if !in_change_set { - callers.insert(caller.clone()); + for caller in syntax.callers(entity) { + if !all_modified_files.contains(&iri::mint_file(&project, &caller.path)) { + callers.insert(format!("{}#{}", caller.path, caller.name)); } } } @@ -842,7 +811,14 @@ pub fn build_report( ) }) .collect(); - let walkthrough = build_walkthrough(&change_paths, &file_module, &file_deps, &task_facts); + let mut walkthrough = build_walkthrough(&change_paths, &file_module, &file_deps, &task_facts); + for layer in &mut walkthrough { + layer.files = layer + .files + .iter() + .map(|file| iri::mint_file(&project, file.strip_prefix("file:").unwrap_or(file))) + .collect(); + } Ok(TriageReport { reference, @@ -1549,7 +1525,7 @@ Edit a.txt. ::file-ref{path=a.txt} :::"; install_intent_body(&repo, &a.intent_file, body_a, None); - let a_node_id = format!("intent:{}", a.uid.to_uppercase()); + let a_node_id = iri::mint_intent(&repo.project_id().unwrap(), &a.uid).unwrap(); // Intent B: remediates A (B --REMEDIATES--> A), status governed by b_done. let b = repo.vault_intent_create(intent_opts("Intent B")).unwrap(); @@ -1690,7 +1666,7 @@ Fix the flaw in A. let a = repo.vault_intent_create(intent_opts("Intent A")).unwrap(); install_intent_body(&repo, &a.intent_file, body_a, None); - let a_node_id = format!("intent:{}", a.uid.to_uppercase()); + let a_node_id = iri::mint_intent(&repo.project_id().unwrap(), &a.uid).unwrap(); repo.create_view_from("feature", &base_view).unwrap(); repo.switch_view(working_copy, "feature").unwrap(); @@ -1850,7 +1826,9 @@ Edit foo. .find(|c| c.id == change_id) .expect("change report present"); assert!( - cr.modifies.iter().any(|m| m == "file:src/foo.rs"), + cr.modifies + .iter() + .any(|m| iri::parse_file(m).is_ok_and(|(_, path)| path == "src/foo.rs")), "modifies should include file:src/foo.rs from change_modified_paths: {:?}", cr.modifies ); @@ -2001,7 +1979,10 @@ Edit bar. assert_eq!(report.walkthrough.len(), 1, "one modified dir → one layer"); let layer = &report.walkthrough[0]; assert_eq!(layer.id, "layer:src"); - assert_eq!(layer.files, vec!["file:src/foo.rs"]); + assert_eq!( + layer.files, + vec![iri::mint_file(&repo.project_id().unwrap(), "src/foo.rs")] + ); assert_eq!(layer.changes, vec![hash.clone()]); assert!(layer.depends_on.is_empty()); } @@ -2045,7 +2026,7 @@ Edit a.txt. body_a, &[("attributedTo", WORK_AUTHOR), ("status", "in-progress")], ); - let a_node_id = format!("intent:{}", a.uid.to_uppercase()); + let a_node_id = iri::mint_intent(&repo.project_id().unwrap(), &a.uid).unwrap(); // Optional review intent R: kind=review, `reviews` A. if let Some((review_author, done)) = review { diff --git a/atomic-repository/tests/kg_projection.rs b/atomic-repository/tests/kg_projection.rs new file mode 100644 index 00000000..e9cb916f --- /dev/null +++ b/atomic-repository/tests/kg_projection.rs @@ -0,0 +1,146 @@ +use atomic_core::{ + change::{Change, ChangeHeader, ProvenanceGraph}, + kg::{ + iri, + ontology::{Class as C, Predicate as P}, + }, + pristine::{KgTxnT, VaultEntryType}, + types::Merkle, +}; +use atomic_repository::{Repository, UnrecordOptions, ViewManifest}; + +const UID: &str = "01ARZ3NDEKTSV4RRFFQ69G5FAV"; + +#[test] +fn later_intent_ingestion_repairs_session_links_and_deletion_restores_gap_findings() { + let directory = tempfile::tempdir().unwrap(); + let repo = Repository::init(directory.path()).unwrap(); + repo.init_vault().unwrap(); + let hash = atomic_core::types::Hash::of(b"historical generated change"); + repo.save_provenance_graph( + &ProvenanceGraph::builder("late-plan", "opencode") + .plan_id(UID) + .add_change_explained(hash) + .build(), + ) + .unwrap(); + let project = repo.project_id().unwrap(); + let change = iri::mint_change(&hash); + let missing = iri::mint_finding(&project, "MissingPlanId", &change); + assert!(repo.vault_kg_node(&missing).unwrap().is_some()); + let path = format!("intents/{UID}/intent.md"); + repo.vault_store(&path, VaultEntryType::Intent, b":::why\nResolve an old turn.\n:::\n".to_vec(), + serde_json::json!({"uid":UID,"id":"TEST::alice::1","title":"Late plan","status":"in_progress","kind":"feature"}).to_string()).unwrap(); + repo.vault_reconcile_intent_manifest().unwrap(); + repo.vault_reindex_kg().unwrap(); + let intent = iri::mint_intent(&project, UID).unwrap(); + assert!(repo + .workflow_snapshot() + .unwrap() + .edges + .iter() + .any(|edge| edge.from_id == change && edge.kind == P::Implements && edge.to_id == intent)); + assert!(repo.vault_kg_node(&missing).unwrap().is_none()); + repo.vault_delete(&path).unwrap(); + assert!(repo.vault_kg_node(&intent).unwrap().is_none()); + assert!(repo.vault_kg_node(&missing).unwrap().is_some()); + assert!(!repo + .workflow_snapshot() + .unwrap() + .edges + .iter() + .any(|edge| edge.from_id == change && edge.kind == P::Implements)); +} + +#[test] +fn legacy_memory_references_resolve_case_insensitively_and_repair_when_targets_arrive() { + let directory = tempfile::tempdir().unwrap(); + let repo = Repository::init(directory.path()).unwrap(); + repo.init_vault().unwrap(); + repo.vault_store( + "memory/source.md", + VaultEntryType::Memory, + b"source".to_vec(), + serde_json::json!({"derivedFrom":["urn:atomic:memory:TaRgEt"]}).to_string(), + ) + .unwrap(); + let project = repo.project_id().unwrap(); + let source = iri::mint_memory(&project, "source"); + assert!(repo + .vault_kg_node(&iri::mint_finding(&project, "UnresolvedReference", &source)) + .unwrap() + .is_some()); + repo.vault_store( + "memory/target.md", + VaultEntryType::Memory, + b"target".to_vec(), + "{}".into(), + ) + .unwrap(); + let snapshot = repo.workflow_snapshot().unwrap(); + assert!(snapshot.edges.iter().any(|edge| edge.from_id == source + && edge.to_id == iri::mint_memory(&project, "target") + && edge.kind == P::ProvWasDerivedFrom)); + assert!(repo + .vault_kg_node(&iri::mint_finding(&project, "UnresolvedReference", &source)) + .unwrap() + .is_none()); + assert!(snapshot + .nodes + .iter() + .all(|node| node.types.iter().all(|class| C::from_iri(class).is_some()))); +} + +#[test] +fn membership_only_manifest_unrecord_and_reinsert_update_direct_view_facts() { + let directory = tempfile::tempdir().unwrap(); + let mut repo = Repository::init(directory.path()).unwrap(); + repo.init_kg().unwrap(); + repo.create_shared_view("mirror").unwrap(); + let change = Change::new( + ChangeHeader::new("Already stored object"), + vec![], + vec![], + vec![], + ); + let mut bytes = Vec::new(); + let hash = change.serialize(&mut bytes).unwrap(); + repo.import_change_bytes(&hash, &bytes).unwrap(); + let project = repo.project_id().unwrap(); + let change_id = iri::mint_change(&hash); + let view = iri::mint_view(&project, "mirror"); + let membership = |repo: &Repository| { + repo.workflow_snapshot().unwrap().edges.iter().any(|edge| { + edge.from_id == change_id && edge.to_id == view && edge.kind == P::DirectlyOnView + }) + }; + assert!(!membership(&repo)); + let manifest = ViewManifest { + name: "mirror".into(), + scope: atomic_core::pristine::ViewScope::Shared, + parent: None, + changes: vec![hash], + state: Merkle::ZERO.next(&hash), + }; + assert_eq!(repo.reconcile_view_manifest(&manifest).unwrap().replayed, 1); + assert!(membership(&repo)); + assert_eq!(repo.reconcile_view_manifest(&manifest).unwrap().replayed, 0); + repo.unrecord( + &hash, + UnrecordOptions { + view: Some("mirror".into()), + ..Default::default() + }, + ) + .unwrap(); + assert!(!membership(&repo)); + assert!(repo + .pristine() + .read_txn() + .unwrap() + .get_kg_node(&change_id) + .unwrap() + .is_some()); + repo.reinsert_change_on_view("mirror", &hash, None).unwrap(); + assert!(membership(&repo)); +} diff --git a/atomic-repository/tests/project_identity.rs b/atomic-repository/tests/project_identity.rs new file mode 100644 index 00000000..d3be4459 --- /dev/null +++ b/atomic-repository/tests/project_identity.rs @@ -0,0 +1,33 @@ +use atomic_core::kg::project_id::MintReason; +use atomic_repository::Repository; + +#[test] +fn identity_is_local_view_independent_and_minting_is_idempotent() { + let directory = tempfile::tempdir().unwrap(); + let mut repo = Repository::init(directory.path()).unwrap(); + assert!(repo.project_identity().unwrap().is_none()); + let identity = repo.ensure_project_identity(MintReason::Upgrade).unwrap(); + assert_eq!( + repo.ensure_project_identity(MintReason::Created).unwrap(), + identity + ); + repo.create_view("another").unwrap(); + assert_eq!(repo.project_identity().unwrap().unwrap(), identity); + drop(repo); + let repo = Repository::open(directory.path()).unwrap(); + assert_eq!(repo.project_identity().unwrap().unwrap(), identity); +} + +#[test] +fn only_full_writable_open_mints_an_upgrade_identity() { + let directory = tempfile::tempdir().unwrap(); + drop(Repository::init(directory.path()).unwrap()); + let repo = Repository::open_existing(directory.path()).unwrap(); + assert!(repo.project_identity().unwrap().is_none()); + drop(repo); + let repo = Repository::open(directory.path()).unwrap(); + assert_eq!( + repo.project_identity().unwrap().unwrap().mint_reason, + MintReason::Upgrade + ); +} diff --git a/atomic-shacl/Cargo.toml b/atomic-shacl/Cargo.toml new file mode 100644 index 00000000..9ee9f6e1 --- /dev/null +++ b/atomic-shacl/Cargo.toml @@ -0,0 +1,25 @@ +[package] +name = "atomic-shacl" +description = "CLI-only RDF snapshot adaptation and PurRDF SHACL validation" +version.workspace = true +edition.workspace = true +license.workspace = true +rust-version = "1.98" + +[dependencies] +atomic-core.workspace = true +purrdf-shapes = "=3.0.1" +oxrdf = "=0.2.4" +oxttl = "=0.1.8" +serde.workspace = true +serde_json.workspace = true +thiserror.workspace = true +blake3.workspace = true +url.workspace = true + +[dev-dependencies] +tempfile.workspace = true +atomic-repository.workspace = true +atomic-identity.workspace = true +atomic-canonical.workspace = true +redb.workspace = true diff --git a/atomic-shacl/src/adapter.rs b/atomic-shacl/src/adapter.rs new file mode 100644 index 00000000..4b30dd83 --- /dev/null +++ b/atomic-shacl/src/adapter.rs @@ -0,0 +1,109 @@ +use atomic_core::{ + kg::ontology::{Class, Datatype, Predicate}, + KgEdge, KgNode, +}; +use oxrdf::{Literal, NamedNode}; +use std::collections::BTreeSet; + +use crate::ValidationError; + +fn named(iri: &str) -> Result { + NamedNode::new(iri) + .map(|node| node.to_string()) + .map_err(|e| ValidationError::Snapshot(e.to_string())) +} + +/// Export asserted workflow facts only. Unknown JSON metadata stays application +/// data. No domain/range, inverse, or property-chain inference is performed. +pub fn snapshot_ntriples(nodes: &[KgNode], edges: &[KgEdge]) -> Result { + let mut triples = BTreeSet::new(); + const TYPE: &str = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type"; + const SUBCLASS: &str = "http://www.w3.org/2000/01/rdf-schema#subClassOf"; + for class in Class::ALL { + for parent in class.parents() { + triples.insert(format!( + "{} <{SUBCLASS}> {} .\n", + named(class.iri())?, + named(parent.iri())? + )); + } + } + for status in ["Verified", "Failed", "Unverifiable"] { + triples.insert(format!( + " <{TYPE}> <{}> .\n", + Class::VerificationStatus.iri() + )); + } + for node in nodes { + let subject = named(&node.id)?; + for class in &node.types { + if Class::from_iri(class).is_none() { + return Err(ValidationError::Snapshot(format!( + "unknown asserted class: {class}" + ))); + } + triples.insert(format!("{subject} <{TYPE}> {} .\n", named(class)?)); + } + let mut literal = |predicate: Predicate, + value: &str, + datatype: Option| + -> Result<(), ValidationError> { + let literal = match datatype { + Some(datatype) => { + Literal::new_typed_literal(value, NamedNode::new_unchecked(datatype.iri())) + } + None => Literal::new_simple_literal(value), + }; + triples.insert(format!("{subject} <{}> {literal} .\n", predicate.iri())); + Ok(()) + }; + literal(Predicate::RdfsLabel, &node.label, None)?; + if let Some(summary) = &node.summary { + literal(Predicate::DctermsDescription, summary, None)?; + } + if let Some(metadata) = &node.metadata { + for (key, predicate, datatype) in [ + ("status", Predicate::Status, Datatype::String), + ("priority", Predicate::Priority, Datatype::String), + ("finding_code", Predicate::FindingCode, Datatype::String), + ("created_at", Predicate::DctermsCreated, Datatype::DateTime), + ("updated_at", Predicate::DctermsModified, Datatype::DateTime), + ("project_id", Predicate::ProjectId, Datatype::String), + ] { + if let Some(value) = metadata.get(key).filter(|value| !value.is_null()) { + let value = value.as_str().ok_or_else(|| { + ValidationError::Snapshot(format!("{key} must be a string on {}", node.id)) + })?; + literal(predicate, value, Some(datatype))?; + } + } + for (key, predicate) in [ + ("turn_number", Predicate::TurnNumber), + ("schema_version", Predicate::SchemaVersion), + ] { + if let Some(value) = metadata.get(key).filter(|value| !value.is_null()) { + let value = value.as_u64().ok_or_else(|| { + ValidationError::Snapshot(format!( + "{key} must be a non-negative integer on {}", + node.id + )) + })?; + literal( + predicate, + &value.to_string(), + Some(Datatype::NonNegativeInteger), + )?; + } + } + } + } + for edge in edges { + triples.insert(format!( + "{} <{}> {} .\n", + named(&edge.from_id)?, + edge.kind.iri(), + named(&edge.to_id)? + )); + } + Ok(triples.into_iter().collect()) +} diff --git a/atomic-shacl/src/lib.rs b/atomic-shacl/src/lib.rs new file mode 100644 index 00000000..22aefe0d --- /dev/null +++ b/atomic-shacl/src/lib.rs @@ -0,0 +1,283 @@ +//! CLI-only SHACL integration. Atomic adapts snapshots and presents reports; +//! PurRDF owns shape parsing, target selection, and constraint execution. + +pub const PROJECTION_SHAPES: &str = + include_str!("../../atomic-core/ontology/shapes/projection-1.0.0.ttl"); +pub const PUBLICATION_SHAPES: &str = + include_str!("../../atomic-core/ontology/shapes/publication-1.0.0.ttl"); + +mod adapter; +pub use adapter::snapshot_ntriples; + +use purrdf_shapes::{ + engine, report::ValidationResult, shapes::Path as ShapePath, term::Term, ShapesImports, +}; +use serde::{Deserialize, Serialize}; +use serde_json::{json, Value}; +use std::{ + collections::{BTreeMap, BTreeSet}, + path::Path, +}; + +#[derive(Debug, thiserror::Error)] +pub enum ValidationError { + #[error("workflow snapshot adaptation failed: {0}")] + Snapshot(String), + #[error("shape loading failed: {0}")] + Shapes(String), + #[error("SHACL execution failed: {0}")] + Execution(String), + #[error(transparent)] + Io(#[from] std::io::Error), +} + +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum Profile { + #[default] + Projection, + Publication, +} + +#[derive(Clone, Debug)] +pub struct ShapeBundle { + pub iri: String, + pub turtle: String, +} + +#[derive(Clone, Debug, Default)] +pub struct ValidationRequest { + pub profile: Profile, + pub shape: Option, + pub node: Option, + pub custom: Vec, +} + +#[derive(Clone, Debug, Default, Serialize)] +pub struct TargetCoverage { + pub shape: String, + pub applicable: usize, + pub evaluated: usize, +} + +#[derive(Clone, Debug, Serialize)] +pub struct ValidationOutput { + pub engine: &'static str, + pub engine_version: &'static str, + pub profile: Profile, + pub profile_identity: String, + pub snapshot_identity: String, + pub conforms: bool, + pub scope: &'static str, + pub target_coverage: Vec, + pub diagnostics: Vec, + pub results: Vec, + /// The complete standard report, including complex paths, literal language/ + /// datatype, nested details and any additional engine report annotations. + pub report_ntriples: String, +} + +/// Only explicitly supplied local bundles can satisfy imports. This function +/// never resolves imports by making network or additional filesystem requests. +pub fn load_custom_shapes(directory: &Path) -> Result, ValidationError> { + let entries = match std::fs::read_dir(directory) { + Ok(entries) => entries, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(Vec::new()), + Err(error) => return Err(error.into()), + }; + let mut bundles = Vec::new(); + for entry in entries { + let path = entry?.path(); + if path.extension().is_some_and(|extension| extension == "ttl") && path.is_file() { + let path = path.canonicalize()?; + let iri = url::Url::from_file_path(&path) + .map_err(|_| { + ValidationError::Shapes(format!( + "cannot form a base IRI for {}", + path.display() + )) + })? + .to_string(); + bundles.push(ShapeBundle { + iri, + turtle: std::fs::read_to_string(path)?, + }); + } + } + bundles.sort_by(|a, b| a.iri.cmp(&b.iri)); + Ok(bundles) +} + +fn term_value(term: &Term) -> String { + match term { + Term::NamedNode(node) => node.as_str().to_owned(), + _ => term.to_string(), + } +} + +fn path_json(path: &ShapePath) -> Value { + match path { + ShapePath::Predicate(iri) => json!({"predicate": iri.as_str()}), + ShapePath::Inverse(path) => json!({"inverse": path_json(path)}), + ShapePath::Sequence(paths) => { + json!({"sequence": paths.iter().map(path_json).collect::>()}) + } + ShapePath::Alternative(paths) => { + json!({"alternative": paths.iter().map(path_json).collect::>()}) + } + ShapePath::ZeroOrMore(path) => json!({"zero_or_more": path_json(path)}), + ShapePath::OneOrMore(path) => json!({"one_or_more": path_json(path)}), + ShapePath::ZeroOrOne(path) => json!({"zero_or_one": path_json(path)}), + } +} + +fn result_json(result: &ValidationResult) -> Value { + json!({ + "focus_node": term_value(&result.focus_node), + "result_path": result.result_path.as_ref().map(term_value), + "path_structure": result.path_structure.as_ref().map(path_json), + "source_shape": term_value(&result.source_shape), + "source_constraint_component": result.source_constraint_component.as_str(), + "result_severity": result.severity.iri(), + "result_messages": result.messages.iter().map(|message| json!({"value": message.value(), "rdf_literal": Term::Literal(message.clone()).to_string()})).collect::>(), + "value": result.value.as_ref().map(term_value), + "details": result.details.iter().map(result_json).collect::>(), + }) +} + +/// One parsed snapshot and one parsed shapes/import closure are reused for +/// target coverage and validation. Focus selection is performed by PurRDF; +/// Atomic's callback only records coverage and applies explicit CLI selectors. +pub fn validate_ntriples( + data: &str, + snapshot_identity: &str, + request: &ValidationRequest, +) -> Result { + let mut bundles = vec![ShapeBundle { + iri: "https://atomic.dev/ns/shapes/projection/1.0.0".into(), + turtle: PROJECTION_SHAPES.into(), + }]; + if request.profile == Profile::Publication { + bundles.push(ShapeBundle { + iri: "https://atomic.dev/ns/shapes/publication/1.0.0".into(), + turtle: PUBLICATION_SHAPES.into(), + }); + } + bundles.extend(request.custom.clone()); + let mut imports = ShapesImports::new(); + let mut root = String::new(); + let mut identity = blake3::Hasher::new(); + for bundle in &bundles { + imports + .insert_turtle(&bundle.iri, &bundle.turtle) + .map_err(|e| ValidationError::Shapes(e.to_string()))?; + root.push_str(&format!( + " <{}> .\n", + bundle.iri + )); + identity.update(&(bundle.iri.len() as u64).to_be_bytes()); + identity.update(bundle.iri.as_bytes()); + identity.update(&(bundle.turtle.len() as u64).to_be_bytes()); + identity.update(bundle.turtle.as_bytes()); + } + let shapes = engine::parse_shapes_with_config( + &root, + Some("urn:atomic:shapes:selection"), + None, + &imports, + ) + .map_err(|e| ValidationError::Shapes(e.to_string()))?; + if shapes.rules.entailment + || !shapes.rules.global_rules.is_empty() + || !shapes.rules.rule_sets.is_empty() + || shapes + .node_shapes + .iter() + .any(|shape| !shape.rules.is_empty()) + { + return Err(ValidationError::Shapes( + "Phase 1 validates asserted snapshots; inference rules require a later profile".into(), + )); + } + if let Some(shape) = &request.shape { + if !shapes + .node_shapes + .iter() + .any(|candidate| term_value(&candidate.id) == *shape) + { + return Err(ValidationError::Shapes(format!( + "unknown selected shape: {shape}" + ))); + } + } + let dataset = purrdf_shapes::text_ingest::parse_ntriples_to_dataset(data) + .map_err(|errors| ValidationError::Snapshot(errors.join("\n")))?; + let dataset = engine::project_dataset(dataset.as_ref()).map_err(ValidationError::Snapshot)?; + let mut coverage: BTreeMap, BTreeSet)> = shapes + .node_shapes + .iter() + .filter(|shape| { + !shape.deactivated + && !shape.targets.is_empty() + && request + .shape + .as_ref() + .is_none_or(|selected| *selected == term_value(&shape.id)) + }) + .map(|shape| (term_value(&shape.id), Default::default())) + .collect(); + let report = + engine::validate_projected_dataset_with_focus_filter(dataset, &shapes, |shape, node| { + let id = term_value(&shape.id); + if request + .shape + .as_ref() + .is_some_and(|selected| *selected != id) + { + return false; + } + let node = term_value(node); + let counts = coverage.entry(id).or_default(); + counts.0.insert(node.clone()); + let evaluate = request + .node + .as_ref() + .is_none_or(|selected| *selected == node); + if evaluate { + counts.1.insert(node); + } + evaluate + }) + .map_err(|e| ValidationError::Execution(e.to_string()))?; + let coverage: Vec<_> = coverage + .into_iter() + .map(|(shape, (applicable, evaluated))| TargetCoverage { + shape, + applicable: applicable.len(), + evaluated: evaluated.len(), + }) + .collect(); + let mut diagnostics = Vec::new(); + if coverage.iter().all(|coverage| coverage.evaluated == 0) { + diagnostics + .push("No applicable focus nodes were evaluated; conformance is vacuous.".into()); + } + diagnostics.push("Phase 1 conformance is not complete publication/audit evidence; inspect coverage and historical-gap findings.".into()); + Ok(ValidationOutput { + engine: "PurRDF", + engine_version: purrdf_shapes::VERSION, + profile: request.profile, + profile_identity: identity.finalize().to_hex().to_string(), + snapshot_identity: snapshot_identity.into(), + conforms: report.conforms, + scope: if request.node.is_some() || request.shape.is_some() { + "selected" + } else { + "snapshot" + }, + target_coverage: coverage, + diagnostics, + results: report.results.iter().map(result_json).collect(), + report_ntriples: report.to_ntriples(), + }) +} diff --git a/atomic-shacl/tests/fixtures/w3c-class-001.ttl b/atomic-shacl/tests/fixtures/w3c-class-001.ttl new file mode 100644 index 00000000..3918003c --- /dev/null +++ b/atomic-shacl/tests/fixtures/w3c-class-001.ttl @@ -0,0 +1,15 @@ +# W3C SHACL test-suite core/node/class-001, approved test. +# Source: https://github.com/w3c/data-shapes/blob/gh-pages/data-shapes-test-suite/tests/core/node/class-001.ttl +# Data/shape statements retained; test-manifest/report statements omitted. +# W3C Software and Document License: https://www.w3.org/copyright/software-license/ +@prefix ex: . +@prefix rdf: . +@prefix rdfs: . +@prefix sh: . +ex:John rdf:type ex:MalePerson . +ex:MalePerson rdf:type rdfs:Class ; rdfs:subClassOf ex:Person . +ex:Person rdf:type rdfs:Class ; rdfs:subClassOf rdfs:Resource . +ex:Quokki rdf:type ex:Animal . +ex:Someone rdf:type ex:Person . +ex:TestShape rdf:type sh:NodeShape ; sh:class ex:Person ; + sh:targetNode ex:John, ex:Quokki, ex:Someone, ex:Typeless . diff --git a/atomic-shacl/tests/profiles.rs b/atomic-shacl/tests/profiles.rs new file mode 100644 index 00000000..98ddbe91 --- /dev/null +++ b/atomic-shacl/tests/profiles.rs @@ -0,0 +1,237 @@ +use atomic_core::{ + kg::{ + iri, + ontology::{Class as C, Predicate as P}, + project_id::ProjectId, + }, + types::Hash, + KgEdge, KgNode, +}; +use atomic_shacl::*; + +fn fixture() -> (Vec, Vec, String, String) { + let project = ProjectId::generate(); + let session = iri::mint_session(&project, "session"); + let turn = iri::mint_turn(&project, "session", 0); + let intent = iri::mint_intent(&project, "01ARZ3NDEKTSV4RRFFQ69G5FAV").unwrap(); + let change = iri::mint_change(&Hash::of(b"change")); + let provenance = iri::mint_provenance(&Hash::of(b"provenance")); + let attestation = iri::mint_attestation(&Hash::of(b"attestation")); + let signer = format!("did:atomic:{}", project.to_base32()); + let nodes = vec![ + KgNode::new(&session, C::Session, "session", "fixture"), + KgNode::new(&turn, C::SessionTurn, "turn", "fixture").with_type(C::ManagedTurn), + // Only the subclass is asserted: standard SHACL subclass matching + // must satisfy the WorkIntent range without OWL materialization. + KgNode::new(&intent, C::FeatureIntent, "intent", "fixture"), + KgNode::new(&change, C::Change, "change", "fixture"), + KgNode::new(&provenance, C::ProvenanceGraph, "provenance", "fixture"), + KgNode::new(&attestation, C::Attestation, "attestation", "fixture"), + KgNode::new(&signer, C::Identity, "signer", "fixture"), + ]; + let edges = vec![ + KgEdge::new(&session, &turn, P::HasTurn), + KgEdge::new(&turn, &intent, P::HasPlan), + KgEdge::new(&turn, &change, P::GeneratedChange), + KgEdge::new(&turn, &provenance, P::TurnExplainedBy), + KgEdge::new(&change, &intent, P::Implements), + KgEdge::new(&change, &provenance, P::ChangeExplainedBy), + KgEdge::new(&change, &attestation, P::AttestedBy), + KgEdge::new(&attestation, &change, P::CoversChange), + KgEdge::new(&attestation, &session, P::AttestsSession), + KgEdge::new(&attestation, &signer, P::SignedBy), + KgEdge::new( + &attestation, + "https://atomic.dev/ns/workflow#Verified", + P::VerificationResult, + ), + ]; + (nodes, edges, change, project.to_urn()) +} + +#[test] +fn complete_workflow_conforms_to_both_profiles_and_reports_real_coverage() { + let (nodes, edges, _, _) = fixture(); + let data = snapshot_ntriples(&nodes, &edges).unwrap(); + for profile in [Profile::Projection, Profile::Publication] { + let report = validate_ntriples( + &data, + "fixture", + &ValidationRequest { + profile, + ..Default::default() + }, + ) + .unwrap(); + assert!(report.conforms, "{}", report.report_ntriples); + assert!(report + .target_coverage + .iter() + .any(|coverage| coverage.evaluated > 0)); + assert_eq!(report.snapshot_identity, "fixture"); + } +} + +#[test] +fn each_missing_shortcut_needs_its_own_change_finding() { + let (mut nodes, mut edges, change, _) = fixture(); + edges.retain(|edge| { + !matches!( + edge.kind, + P::HasPlan + | P::TurnExplainedBy + | P::CoversChange + | P::Implements + | P::ChangeExplainedBy + | P::AttestedBy + ) + }); + let request = ValidationRequest::default(); + let validate = |nodes: &[KgNode], edges: &[KgEdge]| { + validate_ntriples( + &snapshot_ntriples(nodes, edges).unwrap(), + "fixture", + &request, + ) + .unwrap() + }; + assert!(!validate(&nodes, &edges).conforms); + let project = ProjectId::generate(); + for code in ["MissingPlanId", "MissingProvenance", "MissingAttestation"] { + let finding = iri::mint_finding(&project, code, &change); + nodes.push( + KgNode::new(&finding, C::Finding, code, "fixture") + .with_metadata(serde_json::json!({"finding_code": code})), + ); + edges.push(KgEdge::new(&finding, &change, P::AboutResource)); + } + assert!(validate(&nodes, &edges).conforms); + nodes.last_mut().unwrap().metadata = Some(serde_json::json!({"finding_code": "WrongCode"})); + assert!(!validate(&nodes, &edges).conforms); +} + +#[test] +fn shortcuts_cannot_disagree_with_intermediate_paths() { + let (nodes, mut edges, change, _) = fixture(); + let alternate = nodes + .iter() + .find(|node| node.has_type(C::Session)) + .unwrap() + .id + .clone(); + edges.push(KgEdge::new(change, alternate, P::Implements)); + let report = validate_ntriples( + &snapshot_ntriples(&nodes, &edges).unwrap(), + "fixture", + &Default::default(), + ) + .unwrap(); + assert!(!report.conforms); + assert!(report + .results + .iter() + .any(|result| result["source_constraint_component"] + .as_str() + .unwrap() + .contains("SPARQL"))); +} + +#[test] +fn publication_rejects_unverified_and_mac_signatures() { + let (nodes, mut edges, _, _) = fixture(); + let request = ValidationRequest { + profile: Profile::Publication, + ..Default::default() + }; + for status in ["Failed", "Unverifiable"] { + edges + .iter_mut() + .find(|edge| edge.kind == P::VerificationResult) + .unwrap() + .to_id = format!("https://atomic.dev/ns/workflow#{status}"); + assert!( + !validate_ntriples( + &snapshot_ntriples(&nodes, &edges).unwrap(), + "fixture", + &request + ) + .unwrap() + .conforms + ); + } + edges + .iter_mut() + .find(|edge| edge.kind == P::VerificationResult) + .unwrap() + .to_id = "https://atomic.dev/ns/workflow#Verified".into(); + edges + .iter_mut() + .find(|edge| edge.kind == P::SignedBy) + .unwrap() + .to_id = "urn:atomic:identity:session-mac".into(); + assert!( + !validate_ntriples( + &snapshot_ntriples(&nodes, &edges).unwrap(), + "fixture", + &request + ) + .unwrap() + .conforms + ); +} + +fn custom(turtle: &str) -> ValidationRequest { + ValidationRequest { custom: vec![ShapeBundle { iri: "urn:enterprise:shapes".into(), turtle: format!("@prefix sh: . @prefix atomic: . {turtle}") }], ..Default::default() } +} + +#[test] +fn errors_are_not_vacuous_conformance() { + for turtle in [ + " a sh:NodeShape ; sh:targetClass atomic:Change ; sh:property [ sh:path atomic:implements ; sh:minCount \"one\" ] .", + " a sh:NodeShape ; sh:js [ sh:jsFunctionName \"validate\" ] .", + " a sh:NodeShape ; sh:sparql [ sh:select \"SELECT $this WHERE {{\" ] .", + " .", + ] { assert!(validate_ntriples("", "empty", &custom(turtle)).is_err(), "accepted {turtle}"); } + let report = validate_ntriples("", "empty", &Default::default()).unwrap(); + assert!(report.conforms); + assert!(report + .target_coverage + .iter() + .all(|coverage| coverage.evaluated == 0)); + assert!(report + .diagnostics + .iter() + .any(|message| message.contains("vacuous"))); +} + +#[test] +fn custom_shapes_preserve_nested_paths_and_standard_report_fields() { + let (nodes, edges, change, _) = fixture(); + let request = custom(" a sh:NodeShape ; sh:targetClass atomic:Change ; sh:property [ sh:path ( [ sh:inversePath atomic:generatedChange ] atomic:hasTodo ) ; sh:minCount 1 ; sh:message \"Need a todo\"@en ] ."); + let mut request = request; + request.shape = Some("urn:custom".into()); + request.node = Some(change); + let report = validate_ntriples( + &snapshot_ntriples(&nodes, &edges).unwrap(), + "fixture", + &request, + ) + .unwrap(); + assert!(!report.conforms); + assert!(report.results[0]["path_structure"]["sequence"].is_array()); + assert!(report.results[0]["result_messages"][0]["rdf_literal"] + .as_str() + .unwrap() + .ends_with("@en")); + for field in [ + "focusNode", + "resultPath", + "sourceShape", + "sourceConstraintComponent", + "resultSeverity", + "resultMessage", + "inversePath", + ] { + assert!(report.report_ntriples.contains(field), "missing {field}"); + } +} diff --git a/atomic-shacl/tests/w3c_core.rs b/atomic-shacl/tests/w3c_core.rs new file mode 100644 index 00000000..35f874f4 --- /dev/null +++ b/atomic-shacl/tests/w3c_core.rs @@ -0,0 +1,51 @@ +use atomic_shacl::{validate_ntriples, ShapeBundle, ValidationRequest}; +use oxttl::TurtleParser; + +#[test] +fn w3c_class_001_preserves_subclass_matching_and_exact_standard_results() { + let turtle = include_str!("fixtures/w3c-class-001.ttl"); + let triples = TurtleParser::new() + .for_reader(turtle.as_bytes()) + .map(|triple| format!("{} .\n", triple.unwrap())) + .collect::(); + let report = validate_ntriples( + &triples, + "w3c:core/node/class-001", + &ValidationRequest { + custom: vec![ShapeBundle { + iri: "urn:w3c:class-001".into(), + turtle: turtle.into(), + }], + shape: Some("http://datashapes.org/sh/tests/core/node/class-001.test#TestShape".into()), + ..Default::default() + }, + ) + .unwrap(); + assert!(!report.conforms); + assert_eq!(report.results.len(), 2); + let mut focus: Vec<_> = report + .results + .iter() + .map(|result| result["focus_node"].as_str().unwrap()) + .collect(); + focus.sort(); + assert_eq!( + focus, + [ + "http://datashapes.org/sh/tests/core/node/class-001.test#Quokki", + "http://datashapes.org/sh/tests/core/node/class-001.test#Typeless" + ] + ); + for result in &report.results { + assert_eq!( + result["source_constraint_component"], + "http://www.w3.org/ns/shacl#ClassConstraintComponent" + ); + assert_eq!( + result["result_severity"], + "http://www.w3.org/ns/shacl#Violation" + ); + assert_eq!(result["focus_node"], result["value"]); + } + assert_eq!(report.target_coverage[0].evaluated, 4); +} diff --git a/atomic-shacl/tests/workflow_snapshot.rs b/atomic-shacl/tests/workflow_snapshot.rs new file mode 100644 index 00000000..235a14c7 --- /dev/null +++ b/atomic-shacl/tests/workflow_snapshot.rs @@ -0,0 +1,297 @@ +//! Real repository producer → snapshot → RDF → PurRDF integration. +use atomic_core::MutTxnT; +use atomic_core::{ + change::{AttestAgent, Attestation, Change, ChangeHeader, ProvenanceGraph}, + kg::{ + iri, + ontology::{Class as C, Predicate as P}, + }, + pristine::{KgTxnT, VaultEntryType}, +}; +use atomic_identity::KeyPair; +use atomic_repository::{InsertOptions, Repository}; +use atomic_shacl::*; + +fn validate(repo: &Repository, profile: Profile) -> ValidationOutput { + let snapshot = repo.workflow_snapshot().unwrap(); + let data = snapshot_ntriples(&snapshot.nodes, &snapshot.edges).unwrap(); + validate_ntriples( + &data, + &snapshot.identity, + &ValidationRequest { + profile, + ..Default::default() + }, + ) + .unwrap() +} + +#[test] +fn ingested_sidecars_close_history_without_inventing_view_membership() { + let dir = tempfile::tempdir().unwrap(); + let mut repo = Repository::init(dir.path()).unwrap(); + repo.init_vault().unwrap(); + let parent = repo.current_view().to_owned(); + let uid = "01ARZ3NDEKTSV4RRFFQ69G5FAV"; + repo.vault_store(&format!("intents/{uid}/intent.md"), VaultEntryType::Intent, + b":::why\nA real workflow fixture.\n:::\n".to_vec(), + serde_json::json!({"uid":uid,"id":"TEST::alice::1","kind":"feature","status":"in_progress","title":"Workflow fixture"}).to_string()).unwrap(); + repo.vault_reconcile_intent_manifest().unwrap(); + repo.vault_reindex_kg().unwrap(); + let project = repo.project_id().unwrap(); + let intent = iri::mint_intent(&project, uid).unwrap(); + + let change = Change::new(ChangeHeader::new("fixture change"), vec![], vec![], vec![]); + let mut bytes = Vec::new(); + let hash = change.serialize(&mut bytes).unwrap(); + repo.import_change_bytes(&hash, &bytes).unwrap(); + let change_id = iri::mint_change(&hash); + // Downloaded objects are globally present, but absent from every own log. + assert!(!repo + .workflow_snapshot() + .unwrap() + .edges + .iter() + .any(|edge| edge.from_id == change_id && edge.kind == P::DirectlyOnView)); + let graph = ProvenanceGraph::builder("fixture-session", "opencode") + .plan_id("TEST::alice::1") + .add_change_explained(hash) + .build(); + let provenance = repo.save_provenance_graph(&graph).unwrap(); + let snapshot = repo.workflow_snapshot().unwrap(); + assert!(snapshot.edges.iter().any(|edge| edge.from_id == change_id + && edge.kind == P::Implements + && edge.to_id == intent)); + assert!(snapshot.edges.iter().any(|edge| edge.from_id == change_id + && edge.kind == P::ChangeExplainedBy + && edge.to_id == iri::mint_provenance(&provenance))); + assert!(snapshot + .nodes + .iter() + .any(|node| node.id == iri::mint_finding(&project, "MissingAttestation", &change_id))); + let report = validate(&repo, Profile::Projection); + assert!(report.conforms, "{}", report.report_ntriples); + assert!(!validate(&repo, Profile::Publication).conforms); + + let key = KeyPair::generate(); + let did = atomic_canonical::did::did_for_public_key(&key.public); + let mut attestation = Attestation::builder( + "fixture-session", + AttestAgent::new("fixture", "Fixture", "1"), + ) + .add_change(hash) + .provenance_root(provenance) + .build(); + attestation.sign_with_did(&did, key.secret.as_bytes()); + let attest_hash = repo.save_attestation(&attestation).unwrap(); + let attest_id = iri::mint_attestation(&attest_hash); + let status = |repo: &Repository| { + repo.workflow_snapshot() + .unwrap() + .edges + .into_iter() + .find(|edge| edge.from_id == attest_id && edge.kind == P::VerificationResult) + .unwrap() + .to_id + }; + assert!(status(&repo).ends_with("#Unverifiable")); + assert!(repo + .cache_signer_key(&did, &KeyPair::generate().public) + .is_err()); + repo.cache_signer_key(&did, &key.public).unwrap(); + assert!(status(&repo).ends_with("#Verified")); + let report = validate(&repo, Profile::Publication); + assert!(report.conforms, "{}", report.report_ntriples); + assert!(!repo + .workflow_snapshot() + .unwrap() + .nodes + .iter() + .any(|node| node.id == iri::mint_finding(&project, "MissingAttestation", &change_id))); + + repo.create_draft_view("child", &parent).unwrap(); + repo.insert_change(&hash, InsertOptions::default().view(&parent)) + .unwrap(); + let edges = repo.workflow_snapshot().unwrap().edges; + assert!(edges.iter().any(|edge| edge.from_id == change_id + && edge.kind == P::DirectlyOnView + && edge.to_id == iri::mint_view(&project, &parent))); + assert!(!edges.iter().any(|edge| edge.from_id == change_id + && edge.kind == P::DirectlyOnView + && edge.to_id == iri::mint_view(&project, "child"))); + + let before = repo.workflow_snapshot().unwrap(); + assert!(repo.rebuild_kg(true).unwrap()); + let after = repo.workflow_snapshot().unwrap(); + assert_eq!( + before.identity, after.identity, + "incremental and full projection must agree" + ); + assert!(!repo.rebuild_kg(false).unwrap()); + let txn = repo.pristine().read_txn().unwrap(); + for node in &after.nodes { + let mut types = node.types.clone(); + types.sort(); + assert_eq!(txn.kg_node_types(&node.id).unwrap(), types); + assert!(node.types.iter().all(|class| C::from_iri(class).is_some())); + } + assert!(txn + .get_kg_node(&iri::mint_turn(&project, "fixture-session", 0)) + .unwrap() + .unwrap() + .has_type(C::ManagedTurn)); +} + +#[test] +fn old_generation_is_fenced_and_aborted_rebuild_never_publishes_partial_state() { + let dir = tempfile::tempdir().unwrap(); + let repo = Repository::init(dir.path()).unwrap(); + repo.init_vault().unwrap(); + repo.vault_store( + "memory/upgrade.md", + VaultEntryType::Memory, + b"upgrade".to_vec(), + "{}".into(), + ) + .unwrap(); + let original = repo.workflow_snapshot().unwrap().identity; + { + let mut txn = repo.pristine().write_txn().unwrap(); + txn.begin_kg_rebuild().unwrap(); + // Dropping an uncommitted transaction is a simulated interrupted build. + } + assert_eq!(repo.workflow_snapshot().unwrap().identity, original); + drop(repo); + { + let database = redb::Database::open(dir.path().join(".atomic/atomic.redb")).unwrap(); + let txn = database.begin_write().unwrap(); + txn.open_table(atomic_core::pristine::tables::KG_INDEX_META) + .unwrap() + .remove("kg_generation") + .unwrap(); + txn.commit().unwrap(); + } + let repo = Repository::open_readonly(dir.path()).unwrap(); + assert!(repo.list_views().is_ok()); + assert!(repo + .workflow_snapshot() + .unwrap_err() + .to_string() + .contains("KgNeedsRebuild")); + drop(repo); + let repo = Repository::open(dir.path()).unwrap(); + assert!(repo + .workflow_snapshot() + .unwrap() + .nodes + .iter() + .any(|node| node.has_type(C::Memory))); + assert!(!repo.rebuild_kg(false).unwrap()); +} + +#[test] +fn incomplete_manifest_history_conforms_only_through_precise_gap_findings() { + use atomic_core::{ + change::session::{SessionManifest, SessionTurn}, + types::Hash, + }; + let directory = tempfile::tempdir().unwrap(); + let repo = Repository::init(directory.path()).unwrap(); + let change = Hash::of(b"historical change"); + let provenance = Hash::of(b"absent sidecar"); + repo.ingest_session_manifest(&SessionManifest { + schema_version: 2, + session_id: "historical".into(), + goal_provenance: None, + parent_session: None, + fork_turn: None, + turns: vec![SessionTurn { + session_id: "historical".into(), + turn_number: 0, + goal: None, + provenance_hash: provenance, + change_hashes: vec![change], + previous_provenance: None, + timestamp: 1735689600, + plan_id: None, + todos: vec![], + boundary_start: None, + boundary_end: None, + outcome: None, + }], + }) + .unwrap(); + repo.rebuild_session_index().unwrap(); + let snapshot = repo.workflow_snapshot().unwrap(); + let id = iri::mint_change(&change); + for code in ["MissingPlanId", "MissingProvenance", "MissingAttestation"] { + let finding = iri::mint_finding(&repo.project_id().unwrap(), code, &id); + assert!(snapshot.edges.iter().any(|edge| edge.from_id == finding + && edge.to_id == id + && edge.kind == P::AboutResource)); + } + assert!(!snapshot + .nodes + .iter() + .any(|node| node.id == iri::mint_provenance(&provenance))); + let report = validate(&repo, Profile::Projection); + assert!(report.conforms, "{}", report.report_ntriples); + assert!(!validate(&repo, Profile::Publication).conforms); + let mut txn = repo.pristine().write_txn().unwrap(); + atomic_core::pristine::KgMutTxnT::del_kg_node( + &mut txn, + &iri::mint_finding(&repo.project_id().unwrap(), "MissingProvenance", &id), + ) + .unwrap(); + txn.commit().unwrap(); + assert!(!validate(&repo, Profile::Projection).conforms); +} + +#[test] +fn signed_vault_documents_project_typed_lineage_without_rewriting_signed_bytes() { + let directory = tempfile::tempdir().unwrap(); + let repo = Repository::init(directory.path()).unwrap(); + repo.init_vault().unwrap(); + let uid = "01ARZ3NDEKTSV4RRFFQ69G5FAV"; + let frontmatter = serde_json::json!({"uid":uid,"id":"TEST::alice::1","title":"Signed fixture","status":"backlog","kind":"bug"}); + let body = ":::why\nPreserve a signed representation.\n:::\n"; + repo.vault_store( + &format!("intents/{uid}/intent.md"), + VaultEntryType::Intent, + body.as_bytes().to_vec(), + frontmatter.to_string(), + ) + .unwrap(); + let node = atomic_canonical::lift::lift_intent(frontmatter.as_object().unwrap(), body).unwrap(); + let key = KeyPair::generate(); + let prepared = atomic_canonical::proof::prepare_attestation(node.to_value(), &key.public); + let signature = atomic_identity::Signer::new(&key).sign(&prepared.signing_bytes); + let signed = atomic_canonical::proof::attach_proof(prepared.value, &key.public, &signature); + let bytes = serde_json::to_vec_pretty(&signed).unwrap(); + let path = "attestations/signed/attested.md"; + repo.vault_store( + path, + VaultEntryType::Attestation, + bytes.clone(), + "{}".into(), + ) + .unwrap(); + let subject = iri::mint_intent(&repo.project_id().unwrap(), uid).unwrap(); + let snapshot = repo.workflow_snapshot().unwrap(); + assert!(snapshot + .nodes + .iter() + .find(|node| node.id == subject) + .unwrap() + .has_type(C::BugIntent)); + assert!(snapshot.edges.iter().any(|edge| edge.from_id == subject + && edge.kind == P::AuthoredBy + && edge.to_id == atomic_canonical::did::did_for_public_key(&key.public))); + repo.rebuild_kg(true).unwrap(); + assert_eq!( + repo.vault_retrieve(path).unwrap().unwrap().content_bytes, + bytes + ); + let node: atomic_canonical::CanonicalNode = serde_json::from_slice(&bytes).unwrap(); + atomic_canonical::verify(&node, &key.public).unwrap(); +} diff --git a/docs/knowledge-graph-closure-phase1-plan.md b/docs/knowledge-graph-closure-phase1-plan.md index 25ee2d81..44614319 100644 --- a/docs/knowledge-graph-closure-phase1-plan.md +++ b/docs/knowledge-graph-closure-phase1-plan.md @@ -327,10 +327,11 @@ pub enum Class { WorkIntent, FeatureIntent, ... } pub enum Predicate { HasTurn, HasPlan, ..., ProvWasInformedBy, ProvUsed, ... } // A class's parent may be an atomic class or an external one (owl:Thing, skos:Concept). -pub enum ClassRef { Atomic(Class), External(&'static str) } +pub enum ExternalClass { OwlThing, SkosConcept } +pub enum ClassRef { Atomic(Class), External(ExternalClass) } // A property's range: an atomic class, an external class IRI, or an XSD datatype. -pub enum Range { Class(Class), External(&'static str), Datatype(&'static str) } +pub enum Range { Class(ClassRef), Datatype(Datatype) } impl Class { pub fn iri(&self) -> &'static str; @@ -342,7 +343,7 @@ impl Predicate { pub fn iri(&self) -> &'static str; pub fn curie(&self) -> &'static str; pub fn is_external(&self) -> bool; - pub fn domain(&self) -> Option; + pub fn domain(&self) -> Option; pub fn range(&self) -> Option; pub fn inverse(&self) -> Option; pub fn super_predicate(&self) -> Option; @@ -469,8 +470,11 @@ pristine table outside the view system. > **Phase 1 implementation boundary.** `atomic clone` is HTTP-only (plus a Git bootstrap > path that never contacts storage), so identity transport over clone, pull and push all > depends on server support in `atomic-storage`, which is owned by another effort. In -> Phase 1, implement only the local pieces: minting, the `PROJECT_IDENTITY` table, and the -> CLI. **Do not change push, pull, clone, `atomic-remote`, or the daemon's sync code.** The +> Phase 1, implement the local pieces: minting, the `PROJECT_IDENTITY` table, and the +> CLI. Repository-level KG projection during existing object ingestion and actual +> view-membership mutations, plus signer-key cache refresh through existing identity +> APIs, are in scope for CLI and daemon paths. ProjectIdentity transfer, adoption, +> remote binding, and sync wire-contract changes remain deferred. The > rules below record the target behavior for the later transport work. - **Clone:** the remote supplies its identity object as a repository-level object (not a @@ -1035,8 +1039,9 @@ defined in the ontology, not `kg_nodes` rows. - Minting: `atomic init` and `atomic git import` at creation; every other repository without an identity on first writable open - **Not in Phase 1:** clone/pull/push identity transport. It waits for `atomic-storage` - (see the boundary note in §1.3); leave push, pull, clone, `atomic-remote`, and the daemon's - sync code unchanged + (see the boundary note in §1.3). Existing repository ingestion and membership + APIs maintain the offline projection; CLI/daemon orchestration refreshes + signer keys outside pristine writes without changing the sync wire contract. - CLI: `atomic repo identity show` (`--json`) - Property tests: round-trip, percent-encoding, strict base32 parsing diff --git a/docs/knowledge-graph-phase1-upgrade.md b/docs/knowledge-graph-phase1-upgrade.md new file mode 100644 index 00000000..f1e77a68 --- /dev/null +++ b/docs/knowledge-graph-phase1-upgrade.md @@ -0,0 +1,63 @@ +# Knowledge graph Phase 1 upgrade + +This release changes the workflow KG API. Consumers must use absolute resource, +class, and predicate IRIs. There is no legacy-vocabulary compatibility window. + +- `KgNode.types` replaces `kind` (protobuf: repeated `types`). Each asserted + class is an IRI under `https://atomic.dev/ns/workflow#`. +- Edge `kind` / protobuf `relation` is an absolute predicate IRI. `QueryGraph` + responses carry `api_version: 1`; clients reject unsupported versions. +- Changes, provenance, and session attestations use full Blake3 hashes. Local + workflow resources are qualified by the repository's stable ProjectId. +- Query-plan JSON requires `schema_version: 1`. Filter with `field: "type"` + and a class IRI. Neighborhood steps accept a predicate IRI and `direction` + (`incoming`, `outgoing`, or `both`). +- `query callers` accepts a bare symbol or `path#name` and parses source live. + Code entities/calls/imports/modules are no longer persisted in the KG. + +The first full writable open mints an unsigned upgrade identity if needed and +rebuilds the derived graph once. Lightweight/repair/read-only opens do not run +that migration. KG reads on an old generation report `KgNeedsRebuild`; ordinary +repository reads remain available. Force reconstruction with: + +```sh +atomic repo identity show --json +atomic query reindex --full +``` + +Validation runs locally in the CLI, using PurRDF 3.0.1: + +```sh +atomic query validate --format json +atomic query validate --profile publication --format json +atomic query validate --shape --node +``` + +Projection validation accepts a missing workflow shortcut only with its matching +per-change finding. Publication validation additionally checks managed plans, +provenance, session coverage, and explicit successful DID signature verification. +Historical gaps remain visible. Inspect target coverage: conformance with no +applicable targets is vacuous, not evidence of workflow completeness. + +Local enterprise `.atomic/shapes/*.ttl` policies use the same engine. Relative +IRIs resolve against each file's canonical file URL. Imports resolve only among +explicitly loaded bundles; unavailable imports are errors, without network fetches. +Exit codes are **0** for conformance, **1** for violations, and **3** for shape or +engine errors. JSON preserves the structured results and complete RDF report. + +Signer discovery after pull/reindex uses existing authenticated identity APIs. +Keys are cached only after matching their fingerprint DID. Servers unable to +resolve a fingerprint DID leave the attestation `Unverifiable`. Signature +verification remains separate from SHACL. No OWL inference or mandatory +publication hook is enabled in this phase. + +The CLI and `atomic-shacl` require Rust 1.98. Shared libraries retain Rust 1.90. +ProjectIdentity transport, adoption, and remote binding are deferred; Phase 1 +clones have local upgrade identities until that transport is implemented. + +Run the Phase 1 acceptance harness with a freshly built CLI: + +```sh +cargo build -p atomic-cli --bin atomic +ATOMIC_BIN="$PWD/target/debug/atomic" bash tests/harness/46_knowledge_graph_closure.sh +``` diff --git a/libatomic/Cargo.toml b/libatomic/Cargo.toml index fc022541..0f26e962 100644 --- a/libatomic/Cargo.toml +++ b/libatomic/Cargo.toml @@ -51,6 +51,7 @@ anyhow = { workspace = true } rand = { workspace = true } uuid = { workspace = true } tracing = { workspace = true } +url = { workspace = true } [dev-dependencies] # Loopback protocol acceptance harness only; no production listener is added. diff --git a/libatomic/proto/atomic/services/knowledge/knowledge_messages.proto b/libatomic/proto/atomic/services/knowledge/knowledge_messages.proto index c9619748..cadbc3f3 100644 --- a/libatomic/proto/atomic/services/knowledge/knowledge_messages.proto +++ b/libatomic/proto/atomic/services/knowledge/knowledge_messages.proto @@ -60,7 +60,13 @@ message NeighborsQuery { optional uint32 depth = 3; } message CallersQuery { - string node_id = 1; + string node_id = 1; // `#` or a bare symbol, from live source +} +message CodeCaller { + string path = 1; + string name = 2; + uint32 line = 3; + optional string summary = 4; } message GraphSliceQuery { repeated string node_ids = 1; @@ -106,11 +112,13 @@ message QueryGraphResponse { // the domain AgentResult JSON) plus its elapsed time. optional VersionedBytes ask_result = 9; optional uint64 ask_elapsed_ms = 10; + uint32 api_version = 11; // canonical workflow KG API version 1 + repeated CodeCaller callers = 12; } message KGNode { string id = 1; - string node_type = 2; + repeated string types = 2; // asserted absolute class IRIs; breaking KG cutover optional string name = 3; optional bytes data = 4; // node payload (SHAPE-CONFIRM) optional double score = 5; // search ranking when applicable @@ -123,7 +131,7 @@ message KGNode { message KGEdge { string source = 1; string target = 2; - string relation = 3; + string relation = 3; // absolute canonical predicate IRI // Full KG-edge field (add-only): the domain edge's metadata (JSON). optional bytes metadata = 4; } diff --git a/libatomic/src/daemon/services_agent.rs b/libatomic/src/daemon/services_agent.rs index 450b6fc7..290c57bd 100644 --- a/libatomic/src/daemon/services_agent.rs +++ b/libatomic/src/daemon/services_agent.rs @@ -46,6 +46,7 @@ use super::state::{domain_status, repository_error, DaemonState}; /// Tuning constants, mirroring the CLI retrieval layer's documented recipe. mod context_ranking { + use atomic_core::kg::{iri, project_id::ProjectId}; use atomic_core::pristine::tables::tokenize_for_fts; use atomic_core::pristine::vault::{KgNode, VaultEntry, VaultEntryType}; use atomic_core::types::{Base32, Hasher}; @@ -128,7 +129,7 @@ mod context_ranking { .vault_kg_neighbors(node_id, depth) .map_err(repository_error)?; for node in subgraph.nodes { - if node.kind == "memory" && node.id != node_id { + if node.has_type(atomic_core::kg::ontology::Class::Memory) && node.id != node_id { merge_candidate( candidates, &node.id, @@ -216,12 +217,12 @@ mod context_ranking { .and_then(|path| path.as_str()) .filter(|path| path.starts_with("memory/") && path.ends_with(".md")) .map(String::from) - .or_else(|| legacy_memory_node_id_to_path(&node.id)) + .or_else(|| memory_node_id_to_path(&node.id)) } /// Legacy fallback: `memory:architecture` -> `memory/architecture.md`. - fn legacy_memory_node_id_to_path(node_id: &str) -> Option { - let name = node_id.strip_prefix("memory:")?; + fn memory_node_id_to_path(node_id: &str) -> Option { + let (_, name) = iri::parse_memory(node_id).ok()?; if name.is_empty() { return None; } @@ -229,23 +230,12 @@ mod context_ranking { } /// `memory/architecture.md` -> `memory:architecture`. - fn memory_path_to_node_id(path: &str) -> Option { + fn memory_path_to_node_id(project: &ProjectId, path: &str) -> Option { let name = path.strip_prefix("memory/")?.strip_suffix(".md")?; if name.is_empty() { return None; } - Some(format!("memory:{}", name)) - } - - /// `intents/pimo-1/intent.md` -> `intent:PIMO-1` (nested paths keep - /// their inner segments, uppercased, matching the KG indexer). - fn intent_path_to_node_id(path: &str) -> String { - let id = path - .strip_prefix("intents/") - .and_then(|s| s.strip_suffix("/intent.md")) - .unwrap_or(path) - .to_uppercase(); - format!("intent:{}", id) + Some(iri::mint_memory(project, name)) } // Frontmatter helpers (the stored entries carry JSON-object strings). @@ -367,7 +357,10 @@ mod context_ranking { } for file in files { - let node_id = format!("file:{}", file.trim_start_matches("./")); + let node_id = iri::mint_file( + &repo.project_id().map_err(repository_error)?, + file.trim_start_matches("./"), + ); add_memory_neighbors(repo, &node_id, 1, &mut candidates)?; has_explicit_seeds = true; } @@ -376,7 +369,11 @@ mod context_ranking { if !seed_query.trim().is_empty() { let pool = limit.saturating_mul(SEARCH_POOL_MULTIPLIER); let nodes = repo - .vault_kg_search_by_kind(&seed_query, pool, "memory") + .vault_kg_search_by_kind( + &seed_query, + pool, + atomic_core::kg::ontology::Class::Memory, + ) .map_err(repository_error)?; for (rank, node) in nodes.iter().enumerate() { let base = rank_score(rank); @@ -404,7 +401,10 @@ mod context_ranking { if !is_eligible_memory(&entry) { continue; } - if let Some(node_id) = memory_path_to_node_id(&meta.path) { + if let Some(node_id) = memory_path_to_node_id( + &repo.project_id().map_err(repository_error)?, + &meta.path, + ) { merge_candidate(&mut candidates, &node_id, Some(meta.path), 0.0, false); } if candidates.len() >= limit { @@ -433,7 +433,9 @@ mod context_ranking { .vault_list("memory/", Some(VaultEntryType::Memory)) .map_err(repository_error)?; for meta in metas { - let Some(node_id) = memory_path_to_node_id(&meta.path) else { + let Some(node_id) = + memory_path_to_node_id(&repo.project_id().map_err(repository_error)?, &meta.path) + else { continue; }; let Some(entry) = repo.vault_retrieve(&meta.path).map_err(repository_error)? else { @@ -497,9 +499,12 @@ mod context_ranking { } } - let node_id = intent_path_to_node_id(&intent_path); - // Intent -> referenced file/domain -> memory is commonly two hops. - add_memory_neighbors(repo, &node_id, 2, candidates)?; + if let Ok(node_id) = iri::mint_intent( + &repo.project_id().map_err(repository_error)?, + &summary.uid.to_uppercase(), + ) { + add_memory_neighbors(repo, &node_id, 2, candidates)?; + } Ok(()) } @@ -518,7 +523,7 @@ mod context_ranking { let Some(path) = candidate .path .clone() - .or_else(|| legacy_memory_node_id_to_path(&node_id)) + .or_else(|| memory_node_id_to_path(&node_id)) else { continue; }; @@ -537,11 +542,11 @@ mod context_ranking { let memory_id = frontmatter_memory_id(&entry.frontmatter_json).unwrap_or_else(|| node_id.clone()); let name = frontmatter_name(&entry.frontmatter_json).unwrap_or_else(|| { - node_id - .rsplit_once(':') - .map(|(_, n)| n) - .unwrap_or(&node_id) - .to_string() + std::path::Path::new(&path) + .file_stem() + .unwrap_or_default() + .to_string_lossy() + .into_owned() }); let recency = recency_score(&entry.updated_at, now); let body = if include_body { @@ -3371,11 +3376,11 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl { // The --kind form: search the full pool, keep the // kind's nodes, take the limit AFTER filtering — the // local post-filter order. - let kind = kind_filter.to_lowercase(); + let kind = kind_filter; repo.vault_kg_search(&search.query, pool, Some(pool)) .map_err(repository_error)? .into_iter() - .filter(|node| node.kind.to_lowercase() == kind) + .filter(|node| node.types.contains(&kind)) .take(limit) .map(|node| crate::daemon::services_query::kg_node_proto(node.clone())) .collect::>() @@ -3387,6 +3392,8 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl { .collect() }; Ok::<_, Status>(QueryGraphResponse { + api_version: 1, + callers: Vec::new(), nodes, edges: Vec::new(), hits: Vec::new(), @@ -3407,6 +3414,8 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl { neighbors.depth.unwrap_or(1) as u8, )?; Ok(QueryGraphResponse { + api_version: 1, + callers: Vec::new(), nodes, edges, hits: Vec::new(), @@ -3431,12 +3440,12 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl { ) })?; let mut registry = atomic_semantic::ParserRegistry::new(); - let entities = registry.extract(&source, &entity.node_id); + let entities = registry.extract(&entity.node_id, &source); let nodes = entities .into_iter() .map(|entity| KgNode { id: format!("entity:{}:{}:{}", entity.file, entity.name, entity.line), - node_type: entity.kind.to_string(), + types: Vec::new(), name: Some(entity.name.clone()), data: entity.signature.clone().map(String::into_bytes), score: None, @@ -3444,6 +3453,8 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl { }) .collect(); Ok(QueryGraphResponse { + api_version: 1, + callers: Vec::new(), nodes, edges: Vec::new(), hits: Vec::new(), @@ -3458,53 +3469,21 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl { } QueryOneof::Callers(callers) => { let repo = handle.repository_readonly()?; - // The local body's domain: the 1-hop neighbors subgraph, - // filtered to CALLS edges pointing at the entity. - let (nodes, edges) = - crate::daemon::services_query::neighbors_impl(&repo, &callers.node_id, 1)?; - let subgraph = repo - .vault_kg_neighbors(&callers.node_id, 1) - .map_err(repository_error)?; - let calls_present = subgraph - .edges - .iter() - .any(|edge| edge.kind.to_uppercase() == "CALLS"); - // The caller edges + the caller nodes (their summaries ride - // the nodes so the CLI renders the caller lines). - let caller_edges: Vec = subgraph - .edges - .iter() - .filter(|edge| { - edge.kind.to_uppercase() == "CALLS" && edge.to_id == callers.node_id - }) - .map(|edge| crate::daemon::services_query::kg_edge_proto(edge.clone())) - .collect(); - let caller_ids: std::collections::HashSet<&str> = subgraph - .edges - .iter() - .filter(|edge| { - edge.kind.to_uppercase() == "CALLS" && edge.to_id == callers.node_id + let callers = repo + .query_callers(&callers.node_id) + .map_err(repository_error)? + .into_iter() + .map(|caller| crate::atomic::CodeCaller { + path: caller.path, + name: caller.name, + line: caller.line, + summary: caller.summary, }) - .map(|edge| edge.from_id.as_str()) - .collect(); - let caller_nodes: Vec = subgraph - .nodes - .iter() - .filter(|node| caller_ids.contains(node.id.as_str())) - .map(|node| crate::daemon::services_query::kg_node_proto(node.clone())) .collect(); - let _ = (nodes, edges); Ok(QueryGraphResponse { - nodes: caller_nodes, - edges: caller_edges, - hits: Vec::new(), - answer: None, - plan_result: None, - snapshot: None, - graph_collected: None, - calls_present, - ask_result: None, - ask_elapsed_ms: None, + api_version: 1, + callers, + ..Default::default() }) } QueryOneof::GraphSlice(graph) => { @@ -3531,13 +3510,12 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl { } else { kinds .split(',') - .map(|kind| kind.trim().to_lowercase()) + .map(|kind| kind.trim().to_owned()) .filter(|kind| !kind.is_empty()) .collect() }; - let kind_allowed = |kind: &str| -> bool { - allow_all || allowed_kinds.contains(&kind.to_lowercase()) - }; + let kind_allowed = + |kind: &str| -> bool { allow_all || allowed_kinds.contains(kind) }; let changes_per_seed = graph.changes_per_seed.unwrap_or(5) as usize; let cap_changes = changes_per_seed > 0; // 1. Seed: search, filtered to allowed kinds, take limit. @@ -3545,11 +3523,13 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl { .vault_kg_search(&query, limit * 3, None) .map_err(repository_error)? .into_iter() - .filter(|node| kind_allowed(&node.kind)) + .filter(|node| node.types.iter().any(|class| kind_allowed(class))) .take(limit) .collect(); if seed_nodes.is_empty() { return Ok(QueryGraphResponse { + api_version: 1, + callers: Vec::new(), nodes: Vec::new(), edges: Vec::new(), hits: Vec::new(), @@ -3580,10 +3560,10 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl { .map_err(repository_error)?; let mut changes_for_seed: usize = 0; for node in subgraph.nodes { - if !kind_allowed(&node.kind) { + if !node.types.iter().any(|class| kind_allowed(class)) { continue; } - if node.kind == "change" && cap_changes { + if node.has_type(atomic_core::kg::ontology::Class::Change) && cap_changes { if node_map.contains_key(&node.id) { continue; } @@ -3595,20 +3575,18 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl { node_map.entry(node.id.clone()).or_insert(node); } for edge in subgraph.edges { - let from_kind_ok = allow_all || { - let prefix = edge.from_id.split(':').next().unwrap_or(""); - kind_allowed(prefix) - }; - let to_kind_ok = allow_all || { - let prefix = edge.to_id.split(':').next().unwrap_or(""); - kind_allowed(prefix) - }; + let from_kind_ok = node_map.contains_key(&edge.from_id); + let to_kind_ok = node_map.contains_key(&edge.to_id); if from_kind_ok && to_kind_ok && (node_map.contains_key(&edge.from_id) || node_map.contains_key(&edge.to_id)) { - let key = (edge.from_id.clone(), edge.to_id.clone(), edge.kind.clone()); + let key = ( + edge.from_id.clone(), + edge.to_id.clone(), + edge.kind.to_string(), + ); if edge_set.insert(key) { all_edges.push(edge); } @@ -3654,6 +3632,8 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl { collected = nodes.len() as u32; let _ = &collected; Ok(QueryGraphResponse { + api_version: 1, + callers: Vec::new(), nodes: nodes .iter() .map(|node| crate::daemon::services_query::kg_node_proto(node.clone())) @@ -3685,6 +3665,8 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl { let payload = serde_json::to_vec(&result) .map_err(|error| Status::internal(format!("plan encode: {error}")))?; Ok(QueryGraphResponse { + api_version: 1, + callers: Vec::new(), nodes: Vec::new(), edges: Vec::new(), hits: Vec::new(), @@ -3727,6 +3709,8 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl { let payload = serde_json::to_vec(&result) .map_err(|error| Status::internal(format!("ask encode: {error}")))?; Ok(QueryGraphResponse { + api_version: 1, + callers: Vec::new(), nodes: Vec::new(), edges: Vec::new(), hits: Vec::new(), @@ -3756,6 +3740,16 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl { &self, request: Request, ) -> Result, Status> { + let authorization = request + .metadata() + .get("authorization") + .and_then(|value| value.to_str().ok()) + .map(str::to_owned); + let delegation = request + .metadata() + .get("atomic-delegation") + .and_then(|value| value.to_str().ok()) + .map(str::to_owned); let request = request.into_inner(); let action = KnowledgeMaintainAction::try_from(request.action) .unwrap_or(KnowledgeMaintainAction::Unspecified); @@ -3763,6 +3757,25 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl { self.state.log_rpc("MaintainKnowledgeGraph", Some(&handle)); let gate_handle = handle.clone(); let _gate = gate_handle.exclusive().await; + if action == KnowledgeMaintainAction::Reindex { + if let Some(authorization) = authorization { + let repo = handle.repository()?; + if let Ok((_, remote)) = repo.get_default_remote() { + let mut config = atomic_remote::HttpRemoteConfig::new() + .with_header("Authorization", authorization); + if let Some(delegation) = delegation { + config = config.with_header("Atomic-Delegation", delegation); + } + if let Some(client) = + crate::signer_refresh::client_for_remote(&remote.url, &config) + { + crate::signer_refresh::refresh(&repo, &client) + .await + .map_err(repository_error)?; + } + } + } + } let meta = request.meta.clone(); let response = tokio::task::spawn_blocking(move || { let (repo, workspace) = @@ -3797,31 +3810,16 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl { } // The CLI --rebuild flag rides the scope field ("rebuild"). if request.scope.as_deref() == Some("rebuild") { - repo.kg_clear_derived().map_err(repository_error)?; + repo.rebuild_kg(true).map_err(repository_error)?; } - let mut total: u64 = 0; - total += repo.kg_enrich_views().map_err(repository_error)? as u64; - total += repo - .kg_enrich_files(working_copy) - .map_err(repository_error)? as u64; - total += repo - .kg_enrich_modules(working_copy) - .map_err(repository_error)? as u64; - total += repo - .kg_enrich_changes(working_copy) - .map_err(repository_error)? as u64; - total += repo - .kg_enrich_entities(working_copy) - .map_err(repository_error)? as u64; - total += repo - .kg_enrich_includes(working_copy) - .map_err(repository_error)? as u64; - total += repo - .kg_enrich_calls(working_copy) - .map_err(repository_error)? as u64; - total + repo.kg_enrich_from_vcs(working_copy) + .map_err(repository_error)? + .total() as u64 } KnowledgeMaintainAction::Reindex => { + if request.scope.as_deref() == Some("full") { + repo.rebuild_kg(true).map_err(repository_error)?; + } repo.vault_reindex_kg().map_err(repository_error)? as u64 } // `atomic query embed` — the provider resolves from the diff --git a/libatomic/src/daemon/services_query.rs b/libatomic/src/daemon/services_query.rs index e6d983d7..02312086 100644 --- a/libatomic/src/daemon/services_query.rs +++ b/libatomic/src/daemon/services_query.rs @@ -1173,18 +1173,18 @@ Strategy: code_search first to find the exact line range you need, then read just that range. 5. Answer concisely using what you found. Cite file paths and line numbers. -The knowledge graph contains: -- file nodes (file:path) — tracked files -- entity nodes (entity:file:name:line) — functions, structs, classes from tree-sitter -- change nodes (change:hash) — commit history with dates and messages -- view nodes (view:name) — like branches -- goal/intent/memory nodes — development sessions and work items -- edges: MODIFIES, DEFINES, AUTHORED_BY, DEPENDS_ON, REFERENCES, ON_VIEW, etc."; +The knowledge graph contains canonical Atomic workflow resources with absolute IRIs +and asserted class types. Views filter membership over one common causal graph. +Workflow predicates include atomic:modifiesFile, atomic:authoredBy, +atomic:directlyDependsOnChange, atomic:directlyOnView, atomic:hasPlan, +atomic:generatedChange, atomic:implements, atomic:attestedBy, and atomic:references. +The atomic namespace is https://atomic.dev/ns/workflow#. +Use live tree-sitter entity/reference queries for code structure."; pub(crate) fn kg_node_proto(node: atomic_core::pristine::vault::KgNode) -> KgNode { KgNode { id: node.id, - node_type: node.kind, + types: node.types, name: Some(node.label), data: node.summary.clone().map(String::into_bytes), score: None, @@ -1200,7 +1200,7 @@ pub(crate) fn kg_edge_proto(edge: atomic_core::pristine::vault::KgEdge) -> KgEdg KgEdge { source: edge.from_id, target: edge.to_id, - relation: edge.kind, + relation: edge.kind.to_string(), metadata: edge .metadata .and_then(|metadata| serde_json::to_vec(&metadata).ok()), diff --git a/libatomic/src/daemon/services_sync_remote.rs b/libatomic/src/daemon/services_sync_remote.rs index b77e7244..4ce874af 100644 --- a/libatomic/src/daemon/services_sync_remote.rs +++ b/libatomic/src/daemon/services_sync_remote.rs @@ -1167,6 +1167,7 @@ pub async fn pull_changes_impl( // Connect. let config = remote_config(timeout_secs, insecure, &auth_headers); + let signer_client = crate::signer_refresh::client_for_remote(&remote_url, &config); let remote = HttpRemote::with_config(&remote_url, config).map_err(|error| { let (variant, message) = convert_remote_error_pull(error, &remote_url); sync_error("connect", variant, &remote_url, message) @@ -1410,6 +1411,12 @@ pub async fn pull_changes_impl( .await .map_err(|error| Status::internal(error.to_string()))?? }; + if let Some(client) = &signer_client { + let repo = handle.repository()?; + crate::signer_refresh::refresh(&repo, client) + .await + .map_err(repository_error)?; + } let report = PullReport { remote_name, remote_url, @@ -1516,6 +1523,13 @@ pub async fn pull_changes_impl( .map_err(|error| Status::internal(error.to_string()))?? }; + if let Some(client) = &signer_client { + let repo = handle.repository()?; + crate::signer_refresh::refresh(&repo, client) + .await + .map_err(repository_error)?; + } + // Set-id verification (the server's effective set-id vs the local). let mut set_id_verified = false; let mut set_id_warning: Option = None; diff --git a/libatomic/src/lib.rs b/libatomic/src/lib.rs index 11e1846c..4f7c44c9 100644 --- a/libatomic/src/lib.rs +++ b/libatomic/src/lib.rs @@ -35,6 +35,7 @@ pub use crate::atomic as proto; pub mod daemon; pub mod local; +pub mod signer_refresh; #[cfg(feature = "node")] mod node; diff --git a/libatomic/src/signer_refresh.rs b/libatomic/src/signer_refresh.rs new file mode 100644 index 00000000..c7fb1392 --- /dev/null +++ b/libatomic/src/signer_refresh.rs @@ -0,0 +1,50 @@ +//! Asynchronous signer discovery shared by CLI and daemon orchestration. +//! Projection and cache writes remain offline repository operations. + +use atomic_identity::PublicKey; +use atomic_remote::{HttpRemoteConfig, StorageClient}; +use atomic_repository::{Repository, RepositoryError}; + +/// Reuse the authenticated remote's origin and credentials, never an unrelated +/// globally selected server. Anonymous remotes do not trigger discovery. +pub fn client_for_remote(remote: &str, config: &HttpRemoteConfig) -> Option { + let header = |name: &str| { + config + .extra_headers + .iter() + .find(|(key, _)| key.eq_ignore_ascii_case(name)) + .map(|(_, value)| value.as_str()) + }; + let token = header("authorization")?.strip_prefix("Bearer ")?; + let origin = url::Url::parse(remote).ok()?.origin().ascii_serialization(); + StorageClient::with_delegation(&origin, "", token, header("atomic-delegation")).ok() +} + +/// Discover only unresolved DIDs, with no pristine transaction held over I/O. +/// Older storage versions cannot resolve fingerprint DIDs: that is a cache +/// miss, never permission to interpret the fingerprint bytes as a public key. +pub async fn refresh(repo: &Repository, client: &StorageClient) -> Result { + let signers = repo.unresolved_attestation_signers()?; + let mut cached = 0; + for did in signers { + let response = tokio::time::timeout( + std::time::Duration::from_secs(10), + client.resolve_identity_by_public_key(&did), + ) + .await; + let Some(key) = response + .ok() + .and_then(Result::ok) + .and_then(|identity| identity.public_key) + .and_then(|key| PublicKey::from_base32(&key).ok()) + else { + continue; + }; + if !atomic_canonical::did::did_matches_public_key(&did, &key) { + continue; + } + repo.cache_signer_key(&did, &key)?; + cached += 1; + } + Ok(cached) +} diff --git a/libatomic/tests/signer_refresh.rs b/libatomic/tests/signer_refresh.rs new file mode 100644 index 00000000..a9a89fd5 --- /dev/null +++ b/libatomic/tests/signer_refresh.rs @@ -0,0 +1,112 @@ +use atomic_core::{ + change::{AttestAgent, Attestation}, + types::Hash, +}; +use atomic_identity::KeyPair; +use atomic_remote::{HttpRemoteConfig, StorageClient}; +use atomic_repository::Repository; +use std::{ + io::{Read, Write}, + net::TcpListener, +}; + +fn serve_key(did: String, public_key: String) -> (StorageClient, std::thread::JoinHandle<()>) { + let listener = TcpListener::bind("127.0.0.1:0").unwrap(); + let address = listener.local_addr().unwrap(); + let server = std::thread::spawn(move || { + let (mut stream, _) = listener.accept().unwrap(); + stream + .set_read_timeout(Some(std::time::Duration::from_secs(10))) + .unwrap(); + let mut request = Vec::new(); + let mut buffer = [0; 1024]; + while !request.windows(4).any(|bytes| bytes == b"\r\n\r\n") { + let count = stream.read(&mut buffer).unwrap(); + assert_ne!(count, 0); + request.extend_from_slice(&buffer[..count]); + } + let request = String::from_utf8(request).unwrap(); + assert!(request.starts_with(&format!("GET /identities/by-key/{did} "))); + assert!(request + .to_ascii_lowercase() + .contains("authorization: bearer fixture-token")); + let body = serde_json::json!({"success": true, "data": { + "id": "00000000-0000-0000-0000-000000000001", "name": "remote signer", + "status": "active", "createdAt": "2026-10-15T00:00:00Z", "publicKey": public_key + }}) + .to_string(); + write!(stream, "HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{}", body.len(), body).unwrap(); + }); + let client = StorageClient::new(&format!("http://{address}"), "", "fixture-token").unwrap(); + (client, server) +} + +#[tokio::test] +async fn fetched_keys_are_did_checked_before_cache_and_offline_verification() { + use atomic_core::kg::{iri, ontology::Predicate as P}; + let directory = tempfile::tempdir().unwrap(); + let repo = Repository::init(directory.path()).unwrap(); + let key = KeyPair::generate(); + let did = atomic_canonical::did::did_for_public_key(&key.public); + let mut attestation = + Attestation::builder("remote-session", AttestAgent::new("agent", "Agent", "1")) + .add_change(Hash::of(b"covered change")) + .build(); + attestation.sign_with_did(&did, key.secret.as_bytes()); + let hash = repo.save_attestation(&attestation).unwrap(); + let status = || { + repo.workflow_snapshot() + .unwrap() + .edges + .into_iter() + .find(|edge| { + edge.from_id == iri::mint_attestation(&hash) && edge.kind == P::VerificationResult + }) + .unwrap() + .to_id + }; + assert!(status().ends_with("#Unverifiable")); + + let (client, server) = serve_key(did.clone(), KeyPair::generate().public.to_base32()); + assert_eq!( + libatomic::signer_refresh::refresh(&repo, &client) + .await + .unwrap(), + 0 + ); + server.join().unwrap(); + assert!(repo + .pristine() + .read_txn() + .unwrap() + .signer_key(&did) + .unwrap() + .is_none()); + assert!(status().ends_with("#Unverifiable")); + + let (client, server) = serve_key(did.clone(), key.public.to_base32()); + assert_eq!( + libatomic::signer_refresh::refresh(&repo, &client) + .await + .unwrap(), + 1 + ); + server.join().unwrap(); + assert!(status().ends_with("#Verified")); + // No second fetch: cache hits are offline, even after the server exited. + assert_eq!( + libatomic::signer_refresh::refresh(&repo, &client) + .await + .unwrap(), + 0 + ); +} + +#[test] +fn anonymous_remotes_do_not_start_signer_discovery() { + assert!(libatomic::signer_refresh::client_for_remote( + "https://example.test/project/code", + &HttpRemoteConfig::new() + ) + .is_none()); +} diff --git a/tests/harness/14_vault_kg.sh b/tests/harness/14_vault_kg.sh index ba0e975b..0b2255be 100755 --- a/tests/harness/14_vault_kg.sh +++ b/tests/harness/14_vault_kg.sh @@ -214,7 +214,8 @@ run_cmd atomic vault query search "auth" --json assert_ran "KG search finds 'auth' content" "auth" # Query neighbors of a view node — returns a node/edge graph -run_cmd atomic vault query neighbors "view:dev" --json +project_urn="$(atomic repo identity show --json | python3 -c 'import json,sys; print(json.load(sys.stdin)["urn"])')" +run_cmd atomic vault query neighbors "${project_urn}:view:dev" --json assert_ran "KG neighbors query for view node" '"nodes"' # Reindex @@ -294,7 +295,8 @@ else assert_ran "KG search finds imported content" "uuid" # Neighbors query for the imported default view - run_cmd atomic vault query neighbors "view:main" --json + project_urn="$(atomic repo identity show --json | python3 -c 'import json,sys; print(json.load(sys.stdin)["urn"])')" + run_cmd atomic vault query neighbors "${project_urn}:view:main" --json assert_ran "KG neighbors query after import" '"nodes"' # Reindex vault entries diff --git a/tests/harness/46_knowledge_graph_closure.sh b/tests/harness/46_knowledge_graph_closure.sh new file mode 100644 index 00000000..d68f531a --- /dev/null +++ b/tests/harness/46_knowledge_graph_closure.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +# Phase 1: ontology/identity, real producer closure, RDF/SHACL, CLI, and +# pre-cutover caller/triage result-set parity. All repositories are temporary. +set -euo pipefail +HARNESS_DIR="$(cd "$(dirname "$0")" && pwd)" +source "$HARNESS_DIR/helpers.sh" +python3 "$HARNESS_DIR/kg_closure_e2e.py" --binary "$ATOMIC_BIN" +python3 "$HARNESS_DIR/kg_closure_baseline.py" --binary "$ATOMIC_BIN" --check +cargo test --manifest-path "$HARNESS_DIR/../../Cargo.toml" -p atomic-core --test kg_ontology --test kg_iri --test kg_vocabulary_lint --quiet +cargo test --manifest-path "$HARNESS_DIR/../../Cargo.toml" -p atomic-repository --test project_identity --test kg_projection --quiet +cargo test --manifest-path "$HARNESS_DIR/../../Cargo.toml" -p atomic-shacl --quiet +cargo test --manifest-path "$HARNESS_DIR/../../Cargo.toml" -p libatomic --test signer_refresh --quiet diff --git a/tests/harness/fixtures/kg-closure-baseline.json b/tests/harness/fixtures/kg-closure-baseline.json new file mode 100644 index 00000000..078066cc --- /dev/null +++ b/tests/harness/fixtures/kg-closure-baseline.json @@ -0,0 +1,9 @@ +{ + "callers": { + "src/foundation.rs#foundation": ["src/service.rs#service"], + "src/service.rs#service": ["src/main.rs#main"] + }, + "layers": [ + {"title": "src", "files": ["src/foundation.rs", "src/main.rs", "src/service.rs"]} + ] +} diff --git a/tests/harness/kg_closure_baseline.py b/tests/harness/kg_closure_baseline.py new file mode 100644 index 00000000..9e62e372 --- /dev/null +++ b/tests/harness/kg_closure_baseline.py @@ -0,0 +1,76 @@ +"""Capture/compare result sets, independent of KG IDs and CLI presentation.""" + +import argparse +import json +import os +from pathlib import Path +import subprocess +import tempfile +from urllib.parse import unquote + + +def capture(binary, legacy): + with tempfile.TemporaryDirectory(prefix="atomic-kg-baseline-") as directory: + root = Path(directory) + env = dict(os.environ, ATOMIC_NO_DAEMON="1") + + def run(*args, json_output=False): + result = subprocess.run( + [binary, *args], cwd=root, env=env, text=True, capture_output=True + ) + if result.returncode and not (json_output and result.stdout.lstrip().startswith("{")): + raise AssertionError(f"{args}: {result.returncode}\n{result.stdout}\n{result.stderr}") + return json.loads(result.stdout) if json_output else result.stdout + + run("init", "--view", "main") + run("view", "create", "feature", "--draft", "--parent", "main") + run("view", "switch", "feature") + files = { + "src/foundation.rs": "pub fn foundation() -> u32 { 1 }\n", + "src/service.rs": "use crate::foundation::foundation;\npub fn service() -> u32 { foundation() }\n", + "src/main.rs": "mod foundation;\nmod service;\nfn main() { service::service(); }\n", + } + for path, source in files.items(): + (root / path).parent.mkdir(parents=True, exist_ok=True) + (root / path).write_text(source) + run("add", "src") + run("record", "-m", "Add workflow baseline fixture") + run("query", "enrich") + callers = {} + for path, name, line in [ + ("src/foundation.rs", "foundation", 1), + ("src/service.rs", "service", 2), + ]: + target = f"entity:{path}:{name}:{line}" if legacy else f"{path}#{name}" + rows = run("query", "callers", target, "--json", json_output=True) + identities = [] + for row in rows: + if "path" in row: + identities.append(f"{row['path']}#{row['name']}") + else: + caller = row["caller_id"].removeprefix("entity:").rsplit(":", 2) + identities.append(f"{caller[0]}#{caller[1]}") + callers[f"{path}#{name}"] = sorted(set(identities)) + report = run("triage", "review", "feature", "--into", "main", "--json", json_output=True) + layers = [ + {"title": layer["title"], "files": sorted( + path.removeprefix("file:") if legacy else unquote(path.split(":file:", 1)[1]) + for path in layer["files"])} + for layer in report["walkthrough"] + ] + return {"callers": callers, "layers": layers} + + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument("--binary", required=True) + parser.add_argument("--legacy", action="store_true") + parser.add_argument("--check", action="store_true") + args = parser.parse_args() + result = capture(args.binary, args.legacy) + if args.check: + expected = json.loads((Path(__file__).parent / "fixtures/kg-closure-baseline.json").read_text()) + assert result == expected, json.dumps({"expected": expected, "actual": result}, indent=2) + print("Callers and triage result sets match the captured pre-cutover baseline.") + else: + print(json.dumps(result, indent=2, sort_keys=True)) diff --git a/tests/harness/kg_closure_e2e.py b/tests/harness/kg_closure_e2e.py new file mode 100644 index 00000000..031508a0 --- /dev/null +++ b/tests/harness/kg_closure_e2e.py @@ -0,0 +1,123 @@ +"""Black-box Phase 1 acceptance checks; failures retain command diagnostics.""" +import argparse +import json +import os +from pathlib import Path +import re +import subprocess +import tempfile + + +def check(binary): + with tempfile.TemporaryDirectory(prefix="atomic-kg-closure-") as directory: + base = Path(directory) + root, home = base / "repo", base / "home" + root.mkdir() + home.mkdir() + env = dict(os.environ, HOME=str(home), ATOMIC_NO_DAEMON="1") + + def run(*args, status=0, stdin=None, as_json=False): + result = subprocess.run([binary, *args], cwd=root, env=env, + input=stdin, text=True, capture_output=True) + assert result.returncode == status, ( + f"{args}: expected {status}, got {result.returncode}\n" + f"{result.stdout}\n{result.stderr}" + ) + return json.loads(result.stdout) if as_json else result.stdout + + run("init", "--view", "main") + identity = run("repo", "identity", "show", "--json", as_json=True) + assert re.fullmatch(r"[A-Z2-7]{51}[AQ]", identity["project"]) + assert identity["minted"] == "created" + assert identity["urn"] == "urn:atomic:project:" + identity["project"] + empty = run("query", "validate", "--shape", "https://atomic.dev/ns/shapes/projection/1.0.0#GeneratedChange", "--format", "json", as_json=True) + assert empty["conforms"] and all(c["evaluated"] == 0 for c in empty["target_coverage"]) + assert any("vacuous" in diagnostic for diagnostic in empty["diagnostics"]) + + run("vault", "init") + uid = "01ARZ3NDEKTSV4RRFFQ69G5FAV" + intent = root / ".vault" / "intents" / uid / "intent.md" + intent.parent.mkdir(parents=True, exist_ok=True) + intent.write_text(f"""--- +uid: {uid} +id: TEST::alice::1 +title: Workflow harness +kind: feature +status: in_progress +author: Alice +--- +:::why +Verify the real workflow projection end to end. +::: +:::acceptance-criterion{{#{uid}-ac-1 status=unmet}} +The source is represented by a file resource. +::: +:::task{{#{uid}-1 status=unmet criteria={uid}-ac-1}} +Inspect the source. +::file-ref{{path=src/main.rs}} +::: +""") + (root / "src").mkdir(exist_ok=True) + (root / "src/main.rs").write_text("fn main() {}\n") + run("add", "src/main.rs") + run("record", "-m", "Workflow harness source") + run("vault", "sync") + run("query", "reindex", "--full") + + plan = {"schema_version": 1, "steps": [ + {"type": "kg_search", "query": "Workflow harness", "limit": 100, "bind": "all"}, + {"type": "filter", "input": "all", "field": "type", + "equals": "https://atomic.dev/ns/workflow#WorkIntent", "bind": "intents"}, + {"type": "kg_neighbors", "node_id": "$intents", "direction": "outgoing", + "predicate": "https://atomic.dev/ns/workflow#hasTask", "depth": 1}, + ]} + result = run("query", "plan", "--json", stdin=json.dumps(plan), as_json=True) + assert any(node["id"] == identity["urn"] + ":intent:" + uid for node in result["nodes"]) + assert any(edge["kind"] == "https://atomic.dev/ns/workflow#hasTask" for edge in result["edges"]) + assert all(node["types"] and "kind" not in node for node in result["nodes"]) + for invalid in [{"steps": []}, {"schema_version": 99, "steps": []}, + {"schema_version": 1, "steps": [], "unexpected": True}]: + failure = subprocess.run([binary, "query", "plan", "--json"], cwd=root, env=env, + input=json.dumps(invalid), text=True, capture_output=True) + assert failure.returncode != 0 and '"nodes"' not in failure.stdout + + projection = run("query", "validate", "--format", "json", as_json=True) + assert projection["conforms"] and any(c["evaluated"] > 0 for c in projection["target_coverage"]) + publication = run("query", "validate", "--profile", "publication", "--format", "json", status=1, as_json=True) + assert not publication["conforms"] and publication["results"] + assert publication["report_ntriples"] + for field in ["focus_node", "result_path", "source_shape", "source_constraint_component", "result_severity", "result_messages"]: + assert field in publication["results"][0] + run("query", "reindex", "--full") + rebuilt = run("query", "validate", "--format", "json", as_json=True) + assert rebuilt["snapshot_identity"] == projection["snapshot_identity"] + run("query", "enrich", "--rebuild") + assert run("query", "validate", "--format", "json", as_json=True)["snapshot_identity"] == projection["snapshot_identity"] + assert run("repo", "identity", "show", "--json", as_json=True) == identity + + shapes = root / ".atomic/shapes" + shapes.mkdir() + custom = shapes / "enterprise.ttl" + custom.write_text('''@prefix sh: . +@prefix atomic: . +<#Enterprise> a sh:NodeShape ; sh:targetClass atomic:ProjectIdentity ; + sh:property [ sh:path atomic:status ; sh:minCount 1 ; sh:message "Enterprise status required"@en ] . +''') + report = run("query", "validate", "--format", "json", status=1, as_json=True) + assert not report["conforms"] + selected = run("query", "validate", "--shape", custom.resolve().as_uri() + "#Enterprise", "--format", "json", status=1, as_json=True) + assert selected["scope"] == "selected" + assert any("Enterprise status required" in str(row) for row in selected["results"]) + custom.write_text("<> .") + run("query", "validate", "--format", "json", status=3) + custom.write_text("this is not Turtle") + run("query", "validate", "--format", "json", status=3) + custom.unlink() + assert run("query", "validate", "--format", "json", as_json=True)["conforms"] + print("KG closure CLI: identity, canonical plans, rebuild parity, profiles, custom shapes, and error exits passed.") + + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument("--binary", required=True) + check(str(Path(parser.parse_args().binary).resolve())) From 7f5f6e90530dbaaa54c073c605f4de5459ea21e5 Mon Sep 17 00:00:00 2001 From: Bradley Hilton Date: Fri, 16 Oct 2026 11:51:36 -0500 Subject: [PATCH 12/19] fix: route KG inspection and reindex through RPC --- atomic-cli/src/commands/pull/command.rs | 13 -- atomic-cli/src/commands/query/mod.rs | 44 +--- atomic-cli/src/commands/query/validate.rs | 8 +- atomic-cli/src/commands/repo.rs | 49 ++--- atomic-cli/src/commands/rpc.rs | 54 ++++- atomic-cli/src/service/mod.rs | 40 ++++ .../tests/kg_service_integration_test.rs | 196 ++++++++++++++++++ docs/knowledge-graph-phase1-upgrade.md | 7 + .../knowledge/knowledge_messages.proto | 33 ++- .../knowledge/knowledge_service.proto | 13 ++ libatomic/src/daemon/services.rs | 18 ++ libatomic/src/daemon/services_agent.rs | 133 +++++++++--- libatomic/tests/signer_refresh.rs | 78 ++++++- tests/harness/46_knowledge_graph_closure.sh | 1 + 14 files changed, 561 insertions(+), 126 deletions(-) create mode 100644 atomic-cli/tests/kg_service_integration_test.rs diff --git a/atomic-cli/src/commands/pull/command.rs b/atomic-cli/src/commands/pull/command.rs index 1369ff6b..d914fac1 100644 --- a/atomic-cli/src/commands/pull/command.rs +++ b/atomic-cli/src/commands/pull/command.rs @@ -530,7 +530,6 @@ impl Pull { let config = self .build_remote_config(&remote_url, identity_hint.as_deref()) .await; - let signer_client = libatomic::signer_refresh::client_for_remote(&remote_url, &config); let remote = HttpRemote::with_config(&remote_url, config).map_err(|e| { finish_error(&spinner, "Failed to connect"); convert_remote_error(e, &remote_url) @@ -768,11 +767,6 @@ impl Pull { crate::commands::sidecars::report_sidecars(sidecar_stats); } } - if let Some(client) = &signer_client { - if let Err(error) = libatomic::signer_refresh::refresh(&repo, client).await { - log::warn!("signer key refresh after download failed: {error}"); - } - } print_blank(); print_success(&format!( "Downloaded {} (not inserted - use 'atomic insert' to insert)", @@ -966,13 +960,6 @@ impl Pull { } } - // Ingestion/membership APIs project regardless of download/apply counts. - if let Some(client) = &signer_client { - if let Err(error) = libatomic::signer_refresh::refresh(&repo, client).await { - log::warn!("signer key refresh after pull failed: {error}"); - } - } - // Keep partial output, but return failure when any step failed. print_blank(); if stats.has_failures() || !apply_errors.is_empty() || materialize_failed { diff --git a/atomic-cli/src/commands/query/mod.rs b/atomic-cli/src/commands/query/mod.rs index a66b8bca..18f82cea 100644 --- a/atomic-cli/src/commands/query/mod.rs +++ b/atomic-cli/src/commands/query/mod.rs @@ -1690,49 +1690,7 @@ pub struct Reindex { impl Command for Reindex { fn run(&self) -> CliResult<()> { - let root = find_repository_root()?; - // Key discovery precedes both local and daemon reindex execution. A - // missing remote or authentication leaves attestations Unverifiable. - { - let repo = Repository::open(&root).map_err(CliError::Repository)?; - if let Ok((_, remote)) = repo.get_default_remote() { - let runtime = tokio::runtime::Builder::new_current_thread() - .enable_all() - .build() - .map_err(|error| CliError::Internal(error.into()))?; - runtime.block_on(async { - let config = crate::commands::auth::attach_identity( - atomic_remote::HttpRemoteConfig::new(), - &remote.url, - None, - ) - .await; - if let Some(client) = - libatomic::signer_refresh::client_for_remote(&remote.url, &config) - { - if let Err(error) = libatomic::signer_refresh::refresh(&repo, &client).await - { - log::warn!("signer refresh: {error}"); - } - } - }); - } - } - // Route every form through the service layer (the handler runs - // the same vault_reindex_kg domain call). - if crate::commands::rpc::query_reindex(self.full)? { - return Ok(()); - } - let root = find_repository_root()?; - let repo = Repository::open(&root).map_err(CliError::Repository)?; - - if self.full { - repo.rebuild_kg(true).map_err(CliError::Repository)?; - } - let count = repo.vault_reindex_kg().map_err(CliError::Repository)?; - println!("Indexed {} nodes + edges.", count); - - Ok(()) + crate::commands::rpc::query_reindex(self.full) } } diff --git a/atomic-cli/src/commands/query/validate.rs b/atomic-cli/src/commands/query/validate.rs index 9b8e9a9b..52915268 100644 --- a/atomic-cli/src/commands/query/validate.rs +++ b/atomic-cli/src/commands/query/validate.rs @@ -30,8 +30,9 @@ impl Command for Validate { fn run(&self) -> CliResult<()> { // Validation deliberately runs in the CLI process so neither the // daemon nor shared libraries acquire the SHACL engine's MSRV. - let repo = Repository::open(find_repository_root()?).map_err(CliError::Repository)?; - let snapshot = repo.workflow_snapshot().map_err(CliError::Repository)?; + let root = find_repository_root()?; + let snapshot = crate::commands::rpc::workflow_snapshot(&root)?; + let dot_dir = Repository::canonical_dot_dir(&root).map_err(CliError::Repository)?; let error = |error: atomic_shacl::ValidationError| CliError::ShaclError(error.to_string()); let data = atomic_shacl::snapshot_ntriples(&snapshot.nodes, &snapshot.edges).map_err(error)?; @@ -42,8 +43,7 @@ impl Command for Validate { }, shape: self.shape.clone(), node: self.node.clone(), - custom: atomic_shacl::load_custom_shapes(&repo.dot_dir().join("shapes")) - .map_err(error)?, + custom: atomic_shacl::load_custom_shapes(&dot_dir.join("shapes")).map_err(error)?, }; let report = atomic_shacl::validate_ntriples(&data, &snapshot.identity, &request).map_err(error)?; diff --git a/atomic-cli/src/commands/repo.rs b/atomic-cli/src/commands/repo.rs index dba7e8bc..2575b0c8 100644 --- a/atomic-cli/src/commands/repo.rs +++ b/atomic-cli/src/commands/repo.rs @@ -2,8 +2,8 @@ use super::{find_repository_root, Command}; use crate::error::{CliError, CliResult}; -use atomic_core::types::Base32; -use atomic_repository::Repository; +use crate::service::Service; +use atomic_client::proto as pb; use clap::{Parser, Subcommand}; #[derive(Debug, Parser)] @@ -29,21 +29,13 @@ enum IdentityCommand { impl Command for Repo { fn run(&self) -> CliResult<()> { - let repo = Repository::open(find_repository_root()?).map_err(CliError::Repository)?; - let record = repo - .project_identity() - .map_err(CliError::Repository)? - .ok_or_else(|| CliError::Internal(anyhow::anyhow!("MissingProjectIdentity")))?; - let project = record - .project_id() - .map_err(|e| CliError::Internal(e.into()))?; - let object: serde_json::Value = - serde_json::from_slice(&record.object).map_err(|e| CliError::Internal(e.into()))?; - let signed_by = object - .get("atomic:issuer") - .and_then(|v| v.get("@id")) - .and_then(|v| v.as_str()); - let minted = serde_json::to_value(record.mint_reason).expect("enum serialization"); + let root = find_repository_root()?; + let service = Service::open_root(&root)?.ok_or_else(|| CliError::InvalidArgument { + message: "repository identity inspection requires a local repository service".into(), + })?; + let record = service.get_project_identity(pb::GetProjectIdentityRequest { + repository: Some(service.reference.clone()), + })?; let json = matches!( self.command, RepoCommand::Identity { @@ -51,17 +43,22 @@ impl Command for Repo { } ); if json { - println!("{}", serde_json::to_string_pretty(&serde_json::json!({ - "project": project.to_base32(), "urn": project.to_urn(), "created": record.created_at, - "minted": minted, "signed_by": signed_by, "hash": record.hash.to_base32(), - })).expect("JSON serialization")); + println!( + "{}", + serde_json::to_string_pretty(&serde_json::json!({ + "project": record.project, "urn": record.urn, "created": record.created, + "minted": record.minted, "signed_by": record.signed_by, "hash": record.hash, + })) + .expect("JSON serialization") + ); } else { println!( - "project: {project}\nurn: {}\ncreated: {}\nminted: {}\nsigned by: {}", - project.to_urn(), - record.created_at, - minted.as_str().unwrap_or_default(), - signed_by.unwrap_or("unsigned") + "project: {}\nurn: {}\ncreated: {}\nminted: {}\nsigned by: {}", + record.project, + record.urn, + record.created, + record.minted, + record.signed_by.as_deref().unwrap_or("unsigned") ); } Ok(()) diff --git a/atomic-cli/src/commands/rpc.rs b/atomic-cli/src/commands/rpc.rs index caee8184..2cc8b243 100644 --- a/atomic-cli/src/commands/rpc.rs +++ b/atomic-cli/src/commands/rpc.rs @@ -1319,6 +1319,7 @@ pub fn query_enrich(args: &super::query::QueryEnrich) -> CliResult { action: pb::KnowledgeMaintainAction::Enrich as i32, scope: args.rebuild.then(|| "rebuild".to_string()), changes, + ..Default::default() }; let response = session.maintain_knowledge_graph(request)?; if !args.changes.is_empty() { @@ -6393,6 +6394,7 @@ pub fn query_embed(args: &super::query::QueryEmbed) -> CliResult { action: pb::KnowledgeMaintainAction::Embed as i32, scope: args.path.clone(), changes: Vec::new(), + ..Default::default() }; let response = session.maintain_knowledge_graph(request)?; println!( @@ -6415,20 +6417,64 @@ pub fn query_embed(args: &super::query::QueryEmbed) -> CliResult { /// `atomic query reindex` over MaintainKnowledgeGraph (REINDEX) — the /// handler runs vault_reindex_kg; the CLI prints the local report. -pub fn query_reindex(full: bool) -> CliResult { - let Some(session) = Service::open()? else { - return Ok(false); +pub fn query_reindex(full: bool) -> CliResult<()> { + let root = super::find_repository_root()?; + let session = Service::open_root(&root)?.ok_or_else(|| CliError::InvalidArgument { + message: "knowledge graph reindex requires a local repository service".into(), + })?; + let remote = match sync_resolve_remote(&session, &None) { + Ok((_, url)) => Some(url), + Err(CliError::RemoteNotFound { .. }) => None, + Err(error) => return Err(error), }; + let signer_auth_headers = remote + .as_deref() + .map(|url| sync_auth_headers(url, None)) + .unwrap_or_default(); let request = pb::MaintainKnowledgeGraphRequest { repository: Some(session.reference.clone()), meta: request_meta(), action: pb::KnowledgeMaintainAction::Reindex as i32, scope: full.then(|| "full".to_owned()), changes: Vec::new(), + signer_auth_headers, + signer_remote_url: remote, }; let response = session.maintain_knowledge_graph(request)?; println!("Indexed {} nodes + edges.", response.processed); - Ok(true) + Ok(()) +} + +/// Fetch the asserted graph through the same service for local and reactor +/// modes. The CLI adapts this immutable snapshot to RDF without opening redb. +pub(crate) fn workflow_snapshot( + root: &Path, +) -> CliResult { + let session = Service::open_root(root)?.ok_or_else(|| CliError::InvalidArgument { + message: "workflow validation requires a local repository service".into(), + })?; + let response = session.get_workflow_snapshot(pb::GetWorkflowSnapshotRequest { + repository: Some(session.reference.clone()), + })?; + if response.api_version != 1 + || response.generation != atomic_core::kg::KG_GENERATION + || response.ontology_version != atomic_core::kg::ontology::VERSION_IRI + { + return Err(CliError::Internal(anyhow::anyhow!( + "unsupported workflow snapshot API/generation/ontology" + ))); + } + Ok(atomic_repository::repository::WorkflowSnapshot { + generation: response.generation, + ontology_version: atomic_core::kg::ontology::VERSION_IRI, + identity: response.identity, + nodes: response.nodes.iter().map(kg_node_domain).collect(), + edges: response + .edges + .iter() + .map(kg_edge_domain) + .collect::>()?, + }) } // --------------------------------------------------------------------------- diff --git a/atomic-cli/src/service/mod.rs b/atomic-cli/src/service/mod.rs index 9be82e6a..77040acc 100644 --- a/atomic-cli/src/service/mod.rs +++ b/atomic-cli/src/service/mod.rs @@ -1064,6 +1064,46 @@ impl Service { // --------------------------------------------------------------------------- impl Service { + pub fn get_project_identity( + &self, + request: pb::GetProjectIdentityRequest, + ) -> CliResult { + self.call(move |backend| async move { + match backend { + Backend::Local(state) => services_agent::KnowledgeImpl { state } + .get_project_identity(Request::new(request)) + .await + .map(|response| response.into_inner()), + Backend::Reactor(channel) => { + pb::knowledge_service_client::KnowledgeServiceClient::new(channel) + .get_project_identity(request) + .await + .map(|response| response.into_inner()) + } + } + }) + } + + pub fn get_workflow_snapshot( + &self, + request: pb::GetWorkflowSnapshotRequest, + ) -> CliResult { + self.call(move |backend| async move { + match backend { + Backend::Local(state) => services_agent::KnowledgeImpl { state } + .get_workflow_snapshot(Request::new(request)) + .await + .map(|response| response.into_inner()), + Backend::Reactor(channel) => { + pb::knowledge_service_client::KnowledgeServiceClient::new(channel) + .get_workflow_snapshot(request) + .await + .map(|response| response.into_inner()) + } + } + }) + } + pub fn query_graph(&self, request: pb::QueryGraphRequest) -> CliResult { let response = self.call(move |backend| async move { match backend { diff --git a/atomic-cli/tests/kg_service_integration_test.rs b/atomic-cli/tests/kg_service_integration_test.rs new file mode 100644 index 00000000..38ee6f79 --- /dev/null +++ b/atomic-cli/tests/kg_service_integration_test.rs @@ -0,0 +1,196 @@ +//! Workflow inspection and maintenance use the shared handlers in both modes. +use std::path::Path; +use std::process::{Command, Output}; + +use atomic_core::kg::project_id::MintReason; +use atomic_repository::Repository; +use serde_json::Value; + +fn command(root: &Path, args: &[&str]) -> Command { + let mut command = Command::new(env!("CARGO_BIN_EXE_atomic")); + command + .args(args) + .current_dir(root) + .env("ATOMIC_SERVICE", "local") + .env_remove("ATOMIC_RPC") + .env("ATOMIC_DB_LOCK_WAIT_MS", "100") + .env("ATOMIC_DAEMON_BIN", root.join("no-daemon-binary")) + .env("ATOMIC_DAEMON_SOCKET", root.join("no-daemon.socket")); + command +} + +fn json(output: Output) -> Value { + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + serde_json::from_slice(&output.stdout).unwrap() +} + +fn fixture(root: &Path) { + let repo = Repository::init(root).unwrap(); + repo.ensure_project_identity(MintReason::Created).unwrap(); + repo.rebuild_kg(true).unwrap(); +} + +#[test] +fn local_workflow_reads_are_readonly_and_maintenance_routes_through_rpc() { + let directory = tempfile::tempdir().unwrap(); + let root = directory.path(); + fixture(root); + let database = root.join(".atomic/atomic.redb"); + let before = std::fs::read(&database).unwrap(); + let log = root.join("requests.jsonl"); + let identity = json( + command(root, &["repo", "identity", "show", "--json"]) + .env("ATOMIC_DAEMON_LOG_REQUESTS", &log) + .output() + .unwrap(), + ); + assert_eq!(identity["minted"], "created"); + let report = json( + command(root, &["query", "validate", "--format", "json"]) + .env("ATOMIC_DAEMON_LOG_REQUESTS", &log) + .output() + .unwrap(), + ); + assert_eq!(report["conforms"], true); + assert_eq!( + std::fs::read(&database).unwrap(), + before, + "inspection must not migrate/write the database" + ); + let output = command(root, &["query", "reindex", "--full"]) + .env("ATOMIC_DAEMON_LOG_REQUESTS", &log) + .output() + .unwrap(); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + let requests = std::fs::read_to_string(log).unwrap(); + for method in [ + "GetProjectIdentity", + "GetWorkflowSnapshot", + "ListRemotes", + "MaintainKnowledgeGraph", + ] { + assert!( + requests.contains(method), + "missing RPC {method}: {requests}" + ); + } +} + +#[test] +fn identity_inspection_does_not_silently_mint_a_missing_identity() { + let directory = tempfile::tempdir().unwrap(); + drop(Repository::init(directory.path()).unwrap()); + let database = directory.path().join(".atomic/atomic.redb"); + let before = std::fs::read(&database).unwrap(); + let output = command(directory.path(), &["repo", "identity", "show", "--json"]) + .output() + .unwrap(); + assert!(!output.status.success()); + assert!(String::from_utf8_lossy(&output.stderr).contains("MissingProjectIdentity")); + assert_eq!(std::fs::read(database).unwrap(), before); +} + +#[cfg(unix)] +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn reactor_workflow_commands_use_served_handlers_without_local_fallback() { + use libatomic::atomic::{ + daemon_service_server::DaemonServiceServer, + knowledge_service_server::KnowledgeServiceServer, sync_service_server::SyncServiceServer, + }; + use libatomic::daemon::{ + services::DaemonImpl, services_agent::KnowledgeImpl, services_sync::SyncImpl, + state::DaemonState, + }; + use std::sync::Arc; + + let directory = tempfile::tempdir().unwrap(); + let root = directory.path().to_path_buf(); + fixture(&root); + let local_identity = json( + command(&root, &["repo", "identity", "show", "--json"]) + .output() + .unwrap(), + ); + let local_report = json( + command(&root, &["query", "validate", "--format", "json"]) + .output() + .unwrap(), + ); + let socket_directory = tempfile::Builder::new() + .prefix("kg-rpc-") + .tempdir_in("/tmp") + .unwrap(); + let socket = socket_directory.path().join("rpc.sock"); + let listener = tokio::net::UnixListener::bind(&socket).unwrap(); + let state = Arc::new(DaemonState::new()); + let (stop, stopped) = tokio::sync::oneshot::channel(); + let server = tokio::spawn( + tonic::transport::Server::builder() + .add_service(DaemonServiceServer::new(DaemonImpl { + state: state.clone(), + })) + .add_service(KnowledgeServiceServer::new(KnowledgeImpl { + state: state.clone(), + })) + .add_service(SyncServiceServer::new(SyncImpl { state })) + .serve_with_incoming_shutdown( + tokio_stream::wrappers::UnixListenerStream::new(listener), + async { + let _ = stopped.await; + }, + ), + ); + for (args, expected) in [ + ( + vec!["repo", "identity", "show", "--json"], + Some(local_identity), + ), + ( + vec!["query", "validate", "--format", "json"], + Some(local_report), + ), + (vec!["query", "reindex", "--full"], None), + ] { + let mut cli = command(&root, &args); + cli.env("ATOMIC_SERVICE", "reactor") + .env("ATOMIC_DAEMON_SOCKET", &socket); + let output = tokio::task::spawn_blocking(move || cli.output().unwrap()) + .await + .unwrap(); + if let Some(expected) = expected { + assert_eq!(json(output), expected); + } else { + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + } + } + stop.send(()).unwrap(); + server.await.unwrap().unwrap(); + for args in [ + vec!["repo", "identity", "show", "--json"], + vec!["query", "validate", "--format", "json"], + vec!["query", "reindex", "--full"], + ] { + let mut cli = command(&root, &args); + cli.env("ATOMIC_SERVICE", "reactor") + .env("ATOMIC_DAEMON_SOCKET", &socket); + let output = tokio::task::spawn_blocking(move || cli.output().unwrap()) + .await + .unwrap(); + assert!( + !output.status.success(), + "unavailable reactor must not fall back to direct database access" + ); + } +} diff --git a/docs/knowledge-graph-phase1-upgrade.md b/docs/knowledge-graph-phase1-upgrade.md index f1e77a68..aab2337e 100644 --- a/docs/knowledge-graph-phase1-upgrade.md +++ b/docs/knowledge-graph-phase1-upgrade.md @@ -33,6 +33,13 @@ atomic query validate --profile publication --format json atomic query validate --shape --node ``` +Identity inspection and validation fetch their data through the shared RPC +handlers in both local and reactor modes, using the central repository connection +opener. They are read-only: missing identity/generation state requires an explicit +`atomic query reindex --full`. Reindex also performs signer discovery handler-side; +the CLI resolves credentials without opening a repository database. PurRDF still +runs client-side on the returned immutable snapshot. + Projection validation accepts a missing workflow shortcut only with its matching per-change finding. Publication validation additionally checks managed plans, provenance, session coverage, and explicit successful DID signature verification. diff --git a/libatomic/proto/atomic/services/knowledge/knowledge_messages.proto b/libatomic/proto/atomic/services/knowledge/knowledge_messages.proto index cadbc3f3..be98ca27 100644 --- a/libatomic/proto/atomic/services/knowledge/knowledge_messages.proto +++ b/libatomic/proto/atomic/services/knowledge/knowledge_messages.proto @@ -1,11 +1,12 @@ // atomic-protocol — KnowledgeService: knowledge-graph queries and // maintenance. // -// DRAFT — add-only. The CLI's eleven `query` verbs collapse into TWO RPCs -// (the headline combine of this draft): +// DRAFT — add-only. Interactive graph queries and maintenance use: // * QueryGraph — every read: content/code search, kg search, entity, // neighbors, callers, graph slice, plan execution, RAG ask; // * MaintainKnowledgeGraph — embed/enrich/reindex (the write/rebuild side). +// GetProjectIdentity and GetWorkflowSnapshot provide read-only, repository-wide +// workflow inspection for `repo identity show` and client-side SHACL validation. // Local file-index construction (`query index`) remains client-side. QueryGraph // reads a view-scoped indexed snapshot; it must never build/refresh an index. @@ -15,6 +16,29 @@ package atomic; import "atomic/common/common.proto"; +message GetProjectIdentityRequest { + RepositoryRef repository = 1; +} +message GetProjectIdentityResponse { + string project = 1; + string urn = 2; + string created = 3; + string minted = 4; + optional string signed_by = 5; + string hash = 6; +} +message GetWorkflowSnapshotRequest { + RepositoryRef repository = 1; +} +// One consistent, repository-wide asserted snapshot. SHACL runs client-side. +message GetWorkflowSnapshotResponse { + uint32 api_version = 1; + uint32 generation = 2; + string ontology_version = 3; + string identity = 4; + repeated KGNode nodes = 5; + repeated KGEdge edges = 6; +} message QueryGraphRequest { @@ -154,6 +178,11 @@ message MaintainKnowledgeGraphRequest { KnowledgeMaintainAction action = 3; optional string scope = 4; // e.g. entity-type filter repeated Hash changes = 5; // ENRICH: limit to these changes' nodes + // Client-resolved credentials for optional signer discovery on reindex. + // The handler verifies that the configured remote still matches this URL + // before using these headers. No client-side repository open is required. + map signer_auth_headers = 6; + optional string signer_remote_url = 7; } message MaintainKnowledgeGraphResponse { uint64 processed = 1; diff --git a/libatomic/proto/atomic/services/knowledge/knowledge_service.proto b/libatomic/proto/atomic/services/knowledge/knowledge_service.proto index f86ccb07..0f0a7d10 100644 --- a/libatomic/proto/atomic/services/knowledge/knowledge_service.proto +++ b/libatomic/proto/atomic/services/knowledge/knowledge_service.proto @@ -10,6 +10,19 @@ import "atomic/common/options.proto"; import "atomic/services/knowledge/knowledge_messages.proto"; service KnowledgeService { + // Repository-wide workflow inspection; not a view-scoped sandbox query. + rpc GetProjectIdentity(GetProjectIdentityRequest) returns (GetProjectIdentityResponse) { + option (atomic.execution_scope) = EXECUTION_SCOPE_LOCAL; + option (atomic.required_capability) = "repository.read"; + option (atomic.allowed_caller) = CALLER_CLASS_LOCAL; + option (atomic.effect) = RPC_EFFECT_REPOSITORY_READ; + } + rpc GetWorkflowSnapshot(GetWorkflowSnapshotRequest) returns (GetWorkflowSnapshotResponse) { + option (atomic.execution_scope) = EXECUTION_SCOPE_LOCAL; + option (atomic.required_capability) = "knowledge.read"; + option (atomic.allowed_caller) = CALLER_CLASS_LOCAL; + option (atomic.effect) = RPC_EFFECT_REPOSITORY_READ; + } rpc QueryGraph(QueryGraphRequest) returns (QueryGraphResponse) { option (atomic.execution_scope) = EXECUTION_SCOPE_BOTH; option (atomic.required_capability) = "knowledge.read"; diff --git a/libatomic/src/daemon/services.rs b/libatomic/src/daemon/services.rs index 843f99a7..3bd547bf 100644 --- a/libatomic/src/daemon/services.rs +++ b/libatomic/src/daemon/services.rs @@ -69,6 +69,8 @@ const METHOD_MATRIX: &[(&str, &str)] = &[ ("AttestationService", "RecordAttestation"), ("AttestationService", "VerifyAttestation"), ("KnowledgeService", "QueryGraph"), + ("KnowledgeService", "GetProjectIdentity"), + ("KnowledgeService", "GetWorkflowSnapshot"), ("ProvenanceService", "DispatchTurnEvent"), ("ProvenanceService", "AppendHookLog"), ("ProvenanceService", "LoadHookLog"), @@ -125,6 +127,22 @@ impl daemon_service_server::DaemonService for DaemonImpl { allowed_callers: vec![CallerClass::Local as i32], effects: vec![RpcEffect::RepositoryWrite as i32], } + } else if *service == "KnowledgeService" + && matches!(*method, "GetProjectIdentity" | "GetWorkflowSnapshot") + { + MethodDescriptorInfo { + service: service.to_string(), + method: method.to_string(), + scope: ExecutionScope::Local as i32, + required_capabilities: vec![if *method == "GetProjectIdentity" { + "repository.read" + } else { + "knowledge.read" + } + .to_owned()], + allowed_callers: vec![CallerClass::Local as i32], + effects: vec![RpcEffect::RepositoryRead as i32], + } } else if *service == "ProvenanceService" && matches!(*method, "AppendHookLog" | "LoadHookLog") { diff --git a/libatomic/src/daemon/services_agent.rs b/libatomic/src/daemon/services_agent.rs index 290c57bd..9e5bb727 100644 --- a/libatomic/src/daemon/services_agent.rs +++ b/libatomic/src/daemon/services_agent.rs @@ -3345,6 +3345,92 @@ pub struct KnowledgeImpl { #[tonic::async_trait] impl knowledge_service_server::KnowledgeService for KnowledgeImpl { + async fn get_project_identity( + &self, + request: Request, + ) -> Result, Status> { + let request = request.into_inner(); + let reference = request + .repository + .as_ref() + .ok_or_else(|| Status::invalid_argument("repository required"))?; + let handle = self.state.resolve(reference)?; + self.state.log_rpc("GetProjectIdentity", Some(&handle)); + let gate = handle.exclusive_owned().await; + let response = tokio::task::spawn_blocking(move || { + let _gate = gate; + let repo = handle.repository_readonly()?; + let record = repo + .project_identity() + .map_err(repository_error)? + .ok_or_else(|| { + Status::failed_precondition( + "MissingProjectIdentity; run atomic query reindex --full", + ) + })?; + let project = record + .project_id() + .map_err(|e| Status::internal(e.to_string()))?; + let object: serde_json::Value = serde_json::from_slice(&record.object) + .map_err(|e| Status::internal(e.to_string()))?; + let minted = serde_json::to_value(record.mint_reason) + .map_err(|e| Status::internal(e.to_string()))?; + Ok::<_, Status>(GetProjectIdentityResponse { + project: project.to_base32(), + urn: project.to_urn(), + created: record.created_at, + minted: minted.as_str().unwrap_or_default().to_owned(), + signed_by: object + .get("atomic:issuer") + .and_then(|v| v.get("@id")) + .and_then(|v| v.as_str()) + .map(str::to_owned), + hash: record.hash.to_base32(), + }) + }) + .await + .map_err(|e| Status::internal(e.to_string()))??; + Ok(Response::new(response)) + } + + async fn get_workflow_snapshot( + &self, + request: Request, + ) -> Result, Status> { + let request = request.into_inner(); + let reference = request + .repository + .as_ref() + .ok_or_else(|| Status::invalid_argument("repository required"))?; + let handle = self.state.resolve(reference)?; + self.state.log_rpc("GetWorkflowSnapshot", Some(&handle)); + let gate = handle.exclusive_owned().await; + let response = tokio::task::spawn_blocking(move || { + let _gate = gate; + let repo = handle.repository_readonly()?; + let snapshot = repo.workflow_snapshot().map_err(repository_error)?; + Ok::<_, Status>(GetWorkflowSnapshotResponse { + api_version: 1, + generation: snapshot.generation, + ontology_version: snapshot.ontology_version.to_owned(), + identity: snapshot.identity, + nodes: snapshot + .nodes + .into_iter() + .map(super::services_query::kg_node_proto) + .collect(), + edges: snapshot + .edges + .into_iter() + .map(super::services_query::kg_edge_proto) + .collect(), + }) + }) + .await + .map_err(|e| Status::internal(e.to_string()))??; + Ok(Response::new(response)) + } + async fn query_graph( &self, request: Request, @@ -3740,44 +3826,33 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl { &self, request: Request, ) -> Result, Status> { - let authorization = request - .metadata() - .get("authorization") - .and_then(|value| value.to_str().ok()) - .map(str::to_owned); - let delegation = request - .metadata() - .get("atomic-delegation") - .and_then(|value| value.to_str().ok()) - .map(str::to_owned); let request = request.into_inner(); let action = KnowledgeMaintainAction::try_from(request.action) .unwrap_or(KnowledgeMaintainAction::Unspecified); let handle = self.state.resolve(request.repository.as_ref().unwrap())?; self.state.log_rpc("MaintainKnowledgeGraph", Some(&handle)); - let gate_handle = handle.clone(); - let _gate = gate_handle.exclusive().await; - if action == KnowledgeMaintainAction::Reindex { - if let Some(authorization) = authorization { - let repo = handle.repository()?; - if let Ok((_, remote)) = repo.get_default_remote() { - let mut config = atomic_remote::HttpRemoteConfig::new() - .with_header("Authorization", authorization); - if let Some(delegation) = delegation { - config = config.with_header("Atomic-Delegation", delegation); - } - if let Some(client) = - crate::signer_refresh::client_for_remote(&remote.url, &config) - { - crate::signer_refresh::refresh(&repo, &client) - .await - .map_err(repository_error)?; - } - } + let gate = handle.exclusive_owned().await; + if action == KnowledgeMaintainAction::Reindex && !request.signer_auth_headers.is_empty() { + let repo = handle.repository()?; + let (_, remote) = repo.get_default_remote().map_err(repository_error)?; + if request.signer_remote_url.as_deref() != Some(remote.url.as_str()) { + return Err(Status::failed_precondition( + "signer discovery remote changed; retry reindex with fresh credentials", + )); + } + let mut config = atomic_remote::HttpRemoteConfig::new(); + for (name, value) in &request.signer_auth_headers { + config = config.with_header(name, value); + } + if let Some(client) = crate::signer_refresh::client_for_remote(&remote.url, &config) { + crate::signer_refresh::refresh(&repo, &client) + .await + .map_err(repository_error)?; } } let meta = request.meta.clone(); let response = tokio::task::spawn_blocking(move || { + let _gate = gate; let (repo, workspace) = handle.workspace_repository(atomic_repository::WorkspaceTxnMode::Reconcile)?; let working_copy = workspace.working_copy(); diff --git a/libatomic/tests/signer_refresh.rs b/libatomic/tests/signer_refresh.rs index a9a89fd5..64711363 100644 --- a/libatomic/tests/signer_refresh.rs +++ b/libatomic/tests/signer_refresh.rs @@ -10,7 +10,10 @@ use std::{ net::TcpListener, }; -fn serve_key(did: String, public_key: String) -> (StorageClient, std::thread::JoinHandle<()>) { +fn serve_key( + did: String, + public_key: String, +) -> (String, StorageClient, std::thread::JoinHandle<()>) { let listener = TcpListener::bind("127.0.0.1:0").unwrap(); let address = listener.local_addr().unwrap(); let server = std::thread::spawn(move || { @@ -37,8 +40,9 @@ fn serve_key(did: String, public_key: String) -> (StorageClient, std::thread::Jo .to_string(); write!(stream, "HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{}", body.len(), body).unwrap(); }); - let client = StorageClient::new(&format!("http://{address}"), "", "fixture-token").unwrap(); - (client, server) + let url = format!("http://{address}"); + let client = StorageClient::new(&url, "", "fixture-token").unwrap(); + (url, client, server) } #[tokio::test] @@ -67,7 +71,7 @@ async fn fetched_keys_are_did_checked_before_cache_and_offline_verification() { }; assert!(status().ends_with("#Unverifiable")); - let (client, server) = serve_key(did.clone(), KeyPair::generate().public.to_base32()); + let (_, client, server) = serve_key(did.clone(), KeyPair::generate().public.to_base32()); assert_eq!( libatomic::signer_refresh::refresh(&repo, &client) .await @@ -84,7 +88,7 @@ async fn fetched_keys_are_did_checked_before_cache_and_offline_verification() { .is_none()); assert!(status().ends_with("#Unverifiable")); - let (client, server) = serve_key(did.clone(), key.public.to_base32()); + let (_, client, server) = serve_key(did.clone(), key.public.to_base32()); assert_eq!( libatomic::signer_refresh::refresh(&repo, &client) .await @@ -110,3 +114,67 @@ fn anonymous_remotes_do_not_start_signer_discovery() { ) .is_none()); } + +#[tokio::test] +async fn reindex_rpc_refreshes_signers_with_url_bound_client_credentials() { + use libatomic::atomic::{ + knowledge_service_server::KnowledgeService, GetWorkflowSnapshotRequest, + KnowledgeMaintainAction, MaintainKnowledgeGraphRequest, + }; + use libatomic::daemon::{services_agent::KnowledgeImpl, state::DaemonState}; + use std::sync::Arc; + use tonic::Request; + + let directory = tempfile::tempdir().unwrap(); + let repo = Repository::init(directory.path()).unwrap(); + let key = KeyPair::generate(); + let did = atomic_canonical::did::did_for_public_key(&key.public); + let mut attestation = + Attestation::builder("rpc-session", AttestAgent::new("agent", "Agent", "1")) + .add_change(Hash::of(b"rpc covered change")) + .build(); + attestation.sign_with_did(&did, key.secret.as_bytes()); + let hash = repo.save_attestation(&attestation).unwrap(); + let (url, _, server) = serve_key(did, key.public.to_base32()); + repo.add_remote_default("origin", &url).unwrap(); + drop(repo); + + let state = Arc::new(DaemonState::new()); + let reference = state + .register(directory.path().to_path_buf()) + .repository_ref(); + let service = KnowledgeImpl { state }; + let request = MaintainKnowledgeGraphRequest { + repository: Some(reference.clone()), + action: KnowledgeMaintainAction::Reindex as i32, + scope: Some("full".into()), + signer_remote_url: Some("http://wrong.example".into()), + signer_auth_headers: [("Authorization".into(), "Bearer fixture-token".into())].into(), + ..Default::default() + }; + let error = service + .maintain_knowledge_graph(Request::new(request.clone())) + .await + .unwrap_err(); + assert_eq!(error.code(), tonic::Code::FailedPrecondition); + service + .maintain_knowledge_graph(Request::new(MaintainKnowledgeGraphRequest { + signer_remote_url: Some(url), + ..request + })) + .await + .unwrap(); + server.join().unwrap(); + let snapshot = service + .get_workflow_snapshot(Request::new(GetWorkflowSnapshotRequest { + repository: Some(reference), + })) + .await + .unwrap() + .into_inner(); + assert!(snapshot.edges.iter().any(|edge| { + edge.source == atomic_core::kg::iri::mint_attestation(&hash) + && edge.relation == atomic_core::kg::ontology::Predicate::VerificationResult.iri() + && edge.target.ends_with("#Verified") + })); +} diff --git a/tests/harness/46_knowledge_graph_closure.sh b/tests/harness/46_knowledge_graph_closure.sh index d68f531a..674b7315 100644 --- a/tests/harness/46_knowledge_graph_closure.sh +++ b/tests/harness/46_knowledge_graph_closure.sh @@ -10,3 +10,4 @@ cargo test --manifest-path "$HARNESS_DIR/../../Cargo.toml" -p atomic-core --test cargo test --manifest-path "$HARNESS_DIR/../../Cargo.toml" -p atomic-repository --test project_identity --test kg_projection --quiet cargo test --manifest-path "$HARNESS_DIR/../../Cargo.toml" -p atomic-shacl --quiet cargo test --manifest-path "$HARNESS_DIR/../../Cargo.toml" -p libatomic --test signer_refresh --quiet +cargo test --manifest-path "$HARNESS_DIR/../../Cargo.toml" -p atomic-cli --test kg_service_integration_test --quiet From 5907609619c400da3e3bbeaa5bf5c787e29d1d32 Mon Sep 17 00:00:00 2001 From: Bradley Hilton Date: Fri, 16 Oct 2026 17:01:13 -0500 Subject: [PATCH 13/19] fix: preserve refused doctor repairs and update KG routing tests --- atomic-cli/src/commands/doctor.rs | 40 +---------- atomic-cli/src/commands/rpc.rs | 33 +++++++++ .../tests/kg_service_integration_test.rs | 13 +++- .../native_index_doctor_integration_test.rs | 68 ++++++++++++++++++- atomic-cli/tests/routing_parity_test.rs | 32 ++++++--- .../maintenance/maintenance_messages.proto | 6 ++ libatomic/src/daemon/services_maintenance.rs | 49 +++++++++++-- libatomic/src/daemon/state.rs | 17 +++++ 8 files changed, 198 insertions(+), 60 deletions(-) diff --git a/atomic-cli/src/commands/doctor.rs b/atomic-cli/src/commands/doctor.rs index 23220be4..853b1d4a 100644 --- a/atomic-cli/src/commands/doctor.rs +++ b/atomic-cli/src/commands/doctor.rs @@ -436,45 +436,7 @@ fn print_diagnosis(diagnosis: &atomic_repository::WorkingCopyRegistrationDiagnos impl Command for RepairNativeIndexes { fn run(&self) -> CliResult<()> { - let repo = match require_repository_for_native_repair(None) { - Ok(repo) => repo, - Err(error) => { - print_warning(&format!( - "Native derived indexes are unrepairable because repository authority cannot be opened: {}", - error - )); - return Err(error); - } - }; - print_info("Rebuilding native derived indexes from graph authority..."); - let outcome = match repo.repair_native_derived_indexes() { - Ok(outcome) => outcome, - Err(error) => { - print_warning(&format!( - "Native derived indexes are unrepairable from available authority: {}", - error - )); - print_hint("No repair transaction was committed. Restore missing/corrupt change or graph authority and retry."); - return Err(crate::error::CliError::Repository(error)); - } - }; - if outcome.already_healthy { - print_success(&format!( - "Native derived indexes are already consistent ({} rows).", - outcome.rows_written - )); - } else { - print_success(&format!( - "Repaired {} native derived-index problem(s); wrote {} rows atomically{}.", - outcome.problems_repaired, - outcome.rows_written, - outcome - .operation - .map(|operation| format!("; journaled as operation {operation}")) - .unwrap_or_default() - )); - } - Ok(()) + crate::commands::rpc::doctor_repair_native_indexes() } } diff --git a/atomic-cli/src/commands/rpc.rs b/atomic-cli/src/commands/rpc.rs index 2cc8b243..4ec10af0 100644 --- a/atomic-cli/src/commands/rpc.rs +++ b/atomic-cli/src/commands/rpc.rs @@ -2700,6 +2700,39 @@ pub fn query_neighbors(args: &super::query::QueryNeighbors) -> CliResult { // doctor (MaintenanceService) // --------------------------------------------------------------------------- +/// Native repair refuses unrepairable authority in a read-only handler preflight. +pub fn doctor_repair_native_indexes() -> CliResult<()> { + let root = super::find_repository_root()?; + let session = Service::open_root(&root)?.ok_or_else(|| CliError::InvalidArgument { + message: "native-index repair requires a local repository service".into(), + })?; + print_info("Rebuilding native derived indexes from graph authority..."); + let outcome = session.repair(pb::RepairRequest { + repository: Some(session.reference.clone()), + meta: request_meta(), + action: pb::RepairAction::RebuildNativeIndexes as i32, + force: false, + view: String::new(), + })?; + if outcome.native_already_healthy { + print_success(&format!( + "Native derived indexes are already consistent ({} rows).", + outcome.native_rows_written + )); + } else { + print_success(&format!( + "Repaired {} native derived-index problem(s); wrote {} rows atomically{}.", + outcome.native_problems_repaired, + outcome.native_rows_written, + outcome + .native_operation + .map(|operation| format!("; journaled as operation {operation}")) + .unwrap_or_default() + )); + } + Ok(()) +} + /// `atomic doctor check` over CheckRepository — the read-only consistency /// check. Rendered to mirror the in-process path exactly (non-zero exit /// when problems are found). diff --git a/atomic-cli/tests/kg_service_integration_test.rs b/atomic-cli/tests/kg_service_integration_test.rs index 38ee6f79..4792187e 100644 --- a/atomic-cli/tests/kg_service_integration_test.rs +++ b/atomic-cli/tests/kg_service_integration_test.rs @@ -103,11 +103,13 @@ fn identity_inspection_does_not_silently_mint_a_missing_identity() { async fn reactor_workflow_commands_use_served_handlers_without_local_fallback() { use libatomic::atomic::{ daemon_service_server::DaemonServiceServer, - knowledge_service_server::KnowledgeServiceServer, sync_service_server::SyncServiceServer, + knowledge_service_server::KnowledgeServiceServer, + maintenance_service_server::MaintenanceServiceServer, + sync_service_server::SyncServiceServer, }; use libatomic::daemon::{ - services::DaemonImpl, services_agent::KnowledgeImpl, services_sync::SyncImpl, - state::DaemonState, + services::DaemonImpl, services_agent::KnowledgeImpl, services_maintenance::MaintenanceImpl, + services_sync::SyncImpl, state::DaemonState, }; use std::sync::Arc; @@ -140,6 +142,9 @@ async fn reactor_workflow_commands_use_served_handlers_without_local_fallback() .add_service(KnowledgeServiceServer::new(KnowledgeImpl { state: state.clone(), })) + .add_service(MaintenanceServiceServer::new(MaintenanceImpl { + state: state.clone(), + })) .add_service(SyncServiceServer::new(SyncImpl { state })) .serve_with_incoming_shutdown( tokio_stream::wrappers::UnixListenerStream::new(listener), @@ -158,6 +163,7 @@ async fn reactor_workflow_commands_use_served_handlers_without_local_fallback() Some(local_report), ), (vec!["query", "reindex", "--full"], None), + (vec!["doctor", "repair-native-indexes"], None), ] { let mut cli = command(&root, &args); cli.env("ATOMIC_SERVICE", "reactor") @@ -181,6 +187,7 @@ async fn reactor_workflow_commands_use_served_handlers_without_local_fallback() vec!["repo", "identity", "show", "--json"], vec!["query", "validate", "--format", "json"], vec!["query", "reindex", "--full"], + vec!["doctor", "repair-native-indexes"], ] { let mut cli = command(&root, &args); cli.env("ATOMIC_SERVICE", "reactor") diff --git a/atomic-cli/tests/native_index_doctor_integration_test.rs b/atomic-cli/tests/native_index_doctor_integration_test.rs index 2b7c6d46..2ce7e1e0 100644 --- a/atomic-cli/tests/native_index_doctor_integration_test.rs +++ b/atomic-cli/tests/native_index_doctor_integration_test.rs @@ -123,7 +123,7 @@ fn doctor_check_does_not_recreate_a_missing_change_store() { assert_eq!( std::fs::read(&pristine).unwrap().len(), pristine_before.len(), - "doctor wrote to atomic.redb (length changed); byte-for-byte equality is + "{args:?} wrote to atomic.redb (length changed); byte-for-byte equality is intentionally not required because redb 4.2 rewrites its graceful-shutdown marker on every clean close" ); @@ -189,7 +189,7 @@ fn doctor_rejects_corrupt_current_view_without_repairing_dev() { assert_eq!( std::fs::read(&pristine).unwrap().len(), pristine_before.len(), - "doctor wrote to atomic.redb (length changed); byte-for-byte equality is + "{args:?} wrote to atomic.redb (length changed); byte-for-byte equality is intentionally not required because redb 4.2 rewrites its graceful-shutdown marker on every clean close" ); @@ -252,7 +252,7 @@ fn doctor_reports_missing_change_authority_as_unrepairable_without_writes() { assert_eq!( std::fs::read(&pristine_path).unwrap().len(), pristine_before.len(), - "doctor wrote to atomic.redb (length changed); byte-for-byte equality is + "{args:?} wrote to atomic.redb (length changed); byte-for-byte equality is intentionally not required because redb 4.2 rewrites its graceful-shutdown marker on every clean close" ); @@ -264,3 +264,65 @@ fn doctor_reports_missing_change_authority_as_unrepairable_without_writes() { assert!(preserved_change.is_file()); } } + +#[test] +fn doctor_repairs_missing_native_rows_through_rpc_and_repeated_repair_is_readonly() { + let temp = tempfile::tempdir().unwrap(); + let repo = Repository::init(temp.path()).unwrap(); + let working_copy = repo.require_working_copy_id().unwrap(); + std::fs::write(temp.path().join("f.txt"), b"preserved content\n").unwrap(); + repo.add(working_copy, "f.txt", TrackingOptions::default()) + .unwrap(); + repo.record_all(working_copy, "repair fixture").unwrap(); + let inode = repo.get_file_inode("f.txt").unwrap().unwrap(); + let database = repo.pristine().shared_database().unwrap(); + let transaction = database.begin_write().unwrap(); + transaction + .open_table(atomic_core::pristine::REV_TREE) + .unwrap() + .remove(inode.get()) + .unwrap(); + transaction.commit().unwrap(); + drop(database); + assert!(!repo.verify_native_derived_indexes().unwrap().is_healthy()); + drop(repo); + + let log = temp.path().join("requests.jsonl"); + let repair = || { + Command::new(env!("CARGO_BIN_EXE_atomic")) + .current_dir(temp.path()) + .env("ATOMIC_SERVICE", "local") + .env("ATOMIC_DAEMON_LOG_REQUESTS", &log) + .args(["doctor", "repair-native-indexes"]) + .output() + .unwrap() + }; + let output = repair(); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + assert!(String::from_utf8_lossy(&output.stdout).contains("journaled as operation")); + assert!(std::fs::read_to_string(&log).unwrap().contains("Repair")); + assert!(Repository::open_readonly_for_native_repair(temp.path()) + .unwrap() + .verify_native_derived_indexes() + .unwrap() + .is_healthy()); + assert_eq!( + std::fs::read(temp.path().join("f.txt")).unwrap(), + b"preserved content\n" + ); + + let path = temp.path().join(".atomic/atomic.redb"); + let before = blake3::hash(&std::fs::read(&path).unwrap()); + let output = repair(); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + assert!(String::from_utf8_lossy(&output.stdout).contains("already consistent")); + assert_eq!(blake3::hash(&std::fs::read(path).unwrap()), before); +} diff --git a/atomic-cli/tests/routing_parity_test.rs b/atomic-cli/tests/routing_parity_test.rs index 63962d4d..7ac2d3e4 100644 --- a/atomic-cli/tests/routing_parity_test.rs +++ b/atomic-cli/tests/routing_parity_test.rs @@ -11,6 +11,7 @@ use std::path::Path; use std::process::{Command, Output}; +use atomic_core::kg::{iri, ontology::Class, project_id::ProjectId}; use tempfile::TempDir; const ATOMIC_BIN: &str = env!("CARGO_BIN_EXE_atomic"); @@ -49,6 +50,13 @@ fn fixture(label: &str) -> (TempDir, TempDir) { (repo_tmp, home_tmp) } +fn file_iri(repo_dir: &Path, home_dir: &Path, path: &str) -> String { + let identity = run(repo_dir, home_dir, &["repo", "identity", "show", "--json"]); + let identity: serde_json::Value = serde_json::from_str(&identity).unwrap(); + let project = ProjectId::from_base32(identity["project"].as_str().unwrap()).unwrap(); + iri::mint_file(&project, path) +} + fn ensure_default_identity(repo_dir: &Path, home_dir: &Path) { run( repo_dir, @@ -334,22 +342,25 @@ fn query_neighbors_json_serializes_the_full_domain_subgraph() { std::fs::write(repo_dir.join("kg.txt"), "kg fixture\n").unwrap(); run(repo_dir, home_dir, &["add", "kg.txt"]); run(repo_dir, home_dir, &["record", "-m", "kg fixture change"]); + let file_id = file_iri(repo_dir, home_dir, "kg.txt"); let json = run( repo_dir, home_dir, - &["query", "neighbors", "file:kg.txt", "--json"], + &["query", "neighbors", &file_id, "--json"], ); let subgraph: serde_json::Value = serde_json::from_str(&json).unwrap(); let nodes = subgraph["nodes"].as_array().unwrap(); let file_node = nodes .iter() - .find(|node| node["id"] == "file:kg.txt") + .find(|node| node["id"] == file_id) .expect("the file node"); - // The full domain fields the local JSON serializes: kind, label, + // The full domain fields the local JSON serializes: types, label, // summary, source, metadata. - assert_eq!(file_node["kind"], "file"); + assert_eq!(file_node["types"], serde_json::json!([Class::File.iri()])); + assert!(file_node.get("kind").is_none()); assert_eq!(file_node["label"], "kg.txt"); + assert_eq!(file_node["metadata"]["path"], "kg.txt"); assert!(file_node["source"].is_string()); assert!(subgraph["edges"].is_array()); @@ -357,17 +368,17 @@ fn query_neighbors_json_serializes_the_full_domain_subgraph() { let json = run(repo_dir, home_dir, &["query", "search", "kg.txt", "--json"]); let nodes: Vec = serde_json::from_str(&json).unwrap(); assert!( - nodes.iter().any(|node| node["id"] == "file:kg.txt"), + nodes.iter().any(|node| node["id"] == file_id), "the search surfaces the file node" ); - // The --kind filter narrows to the kind's nodes. + // The --kind filter selects an asserted canonical class. let table = run( repo_dir, home_dir, - &["query", "search", "kg.txt", "--kind", "file"], + &["query", "search", "kg.txt", "--kind", Class::File.iri()], ); - assert!(table.contains("file:kg.txt")); + assert!(table.contains(&file_id)); } #[test] @@ -1350,13 +1361,12 @@ fn kg_extras_reindex_and_graph_route() { assert!(reindexed.contains("Indexed "), "{reindexed}"); assert!(reindexed.contains(" nodes + edges."), "{reindexed}"); let graph = run(repo_dir, home_dir, &["query", "graph", "greet", "--json"]); + let file_id = file_iri(repo_dir, home_dir, "greet.rs"); let value: serde_json::Value = serde_json::from_str(&graph).unwrap(); let nodes = value["nodes"].as_array().unwrap(); assert!(!nodes.is_empty(), "the seeded graph carries nodes"); assert!( - nodes - .iter() - .any(|node| node["id"] == serde_json::json!("file:greet.rs")), + nodes.iter().any(|node| node["id"] == file_id), "the seed file is present: {graph}" ); } diff --git a/libatomic/proto/atomic/services/maintenance/maintenance_messages.proto b/libatomic/proto/atomic/services/maintenance/maintenance_messages.proto index b67c9f4c..a6fcdd1d 100644 --- a/libatomic/proto/atomic/services/maintenance/maintenance_messages.proto +++ b/libatomic/proto/atomic/services/maintenance/maintenance_messages.proto @@ -36,6 +36,7 @@ enum RepairAction { reserved 3; // was CONSISTENCY_CHECK — // split to CheckRepository REPAIR_ACTION_REINDEX_WORKING_COPY = 4; // `status --reindex` + REPAIR_ACTION_REBUILD_NATIVE_INDEXES = 5; // `doctor repair-native-indexes` } message RepairRequest { RepositoryRef repository = 1; @@ -78,6 +79,11 @@ message RepairResponse { repeated string skip_samples = 16; // REINDEX_WORKING_COPY render data. uint64 reindexed = 17; + // REBUILD_NATIVE_INDEXES: authority is checked read-only before a writer opens. + bool native_already_healthy = 18; + uint64 native_rows_written = 19; + uint64 native_problems_repaired = 20; + optional string native_operation = 21; ResponseMeta meta = 100; } message CheckRepositoryRequest { diff --git a/libatomic/src/daemon/services_maintenance.rs b/libatomic/src/daemon/services_maintenance.rs index 61c2f694..ac43f57b 100644 --- a/libatomic/src/daemon/services_maintenance.rs +++ b/libatomic/src/daemon/services_maintenance.rs @@ -31,11 +31,12 @@ fn repair_action(action: i32) -> Result { x if x == Action::RebuildDependencyIndex as i32 => Action::RebuildDependencyIndex, x if x == Action::MaterializeCrdt as i32 => Action::MaterializeCrdt, x if x == Action::ReindexWorkingCopy as i32 => Action::ReindexWorkingCopy, + x if x == Action::RebuildNativeIndexes as i32 => Action::RebuildNativeIndexes, _ => { return Err(domain_status( ErrorCode::InvalidArgument, "repair action is required (REBUILD_DEPENDENCY_INDEX, MATERIALIZE_CRDT, \ - or REINDEX_WORKING_COPY)", + REINDEX_WORKING_COPY, or REBUILD_NATIVE_INDEXES)", )) } }) @@ -68,9 +69,46 @@ impl MaintenanceService for MaintenanceImpl { let action = repair_action(request.action)?; let force = request.force; let view = request.view; - let gate_handle = handle.clone(); - let _gate = gate_handle.exclusive().await; + let gate = handle.exclusive_owned().await; let result = tokio::task::spawn_blocking(move || -> Result { + let _gate = gate; + if action == RepairAction::RebuildNativeIndexes { + // Even opening a writable redb handle can truncate unused tail + // pages on close. Refuse missing/corrupt authority before that + // open, rather than treating "no committed repair" as read-only. + let report = handle + .native_index_repository(true)? + .verify_native_derived_indexes() + .map_err(repository_error)?; + if let Some(problem) = report.problems.iter().find(|problem| { + problem.kind == atomic_repository::NativeIndexProblemKind::Unrepairable + }) { + return Err(Status::failed_precondition(format!( + "Native derived indexes are unrepairable: {problem}" + ))); + } + if report.is_healthy() { + return Ok(RepairResponse { + native_already_healthy: true, + native_rows_written: report.expected_rows as u64, + ..Default::default() + }); + } + let repo = handle.native_index_repository(false)?; + let outcome = repo.repair_native_derived_indexes().map_err(|error| { + Status::failed_precondition(format!( + "Native derived indexes are unrepairable: {error}" + )) + })?; + return Ok(RepairResponse { + repaired: outcome.problems_repaired as u64, + native_already_healthy: outcome.already_healthy, + native_rows_written: outcome.rows_written as u64, + native_problems_repaired: outcome.problems_repaired as u64, + native_operation: outcome.operation.map(|operation| operation.to_string()), + ..Default::default() + }); + } let mut repo = handle.repository()?; match action { RepairAction::RebuildDependencyIndex => { @@ -137,9 +175,12 @@ impl MaintenanceService for MaintenanceImpl { }), skip_samples: outcome.skip_samples.clone(), meta: None, + ..Default::default() }) } - RepairAction::Unspecified => unreachable!("validated above"), + RepairAction::RebuildNativeIndexes | RepairAction::Unspecified => { + unreachable!("handled or validated above") + } } }) .await diff --git a/libatomic/src/daemon/state.rs b/libatomic/src/daemon/state.rs index 8d78df14..340df1ac 100644 --- a/libatomic/src/daemon/state.rs +++ b/libatomic/src/daemon/state.rs @@ -209,6 +209,23 @@ impl RepoHandle { }) } + /// Native-index diagnostics must tolerate incomplete derived schemas and + /// must not trigger automatic migration or recovery. Repair callers first + /// inspect authority read-only, then open writable only if it is repairable. + pub(super) fn native_index_repository(&self, readonly: bool) -> Result { + let result = if readonly { + Repository::open_readonly_for_native_repair(&self.root) + } else { + Repository::open_for_native_repair(&self.root) + }; + result.map_err(|error| { + domain_status( + ErrorCode::Repository, + format!("Native derived indexes are unrepairable: {error}"), + ) + }) + } + /// The redb-native change/provenance store (atomic.redb — the merged /// database) for this request — opened by path WITHOUT opening the /// repository handle. redb refuses a second concurrent handle on the From 98fd55c85df1650f26319edee2436900c09a37ad Mon Sep 17 00:00:00 2001 From: Bradley Hilton Date: Thu, 22 Oct 2026 21:16:54 -0500 Subject: [PATCH 14/19] fix: repair knowledge graph projection and consumers Preserve authoritative vault child evidence, repair late-arrival references and dependency facts, and resolve canonical resource IRIs exactly. Analyze change impact from recorded syntax in the requested Atomic view, including TypeScript imports and public-visibility changes. Update typed fixtures, closure CI coverage, and deterministic transport/watcher tests. Validated on macOS with the full workspace CI test command, all-target Check and Clippy, rustdoc, formatting, packaging tests, CLI harnesses 09/14/46, and scoped Rust-version checks. GitHub publication snapshot for ATOM::bradley::1; independent Atomic re-review and hook-recorded candidate coverage remain separate. --- .github/workflows/ci.yml | 5 + atomic-cli/src/commands/query/mod.rs | 8 +- atomic-cli/tests/git_shim_cb13d_test.rs | 8 + atomic-cli/tests/provenance_rpc_sink.rs | 126 +++++++--- atomic-core/src/pristine/vault.rs | 8 +- atomic-core/tests/kg_vocabulary_lint.rs | 1 + atomic-repository/src/repository/changes.rs | 7 + atomic-repository/src/repository/content.rs | 71 +++--- .../src/repository/syntax_index.rs | 59 ++++- .../src/repository/vault_kg_enrich.rs | 42 ++++ .../src/repository/vault_projection.rs | 77 +++--- .../src/repository/vault_triples.rs | 17 +- atomic-repository/tests/kg_projection.rs | 226 +++++++++++++++++- atomic-semantic/src/parser.rs | 45 ++++ .../replication_acceptance/fixture.rs | 2 +- libatomic/proto/CONTRACT.md | 10 +- .../repository_query_messages.proto | 4 +- libatomic/src/daemon/change_impact.rs | 189 ++++++++------- libatomic/src/daemon/services_query.rs | 17 +- libatomic/tests/local_service_test.rs | 137 ++++++++--- tests/harness/14_vault_kg.sh | 58 +++-- 21 files changed, 857 insertions(+), 260 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 336358e0..7266b1a2 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -73,6 +73,11 @@ jobs: shell: bash run: ATOMIC_BIN="$PWD/target/debug/atomic" bash tests/harness/run_all.sh 09 + - name: Run knowledge graph closure harness + if: matrix.os == 'ubuntu-latest' + shell: bash + run: ATOMIC_BIN="$PWD/target/debug/atomic" bash tests/harness/46_knowledge_graph_closure.sh + fmt: name: Format runs-on: ubuntu-latest diff --git a/atomic-cli/src/commands/query/mod.rs b/atomic-cli/src/commands/query/mod.rs index 18f82cea..9c3ddbd6 100644 --- a/atomic-cli/src/commands/query/mod.rs +++ b/atomic-cli/src/commands/query/mod.rs @@ -1051,10 +1051,10 @@ pub struct QueryNodes { #[arg(long, short = 'k', default_value = "10")] pub limit: usize, - /// Filter by node kind (change, entity, file, view, memory, intent, goal). + /// Filter by canonical class IRI (for example, ). /// - /// When set, only nodes of this kind are returned. - /// Without this flag, all kinds are returned. + /// When set, only nodes asserting this type are returned. + /// Without this flag, all types are returned. #[arg(long, short = 't')] pub kind: Option, @@ -1142,7 +1142,7 @@ impl Command for QueryNodes { /// Get the neighborhood of a KG node. #[derive(Parser, Debug)] pub struct QueryNeighbors { - /// Node ID (e.g., "change:abc123", "file:src/auth.rs", "intent:PIMO-1"). + /// Canonical resource IRI, copied from query search or graph JSON. pub node_id: String, /// Traversal depth (1 or 2). diff --git a/atomic-cli/tests/git_shim_cb13d_test.rs b/atomic-cli/tests/git_shim_cb13d_test.rs index 247978ce..c51a6fd7 100644 --- a/atomic-cli/tests/git_shim_cb13d_test.rs +++ b/atomic-cli/tests/git_shim_cb13d_test.rs @@ -39,6 +39,8 @@ fn atomic(root: &Path, home: &Path, args: &[&str]) -> Output { .current_dir(root) .env("HOME", home) .env("ATOMIC_HOME", home.join(".atomic")) + .env("GIT_CONFIG_NOSYSTEM", "1") + .env("GIT_CONFIG_GLOBAL", "/dev/null") .env("GIT_AUTHOR_NAME", "CB-13D Tests") .env("GIT_AUTHOR_EMAIL", "cb13d@example.com") .env("GIT_COMMITTER_NAME", "CB-13D Tests") @@ -80,6 +82,10 @@ fn git(root: &Path, args: &[&str]) -> Output { Command::new("git") .args(args) .current_dir(root) + // These fixtures compare commit OIDs. User/system signing and hooks + // configuration must not introduce signatures or mutate their commits. + .env("GIT_CONFIG_NOSYSTEM", "1") + .env("GIT_CONFIG_GLOBAL", "/dev/null") .env("GIT_AUTHOR_NAME", "CB-13D Tests") .env("GIT_AUTHOR_EMAIL", "cb13d@example.com") .env("GIT_COMMITTER_NAME", "CB-13D Tests") @@ -281,6 +287,8 @@ impl WatchFixture { "--detach", linked_root.path().to_str().expect("utf8"), ]) + .env("GIT_CONFIG_NOSYSTEM", "1") + .env("GIT_CONFIG_GLOBAL", "/dev/null") .env("GIT_AUTHOR_DATE", "@1726732800 +0000") .env("GIT_COMMITTER_DATE", "@1726732800 +0000") .output() diff --git a/atomic-cli/tests/provenance_rpc_sink.rs b/atomic-cli/tests/provenance_rpc_sink.rs index fd0be968..61a94017 100644 --- a/atomic-cli/tests/provenance_rpc_sink.rs +++ b/atomic-cli/tests/provenance_rpc_sink.rs @@ -1,14 +1,14 @@ //! Integration tests for the RPC-based provenance journal sink -//! (`ProvenanceRpcSink`): the real daemon binary from the Reactor -//! checkout, the real `atomic` binary driving the real sink through the -//! full ProvenanceJournalSink trait flow over the real socket. +//! (`ProvenanceRpcSink`): the real `atomic` binary driving the real sink +//! and shared service handlers over a socket. The trait-flow test owns its +//! server in-process and requires no sibling checkout or external binary. //! //! The sink itself lives inside the (binary-only) CLI crate, so the suite //! drives it through the hidden `agent journal-rpc-selftest` harness //! command, which exercises exactly the trait surface the orchestrator //! uses: reserve → append (chunked under the wire budget) → prepare → //! load_frozen_envelopes (paged reassembly) → bind → acknowledge, plus -//! stop_turn/turn_status. A second test proves `agent hooks` journals over +//! stop_turn/turn_status. Optional Reactor tests prove `agent hooks` journals over //! the daemon's ProvenanceService when the daemon is already reachable //! (the daemon's RPC log must show the journal calls), and a third proves //! the D4 cutover: with NO daemon pre-started, a hook flow auto-starts @@ -50,17 +50,73 @@ fn reactor_root() -> Option { .filter(|root| root.join("Cargo.toml").exists()) } -/// `None` when the Reactor checkout (and thus the daemon binary) is -/// unavailable; every test in this suite skips in that case. +/// Discover an actual daemon target for the optional external transport tests. +/// The mandatory trait-flow test serves the same handlers in this process. fn daemon_binary() -> Option { - Some(reactor_root()?.join("target/debug/atomicd")) + let root = reactor_root()?; + // A sibling directory named reactor is not proof it contains atomicd. + // Cargo also owns the target directory (including CARGO_TARGET_DIR). + let output = Command::new("cargo") + .args([ + "metadata", + "--format-version", + "1", + "--no-deps", + "--locked", + "--offline", + "--manifest-path", + ]) + .arg(root.join("Cargo.toml")) + .output() + .expect("inspect Reactor targets"); + assert!( + output.status.success(), + "cannot inspect Reactor targets: {}", + String::from_utf8_lossy(&output.stderr) + ); + let metadata: serde_json::Value = + serde_json::from_slice(&output.stdout).expect("Cargo metadata JSON"); + let has_daemon = metadata["packages"] + .as_array() + .expect("Cargo packages") + .iter() + .any(|package| { + package["targets"] + .as_array() + .expect("Cargo targets") + .iter() + .any(|target| { + target["name"] == "atomicd" + && target["kind"] + .as_array() + .is_some_and(|kinds| kinds.iter().any(|kind| kind == "bin")) + }) + }); + if !has_daemon { + assert!( + std::env::var_os("ATOMIC_REACTOR_ROOT").is_none(), + "ATOMIC_REACTOR_ROOT explicitly names a workspace without an atomicd binary target" + ); + return None; + } + Some( + PathBuf::from( + metadata["target_directory"] + .as_str() + .expect("Cargo target directory"), + ) + .join("debug") + .join(format!("atomicd{}", std::env::consts::EXE_SUFFIX)), + ) } /// Skip-label for tests that require the Reactor checkout. fn skip_without_reactor(what: &str) { // cargo prints skipped-test output on stderr; keep the skip visible // in the run log without failing the suite. - eprintln!("skipping {what}: no Reactor checkout (set ATOMIC_REACTOR_ROOT to enable)"); + eprintln!( + "skipping {what}: no built Reactor atomicd target (set ATOMIC_REACTOR_ROOT to enable)" + ); } /// Guard that restores the socket override when dropped. @@ -179,39 +235,42 @@ fn run_atomic(args: &[&str], current_dir: &Path, stdin: Option<&[u8]>) -> std::p output } -#[test] +#[cfg(unix)] +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] #[serial] -fn rpc_sink_drives_the_full_trait_flow_over_the_socket() { +async fn rpc_sink_drives_the_full_trait_flow_over_the_socket() { + use libatomic::atomic::{ + daemon_service_server::DaemonServiceServer, + provenance_service_server::ProvenanceServiceServer, + }; + use libatomic::daemon::{ + services::DaemonImpl, services_agent::ProvenanceImpl, state::DaemonState, + }; + use std::sync::Arc; // The selftest is the REACTOR transport's exercise — opt in explicitly // (local mode is the CLI default and would refuse the command). let _service = VarEnv::set("ATOMIC_SERVICE", "reactor"); - // Build the daemon from the Reactor checkout when it is missing (the - // normal verification flow builds it first). - let (Some(reactor), Some(daemon)) = (reactor_root(), daemon_binary()) else { - skip_without_reactor("rpc_sink_drives_the_full_trait_flow_over_the_socket"); - return; - }; - if !daemon.exists() { - let output = Command::new("cargo") - .arg("build") - .arg("--manifest-path") - .arg(reactor.join("Cargo.toml")) - .output() - .expect("cargo build reactor"); - assert!( - output.status.success(), - "building atomicd failed: {}", - String::from_utf8_lossy(&output.stderr) - ); - } - let temp = TempDir::new().unwrap(); let repo = temp.path().join("repo"); drop(Repository::init(&repo).unwrap()); let socket = temp.path().join("atomicd.sock"); let _env = SocketEnv::set(&socket); - let mut daemon = KillOnDrop(spawn_daemon(&socket, &[])); - wait_for_daemon(&socket, &mut daemon.0); + let listener = tokio::net::UnixListener::bind(&socket).unwrap(); + let state = Arc::new(DaemonState::new()); + let (stop, stopped) = tokio::sync::oneshot::channel(); + let server = tokio::spawn( + tonic::transport::Server::builder() + .add_service(DaemonServiceServer::new(DaemonImpl { + state: state.clone(), + })) + .add_service(ProvenanceServiceServer::new(ProvenanceImpl { state })) + .serve_with_incoming_shutdown( + tokio_stream::wrappers::UnixListenerStream::new(listener), + async { + let _ = stopped.await; + }, + ), + ); // Six ~450 KiB wire envelopes (payload plus JSON framing) against // the sink's 1 MiB wire chunk budget: the batch must split into @@ -269,7 +328,8 @@ fn rpc_sink_drives_the_full_trait_flow_over_the_socket() { // The daemon really owns the journal: inspect the store the daemon // wrote (its per-request handles are closed between RPCs). - drop(daemon); + stop.send(()).unwrap(); + server.await.unwrap().unwrap(); let store = atomic_repository::redb_change_store::RedbChangeStore::open_existing( Repository::canonical_database_path(&repo).unwrap(), ) diff --git a/atomic-core/src/pristine/vault.rs b/atomic-core/src/pristine/vault.rs index e83d45be..5b50e03e 100644 --- a/atomic-core/src/pristine/vault.rs +++ b/atomic-core/src/pristine/vault.rs @@ -495,7 +495,7 @@ pub struct KgNode { /// Optional longer description (commit message, entity signature, etc.). #[serde(skip_serializing_if = "Option::is_none")] pub summary: Option, - /// Where this node was derived from (e.g., `"change_graph"`, `"ast_extraction"`, `"vault"`). + /// Where this node was derived from (e.g., `"vcs"`, `"session_ledger"`, `"vault"`). pub source: String, /// Arbitrary metadata as JSON. #[serde(skip_serializing_if = "Option::is_none")] @@ -554,15 +554,15 @@ impl KgNode { /// A directed edge in the knowledge graph. /// -/// Edges encode relationships: `MODIFIES`, `DEFINES`, `DEPENDS_ON`, -/// `ON_VIEW`, `CHILD_OF`, `LINKED_INTENT`, etc. +/// Only predicates from the canonical workflow ontology can be stored. +/// Source-code relationships belong to the request-scoped syntax index. #[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] pub struct KgEdge { /// Source node ID. pub from_id: String, /// Target node ID. pub to_id: String, - /// Relationship type (e.g., `"MODIFIES"`, `"DEFINES"`, `"DEPENDS_ON"`). + /// Canonical workflow relationship. pub kind: crate::kg::ontology::Predicate, /// Optional edge metadata. #[serde(skip_serializing_if = "Option::is_none")] diff --git a/atomic-core/tests/kg_vocabulary_lint.rs b/atomic-core/tests/kg_vocabulary_lint.rs index 73b50d33..9eaa4cd4 100644 --- a/atomic-core/tests/kg_vocabulary_lint.rs +++ b/atomic-core/tests/kg_vocabulary_lint.rs @@ -66,6 +66,7 @@ fn workspace_has_no_retired_kg_vocabulary_literals() { if path.is_dir() && (name.starts_with("atomic-") || name == "libatomic") { scan(&path.join("src"), &forbidden, &mut failures); scan(&path.join("tests"), &forbidden, &mut failures); + scan(&path.join("examples"), &forbidden, &mut failures); } } assert!( diff --git a/atomic-repository/src/repository/changes.rs b/atomic-repository/src/repository/changes.rs index c597d4b6..8e224c96 100644 --- a/atomic-repository/src/repository/changes.rs +++ b/atomic-repository/src/repository/changes.rs @@ -143,9 +143,16 @@ impl Repository { let stored = self.save_change_bytes(hash, bytes, &change)?; self.ensure_project_identity(atomic_core::kg::project_id::MintReason::Upgrade)?; if !change.kind().is_snapshot() { + let hashes = self + .change_store + .iter_changes() + .collect::, _>>() + .map_err(|e| RepositoryError::Database(e.to_string()))?; let mut txn = self.pristine.write_txn()?; if txn.kg_writes_enabled()? { self.project_change_kg(&mut txn, hash, &change)?; + self.repair_change_dependencies_kg(&mut txn, hash)?; + self.project_vault_kg(&mut txn, hashes)?; } txn.commit()?; } diff --git a/atomic-repository/src/repository/content.rs b/atomic-repository/src/repository/content.rs index 69935269..bb288fc9 100644 --- a/atomic-repository/src/repository/content.rs +++ b/atomic-repository/src/repository/content.rs @@ -646,32 +646,7 @@ impl Repository { path: P, change_hash: &Hash, ) -> Result>, RepositoryError> { - let path = path.as_ref(); - let normalized = normalize_path(path); - - let txn = self - .pristine - .read_txn() - .map_err(|e| RepositoryError::Database(e.to_string()))?; - - // Get the current view - let view = txn - .get_view(&self.current_view) - .map_err(|e| RepositoryError::Database(e.to_string()))? - .ok_or_else(|| RepositoryError::ViewNotFound { - name: self.current_view.clone(), - })?; - - let Some(visibility) = graph_visibility_at_change(&txn, &view, change_hash, false)? else { - return Ok(None); - }; - - let cached_txn = - CachedGraphTxn::new(&txn).map_err(|e| RepositoryError::Database(e.to_string()))?; - match self.get_materialized_entry_with_visibility(&cached_txn, &normalized, visibility)? { - MaterializedEntry::Absent { .. } => Ok(None), - MaterializedEntry::Present { bytes, .. } => Ok(Some(bytes)), - } + self.get_file_content_before_change_on_view(path, change_hash, &self.current_view) } /// Get file content as it was AFTER a specific change was applied. @@ -711,7 +686,40 @@ impl Repository { path: P, change_hash: &Hash, ) -> Result>, RepositoryError> { - let path = path.as_ref(); + self.get_file_content_after_change_on_view(path, change_hash, &self.current_view) + } + + /// Reconstruct the state before a change in the named view's effective + /// history, without switching the working copy. Returns `None` when the + /// file or change is absent in that view. + pub fn get_file_content_before_change_on_view>( + &self, + path: P, + change_hash: &Hash, + view_name: &str, + ) -> Result>, RepositoryError> { + self.file_content_at_change_on_view(path.as_ref(), change_hash, view_name, false) + } + + /// Reconstruct the state after a change in the named view's effective + /// history, without switching the working copy. Returns `None` when the + /// file or change is absent in that view. + pub fn get_file_content_after_change_on_view>( + &self, + path: P, + change_hash: &Hash, + view_name: &str, + ) -> Result>, RepositoryError> { + self.file_content_at_change_on_view(path.as_ref(), change_hash, view_name, true) + } + + fn file_content_at_change_on_view( + &self, + path: &Path, + change_hash: &Hash, + view_name: &str, + include_change: bool, + ) -> Result>, RepositoryError> { let normalized = normalize_path(path); let txn = self @@ -719,15 +727,16 @@ impl Repository { .read_txn() .map_err(|e| RepositoryError::Database(e.to_string()))?; - // Get the current view let view = txn - .get_view(&self.current_view) + .get_view(view_name) .map_err(|e| RepositoryError::Database(e.to_string()))? .ok_or_else(|| RepositoryError::ViewNotFound { - name: self.current_view.clone(), + name: view_name.to_owned(), })?; - let Some(visibility) = graph_visibility_at_change(&txn, &view, change_hash, true)? else { + let Some(visibility) = + graph_visibility_at_change(&txn, &view, change_hash, include_change)? + else { return Ok(None); }; diff --git a/atomic-repository/src/repository/syntax_index.rs b/atomic-repository/src/repository/syntax_index.rs index 4a186c6b..d9c1aa3a 100644 --- a/atomic-repository/src/repository/syntax_index.rs +++ b/atomic-repository/src/repository/syntax_index.rs @@ -27,6 +27,37 @@ fn callable(entity: &Entity) -> bool { matches!(entity.kind, EntityKind::Function | EntityKind::Method) } +fn relative_script_import(importer: &str, module: &str, target: &str) -> Option { + let script = |path: &Path| { + matches!( + path.extension().and_then(|ext| ext.to_str()), + Some("ts" | "tsx" | "js" | "jsx" | "mjs" | "mts" | "cjs" | "cts") + ) + }; + if !script(Path::new(importer)) || !module.starts_with('.') { + return None; + } + if !script(Path::new(target)) { + return Some(false); + } + let mut resolved = std::path::PathBuf::new(); + let joined = Path::new(importer) + .parent() + .unwrap_or_else(|| Path::new("")) + .join(module); + for component in joined.components() { + match component { + std::path::Component::Normal(name) => resolved.push(name), + std::path::Component::ParentDir if !resolved.pop() => return Some(false), + std::path::Component::ParentDir | std::path::Component::CurDir => {} + _ => return Some(false), + } + } + let target = Path::new(target); + let stem = target.with_extension(""); + Some(target == resolved || stem == resolved || stem == resolved.join("index")) +} + impl SyntaxIndex { pub fn from_sources(sources: &[(String, String)]) -> Self { let mut parser = ParserRegistry::new(); @@ -88,7 +119,9 @@ impl SyntaxIndex { .file_stem() .and_then(|s| s.to_str()) .unwrap_or_default(); - if target != *path && tokens.contains(stem) { + let matches = relative_script_import(path, &import.name, target) + .unwrap_or_else(|| tokens.contains(stem)); + if target != *path && matches { index.file_deps.insert(((*path).clone(), target.clone())); } } @@ -185,3 +218,27 @@ impl Repository { Ok(self.syntax_index(None)?.callers(target)) } } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn script_imports_follow_relative_paths_instead_of_duplicate_file_stems() { + let index = SyntaxIndex::from_sources(&[ + ("src/ui.ts".into(), "import { changed } from './api';\nimport '../shared';\nexport { api } from './nested';\n".into()), + ("src/api.ts".into(), "export function changed() {}".into()), + ("tests/api.ts".into(), "function other() {}".into()), + ("shared.ts".into(), "export const value = 1;".into()), + ("src/nested/index.ts".into(), "export const api = 1;".into()), + ]); + assert_eq!( + index.file_deps, + BTreeSet::from([ + ("src/ui.ts".into(), "src/api.ts".into()), + ("src/ui.ts".into(), "shared.ts".into()), + ("src/ui.ts".into(), "src/nested/index.ts".into()), + ]) + ); + } +} diff --git a/atomic-repository/src/repository/vault_kg_enrich.rs b/atomic-repository/src/repository/vault_kg_enrich.rs index e6c80bcf..18650c33 100644 --- a/atomic-repository/src/repository/vault_kg_enrich.rs +++ b/atomic-repository/src/repository/vault_kg_enrich.rs @@ -143,6 +143,7 @@ impl Repository { txn.upsert_kg_node(&KgNode::new(&author_id, C::Identity, &author.name, "vcs"))?; txn.upsert_kg_edge(&KgEdge::new(&id, author_id, P::AuthoredBy))?; } + let mut missing_dependency = false; for hash in change.dependencies() { if self.has_change(hash) { let target = iri::mint_change(hash); @@ -156,6 +157,7 @@ impl Repository { } txn.upsert_kg_edge(&KgEdge::new(&id, target, P::DirectlyDependsOnChange))?; } else { + missing_dependency = true; txn.kg_finding( "UnresolvedReference", &id, @@ -163,6 +165,9 @@ impl Repository { )?; } } + if !missing_dependency { + txn.del_kg_node(&iri::mint_finding(&project, "UnresolvedReference", &id))?; + } let rules = IgnoreRules::load_for_enrichment(self.root()); let mut paths = std::collections::BTreeSet::new(); for file in change.file_ops() { @@ -186,6 +191,43 @@ impl Repository { Ok(paths.len()) } + pub(super) fn repair_change_dependencies_kg( + &self, + txn: &mut atomic_core::WriteTxn<'_>, + arrived: &Hash, + ) -> Result<(), RepositoryError> { + let mut pending = std::collections::BTreeSet::new(); + for edge in txn.all_kg_edges()? { + if edge.kind != P::AboutResource { + continue; + } + let Some(finding) = txn.get_kg_node(&edge.from_id)? else { + continue; + }; + if finding + .metadata + .as_ref() + .and_then(|m| m.get("finding_code")) + .and_then(serde_json::Value::as_str) + != Some("UnresolvedReference") + { + continue; + } + if let Ok(hash) = iri::parse_change(&edge.to_id) { + pending.insert(hash); + } + } + for hash in pending { + if self.has_change(&hash) { + let change = self.load_change(&hash)?; + if change.dependencies().contains(arrived) { + self.project_change_kg(txn, &hash, &change)?; + } + } + } + Ok(()) + } + pub(super) fn project_view_kg( &self, txn: &mut atomic_core::WriteTxn<'_>, diff --git a/atomic-repository/src/repository/vault_projection.rs b/atomic-repository/src/repository/vault_projection.rs index d2ca7218..44e89798 100644 --- a/atomic-repository/src/repository/vault_projection.rs +++ b/atomic-repository/src/repository/vault_projection.rs @@ -14,7 +14,10 @@ use std::collections::BTreeMap; pub(super) struct VaultProjection { project: ProjectId, + // Canonical IRIs are exact identities. Only historical aliases may fold + // case, and a collision must stay unresolved rather than pick a winner. subjects: BTreeMap, + aliases: BTreeMap>, paths: BTreeMap, hashes: BTreeMap, } @@ -40,6 +43,7 @@ impl VaultProjection { let mut context = Self { project, subjects: BTreeMap::new(), + aliases: BTreeMap::new(), paths: BTreeMap::new(), hashes, }; @@ -101,25 +105,22 @@ impl VaultProjection { context.paths.insert(meta.path.clone(), subject.clone()); context .subjects - .insert(subject.to_lowercase(), (subject.clone(), class)); - context.subjects.insert( - format!("urn:atomic:{local_kind}:{name}").to_lowercase(), + .insert(subject.clone(), (subject.clone(), class)); + context.add_alias( + format!("urn:atomic:{local_kind}:{name}"), (subject.clone(), class), ); if entry.entry_type == VaultEntryType::Intent { for (key, summary) in &manifest.intents { if summary.vault_path == meta.path { - context.subjects.insert( - format!("urn:atomic:intent:{key}").to_lowercase(), + context.add_alias( + format!("urn:atomic:intent:{key}"), (subject.clone(), class), ); } } if let Some(id) = fm.get("id").and_then(|v| v.as_str()) { - context.subjects.insert( - format!("urn:atomic:intent:{id}").to_lowercase(), - (subject.clone(), class), - ); + context.add_alias(format!("urn:atomic:intent:{id}"), (subject.clone(), class)); } if let Some(map) = fm.as_object() { if let Ok(node) = atomic_canonical::lift::lift_intent( @@ -158,10 +159,8 @@ impl VaultProjection { message: e.to_string(), } })?; - context - .subjects - .insert(raw.to_lowercase(), (id.clone(), class)); - context.subjects.insert(id.to_lowercase(), (id, class)); + context.add_alias(raw.to_owned(), (id.clone(), class)); + context.subjects.insert(id.clone(), (id, class)); } } } @@ -170,13 +169,27 @@ impl VaultProjection { Ok(context) } + fn add_alias(&mut self, alias: String, target: (String, C)) { + self.aliases + .entry(alias.to_lowercase()) + .and_modify(|existing| { + if existing.as_ref() != Some(&target) { + *existing = None; + } + }) + .or_insert(Some(target)); + } + fn resolve(&self, raw: &str) -> Option<(String, C)> { if raw.starts_with("memory:") || raw.starts_with("intent:") { return self.resolve(&format!("urn:atomic:{raw}")); } - if let Some(target) = self.subjects.get(&raw.to_lowercase()) { + if let Some(target) = self.subjects.get(raw) { return Some(target.clone()); } + if let Some(target) = self.aliases.get(&raw.to_lowercase()) { + return target.clone(); + } if let Ok(hash) = iri::parse_change(raw) { return self .hashes @@ -225,6 +238,8 @@ impl VaultProjection { fn link(&self, graph: &mut KgSubgraph, subject: &str, raw: &str, predicate: P, path: &str) { if let Some((target, class)) = self.resolve(raw) { + // The referring entry owns the edge, never the target's metadata. + // In particular, a child type does not imply local ownership. graph .nodes .push(KgNode::new(&target, class, raw, "vault_reference")); @@ -277,18 +292,14 @@ impl VaultProjection { }; let fm: serde_json::Value = serde_json::from_str(&entry.frontmatter_json).unwrap_or_default(); - let class = self - .subjects - .get(&subject.to_lowercase()) - .expect("indexed subject") - .1; + let class = self.subjects.get(subject).expect("indexed subject").1; let label = fm .get("title") .or_else(|| fm.get("name")) .and_then(|v| v.as_str()) .unwrap_or(path); let mut node = KgNode::new(subject, class, label, "vault") - .with_metadata(serde_json::json!({"vault_path": path, "created_at": entry.created_at, "updated_at": entry.updated_at, + .with_metadata(serde_json::json!({"vault_path": path, "derived_from_vault_path": path, "created_at": entry.created_at, "updated_at": entry.updated_at, "status": fm.get("status"), "priority": fm.get("priority"), "human_key": fm.get("id"), "uid": fm.get("uid")})); if let Some(description) = fm .get("description") @@ -388,7 +399,7 @@ impl VaultProjection { for ac in &intent.has_acceptance_criterion { if let Some((id, class)) = self.resolve(&ac.id) { graph.nodes.push(KgNode::new(&id, class, &ac.text, "vault").with_summary(&ac.text) - .with_metadata(serde_json::json!({"status": ac.ac_status, "verifiedBy": ac.verified_by, "evidence": ac.evidence}))); + .with_metadata(serde_json::json!({"status": ac.ac_status, "verifiedBy": ac.verified_by, "evidence": ac.evidence, "derived_from_vault_path": path}))); graph .edges .push(KgEdge::new(subject, id, P::HasAcceptanceCriterion)); @@ -399,7 +410,7 @@ impl VaultProjection { graph.nodes.push( KgNode::new(&id, class, &task.text, "vault") .with_summary(&task.text) - .with_metadata(serde_json::json!({"status": task.task_status})), + .with_metadata(serde_json::json!({"status": task.task_status, "derived_from_vault_path": path})), ); graph.edges.push(KgEdge::new(subject, &id, P::HasTask)); for ac in task.satisfies.as_slice() { @@ -418,7 +429,10 @@ impl VaultProjection { if let Some((id, class)) = self.resolve(&item.id) { graph.nodes.push( KgNode::new(&id, class, &item.text, "vault") - .with_summary(&item.text), + .with_summary(&item.text) + .with_metadata( + serde_json::json!({"derived_from_vault_path": path}), + ), ); graph.edges.push(KgEdge::new(subject, id, predicate)); } @@ -433,7 +447,10 @@ impl VaultProjection { if let Some((id, class)) = self.resolve(&constraint.id) { graph.nodes.push( KgNode::new(&id, class, &constraint.text, "vault") - .with_summary(&constraint.text), + .with_summary(&constraint.text) + .with_metadata( + serde_json::json!({"derived_from_vault_path": path}), + ), ); graph.edges.push(KgEdge::new(subject, id, P::HasConstraint)); } @@ -492,18 +509,6 @@ impl VaultProjection { } } graph.nodes.push(node); - for node in &mut graph.nodes { - if node.id == *subject - || node.has_type(C::Task) - || node.has_type(C::AcceptanceCriterion) - || node.has_type(C::ScopeItem) - || node.has_type(C::Constraint) - || node.has_type(C::Finding) - { - let metadata = node.metadata.get_or_insert_with(|| serde_json::json!({})); - metadata["derived_from_vault_path"] = serde_json::json!(path); - } - } for edge in &mut graph.edges { let metadata = edge.metadata.get_or_insert_with(|| serde_json::json!({})); metadata["derived_from_vault_path"] = serde_json::json!(path); diff --git a/atomic-repository/src/repository/vault_triples.rs b/atomic-repository/src/repository/vault_triples.rs index 61f3b387..fcbd25d3 100644 --- a/atomic-repository/src/repository/vault_triples.rs +++ b/atomic-repository/src/repository/vault_triples.rs @@ -136,18 +136,27 @@ impl Repository { if !txn.kg_writes_enabled()? { return Ok(0); } - let context = VaultProjection::new(&txn, txn.kg_project()?, hashes)?; + let total = self.project_vault_kg(&mut txn, hashes)?; + txn.commit()?; + self.refresh_session_kg()?; + Ok(total) + } + + pub(super) fn project_vault_kg( + &self, + txn: &mut atomic_core::WriteTxn<'_>, + hashes: Vec, + ) -> Result { + let context = VaultProjection::new(txn, txn.kg_project()?, hashes)?; let entries = txn.list_vault_entries("", None)?; let mut total = 0; for meta in entries { if let Some(entry) = txn.get_vault_entry(&meta.path)? { let graph = context.extract(&meta.path, &entry)?; - store_projection(&mut txn, &meta.path, &graph.nodes, &graph.edges)?; + store_projection(txn, &meta.path, &graph.nodes, &graph.edges)?; total += graph.nodes.len() + graph.edges.len(); } } - txn.commit()?; - self.refresh_session_kg()?; Ok(total) } diff --git a/atomic-repository/tests/kg_projection.rs b/atomic-repository/tests/kg_projection.rs index e9cb916f..5e869867 100644 --- a/atomic-repository/tests/kg_projection.rs +++ b/atomic-repository/tests/kg_projection.rs @@ -5,12 +5,236 @@ use atomic_core::{ ontology::{Class as C, Predicate as P}, }, pristine::{KgTxnT, VaultEntryType}, - types::Merkle, + types::{Base32, Merkle}, }; use atomic_repository::{Repository, UnrecordOptions, ViewManifest}; const UID: &str = "01ARZ3NDEKTSV4RRFFQ69G5FAV"; +#[test] +fn references_never_take_ownership_of_another_intents_children() { + for reference_first in [false, true] { + let directory = tempfile::tempdir().unwrap(); + let repo = Repository::init(directory.path()).unwrap(); + repo.init_vault().unwrap(); + let project = repo.project_id().unwrap(); + let owner = format!("intents/{UID}/intent.md"); + let reference = || { + repo.vault_store( + "memory/reference.md", + VaultEntryType::Memory, + b"Reference, not ownership.".to_vec(), + serde_json::json!({"derivedFrom":["urn:atomic:ac:target-ac", "urn:atomic:task:target-task"]}).to_string(), + ).unwrap(); + }; + if reference_first { + reference(); + } + repo.vault_store( + &owner, + VaultEntryType::Intent, + b":::why\nOwner.\n:::\n:::acceptance-criterion{#target-ac status=met verifiedBy=review evidence=test}\nAuthoritative acceptance text.\n:::\n:::task{#target-task status=done criteria=target-ac}\nAuthoritative task text.\n:::\n".to_vec(), + serde_json::json!({"uid":UID,"id":"TEST::owner::1","title":"Owner","status":"in_progress"}).to_string(), + ).unwrap(); + if !reference_first { + reference(); + } + let ac = iri::mint_child( + &project, + UID, + iri::ChildKind::AcceptanceCriterion, + "target-ac", + ) + .unwrap(); + let task = iri::mint_child(&project, UID, iri::ChildKind::Task, "target-task").unwrap(); + let assert_children = || { + for (id, label, status) in [ + (&ac, "Authoritative acceptance text.", "met"), + (&task, "Authoritative task text.", "done"), + ] { + let node = repo.vault_kg_node(id).unwrap().unwrap(); + assert_eq!(node.source, "vault"); + assert_eq!(node.label, label); + let metadata = node.metadata.unwrap(); + assert_eq!(metadata["status"], status); + assert_eq!(metadata["derived_from_vault_path"], owner); + } + let node = repo.vault_kg_node(&ac).unwrap().unwrap(); + assert_eq!(node.metadata.as_ref().unwrap()["evidence"], "test"); + }; + assert_children(); + repo.vault_reindex_kg().unwrap(); + assert_children(); + repo.rebuild_kg(true).unwrap(); + assert_children(); + repo.vault_delete("memory/reference.md").unwrap(); + assert_children(); + reference(); + repo.vault_delete(&owner).unwrap(); + assert!(repo.vault_kg_node(&ac).unwrap().is_none()); + assert!(repo.vault_kg_node(&task).unwrap().is_none()); + let source = iri::mint_memory(&project, "reference"); + assert!(repo + .vault_kg_node(&iri::mint_finding(&project, "UnresolvedReference", &source)) + .unwrap() + .is_some()); + } +} + +#[test] +fn received_changes_repair_vault_references_without_changing_view_membership() { + for change_first in [false, true] { + let directory = tempfile::tempdir().unwrap(); + let repo = Repository::init(directory.path()).unwrap(); + repo.init_vault().unwrap(); + let change = Change::empty(ChangeHeader::new("late dependency")); + let mut bytes = Vec::new(); + let hash = change.serialize(&mut bytes).unwrap(); + if change_first { + repo.import_change_bytes(&hash, &bytes).unwrap(); + } + repo.vault_store( + "memory/source.md", + VaultEntryType::Memory, + b"Source.".to_vec(), + serde_json::json!({"derivedFrom":[format!("urn:atomic:change:{}", hash.to_base32())]}) + .to_string(), + ) + .unwrap(); + let project = repo.project_id().unwrap(); + let source = iri::mint_memory(&project, "source"); + let finding = iri::mint_finding(&project, "UnresolvedReference", &source); + assert_eq!( + repo.vault_kg_node(&finding).unwrap().is_some(), + !change_first + ); + for _ in 0..2 { + repo.import_change_bytes(&hash, &bytes).unwrap(); + let snapshot = repo.workflow_snapshot().unwrap(); + assert!(snapshot.edges.iter().any(|edge| edge.from_id == source + && edge.to_id == iri::mint_change(&hash) + && edge.kind == P::ProvWasDerivedFrom)); + assert!(repo.vault_kg_node(&finding).unwrap().is_none()); + assert!(!snapshot + .edges + .iter() + .any(|edge| edge.from_id == iri::mint_change(&hash) + && edge.kind == P::DirectlyOnView)); + } + } +} + +#[test] +fn late_dependencies_repair_change_edges_and_remove_only_resolved_findings() { + let directory = tempfile::tempdir().unwrap(); + let repo = Repository::init(directory.path()).unwrap(); + repo.init_kg().unwrap(); + let mut objects = Vec::new(); + for name in ["first parent", "second parent"] { + let mut bytes = Vec::new(); + let hash = Change::empty(ChangeHeader::new(name)) + .serialize(&mut bytes) + .unwrap(); + objects.push((hash, bytes)); + } + // Change headers contain timestamps, so fixture creation order is not a + // canonical dependency order. Serialization correctly rejects unsorted hashes. + objects.sort_by_key(|(hash, _)| *hash); + let child = Change::new( + ChangeHeader::new("child"), + vec![], + vec![], + objects.iter().map(|(hash, _)| *hash).collect(), + ); + let mut bytes = Vec::new(); + let hash = child.serialize(&mut bytes).unwrap(); + repo.import_change_bytes(&hash, &bytes).unwrap(); + let project = repo.project_id().unwrap(); + let child = iri::mint_change(&hash); + let finding = iri::mint_finding(&project, "UnresolvedReference", &child); + assert!(repo.vault_kg_node(&finding).unwrap().is_some()); + for (i, (hash, bytes)) in objects.iter().enumerate() { + repo.import_change_bytes(hash, bytes).unwrap(); + assert!(repo + .workflow_snapshot() + .unwrap() + .edges + .iter() + .any(|edge| edge.from_id == child + && edge.to_id == iri::mint_change(hash) + && edge.kind == P::DirectlyDependsOnChange)); + assert_eq!(repo.vault_kg_node(&finding).unwrap().is_some(), i == 0); + } +} + +#[test] +fn canonical_references_are_exact_and_ambiguous_legacy_aliases_fail_closed() { + let directory = tempfile::tempdir().unwrap(); + let repo = Repository::init(directory.path()).unwrap(); + repo.init_vault().unwrap(); + for name in ["A", "a"] { + repo.vault_store( + &format!("memory/{name}.md"), + VaultEntryType::Memory, + name.as_bytes().to_vec(), + "{}".into(), + ) + .unwrap(); + } + let project = repo.project_id().unwrap(); + let source = iri::mint_memory(&project, "source"); + let upper = iri::mint_memory(&project, "A"); + repo.vault_store( + "memory/source.md", + VaultEntryType::Memory, + b"Exact canonical target".to_vec(), + serde_json::json!({"derivedFrom":[upper]}).to_string(), + ) + .unwrap(); + for full in [false, true] { + if full { + repo.rebuild_kg(true).unwrap(); + } + let edges = repo.workflow_snapshot().unwrap().edges; + assert!(edges.iter().any(|edge| edge.from_id == source + && edge.to_id == upper + && edge.kind == P::ProvWasDerivedFrom)); + assert!(!edges.iter().any(|edge| edge.from_id == source + && edge.to_id == iri::mint_memory(&project, "a") + && edge.kind == P::ProvWasDerivedFrom)); + } + repo.vault_store( + "memory/source.md", + VaultEntryType::Memory, + b"Ambiguous legacy alias".to_vec(), + serde_json::json!({"derivedFrom":["urn:atomic:memory:a"]}).to_string(), + ) + .unwrap(); + assert!(!repo + .workflow_snapshot() + .unwrap() + .edges + .iter() + .any(|edge| edge.from_id == source && edge.kind == P::ProvWasDerivedFrom)); + assert!(repo + .vault_kg_node(&iri::mint_finding(&project, "UnresolvedReference", &source)) + .unwrap() + .is_some()); + repo.vault_delete("memory/a.md").unwrap(); + assert!(repo + .workflow_snapshot() + .unwrap() + .edges + .iter() + .any(|edge| edge.from_id == source + && edge.to_id == upper + && edge.kind == P::ProvWasDerivedFrom)); + assert!(repo + .vault_kg_node(&iri::mint_finding(&project, "UnresolvedReference", &source)) + .unwrap() + .is_none()); +} + #[test] fn later_intent_ingestion_repairs_session_links_and_deletion_restores_gap_findings() { let directory = tempfile::tempdir().unwrap(); diff --git a/atomic-semantic/src/parser.rs b/atomic-semantic/src/parser.rs index a304a3e1..19a27a5d 100644 --- a/atomic-semantic/src/parser.rs +++ b/atomic-semantic/src/parser.rs @@ -555,6 +555,23 @@ impl TypeScriptExtractor { let child_in_export = in_export || is_export; match node.kind() { + "import_statement" | "export_statement" => { + if let Some(module) = node.child_by_field_name("source") { + if let Some(text) = self.node_text(&module, source) { + entities.push( + Entity::new( + text.trim_matches(['\'', '"']), + EntityKind::Import, + file_path, + node.start_position().row as u32 + 1, + node.end_position().row as u32 + 1, + ) + .with_signature(self.node_text(node, source).unwrap_or_default()), + ); + } + } + } + // Function declarations "function_declaration" => { if let Some(entity) = @@ -1181,6 +1198,34 @@ impl TypeScriptExtractor { mod tests { use super::*; + #[test] + fn extracts_import_and_reexport_module_sources_not_imported_symbol_names() { + let mut parser = TypeScriptExtractor::new(); + let entities = parser.extract( + "import { changed } from './api';\nimport './startup';\nexport { value } from '../shared';\n// import './not-real';\nexport function kept() {}\n", + "src/ui.ts", + ); + let imports: Vec<_> = entities + .iter() + .filter(|entity| entity.kind == EntityKind::Import) + .collect(); + assert_eq!( + imports + .iter() + .map(|entity| entity.name.as_str()) + .collect::>(), + ["./api", "./startup", "../shared"] + ); + assert_eq!(imports[0].line, 1); + assert_eq!( + imports[0].signature.as_deref(), + Some("import { changed } from './api';") + ); + assert!(entities + .iter() + .any(|entity| entity.name == "kept" && entity.exported)); + } + #[test] fn test_extract_function() { let mut extractor = TypeScriptExtractor::new(); diff --git a/libatomic/examples/replication_acceptance/fixture.rs b/libatomic/examples/replication_acceptance/fixture.rs index d6e14dc2..bffd131f 100644 --- a/libatomic/examples/replication_acceptance/fixture.rs +++ b/libatomic/examples/replication_acceptance/fixture.rs @@ -162,7 +162,7 @@ pub fn create(binary: &Path, root: &Path, config: &Path) -> Result { let mut txn = repo.pristine().write_txn()?; txn.upsert_kg_node(&atomic_core::pristine::vault::KgNode::new( "fixture:authored-only", - "intent", + atomic_core::kg::ontology::Class::WorkIntent, "Authored DB-only relationship source", "authored", ))?; diff --git a/libatomic/proto/CONTRACT.md b/libatomic/proto/CONTRACT.md index 379645cc..5ed37b22 100644 --- a/libatomic/proto/CONTRACT.md +++ b/libatomic/proto/CONTRACT.md @@ -223,12 +223,14 @@ empty graph change; absent summary means an older producer, not zero. `GetChange.includes.blast_radius` opts into the additive `blast_radius` response (field 12). It binds directly changed paths to the selected hash and observes -incoming `CALLS`/`INCLUDES` from the current repository KG. Public API changes +incoming `calls`/`includes` from a request-scoped syntax index of recorded files +visible in the selected Atomic view (the current view by default). Public API changes are found by parsing the recorded before/after sources and matching changed -line ranges in both versions, including deletions. Entity IDs come from indexed -DEFINES edges, not historical-line guesses. No working-copy source reads, KG +line ranges in both versions, including deletions. Callables use `#` +selectors, not persistent entity IDs or historical-line guesses. No working-copy source reads, KG enrichment, co-change traversal or write-open occurs. The response is not a -historical/view-isolated dependency snapshot and says so explicitly. Missing +historical dependency snapshot and says so explicitly. The selected view scopes +both historical reconstruction and current dependency context. Missing parsers/content/definitions, ambiguity and analysis/display limits yield partial coverage, never a private-change assertion. Results deduplicate file paths and preserve hop 1 when a direct file also appears as a caller. No random impact or diff --git a/libatomic/proto/atomic/services/repository_query/repository_query_messages.proto b/libatomic/proto/atomic/services/repository_query/repository_query_messages.proto index da16f711..e04c7da2 100644 --- a/libatomic/proto/atomic/services/repository_query/repository_query_messages.proto +++ b/libatomic/proto/atomic/services/repository_query/repository_query_messages.proto @@ -183,7 +183,7 @@ message ChangeBlastRadius { Hash change_hash = 1; string mode = 2; // structural | private | partial repeated ChangeImpactFile files = 3; // distinct paths, direct files win - repeated ChangeImpactEdge edges = 4; // incoming CALLS / INCLUDES, never co-change + repeated ChangeImpactEdge edges = 4; // incoming calls / includes from view syntax, never co-change repeated string warnings = 5; string context = 6; } @@ -194,7 +194,7 @@ message ChangeImpactFile { message ChangeImpactEdge { string from_path = 1; // caller/importer string to_path = 2; // directly changed code - string relation = 3; // CALLS | INCLUDES + string relation = 3; // calls | includes (syntax relationships, not KG predicates) } // Recorded values, not estimates or signature-verification results. No prompt, diff --git a/libatomic/src/daemon/change_impact.rs b/libatomic/src/daemon/change_impact.rs index bd0c72df..b7693008 100644 --- a/libatomic/src/daemon/change_impact.rs +++ b/libatomic/src/daemon/change_impact.rs @@ -1,10 +1,11 @@ //! Change-local structural impact. Never read working-copy sources, rebuild the //! KG, traverse co-change history, or turn missing analysis into "private". +//! Dependency context comes from recorded bytes visible in the selected Atomic +//! view, never from a persisted code-structure graph or working-copy buffers. use crate::atomic::{ChangeBlastRadius, ChangeImpactEdge, ChangeImpactFile}; use atomic_core::change::Change; -use atomic_core::pristine::vault::KgSubgraph; use atomic_core::types::Hash; -use atomic_repository::Repository; +use atomic_repository::{Repository, SyntaxIndex}; use atomic_semantic::{Entity, ParserRegistry}; use similar::{ChangeTag, TextDiff}; use std::collections::{BTreeMap, BTreeSet}; @@ -13,7 +14,12 @@ const MAX_FILES: usize = 2000; const MAX_SOURCE_BYTES: usize = 2 * 1024 * 1024; const MAX_CALLERS: usize = 30; -pub(super) fn project(repo: &Repository, change: &Change, hash: &Hash) -> ChangeBlastRadius { +pub(super) fn project( + repo: &Repository, + change: &Change, + hash: &Hash, + view: &str, +) -> ChangeBlastRadius { let mut paths = atomic_repository::get_files_in_change(change); paths.extend(change.file_ops().iter().map(|op| op.path().to_string())); paths.sort(); @@ -29,6 +35,7 @@ pub(super) fn project(repo: &Repository, change: &Change, hash: &Hash) -> Change let mut registry = ParserRegistry::new(); let mut analyzed = 0; let mut public_change = false; + let mut syntax = None; for (index, path) in paths.into_iter().enumerate() { if index >= 100 { warnings.push("Structural analysis limited to 100 direct files; remaining direct paths are retained.".into()); @@ -38,8 +45,8 @@ pub(super) fn project(repo: &Repository, change: &Change, hash: &Hash) -> Change warnings.push(format!("{path}: no structural parser; direct impact only.")); continue; } - let before = repo.get_file_content_before_change(&path, hash); - let after = repo.get_file_content_after_change(&path, hash); + let before = repo.get_file_content_before_change_on_view(&path, hash, view); + let after = repo.get_file_content_after_change_on_view(&path, hash, view); let (Ok(before), Ok(after)) = (before, after) else { warnings.push(format!("{path}: historical content reconstruction failed.")); continue; @@ -80,60 +87,44 @@ pub(super) fn project(repo: &Repository, change: &Change, hash: &Hash) -> Change continue; } public_change = true; - let file_id = format!("file:{path}"); - let Ok(graph) = repo.vault_kg_neighbors(&file_id, 1) else { - warnings.push(format!("{path}: knowledge graph unavailable.")); - continue; + let index = syntax.get_or_insert_with(|| repo.syntax_index(Some(view))); + let index = match index { + Ok(index) => index, + Err(error) => { + warnings.push(format!("{path}: view syntax unavailable: {error}")); + continue; + } }; - let defined: BTreeSet<_> = graph - .edges - .iter() - .filter(|e| e.from_id == file_id && e.kind.eq_ignore_ascii_case("DEFINES")) - .map(|e| e.to_id.as_str()) - .collect(); - if defined.is_empty() { - warnings.push(format!( - "{path}: no indexed definitions; structural coverage unknown." - )); - continue; - } for entity in &public { if !matches!(entity.kind.as_str(), "function" | "method") { continue; } - // Resolve stored IDs from indexed DEFINES edges. Do not fabricate an - // entity ID at a historical line that may no longer exist in the KG. - let matches: Vec<_> = graph - .nodes + let matches: Vec<_> = index + .entities .iter() - .filter(|node| defined.contains(node.id.as_str()) && node.label == entity.name) + .filter(|candidate| candidate.file == path && candidate.name == entity.name) .collect(); if matches.len() != 1 { warnings.push(format!( - "{path}::{}: indexed identity absent/ambiguous.", + "{path}::{}: definition absent/ambiguous in the selected view.", entity.name )); continue; } - match repo.vault_kg_neighbors(&matches[0].id, 2) { - Ok(callers) => collect_callers( - &callers, - &matches[0].id, - &path, - &mut files, - &mut links, - &mut warnings, - ), - Err(_) => warnings.push(format!("{path}::{}: callers unavailable.", entity.name)), - } + collect_callers( + index, + &format!("{path}#{}", entity.name), + &path, + &mut files, + &mut links, + &mut warnings, + ); } // Importers are genuine incoming file dependencies. Include them for all // public API changes; do not hide type importers when callers also exist. - for edge in &graph.edges { - if edge.to_id == file_id && edge.kind.eq_ignore_ascii_case("INCLUDES") { - if let Some(importer) = edge.from_id.strip_prefix("file:") { - add_link(importer, &path, "INCLUDES", &mut files, &mut links); - } + for (importer, target) in &index.file_deps { + if target == &path { + add_link(importer, &path, "includes", &mut files, &mut links); } } } @@ -174,7 +165,7 @@ pub(super) fn project(repo: &Repository, change: &Change, hash: &Hash) -> Change mode: if warnings.is_empty() { mode } else { "partial" }.into(), files: rows.into_iter().map(|(path, hop)| ChangeImpactFile { path, hop }).collect(), edges, warnings: warnings.into_iter().take(100).collect(), - context: "Recorded change sources + current repository KG; not a historical or view-isolated dependency snapshot.".into(), + context: format!("Recorded change sources + syntax visible in Atomic view '{view}'; not a historical dependency snapshot."), } } @@ -192,8 +183,14 @@ fn changed_entities( before: &str, after: &str, ) -> Option> { - let old = registry.extract(path, before); - let new = registry.extract(path, after); + let definitions = |entities: Vec| { + entities + .into_iter() + .filter(|entity| entity.kind != atomic_semantic::EntityKind::Import) + .collect::>() + }; + let old = definitions(registry.extract(path, before)); + let new = definitions(registry.extract(path, after)); if old.is_empty() && new.is_empty() { return None; } @@ -217,14 +214,21 @@ fn changed_entities( _ => {} } } - let mut entities = Vec::new(); + let mut entities: Vec = Vec::new(); for (candidates, lines) in [(old, old_lines), (new, new_lines)] { for entity in candidates { - if lines.range(entity.line..=entity.end_line).next().is_some() - && !entities - .iter() - .any(|e: &Entity| e.name == entity.name && e.kind == entity.kind) + if lines.range(entity.line..=entity.end_line).next().is_none() { + continue; + } + if let Some(existing) = entities + .iter_mut() + .find(|e| e.name == entity.name && e.kind == entity.kind) { + // Becoming public is an API change just as becoming private is. + // Retain visibility from either recorded version, not whichever + // version happened to be visited first. + existing.exported |= entity.exported; + } else { entities.push(entity); } } @@ -247,19 +251,14 @@ fn add_link( } fn collect_callers( - graph: &KgSubgraph, + index: &SyntaxIndex, target: &str, path: &str, files: &mut BTreeMap, links: &mut BTreeSet<(String, String, String)>, warnings: &mut Vec, ) { - let callers: BTreeSet<_> = graph - .edges - .iter() - .filter(|e| e.to_id == target && e.kind.eq_ignore_ascii_case("CALLS")) - .map(|e| e.from_id.as_str()) - .collect(); + let callers = index.callers(target); if callers.len() > MAX_CALLERS { warnings.push(format!( "{path}: common/high-fanout entity skipped (over {MAX_CALLERS} callers)." @@ -267,18 +266,7 @@ fn collect_callers( return; } for caller in callers { - let definitions: Vec<_> = graph - .edges - .iter() - .filter(|e| e.to_id == caller && e.kind.eq_ignore_ascii_case("DEFINES")) - .filter_map(|e| e.from_id.strip_prefix("file:")) - .collect(); - if definitions.is_empty() { - warnings.push(format!("{path}: caller definition is not indexed.")); - } - for source in definitions { - add_link(source, path, "CALLS", files, links); - } + add_link(&caller.path, path, "calls", files, links); } } @@ -299,25 +287,41 @@ mod tests { .iter() .any(|e| e.name == "removed" && public_api(e))); assert!(changed_entities(&mut parser, "a.json", "{}", "{\"a\":1}").is_none()); + assert!( + changed_entities(&mut parser, "a.ts", "import './old';", "import './new';").is_none() + ); + } + #[test] + fn visibility_changes_use_public_api_from_either_recorded_version() { + let mut parser = ParserRegistry::new(); + let private = "function changed() { return 1; }\n"; + let public = "export function changed() { return 1; }\n"; + for (before, after) in [(private, public), (public, private)] { + let entities = changed_entities(&mut parser, "a.ts", before, after).unwrap(); + assert_eq!(entities.len(), 1); + assert!(public_api(&entities[0])); + } } + #[test] fn callers_are_directed_and_direct_files_are_not_counted_twice() { - use atomic_core::pristine::vault::KgEdge; - let graph = KgSubgraph { - nodes: vec![], - edges: vec![ - KgEdge::new("entity:caller", "entity:target", "CALLS"), - KgEdge::new("file:tests/caller.ts", "entity:caller", "DEFINES"), - KgEdge::new("entity:target", "entity:outgoing", "CALLS"), - KgEdge::new("file:wrong.ts", "entity:outgoing", "DEFINES"), - ], - }; + let index = SyntaxIndex::from_sources(&[ + ( + "src/a.ts".into(), + "export function target() { outgoing(); }".into(), + ), + ( + "tests/caller.ts".into(), + "function caller() { target(); }".into(), + ), + ("wrong.ts".into(), "function outgoing() {}".into()), + ]); let mut files = BTreeMap::from([("src/a.ts".into(), 1)]); let mut links = BTreeSet::new(); let mut warnings = vec![]; collect_callers( - &graph, - "entity:target", + &index, + "src/a.ts#target", "src/a.ts", &mut files, &mut links, @@ -328,7 +332,7 @@ mod tests { add_link( "src/a.ts", "tests/caller.ts", - "INCLUDES", + "includes", &mut files, &mut links, ); @@ -338,19 +342,20 @@ mod tests { #[test] fn common_callable_names_produce_a_partial_coverage_warning_not_arbitrary_callers() { - use atomic_core::pristine::vault::KgEdge; - let graph = KgSubgraph { - nodes: vec![], - edges: (0..31) - .map(|i| KgEdge::new(format!("caller:{i}"), "target", "CALLS")) - .collect(), - }; + let mut sources = vec![("src/a.ts".into(), "export function target() {}".into())]; + sources.extend((0..31).map(|i| { + ( + format!("caller{i}.ts"), + format!("function caller{i}() {{ target(); }}"), + ) + })); + let index = SyntaxIndex::from_sources(&sources); let mut files = BTreeMap::from([("src/a.ts".into(), 1)]); let mut links = BTreeSet::new(); let mut warnings = vec![]; collect_callers( - &graph, - "target", + &index, + "src/a.ts#target", "src/a.ts", &mut files, &mut links, diff --git a/libatomic/src/daemon/services_query.rs b/libatomic/src/daemon/services_query.rs index 02312086..278b8fe1 100644 --- a/libatomic/src/daemon/services_query.rs +++ b/libatomic/src/daemon/services_query.rs @@ -447,8 +447,11 @@ pub async fn get_change_impl( paths .into_iter() .map(|path| { - let before_result = repo.get_file_content_before_change(&path, &hash); - let after_result = repo.get_file_content_after_change(&path, &hash); + let view_name = view.as_deref().unwrap_or_else(|| repo.current_view()); + let before_result = + repo.get_file_content_before_change_on_view(&path, &hash, view_name); + let after_result = + repo.get_file_content_after_change_on_view(&path, &hash, view_name); let before_exists = before_result.as_ref().ok().map(|content| content.is_some()); let after_exists = after_result.as_ref().ok().map(|content| content.is_some()); @@ -473,8 +476,14 @@ pub async fn get_change_impl( } else { Vec::new() }; - let blast_radius = - wants_blast_radius.then(|| super::change_impact::project(&repo, &change, &hash)); + let blast_radius = wants_blast_radius.then(|| { + super::change_impact::project( + &repo, + &change, + &hash, + view.as_deref().unwrap_or_else(|| repo.current_view()), + ) + }); Ok::<_, Status>(( change, hash, diff --git a/libatomic/tests/local_service_test.rs b/libatomic/tests/local_service_test.rs index e1aafb9c..9bd384c3 100644 --- a/libatomic/tests/local_service_test.rs +++ b/libatomic/tests/local_service_test.rs @@ -494,12 +494,8 @@ fn embedded_descriptor_contains_the_status_contract() { } #[tokio::test] -async fn change_blast_radius_is_readonly_change_local_and_uses_incoming_structural_edges() { +async fn change_blast_radius_is_readonly_change_local_and_uses_view_syntax() { use atomic_core::change::ChangeHeader; - use atomic_core::pristine::{ - vault::{KgEdge, KgNode}, - KgMutTxnT, MutTxnT, - }; use atomic_repository::RecordOptions; let dir = tempfile::tempdir().unwrap(); let repo = Repository::init(dir.path()).unwrap(); @@ -512,6 +508,15 @@ async fn change_blast_radius_is_readonly_change_local_and_uses_incoming_structur ) .unwrap(); repo.add(wc, "src/api.ts", Default::default()).unwrap(); + std::fs::create_dir(dir.path().join("tests")).unwrap(); + for (path, source) in [ + ("tests/api.ts", "function test() { changed(); }\n"), + ("src/ui.ts", "import { changed } from './api';\n"), + ("unrelated.ts", "function unrelated() { return 1; }\n"), + ] { + std::fs::write(dir.path().join(path), source).unwrap(); + repo.add(wc, path, Default::default()).unwrap(); + } let options = || { RecordOptions::new() .with_all(true) @@ -529,33 +534,11 @@ async fn change_blast_radius_is_readonly_change_local_and_uses_incoming_structur .record(wc, ChangeHeader::new("public change"), options()) .unwrap() .hash(); - let target = "entity:src/api.ts:changed:1"; - let caller = "entity:tests/api.ts:test:1"; - let mut txn = repo.pristine().write_txn().unwrap(); - for (id, kind, label) in [ - ("file:src/api.ts", "file", "api.ts"), - ("file:tests/api.ts", "file", "api.ts"), - ("file:src/ui.ts", "file", "ui.ts"), - (target, "entity", "changed"), - (caller, "entity", "test"), - ] { - txn.upsert_kg_node(&KgNode::new(id, kind, label, "ast")) - .unwrap(); - } - for (from, to, kind) in [ - ("file:src/api.ts", target, "DEFINES"), - ("file:tests/api.ts", caller, "DEFINES"), - (caller, target, "CALLS"), - ("file:src/ui.ts", "file:src/api.ts", "INCLUDES"), - (target, caller, "CALLS"), - ("file:unrelated.ts", "file:src/api.ts", "CO_CHANGED"), - ] { - txn.upsert_kg_edge(&KgEdge::new(from, to, kind)).unwrap(); - } - txn.commit().unwrap(); // The working copy is now unrelated to the recorded change. Analysis must // still use the exact before/after change sources rather than this buffer. std::fs::write(&file, "function totallyDifferent() {}\n").unwrap(); + std::fs::write(dir.path().join("tests/api.ts"), "function test() {}\n").unwrap(); + std::fs::write(dir.path().join("src/ui.ts"), "").unwrap(); drop(repo); let service = LocalService::new(); let reference = resolve(&service, dir.path()).await; @@ -604,6 +587,102 @@ async fn change_blast_radius_is_readonly_change_local_and_uses_incoming_structur ); } +#[tokio::test] +async fn change_impact_and_recorded_bytes_use_the_selected_view_not_the_working_copy_view() { + use atomic_core::change::ChangeHeader; + use atomic_repository::RecordOptions; + let dir = tempfile::tempdir().unwrap(); + let mut repo = Repository::init(dir.path()).unwrap(); + let wc = repo.require_working_copy_id().unwrap(); + let selected_view = repo.current_view().to_owned(); + let options = || { + RecordOptions::new() + .with_all(true) + .save_to_store(true) + .apply_after_record(true) + }; + for (path, source) in [ + ("api.ts", "function changed() { return 1; }\n"), + ("caller.ts", "function caller() { changed(); }\n"), + ] { + std::fs::write(dir.path().join(path), source).unwrap(); + repo.add(wc, path, Default::default()).unwrap(); + } + repo.record(wc, ChangeHeader::new("selected baseline"), options()) + .unwrap(); + std::fs::write( + dir.path().join("api.ts"), + "export function changed() { return 2; }\n", + ) + .unwrap(); + let hash = *repo + .record(wc, ChangeHeader::new("selected public change"), options()) + .unwrap() + .hash(); + repo.create_shared_view("unrelated").unwrap(); + repo.switch_view(wc, "unrelated").unwrap(); + std::fs::write( + dir.path().join("unrelated.ts"), + "function unrelated() { changed(); }\n", + ) + .unwrap(); + repo.add(wc, "unrelated.ts", Default::default()).unwrap(); + repo.record(wc, ChangeHeader::new("different view caller"), options()) + .unwrap(); + drop(repo); + + let service = LocalService::new(); + let reference = resolve(&service, dir.path()).await; + let database = dir.path().join(".atomic/atomic.redb"); + let before = std::fs::read(&database).unwrap(); + let response: proto::GetChangeResponse = invoke( + &service, + "/atomic.RepositoryQueryService/GetChange", + &proto::GetChangeRequest { + repository: Some(reference), + r#ref: Some(proto::ChangeRef { + kind: Some(proto::change_ref::Kind::Hash(proto::Hash { + value: hash.0.to_vec(), + ..Default::default() + })), + }), + view: Some(proto::ViewRef { + name: Some(selected_view.clone()), + ..Default::default() + }), + includes: Some(proto::ChangeIncludes { + blast_radius: true, + ..Default::default() + }), + include_file_contents: true, + }, + ) + .await; + let impact = response.blast_radius.unwrap(); + assert_eq!(impact.mode, "structural", "{:?}", impact.warnings); + assert!(impact.context.contains(&selected_view)); + assert_eq!(impact.files.len(), 2); + assert_eq!(impact.edges.len(), 1); + assert_eq!(impact.edges[0].from_path, "caller.ts"); + assert_eq!(impact.edges[0].to_path, "api.ts"); + let file = response + .file_contents + .iter() + .find(|file| file.path == "api.ts") + .unwrap(); + assert_eq!( + file.before.as_deref(), + Some(&b"function changed() { return 1; }\n"[..]) + ); + assert_eq!( + file.after.as_deref(), + Some(&b"export function changed() { return 2; }\n"[..]) + ); + assert_eq!(before, std::fs::read(database).unwrap()); + let repo = Repository::open_readonly(dir.path()).unwrap(); + assert_eq!(repo.current_view(), "unrelated"); +} + #[tokio::test] async fn private_blast_radius_requires_successfully_analyzed_recorded_sources() { use atomic_core::change::ChangeHeader; diff --git a/tests/harness/14_vault_kg.sh b/tests/harness/14_vault_kg.sh index 0b2255be..667aff03 100755 --- a/tests/harness/14_vault_kg.sh +++ b/tests/harness/14_vault_kg.sh @@ -53,6 +53,42 @@ assert_ran() { fi } +# Discover a canonical view IRI from the query result, then require a real +# incoming membership edge. An empty neighbors response is not a passing graph. +assert_view_membership() { + local desc="$1" + local view_id + run_cmd atomic vault query search main --kind 'https://atomic.dev/ns/workflow#View' --json + if [[ $RC -ne 0 ]]; then + _fail "$desc" "view search exited $RC: $OUT" + return + fi + if ! view_id="$(echo "$OUT" | python3 -c ' +import json, sys +nodes = json.load(sys.stdin) +print(next(node["id"] for node in nodes if node["label"] == "main" and "https://atomic.dev/ns/workflow#View" in node["types"])) +')"; then + _fail "$desc" "canonical main view missing: $OUT" + return + fi + run_cmd atomic vault query neighbors "$view_id" --json + if [[ $RC -ne 0 ]]; then + _fail "$desc" "neighbors exited $RC: $OUT" + elif echo "$OUT" | python3 -c ' +import json, sys +graph = json.load(sys.stdin) +view_id = sys.argv[1] +ns = "https://atomic.dev/ns/workflow#" +nodes = {node["id"]: node for node in graph["nodes"]} +assert ns + "View" in nodes[view_id]["types"] +assert any(edge["to_id"] == view_id and edge["kind"] == ns + "directlyOnView" and ns + "Change" in nodes[edge["from_id"]]["types"] for edge in graph["edges"]) +' "$view_id"; then + _pass "$desc" + else + _fail "$desc" "canonical view/change membership missing: $OUT" + fi +} + # ════════════════════════════════════════════════════════════════════════════ # Section 1: Vault Initialization # ════════════════════════════════════════════════════════════════════════════ @@ -197,26 +233,23 @@ assert_ran "KG search finds recorded content" "change:|file:src/main\.rs" begin_section "Vault: KG Enrichment" make_temp_repo "vault-kg" -init_repo --vault +init_repo --vault --view main # Create some tracked files create_file "src/auth.rs" "pub fn authenticate() { }\npub fn verify() { }" create_file "src/main.rs" "fn main() { authenticate(); }" add_files "src/auth.rs" "src/main.rs" -record_change "Add auth module" >/dev/null 2>&1 || true +record_change "Add auth module" >/dev/null 2>&1 # Run explicit KG enrichment — reports what was enriched run_cmd atomic vault query enrich -assert_ran "KG enrich from VCS data" "Enriched: .*views.*files.*changes" +assert_ran "KG enrich from VCS data" 'Enriched knowledge graph: [1-9][0-9]* item\(s\) processed\.' -# Search for entities +# Search for workflow file resources (AST entities are not stored in the KG). run_cmd atomic vault query search "auth" --json assert_ran "KG search finds 'auth' content" "auth" -# Query neighbors of a view node — returns a node/edge graph -project_urn="$(atomic repo identity show --json | python3 -c 'import json,sys; print(json.load(sys.stdin)["urn"])')" -run_cmd atomic vault query neighbors "${project_urn}:view:dev" --json -assert_ran "KG neighbors query for view node" '"nodes"' +assert_view_membership "KG neighbors contain canonical view membership" # Reindex run_cmd atomic vault query reindex @@ -272,7 +305,7 @@ else else cd "$GIT_KG_DIR" - run_cmd atomic init --vault + run_cmd atomic init --vault --view main if [[ $RC -eq 0 ]]; then _pass "init --vault in git checkout" else @@ -288,16 +321,13 @@ else # Enrich KG from imported data run_cmd atomic vault query enrich - assert_ran "KG enrichment after git import" "Enriched: .*views.*files.*changes" + assert_ran "KG enrichment after git import" 'Enriched knowledge graph: [1-9][0-9]* item\(s\) processed\.' # Search should find content from the imported repo run_cmd atomic vault query search "uuid" --json assert_ran "KG search finds imported content" "uuid" - # Neighbors query for the imported default view - project_urn="$(atomic repo identity show --json | python3 -c 'import json,sys; print(json.load(sys.stdin)["urn"])')" - run_cmd atomic vault query neighbors "${project_urn}:view:main" --json - assert_ran "KG neighbors query after import" '"nodes"' + assert_view_membership "KG neighbors contain imported view membership" # Reindex vault entries run_cmd atomic vault query reindex From 3089d1a1c44032f4927ded778dfa10b66ca61ac0 Mon Sep 17 00:00:00 2001 From: Lee Faus Date: Fri, 23 Oct 2026 15:57:07 -0400 Subject: [PATCH 15/19] detailed graph layout --- Cargo.toml | 2 +- README.md | 26 + .../src/turn/orchestrator/provenance.rs | 14 +- atomic-agent/src/turn/orchestrator/turn.rs | 4 + atomic-agent/src/turn/session.rs | 218 +++++ atomic-canonical/ns/ctx.jsonld | 1 + atomic-canonical/src/gate.rs | 16 + atomic-canonical/src/lift.rs | 19 +- atomic-canonical/src/node.rs | 12 +- atomic-canonical/src/render.rs | 33 +- atomic-canonical/src/vocab.rs | 23 +- .../tests/implementation_declaration.rs | 62 ++ atomic-cli/src/commands/intent/graph.rs | 284 ++++++ atomic-cli/src/commands/intent/mod.rs | 7 + atomic-cli/src/commands/intent/update.rs | 68 +- .../src/commands/query/graph_inspector.css | 40 + .../src/commands/query/graph_inspector.js | 195 ++++ atomic-cli/src/commands/query/mod.rs | 248 ++++- atomic-cli/src/commands/rpc.rs | 77 ++ .../tests/fixtures/graph_canvas_test.js | 82 ++ .../tests/fixtures/graph_inspector_test.html | 98 ++ atomic-cli/tests/intent_graph_test.rs | 453 +++++++++ atomic-cli/tests/intent_turn_binding_test.rs | 288 ++++++ .../ontology/shapes/projection-1.0.0.ttl | 15 +- atomic-core/ontology/workflow-1.0.0.ttl | 7 +- atomic-core/src/kg/mod.rs | 2 +- atomic-core/src/kg/ontology.rs | 5 +- .../src/pristine/txn/write/session_kg.rs | 6 +- atomic-core/tests/kg_view_ancestry.rs | 165 ++++ atomic-repository/src/intent_graph.rs | 405 ++++++++ atomic-repository/src/lib.rs | 3 + atomic-repository/src/patch_evidence.rs | 219 +++++ atomic-repository/src/query_plan.rs | 37 + .../src/repository/kg_rebuild.rs | 2 + .../src/repository/vault_kg_enrich.rs | 86 +- .../src/repository/vault_projection.rs | 17 + .../src/repository/vault_triples.rs | 9 + atomic-repository/src/workflow_usage.rs | 891 ++++++++++++++++++ .../tests/implementation_declaration.rs | 65 ++ atomic-repository/tests/intent_graph.rs | 295 ++++++ .../tests/kg_change_availability.rs | 127 +++ atomic-repository/tests/kg_recorded_impact.rs | 99 ++ atomic-repository/tests/patch_evidence.rs | 359 +++++++ atomic-repository/tests/workflow_usage.rs | 118 +++ atomic-shacl/src/adapter.rs | 10 + atomic-shacl/tests/change_availability.rs | 33 + docs/intent-graph.md | 410 ++++---- libatomic/src/daemon/services_agent.rs | 4 +- libatomic/src/daemon/services_query.rs | 4 +- rust-toolchain.toml | 4 + 50 files changed, 5410 insertions(+), 257 deletions(-) create mode 100644 atomic-canonical/tests/implementation_declaration.rs create mode 100644 atomic-cli/src/commands/intent/graph.rs create mode 100644 atomic-cli/src/commands/query/graph_inspector.css create mode 100644 atomic-cli/src/commands/query/graph_inspector.js create mode 100644 atomic-cli/tests/fixtures/graph_canvas_test.js create mode 100644 atomic-cli/tests/fixtures/graph_inspector_test.html create mode 100644 atomic-cli/tests/intent_graph_test.rs create mode 100644 atomic-cli/tests/intent_turn_binding_test.rs create mode 100644 atomic-core/tests/kg_view_ancestry.rs create mode 100644 atomic-repository/src/intent_graph.rs create mode 100644 atomic-repository/src/patch_evidence.rs create mode 100644 atomic-repository/src/workflow_usage.rs create mode 100644 atomic-repository/tests/implementation_declaration.rs create mode 100644 atomic-repository/tests/intent_graph.rs create mode 100644 atomic-repository/tests/kg_change_availability.rs create mode 100644 atomic-repository/tests/kg_recorded_impact.rs create mode 100644 atomic-repository/tests/patch_evidence.rs create mode 100644 atomic-repository/tests/workflow_usage.rs create mode 100644 atomic-shacl/tests/change_availability.rs create mode 100644 rust-toolchain.toml diff --git a/Cargo.toml b/Cargo.toml index f6d43f7d..8cf50050 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -25,7 +25,7 @@ authors = ["Atomic Contributors"] license = "Apache-2.0" repository = "https://github.com/atomicdotdev/atomic" homepage = "https://atomic.dev" -rust-version = "1.90" +rust-version = "1.98" [workspace.dependencies] # Core diff --git a/README.md b/README.md index 97d8ecf8..af55b426 100644 --- a/README.md +++ b/README.md @@ -514,6 +514,32 @@ Full documentation at **[docs.atomic.dev](https://docs.atomic.dev/)**. If replacing an older binary, first follow [the legacy upgrade steps](#upgrading-from-legacy-database-owners-0191-and-earlier). +Source builds require **Rust 1.98 or newer** (including the SHACL dependencies). +`rust-toolchain.toml` pins the supported development toolchain. Use rustup's +`cargo`/`rustc` shims, not an older standalone Homebrew Rust installation: + +```sh +rustup toolchain install 1.98.0 --profile minimal --component rustfmt --component clippy +export PATH="$HOME/.cargo/bin:$PATH" +rustup show active-toolchain +rustc --version +cargo build -r +``` + +If Cargo still reports `rustc 1.97.1`, inspect `type -a cargo rustc`: a direct +`/opt/homebrew/bin/cargo` or `rustc` ahead of `~/.cargo/bin` bypasses rustup and +the repository pin. Put the export above **after** Homebrew's PATH setup in your +shell configuration. Do not lower `rust-version` or use `--ignore-rust-version`: +the dependency itself requires the newer compiler. For a one-command build +without changing the shell configuration: + +```sh +PATH="$HOME/.cargo/bin:$PATH" cargo build -r +``` + +Also check for explicit `RUSTC`, `RUSTDOC`, `RUSTUP_TOOLCHAIN`, or Cargo config +overrides if the shims still select the wrong compiler. + ```bash # Build all crates cargo build --release diff --git a/atomic-agent/src/turn/orchestrator/provenance.rs b/atomic-agent/src/turn/orchestrator/provenance.rs index f0b73181..d22967aa 100644 --- a/atomic-agent/src/turn/orchestrator/provenance.rs +++ b/atomic-agent/src/turn/orchestrator/provenance.rs @@ -925,10 +925,16 @@ impl TurnOrchestrator { agent_vendor: session.agent_vendor.clone(), change_hashes: change_hashes.to_vec(), previous_provenance, - plan_id: session - .managed_run - .as_ref() - .and_then(|run| run.work_item_id.clone()), + plan_id: self + .session_store + .turn_intent_binding(session_id, session.turn_count.max(1))? + .map(|binding| binding.intent_uid) + .or_else(|| { + session + .managed_run + .as_ref() + .and_then(|run| run.work_item_id.clone()) + }), ledger_turn_number, }; let reservation = sink diff --git a/atomic-agent/src/turn/orchestrator/turn.rs b/atomic-agent/src/turn/orchestrator/turn.rs index 286c732e..aeca2c98 100644 --- a/atomic-agent/src/turn/orchestrator/turn.rs +++ b/atomic-agent/src/turn/orchestrator/turn.rs @@ -297,6 +297,8 @@ impl TurnOrchestrator { // The load_or_create + Idle consultation below handles it. } let turn_number = session.end_turn(); + self.session_store + .freeze_turn_intent(session_id, turn_number)?; let boundary_end = atomic_repository::Repository::open_readonly(&self.repo_root) .ok() .and_then(|repo| { @@ -466,6 +468,8 @@ impl TurnOrchestrator { // OpenCode writes no transcript file of its own. self.enrich_opencode_turn(&mut session, &mut event)?; let pending_turn_number = session.turn_count.saturating_add(1); + self.session_store + .freeze_turn_intent(session_id, pending_turn_number)?; self.commit_turn_completion_events(&session, &event, pending_turn_number)?; // A read-only turn still owns a journal turn number and immutable diff --git a/atomic-agent/src/turn/session.rs b/atomic-agent/src/turn/session.rs index 45e5b1cd..2b719df6 100644 --- a/atomic-agent/src/turn/session.rs +++ b/atomic-agent/src/turn/session.rs @@ -760,7 +760,181 @@ pub struct SessionStore { sessions_dir: PathBuf, } +/// An explicit work-start declaration for one hook turn (1-indexed). +/// Kept separately from mutable session JSON so later hook saves cannot erase it. +#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] +pub struct TurnIntentBinding { + pub session_id: String, + pub turn_number: u32, + pub intent_uid: String, +} + impl SessionStore { + /// Bind the currently active turn before recording. Repeating the same + /// declaration is idempotent; replacing it or binding a completed turn fails. + /// `expected_turn` lets integrations reject delayed commands from an old turn. + pub fn bind_turn_intent( + &self, + session_id: &str, + expected_turn: Option, + intent_uid: &str, + ) -> AgentResult { + use fs2::FileExt; + use std::io::Write; + + validate_session_id(session_id)?; + let failure = |reason: String| AgentError::SessionSaveFailed { + session_id: session_id.to_string(), + reason, + }; + let dir = self.sessions_dir.join(session_id); + std::fs::create_dir_all(&dir)?; + // The recorder holds this lock from session load through publication. + let lock = std::fs::OpenOptions::new() + .create(true) + .write(true) + .truncate(false) + .open(dir.join("turn-end.lock"))?; + lock.try_lock_exclusive().map_err(|error| { + failure(format!( + "cannot bind intent while turn is recording: {error}" + )) + })?; + let session = self + .load(session_id)? + .ok_or_else(|| failure("unknown session".into()))?; + if session.session_id != session_id + || !session.is_turn_active() + || session.is_ended() + || session.incomplete().is_some() + { + return Err(failure( + "intent binding requires an active, complete session turn".into(), + )); + } + let turn_number = session.turn_count.saturating_add(1); + if dir + .join(format!("intent-{turn_number}.frozen")) + .try_exists()? + { + return Err(failure( + "turn recording has already started; governing intent is frozen".into(), + )); + } + if expected_turn.is_some_and(|expected| expected != turn_number) { + return Err(failure(format!( + "stale intent binding: active turn is {turn_number}" + ))); + } + if intent_uid.trim().is_empty() { + return Err(failure("intent UID must not be empty".into())); + } + // Managed work items remain authoritative. Non-UID managed references + // retain their existing checkpoint fallback rather than being replaced. + if let Some(work_item) = session + .managed_run + .as_ref() + .and_then(|run| run.work_item_id.as_deref()) + { + if work_item != intent_uid { + return Err(failure( + "explicit intent conflicts with the managed work item".into(), + )); + } + } + let binding = TurnIntentBinding { + session_id: session_id.to_string(), + turn_number, + intent_uid: intent_uid.to_string(), + }; + if let Some(existing) = self.turn_intent_binding(session_id, turn_number)? { + return if existing == binding { + Ok(existing) + } else { + Err(failure( + "turn already has a different governing intent".into(), + )) + }; + } + let path = dir.join(format!("intent-{turn_number}.json")); + let tmp = dir.join(format!( + "intent-{turn_number}-{}-{}.tmp", + std::process::id(), + SESSION_TMP_SEQUENCE.fetch_add(1, Ordering::Relaxed) + )); + let result = (|| -> AgentResult<()> { + let mut file = std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .open(&tmp)?; + file.write_all( + &serde_json::to_vec(&binding).map_err(|error| failure(error.to_string()))?, + )?; + file.sync_all()?; + std::fs::rename(&tmp, &path)?; + #[cfg(unix)] + std::fs::File::open(&dir)?.sync_all()?; + Ok(()) + })(); + if result.is_err() { + let _ = std::fs::remove_file(&tmp); + } + result?; + Ok(binding) + } + + /// Read only this turn's declaration; never inherit governance from an + /// earlier turn or infer it from files. Corrupt declarations fail closed. + pub fn turn_intent_binding( + &self, + session_id: &str, + turn_number: u32, + ) -> AgentResult> { + validate_session_id(session_id)?; + let path = self + .sessions_dir + .join(session_id) + .join(format!("intent-{turn_number}.json")); + let bytes = match std::fs::read(path) { + Ok(bytes) => bytes, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None), + Err(error) => return Err(error.into()), + }; + let failure = |reason: String| AgentError::SessionLoadFailed { + session_id: session_id.to_string(), + reason, + }; + let binding: TurnIntentBinding = + serde_json::from_slice(&bytes).map_err(|error| failure(error.to_string()))?; + if binding.session_id != session_id + || binding.turn_number != turn_number + || binding.intent_uid.trim().is_empty() + { + return Err(failure("invalid turn intent binding".into())); + } + Ok(Some(binding)) + } + + /// Seal governance before any recording attempt, including one that fails + /// before publishing a checkpoint. The caller holds `turn-end.lock`. + pub(crate) fn freeze_turn_intent(&self, session_id: &str, turn_number: u32) -> AgentResult<()> { + validate_session_id(session_id)?; + // Validate existing evidence before recording any content. + self.turn_intent_binding(session_id, turn_number)?; + let dir = self.sessions_dir.join(session_id); + std::fs::create_dir_all(&dir)?; + let path = dir.join(format!("intent-{turn_number}.frozen")); + let file = std::fs::OpenOptions::new() + .write(true) + .create(true) + .truncate(false) + .open(path)?; + file.sync_all()?; + #[cfg(unix)] + std::fs::File::open(dir)?.sync_all()?; + Ok(()) + } + /// Create a new session store at the given directory path. /// /// Creates the directory if it doesn't exist. @@ -1295,6 +1469,50 @@ mod tests { (dir, store) } + #[test] + fn frozen_turn_rejects_late_binding_even_if_recording_failed() { + let (_dir, store) = make_store(); + let mut session = make_session(); + session.begin_turn(); + session.phase = Phase::Active; + store.save(&session).unwrap(); + store.freeze_turn_intent(&session.session_id, 1).unwrap(); + assert!(store + .bind_turn_intent(&session.session_id, Some(1), "01ARZ3NDEKTSV4RRFFQ69G5FAV") + .is_err()); + assert!(store + .turn_intent_binding(&session.session_id, 1) + .unwrap() + .is_none()); + } + + #[test] + fn binding_survives_session_saves_and_rejects_corrupt_identity() { + let (_dir, store) = make_store(); + let mut session = make_session(); + session.begin_turn(); + session.phase = Phase::Active; + store.save(&session).unwrap(); + let binding = store + .bind_turn_intent(&session.session_id, None, "01ARZ3NDEKTSV4RRFFQ69G5FAV") + .unwrap(); + store.save(&session).unwrap(); + assert_eq!( + store.turn_intent_binding(&session.session_id, 1).unwrap(), + Some(binding) + ); + let path = store + .sessions_dir + .join(&session.session_id) + .join("intent-1.json"); + std::fs::write( + path, + br#"{"session_id":"another-session","turn_number":1,"intent_uid":"intent"}"#, + ) + .unwrap(); + assert!(store.freeze_turn_intent(&session.session_id, 1).is_err()); + } + // AgentSession construction #[test] diff --git a/atomic-canonical/ns/ctx.jsonld b/atomic-canonical/ns/ctx.jsonld index f7c89417..3c6e1848 100644 --- a/atomic-canonical/ns/ctx.jsonld +++ b/atomic-canonical/ns/ctx.jsonld @@ -45,6 +45,7 @@ "to": { "@id": "atom:to", "@type": "@id" }, "edge": "atom:edge", + "declarationEvidence": "http://purl.org/dc/terms/description", "memoryKind": "atom:memoryKind", "about": { "@id": "atom:about", "@type": "@id" }, diff --git a/atomic-canonical/src/gate.rs b/atomic-canonical/src/gate.rs index bfb71ae5..bc533896 100644 --- a/atomic-canonical/src/gate.rs +++ b/atomic-canonical/src/gate.rs @@ -132,6 +132,21 @@ pub fn validate_intent(node: &CanonicalNode) -> ValidationReport { } // attributedTo must be present (sh:class prov:Agent deferred — see M0 notes). + for reference in &node.depends_on { + if reference.edge == "declaresImplementation" + && (!vocab::is_canonical_change_iri(&reference.to) + || reference + .declaration_evidence + .as_deref() + .is_none_or(|value| value.trim().is_empty())) + { + out.push(Violation { + focus_node: focus.clone(), shape: "ImplementationDeclarationShape".into(), + path: Some("dependsOn".into()), + message: "implementation declarations require a canonical full-hash change IRI and nonempty declarationEvidence".into(), + }); + } + } if node.attributed_to.as_deref().unwrap_or("").is_empty() { out.push(Violation { focus_node: focus.clone(), @@ -521,6 +536,7 @@ mod tests { /// A `:::ref{edge=reviews}` targeting another intent. fn reviews_ref(target: &str) -> Ref { Ref { + declaration_evidence: None, type_: None, id: None, to: target.to_string(), diff --git a/atomic-canonical/src/lift.rs b/atomic-canonical/src/lift.rs index 40f19cf9..6986755c 100644 --- a/atomic-canonical/src/lift.rs +++ b/atomic-canonical/src/lift.rs @@ -282,7 +282,8 @@ fn lift_constraint(d: &Directive, id_base: &str, n: usize) -> Constraint { /// Lift a `:::ref{to= edge=}` leaf into a typed reference edge. Both `to` and /// `edge` are required (a ref with no target or no edge type is a lift error), /// and `edge` must be a known *ref* edge: a dependency edge (`depends`/ -/// `blockedBy`) or a remediation edge (`remediates`). Edges like +/// `blockedBy`), review/remediation edge, or an evidence-bearing explicit +/// implementation declaration. Edges like /// `verifiedBy`/`about` are rejected even though they are valid edges elsewhere. fn lift_ref(d: &Directive) -> Result { let to = d @@ -295,16 +296,28 @@ fn lift_ref(d: &Directive) -> Result { .to_string(); if !vocab::is_known_ref_edge(&edge) { return Err(CanonicalError::Lift(format!( - "':::ref' edge '{edge}' is not a valid ref edge (dependency {:?} or remediation {:?})", + "':::ref' edge '{edge}' is not a valid ref edge (dependency {:?}, remediation {:?}, review {:?}, or implementation {:?})", vocab::DEPENDENCY_EDGES, - vocab::REMEDIATION_EDGES + vocab::REMEDIATION_EDGES, + vocab::REVIEW_EDGES, + vocab::IMPLEMENTATION_EDGES ))); } + let declaration_evidence = d.attr("declarationEvidence").map(str::to_owned); + if edge == "declaresImplementation" + && (!vocab::is_canonical_change_iri(&to) + || declaration_evidence + .as_deref() + .is_none_or(|e| e.trim().is_empty())) + { + return Err(CanonicalError::Lift("implementation declarations require a canonical full-hash change IRI and nonempty declarationEvidence".into())); + } Ok(Ref { type_: d.id.as_ref().map(|_| NodeType::Ref.as_str().to_string()), id: d.id.as_ref().map(|id| as_urn("ref", id)), to, edge, + declaration_evidence, }) } diff --git a/atomic-canonical/src/node.rs b/atomic-canonical/src/node.rs index 8ac55ce0..d545bf4e 100644 --- a/atomic-canonical/src/node.rs +++ b/atomic-canonical/src/node.rs @@ -110,7 +110,8 @@ pub struct Constraint { pub text: String, } -/// A typed dependency edge leaf, lifted from a `:::ref{to= edge=}` directive. +/// A typed reference leaf, lifted from a `::ref{to= edge=}` directive. +/// Includes dependency, review, remediation and explicit implementation claims. /// `@type`/`@id` are omitted when absent (the doc's `:::ref` carries only /// `to`/`edge`), so no empty JSON-LD keys leak into the hash. #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] @@ -122,6 +123,13 @@ pub struct Ref { pub id: Option, pub to: String, pub edge: String, + /// Evidence for an explicit implementation declaration, not captured turn provenance. + #[serde( + rename = "declarationEvidence", + default, + skip_serializing_if = "Option::is_none" + )] + pub declaration_evidence: Option, } /// The acceptance criteria a task fulfills, in whichever JSON shape it was read. @@ -309,6 +317,8 @@ pub struct CanonicalNode { skip_serializing_if = "Vec::is_empty" )] pub has_constraint: Vec, + /// Historical wire name; inspect each reference's edge before treating it + /// as a dependency. Reviews and implementation declarations are not deps. #[serde(rename = "dependsOn", default, skip_serializing_if = "Vec::is_empty")] pub depends_on: Vec, diff --git a/atomic-canonical/src/render.rs b/atomic-canonical/src/render.rs index 1d7189ef..b178f596 100644 --- a/atomic-canonical/src/render.rs +++ b/atomic-canonical/src/render.rs @@ -123,12 +123,40 @@ fn render_cli(node: &CanonicalNode) -> String { } // Dependencies — each ref rendered as 'edge -> to'. - if !node.depends_on.is_empty() { + if node + .depends_on + .iter() + .any(|r| r.edge != "declaresImplementation") + { s.push_str("\nDependencies\n"); - for r in &node.depends_on { + for r in node + .depends_on + .iter() + .filter(|r| r.edge != "declaresImplementation") + { s.push_str(&format!(" {} -> {}\n", r.edge, r.to)); } } + if node + .depends_on + .iter() + .any(|r| r.edge == "declaresImplementation") + { + s.push_str("\nDeclared implementation (not captured turn provenance)\n"); + for r in node + .depends_on + .iter() + .filter(|r| r.edge == "declaresImplementation") + { + s.push_str(&format!( + " {}\n {}\n", + r.to, + r.declaration_evidence + .as_deref() + .unwrap_or("missing evidence") + )); + } + } // Provenance footer. s.push('\n'); @@ -255,6 +283,7 @@ mod tests { }, ], depends_on: vec![Ref { + declaration_evidence: None, type_: None, id: None, to: "urn:atomic:intent:xyz".to_string(), diff --git a/atomic-canonical/src/vocab.rs b/atomic-canonical/src/vocab.rs index 8076a04c..ac632610 100644 --- a/atomic-canonical/src/vocab.rs +++ b/atomic-canonical/src/vocab.rs @@ -190,6 +190,7 @@ pub const EDGE_NAMES: &[&str] = &[ "attributedTo", "remediates", "reviews", + "declaresImplementation", ]; /// Is this a directive name the system recognizes? Unknown ⇒ gate error. @@ -219,6 +220,22 @@ pub const REMEDIATION_EDGES: &[&str] = &["remediates"]; /// are kept out of [`DEPENDENCY_EDGES`] but folded into [`is_known_ref_edge`]. pub const REVIEW_EDGES: &[&str] = &["reviews"]; +/// Explicit intent-to-change claims are not dependency or generating-turn edges. +pub const IMPLEMENTATION_EDGES: &[&str] = &["declaresImplementation"]; + +/// Strict full-hash identity, independent of the repository-local node index. +pub fn is_canonical_change_iri(value: &str) -> bool { + let Some(hash) = value.strip_prefix("urn:atomic:change:blake3-256:") else { + return false; + }; + hash.len() == 52 + && data_encoding::BASE32_NOPAD + .decode(hash.as_bytes()) + .is_ok_and(|bytes| { + bytes.len() == 32 && data_encoding::BASE32_NOPAD.encode(&bytes) == hash + }) +} + /// Is this a typed edge name the system recognizes at all? pub fn is_known_edge(name: &str) -> bool { EDGE_NAMES.contains(&name) @@ -232,12 +249,14 @@ pub fn is_known_dependency_edge(name: &str) -> bool { } /// Is this an edge a `:::ref` leaf may declare? The closed set the lift -/// validates `edge=` against: the dependency subset plus the remediation and -/// review intent→intent edges. An edge outside this union is a lift error. +/// validates `edge=` against: dependencies, remediation/review intent links, +/// and explicit intent-to-change declarations. Only the dependency subset +/// participates in dependency traversal. An unknown edge is a lift error. pub fn is_known_ref_edge(name: &str) -> bool { is_known_dependency_edge(name) || REMEDIATION_EDGES.contains(&name) || REVIEW_EDGES.contains(&name) + || IMPLEMENTATION_EDGES.contains(&name) } /// Is this a valid intent status value? diff --git a/atomic-canonical/tests/implementation_declaration.rs b/atomic-canonical/tests/implementation_declaration.rs new file mode 100644 index 00000000..47c5885a --- /dev/null +++ b/atomic-canonical/tests/implementation_declaration.rs @@ -0,0 +1,62 @@ +use atomic_canonical::{lift::lift_intent, node::Ref, validate_intent, vocab}; +use serde_json::json; + +fn body(target: &str, evidence: &str) -> String { + format!(":::why\nRetain an honest explicit implementation claim.\n:::\n::ref{{to={target} edge=declaresImplementation declarationEvidence=\"{evidence}\"}}\n") +} +fn fm() -> serde_json::Map { + json!({"id":"TEST::alice::1","uid":"01ARZ3NDEKTSV4RRFFQ69G5FAV","title":"Repair claim","status":"in_progress"}).as_object().unwrap().clone() +} + +#[test] +fn declaration_is_signed_with_its_evidence_and_is_not_a_dependency() { + use atomic_identity::{identity::Identity, keypair::KeyPair}; + let target = format!("urn:atomic:change:blake3-256:{}", "A".repeat(52)); + let key = KeyPair::generate(); + let identity = Identity::new("alice", &key); + let signed = atomic_canonical::lift_and_attest( + &fm(), + &body( + &target, + "Operator repaired a failed child publication; original plan is unknown.", + ), + &identity, + &key, + ) + .unwrap(); + assert!(validate_intent(&signed).conforms); + assert!(!vocab::is_known_dependency_edge("declaresImplementation")); + let bytes = serde_json::to_vec(&signed).unwrap(); + let mut decoded = serde_json::from_slice::(&bytes).unwrap(); + atomic_canonical::verify(&decoded, &key.public).unwrap(); + decoded.depends_on[0].declaration_evidence = Some("Forged evidence".into()); + assert!(atomic_canonical::verify(&decoded, &key.public).is_err()); + decoded.depends_on[0].declaration_evidence = None; + assert!(validate_intent(&decoded) + .results + .iter() + .any(|r| r.shape == "ImplementationDeclarationShape")); +} + +#[test] +fn malformed_targets_and_missing_evidence_are_rejected() { + let valid = format!("urn:atomic:change:blake3-256:{}", "A".repeat(52)); + for target in [ + "6FEJ".to_string(), + format!("urn:atomic:change:blake3-256:{}", "a".repeat(52)), + format!("urn:atomic:change:blake3-256:{}B", "A".repeat(51)), + format!("{valid}#fragment"), + ] { + assert!(lift_intent(&fm(), &body(&target, "evidence")).is_err()); + } + assert!(lift_intent(&fm(), &body(&valid, " ")).is_err()); + let missing = format!("::ref{{to={valid} edge=declaresImplementation}}\n"); + assert!(lift_intent(&fm(), &missing).is_err()); +} + +#[test] +fn legacy_reference_json_remains_byte_shape_compatible() { + let old = json!({"to":"urn:atomic:intent:old","edge":"depends"}); + let reference: Ref = serde_json::from_value(old.clone()).unwrap(); + assert_eq!(serde_json::to_value(reference).unwrap(), old); +} diff --git a/atomic-cli/src/commands/intent/graph.rs b/atomic-cli/src/commands/intent/graph.rs new file mode 100644 index 00000000..d45b4e53 --- /dev/null +++ b/atomic-cli/src/commands/intent/graph.rs @@ -0,0 +1,284 @@ +//! `atomic intent graph ` — semantic lineage, not a search neighborhood. + +use crate::{ + commands::Command, + error::{CliError, CliResult}, +}; +use atomic_core::kg::ontology::compact; +use atomic_repository::intent_graph::{IntentGraph as Graph, DEFAULT_MAX_NODES}; +use clap::Parser; + +#[derive(Debug, Parser)] +pub struct IntentGraph { + /// Human intent key, UID, or canonical intent IRI. + pub id: String, + /// Emit canonical nodes, predicates, roles, snapshot identity, and diagnostics. + #[arg(long, conflicts_with_all = ["dot", "html"])] + pub json: bool, + /// Emit Graphviz DOT (pipe to `dot -Tsvg` to render). + #[arg(long, conflicts_with = "html")] + pub dot: bool, + /// Save an interactive HTML graph and open it in the default browser. + #[arg(long)] + pub html: bool, + /// Do not open a browser; emit HTML to stdout or the --output file. + #[arg(long, requires = "html")] + pub no_open: bool, + /// Maximum selected nodes; limits are explicit, never silent truncation. + #[arg(long, default_value_t = DEFAULT_MAX_NODES as u32, value_parser = clap::value_parser!(u32).range(1..=10000))] + pub max_nodes: u32, + /// Write output to this file (HTML also opens it unless --no-open is set). + #[arg(short, long)] + pub output: Option, +} + +impl Command for IntentGraph { + fn run(&self) -> CliResult<()> { + let graph = crate::commands::rpc::intent_graph(&self.id, self.max_nodes as usize) + .map_err(|error| { + if error.to_string().contains("KgNeedsRebuild") { + CliError::InvalidArgument { + message: "workflow graph needs an explicit upgrade; run atomic query reindex --full, then retry intent graph".into(), + } + } else { error } + })?; + let output = if self.json { + serde_json::to_string_pretty(&graph).map_err(|e| CliError::Internal(e.into()))? + "\n" + } else if self.dot { + let mut nodes = graph.nodes.clone(); + annotate(&graph, &mut nodes); + let mut dot = crate::commands::query::emit_dot(&nodes, &graph.edges); + let notes = graph + .diagnostics + .iter() + .map(|s| format!("// {}\n", s.replace(['\n', '\r'], " "))) + .collect::(); + dot.insert_str(0, ¬es); + dot + } else if self.html { + let mut nodes = graph.nodes.clone(); + annotate(&graph, &mut nodes); + let title = format!( + "Intent {} — {} implementing, {} intent-artifact recording, {} dependency-only changes{}", + self.id, + graph.implementing_changes.len(), + graph.recording_changes.len(), + graph.dependency_changes.len(), + if graph.truncated { " [TRUNCATED]" } else { "" } + ); + let html = crate::commands::query::emit_html(&title, &nodes, &graph.edges); + let notes = format!( + "
Scope and evidence notes
{}
", + escape_html(&format!("Snapshot: {}\nScope: {}\n{}", graph.snapshot_identity, graph.scope, graph.diagnostics.join("\n"))) + ); + html.replacen( + "", + &format!("{notes}"), + 1, + ) + } else { + render_text(&graph) + }; + for diagnostic in &graph.diagnostics { + eprintln!("Note: {diagnostic}"); + } + if self.html && !self.no_open { + write_and_open_html(&output, self.output.as_deref(), |url| open::that(url))?; + } else if let Some(path) = &self.output { + std::fs::write(path, output)?; + } else { + print!("{output}"); + } + Ok(()) + } +} + +fn write_and_open_html( + html: &str, + output: Option<&std::path::Path>, + open_browser: impl FnOnce(&str) -> std::io::Result<()>, +) -> CliResult { + use std::io::Write; + + let path = match output { + Some(path) => { + std::fs::write(path, html)?; + path.to_path_buf() + } + None => { + // Keep the file after the process exits so asynchronous browsers + // can load it. Unique names avoid replacing another open graph. + let path = std::env::temp_dir() + .join(format!("atomic-intent-graph-{}.html", uuid::Uuid::new_v4())); + let mut file = std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .open(&path)?; + file.write_all(html.as_bytes())?; + path + } + }; + let path = std::fs::canonicalize(path)?; + let url = url::Url::from_file_path(&path).map_err(|_| CliError::InvalidArgument { + message: format!("cannot convert HTML path to a file URL: {}", path.display()), + })?; + println!("Wrote intent graph to {}", path.display()); + if let Err(error) = open_browser(url.as_str()) { + eprintln!("Could not open a browser: {error}. Open {url} manually."); + } + Ok(path) +} + +fn annotate(graph: &Graph, nodes: &mut [atomic_core::KgNode]) { + for node in nodes { + let role = if node.id == graph.root { + "root intent" + } else if graph.declared_changes.contains(&node.id) + && !graph.captured_changes.contains(&node.id) + { + "declared implementation; original turn provenance not established" + } else if graph.implementing_changes.contains(&node.id) { + "implementing change" + } else if graph.recording_changes.contains(&node.id) { + "records intent artifact; implementation attribution not asserted" + } else if graph.intent_artifact.as_ref() == Some(&node.id) { + "selected intent's recorded artifact (matched by stored vault_path)" + } else if graph.dependency_changes.contains(&node.id) { + "dependency-only change" + } else { + "context" + }; + let state = node.metadata.as_ref().map(|m| { + format!( + "object={}, projection={}", + m.get("object_availability") + .and_then(|v| v.as_str()) + .unwrap_or("unknown"), + m.get("projection_completeness") + .and_then(|v| v.as_str()) + .unwrap_or("reference-only") + ) + }); + let evidence = if node.has_type(atomic_core::kg::ontology::Class::Change) { + state.unwrap_or_else(|| "object=unknown, projection=reference-only".into()) + } else { + String::new() + }; + node.summary = Some(format!( + "[{role}] {evidence} {}", + node.summary.as_deref().unwrap_or("") + )); + } +} + +pub(crate) fn render_text(graph: &Graph) -> String { + let mut out = format!("Intent semantic change graph\nRoot: {}\nSnapshot: {}\nScope: {}\n{} nodes, {} edges; {} implementing changes, {} intent-artifact recording changes, {} dependency-only changes{}\n\n", graph.root, graph.snapshot_identity, graph.scope, graph.nodes.len(), graph.edges.len(), graph.implementing_changes.len(), graph.recording_changes.len(), graph.dependency_changes.len(), if graph.truncated { " [TRUNCATED]" } else { "" }); + let mut nodes = graph.nodes.clone(); + annotate(graph, &mut nodes); + for node in &nodes { + out.push_str(&format!( + "[{}] {}\n {}\n", + compact(node.primary_type()), + one_line(&node.label), + node.id + )); + if let Some(summary) = &node.summary { + out.push_str(&format!(" {}\n", one_line(summary))); + } + for edge in graph.edges.iter().filter(|e| e.from_id == node.id) { + out.push_str(&format!( + " --{}--> {}\n", + compact(edge.kind.iri()), + edge.to_id + )); + if let Some(evidence) = edge + .metadata + .as_ref() + .and_then(|m| m.get("declarationEvidence")) + .and_then(|v| v.as_str()) + { + out.push_str(&format!( + " Declaration evidence: {}\n", + one_line(evidence) + )); + } + } + } + for id in &graph.missing_resources { + out.push_str(&format!("MISSING: {id}\n")); + } + for diagnostic in &graph.diagnostics { + out.push_str(&format!("Note: {diagnostic}\n")); + } + out +} + +fn one_line(text: &str) -> String { + text.chars() + .map(|c| if c.is_control() { ' ' } else { c }) + .collect() +} + +fn escape_html(text: &str) -> String { + text.replace('&', "&") + .replace('<', "<") + .replace('>', ">") +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn html_is_complete_before_browser_opens_and_file_url_is_encoded() { + let directory = tempfile::tempdir().unwrap(); + let path = directory.path().join("intent #1 100%.html"); + let html = "Intent graph"; + let mut called = false; + let written = write_and_open_html(html, Some(&path), |target| { + called = true; + let url = url::Url::parse(target).unwrap(); + assert_eq!(url.scheme(), "file"); + assert!(target.contains("%23") && target.contains("%25") && target.contains("%20")); + assert_eq!( + url.to_file_path().unwrap(), + std::fs::canonicalize(&path).unwrap() + ); + assert_eq!(std::fs::read_to_string(&path).unwrap(), html); + Ok(()) + }) + .unwrap(); + assert!(called); + assert_eq!(written, std::fs::canonicalize(path).unwrap()); + } + + #[test] + fn automatic_html_files_are_unique_and_survive_browser_failure() { + let first = write_and_open_html("first graph", None, |_| { + Err(std::io::Error::new( + std::io::ErrorKind::NotFound, + "no browser available", + )) + }) + .unwrap(); + let second = write_and_open_html("second graph", None, |_| Ok(())).unwrap(); + assert_ne!(first, second); + assert_eq!(first.extension().unwrap(), "html"); + assert_eq!(std::fs::read_to_string(&first).unwrap(), "first graph"); + assert_eq!(std::fs::read_to_string(&second).unwrap(), "second graph"); + std::fs::remove_file(first).unwrap(); + std::fs::remove_file(second).unwrap(); + } + + #[test] + fn failed_html_write_does_not_launch_browser() { + let directory = tempfile::tempdir().unwrap(); + let mut called = false; + assert!(write_and_open_html("graph", Some(directory.path()), |_| { + called = true; + Ok(()) + }) + .is_err()); + assert!(!called); + } +} diff --git a/atomic-cli/src/commands/intent/mod.rs b/atomic-cli/src/commands/intent/mod.rs index 2125c30b..6da0acf7 100644 --- a/atomic-cli/src/commands/intent/mod.rs +++ b/atomic-cli/src/commands/intent/mod.rs @@ -11,6 +11,7 @@ //! ```text //! atomic intent validate Gate an intent against the canonical shapes //! atomic intent show Render an intent as a read-time projection +//! atomic intent graph Show the intent's semantic change lineage //! atomic intent new Scaffold a directive-based intent //! atomic intent attest <ID> Sign an intent into a tracked attestation entry //! atomic intent verify <ID> Verify a signed intent's attestation @@ -40,6 +41,7 @@ pub mod bridge { pub use atomic_repository::intent_bridge::*; } pub mod delete; +pub mod graph; pub mod key_cache; pub mod link; pub mod list; @@ -51,6 +53,7 @@ pub mod verify; pub use attest::IntentAttest; pub use delete::IntentDelete; +pub use graph::IntentGraph; pub use link::IntentLink; pub use list::IntentList; pub use new::IntentNew; @@ -72,6 +75,9 @@ pub enum IntentCommands { /// Show an intent as a rendered read-time projection. Show(IntentShow), + /// Show an intent-rooted semantic change graph, including dependencies and evidence links. + Graph(IntentGraph), + /// Scaffold a new directive-based intent into the vault. New(IntentNew), @@ -112,6 +118,7 @@ impl Command for Intent { match &self.command { IntentCommands::Validate(cmd) => cmd.run(), IntentCommands::Show(cmd) => cmd.run(), + IntentCommands::Graph(cmd) => cmd.run(), IntentCommands::New(cmd) => cmd.run(), IntentCommands::Attest(cmd) => cmd.run(), IntentCommands::Verify(cmd) => cmd.run(), diff --git a/atomic-cli/src/commands/intent/update.rs b/atomic-cli/src/commands/intent/update.rs index b15c59b3..34cc4c18 100644 --- a/atomic-cli/src/commands/intent/update.rs +++ b/atomic-cli/src/commands/intent/update.rs @@ -2,9 +2,9 @@ //! //! The canonical-family analogue of `atomic vault intent update`. It delegates //! to the exact same [`Repository::vault_intent_update`] write path, so its -//! behavior (frontmatter/status/priority/title/reason/source-memory updates, -//! body replacement, the started-or-linked guard behind `--force`) is -//! byte-for-byte identical to the vault verb it supersedes. +//! field updates and the started-or-linked guard behind `--force` share the +//! vault verb's implementation. Work-start declarations additionally persist +//! explicit agent turn context before updating status. use clap::Parser; @@ -28,6 +28,15 @@ pub struct IntentUpdate { #[arg(long)] pub status: Option<String>, + /// Bind work start to this active agent session (or ATOMIC_SESSION_ID). + /// Only valid with --status in_progress; never inferred from active sessions. + #[arg(long)] + pub session_id: Option<String>, + + /// Expected 1-indexed hook turn, to reject delayed work-start declarations. + #[arg(long)] + pub turn: Option<u32>, + /// New assignee. #[arg(long)] pub assignee: Option<String>, @@ -63,6 +72,58 @@ pub struct IntentUpdate { } impl IntentUpdate { + fn bind_work_start(&self) -> CliResult<()> { + let starting = matches!(self.status.as_deref(), Some("in_progress" | "in-progress")); + if !starting { + if self.session_id.is_some() || self.turn.is_some() { + return Err(CliError::InvalidArgument { + message: "--session-id and --turn require --status in_progress".into(), + }); + } + return Ok(()); + } + let session_id = self + .session_id + .clone() + .or_else(|| std::env::var("ATOMIC_SESSION_ID").ok()); + let Some(session_id) = session_id else { + if self.turn.is_some() { + return Err(CliError::InvalidArgument { + message: "--turn requires --session-id or ATOMIC_SESSION_ID".into(), + }); + } + return Ok(()); // Human status-only updates remain supported. + }; + let root = find_repository_root()?; + let uid = { + let repo = + crate::commands::open_readonly_repository(&root).map_err(CliError::Repository)?; + let inputs = + super::bridge::read_intent(&repo, &self.id).map_err(CliError::Repository)?; + inputs + .frontmatter + .get("uid") + .and_then(|value| value.as_str()) + .filter(|uid| !uid.is_empty()) + .ok_or_else(|| CliError::InvalidArgument { + message: "intent has no canonical UID; cannot bind work start".into(), + })? + .to_string() + }; + let dot = Repository::canonical_dot_dir(&root).map_err(CliError::Repository)?; + let store = atomic_agent::turn::session::SessionStore::new(dot.join("sessions")) + .map_err(|error| CliError::Internal(error.into()))?; + // Persist the declaration before status mutation or any record hook. + let binding = store + .bind_turn_intent(&session_id, self.turn, &uid) + .map_err(|error| CliError::Internal(error.into()))?; + println!( + "Bound governing intent: {} (session {}, turn {})", + binding.intent_uid, binding.session_id, binding.turn_number + ); + Ok(()) + } + /// Resolve the body override: inline `--body` or `--body-stdin`. /// /// Shared by the local body and the service-layer hook — reading the @@ -86,6 +147,7 @@ impl IntentUpdate { impl Command for IntentUpdate { fn run(&self) -> CliResult<()> { + self.bind_work_start()?; if crate::commands::rpc::intent_update(self)? { return Ok(()); } diff --git a/atomic-cli/src/commands/query/graph_inspector.css b/atomic-cli/src/commands/query/graph_inspector.css new file mode 100644 index 00000000..0a0fe538 --- /dev/null +++ b/atomic-cli/src/commands/query/graph_inspector.css @@ -0,0 +1,40 @@ +#header { height: 50px; } +#header h1, #header .stats { flex-shrink: 0; white-space: nowrap; } +#header .query { min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } +#graph-layout { position: fixed; inset: 50px 0 0; display: flex; } +#graph-view { flex: 1; min-width: 0; min-height: 0; position: relative; overflow: hidden; } +#node-inspector { width: min(420px, 38vw); flex-shrink: 0; overflow-y: auto; padding: 16px; background: #161628; border-left: 1px solid #3a3a50; font-size: 12px; line-height: 1.5; } +.inspector-heading { display: flex; align-items: center; justify-content: space-between; gap: 12px; } +.inspector-heading h2 { font-size: 16px; color: #e8e8fa; } +.inspector-clear, .inspector-picker { background: #25253c; color: #eee; border: 1px solid #50506a; border-radius: 4px; padding: 6px; } +.inspector-clear { cursor: pointer; } +.inspector-clear:disabled { opacity: .4; cursor: default; } +.inspector-picker-label { display: block; margin-top: 14px; color: #bdbdd2; } +.inspector-picker { width: 100%; margin-top: 4px; } +.inspector-status { margin-top: 12px; color: #9fd6ff; overflow-wrap: anywhere; } +.inspector-section { margin-top: 18px; } +.inspector-section h3, .inspector-section > summary { color: #aaaaca; font-size: 11px; text-transform: uppercase; letter-spacing: .5px; margin-bottom: 6px; } +.inspector-section h4 { color: #b9b9d4; font-size: 11px; font-weight: 600; margin-top: 8px; } +.inspector-value { font: inherit; color: #e0e0ee; white-space: pre-wrap; overflow-wrap: anywhere; } +.inspector-muted { margin-top: 10px; color: #9393ae; } +.inspector-relation { border-left: 2px solid #474768; padding-left: 10px; margin-top: 10px; } +.inspector-predicate { color: #bca6ed; font-size: 11px; } +.inspector-neighbor { display: block; border: 0; padding: 3px 0; text-align: left; background: none; color: #7ccaff; font: inherit; cursor: pointer; overflow-wrap: anywhere; } +.inspector-neighbor:hover { text-decoration: underline; } +.inspector-node-id { display: block; color: #a0a0b8; font-size: 10px; overflow-wrap: anywhere; } +.inspector-usage-cost { font-size: 21px; font-weight: 600; color: #a9e6c3; margin: 12px 0; } +.inspector-usage-coverage { color: #bcbcd2; margin-top: 8px; } +.inspector-usage-warning { color: #ffc484; margin-top: 8px; } +.inspector-token-table { width: 100%; border-collapse: collapse; margin: 8px 0; } +.inspector-token-table th { text-align: left; font-weight: 400; color: #bcbcd2; } +.inspector-token-table td { text-align: right; font-variant-numeric: tabular-nums; color: #e7e7f5; } +.inspector-token-table th, .inspector-token-table td { padding: 4px 0; border-bottom: 1px solid #292940; } +.inspector-attribution { border-left: 2px solid #526f99; padding-left: 10px; margin-top: 10px; overflow-wrap: anywhere; } +.inspector-usage-notes { margin-top: 10px; color: #aaaaca; } +.inspector-relation details { margin-top: 6px; color: #aaaaca; } +#node-inspector summary { cursor: pointer; } +#node-inspector button:focus-visible, #node-inspector select:focus-visible { outline: 2px solid #7ccaff; outline-offset: 2px; } +@media (max-width: 760px) { + #graph-layout { flex-direction: column; } + #node-inspector { width: 100%; height: 45%; flex: 0 0 45%; border-left: 0; border-top: 1px solid #3a3a50; } +} diff --git a/atomic-cli/src/commands/query/graph_inspector.js b/atomic-cli/src/commands/query/graph_inspector.js new file mode 100644 index 00000000..5193b989 --- /dev/null +++ b/atomic-cli/src/commands/query/graph_inspector.js @@ -0,0 +1,195 @@ +// Embedded into exported HTML. This inspector reads only the exported snapshot; +// it never fetches from a service or treats graph-controlled strings as HTML. +function createGraphInspector(container, graphData, onSelect) { + var doc = container.ownerDocument; + var byId = new Map(graphData.nodes.map(function(node) { return [node.id, node]; })); + var incoming = new Map(); + var outgoing = new Map(); + function endpoint(value) { return typeof value === 'object' && value ? value.id : value; } + function compact(value) { return String(value).replace('https://atomic.dev/ns/workflow#', 'atomic:'); } + function element(tag, text, className) { + var el = doc.createElement(tag); + if (text !== undefined) el.textContent = text; + if (className) el.className = className; + return el; + } + graphData.links.forEach(function(edge) { + var source = endpoint(edge.source); + var target = endpoint(edge.target); + if (!outgoing.has(source)) outgoing.set(source, []); + if (!incoming.has(target)) incoming.set(target, []); + outgoing.get(source).push({ edge: edge, other: target }); + incoming.get(target).push({ edge: edge, other: source }); + }); + + var heading = element('div', undefined, 'inspector-heading'); + heading.appendChild(element('h2', 'Node details')); + var clear = element('button', 'Clear', 'inspector-clear'); + clear.type = 'button'; + clear.setAttribute('aria-label', 'Clear selected node'); + heading.appendChild(clear); + var label = element('label', 'Select a node', 'inspector-picker-label'); + label.htmlFor = 'node-picker'; + var picker = element('select', undefined, 'inspector-picker'); + picker.id = 'node-picker'; + var placeholder = element('option', 'Click a graph node or choose here'); + placeholder.value = ''; + picker.appendChild(placeholder); + graphData.nodes.forEach(function(node) { + var text = node.label || node.id; + var option = element('option', text.length > 100 ? text.slice(0, 97) + '...' : text); + option.value = node.id; + picker.appendChild(option); + }); + var status = element('p', '', 'inspector-status'); + status.setAttribute('role', 'status'); + var content = element('div', undefined, 'inspector-content'); + container.replaceChildren(heading, label, picker, status, content); + + function section(title) { + var result = element('section', undefined, 'inspector-section'); + result.appendChild(element('h3', title)); + content.appendChild(result); + return result; + } + function field(title, text) { + section(title).appendChild(element('pre', text, 'inspector-value')); + } + function tokenTable(parent, tokens) { + var table = element('table', undefined, 'inspector-token-table'); + [['Input', 'input'], ['Output', 'output'], ['Reasoning', 'reasoning'], + ['Cache read', 'cache_read'], ['Cache write', 'cache_write'], ['Recorded total', 'total']].forEach(function(pair) { + var row = element('tr'); + row.appendChild(element('th', pair[0])); + row.appendChild(element('td', tokens ? Number(tokens[pair[1]]).toLocaleString('en-US') : 'Not recorded')); + table.appendChild(row); + }); + parent.appendChild(table); + } + function usageDetails(node) { + var usage = node.metadata && node.metadata.ai_usage; + var types = node.types || [node.kind]; + var relevant = types.some(function(type) { + return /#(Change|View|WorkIntent|FeatureIntent|BugIntent|ReviewIntent|ManagedIntent)$/.test(type); + }); + if (!usage) { + if (relevant) section('AI usage').appendChild(element('p', 'Usage was not included in this export. Regenerate it with a current Atomic CLI.', 'inspector-muted')); + return; + } + var result = section('AI usage'); + result.classList.add('inspector-ai-usage'); + var scope = { + change_provenance: 'This change’s recorded provenance.', + intent_implementation_and_recording_evidence_excluding_dependencies: 'Intent implementation and intent-artifact recording evidence; dependency-only changes excluded.', + view_effective_membership_including_ancestors_and_dependencies: 'Effective view membership, including ancestor views and dependencies.', + view_canonical_effective_membership_including_ancestors_and_dependencies: 'Effective view membership from canonical repository data, including ancestor views and dependencies.' + }; + result.appendChild(element('p', scope[usage.basis] || usage.basis, 'inspector-muted')); + var coverage = usage.coverage; + result.appendChild(element('p', coverage.with_provenance + ' of ' + coverage.changes + ' unique changes have AI provenance; ' + coverage.without_provenance + ' without provenance, ' + coverage.unavailable + ' unavailable.', 'inspector-usage-coverage')); + if (!usage.scope_complete) result.appendChild(element('p', 'Incomplete membership scope: ' + (usage.scope_error || 'some ancestor views are unavailable') + '. These totals are not complete.', 'inspector-usage-warning')); + if (coverage.duplicate_usage_records) result.appendChild(element('p', coverage.duplicate_usage_records + ' duplicate usage record(s) counted once.', 'inspector-muted')); + if (coverage.conflicting_usage_records) result.appendChild(element('p', coverage.conflicting_usage_records + ' conflicting usage record(s) excluded from totals.', 'inspector-usage-warning')); + if (coverage.records_without_cost) result.appendChild(element('p', coverage.records_without_cost + ' usage record(s) have no usable cost. Amount below is a known subtotal.', 'inspector-usage-warning')); + if (coverage.records_without_tokens) result.appendChild(element('p', coverage.records_without_tokens + ' usage record(s) have no reported token counts. Token values below are known subtotals.', 'inspector-usage-warning')); + result.appendChild(element('div', usage.cost ? '$' + usage.cost.amount + ' ' + usage.cost.currency : 'Cost not recorded', 'inspector-usage-cost')); + tokenTable(result, usage.tokens); + result.appendChild(element('p', 'Cache and reasoning categories are provider-defined: they may overlap or be excluded from the recorded total.', 'inspector-muted')); + if (usage.contributors && usage.contributors.length) { + var attribution = section('Models, tools & vendors'); + usage.contributors.forEach(function(profile) { + var card = element('div', undefined, 'inspector-attribution'); + card.appendChild(element('h4', profile.model || '(model not recorded)')); + card.appendChild(element('p', 'Vendor: ' + (profile.vendor || '(not recorded)'))); + card.appendChild(element('p', 'Tool: ' + (profile.tool || '(not recorded)'))); + card.appendChild(element('p', profile.cost_usd === null ? 'Cost not recorded' : '$' + profile.cost_usd + ' USD')); + var breakdown = element('details'); + breakdown.appendChild(element('summary', 'Token breakdown for this model/tool')); + tokenTable(breakdown, profile.tokens); + card.appendChild(breakdown); + attribution.appendChild(card); + }); + } + var notes = element('details', undefined, 'inspector-usage-notes'); + notes.appendChild(element('summary', 'How these totals are calculated')); + (usage.notes || []).forEach(function(note) { notes.appendChild(element('p', note, 'inspector-muted')); }); + result.appendChild(notes); + } + function relationships(title, entries) { + var list = section(title + ' (' + entries.length + ')'); + if (!entries.length) list.appendChild(element('p', 'None in this exported graph.', 'inspector-muted')); + entries.forEach(function(entry) { + var row = element('div', undefined, 'inspector-relation'); + row.appendChild(element('div', compact(entry.edge.kind), 'inspector-predicate')); + var neighbor = byId.get(entry.other); + if (neighbor) { + var button = element('button', neighbor.label || neighbor.id, 'inspector-neighbor'); + button.type = 'button'; + button.title = entry.other; + button.addEventListener('click', function() { select(entry.other); picker.focus(); }); + row.appendChild(button); + } + row.appendChild(element('code', entry.other, 'inspector-node-id')); + if (entry.edge.metadata !== undefined && entry.edge.metadata !== null) { + var details = element('details'); + details.appendChild(element('summary', 'Relationship metadata')); + details.appendChild(element('pre', JSON.stringify(entry.edge.metadata, null, 2), 'inspector-value')); + row.appendChild(details); + } + list.appendChild(row); + }); + } + function select(id) { + var node = byId.get(id); + picker.value = node ? node.id : ''; + clear.disabled = !node; + container.dataset.selectedNode = node ? node.id : ''; + content.replaceChildren(); + if (!node) { + status.textContent = 'No node selected.'; + content.appendChild(element('p', 'Click a node to pin its details here. Follow a relationship to inspect its other endpoint. Escape or Clear resets selection.', 'inspector-muted')); + } else { + status.textContent = 'Selected: ' + (node.label || node.id); + field('Label', node.label || '(no label)'); + field('Canonical ID', node.id); + field('Types', (node.types || [node.kind]).join('\n')); + field('Source', node.source || '(not recorded)'); + field('Summary', node.summary || '(no summary recorded)'); + usageDetails(node); + var metadata = section('Metadata'); + if (node.metadata === undefined || node.metadata === null) { + metadata.appendChild(element('p', 'No metadata exported.', 'inspector-muted')); + } else if (typeof node.metadata === 'object' && !Array.isArray(node.metadata)) { + Object.keys(node.metadata).filter(function(key) { + return !['ai_usage', 'ai_usage_records', 'ai_usage_state', 'ai_usage_record_version'].includes(key); + }).sort().forEach(function(key) { + metadata.appendChild(element('h4', key)); + var value = node.metadata[key]; + metadata.appendChild(element('pre', typeof value === 'string' ? value : JSON.stringify(value, null, 2), 'inspector-value')); + }); + } else { + metadata.appendChild(element('pre', JSON.stringify(node.metadata, null, 2), 'inspector-value')); + } + relationships('Outgoing relationships', outgoing.get(node.id) || []); + relationships('Incoming relationships', incoming.get(node.id) || []); + var raw = element('details', undefined, 'inspector-section'); + raw.appendChild(element('summary', 'Raw exported node')); + raw.appendChild(element('pre', JSON.stringify(node, null, 2), 'inspector-value')); + content.appendChild(raw); + content.appendChild(element('p', 'Snapshot data only. File contents and historical patch bodies are not fetched.', 'inspector-muted')); + } + if (onSelect) onSelect(node ? node.id : null); + } + picker.addEventListener('change', function() { select(picker.value); }); + clear.addEventListener('click', function() { select(null); picker.focus(); }); + doc.addEventListener('keydown', function(event) { + if (event.key === 'Escape') { select(null); picker.focus(); } + }); + // Build the initial empty state without calling the graph's redraw callback + // before its canvas/simulation initialization has finished. + var callback = onSelect; + onSelect = null; + select(null); + onSelect = callback; + return { select: select }; +} diff --git a/atomic-cli/src/commands/query/mod.rs b/atomic-cli/src/commands/query/mod.rs index 9c3ddbd6..d2ff3807 100644 --- a/atomic-cli/src/commands/query/mod.rs +++ b/atomic-cli/src/commands/query/mod.rs @@ -436,6 +436,9 @@ impl Command for QueryGraph { }); } + repo.enrich_workflow_usage(&mut nodes) + .map_err(CliError::Repository)?; + // 4. Output if self.json { let output = serde_json::json!({ @@ -628,6 +631,50 @@ fn dot_escape_label(s: &str) -> String { s.replace('\\', "\\\\").replace('"', "\\\"") } +/// Stable categorical colors with contrast against the HTML graph's dark canvas. +/// Exhaustive matching keeps ontology additions from silently sharing a fallback. +fn html_node_color(kind: &str) -> String { + let Some(class) = Class::from_iri(kind) else { + // Unlike a palette indexed by the types currently visible, hashing the + // full IRI keeps an external type's color stable between exports. + let hash = kind.bytes().fold(2166136261u32, |hash, byte| { + (hash ^ u32::from(byte)).wrapping_mul(16777619) + }); + return format!("hsl({}, 65%, 68%)", hash % 360); + }; + match class { + Class::WorkIntent => "#63d5ea", + Class::FeatureIntent => "#56cfe1", + Class::BugIntent => "#ff8a9a", + Class::ReviewIntent => "#a5b4fc", + Class::ManagedIntent => "#8be4f0", + Class::Goal => "#6ed4ae", + Class::Memory => "#e2bc81", + Class::Skill => "#a4d98b", + Class::ToolResult => "#8da9bd", + Class::Task => "#dd81ef", + Class::AcceptanceCriterion => "#c6e96e", + Class::ScopeItem => "#c5a582", + Class::Constraint => "#f59ac2", + Class::Todo => "#d79e64", + Class::Session => "#b6a4ff", + Class::SessionTurn => "#39b8bd", + Class::ManagedTurn => "#8ad4dc", + Class::ProvenanceGraph => "#72a7ff", + Class::Attestation => "#f0c4ec", + Class::Finding => "#ffad66", + Class::Change => "#ff786f", + Class::View => "#9878e8", + Class::File => "#53d89b", + Class::Identity => "#f4d06f", + Class::Concept => "#aac4a0", + Class::Project => "#adcbe7", + Class::ProjectIdentity => "#bba2a0", + Class::VerificationStatus => "#d4d8e3", + } + .to_owned() +} + /// Generate a self-contained HTML page with a D3.js force-directed graph. pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> String { // Build JSON data for embedding @@ -661,9 +708,12 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri serde_json::json!({ "id": n.id, "kind": n.primary_type(), + "color": html_node_color(n.primary_type()), "types": n.types, "label": n.label, "summary": n.summary.as_deref().unwrap_or(""), + "source": n.source, + "metadata": n.metadata, "content_matches": content_matches, "entity_kind": entity_kind, "line": line, @@ -679,6 +729,7 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri "source": e.from_id, "target": e.to_id, "kind": e.kind, + "metadata": e.metadata, }) }) .collect(); @@ -688,7 +739,11 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri "links": graph_edges, }); - let data_json = serde_json::to_string(&graph_data).unwrap(); + let data_json = serde_json::to_string(&graph_data) + .unwrap() + .replace('<', "\\u003c") + .replace('>', "\\u003e") + .replace('&', "\\u0026"); let escaped_query = query .replace('&', "&") .replace('<', "<") @@ -707,16 +762,17 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri #header h1 {{ font-size: 14px; font-weight: 600; color: #a0a0c0; letter-spacing: 1px; }} #header .query {{ font-size: 14px; color: #6ec6ff; }} #header .stats {{ font-size: 12px; color: #888; margin-left: auto; }} - canvas {{ display: block; width: 100vw; height: 100vh; }} + canvas {{ display: block; width: 100%; height: 100%; }} #tooltip {{ position: fixed; padding: 10px 14px; background: rgba(0,0,0,0.92); border: 1px solid #555; border-radius: 6px; font-size: 12px; pointer-events: none; display: none; z-index: 20; max-width: 420px; line-height: 1.5; }} #tooltip .kind {{ color: #888; font-size: 10px; text-transform: uppercase; letter-spacing: 0.5px; }} #tooltip .label {{ color: #fff; font-weight: 600; margin: 3px 0; word-break: break-all; }} #tooltip .summary {{ color: #bbb; font-size: 11px; margin-bottom: 2px; }} #tooltip .detail {{ color: #aaa; font-size: 11px; }} - #legend {{ position: fixed; bottom: 16px; right: 16px; z-index: 10; background: rgba(26,26,46,0.92); border: 1px solid #333; border-radius: 8px; padding: 10px 14px; font-size: 11px; }} + #legend {{ position: absolute; bottom: 16px; right: 16px; z-index: 10; background: rgba(26,26,46,0.92); border: 1px solid #333; border-radius: 8px; padding: 10px 14px; font-size: 11px; }} #legend .legend-title {{ color: #a0a0c0; font-weight: 600; font-size: 10px; text-transform: uppercase; letter-spacing: 0.5px; margin-bottom: 6px; }} #legend .legend-item {{ display: flex; align-items: center; gap: 8px; margin: 3px 0; color: #ccc; }} #legend .legend-dot {{ width: 10px; height: 10px; border-radius: 50%; flex-shrink: 0; }} + {inspector_css} </style> </head> <body> @@ -731,23 +787,21 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri <div class="summary" id="tt-summary"></div> <div class="detail" id="tt-detail"></div> </div> -<canvas id="graph"></canvas> -<div id="legend"></div> +<div id="graph-layout"> + <div id="graph-view"> + <canvas id="graph"></canvas> + <div id="legend"></div> + </div> + <aside id="node-inspector" aria-label="Node details"></aside> +</div> <script src="https://cdn.jsdelivr.net/npm/d3@7"></script> <script> (function() {{ + {inspector_js} // Embedded graph data var graphData = {data_json}; - var COLORS = {{ - 'https://atomic.dev/ns/workflow#File': '#2ecc71', - 'https://atomic.dev/ns/workflow#Change': '#e74c3c', - 'https://atomic.dev/ns/workflow#View': '#9b59b6', - 'https://atomic.dev/ns/workflow#Identity': '#f1c40f', - 'https://atomic.dev/ns/workflow#Goal': '#1abc9c', - 'https://atomic.dev/ns/workflow#WorkIntent': '#1abc9c', - 'https://atomic.dev/ns/workflow#Memory': '#f39c12', - }}; + var kindColors = new Map(graphData.nodes.map(function(n) {{ return [n.kind, n.color]; }})); var EDGE_COLORS = {{ 'https://atomic.dev/ns/workflow#modifiesFile': '#e74c3c', 'https://atomic.dev/ns/workflow#authoredBy': '#f1c40f', @@ -755,7 +809,6 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri 'https://atomic.dev/ns/workflow#directlyOnView': '#9b59b6', 'https://atomic.dev/ns/workflow#references': '#95a5a6', }}; - var DEFAULT_COLOR = '#7f8c8d'; function nodeRadius(d) {{ if (d.kind === 'https://atomic.dev/ns/workflow#File') return 5 + Math.min(d.content_matches, 100) * 0.12; @@ -763,15 +816,16 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri return 5; }} - function nodeColor(d) {{ return COLORS[d.kind] || DEFAULT_COLOR; }} + function nodeColor(d) {{ return d.color; }} function edgeColor(d) {{ return EDGE_COLORS[d.kind] || '#555'; }} // Canvas setup var canvas = document.getElementById('graph'); + var graphView = document.getElementById('graph-view'); var ctx = canvas.getContext('2d'); var dpr = window.devicePixelRatio || 1; - var width = window.innerWidth; - var height = window.innerHeight; + var width = graphView.clientWidth; + var height = graphView.clientHeight; canvas.width = width * dpr; canvas.height = height * dpr; canvas.style.width = width + 'px'; @@ -789,6 +843,7 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri nodes.forEach(function(n) {{ n.r = nodeRadius(n); }}); var nodeById = new Map(nodes.map(function(n) {{ return [n.id, n]; }})); + var selectedNodeId = null; var links = graphData.links.filter(function(l) {{ return nodeById.has(l.source) && nodeById.has(l.target); @@ -799,14 +854,29 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri // Build legend from kinds present in data var kindsPresent = new Set(nodes.map(function(n) {{ return n.kind; }})); var legendEl = document.getElementById('legend'); - var legendHTML = '<div class="legend-title">Node Types</div>'; + var legendTitle = document.createElement('div'); + legendTitle.className = 'legend-title'; + legendTitle.textContent = 'Node Types'; + legendEl.appendChild(legendTitle); var kindOrder = Array.from(kindsPresent).sort(); kindOrder.forEach(function(k) {{ if (kindsPresent.has(k)) {{ - legendHTML += '<div class="legend-item"><span class="legend-dot" style="background:' + (COLORS[k] || DEFAULT_COLOR) + '"></span>' + k.replace('https://atomic.dev/ns/workflow#', 'atomic:') + '</div>'; + var item = document.createElement('div'); + item.className = 'legend-item'; + item.dataset.kind = k; + var dot = document.createElement('span'); + dot.className = 'legend-dot'; + dot.style.background = kindColors.get(k); + item.appendChild(dot); + item.appendChild(document.createTextNode(k.replace('https://atomic.dev/ns/workflow#', 'atomic:'))); + legendEl.appendChild(item); }} }}); - legendEl.innerHTML = legendHTML; + var inspector = createGraphInspector(document.getElementById('node-inspector'), graphData, function(id) {{ + selectedNodeId = id; + hideTooltip(); + draw(); + }}); // Current zoom transform var currentTransform = d3.zoomIdentity; @@ -832,6 +902,9 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri links.forEach(function(l) {{ var s = l.source; var t = l.target; + var highlighted = selectedNodeId && (s.id === selectedNodeId || t.id === selectedNodeId); + ctx.globalAlpha = selectedNodeId ? (highlighted ? 0.9 : 0.12) : 0.4; + ctx.lineWidth = highlighted ? 1.8 : 0.6; ctx.strokeStyle = edgeColor(l); ctx.beginPath(); ctx.moveTo(s.x, s.y); @@ -852,6 +925,13 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri ctx.lineWidth = 1.2; ctx.globalAlpha = 1.0; ctx.stroke(); + if (d.id === selectedNodeId) {{ + ctx.beginPath(); + ctx.arc(d.x, d.y, r + 4 / currentTransform.k, 0, 2 * Math.PI); + ctx.strokeStyle = '#9fd6ff'; + ctx.lineWidth = 2 / currentTransform.k; + ctx.stroke(); + }} }}); // Draw labels for large nodes (modules and high-match files) @@ -860,7 +940,7 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri ctx.textAlign = 'center'; ctx.textBaseline = 'top'; nodes.forEach(function(d) {{ - if (d.r > 10) {{ + if (d.r > 10 || d.id === selectedNodeId) {{ var label = d.label; if (d.kind === 'module') label += '/'; if (label.length > 28) label = label.slice(0, 25) + '...'; @@ -876,6 +956,18 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri // Zoom and pan via d3.zoom var zoomBehavior = d3.zoom() + .filter(function(event) {{ + if (event.button || (event.ctrlKey && event.type !== 'wheel')) return false; + if (event.type === 'mousedown') {{ + dragStart = [event.clientX, event.clientY]; + dragMoved = false; + var rect = canvas.getBoundingClientRect(); + // Let our node-drag handler own this gesture, rather than D3 starting + // a pan and stopping propagation before the node handler can run. + return !findNode(event.clientX - rect.left, event.clientY - rect.top); + }} + return true; + }}) .scaleExtent([0.05, 10]) .on('zoom', function(event) {{ currentTransform = event.transform; @@ -951,8 +1043,13 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri // Drag state var dragNode = null; var dragSubject = null; + var dragStart = null; + var dragMoved = false; canvas.addEventListener('mousemove', function(e) {{ + if (dragStart && Math.hypot(e.clientX - dragStart[0], e.clientY - dragStart[1]) > 4) {{ + dragMoved = true; + }} var rect = canvas.getBoundingClientRect(); var sx = e.clientX - rect.left; var sy = e.clientY - rect.top; @@ -977,6 +1074,8 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri }}); canvas.addEventListener('mousedown', function(e) {{ + dragStart = [e.clientX, e.clientY]; + dragMoved = false; var rect = canvas.getBoundingClientRect(); var sx = e.clientX - rect.left; var sy = e.clientY - rect.top; @@ -995,6 +1094,7 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri }}); window.addEventListener('mouseup', function() {{ + dragStart = null; if (dragNode) {{ dragNode.fx = null; dragNode.fy = null; @@ -1010,10 +1110,17 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri hideTooltip(); }}); + canvas.addEventListener('click', function(e) {{ + if (dragMoved) return; + var rect = canvas.getBoundingClientRect(); + var found = findNode(e.clientX - rect.left, e.clientY - rect.top); + inspector.select(found ? found.id : null); + }}); + // Handle resize - window.addEventListener('resize', function() {{ - width = window.innerWidth; - height = window.innerHeight; + function resizeGraph() {{ + width = graphView.clientWidth; + height = graphView.clientHeight; canvas.width = width * dpr; canvas.height = height * dpr; canvas.style.width = width + 'px'; @@ -1021,13 +1128,18 @@ pub(crate) fn emit_html(query: &str, nodes: &[KgNode], edges: &[KgEdge]) -> Stri ctx.scale(dpr, dpr); simulation.force('center', d3.forceCenter(width / 2, height / 2)); simulation.alpha(0.1).restart(); - }}); + draw(); + }} + window.addEventListener('resize', resizeGraph); + if (window.ResizeObserver) new ResizeObserver(resizeGraph).observe(graphView); }})(); </script> </body> </html>"##, query = escaped_query, data_json = data_json, + inspector_js = include_str!("graph_inspector.js"), + inspector_css = include_str!("graph_inspector.css"), ) } @@ -1978,6 +2090,90 @@ impl Command for QueryAsk { mod tests { use super::truncate_display; + #[test] + fn html_palette_covers_every_workflow_class_with_unique_high_contrast_colors() { + use atomic_core::kg::ontology::Class; + fn luminance(hex: &str) -> f64 { + let channel = |offset| { + let value = + f64::from(u8::from_str_radix(&hex[offset..offset + 2], 16).unwrap()) / 255.0; + if value <= 0.04045 { + value / 12.92 + } else { + ((value + 0.055) / 1.055).powf(2.4) + } + }; + 0.2126 * channel(1) + 0.7152 * channel(3) + 0.0722 * channel(5) + } + let colors: std::collections::HashSet<_> = Class::ALL + .iter() + .map(|class| super::html_node_color(class.iri())) + .collect(); + assert_eq!( + colors.len(), + Class::ALL.len(), + "known types must not share colors" + ); + let background = luminance("#1a1a2e"); + for color in colors { + assert!(color.starts_with('#') && color.len() == 7); + assert!( + (luminance(&color) + 0.05) / (background + 0.05) >= 4.5, + "low contrast: {color}" + ); + } + } + + #[test] + fn html_external_type_colors_are_stable_and_not_the_old_gray_fallback() { + let first = super::html_node_color("https://example.test/TypeA"); + assert_eq!(first, super::html_node_color("https://example.test/TypeA")); + assert!(first.starts_with("hsl(")); + assert_ne!(first, super::html_node_color("https://example.test/TypeB")); + } + + #[test] + fn html_inspector_embeds_full_snapshot_data_without_script_injection() { + use atomic_core::{ + kg::ontology::{Class, Predicate}, + KgEdge, KgNode, + }; + let hostile = "</script><script>alert('metadata')</script>"; + let mut node = KgNode::new("urn:node:first", Class::Change, hostile, "vcs"); + node.summary = Some("full summary ".repeat(40)); + node.metadata = + Some(serde_json::json!({"hash":"FULL_HASH", "nested":{"evidence":hostile}})); + let neighbor = KgNode::new("urn:node:second", Class::File, "src/cli.ts", "vcs"); + let mut edge = KgEdge::new(&node.id, &neighbor.id, Predicate::ModifiesFile); + edge.metadata = Some(serde_json::json!({"evidence":hostile})); + let html = super::emit_html( + "Inspect <metadata>", + &[node.clone(), neighbor], + &[edge.clone()], + ); + let json = html + .split("var graphData = ") + .nth(1) + .unwrap() + .split(";\n") + .next() + .unwrap(); + let data: serde_json::Value = serde_json::from_str(json).unwrap(); + assert_eq!(data["nodes"][0]["metadata"], node.metadata.unwrap()); + assert_eq!(data["nodes"][0]["summary"], node.summary.unwrap()); + assert_eq!(data["nodes"][0]["source"], "vcs"); + assert_eq!( + data["nodes"][0]["color"], + super::html_node_color(Class::Change.iri()) + ); + assert_eq!(data["links"][0]["metadata"], edge.metadata.unwrap()); + assert!(!html.contains(hostile)); + assert!(html.contains("\\u003c/script\\u003e")); + assert!(html.contains("aria-label=\"Node details\"")); + assert!(html.contains("if (dragMoved) return;")); + assert!(html.contains("graphView.clientWidth")); + } + #[test] fn test_truncate_display_short_ascii_unchanged() { assert_eq!(truncate_display("hello", 60), "hello"); diff --git a/atomic-cli/src/commands/rpc.rs b/atomic-cli/src/commands/rpc.rs index 4ec10af0..f36d053c 100644 --- a/atomic-cli/src/commands/rpc.rs +++ b/atomic-cli/src/commands/rpc.rs @@ -6489,6 +6489,12 @@ pub(crate) fn workflow_snapshot( let response = session.get_workflow_snapshot(pb::GetWorkflowSnapshotRequest { repository: Some(session.reference.clone()), })?; + decode_workflow_snapshot(response) +} + +fn decode_workflow_snapshot( + response: pb::GetWorkflowSnapshotResponse, +) -> CliResult<atomic_repository::repository::WorkflowSnapshot> { if response.api_version != 1 || response.generation != atomic_core::kg::KG_GENERATION || response.ontology_version != atomic_core::kg::ontology::VERSION_IRI @@ -6510,6 +6516,77 @@ pub(crate) fn workflow_snapshot( }) } +/// Compose existing local/reactor read endpoints without opening redb client-side. +/// The selection itself runs purely over one consistent workflow snapshot. +pub(crate) fn intent_graph( + id: &str, + max_nodes: usize, +) -> CliResult<atomic_repository::intent_graph::IntentGraph> { + let root = super::find_repository_root()?; + let session = Service::open_root(&root)?.ok_or_else(|| CliError::InvalidArgument { + message: "intent graphs require a local repository service (local or reactor mode)".into(), + })?; + let canonical = atomic_core::kg::iri::parse_intent(id) + .ok() + .map(|_| id.to_owned()); + let uid = if canonical.is_none() { + let response = session.get_vault_entry(pb::GetVaultEntryRequest { + repository: Some(session.reference.clone()), + kind: Kind::Intent as i32, + id: id.into(), + path: None, + include_bundle: true, + view: None, + })?; + let bundle = response + .entry_bundle + .as_ref() + .map(parse_entry_bundle) + .transpose()? + .ok_or_else(|| { + CliError::Internal(anyhow::anyhow!("intent service returned no entry bundle")) + })?; + let inputs = crate::commands::intent::bridge::inputs_from_entry(&bundle.entry)?; + Some( + inputs + .frontmatter + .get("uid") + .and_then(|v| v.as_str()) + .ok_or_else(|| CliError::InvalidArgument { + message: "intent has no canonical UID".into(), + })? + .to_uppercase(), + ) + } else { + None + }; + let snapshot = decode_workflow_snapshot(session.get_workflow_snapshot( + pb::GetWorkflowSnapshotRequest { + repository: Some(session.reference.clone()), + }, + )?)?; + let canonical = match canonical { + Some(id) => id, + None => { + let candidates: Vec<_> = snapshot + .nodes + .iter() + .filter(|node| { + atomic_core::kg::iri::parse_intent(&node.id) + .ok() + .is_some_and(|(_, value)| Some(&value) == uid.as_ref()) + }) + .collect(); + if candidates.len() != 1 { + return Err(CliError::InvalidArgument { message: "intent is missing or ambiguous in the workflow snapshot; run atomic query enrich".into() }); + } + candidates[0].id.clone() + } + }; + atomic_repository::intent_graph::IntentGraph::from_snapshot(&snapshot, &canonical, max_nodes) + .map_err(CliError::Repository) +} + // --------------------------------------------------------------------------- // vault init / project init (bootstrap boundary) // --------------------------------------------------------------------------- diff --git a/atomic-cli/tests/fixtures/graph_canvas_test.js b/atomic-cli/tests/fixtures/graph_canvas_test.js new file mode 100644 index 00000000..8fb317bf --- /dev/null +++ b/atomic-cli/tests/fixtures/graph_canvas_test.js @@ -0,0 +1,82 @@ +// Injected only by the opt-in headless Chrome regression test. +setTimeout(function() { + var results = document.createElement('pre'); + results.id = 'canvas-test-results'; + document.body.appendChild(results); + function assert(value, message) { if (!value) throw new Error(message); } + try { + assert(window.d3, 'D3 must load for the canvas interaction regression'); + assert(!window.graphTestErrors.length, 'export must initialize without JavaScript errors'); + var canvas = document.getElementById('graph'); + var pane = document.getElementById('node-inspector'); + var picker = document.getElementById('node-picker'); + var dataScript = Array.from(document.scripts).find(function(script) { return script.textContent.includes('var graphData = '); }); + var graphData = JSON.parse(dataScript.textContent.split('var graphData = ')[1].split(';\n')[0]); + var colors = new Map(graphData.nodes.map(function(node) { return [node.kind, node.color]; })); + var legendRows = document.querySelectorAll('#legend .legend-item'); + assert(legendRows.length === colors.size, 'every visible type needs a legend entry'); + legendRows.forEach(function(row) { + var expected = document.createElement('span'); + expected.style.backgroundColor = colors.get(row.dataset.kind); + assert(row.querySelector('.legend-dot').style.backgroundColor === expected.style.backgroundColor, 'legend and canvas must use the same exported type color'); + }); + assert(new Set(colors.values()).size === colors.size, 'known visible types must have distinct colors'); + ['https://atomic.dev/ns/workflow#Change', 'https://atomic.dev/ns/workflow#FeatureIntent'].forEach(function(kind) { + var usageNode = graphData.nodes.find(function(node) { return node.kind === kind; }); + assert(usageNode, 'usage fixture node must be exported'); + picker.value = usageNode.id; + picker.dispatchEvent(new Event('change')); + var usagePane = pane.querySelector('.inspector-ai-usage'); + assert(usagePane && usagePane.textContent.includes('$0.125000 USD'), 'change/intent must show recorded cost'); + assert(usagePane.textContent.includes('Cache read') && usagePane.textContent.includes('Recorded total'), 'token categories must be shown'); + assert(pane.textContent.includes('graph-test-model') && pane.textContent.includes('Vendor: openai') && pane.textContent.includes('Tool: CLI: graph-test-agent'), 'model, vendor and tool attribution must be visible'); + }); + document.dispatchEvent(new KeyboardEvent('keydown', {key: 'Escape'})); + var view = document.getElementById('graph-view').getBoundingClientRect(); + var panel = pane.getBoundingClientRect(); + if (window.innerWidth > 760) assert(view.right <= panel.left + 1, 'right pane must not cover graph'); + else assert(view.bottom <= panel.top + 1 && panel.width > window.innerWidth - 4, 'narrow layout must stack pane below graph'); + var rect = canvas.getBoundingClientRect(); + var point = null; + for (var y = 8; y < rect.height && !point; y += 8) { + for (var x = 8; x < rect.width; x += 8) { + canvas.dispatchEvent(new MouseEvent('click', {clientX: rect.left + x, clientY: rect.top + y, view: window})); + if (pane.dataset.selectedNode) { point = [rect.left + x, rect.top + y]; break; } + } + } + assert(point, 'canvas click must select a visible node'); + var first = pane.dataset.selectedNode; + var other = Array.from(picker.options).find(function(option) { return option.value && option.value !== first; }); + assert(other, 'fixture must have multiple selectable nodes'); + function selectOther() { picker.value = other.value; picker.dispatchEvent(new Event('change')); } + function mouse(target, type, x, y) { + target.dispatchEvent(new MouseEvent(type, {clientX: x, clientY: y, bubbles: true, view: window, button: 0})); + } + selectOther(); + mouse(canvas, 'mousedown', point[0], point[1]); + mouse(window, 'mouseup', point[0], point[1]); + mouse(canvas, 'click', point[0], point[1]); + assert(pane.dataset.selectedNode === first, 'real down/up/click gesture must select a node'); + assert(pane.textContent.includes('Canonical ID') && pane.textContent.includes('Metadata'), 'click shows rich details'); + selectOther(); + mouse(canvas, 'mousedown', point[0], point[1]); + mouse(canvas, 'mousemove', point[0] + 25, point[1] + 15); + mouse(window, 'mouseup', point[0] + 25, point[1] + 15); + mouse(canvas, 'click', point[0] + 25, point[1] + 15); + assert(pane.dataset.selectedNode === other.value, 'dragging must not change pinned selection'); + canvas.dispatchEvent(new WheelEvent('wheel', {clientX: point[0], clientY: point[1], deltaY: -80, bubbles: true, view: window})); + assert(pane.dataset.selectedNode === other.value, 'zoom must preserve pinned selection'); + var neighbor = pane.querySelector('.inspector-neighbor'); + assert(neighbor, 'relationships must be navigable in the real export'); + neighbor.click(); + assert(pane.dataset.selectedNode && pane.dataset.selectedNode !== other.value, 'relationship click changes selected node'); + document.dispatchEvent(new KeyboardEvent('keydown', {key: 'Escape'})); + assert(!pane.dataset.selectedNode, 'Escape clears real canvas selection'); + assert(!window.graphTestErrors.length, 'canvas and inspector interactions must not throw JavaScript errors'); + document.body.dataset.canvasTestStatus = 'passed'; + results.textContent = 'Canvas selection, dragging, zoom, relationship navigation and layout passed'; + } catch (error) { + document.body.dataset.canvasTestStatus = 'failed'; + results.textContent = error.stack; + } +}, 1200); diff --git a/atomic-cli/tests/fixtures/graph_inspector_test.html b/atomic-cli/tests/fixtures/graph_inspector_test.html new file mode 100644 index 00000000..eee3207b --- /dev/null +++ b/atomic-cli/tests/fixtures/graph_inspector_test.html @@ -0,0 +1,98 @@ +<!DOCTYPE html> +<html lang="en"> +<meta charset="utf-8"> +<title>Atomic graph inspector DOM regression + + +

+
+
+
+
diff --git a/atomic-cli/tests/intent_graph_test.rs b/atomic-cli/tests/intent_graph_test.rs
new file mode 100644
index 00000000..596f93ad
--- /dev/null
+++ b/atomic-cli/tests/intent_graph_test.rs
@@ -0,0 +1,453 @@
+use atomic_core::{
+    change::{Change, ChangeHeader, ProvenanceGraph},
+    kg::iri,
+    pristine::VaultEntryType,
+    types::Base32,
+};
+use atomic_repository::Repository;
+use std::{
+    path::Path,
+    process::{Command, Output},
+};
+
+const UID: &str = "01ARZ3NDEKTSV4RRFFQ69G5FAV";
+const KEY: &str = "TEST::tester::1";
+
+fn command(root: &Path, args: &[&str]) -> Command {
+    let mut command = Command::new(env!("CARGO_BIN_EXE_atomic"));
+    command
+        .current_dir(root)
+        .args(args)
+        .env("ATOMIC_SERVICE", "local")
+        .env_remove("ATOMIC_RPC");
+    command
+}
+fn success(output: Output) -> String {
+    assert!(
+        output.status.success(),
+        "{}",
+        String::from_utf8_lossy(&output.stderr)
+    );
+    String::from_utf8(output.stdout).unwrap()
+}
+fn fixture(root: &Path) -> String {
+    let repo = Repository::init(root).unwrap();
+    repo.init_vault().unwrap();
+    repo.vault_store(&format!("intents/{UID}/intent.md"), VaultEntryType::Intent,
+        b":::why\nInspect exact lineage.\n:::\n".to_vec(),
+        serde_json::json!({"uid":UID,"id":KEY,"title":"Graph ","kind":"feature","status":"in_progress"}).to_string()).unwrap();
+    repo.vault_reconcile_intent_manifest().unwrap();
+    let mut bytes = vec![];
+    let mut change = Change::empty(ChangeHeader::new("Exact recorded change"));
+    let mut provenance = atomic_core::change::Provenance::builder()
+        .vendor(atomic_core::change::AIVendor::OpenAI)
+        .model("graph-test-model")
+        .tool(atomic_core::change::AITool::cli("graph-test-agent"))
+        .request_id("graph-test-usage")
+        .cost_usd(0.125)
+        .build();
+    provenance.tokens = atomic_core::change::TokenUsage::full(100, 40, 10, 60, 20);
+    change.add_provenance(provenance);
+    let hash = change.serialize(&mut bytes).unwrap();
+    repo.import_change_bytes(&hash, &bytes).unwrap();
+    repo.save_provenance_graph(
+        &ProvenanceGraph::builder("graph-test-session", "opencode")
+            .plan_id(UID)
+            .add_change_explained(hash)
+            .build(),
+    )
+    .unwrap();
+    repo.rebuild_kg(true).unwrap();
+    let root = iri::mint_intent(&repo.project_id().unwrap(), UID).unwrap();
+    assert_eq!(repo.intent_graph(KEY, 100).unwrap().root, root);
+    assert_eq!(repo.intent_graph(&root, 100).unwrap().root, root);
+    root
+}
+
+#[test]
+fn graph_resolves_ids_renders_all_formats_and_preserves_database() {
+    let directory = tempfile::tempdir().unwrap();
+    let root = directory.path();
+    let canonical = fixture(root);
+    let database = root.join(".atomic/atomic.redb");
+    let before = std::fs::read(&database).unwrap();
+    let mut expected = None;
+    for id in [KEY, UID, canonical.as_str()] {
+        let output = success(
+            command(root, &["intent", "graph", id, "--json"])
+                .output()
+                .unwrap(),
+        );
+        let json: serde_json::Value = serde_json::from_str(&output).unwrap();
+        assert_eq!(json["root"], canonical);
+        assert_eq!(json["implementing_changes"].as_array().unwrap().len(), 1);
+        assert_eq!(json["truncated"], false);
+        let root_node = json["nodes"]
+            .as_array()
+            .unwrap()
+            .iter()
+            .find(|node| node["id"] == canonical)
+            .unwrap();
+        assert_eq!(
+            root_node["metadata"]["ai_usage"]["cost"]["amount"],
+            "0.125000"
+        );
+        assert_eq!(
+            root_node["metadata"]["ai_usage"]["contributors"][0]["model"],
+            "graph-test-model"
+        );
+        let change_id = json["implementing_changes"][0].as_str().unwrap();
+        let neighbors: serde_json::Value = serde_json::from_str(&success(
+            command(root, &["query", "neighbors", change_id, "--json"])
+                .output()
+                .unwrap(),
+        ))
+        .unwrap();
+        let change_node = neighbors["nodes"]
+            .as_array()
+            .unwrap()
+            .iter()
+            .find(|node| node["id"] == change_id)
+            .unwrap();
+        assert_eq!(
+            change_node["metadata"]["ai_usage"]["cost"]["amount"],
+            "0.125000"
+        );
+        if let Some(expected) = &expected {
+            assert_eq!(&json, expected);
+        }
+        expected = Some(json);
+    }
+    let text = success(command(root, &["intent", "graph", KEY]).output().unwrap());
+    assert!(text.contains("implementing change"));
+    assert!(text.contains("atomic:generatedChange"));
+    let dot = success(
+        command(root, &["intent", "graph", KEY, "--dot"])
+            .output()
+            .unwrap(),
+    );
+    assert!(dot.contains("digraph query"));
+    assert!(dot.contains("atomic:implements"));
+    let html_path = root.join("intent.html");
+    success(
+        command(
+            root,
+            &[
+                "intent",
+                "graph",
+                KEY,
+                "--html",
+                "--no-open",
+                "-o",
+                html_path.to_str().unwrap(),
+            ],
+        )
+        .output()
+        .unwrap(),
+    );
+    let html = std::fs::read_to_string(html_path).unwrap();
+    assert!(html.contains(""));
+    assert!(!html.contains(""));
+    assert!(html.contains("\\u003c/script\\u003e"));
+    assert!(html.contains("id=\"node-inspector\""));
+    assert!(html.contains("createGraphInspector"));
+    assert!(html.contains("canvas.addEventListener('click'"));
+    assert!(html.contains("Incoming relationships"));
+    let capped = success(
+        command(
+            root,
+            &["intent", "graph", KEY, "--json", "--max-nodes", "1"],
+        )
+        .output()
+        .unwrap(),
+    );
+    let capped: serde_json::Value = serde_json::from_str(&capped).unwrap();
+    assert_eq!(capped["nodes"].as_array().unwrap().len(), 1);
+    assert_eq!(capped["truncated"], true);
+    assert_eq!(
+        std::fs::read(database).unwrap(),
+        before,
+        "graph reads must not rebuild or mutate the KG"
+    );
+}
+
+/// Run explicitly on machines with Chrome; normal tests never launch GUI apps.
+#[test]
+#[ignore = "requires Chrome; set ATOMIC_TEST_CHROME and run with --ignored"]
+fn graph_inspector_dom_interactions_in_chrome() {
+    let fixture =
+        Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures/graph_inspector_test.html");
+    let dom = chrome_dom(&fixture, "1200,900");
+    assert!(dom.contains("data-test-status=\"passed\""), "{dom}");
+}
+
+fn chrome_dom(path: &Path, window_size: &str) -> String {
+    let chrome = std::env::var_os("ATOMIC_TEST_CHROME")
+        .expect("set ATOMIC_TEST_CHROME to a Chrome/Chromium executable");
+    let profile = tempfile::tempdir().unwrap();
+    let url = url::Url::from_file_path(path).unwrap();
+    let output = std::process::Command::new(chrome)
+        .args([
+            "--headless=new",
+            "--disable-gpu",
+            "--no-first-run",
+            "--disable-background-networking",
+            "--disable-extensions",
+            "--dump-dom",
+            "--virtual-time-budget=3000",
+            "--allow-file-access-from-files",
+        ])
+        .arg(format!("--window-size={window_size}"))
+        .arg(format!("--user-data-dir={}", profile.path().display()))
+        .arg(url.as_str())
+        .output()
+        .unwrap();
+    success(output)
+}
+
+#[test]
+#[ignore = "requires Chrome and D3; set ATOMIC_TEST_CHROME and optionally ATOMIC_TEST_D3"]
+fn exported_graph_canvas_selection_drag_zoom_and_responsive_layout() {
+    let directory = tempfile::tempdir().unwrap();
+    fixture(directory.path());
+    let mut html = success(
+        command(
+            directory.path(),
+            &["intent", "graph", KEY, "--html", "--no-open"],
+        )
+        .output()
+        .unwrap(),
+    );
+    if let Some(d3) = std::env::var_os("ATOMIC_TEST_D3") {
+        let url = url::Url::from_file_path(std::fs::canonicalize(d3).unwrap()).unwrap();
+        html = html.replace("https://cdn.jsdelivr.net/npm/d3@7", url.as_str());
+    }
+    html = html.replace("", "");
+    html = html.replace(
+        "",
+        &format!(
+            "",
+            include_str!("fixtures/graph_canvas_test.js")
+        ),
+    );
+    let path = directory.path().join("canvas-inspector.html");
+    std::fs::write(&path, html).unwrap();
+    for size in ["1200,900", "600,900"] {
+        let dom = chrome_dom(&path, size);
+        assert!(
+            dom.contains("data-canvas-test-status=\"passed\""),
+            "{size}: {dom}"
+        );
+    }
+}
+
+#[test]
+fn graph_rejects_unknown_intents_conflicting_formats_and_invalid_limits() {
+    let directory = tempfile::tempdir().unwrap();
+    fixture(directory.path());
+    for args in [
+        vec!["intent", "graph", "TEST::tester::999", "--json"],
+        vec!["intent", "graph", KEY, "--json", "--html"],
+        vec!["intent", "graph", KEY, "--max-nodes", "0"],
+        vec!["intent", "graph", KEY, "--max-nodes", "10001"],
+        vec!["intent", "graph", KEY, "--no-open"],
+    ] {
+        assert!(!command(directory.path(), &args)
+            .output()
+            .unwrap()
+            .status
+            .success());
+    }
+}
+
+#[test]
+fn graph_distinguishes_explicit_declaration_from_captured_implementation() {
+    use atomic_repository::IntentUpdateOptions;
+    let directory = tempfile::tempdir().unwrap();
+    fixture(directory.path());
+    let repo = Repository::open(directory.path()).unwrap();
+    let mut bytes = vec![];
+    let hash = Change::empty(ChangeHeader::new("Mixed parent change"))
+        .serialize(&mut bytes)
+        .unwrap();
+    repo.import_change_bytes(&hash, &bytes).unwrap();
+    let entry = repo.vault_intent_show(KEY).unwrap();
+    let body = format!("{}\n::ref{{to={} edge=declaresImplementation declarationEvidence=\"Author explicitly repairs a lost child association; no original plan provenance is claimed.\"}}\n", String::from_utf8(entry.content_bytes).unwrap(), iri::mint_change(&hash));
+    repo.vault_intent_update(
+        KEY,
+        IntentUpdateOptions {
+            content: Some(body),
+            force: true,
+            ..Default::default()
+        },
+    )
+    .unwrap();
+    drop(repo);
+    let json: serde_json::Value = serde_json::from_str(&success(
+        command(directory.path(), &["intent", "graph", KEY, "--json"])
+            .output()
+            .unwrap(),
+    ))
+    .unwrap();
+    assert_eq!(json["implementing_changes"].as_array().unwrap().len(), 2);
+    assert_eq!(json["captured_changes"].as_array().unwrap().len(), 1);
+    assert_eq!(
+        json["declared_changes"],
+        serde_json::json!([iri::mint_change(&hash)])
+    );
+    let text = success(
+        command(directory.path(), &["intent", "graph", KEY])
+            .output()
+            .unwrap(),
+    );
+    assert!(text.contains("declared implementation; original turn provenance not established"));
+    assert!(text.contains("Declaration evidence: Author explicitly repairs"));
+}
+
+/// The demo-2 failure: real, fully recorded source and intent artifacts exist,
+/// but there is no plan_id, implementation assertion, or retrospective claim.
+#[cfg(unix)]
+#[test]
+fn graph_exposes_real_recording_evidence_without_plan_id_or_database_mutation() {
+    use std::fs;
+    let directory = tempfile::tempdir().unwrap();
+    let home = tempfile::tempdir().unwrap();
+    let root = directory.path();
+    let run = |args: &[&str]| {
+        success(
+            command(root, args)
+                .env("HOME", home.path())
+                .env("ATOMIC_CONFIG_DIR", home.path().join(".atomic"))
+                .env("ATOMIC_NONINTERACTIVE", "1")
+                .env_remove("ATOMIC_SESSION_ID")
+                .env_remove("ATOMIC_AGENT_IDENTITY")
+                .output()
+                .unwrap(),
+        )
+    };
+    run(&[
+        "identity",
+        "new",
+        "tester",
+        "--email",
+        "tester@example.invalid",
+        "--set-default",
+    ]);
+    run(&["init"]);
+    {
+        let repo = Repository::open(root).unwrap();
+        repo.vault_store(&format!("intents/{UID}/intent.md"), VaultEntryType::Intent,
+            b":::why\nBuild the CLI.\n:::\n".to_vec(),
+            serde_json::json!({"uid":UID,"id":KEY,"title":"Build the CLI","kind":"feature","status":"in_progress"}).to_string()).unwrap();
+        repo.vault_reconcile_intent_manifest().unwrap();
+    }
+    let artifact_path = format!(".vault/intents/{UID}/intent.md");
+    fs::create_dir_all(root.join(format!(".vault/intents/{UID}"))).unwrap();
+    fs::write(root.join(&artifact_path), format!("---\nid: {KEY}\nuid: {UID}\ntitle: Build the CLI\nstatus: in_progress\n---\n:::why\nBuild the CLI.\n:::\n")).unwrap();
+    fs::create_dir_all(root.join("src")).unwrap();
+    fs::write(root.join("src/cli.ts"), "console.log('Hello, world!');\n").unwrap();
+    run(&["add", &artifact_path, "src/cli.ts"]);
+    run(&["record", "-m", "Record the CLI and its intent"]);
+    let log: serde_json::Value =
+        serde_json::from_str(&run(&["log", "-n", "1", "-f", "json"])).unwrap();
+    let recorded_hash =
+        atomic_core::types::Hash::from_base32(log[0]["hash"].as_str().unwrap().as_bytes()).unwrap();
+    let recorded = iri::mint_change(&recorded_hash);
+    // A later change shares the source file but not the intent artifact.
+    fs::write(root.join("src/cli.ts"), "console.log('Unrelated edit');\n").unwrap();
+    run(&["record", "-m", "Another change to the shared source file"]);
+    {
+        let repo = Repository::open(root).unwrap();
+        repo.rebuild_kg(true).unwrap();
+    }
+    let database = root.join(".atomic/atomic.redb");
+    let before = fs::read(&database).unwrap();
+    let json: serde_json::Value =
+        serde_json::from_str(&run(&["intent", "graph", KEY, "--json"])).unwrap();
+    assert_eq!(json["recording_changes"], serde_json::json!([recorded]));
+    assert_eq!(json["implementing_changes"], serde_json::json!([]));
+    assert_eq!(json["captured_changes"], serde_json::json!([]));
+    assert_eq!(json["declared_changes"], serde_json::json!([]));
+    assert!(json["nodes"]
+        .as_array()
+        .unwrap()
+        .iter()
+        .any(|n| n["metadata"]["path"] == "src/cli.ts"));
+    let text = run(&["intent", "graph", KEY]);
+    assert!(text.contains("1 intent-artifact recording changes"));
+    assert!(text.contains("records intent artifact; implementation attribution not asserted"));
+    assert!(!text.contains("No implementing changes are linked"));
+    let dot = run(&["intent", "graph", KEY, "--dot"]);
+    assert!(dot.contains("records intent artifact"));
+    let html = run(&["intent", "graph", KEY, "--html", "--no-open"]);
+    assert!(html.contains("1 intent-artifact recording"));
+    assert_eq!(
+        fs::read(database).unwrap(),
+        before,
+        "semantic graph reads must not write to the repository"
+    );
+}
+
+#[cfg(unix)]
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn graph_has_local_reactor_parity_through_shared_read_endpoints() {
+    use libatomic::atomic::{
+        daemon_service_server::DaemonServiceServer,
+        knowledge_service_server::KnowledgeServiceServer, vault_service_server::VaultServiceServer,
+    };
+    use libatomic::daemon::{
+        services::DaemonImpl,
+        services_agent::{KnowledgeImpl, VaultImpl},
+        state::DaemonState,
+    };
+    use std::sync::Arc;
+    let directory = tempfile::tempdir().unwrap();
+    let root = directory.path().to_path_buf();
+    fixture(&root);
+    let local = success(
+        command(&root, &["intent", "graph", KEY, "--json"])
+            .output()
+            .unwrap(),
+    );
+    // Keep the Unix socket short enough for sockaddr_un on macOS.
+    let socket_dir = tempfile::Builder::new()
+        .prefix("ig-")
+        .tempdir_in(std::env::temp_dir())
+        .unwrap();
+    let socket = socket_dir.path().join("s");
+    let listener = tokio::net::UnixListener::bind(&socket).unwrap();
+    let state = Arc::new(DaemonState::new());
+    let (stop, stopped) = tokio::sync::oneshot::channel();
+    let server = tokio::spawn(
+        tonic::transport::Server::builder()
+            .add_service(DaemonServiceServer::new(DaemonImpl {
+                state: state.clone(),
+            }))
+            .add_service(KnowledgeServiceServer::new(KnowledgeImpl {
+                state: state.clone(),
+            }))
+            .add_service(VaultServiceServer::new(VaultImpl { state }))
+            .serve_with_incoming_shutdown(
+                tokio_stream::wrappers::UnixListenerStream::new(listener),
+                async {
+                    let _ = stopped.await;
+                },
+            ),
+    );
+    let output = tokio::task::spawn_blocking(move || {
+        command(&root, &["intent", "graph", KEY, "--json"])
+            .env("ATOMIC_SERVICE", "reactor")
+            .env("ATOMIC_DAEMON_SOCKET", socket)
+            .output()
+            .unwrap()
+    })
+    .await
+    .unwrap();
+    let _ = stop.send(());
+    server.await.unwrap().unwrap();
+    let reactor = success(output);
+    assert_eq!(
+        serde_json::from_str::(&local).unwrap(),
+        serde_json::from_str::(&reactor).unwrap()
+    );
+}
diff --git a/atomic-cli/tests/intent_turn_binding_test.rs b/atomic-cli/tests/intent_turn_binding_test.rs
new file mode 100644
index 00000000..163b96e4
--- /dev/null
+++ b/atomic-cli/tests/intent_turn_binding_test.rs
@@ -0,0 +1,288 @@
+//! Authoring → work start → attestation → real hook recording. No fixture
+//! supplies plan_id or manufactures a provenance graph/SessionTurn.
+#![cfg(unix)]
+
+use atomic_agent::turn::session::{ManagedRunStamp, SessionStore};
+use atomic_repository::Repository;
+use std::{
+    fs,
+    io::Write,
+    path::Path,
+    process::{Command, Output, Stdio},
+};
+
+struct Fixture {
+    root: tempfile::TempDir,
+    home: tempfile::TempDir,
+}
+
+impl Fixture {
+    fn new() -> Self {
+        let fixture = Self {
+            root: tempfile::tempdir().unwrap(),
+            home: tempfile::tempdir().unwrap(),
+        };
+        fixture.run(&[
+            "identity",
+            "new",
+            "tester",
+            "--email",
+            "tester@example.invalid",
+            "--set-default",
+        ]);
+        fixture.run(&["init"]);
+        fixture
+    }
+
+    fn root(&self) -> &Path {
+        self.root.path()
+    }
+
+    fn command(&self) -> Command {
+        let mut command = Command::new(env!("CARGO_BIN_EXE_atomic"));
+        command
+            .current_dir(self.root())
+            .env("HOME", self.home.path())
+            .env("ATOMIC_CONFIG_DIR", self.home.path().join(".atomic"))
+            .env("ATOMIC_SERVICE", "local")
+            .env("ATOMIC_NONINTERACTIVE", "1")
+            .env_remove("ATOMIC_RPC")
+            .env_remove("ATOMIC_SESSION_ID")
+            .env_remove("ATOMIC_AGENT_IDENTITY");
+        command
+    }
+
+    fn run(&self, args: &[&str]) -> String {
+        success(self.command().args(args).output().unwrap())
+    }
+
+    fn hook(&self, session: &str, verb: &str) -> serde_json::Value {
+        let mut child = self
+            .command()
+            .args(["agent", "hooks", "opencode", verb, "--foreground", "--json"])
+            .stdin(Stdio::piped())
+            .stdout(Stdio::piped())
+            .stderr(Stdio::piped())
+            .spawn()
+            .unwrap();
+        let payload = serde_json::json!({"session_id": session, "cwd": self.root(),
+            "prompt": "Implement a greeting", "model": "test", "provider": "test",
+            "response": "Implemented and tested a greeting"});
+        child
+            .stdin
+            .take()
+            .unwrap()
+            .write_all(payload.to_string().as_bytes())
+            .unwrap();
+        let text = success(child.wait_with_output().unwrap());
+        serde_json::from_str(text.lines().last().unwrap()).unwrap()
+    }
+
+    fn new_intent(&self) -> (String, String) {
+        let output = self.run(&["intent", "new", "Implement a greeting"]);
+        let id = output
+            .lines()
+            .find_map(|line| line.strip_prefix("Created intent: "))
+            .unwrap()
+            .to_string();
+        let repo = Repository::open_readonly(self.root()).unwrap();
+        let entry = repo.vault_intent_show(&id).unwrap();
+        let frontmatter: serde_json::Value = serde_json::from_str(&entry.frontmatter_json).unwrap();
+        (id, frontmatter["uid"].as_str().unwrap().to_string())
+    }
+}
+
+fn success(output: Output) -> String {
+    assert!(
+        output.status.success(),
+        "stdout: {}\nstderr: {}",
+        String::from_utf8_lossy(&output.stdout),
+        String::from_utf8_lossy(&output.stderr)
+    );
+    String::from_utf8(output.stdout).unwrap()
+}
+
+#[test]
+fn ordinary_opencode_authoring_records_explicit_governing_intent() {
+    for via_environment in [false, true] {
+        let f = Fixture::new();
+        let session = "ordinary-intent-session";
+        f.hook(session, "session-start");
+        f.hook(session, "user-prompt");
+        let (id, uid) = f.new_intent();
+        let path = f.root().join(format!(".vault/intents/{uid}/intent.md"));
+        let scaffold = fs::read_to_string(&path).unwrap();
+        let spine = scaffold.split(":::why").next().unwrap();
+        fs::write(&path, format!("{spine}:::why\nMake the greeting usable.\n:::\n\n:::acceptance-criterion{{#{uid}-ac-1 status=unmet}}\nA greeting exists.\n:::\n\n:::task{{#{uid}-1 status=open criteria={uid}-ac-1}}\nImplement the greeting.\n::file-ref{{path=greeting.txt}}\n:::\n")).unwrap();
+        f.run(&["vault", "sync"]);
+
+        let mut start = f.command();
+        start.args([
+            "intent",
+            "update",
+            &id,
+            "--status",
+            "in_progress",
+            "--turn",
+            "1",
+        ]);
+        if via_environment {
+            start.env("ATOMIC_SESSION_ID", session);
+        } else {
+            start.args(["--session-id", session]);
+        }
+        success(start.output().unwrap());
+
+        let store = SessionStore::for_repo(f.root()).unwrap();
+        assert!(store.load(session).unwrap().unwrap().managed_run.is_none());
+        assert_eq!(
+            store
+                .turn_intent_binding(session, 1)
+                .unwrap()
+                .unwrap()
+                .intent_uid,
+            uid
+        );
+        // Work starts only after the durable declaration is observable.
+        fs::write(f.root().join("greeting.txt"), "Hello, world!\n").unwrap();
+        assert_eq!(
+            fs::read_to_string(f.root().join("greeting.txt")).unwrap(),
+            "Hello, world!\n"
+        );
+        let intent = fs::read_to_string(&path).unwrap()
+            .replace("status=unmet", "status=met verifiedBy=\"greeting-test\" evidence=\"greeting.txt content asserted\"")
+            .replace("status=open", "status=done");
+        fs::write(&path, intent).unwrap();
+        f.run(&["vault", "sync"]);
+        f.run(&["intent", "attest", &id]);
+        f.run(&["intent", "validate", &id]);
+        f.run(&["intent", "update", &id, "--status", "needs-review"]);
+        let stop = f.hook(session, "stop");
+        assert_eq!(stop["recorded"], true);
+        {
+            let repo = Repository::open_readonly(f.root()).unwrap();
+            let (_, turns) = repo.get_session_ledger(session).unwrap().unwrap();
+            assert_eq!(turns.len(), 1);
+            assert_eq!(turns[0].plan_id.as_deref(), Some(uid.as_str()));
+            assert!(!turns[0].change_hashes.is_empty());
+            let graph = repo
+                .load_provenance_graph(&turns[0].provenance_hash)
+                .unwrap();
+            assert_eq!(graph.plan_id.as_deref(), Some(uid.as_str()));
+            assert_eq!(graph.changes_explained, turns[0].change_hashes);
+            let report = repo.intent_graph(&id, 1000).unwrap();
+            assert!(
+                !report.implementing_changes.is_empty(),
+                "the projection must see the real binding"
+            );
+        }
+        // A late declaration may not repair an already recorded turn.
+        assert!(store.bind_turn_intent(session, Some(1), &uid).is_err());
+        f.hook(session, "user-prompt");
+        assert!(store.turn_intent_binding(session, 2).unwrap().is_none());
+        fs::write(f.root().join("greeting.txt"), "Hello again!\n").unwrap();
+        f.hook(session, "stop");
+        let repo = Repository::open_readonly(f.root()).unwrap();
+        let (_, turns) = repo.get_session_ledger(session).unwrap().unwrap();
+        assert_eq!(turns.len(), 2);
+        assert_eq!(
+            turns[1].plan_id, None,
+            "never inherit the prior turn's intent"
+        );
+    }
+}
+
+#[test]
+fn binding_rejects_conflicts_stale_turns_and_concurrent_recording() {
+    let f = Fixture::new();
+    let session = "binding-guards";
+    f.hook(session, "session-start");
+    f.hook(session, "user-prompt");
+    let (id, uid) = f.new_intent();
+    let start = [
+        "intent",
+        "update",
+        &id,
+        "--status",
+        "in_progress",
+        "--session-id",
+        session,
+        "--turn",
+        "1",
+    ];
+    f.run(&start);
+    f.run(&start); // Idempotent retry.
+    let store = SessionStore::for_repo(f.root()).unwrap();
+    assert!(store.bind_turn_intent(session, Some(2), &uid).is_err());
+    assert!(store
+        .bind_turn_intent(session, Some(1), "01ARZ3NDEKTSV4RRFFQ69G5FAV")
+        .is_err());
+    let lock = fs::OpenOptions::new()
+        .write(true)
+        .open(
+            f.root()
+                .join(format!(".atomic/sessions/{session}/turn-end.lock")),
+        )
+        .unwrap();
+    fs2::FileExt::try_lock_exclusive(&lock).unwrap();
+    assert!(store.bind_turn_intent(session, Some(1), &uid).is_err());
+    drop(lock);
+    assert_eq!(
+        store
+            .turn_intent_binding(session, 1)
+            .unwrap()
+            .unwrap()
+            .intent_uid,
+        uid
+    );
+    assert!(store.bind_turn_intent("../escape", Some(1), &uid).is_err());
+}
+
+#[test]
+fn status_and_corecorded_intent_file_do_not_invent_governance() {
+    let f = Fixture::new();
+    let session = "status-only-session";
+    f.hook(session, "session-start");
+    f.hook(session, "user-prompt");
+    let (id, _) = f.new_intent();
+    f.run(&["intent", "update", &id, "--status", "in_progress"]);
+    fs::write(f.root().join("greeting.txt"), "Hello!\n").unwrap();
+    assert_eq!(f.hook(session, "stop")["recorded"], true);
+    let repo = Repository::open_readonly(f.root()).unwrap();
+    let (_, turns) = repo.get_session_ledger(session).unwrap().unwrap();
+    assert_eq!(turns.len(), 1);
+    assert_eq!(turns[0].plan_id, None);
+    assert_eq!(
+        repo.load_provenance_graph(&turns[0].provenance_hash)
+            .unwrap()
+            .plan_id,
+        None
+    );
+}
+
+#[test]
+fn managed_work_item_capture_remains_available_without_a_declaration() {
+    let f = Fixture::new();
+    let session = "managed-intent-session";
+    f.hook(session, "session-start");
+    f.hook(session, "user-prompt");
+    let (_, uid) = f.new_intent();
+    let store = SessionStore::for_repo(f.root()).unwrap();
+    let mut state = store.load(session).unwrap().unwrap();
+    state.managed_run = Some(ManagedRunStamp {
+        run_id: "test-managed-run".into(),
+        owner_agent: "test".into(),
+        owner_session_id: "owner".into(),
+        work_item_id: Some(uid.clone()),
+    });
+    store.save(&state).unwrap();
+    assert!(store
+        .bind_turn_intent(session, Some(1), "01ARZ3NDEKTSV4RRFFQ69G5FAV")
+        .is_err());
+    fs::write(f.root().join("greeting.txt"), "Hello!\n").unwrap();
+    f.hook(session, "stop");
+    let repo = Repository::open_readonly(f.root()).unwrap();
+    let (_, turns) = repo.get_session_ledger(session).unwrap().unwrap();
+    assert_eq!(turns[0].plan_id.as_deref(), Some(uid.as_str()));
+    assert!(store.turn_intent_binding(session, 1).unwrap().is_none());
+}
diff --git a/atomic-core/ontology/shapes/projection-1.0.0.ttl b/atomic-core/ontology/shapes/projection-1.0.0.ttl
index f0aa5bc1..1ac4e960 100644
--- a/atomic-core/ontology/shapes/projection-1.0.0.ttl
+++ b/atomic-core/ontology/shapes/projection-1.0.0.ttl
@@ -3,6 +3,19 @@
 @prefix sh:  .
 @prefix rdf:  .
 @prefix prov:  .
+@prefix xsd:  .
+
+# Optional for older snapshots; when asserted these are closed, single-valued
+# observations. SHACL checks consistency, not object-store truth: only the
+# repository projector has access to the canonical store and can verify it.
+shape:ChangeAvailability a sh:NodeShape ;
+  sh:targetClass atomic:Change ;
+  sh:property [ sh:path atomic:objectAvailability ; sh:maxCount 1 ; sh:datatype xsd:string ; sh:in ("unknown" "absent" "available") ] ;
+  sh:property [ sh:path atomic:projectionCompleteness ; sh:maxCount 1 ; sh:datatype xsd:string ; sh:in ("reference-only" "projected") ] ;
+  sh:or (
+    [ sh:not [ sh:property [ sh:path atomic:projectionCompleteness ; sh:hasValue "projected" ] ] ]
+    [ sh:property [ sh:path atomic:objectAvailability ; sh:hasValue "available" ] ]
+  ) .
 
 shape:GeneratedChange a sh:NodeShape ;
   sh:targetObjectsOf atomic:generatedChange ;
@@ -48,7 +61,7 @@ shape:ReferencedResource a sh:NodeShape ;
     atomic:directlyOnView, atomic:sessionViewContext, atomic:childViewOf,
     atomic:authoredBy, atomic:coversChange, atomic:attests, atomic:attestsSession,
     atomic:signedBy, atomic:previousAttestation, atomic:implements,
-    atomic:attestedBy, atomic:aboutResource, atomic:verificationResult,
+    atomic:attestedBy, atomic:aboutResource, atomic:verificationResult, atomic:declaresImplementation,
     atomic:hasIdentity, atomic:issuer, atomic:explainedBy, atomic:dependsOn,
     atomic:modifies, atomic:hasScopeItem, atomic:ancestorView,
     atomic:turnOfSession, atomic:governsTurn, atomic:generatedBy, atomic:reviewedBy,
diff --git a/atomic-core/ontology/workflow-1.0.0.ttl b/atomic-core/ontology/workflow-1.0.0.ttl
index 7be0a639..2ef6744c 100644
--- a/atomic-core/ontology/workflow-1.0.0.ttl
+++ b/atomic-core/ontology/workflow-1.0.0.ttl
@@ -73,7 +73,7 @@ atomic:dependsOnIntent a owl:ObjectProperty ; rdfs:domain atomic:WorkIntent ; rd
 atomic:modifiesFile a owl:ObjectProperty ; rdfs:domain atomic:Change ; rdfs:range atomic:File ; rdfs:subPropertyOf atomic:modifies .
 atomic:directlyOnView a owl:ObjectProperty ; rdfs:domain atomic:Change ; rdfs:range atomic:View .
 atomic:sessionViewContext a owl:ObjectProperty ; rdfs:domain atomic:Session ; rdfs:range atomic:View .
-atomic:childViewOf a owl:ObjectProperty ; rdfs:domain atomic:View ; rdfs:range atomic:View .
+atomic:childViewOf a owl:ObjectProperty ; rdfs:domain atomic:View ; rdfs:range atomic:View ; rdfs:subPropertyOf atomic:ancestorView .
 atomic:visibleFrom a owl:ObjectProperty ; rdfs:comment "Query-time visibility operator; never stored as direct membership." .
 atomic:authoredBy a owl:ObjectProperty ; rdfs:domain owl:Thing ; rdfs:range atomic:Identity ; rdfs:subPropertyOf prov:wasAttributedTo .
 atomic:coversChange a owl:ObjectProperty ; rdfs:domain atomic:Attestation ; rdfs:range atomic:Change ; owl:inverseOf atomic:attestedBy .
@@ -82,6 +82,7 @@ atomic:attestsSession a owl:ObjectProperty ; rdfs:domain atomic:Attestation ; rd
 atomic:signedBy a owl:ObjectProperty ; rdfs:domain atomic:Attestation ; rdfs:range atomic:Identity .
 atomic:previousAttestation a owl:ObjectProperty ; rdfs:domain atomic:Attestation ; rdfs:range atomic:Attestation .
 atomic:implements a owl:ObjectProperty ; rdfs:domain atomic:Change ; rdfs:range atomic:WorkIntent ; owl:propertyChainAxiom (atomic:generatedBy atomic:hasPlan) .
+atomic:declaresImplementation a owl:ObjectProperty ; rdfs:domain atomic:WorkIntent ; rdfs:range atomic:Change ; rdfs:comment "Explicit, evidence-bearing author declaration, potentially retrospective. Not an inferred implements shortcut and not proof of a generating turn." .
 atomic:attestedBy a owl:ObjectProperty ; rdfs:domain atomic:Change ; rdfs:range atomic:Attestation ; owl:inverseOf atomic:coversChange .
 atomic:aboutResource a owl:ObjectProperty ; rdfs:domain atomic:Finding ; rdfs:range owl:Thing .
 atomic:dependsOn a owl:ObjectProperty, owl:TransitiveProperty .
@@ -100,6 +101,10 @@ atomic:issuer a owl:ObjectProperty ; rdfs:domain atomic:ProjectIdentity ; rdfs:r
 atomic:status a owl:DatatypeProperty ; rdfs:range xsd:string .
 atomic:priority a owl:DatatypeProperty ; rdfs:range xsd:string .
 atomic:findingCode a owl:DatatypeProperty ; rdfs:domain atomic:Finding ; rdfs:range xsd:string .
+atomic:objectAvailability a owl:DatatypeProperty ; rdfs:domain atomic:Change ; rdfs:range xsd:string ;
+    rdfs:comment "Local canonical-object observation: unknown, absent, or available. Only successful verified loading establishes available; a ledger reference or graph registration does not. This derived observation is repository-local, not an immutable property of the change." .
+atomic:projectionCompleteness a owl:DatatypeProperty ; rdfs:domain atomic:Change ; rdfs:range xsd:string ;
+    rdfs:comment "reference-only means the node identifies a change without projecting its immutable object; projected means the verified local object's header, dependencies and eligible file operations were projected. This does not assert dependency closure, view membership, provenance or attestation completeness." .
 atomic:projectId a owl:DatatypeProperty ; rdfs:domain atomic:ProjectIdentity ; rdfs:range xsd:string .
 atomic:schemaVersion a owl:DatatypeProperty ; rdfs:domain atomic:ProjectIdentity ; rdfs:range xsd:nonNegativeInteger .
 atomic:turnNumber a owl:DatatypeProperty ; rdfs:domain atomic:SessionTurn ; rdfs:range xsd:nonNegativeInteger .
diff --git a/atomic-core/src/kg/mod.rs b/atomic-core/src/kg/mod.rs
index 60bde518..31e39b03 100644
--- a/atomic-core/src/kg/mod.rs
+++ b/atomic-core/src/kg/mod.rs
@@ -6,7 +6,7 @@ pub mod ontology;
 pub mod project_id;
 
 /// Storage generation of the derived canonical workflow graph.
-pub const KG_GENERATION: u32 = 1;
+pub const KG_GENERATION: u32 = 2;
 
 #[derive(Clone, Copy, Debug, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
 #[serde(rename_all = "snake_case")]
diff --git a/atomic-core/src/kg/ontology.rs b/atomic-core/src/kg/ontology.rs
index 2c023552..3fe1079d 100644
--- a/atomic-core/src/kg/ontology.rs
+++ b/atomic-core/src/kg/ontology.rs
@@ -167,7 +167,7 @@ predicates! {
     ModifiesFile => ("https://atomic.dev/ns/workflow#modifiesFile", "atomic:modifiesFile", domain(Change), range(File), None, Some(Modifies)),
     DirectlyOnView => ("https://atomic.dev/ns/workflow#directlyOnView", "atomic:directlyOnView", domain(Change), range(View), None, None),
     SessionViewContext => ("https://atomic.dev/ns/workflow#sessionViewContext", "atomic:sessionViewContext", domain(Session), range(View), None, None),
-    ChildViewOf => ("https://atomic.dev/ns/workflow#childViewOf", "atomic:childViewOf", domain(View), range(View), None, None),
+    ChildViewOf => ("https://atomic.dev/ns/workflow#childViewOf", "atomic:childViewOf", domain(View), range(View), None, Some(AncestorView)),
     VisibleFrom => ("https://atomic.dev/ns/workflow#visibleFrom", "atomic:visibleFrom", None, None, None, None),
     AuthoredBy => ("https://atomic.dev/ns/workflow#authoredBy", "atomic:authoredBy", Some(E(OwlThing)), range(Identity), None, Some(ProvWasAttributedTo)),
     CoversChange => ("https://atomic.dev/ns/workflow#coversChange", "atomic:coversChange", domain(Attestation), range(Change), Some(AttestedBy), None),
@@ -176,6 +176,7 @@ predicates! {
     SignedBy => ("https://atomic.dev/ns/workflow#signedBy", "atomic:signedBy", domain(Attestation), range(Identity), None, None),
     PreviousAttestation => ("https://atomic.dev/ns/workflow#previousAttestation", "atomic:previousAttestation", domain(Attestation), range(Attestation), None, None),
     Implements => ("https://atomic.dev/ns/workflow#implements", "atomic:implements", domain(Change), range(WorkIntent), None, None),
+    DeclaresImplementation => ("https://atomic.dev/ns/workflow#declaresImplementation", "atomic:declaresImplementation", domain(WorkIntent), range(Change), None, None),
     AttestedBy => ("https://atomic.dev/ns/workflow#attestedBy", "atomic:attestedBy", domain(Change), range(Attestation), Some(CoversChange), None),
     AboutResource => ("https://atomic.dev/ns/workflow#aboutResource", "atomic:aboutResource", domain(Finding), Some(Range::Class(E(OwlThing))), None, None),
     DependsOn => ("https://atomic.dev/ns/workflow#dependsOn", "atomic:dependsOn", None, None, None, None),
@@ -193,6 +194,8 @@ predicates! {
     Status => ("https://atomic.dev/ns/workflow#status", "atomic:status", None, Some(Range::Datatype(Datatype::String)), None, None),
     Priority => ("https://atomic.dev/ns/workflow#priority", "atomic:priority", None, Some(Range::Datatype(Datatype::String)), None, None),
     FindingCode => ("https://atomic.dev/ns/workflow#findingCode", "atomic:findingCode", domain(Finding), Some(Range::Datatype(Datatype::String)), None, None),
+    ObjectAvailability => ("https://atomic.dev/ns/workflow#objectAvailability", "atomic:objectAvailability", domain(Change), Some(Range::Datatype(Datatype::String)), None, None),
+    ProjectionCompleteness => ("https://atomic.dev/ns/workflow#projectionCompleteness", "atomic:projectionCompleteness", domain(Change), Some(Range::Datatype(Datatype::String)), None, None),
     ProjectId => ("https://atomic.dev/ns/workflow#projectId", "atomic:projectId", domain(ProjectIdentity), Some(Range::Datatype(Datatype::String)), None, None),
     SchemaVersion => ("https://atomic.dev/ns/workflow#schemaVersion", "atomic:schemaVersion", domain(ProjectIdentity), Some(Range::Datatype(Datatype::NonNegativeInteger)), None, None),
     TurnNumber => ("https://atomic.dev/ns/workflow#turnNumber", "atomic:turnNumber", domain(SessionTurn), Some(Range::Datatype(Datatype::NonNegativeInteger)), None, None),
diff --git a/atomic-core/src/pristine/txn/write/session_kg.rs b/atomic-core/src/pristine/txn/write/session_kg.rs
index d01299f2..33725241 100644
--- a/atomic-core/src/pristine/txn/write/session_kg.rs
+++ b/atomic-core/src/pristine/txn/write/session_kg.rs
@@ -322,7 +322,11 @@ impl WriteTxn<'_> {
                         &hash.to_base32()[..12],
                         "session_ledger",
                     )
-                    .with_metadata(serde_json::json!({"hash": hash.to_base32()})),
+                    // The ledger proves identity, not object-store availability.
+                    // Repository projection upgrades these conservative facts
+                    // after loading and verifying the immutable object.
+                    .with_metadata(serde_json::json!({"hash": hash.to_base32(),
+                        "object_availability": "unknown", "projection_completeness": "reference-only"})),
                 )?;
             }
             self.upsert_kg_edge(&edge(&id, &change, Predicate::GeneratedChange))?;
diff --git a/atomic-core/tests/kg_view_ancestry.rs b/atomic-core/tests/kg_view_ancestry.rs
new file mode 100644
index 00000000..b9809afe
--- /dev/null
+++ b/atomic-core/tests/kg_view_ancestry.rs
@@ -0,0 +1,165 @@
+//! Ancestry axioms describe entailments, not automatic KG query expansion.
+
+use atomic_core::{
+    kg::{
+        ontology::{Class, Predicate, TTL},
+        Direction,
+    },
+    pristine::{KgMutTxnT, KgTxnT, MutTxnT, Pristine},
+    KgEdge, KgNode,
+};
+use oxrdf::{Subject, Term};
+use oxttl::TurtleParser;
+use std::collections::BTreeSet;
+
+type Fact = (&'static str, Predicate, &'static str);
+
+fn asserted() -> BTreeSet {
+    BTreeSet::from([
+        ("urn:view:child", Predicate::ChildViewOf, "urn:view:parent"),
+        ("urn:view:parent", Predicate::ChildViewOf, "urn:view:root"),
+        ("urn:change:one", Predicate::DirectlyOnView, "urn:view:root"),
+    ])
+}
+
+// Test-only application of subproperty and transitivity rules. Production KG
+// queries do not run an OWL reasoner; keep that boundary explicit below.
+fn declared_closure(mut facts: BTreeSet) -> BTreeSet {
+    loop {
+        let previous = facts.clone();
+        for &(subject, predicate, object) in &previous {
+            if let Some(parent) = predicate.super_predicate() {
+                facts.insert((subject, parent, object));
+            }
+            if predicate.is_transitive() {
+                for &(next_subject, next_predicate, next_object) in &previous {
+                    if object == next_subject && predicate == next_predicate {
+                        facts.insert((subject, predicate, next_object));
+                    }
+                }
+            }
+        }
+        if facts == previous {
+            return facts;
+        }
+    }
+}
+
+#[test]
+fn turtle_and_registry_link_child_to_ancestor_not_the_reverse() {
+    let graph = TurtleParser::new()
+        .for_reader(TTL.as_bytes())
+        .collect::, _>>()
+        .unwrap();
+    let objects = |subject: &str, predicate: &str| -> BTreeSet {
+        graph
+            .iter()
+            .filter_map(|triple| match (&triple.subject, &triple.object) {
+                (Subject::NamedNode(s), Term::NamedNode(o))
+                    if s.as_str() == subject && triple.predicate.as_str() == predicate =>
+                {
+                    Some(o.as_str().to_owned())
+                }
+                _ => None,
+            })
+            .collect()
+    };
+    let subproperty = "http://www.w3.org/2000/01/rdf-schema#subPropertyOf";
+    assert_eq!(
+        objects(Predicate::ChildViewOf.iri(), subproperty),
+        BTreeSet::from([Predicate::AncestorView.iri().to_owned()])
+    );
+    assert!(objects(Predicate::AncestorView.iri(), subproperty).is_empty());
+    assert!(objects(Predicate::DirectlyOnView.iri(), subproperty).is_empty());
+    assert!(objects(
+        Predicate::ChildViewOf.iri(),
+        "http://www.w3.org/2002/07/owl#inverseOf"
+    )
+    .is_empty());
+    assert!(objects(
+        Predicate::AncestorView.iri(),
+        "http://www.w3.org/1999/02/22-rdf-syntax-ns#type"
+    )
+    .contains("http://www.w3.org/2002/07/owl#TransitiveProperty"));
+    assert_eq!(
+        Predicate::ChildViewOf.super_predicate(),
+        Some(Predicate::AncestorView)
+    );
+    assert_eq!(Predicate::ChildViewOf.inverse(), None);
+    assert!(!Predicate::ChildViewOf.is_transitive());
+    assert!(Predicate::AncestorView.is_transitive());
+    assert_eq!(Predicate::AncestorView.super_predicate(), None);
+    assert_eq!(Predicate::DirectlyOnView.super_predicate(), None);
+}
+
+#[test]
+fn three_level_ancestry_closure_preserves_direction_and_direct_membership() {
+    let facts = asserted();
+    let closure = declared_closure(facts.clone());
+    let mut expected = facts;
+    expected.extend([
+        ("urn:view:child", Predicate::AncestorView, "urn:view:parent"),
+        ("urn:view:parent", Predicate::AncestorView, "urn:view:root"),
+        ("urn:view:child", Predicate::AncestorView, "urn:view:root"),
+    ]);
+    // Exact equality excludes reverse/self ancestry, child->root direct
+    // parenthood, and inherited directlyOnView membership for the change.
+    assert_eq!(closure, expected);
+    assert!(!closure.iter().any(|(s, _, o)| s == o));
+    assert_eq!(
+        closure
+            .iter()
+            .filter(|(_, p, _)| *p == Predicate::DirectlyOnView)
+            .count(),
+        1
+    );
+}
+
+#[test]
+fn ordinary_kg_queries_return_asserted_edges_not_ancestry_entailments() {
+    let dir = tempfile::tempdir().unwrap();
+    let pristine = Pristine::open(dir.path().join("db")).unwrap();
+    let mut txn = pristine.write_txn().unwrap();
+    txn.init_kg().unwrap();
+    for id in ["urn:view:child", "urn:view:parent", "urn:view:root"] {
+        txn.upsert_kg_node(&KgNode::new(id, Class::View, id, "test"))
+            .unwrap();
+    }
+    txn.upsert_kg_node(&KgNode::new(
+        "urn:change:one",
+        Class::Change,
+        "change",
+        "test",
+    ))
+    .unwrap();
+    for (subject, predicate, object) in asserted() {
+        txn.upsert_kg_edge(&KgEdge::new(subject, object, predicate))
+            .unwrap();
+    }
+    txn.commit().unwrap();
+
+    let txn = pristine.read_txn().unwrap();
+    let outgoing = txn.get_kg_edges_from("urn:view:child").unwrap();
+    assert_eq!(outgoing.len(), 1);
+    assert_eq!(outgoing[0].kind, Predicate::ChildViewOf);
+    assert_eq!(outgoing[0].to_id, "urn:view:parent");
+    let ancestors = txn
+        .kg_neighbors_filtered(
+            "urn:view:child",
+            2,
+            Some(Predicate::AncestorView),
+            Direction::Outgoing,
+        )
+        .unwrap();
+    assert!(ancestors.edges.is_empty());
+    let parents = txn
+        .kg_neighbors_filtered(
+            "urn:view:child",
+            2,
+            Some(Predicate::ChildViewOf),
+            Direction::Outgoing,
+        )
+        .unwrap();
+    assert_eq!(parents.edges.len(), 2);
+    assert_eq!(txn.count_kg_edges().unwrap(), asserted().len());
+}
diff --git a/atomic-repository/src/intent_graph.rs b/atomic-repository/src/intent_graph.rs
new file mode 100644
index 00000000..6fea26af
--- /dev/null
+++ b/atomic-repository/src/intent_graph.rs
@@ -0,0 +1,405 @@
+//! Intent-rooted, predicate-scoped selection over one semantic workflow snapshot.
+//!
+//! Shared sessions, files and identities are boundaries, not routes to unrelated
+//! work. This is a read model, not OWL inference or source-code interpretation.
+
+use std::collections::{BTreeMap, BTreeSet, VecDeque};
+
+use atomic_core::{
+    kg::{
+        iri,
+        ontology::{Class as C, Predicate as P},
+    },
+    KgEdge, KgNode,
+};
+use serde::Serialize;
+
+use crate::{repository::WorkflowSnapshot, Repository, RepositoryError};
+
+pub const DEFAULT_MAX_NODES: usize = 5000;
+pub const HARD_MAX_NODES: usize = 10_000;
+
+#[derive(Clone, Debug, Serialize)]
+pub struct IntentGraph {
+    pub schema_version: u32,
+    pub root: String,
+    pub snapshot_identity: String,
+    pub generation: u32,
+    pub ontology_version: String,
+    /// Completeness is relative to the asserted snapshot, not all historical work.
+    pub scope: &'static str,
+    pub nodes: Vec,
+    pub edges: Vec,
+    pub implementing_changes: Vec,
+    /// Implementation backed by a captured turn path or its existing shortcut.
+    pub captured_changes: Vec,
+    /// Explicit author declarations; never imply reconstructed turn provenance.
+    pub declared_changes: Vec,
+    /// The selected intent's own vault artifact, joined from stored vault_path
+    /// metadata to a canonical File node. No intent-to-file edge is synthesized.
+    pub intent_artifact: Option,
+    /// Changes with an asserted modifiesFile edge to that exact artifact.
+    /// Recording an intent is evidence, not an implementation assertion.
+    pub recording_changes: Vec,
+    pub dependency_changes: Vec,
+    pub truncated: bool,
+    pub missing_resources: Vec,
+    pub diagnostics: Vec,
+}
+
+impl Repository {
+    /// Resolve a human intent key, UID, or canonical intent IRI and select its
+    /// asserted semantic lineage. Does not enrich/rebuild the graph or read files.
+    pub fn intent_graph(&self, id: &str, max_nodes: usize) -> Result {
+        let uid = if let Ok((project, uid)) = iri::parse_intent(id) {
+            if project != self.project_id()? {
+                return Err(invalid("intent belongs to a different project"));
+            }
+            uid
+        } else {
+            let inputs = crate::intent_bridge::read_intent(self, id)?;
+            inputs
+                .frontmatter
+                .get("uid")
+                .and_then(|v| v.as_str())
+                .ok_or_else(|| invalid("intent has no canonical UID"))?
+                .to_uppercase()
+        };
+        let root = iri::mint_intent(&self.project_id()?, &uid)
+            .map_err(|error| invalid(&error.to_string()))?;
+        IntentGraph::from_snapshot(&self.workflow_snapshot()?, &root, max_nodes)
+    }
+}
+
+fn invalid(message: &str) -> RepositoryError {
+    RepositoryError::InvalidOperation {
+        message: message.into(),
+    }
+}
+
+/// Traversal roles are deliberately separate from class assertions: a related
+/// intent is a boundary unless it is the selected root.
+#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
+enum Role {
+    Root,
+    IntentArtifact,
+    Detail,
+    Change,
+    Turn,
+    Attestation,
+    View,
+}
+
+impl IntentGraph {
+    /// Select from a consistent asserted snapshot. Limits stop traversal rather
+    /// than silently treating a depth-limited neighborhood as complete lineage.
+    pub fn from_snapshot(
+        snapshot: &WorkflowSnapshot,
+        root: &str,
+        max_nodes: usize,
+    ) -> Result {
+        if !(1..=HARD_MAX_NODES).contains(&max_nodes) {
+            return Err(invalid("max-nodes must be between 1 and 10000"));
+        }
+        if snapshot.generation != atomic_core::kg::KG_GENERATION
+            || snapshot.ontology_version != atomic_core::kg::ontology::VERSION_IRI
+        {
+            return Err(invalid(
+                "unsupported workflow snapshot generation or ontology; rebuild the knowledge graph",
+            ));
+        }
+        iri::parse_intent(root).map_err(|error| invalid(&error.to_string()))?;
+        // These three resources are defined by the embedded ontology rather than
+        // persisted as KG rows. Include them when an asserted edge references one.
+        let statuses: Vec<_> = ["Verified", "Failed", "Unverifiable"]
+            .into_iter()
+            .map(|name| {
+                KgNode::new(
+                    format!("https://atomic.dev/ns/workflow#{name}"),
+                    C::VerificationStatus,
+                    name,
+                    "ontology",
+                )
+            })
+            .collect();
+        let nodes: BTreeMap<&str, &KgNode> = snapshot
+            .nodes
+            .iter()
+            .chain(statuses.iter())
+            .map(|n| (n.id.as_str(), n))
+            .collect();
+        if !nodes.get(root).is_some_and(|node| {
+            node.has_type(C::WorkIntent)
+                || node
+                    .types
+                    .iter()
+                    .filter_map(|t| C::from_iri(t))
+                    .any(|c| c.parents().iter().any(|p| p.iri() == C::WorkIntent.iri()))
+        }) {
+            return Err(invalid("intent is not present in the workflow snapshot; run atomic query enrich to rebuild it"));
+        }
+        let mut outgoing: BTreeMap<&str, Vec<&KgEdge>> = BTreeMap::new();
+        let mut incoming: BTreeMap<&str, Vec<&KgEdge>> = BTreeMap::new();
+        for edge in &snapshot.edges {
+            outgoing.entry(&edge.from_id).or_default().push(edge);
+            incoming.entry(&edge.to_id).or_default().push(edge);
+        }
+        // Sorting makes capped selection independent of storage/serialization order.
+        for edges in outgoing.values_mut().chain(incoming.values_mut()) {
+            edges.sort_by(|a, b| {
+                (&a.from_id, a.kind, &a.to_id).cmp(&(&b.from_id, b.kind, &b.to_id))
+            });
+        }
+        let mut selected = BTreeSet::from([root.to_owned()]);
+        let mut queue = VecDeque::from([(root.to_owned(), Role::Root)]);
+        let mut visited = BTreeSet::new();
+        let mut missing = BTreeSet::new();
+        let mut implementing = BTreeSet::new();
+        let mut captured = BTreeSet::new();
+        let mut declared = BTreeSet::new();
+        let mut recording = BTreeSet::new();
+        let mut dependencies = BTreeSet::new();
+        let mut truncated = false;
+        // Join two facts already present in the snapshot: the vault entry's
+        // repository path and a recorded File node. Do not use task file refs,
+        // shared source files, or naming/timestamp similarity as attribution.
+        let intent_artifact = crate::workflow_usage::intent_artifact_id(nodes[root]).filter(|id| {
+            nodes
+                .get(id.as_str())
+                .is_some_and(|node| node.has_type(C::File))
+        });
+        if let Some(artifact) = &intent_artifact {
+            if selected.len() < max_nodes {
+                selected.insert(artifact.clone());
+                queue.push_back((artifact.clone(), Role::IntentArtifact));
+            } else {
+                truncated = true;
+            }
+        }
+        while let Some((id, role)) = queue.pop_front() {
+            if !visited.insert((id.clone(), role)) {
+                continue;
+            }
+            let out = outgoing.get(id.as_str()).into_iter().flatten().copied();
+            let inc = incoming.get(id.as_str()).into_iter().flatten().copied();
+            for (edge, forward) in out.map(|e| (e, true)).chain(inc.map(|e| (e, false))) {
+                let next = if forward { &edge.to_id } else { &edge.from_id };
+                let predicate = edge.kind;
+                let transition: Option> = match (role, forward, predicate) {
+                    (Role::IntentArtifact, false, P::ModifiesFile)
+                        if nodes
+                            .get(next.as_str())
+                            .is_some_and(|node| node.has_type(C::Change)) =>
+                    {
+                        Some(Some(Role::Change))
+                    }
+                    (Role::Root, false, P::Implements) => Some(Some(Role::Change)),
+                    (Role::Root, true, P::DeclaresImplementation) => Some(Some(Role::Change)),
+                    (Role::Root, false, P::HasPlan) | (Role::Root, true, P::GovernsTurn) => {
+                        Some(Some(Role::Turn))
+                    }
+                    (
+                        Role::Root,
+                        true,
+                        P::HasTask
+                        | P::HasAcceptanceCriterion
+                        | P::HasScopeIn
+                        | P::HasScopeOut
+                        | P::HasScopeItem
+                        | P::HasConstraint,
+                    ) => Some(Some(Role::Detail)),
+                    (Role::Root, true, _) => Some(None),
+                    (
+                        Role::Root,
+                        false,
+                        P::Reviews
+                        | P::ReviewedBy
+                        | P::Remediates
+                        | P::LinkedIntent
+                        | P::LinkedGoal
+                        | P::ProvWasDerivedFrom
+                        | P::InformedBy
+                        | P::DependsOnIntent
+                        | P::Blocks
+                        | P::BlockedBy
+                        | P::References,
+                    ) => Some(None),
+                    (Role::Detail, true, P::Touches | P::Satisfies | P::References) => Some(None),
+                    (Role::Change, true, P::DirectlyDependsOnChange) => Some(Some(Role::Change)),
+                    (Role::Change, true, P::AttestedBy)
+                    | (Role::Change, false, P::CoversChange) => Some(Some(Role::Attestation)),
+                    (Role::Change, false, P::GeneratedChange)
+                    | (Role::Change, true, P::GeneratedBy) => Some(Some(Role::Turn)),
+                    (Role::Change, true, P::DirectlyOnView) => Some(Some(Role::View)),
+                    (
+                        Role::Change,
+                        true,
+                        P::ModifiesFile
+                        | P::Modifies
+                        | P::AuthoredBy
+                        | P::ChangeExplainedBy
+                        | P::ExplainedBy
+                        | P::Implements,
+                    ) => Some(None),
+                    // Only root-governed turns can seed implementing changes.
+                    (Role::Turn, true, P::GeneratedChange)
+                        if outgoing.get(id.as_str()).is_some_and(|edges| {
+                            edges
+                                .iter()
+                                .any(|e| e.kind == P::HasPlan && e.to_id == root)
+                        }) || incoming.get(id.as_str()).is_some_and(|edges| {
+                            edges
+                                .iter()
+                                .any(|e| e.kind == P::GovernsTurn && e.from_id == root)
+                        }) =>
+                    {
+                        Some(Some(Role::Change))
+                    }
+                    (Role::Turn, false, P::Attests) => Some(Some(Role::Attestation)),
+                    (
+                        Role::Turn,
+                        true,
+                        P::HasPlan
+                        | P::HasTodo
+                        | P::TurnExplainedBy
+                        | P::ExplainedBy
+                        | P::TurnOfSession,
+                    )
+                    | (Role::Turn, false, P::HasTurn) => Some(None),
+                    // Coverage is a boundary: never acquire sibling changes via an attestation.
+                    (
+                        Role::Attestation,
+                        true,
+                        P::SignedBy
+                        | P::AttestsSession
+                        | P::Attests
+                        | P::PreviousAttestation
+                        | P::VerificationResult,
+                    ) => Some(None),
+                    (Role::View, true, P::ChildViewOf) => Some(Some(Role::View)),
+                    (_, false, P::AboutResource) => Some(None),
+                    _ => None,
+                };
+                let Some(next_role) = transition else {
+                    continue;
+                };
+                if !nodes.contains_key(next.as_str()) {
+                    missing.insert(next.clone());
+                    continue;
+                }
+                if !selected.contains(next) {
+                    if selected.len() == max_nodes {
+                        truncated = true;
+                        continue;
+                    }
+                    selected.insert(next.clone());
+                }
+                if next_role == Some(Role::Change) {
+                    if role == Role::IntentArtifact && predicate == P::ModifiesFile {
+                        recording.insert(next.clone());
+                    } else if predicate == P::Implements
+                        || (role == Role::Turn && predicate == P::GeneratedChange)
+                    {
+                        implementing.insert(next.clone());
+                        captured.insert(next.clone());
+                    } else if predicate == P::DeclaresImplementation {
+                        implementing.insert(next.clone());
+                        declared.insert(next.clone());
+                    } else if predicate == P::DirectlyDependsOnChange {
+                        dependencies.insert(next.clone());
+                    }
+                }
+                if let Some(next_role) = next_role {
+                    queue.push_back((next.clone(), next_role));
+                }
+            }
+        }
+        // Include findings on boundary nodes too, without traversing from them.
+        for id in selected.clone() {
+            for edge in incoming
+                .get(id.as_str())
+                .into_iter()
+                .flatten()
+                .filter(|e| e.kind == P::AboutResource)
+            {
+                if !nodes.contains_key(edge.from_id.as_str()) {
+                    missing.insert(edge.from_id.clone());
+                } else if selected.len() < max_nodes || selected.contains(&edge.from_id) {
+                    selected.insert(edge.from_id.clone());
+                } else {
+                    truncated = true;
+                }
+            }
+        }
+        dependencies.retain(|id| !implementing.contains(id) && !recording.contains(id));
+        let selected_nodes: Vec<_> = selected
+            .iter()
+            .filter_map(|id| nodes.get(id.as_str()).map(|n| (*n).clone()))
+            .collect();
+        let mut edges: Vec<_> = snapshot
+            .edges
+            .iter()
+            .filter(|e| selected.contains(&e.from_id) && selected.contains(&e.to_id))
+            .cloned()
+            .collect();
+        edges.sort_by(|a, b| (&a.from_id, a.kind, &a.to_id).cmp(&(&b.from_id, b.kind, &b.to_id)));
+        let mut diagnostics = vec!["Stored semantic graph facts, including exact intent-artifact recording evidence; no source interpretation, implicit ancestry, or historical patch/preimage expansion.".into()];
+        if implementing.is_empty() {
+            if recording.is_empty() {
+                diagnostics.push("No implementing changes or intent-artifact recording changes are linked in this snapshot; this is not proof that no work occurred.".into());
+            } else {
+                diagnostics.push(format!("{} recorded change(s) modify this intent's artifact and are shown as recording evidence. No explicit implementation association is asserted; recording evidence does not establish governing-turn provenance.", recording.len()));
+            }
+        }
+        if !declared.is_empty() {
+            diagnostics.push(format!("{} implementing change(s) have explicit author declarations. Declarations do not repair or prove missing original plan_id/generating-turn provenance.", declared.len()));
+        }
+        if truncated {
+            diagnostics.push("Node limit reached: selection and dependency closure are incomplete. Increase --max-nodes.".into());
+        }
+        if !missing.is_empty() {
+            diagnostics.push("Some referenced resources are missing from the snapshot.".into());
+        }
+        for node in &selected_nodes {
+            if node.has_type(C::Change) {
+                let availability = node
+                    .metadata
+                    .as_ref()
+                    .and_then(|m| m.get("object_availability"))
+                    .and_then(|v| v.as_str())
+                    .unwrap_or("unknown");
+                let completeness = node
+                    .metadata
+                    .as_ref()
+                    .and_then(|m| m.get("projection_completeness"))
+                    .and_then(|v| v.as_str())
+                    .unwrap_or("reference-only");
+                if availability != "available" || completeness != "projected" {
+                    diagnostics.push(format!(
+                        "{}: object {availability}, projection {completeness}",
+                        node.id
+                    ));
+                }
+            }
+        }
+        Ok(Self {
+            schema_version: 1,
+            root: root.into(),
+            snapshot_identity: snapshot.identity.clone(),
+            generation: snapshot.generation,
+            ontology_version: snapshot.ontology_version.into(),
+            scope: "intent_asserted_lineage_and_recording_evidence_with_dependency_closure",
+            nodes: selected_nodes,
+            edges,
+            implementing_changes: implementing.into_iter().collect(),
+            captured_changes: captured.into_iter().collect(),
+            declared_changes: declared.into_iter().collect(),
+            intent_artifact,
+            recording_changes: recording.into_iter().collect(),
+            dependency_changes: dependencies.into_iter().collect(),
+            truncated,
+            missing_resources: missing.into_iter().collect(),
+            diagnostics,
+        })
+    }
+}
diff --git a/atomic-repository/src/lib.rs b/atomic-repository/src/lib.rs
index 83c9cadc..e5db9d2d 100644
--- a/atomic-repository/src/lib.rs
+++ b/atomic-repository/src/lib.rs
@@ -96,6 +96,7 @@ pub mod error;
 pub mod git_binding;
 pub mod ignore;
 pub mod intent_bridge;
+pub mod intent_graph;
 pub mod manifest;
 pub mod record;
 pub mod repository;
@@ -103,6 +104,7 @@ pub mod status;
 pub mod tracking;
 pub mod triage;
 pub mod wip;
+pub(crate) mod workflow_usage;
 
 // Phase 7 additions
 pub mod archive;
@@ -131,6 +133,7 @@ pub mod content_search;
 pub mod oci;
 
 // Query plan schema and executor
+pub mod patch_evidence;
 pub mod query_plan;
 
 // Content search re-exports
diff --git a/atomic-repository/src/patch_evidence.rs b/atomic-repository/src/patch_evidence.rs
new file mode 100644
index 00000000..b719e60b
--- /dev/null
+++ b/atomic-repository/src/patch_evidence.rs
@@ -0,0 +1,219 @@
+//! Bounded evidence from immutable change objects, never from a working copy.
+//!
+//! The content blob contains insertion bytes (and structural data such as names),
+//! addressed by the recorded operations. It is NOT a reconstructed historical file
+//! or a textual diff. Deletion operations are returned verbatim; resolving their
+//! preimages through dependency graphs is deliberately unsupported here.
+
+use std::io::{self, Write};
+
+use atomic_core::change::{FileOps, GraphOp};
+use atomic_core::kg::iri;
+use atomic_core::types::{Base32, Hash};
+use serde::{Deserialize, Serialize};
+
+use crate::{Repository, RepositoryError};
+
+pub const MAX_EVIDENCE_OPERATIONS: usize = 64;
+pub const MAX_EVIDENCE_OPERATION_BYTES: usize = 65_536;
+pub const MAX_EVIDENCE_CONTENT_BYTES: usize = 16_384;
+/// Bounds accepted encoded objects before invoking the existing change decoder.
+/// This is not a bound on decoder memory (compressed objects can expand).
+pub const MAX_EVIDENCE_OBJECT_BYTES: u64 = 8 * 1024 * 1024;
+
+/// Limits apply to the combined GraphOps/FileOps count and compact JSON bytes,
+/// and separately to raw content-blob bytes. Oversized limits are rejected.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
+#[serde(default, deny_unknown_fields)]
+pub struct PatchEvidenceLimits {
+    pub max_operations: usize,
+    pub max_operation_bytes: usize,
+    pub max_content_bytes: usize,
+}
+
+impl Default for PatchEvidenceLimits {
+    fn default() -> Self {
+        Self {
+            max_operations: 32,
+            max_operation_bytes: 32_768,
+            max_content_bytes: 8192,
+        }
+    }
+}
+
+impl PatchEvidenceLimits {
+    pub fn validate(&self) -> Result<(), RepositoryError> {
+        if self.max_operations > MAX_EVIDENCE_OPERATIONS
+            || self.max_operation_bytes > MAX_EVIDENCE_OPERATION_BYTES
+            || self.max_content_bytes > MAX_EVIDENCE_CONTENT_BYTES
+        {
+            return Err(RepositoryError::Serialization(
+                "patch evidence limits exceed hard maxima".into(),
+            ));
+        }
+        Ok(())
+    }
+}
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
+#[serde(rename_all = "snake_case")]
+pub enum PatchEvidenceStatus {
+    Found,
+    /// No locally stored change object. Does not mean that the change never existed.
+    Missing,
+    /// Encoded object exceeds the hard source-size limit; nothing was decoded.
+    Unsupported,
+}
+
+/// Exact prefixes of the stored operation sequences, cut only at whole-operation
+/// boundaries. An empty sequence with total=0 means no operations were recorded,
+/// not a regenerated semantic layer. `None` means the object was not loaded.
+#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
+pub struct RecordedOperations {
+    pub operations: Vec,
+    pub total: usize,
+    pub truncated: bool,
+}
+
+#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
+pub struct RecordedContentBlob {
+    /// Canonical hash of the entire recorded blob, verified before returning bytes.
+    pub hash: String,
+    /// Exact bytes at offset zero, not lossy UTF-8 or current filesystem content.
+    pub bytes: Vec,
+    pub total_bytes: usize,
+    pub truncated: bool,
+}
+
+#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
+#[serde(rename_all = "snake_case")]
+pub enum RemovedPreimageStatus {
+    /// Recorded delete/replace references remain in GraphOps/FileOps. This small
+    /// operator does not reconstruct removed bytes or follow dependency objects.
+    Unsupported,
+}
+
+#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
+pub struct PatchEvidence {
+    pub change_iri: String,
+    pub status: PatchEvidenceStatus,
+    pub graph_ops: Option>>>,
+    pub file_ops: Option>,
+    pub content_blob: Option,
+    pub removed_preimage: RemovedPreimageStatus,
+}
+
+impl Repository {
+    /// Retrieve exact recorded patch data using only a canonical full-hash change
+    /// IRI (`urn:atomic:change:blake3-256:<52 uppercase Base32 characters>`).
+    ///
+    /// Hash prefixes, raw hashes, other resource kinds and noncanonical spellings
+    /// are errors. Missing objects and oversized encoded objects have explicit
+    /// statuses; corruption and I/O errors fail closed. No view or working-copy
+    /// state is consulted, and no KG nodes or semantic claims are synthesized.
+    ///
+    /// GraphOps consume the shared operation budget first, then FileOps. Result
+    /// limits bound returned data, not the existing decoder's memory usage.
+    pub fn patch_evidence(
+        &self,
+        change_iri: &str,
+        limits: PatchEvidenceLimits,
+    ) -> Result {
+        limits.validate()?;
+        let hash = iri::parse_change(change_iri)
+            .map_err(|error| RepositoryError::Serialization(error.to_string()))?;
+        let mut result = PatchEvidence {
+            change_iri: iri::mint_change(&hash),
+            status: PatchEvidenceStatus::Missing,
+            graph_ops: None,
+            file_ops: None,
+            content_blob: None,
+            removed_preimage: RemovedPreimageStatus::Unsupported,
+        };
+        let metadata = match std::fs::metadata(self.change_store().change_path(&hash)) {
+            Ok(metadata) => metadata,
+            Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(result),
+            Err(error) => return Err(error.into()),
+        };
+        if metadata.len() > MAX_EVIDENCE_OBJECT_BYTES {
+            result.status = PatchEvidenceStatus::Unsupported;
+            return Ok(result);
+        }
+        let change = match self.load_change(&hash) {
+            Ok(change) => change,
+            Err(RepositoryError::ChangeNotFound { .. }) => return Ok(result),
+            Err(error) => return Err(error),
+        };
+        if Hash::of(&change.contents) != change.hashed.contents_hash {
+            return Err(RepositoryError::Serialization(
+                "patch evidence content-blob hash mismatch".into(),
+            ));
+        }
+        let mut count = limits.max_operations;
+        let mut bytes = limits.max_operation_bytes;
+        result.graph_ops = Some(bounded_operations(
+            &change.hashed.hunks,
+            &mut count,
+            &mut bytes,
+        )?);
+        result.file_ops = Some(bounded_operations(
+            change.file_ops(),
+            &mut count,
+            &mut bytes,
+        )?);
+        result.content_blob = Some(RecordedContentBlob {
+            hash: change.hashed.contents_hash.to_base32(),
+            bytes: change
+                .contents
+                .iter()
+                .take(limits.max_content_bytes)
+                .copied()
+                .collect(),
+            total_bytes: change.contents.len(),
+            truncated: change.contents.len() > limits.max_content_bytes,
+        });
+        result.status = PatchEvidenceStatus::Found;
+        Ok(result)
+    }
+}
+
+// Count serialization through a bounded sink rather than allocating an unbounded
+// JSON string for a single potentially enormous FileOps/GraphOp.
+struct ByteBudget(usize);
+impl Write for ByteBudget {
+    fn write(&mut self, bytes: &[u8]) -> io::Result {
+        if bytes.len() > self.0 {
+            return Err(io::Error::other("operation byte budget exhausted"));
+        }
+        self.0 -= bytes.len();
+        Ok(bytes.len())
+    }
+    fn flush(&mut self) -> io::Result<()> {
+        Ok(())
+    }
+}
+
+fn bounded_operations(
+    source: &[T],
+    remaining_count: &mut usize,
+    remaining_bytes: &mut usize,
+) -> Result, RepositoryError> {
+    let mut operations = Vec::new();
+    for operation in source.iter().take(*remaining_count) {
+        let mut budget = ByteBudget(*remaining_bytes);
+        if let Err(error) = serde_json::to_writer(&mut budget, operation) {
+            if error.is_io() {
+                break;
+            }
+            return Err(RepositoryError::Serialization(error.to_string()));
+        }
+        *remaining_bytes = budget.0;
+        operations.push(operation.clone());
+    }
+    *remaining_count -= operations.len();
+    Ok(RecordedOperations {
+        truncated: operations.len() < source.len(),
+        operations,
+        total: source.len(),
+    })
+}
diff --git a/atomic-repository/src/query_plan.rs b/atomic-repository/src/query_plan.rs
index 430ef8cb..d82417b5 100644
--- a/atomic-repository/src/query_plan.rs
+++ b/atomic-repository/src/query_plan.rs
@@ -12,6 +12,7 @@ use std::collections::HashMap;
 
 use serde::{Deserialize, Serialize};
 
+use crate::patch_evidence::{PatchEvidence, PatchEvidenceLimits};
 use crate::repository::Repository;
 use crate::RepositoryError;
 use atomic_core::kg::{
@@ -36,6 +37,13 @@ pub struct QueryPlan {
 #[derive(Debug, Clone, Serialize, Deserialize)]
 #[serde(tag = "type", rename_all = "snake_case", deny_unknown_fields)]
 pub enum QueryStep {
+    /// Exact immutable recorded patch evidence, not current file content or KG
+    /// assertions. One literal canonical full-hash change IRI per step.
+    PatchEvidence {
+        change_iri: String,
+        #[serde(default)]
+        limits: PatchEvidenceLimits,
+    },
     /// Full-text keyword search over KG nodes.
     KgSearch {
         /// Search query text.
@@ -124,6 +132,9 @@ fn default_max_chars() -> usize {
 /// Result of executing a query plan.
 #[derive(Debug, Clone, Serialize, Deserialize)]
 pub struct PlanResult {
+    /// Typed immutable patch evidence, separate from graph assertions and vault content.
+    #[serde(default, skip_serializing_if = "Vec::is_empty")]
+    pub patch_evidence: Vec,
     /// Nodes collected across all steps.
     pub nodes: Vec,
     /// Edges collected across all steps.
@@ -164,11 +175,34 @@ pub fn execute_plan(repo: &Repository, plan: &QueryPlan) -> Result = Vec::new();
     let mut content: HashMap = HashMap::new();
     let mut step_stats: Vec = Vec::new();
+    let mut patch_evidence = Vec::new();
+    // Bound evidence fan-out even for programmatically constructed plans.
+    if plan
+        .steps
+        .iter()
+        .filter(|step| matches!(step, QueryStep::PatchEvidence { .. }))
+        .count()
+        > 8
+    {
+        return Err(RepositoryError::Serialization(
+            "at most 8 patch evidence steps per plan".into(),
+        ));
+    }
 
     for step in &plan.steps {
         let step_start = std::time::Instant::now();
 
         match step {
+            QueryStep::PatchEvidence { change_iri, limits } => {
+                let evidence = repo.patch_evidence(change_iri, *limits)?;
+                let found = evidence.status == crate::patch_evidence::PatchEvidenceStatus::Found;
+                patch_evidence.push(evidence);
+                step_stats.push(StepStat {
+                    step_type: "patch_evidence".into(),
+                    result_count: usize::from(found),
+                    elapsed_ms: step_start.elapsed().as_millis() as u64,
+                });
+            }
             QueryStep::KgSearch { query, limit, bind } => {
                 let nodes = repo.vault_kg_search(query, *limit, None)?;
                 let count = nodes.len();
@@ -390,6 +424,7 @@ pub fn execute_plan(repo: &Repository, plan: &QueryPlan) -> Result true,
+                Err(RepositoryError::ChangeNotFound { .. }) => false,
+                Err(error) => return Err(error),
+            };
+            if available {
                 let target = iri::mint_change(hash);
                 if txn.get_kg_node(&target)?.is_none() {
                     txn.upsert_kg_node(&KgNode::new(
@@ -153,7 +166,8 @@ impl Repository {
                         C::Change,
                         &hash.to_base32()[..12],
                         "vcs",
-                    ))?;
+                    ).with_metadata(serde_json::json!({"hash": hash.to_base32(),
+                        "object_availability": "available", "projection_completeness": "reference-only"})))?;
                 }
                 txn.upsert_kg_edge(&KgEdge::new(&id, target, P::DirectlyDependsOnChange))?;
             } else {
@@ -168,11 +182,12 @@ impl Repository {
         if !missing_dependency {
             txn.del_kg_node(&iri::mint_finding(&project, "UnresolvedReference", &id))?;
         }
-        let rules = IgnoreRules::load_for_enrichment(self.root());
+        // Recorded operations are immutable impact facts. Current working-copy
+        // indexing exclusions must not change their historical projection.
         let mut paths = std::collections::BTreeSet::new();
         for file in change.file_ops() {
             let path = file.path();
-            if path.is_empty() || rules.is_ignored(std::path::Path::new(path), false) {
+            if path.is_empty() {
                 continue;
             }
             paths.insert(path);
@@ -191,6 +206,65 @@ impl Repository {
         Ok(paths.len())
     }
 
+    /// Reconcile reference nodes against verified canonical objects. In
+    /// particular a session ledger may mention an object never received here.
+    /// This preserves node identity and independent edges, and never treats a
+    /// malformed/unreadable object as absent or available.
+    pub(super) fn refresh_change_availability_kg(
+        &self,
+        txn: &mut atomic_core::WriteTxn<'_>,
+    ) -> Result<(), RepositoryError> {
+        for mut node in txn.all_kg_nodes()? {
+            if !node.has_type(C::Change) {
+                continue;
+            }
+            let hash = iri::parse_change(&node.id)
+                .map_err(|error| RepositoryError::Serialization(error.to_string()))?;
+            let availability = match self.load_change(&hash) {
+                Ok(change) if !change.kind().is_snapshot() => {
+                    if node
+                        .metadata
+                        .as_ref()
+                        .and_then(|metadata| metadata.get("projection_completeness"))
+                        .and_then(serde_json::Value::as_str)
+                        != Some("projected")
+                    {
+                        self.project_change_kg(txn, &hash, &change)?;
+                    }
+                    continue;
+                }
+                Ok(_) => "available",
+                Err(RepositoryError::ChangeNotFound { .. }) => "absent",
+                Err(error) => return Err(error),
+            };
+            let metadata = node.metadata.get_or_insert_with(|| serde_json::json!({}));
+            metadata["object_availability"] = availability.into();
+            metadata["projection_completeness"] = "reference-only".into();
+            if let Some(metadata) = metadata.as_object_mut() {
+                metadata.remove("created_at");
+                for field in [
+                    "ai_usage",
+                    "ai_usage_records",
+                    "ai_usage_state",
+                    "ai_usage_record_version",
+                ] {
+                    metadata.remove(field);
+                }
+            }
+            node.summary = None;
+            for edge in txn.get_kg_edges_from(&node.id)? {
+                if matches!(
+                    edge.kind,
+                    P::AuthoredBy | P::DirectlyDependsOnChange | P::ModifiesFile
+                ) {
+                    txn.del_kg_edge(&edge.from_id, &edge.to_id, &edge.kind)?;
+                }
+            }
+            txn.upsert_kg_node(&node)?;
+        }
+        Ok(())
+    }
+
     pub(super) fn repair_change_dependencies_kg(
         &self,
         txn: &mut atomic_core::WriteTxn<'_>,
diff --git a/atomic-repository/src/repository/vault_projection.rs b/atomic-repository/src/repository/vault_projection.rs
index 44e89798..8783e870 100644
--- a/atomic-repository/src/repository/vault_projection.rs
+++ b/atomic-repository/src/repository/vault_projection.rs
@@ -461,6 +461,7 @@ impl VaultProjection {
                             "blockedBy" => P::BlockedBy,
                             "remediates" => P::Remediates,
                             "reviews" => P::Reviews,
+                            "declaresImplementation" => P::DeclaresImplementation,
                             _ => P::References,
                         };
                         let raw = if reference.to.starts_with("urn:") {
@@ -469,6 +470,15 @@ impl VaultProjection {
                             format!("urn:atomic:intent:{}", reference.to)
                         };
                         self.link(&mut graph, subject, &raw, predicate, path);
+                        if predicate == P::DeclaresImplementation {
+                            if let Some(edge) =
+                                graph.edges.last_mut().filter(|edge| edge.kind == predicate)
+                            {
+                                edge.metadata.get_or_insert_with(|| serde_json::json!({}))
+                                    ["declarationEvidence"] =
+                                    serde_json::json!(reference.declaration_evidence);
+                            }
+                        }
                     }
                 }
             }
@@ -534,6 +544,13 @@ impl VaultProjection {
                 .map(|target| (target, P::InformedBy))
                 .collect();
             for reference in node.depends_on {
+                // Implementation declarations are projected from the current
+                // intent source only. A stale signed snapshot must not restore a
+                // declaration removed from that source; its original bytes and
+                // proof remain inspectable in the attestation object.
+                if reference.edge == "declaresImplementation" {
+                    continue;
+                }
                 let predicate = match reference.edge.as_str() {
                     "dependsOn" => P::DependsOnIntent,
                     "blockedBy" => P::BlockedBy,
diff --git a/atomic-repository/src/repository/vault_triples.rs b/atomic-repository/src/repository/vault_triples.rs
index fcbd25d3..20697ed0 100644
--- a/atomic-repository/src/repository/vault_triples.rs
+++ b/atomic-repository/src/repository/vault_triples.rs
@@ -164,6 +164,15 @@ impl Repository {
         let txn = self.pristine.read_txn()?;
         let mut nodes = txn.all_kg_nodes()?;
         let mut edges = txn.all_kg_edges()?;
+        let project = txn
+            .project_identity()?
+            .ok_or(atomic_core::PristineError::MissingProjectIdentity)?
+            .project_id()
+            .map_err(|error| RepositoryError::Serialization(error.to_string()))?;
+        let view_scopes = crate::workflow_usage::view_scopes(self, &txn, &project)?;
+        drop(txn);
+        self.hydrate_workflow_usage(&mut nodes)?;
+        crate::workflow_usage::annotate_with_views(&mut nodes, &edges, Some(&view_scopes))?;
         nodes.sort_by(|a, b| a.id.cmp(&b.id));
         edges.sort_by(|a, b| (&a.from_id, &a.to_id, a.kind).cmp(&(&b.from_id, &b.to_id, b.kind)));
         let bytes =
diff --git a/atomic-repository/src/workflow_usage.rs b/atomic-repository/src/workflow_usage.rs
new file mode 100644
index 00000000..9bc1f335
--- /dev/null
+++ b/atomic-repository/src/workflow_usage.rs
@@ -0,0 +1,891 @@
+//! Read-only usage summaries over immutable change provenance and KG membership.
+//! Totals are scoped before graph selection/caps, never inferred from shared files.
+
+use std::collections::{BTreeMap, BTreeSet};
+
+use atomic_core::{
+    change::Change,
+    kg::{
+        iri,
+        ontology::{Class as C, Predicate as P},
+    },
+    types::{Base32, Hash},
+    KgEdge, KgNode,
+};
+use serde::{Deserialize, Serialize};
+
+use crate::{Repository, RepositoryError};
+
+pub(crate) struct ViewUsageScope {
+    changes: BTreeSet,
+    error: Option,
+}
+
+/// Capture authoritative view filters in the same read transaction as KG rows.
+/// KG membership edges can lag record/insert operations and are not accounting
+/// authority for effective view membership.
+pub(crate) fn view_scopes(
+    repo: &Repository,
+    txn: &impl atomic_core::pristine::ViewTxnT,
+    project: &atomic_core::kg::project_id::ProjectId,
+) -> Result, RepositoryError> {
+    let mut scopes = BTreeMap::new();
+    let mut dependency_cache = BTreeMap::>::new();
+    for name in txn.list_views()? {
+        let Some(view) = txn.get_view(&name)? else {
+            continue;
+        };
+        let mut scope = ViewUsageScope {
+            changes: BTreeSet::new(),
+            error: None,
+        };
+        // Accounting does not traverse the byte graph. Resolve canonical view
+        // membership, then verify dependency headers directly rather than
+        // requiring old repositories to backfill a traversal acceleration index.
+        match crate::repository::view_membership(txn, &view) {
+            Ok(membership) => {
+                let mut pending = Vec::new();
+                for change in membership.iter() {
+                    if let Some(hash) = txn.get_external(*change)? {
+                        pending.push(hash);
+                    } else {
+                        scope.error =
+                            Some("A canonical view member has no external change hash".into());
+                    }
+                }
+                while let Some(hash) = pending.pop() {
+                    if !scope.changes.insert(iri::mint_change(&hash)) {
+                        continue;
+                    }
+                    if !dependency_cache.contains_key(&hash) {
+                        match repo.load_change(&hash) {
+                            Ok(change) => {
+                                dependency_cache.insert(hash, change.dependencies().to_vec());
+                            }
+                            Err(RepositoryError::ChangeNotFound { .. }) => {
+                                scope.error = Some(format!("View change {} is unavailable; dependency closure may be incomplete", hash.to_base32()));
+                                continue;
+                            }
+                            Err(error) => return Err(error),
+                        }
+                    }
+                    for dependency in &dependency_cache[&hash] {
+                        if txn.get_internal(dependency)?.is_none() {
+                            scope.error = Some(format!(
+                                "View dependency {} is not registered in the canonical graph",
+                                dependency.to_base32()
+                            ));
+                        }
+                        pending.push(*dependency);
+                    }
+                }
+            }
+            Err(error) => scope.error = Some(error.to_string()),
+        }
+        scopes.insert(iri::mint_view(project, &name), scope);
+    }
+    Ok(scopes)
+}
+
+impl Repository {
+    /// Hydrate older KG projections from verified immutable change headers.
+    /// This only changes the returned read model, never the repository database.
+    pub(crate) fn hydrate_workflow_usage(
+        &self,
+        nodes: &mut [KgNode],
+    ) -> Result<(), RepositoryError> {
+        for node in nodes.iter_mut().filter(|node| node.has_type(C::Change)) {
+            let metadata = node.metadata.as_ref();
+            let availability = metadata
+                .and_then(|m| m.get("object_availability"))
+                .and_then(|v| v.as_str());
+            if matches!(availability, Some("absent" | "unknown")) {
+                set_change_records(node, None)?;
+                continue;
+            }
+            if metadata
+                .and_then(|m| m.get("ai_usage_record_version"))
+                .and_then(|v| v.as_u64())
+                == Some(1)
+            {
+                continue;
+            }
+            let hash = iri::parse_change(&node.id)
+                .map_err(|error| RepositoryError::Serialization(error.to_string()))?;
+            match self.load_change(&hash) {
+                Ok(change) => set_change_records(node, Some(change_records(&change, &hash)))?,
+                Err(RepositoryError::ChangeNotFound { .. }) => set_change_records(node, None)?,
+                Err(error) => return Err(error),
+            }
+        }
+        Ok(())
+    }
+
+    /// Copy full-snapshot usage summaries into a selected subgraph. Aggregate
+    /// scope does not shrink when the exported graph is capped or filtered.
+    pub fn enrich_workflow_usage(&self, nodes: &mut [KgNode]) -> Result<(), RepositoryError> {
+        if !nodes
+            .iter()
+            .any(|node| node.has_type(C::Change) || node.has_type(C::View) || is_intent(node))
+        {
+            return Ok(());
+        }
+        let snapshot = self.workflow_snapshot()?;
+        let summaries: BTreeMap<_, _> = snapshot
+            .nodes
+            .iter()
+            .map(|node| (node.id.as_str(), node))
+            .collect();
+        for node in nodes {
+            if let Some(usage) = summaries
+                .get(node.id.as_str())
+                .and_then(|source| source.metadata.as_ref())
+                .and_then(|metadata| metadata.get("ai_usage"))
+            {
+                metadata_object(node)?["ai_usage"] = usage.clone();
+            }
+        }
+        Ok(())
+    }
+}
+
+fn is_intent(node: &KgNode) -> bool {
+    node.types
+        .iter()
+        .filter_map(|kind| C::from_iri(kind))
+        .any(|kind| {
+            kind == C::WorkIntent
+                || kind
+                    .parents()
+                    .iter()
+                    .any(|parent| parent.iri() == C::WorkIntent.iri())
+        })
+}
+
+#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
+struct Tokens {
+    input: u64,
+    output: u64,
+    reasoning: u64,
+    cache_read: u64,
+    cache_write: u64,
+    total: u64,
+}
+
+impl Tokens {
+    fn add(&mut self, other: &Self) -> Result<(), RepositoryError> {
+        self.input = add(self.input, other.input)?;
+        self.output = add(self.output, other.output)?;
+        self.reasoning = add(self.reasoning, other.reasoning)?;
+        self.cache_read = add(self.cache_read, other.cache_read)?;
+        self.cache_write = add(self.cache_write, other.cache_write)?;
+        self.total = add(self.total, other.total)?;
+        Ok(())
+    }
+}
+
+fn invalid(message: &str) -> RepositoryError {
+    RepositoryError::InvalidOperation {
+        message: message.into(),
+    }
+}
+fn add(a: u64, b: u64) -> Result {
+    a.checked_add(b)
+        .ok_or_else(|| invalid("AI usage total exceeds u64; refusing to wrap or understate usage"))
+}
+
+#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
+pub(crate) struct UsageRecord {
+    id: String,
+    vendor: String,
+    model: String,
+    tool: String,
+    tokens: Option,
+    cost_usd_micros: Option,
+}
+
+/// Only usage/attribution fields are exported: never prompts or reasoning text.
+pub(crate) fn change_records(change: &Change, hash: &Hash) -> Vec {
+    change
+        .provenance()
+        .iter()
+        .enumerate()
+        .map(|(index, provenance)| {
+            let vendor = provenance.vendor.name().to_owned();
+            let model = provenance.model.clone();
+            let tool = provenance.tool.description();
+            let turn = provenance
+                .metadata
+                .iter()
+                .find(|(key, _)| key == "turn_number")
+                .map(|(_, value)| value)
+                .filter(|value| !value.is_empty());
+            let identity = if let Some(request) = provenance
+                .request_id
+                .as_deref()
+                .filter(|value| !value.is_empty())
+            {
+                serde_json::to_vec(&("request", &vendor, request))
+            } else if let (Some(session), Some(turn)) = (
+                provenance
+                    .session_id
+                    .as_deref()
+                    .filter(|value| !value.is_empty()),
+                turn,
+            ) {
+                serde_json::to_vec(&("session_turn", session, turn, &vendor, &model, &tool))
+            } else {
+                // No identifiable shared generation: do not guess that otherwise
+                // identical records on distinct changes are the same paid request.
+                serde_json::to_vec(&("change_record", hash.to_base32(), index))
+            }
+            .expect("usage identity serialization");
+            let tokens = &provenance.tokens;
+            let cost = provenance.cost;
+            let zero_reported = provenance
+                .metadata
+                .iter()
+                .any(|(key, value)| key == "cost_reported" && value == "true");
+            let cost_usd_micros = if cost.micro_usd > 0 || (cost.usd == 0.0 && zero_reported) {
+                Some(cost.micro_usd)
+            } else if cost.usd.is_finite()
+                && cost.usd > 0.0
+                && cost.usd * 1_000_000.0 < u64::MAX as f64
+            {
+                // Legacy records may predate the integer micro-dollar field.
+                Some((cost.usd * 1_000_000.0).round() as u64)
+            } else {
+                None
+            };
+            UsageRecord {
+                id: Hash::of(&identity).to_base32(),
+                vendor,
+                model,
+                tool,
+                tokens: if [
+                    tokens.input_tokens,
+                    tokens.output_tokens,
+                    tokens.reasoning_tokens,
+                    tokens.cache_read_tokens,
+                    tokens.cache_write_tokens,
+                    tokens.total_tokens,
+                ]
+                .iter()
+                .all(|value| *value == 0)
+                {
+                    None
+                } else {
+                    Some(Tokens {
+                        input: tokens.input_tokens,
+                        output: tokens.output_tokens,
+                        reasoning: tokens.reasoning_tokens,
+                        cache_read: tokens.cache_read_tokens,
+                        cache_write: tokens.cache_write_tokens,
+                        total: tokens.total_tokens,
+                    })
+                },
+                cost_usd_micros,
+            }
+        })
+        .collect()
+}
+
+fn metadata_object(node: &mut KgNode) -> Result<&mut serde_json::Value, RepositoryError> {
+    let metadata = node.metadata.get_or_insert_with(|| serde_json::json!({}));
+    if !metadata.is_object() {
+        return Err(invalid(
+            "KG metadata must be an object for AI usage projection",
+        ));
+    }
+    Ok(metadata)
+}
+
+pub(crate) fn set_change_records(
+    node: &mut KgNode,
+    records: Option>,
+) -> Result<(), RepositoryError> {
+    let metadata = metadata_object(node)?;
+    metadata["ai_usage_record_version"] = 1.into();
+    metadata["ai_usage_state"] = match &records {
+        None => "unavailable",
+        Some(records) if records.is_empty() => "not_recorded",
+        Some(_) => "recorded",
+    }
+    .into();
+    metadata["ai_usage_records"] = serde_json::to_value(records).expect("usage serialization");
+    Ok(())
+}
+
+pub(crate) fn intent_artifact_id(node: &KgNode) -> Option {
+    let (project, _) = iri::parse_intent(&node.id).ok()?;
+    let path = node.metadata.as_ref()?.get("vault_path")?.as_str()?;
+    if path.is_empty()
+        || path.contains('\\')
+        || path.split('/').any(|part| matches!(part, "" | "." | ".."))
+    {
+        return None;
+    }
+    Some(iri::mint_file(&project, &format!(".vault/{path}")))
+}
+
+#[derive(Default, Serialize)]
+struct Coverage {
+    changes: usize,
+    with_provenance: usize,
+    without_provenance: usize,
+    unavailable: usize,
+    unique_usage_records: usize,
+    duplicate_usage_records: usize,
+    conflicting_usage_records: usize,
+    records_without_cost: usize,
+    records_without_tokens: usize,
+}
+
+#[derive(Default, Serialize)]
+struct Attribution {
+    vendor: String,
+    model: String,
+    tool: String,
+    records: usize,
+    tokens: Option,
+    cost_usd_micros: Option,
+    cost_usd: Option,
+}
+
+fn usd(micros: u64) -> String {
+    format!("{}.{:06}", micros / 1_000_000, micros % 1_000_000)
+}
+
+fn summarize(
+    selected: &BTreeSet,
+    nodes: &BTreeMap<&str, &KgNode>,
+    basis: &str,
+    scope_complete: bool,
+) -> Result {
+    let mut coverage = Coverage {
+        changes: selected.len(),
+        ..Default::default()
+    };
+    let mut records = BTreeMap::::new();
+    let mut conflicts = BTreeSet::new();
+    let mut attribution_keys = BTreeSet::new();
+    for id in selected {
+        let Some(node) = nodes
+            .get(id.as_str())
+            .filter(|node| node.has_type(C::Change))
+        else {
+            coverage.unavailable += 1;
+            continue;
+        };
+        let metadata = node.metadata.as_ref();
+        match metadata
+            .and_then(|m| m.get("ai_usage_state"))
+            .and_then(|value| value.as_str())
+        {
+            Some("not_recorded") => {
+                coverage.without_provenance += 1;
+                continue;
+            }
+            Some("recorded") => coverage.with_provenance += 1,
+            _ => {
+                coverage.unavailable += 1;
+                continue;
+            }
+        }
+        let entries: Vec = serde_json::from_value(
+            metadata
+                .and_then(|m| m.get("ai_usage_records"))
+                .cloned()
+                .unwrap_or_default(),
+        )
+        .map_err(|error| {
+            RepositoryError::Serialization(format!("invalid AI usage projection: {error}"))
+        })?;
+        for record in entries {
+            attribution_keys.insert((
+                record.vendor.clone(),
+                record.model.clone(),
+                record.tool.clone(),
+            ));
+            if let Some(existing) = records.get(&record.id) {
+                if existing == &record {
+                    coverage.duplicate_usage_records += 1;
+                } else {
+                    conflicts.insert(record.id.clone());
+                }
+            } else {
+                records.insert(record.id.clone(), record);
+            }
+        }
+    }
+    coverage.conflicting_usage_records = conflicts.len();
+    records.retain(|id, _| !conflicts.contains(id));
+    coverage.unique_usage_records = records.len();
+    let mut totals = None::;
+    let mut cost = None;
+    let mut groups = BTreeMap::<(String, String, String), Attribution>::new();
+    for record in records.values() {
+        let group = groups
+            .entry((
+                record.vendor.clone(),
+                record.model.clone(),
+                record.tool.clone(),
+            ))
+            .or_insert_with(|| Attribution {
+                vendor: record.vendor.clone(),
+                model: record.model.clone(),
+                tool: record.tool.clone(),
+                ..Default::default()
+            });
+        group.records += 1;
+        if let Some(tokens) = &record.tokens {
+            totals.get_or_insert_with(Tokens::default).add(tokens)?;
+            group
+                .tokens
+                .get_or_insert_with(Tokens::default)
+                .add(tokens)?;
+        } else {
+            coverage.records_without_tokens += 1;
+        }
+        if let Some(amount) = record.cost_usd_micros {
+            cost = Some(add(cost.unwrap_or(0), amount)?);
+            group.cost_usd_micros = Some(add(group.cost_usd_micros.unwrap_or(0), amount)?);
+        } else {
+            coverage.records_without_cost += 1;
+        }
+    }
+    // Attribution remains useful even when conflicting usage cannot be billed.
+    for (vendor, model, tool) in attribution_keys {
+        groups
+            .entry((vendor.clone(), model.clone(), tool.clone()))
+            .or_insert_with(|| Attribution {
+                vendor,
+                model,
+                tool,
+                ..Default::default()
+            });
+    }
+    let contributors: Vec<_> = groups
+        .into_values()
+        .map(|mut group| {
+            group.cost_usd = group.cost_usd_micros.map(usd);
+            group
+        })
+        .collect();
+    Ok(serde_json::json!({
+        "schema_version": 1, "basis": basis, "scope_complete": scope_complete,
+        "change_set_hash": Hash::of(&serde_json::to_vec(selected).expect("usage scope serialization")).to_base32(),
+        "coverage": coverage,
+        "tokens": totals,
+        "cost": cost.map(|amount| serde_json::json!({"currency":"USD", "amount_micros":amount, "amount":usd(amount)})),
+        "contributors": contributors,
+        "notes": [
+            "Recorded AI usage only, not total development cost. Missing provenance is not zero usage.",
+            "Token totals are preserved as recorded. Cache and reasoning categories may overlap provider totals; do not add rows to reconstruct total.",
+            "Zero token fields are stored values; older captures may default unreported fields to zero. All-zero token records and unmarked zero costs are treated as unreported, not free.",
+            "Shared request/session-turn usage is counted once; conflicting copies are excluded rather than guessed."
+        ]
+    }))
+}
+
+/// Attach derived usage to change/intent/view nodes, using the complete snapshot.
+#[cfg(test)]
+pub(crate) fn annotate(nodes: &mut [KgNode], edges: &[KgEdge]) -> Result<(), RepositoryError> {
+    annotate_with_views(nodes, edges, None)
+}
+
+pub(crate) fn annotate_with_views(
+    nodes: &mut [KgNode],
+    edges: &[KgEdge],
+    canonical_views: Option<&BTreeMap>,
+) -> Result<(), RepositoryError> {
+    let by_id: BTreeMap<_, _> = nodes.iter().map(|node| (node.id.as_str(), node)).collect();
+    let mut generated = BTreeMap::<&str, BTreeSet>::new();
+    let mut intent_changes = BTreeMap::<&str, BTreeSet>::new();
+    let mut recording = BTreeMap::<&str, BTreeSet>::new();
+    let mut membership = BTreeMap::<&str, BTreeSet>::new();
+    let mut parents = BTreeMap::<&str, BTreeSet>::new();
+    let mut dependencies = BTreeMap::<&str, BTreeSet>::new();
+    for edge in edges {
+        let map_and_pair = match edge.kind {
+            P::GeneratedChange => Some((&mut generated, (&edge.from_id, &edge.to_id))),
+            P::Implements => Some((&mut intent_changes, (&edge.to_id, &edge.from_id))),
+            P::DeclaresImplementation => Some((&mut intent_changes, (&edge.from_id, &edge.to_id))),
+            P::ModifiesFile => Some((&mut recording, (&edge.to_id, &edge.from_id))),
+            P::DirectlyOnView => Some((&mut membership, (&edge.to_id, &edge.from_id))),
+            P::ChildViewOf => Some((&mut parents, (&edge.from_id, &edge.to_id))),
+            P::DirectlyDependsOnChange => Some((&mut dependencies, (&edge.from_id, &edge.to_id))),
+            _ => None,
+        };
+        if let Some((map, (from, to))) = map_and_pair {
+            map.entry(from).or_default().insert(to.clone());
+        }
+    }
+    for edge in edges {
+        let (intent, turn) = match edge.kind {
+            P::HasPlan => (&edge.to_id, &edge.from_id),
+            P::GovernsTurn => (&edge.from_id, &edge.to_id),
+            _ => continue,
+        };
+        if let Some(changes) = generated.get(turn.as_str()) {
+            intent_changes
+                .entry(intent)
+                .or_default()
+                .extend(changes.iter().cloned());
+        }
+    }
+    let mut summaries = Vec::new();
+    for (index, node) in nodes.iter().enumerate() {
+        let (mut selected, mut basis, mut scope_complete) = if node.has_type(C::Change) {
+            (BTreeSet::from([node.id.clone()]), "change_provenance", true)
+        } else if is_intent(node) {
+            let mut selected = intent_changes
+                .get(node.id.as_str())
+                .cloned()
+                .unwrap_or_default();
+            if let Some(artifact) = intent_artifact_id(node)
+                .filter(|id| by_id.get(id.as_str()).is_some_and(|n| n.has_type(C::File)))
+            {
+                if let Some(changes) = recording.get(artifact.as_str()) {
+                    selected.extend(changes.iter().cloned());
+                }
+            }
+            (
+                selected,
+                "intent_implementation_and_recording_evidence_excluding_dependencies",
+                true,
+            )
+        } else if node.has_type(C::View) {
+            (
+                BTreeSet::new(),
+                "view_effective_membership_including_ancestors_and_dependencies",
+                true,
+            )
+        } else {
+            continue;
+        };
+        let mut scope_error = None;
+        if node.has_type(C::View) && canonical_views.is_some() {
+            basis = "view_canonical_effective_membership_including_ancestors_and_dependencies";
+            match canonical_views.and_then(|views| views.get(&node.id)) {
+                Some(scope) => {
+                    selected = scope.changes.clone();
+                    scope_complete = scope.error.is_none();
+                    scope_error = scope.error.clone();
+                }
+                None => {
+                    scope_complete = false;
+                    scope_error =
+                        Some("View is not present in canonical repository metadata".into());
+                }
+            }
+        } else if node.has_type(C::View) {
+            let mut pending = vec![node.id.clone()];
+            let mut visited = BTreeSet::new();
+            while let Some(view) = pending.pop() {
+                if !visited.insert(view.clone()) {
+                    continue;
+                }
+                scope_complete &= by_id
+                    .get(view.as_str())
+                    .is_some_and(|node| node.has_type(C::View));
+                if let Some(changes) = membership.get(view.as_str()) {
+                    selected.extend(changes.iter().cloned());
+                }
+                if let Some(ancestors) = parents.get(view.as_str()) {
+                    pending.extend(ancestors.iter().cloned());
+                }
+            }
+            let mut pending: Vec<_> = selected.iter().cloned().collect();
+            while let Some(change) = pending.pop() {
+                if let Some(deps) = dependencies.get(change.as_str()) {
+                    for dep in deps {
+                        if selected.insert(dep.clone()) {
+                            pending.push(dep.clone());
+                        }
+                    }
+                }
+            }
+        }
+        let mut summary = summarize(&selected, &by_id, basis, scope_complete)?;
+        if let Some(error) = scope_error {
+            summary["scope_error"] = error.into();
+        }
+        summaries.push((index, summary));
+    }
+    for (index, summary) in summaries {
+        metadata_object(&mut nodes[index])?["ai_usage"] = summary;
+    }
+    Ok(())
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use atomic_core::change::{AITool, AIVendor, ChangeHeader, Provenance, TokenUsage};
+
+    fn change(name: &str, amount: u64, request: &str) -> KgNode {
+        let mut change = Change::empty(ChangeHeader::new(name));
+        let mut provenance = Provenance::builder()
+            .vendor(AIVendor::OpenAI)
+            .model("model-a")
+            .tool(AITool::cli("test-agent"))
+            .request_id(request)
+            .prompt_full("PRIVATE PROMPT")
+            .reasoning_text("PRIVATE REASONING")
+            .build();
+        provenance.tokens = TokenUsage::full(100, 40, 10, 60, 20);
+        provenance.tokens.total_tokens = 140; // Reasoning already included by this provider.
+        provenance.cost = atomic_core::change::Cost::from_micro_usd(amount);
+        change.add_provenance(provenance);
+        let hash = Hash::of(name.as_bytes());
+        let mut node = KgNode::new(iri::mint_change(&hash), C::Change, name, "vcs");
+        set_change_records(&mut node, Some(change_records(&change, &hash))).unwrap();
+        node
+    }
+    fn usage<'a>(nodes: &'a [KgNode], id: &str) -> &'a serde_json::Value {
+        &nodes
+            .iter()
+            .find(|node| node.id == id)
+            .unwrap()
+            .metadata
+            .as_ref()
+            .unwrap()["ai_usage"]
+    }
+
+    #[test]
+    fn intent_scopes_exclude_dependency_only_history_and_deduplicate_all_associations() {
+        let first = change("first", 86_176, "request-a");
+        let dependency = change("dependency", 90_964, "request-b");
+        let project = atomic_core::kg::project_id::ProjectId::from_base32(&"A".repeat(52)).unwrap();
+        let uid = "01ARZ3NDEKTSV4RRFFQ69G5FAV";
+        let intent = KgNode::new(
+            iri::mint_intent(&project, uid).unwrap(),
+            C::FeatureIntent,
+            "intent",
+            "vault",
+        )
+        .with_metadata(serde_json::json!({"vault_path":format!("intents/{uid}/intent.md")}));
+        let artifact = KgNode::new(
+            intent_artifact_id(&intent).unwrap(),
+            C::File,
+            "intent.md",
+            "vcs",
+        );
+        let edges = vec![
+            KgEdge::new(&first.id, &intent.id, P::Implements),
+            KgEdge::new(&intent.id, &first.id, P::DeclaresImplementation),
+            KgEdge::new(&first.id, &artifact.id, P::ModifiesFile),
+            KgEdge::new("turn", &intent.id, P::HasPlan),
+            KgEdge::new("turn", &first.id, P::GeneratedChange),
+            KgEdge::new(&first.id, &dependency.id, P::DirectlyDependsOnChange),
+        ];
+        let mut nodes = vec![first.clone(), dependency, intent.clone(), artifact];
+        annotate(&mut nodes, &edges).unwrap();
+        let summary = usage(&nodes, &intent.id);
+        assert_eq!(summary["coverage"]["changes"], 1);
+        assert_eq!(summary["cost"]["amount_micros"], 86_176);
+        assert_eq!(summary["cost"]["amount"], "0.086176");
+        assert_eq!(summary["tokens"]["total"], 140);
+        assert_eq!(summary["tokens"]["reasoning"], 10);
+        assert_eq!(summary["tokens"]["cache_read"], 60);
+        assert_eq!(summary["tokens"]["cache_write"], 20);
+        assert_eq!(summary["contributors"][0]["model"], "model-a");
+        assert_eq!(summary["contributors"][0]["vendor"], "openai");
+        assert_eq!(summary["contributors"][0]["tool"], "CLI: test-agent");
+        let serialized = serde_json::to_string(&nodes).unwrap();
+        assert!(!serialized.contains("PRIVATE"));
+    }
+
+    #[test]
+    fn view_costs_include_ancestor_membership_and_dependencies_once() {
+        let first = change("first", 86_176, "a");
+        let second = change("second", 90_964, "b");
+        let child = KgNode::new("child", C::View, "child", "vcs");
+        let parent = KgNode::new("parent", C::View, "parent", "vcs");
+        let edges = vec![
+            KgEdge::new("child", "parent", P::ChildViewOf),
+            KgEdge::new("parent", "child", P::ChildViewOf), // cycles must terminate
+            KgEdge::new(&first.id, "child", P::DirectlyOnView),
+            KgEdge::new(&first.id, "parent", P::DirectlyOnView),
+            KgEdge::new(&first.id, &second.id, P::DirectlyDependsOnChange),
+        ];
+        let mut nodes = vec![first, second, child, parent];
+        annotate(&mut nodes, &edges).unwrap();
+        let summary = usage(&nodes, "child");
+        assert_eq!(summary["coverage"]["changes"], 2);
+        assert_eq!(summary["cost"]["amount_micros"], 177_140);
+        assert_eq!(summary["tokens"]["total"], 280);
+    }
+
+    #[test]
+    fn shared_requests_are_not_billed_twice_and_conflicts_are_not_guessed() {
+        let first = change("first", 123, "shared-request");
+        let second = change("second", 123, "shared-request");
+        let view = KgNode::new("view", C::View, "view", "vcs");
+        let edges = vec![
+            KgEdge::new(&first.id, "view", P::DirectlyOnView),
+            KgEdge::new(&second.id, "view", P::DirectlyOnView),
+        ];
+        let mut nodes = vec![first.clone(), second, view.clone()];
+        annotate(&mut nodes, &edges).unwrap();
+        assert_eq!(usage(&nodes, "view")["cost"]["amount_micros"], 123);
+        assert_eq!(
+            usage(&nodes, "view")["coverage"]["duplicate_usage_records"],
+            1
+        );
+        nodes[1] = change("second", 456, "shared-request");
+        annotate(&mut nodes, &edges).unwrap();
+        assert!(usage(&nodes, "view")["cost"].is_null());
+        assert!(usage(&nodes, "view")["tokens"].is_null());
+        assert_eq!(
+            usage(&nodes, "view")["coverage"]["conflicting_usage_records"],
+            1
+        );
+        assert_eq!(usage(&nodes, "view")["contributors"][0]["model"], "model-a");
+    }
+
+    #[test]
+    fn missing_provenance_is_unknown_not_free_and_dangling_members_are_counted() {
+        let mut human = KgNode::new("human", C::Change, "human", "vcs");
+        set_change_records(&mut human, Some(vec![])).unwrap();
+        let mut absent = KgNode::new("absent", C::Change, "absent", "vcs");
+        set_change_records(&mut absent, None).unwrap();
+        let view = KgNode::new("view", C::View, "view", "vcs");
+        let edges = vec![
+            KgEdge::new("human", "view", P::DirectlyOnView),
+            KgEdge::new("absent", "view", P::DirectlyOnView),
+            KgEdge::new("missing", "view", P::DirectlyOnView),
+            KgEdge::new("view", "missing-parent", P::ChildViewOf),
+        ];
+        let mut nodes = vec![human, absent, view];
+        annotate(&mut nodes, &edges).unwrap();
+        let summary = usage(&nodes, "view");
+        assert!(summary["cost"].is_null() && summary["tokens"].is_null());
+        assert_eq!(summary["coverage"]["without_provenance"], 1);
+        assert_eq!(summary["coverage"]["unavailable"], 2);
+        assert_eq!(summary["scope_complete"], false);
+    }
+
+    #[test]
+    fn checked_totals_reject_overflow_and_invalid_metadata() {
+        let mut tokens = Tokens {
+            input: u64::MAX,
+            ..Default::default()
+        };
+        assert!(tokens
+            .add(&Tokens {
+                input: 1,
+                ..Default::default()
+            })
+            .is_err());
+        assert!(add(u64::MAX, 1).is_err());
+        let mut node = KgNode::new("malformed", C::Change, "malformed", "vcs")
+            .with_metadata(serde_json::json!("not an object"));
+        assert!(set_change_records(&mut node, None).is_err());
+    }
+
+    #[test]
+    fn attribution_preserves_multiple_models_and_unreported_usage_is_not_zero() {
+        let mut first = change("first", 123, "a");
+        let second = change("second", 456, "b");
+        let mut records: Vec =
+            serde_json::from_value(first.metadata.as_ref().unwrap()["ai_usage_records"].clone())
+                .unwrap();
+        let mut additional: Vec =
+            serde_json::from_value(second.metadata.as_ref().unwrap()["ai_usage_records"].clone())
+                .unwrap();
+        additional[0].model = "model-b".into();
+        additional[0].vendor = "anthropic".into();
+        additional[0].tool = "Editor: example".into();
+        records.extend(additional);
+        set_change_records(&mut first, Some(records)).unwrap();
+        let id = first.id.clone();
+        let mut nodes = vec![first];
+        annotate(&mut nodes, &[]).unwrap();
+        assert_eq!(
+            usage(&nodes, &id)["contributors"].as_array().unwrap().len(),
+            2
+        );
+        assert_eq!(usage(&nodes, &id)["cost"]["amount_micros"], 579);
+
+        let mut change = Change::empty(ChangeHeader::new("model known, usage unreported"));
+        change.add_provenance(
+            Provenance::builder()
+                .vendor(AIVendor::OpenAI)
+                .model("model-a")
+                .build(),
+        );
+        let hash = Hash::of(b"unreported");
+        let mut node = KgNode::new(iri::mint_change(&hash), C::Change, "unreported", "vcs");
+        set_change_records(&mut node, Some(change_records(&change, &hash))).unwrap();
+        let id = node.id.clone();
+        let mut nodes = vec![node];
+        annotate(&mut nodes, &[]).unwrap();
+        assert!(usage(&nodes, &id)["tokens"].is_null());
+        assert!(usage(&nodes, &id)["cost"].is_null());
+        assert_eq!(usage(&nodes, &id)["contributors"][0]["model"], "model-a");
+        assert_eq!(usage(&nodes, &id)["coverage"]["records_without_cost"], 1);
+        assert_eq!(usage(&nodes, &id)["coverage"]["records_without_tokens"], 1);
+    }
+
+    #[test]
+    fn session_turn_identity_deduplicates_split_changes_but_unidentified_records_remain_distinct() {
+        let mut change = Change::empty(ChangeHeader::new("split turn"));
+        change.add_provenance(
+            Provenance::builder()
+                .vendor(AIVendor::OpenAI)
+                .model("model-a")
+                .tokens(10, 20)
+                .session_id("session")
+                .metadata("turn_number", "1")
+                .cost_usd(0.01)
+                .build(),
+        );
+        let first = change_records(&change, &Hash::of(b"first"));
+        let second = change_records(&change, &Hash::of(b"second"));
+        assert_eq!(first[0].id, second[0].id);
+        change.hashed.provenance[0].session_id = None;
+        let first = change_records(&change, &Hash::of(b"first"));
+        let second = change_records(&change, &Hash::of(b"second"));
+        assert_ne!(first[0].id, second[0].id);
+        change.hashed.provenance[0].cost = atomic_core::change::Cost::zero();
+        assert!(change_records(&change, &Hash::of(b"zero"))[0]
+            .cost_usd_micros
+            .is_none());
+        change.hashed.provenance[0]
+            .metadata
+            .push(("cost_reported".into(), "true".into()));
+        assert_eq!(
+            change_records(&change, &Hash::of(b"zero"))[0].cost_usd_micros,
+            Some(0)
+        );
+    }
+
+    #[test]
+    fn canonical_view_scopes_override_stale_edges_and_identify_the_actual_change_set() {
+        let first = change("first", 123, "a");
+        let second = change("second", 123, "b");
+        let view = KgNode::new("view", C::View, "view", "vcs");
+        let stale_edges = vec![KgEdge::new(&first.id, "view", P::DirectlyOnView)];
+        let mut scopes = BTreeMap::from([(
+            "view".into(),
+            ViewUsageScope {
+                changes: BTreeSet::from([second.id.clone()]),
+                error: None,
+            },
+        )]);
+        let mut nodes = vec![first.clone(), second, view];
+        annotate_with_views(&mut nodes, &stale_edges, Some(&scopes)).unwrap();
+        let second_hash = usage(&nodes, "view")["change_set_hash"].clone();
+        assert_eq!(usage(&nodes, "view")["cost"]["amount_micros"], 123);
+        scopes.get_mut("view").unwrap().changes = BTreeSet::from([first.id]);
+        annotate_with_views(&mut nodes, &stale_edges, Some(&scopes)).unwrap();
+        assert_ne!(usage(&nodes, "view")["change_set_hash"], second_hash);
+        scopes.get_mut("view").unwrap().changes.clear();
+        scopes.get_mut("view").unwrap().error = Some("unavailable canonical membership".into());
+        annotate_with_views(&mut nodes, &stale_edges, Some(&scopes)).unwrap();
+        assert_eq!(usage(&nodes, "view")["scope_complete"], false);
+        assert!(usage(&nodes, "view")["cost"].is_null());
+    }
+}
diff --git a/atomic-repository/tests/implementation_declaration.rs b/atomic-repository/tests/implementation_declaration.rs
new file mode 100644
index 00000000..2444abaa
--- /dev/null
+++ b/atomic-repository/tests/implementation_declaration.rs
@@ -0,0 +1,65 @@
+use atomic_core::{
+    change::{Change, ChangeHeader},
+    kg::{iri, ontology::Predicate as P},
+    pristine::VaultEntryType,
+};
+use atomic_repository::Repository;
+
+const UID: &str = "01ARZ3NDEKTSV4RRFFQ69G5FAV";
+
+#[test]
+fn explicit_declaration_projects_and_rebuilds_without_manufacturing_captured_provenance() {
+    let dir = tempfile::tempdir().unwrap();
+    let repo = Repository::init(dir.path()).unwrap();
+    repo.init_vault().unwrap();
+    let mut bytes = vec![];
+    let hash = Change::empty(ChangeHeader::new("Mixed parent change"))
+        .serialize(&mut bytes)
+        .unwrap();
+    let change = iri::mint_change(&hash);
+    let path = format!("intents/{UID}/intent.md");
+    let evidence = "Explicit author repair; original child publication failed, parent change includes implementation.";
+    repo.vault_store(&path, VaultEntryType::Intent,
+        format!(":::why\nDeclare without inventing an original turn.\n:::\n::ref{{to={change} edge=declaresImplementation declarationEvidence=\"{evidence}\"}}\n").into_bytes(),
+        serde_json::json!({"uid":UID,"id":"TEST::alice::1","title":"Repair declaration","status":"in_progress"}).to_string()).unwrap();
+    repo.vault_reconcile_intent_manifest().unwrap();
+    let root = iri::mint_intent(&repo.project_id().unwrap(), UID).unwrap();
+    let snapshot = repo.workflow_snapshot().unwrap();
+    assert!(!snapshot
+        .edges
+        .iter()
+        .any(|e| e.kind == P::DeclaresImplementation));
+    assert!(snapshot
+        .nodes
+        .iter()
+        .any(|n| n.label == "UnresolvedReference"));
+    repo.import_change_bytes(&hash, &bytes).unwrap();
+    for _ in 0..2 {
+        repo.rebuild_kg(true).unwrap();
+        let graph = repo.intent_graph(&root, 100).unwrap();
+        assert_eq!(graph.implementing_changes, vec![change.clone()]);
+        assert_eq!(graph.declared_changes, vec![change.clone()]);
+        assert!(graph.captured_changes.is_empty());
+        let edge = graph
+            .edges
+            .iter()
+            .find(|e| e.kind == P::DeclaresImplementation)
+            .unwrap();
+        assert_eq!(
+            edge.metadata.as_ref().unwrap()["declarationEvidence"],
+            evidence
+        );
+        assert!(!graph
+            .edges
+            .iter()
+            .any(|e| matches!(e.kind, P::Implements | P::HasPlan | P::GeneratedChange)));
+        assert!(graph
+            .diagnostics
+            .iter()
+            .any(|d| d.contains("do not repair or prove")));
+    }
+    assert_eq!(
+        repo.load_change(&hash).unwrap().hashed.header.message,
+        "Mixed parent change"
+    );
+}
diff --git a/atomic-repository/tests/intent_graph.rs b/atomic-repository/tests/intent_graph.rs
new file mode 100644
index 00000000..374f5cf0
--- /dev/null
+++ b/atomic-repository/tests/intent_graph.rs
@@ -0,0 +1,295 @@
+use atomic_core::{
+    kg::{
+        iri,
+        ontology::{Class as C, Predicate as P, VERSION_IRI},
+        project_id::ProjectId,
+    },
+    types::Hash,
+    KgEdge, KgNode,
+};
+use atomic_repository::{intent_graph::IntentGraph, repository::WorkflowSnapshot};
+
+fn fixture() -> (WorkflowSnapshot, String, String, String, String) {
+    let project = ProjectId::from_base32(&"A".repeat(52)).unwrap();
+    let root = iri::mint_intent(&project, "01ARZ3NDEKTSV4RRFFQ69G5FAV").unwrap();
+    let other = iri::mint_intent(&project, "01ARZ3NDEKTSV4RRFFQ69G5FAW").unwrap();
+    let change = iri::mint_change(&Hash::of(b"implementation"));
+    let dependency = iri::mint_change(&Hash::of(b"dependency"));
+    let unrelated = iri::mint_change(&Hash::of(b"unrelated"));
+    let mut nodes = vec![];
+    for (id, class) in [
+        (root.as_str(), C::FeatureIntent),
+        (other.as_str(), C::WorkIntent),
+        (change.as_str(), C::Change),
+        (dependency.as_str(), C::Change),
+        (unrelated.as_str(), C::Change),
+        ("urn:turn:one", C::SessionTurn),
+        ("urn:turn:two", C::SessionTurn),
+        ("urn:session:shared", C::Session),
+        ("urn:file:shared", C::File),
+        ("urn:author:shared", C::Identity),
+        ("urn:attestation:shared", C::Attestation),
+        ("urn:provenance:one", C::ProvenanceGraph),
+        ("urn:view:child", C::View),
+        ("urn:view:parent", C::View),
+        ("urn:task:one", C::Task),
+        ("urn:ac:one", C::AcceptanceCriterion),
+        ("urn:finding:gap", C::Finding),
+    ] {
+        nodes.push(KgNode::new(id, class, id, "fixture"));
+    }
+    let mut edges = vec![];
+    for (from, to, p) in [
+        (change.as_str(), root.as_str(), P::Implements),
+        (
+            change.as_str(),
+            dependency.as_str(),
+            P::DirectlyDependsOnChange,
+        ),
+        (
+            dependency.as_str(),
+            change.as_str(),
+            P::DirectlyDependsOnChange,
+        ), // cycle safety
+        ("urn:turn:one", root.as_str(), P::HasPlan),
+        ("urn:turn:one", change.as_str(), P::GeneratedChange),
+        ("urn:turn:two", other.as_str(), P::HasPlan),
+        ("urn:turn:two", unrelated.as_str(), P::GeneratedChange),
+        ("urn:session:shared", "urn:turn:one", P::HasTurn),
+        ("urn:session:shared", "urn:turn:two", P::HasTurn),
+        (change.as_str(), "urn:file:shared", P::ModifiesFile),
+        (unrelated.as_str(), "urn:file:shared", P::ModifiesFile),
+        (change.as_str(), "urn:author:shared", P::AuthoredBy),
+        (unrelated.as_str(), "urn:author:shared", P::AuthoredBy),
+        ("urn:attestation:shared", change.as_str(), P::CoversChange),
+        (
+            "urn:attestation:shared",
+            unrelated.as_str(),
+            P::CoversChange,
+        ),
+        (
+            "urn:attestation:shared",
+            "https://atomic.dev/ns/workflow#Verified",
+            P::VerificationResult,
+        ),
+        ("urn:turn:one", "urn:provenance:one", P::TurnExplainedBy),
+        (change.as_str(), "urn:view:child", P::DirectlyOnView),
+        ("urn:view:child", "urn:view:parent", P::ChildViewOf),
+        (root.as_str(), "urn:task:one", P::HasTask),
+        (root.as_str(), "urn:ac:one", P::HasAcceptanceCriterion),
+        ("urn:task:one", "urn:ac:one", P::Satisfies),
+        ("urn:finding:gap", dependency.as_str(), P::AboutResource),
+    ] {
+        edges.push(KgEdge::new(from, to, p));
+    }
+    (
+        WorkflowSnapshot {
+            generation: atomic_core::kg::KG_GENERATION,
+            ontology_version: VERSION_IRI,
+            identity: "consistent-snapshot".into(),
+            nodes,
+            edges,
+        },
+        root,
+        change,
+        dependency,
+        unrelated,
+    )
+}
+
+#[test]
+fn selects_semantic_lineage_without_acquiring_shared_resource_neighbors() {
+    let (snapshot, root, change, dependency, unrelated) = fixture();
+    let graph = IntentGraph::from_snapshot(&snapshot, &root, 100).unwrap();
+    assert_eq!(graph.implementing_changes, vec![change]);
+    assert_eq!(graph.dependency_changes, vec![dependency]);
+    for id in [
+        "urn:session:shared",
+        "urn:provenance:one",
+        "urn:attestation:shared",
+        "urn:finding:gap",
+        "urn:ac:one",
+        "urn:view:parent",
+        "https://atomic.dev/ns/workflow#Verified",
+    ] {
+        assert!(graph.nodes.iter().any(|n| n.id == id), "missing {id}");
+    }
+    assert!(!graph
+        .nodes
+        .iter()
+        .any(|n| n.id == unrelated || n.id == "urn:turn:two"));
+    assert!(!graph.truncated);
+    assert!(graph.missing_resources.is_empty());
+    assert!(graph.edges.iter().any(|e| e.kind == P::Satisfies));
+    assert_eq!(graph.snapshot_identity, "consistent-snapshot");
+    assert!(graph
+        .diagnostics
+        .iter()
+        .any(|d| d.contains("object unknown")));
+}
+
+#[test]
+fn selection_and_limits_are_deterministic_and_explicit() {
+    let (mut snapshot, root, _, _, _) = fixture();
+    let graph = IntentGraph::from_snapshot(&snapshot, &root, 8).unwrap();
+    assert!(graph.truncated);
+    assert_eq!(graph.nodes.len(), 8);
+    assert!(graph.nodes.iter().any(|n| n.id == root));
+    snapshot.nodes.reverse();
+    snapshot.edges.reverse();
+    let reversed = IntentGraph::from_snapshot(&snapshot, &root, 8).unwrap();
+    assert_eq!(
+        serde_json::to_value(graph).unwrap(),
+        serde_json::to_value(reversed).unwrap()
+    );
+    for limit in [0, 10001] {
+        assert!(IntentGraph::from_snapshot(&snapshot, &root, limit).is_err());
+    }
+    assert!(IntentGraph::from_snapshot(&snapshot, "intent:abbreviated", 100).is_err());
+    snapshot.generation = 0;
+    assert!(IntentGraph::from_snapshot(&snapshot, &root, 100).is_err());
+}
+
+#[test]
+fn missing_resources_and_empty_implementation_are_not_silent_success_claims() {
+    let (mut snapshot, root, _, _, _) = fixture();
+    snapshot.edges.retain(|e| e.from_id == root);
+    snapshot
+        .edges
+        .push(KgEdge::new(&root, "urn:missing:task", P::HasTask));
+    let graph = IntentGraph::from_snapshot(&snapshot, &root, 100).unwrap();
+    assert!(graph.implementing_changes.is_empty());
+    assert_eq!(graph.missing_resources, vec!["urn:missing:task"]);
+    assert!(graph.diagnostics.iter().any(|d| d.contains("not proof")));
+    snapshot.nodes.retain(|n| n.id != root);
+    assert!(IntentGraph::from_snapshot(&snapshot, &root, 100).is_err());
+}
+
+#[test]
+fn authoritative_turn_path_seeds_changes_without_requiring_shortcuts() {
+    let (mut snapshot, root, change, dependency, _) = fixture();
+    snapshot.edges.retain(|e| e.kind != P::Implements);
+    let graph = IntentGraph::from_snapshot(&snapshot, &root, 100).unwrap();
+    assert_eq!(graph.implementing_changes, vec![change]);
+    assert_eq!(graph.dependency_changes, vec![dependency]);
+}
+
+fn recording_fixture() -> (WorkflowSnapshot, String, String, String, String, String) {
+    let (mut snapshot, root, change, dependency, unrelated) = fixture();
+    snapshot
+        .edges
+        .retain(|e| !matches!(e.kind, P::Implements | P::HasPlan | P::GovernsTurn));
+    let (project, uid) = iri::parse_intent(&root).unwrap();
+    let path = format!("intents/{uid}/intent.md");
+    snapshot
+        .nodes
+        .iter_mut()
+        .find(|n| n.id == root)
+        .unwrap()
+        .metadata = Some(serde_json::json!({"vault_path": path}));
+    let artifact = iri::mint_file(&project, &format!(".vault/{path}"));
+    let other_artifact = iri::mint_file(
+        &project,
+        ".vault/intents/01ARZ3NDEKTSV4RRFFQ69G5FAW/intent.md",
+    );
+    for file in [&artifact, &other_artifact] {
+        snapshot.nodes.push(KgNode::new(file, C::File, file, "vcs"));
+    }
+    snapshot
+        .edges
+        .push(KgEdge::new(&change, &artifact, P::ModifiesFile));
+    snapshot
+        .edges
+        .push(KgEdge::new(&unrelated, &other_artifact, P::ModifiesFile));
+    (snapshot, root, change, dependency, unrelated, artifact)
+}
+
+#[test]
+fn exposes_recorded_intent_evidence_without_inventing_implementation() {
+    let (snapshot, root, change, dependency, unrelated, artifact) = recording_fixture();
+    let graph = IntentGraph::from_snapshot(&snapshot, &root, 100).unwrap();
+    assert_eq!(graph.recording_changes, vec![change.clone()]);
+    assert_eq!(graph.intent_artifact.as_ref(), Some(&artifact));
+    assert_eq!(graph.dependency_changes, vec![dependency]);
+    assert!(graph.implementing_changes.is_empty());
+    assert!(graph.captured_changes.is_empty());
+    assert!(graph.declared_changes.is_empty());
+    for id in [
+        &change,
+        &artifact,
+        "urn:turn:one",
+        "urn:provenance:one",
+        "urn:file:shared",
+    ] {
+        assert!(graph.nodes.iter().any(|n| n.id == id), "missing {id}");
+    }
+    assert!(!graph
+        .nodes
+        .iter()
+        .any(|n| n.id == unrelated || n.id == "urn:turn:two"));
+    assert!(
+        graph.edges.iter().all(|edge| snapshot.edges.contains(edge)),
+        "the read must not synthesize semantic edges"
+    );
+    assert!(graph
+        .diagnostics
+        .iter()
+        .any(|d| d.contains("recording evidence")));
+    assert!(!graph
+        .diagnostics
+        .iter()
+        .any(|d| d.starts_with("No implementing changes")));
+}
+
+#[test]
+fn recording_evidence_never_fans_out_from_shared_files_or_other_projects() {
+    let (mut snapshot, root, change, _, _, _) = recording_fixture();
+    snapshot.edges.retain(|edge| {
+        !(edge.from_id == change && edge.kind == P::ModifiesFile && edge.to_id.contains(".vault/"))
+    });
+    let (project, uid) = iri::parse_intent(&root).unwrap();
+    let foreign = ProjectId::from_base32(&format!("B{}", "A".repeat(51))).unwrap();
+    assert_ne!(project, foreign);
+    let foreign_artifact = iri::mint_file(&foreign, &format!(".vault/intents/{uid}/intent.md"));
+    snapshot.nodes.push(KgNode::new(
+        &foreign_artifact,
+        C::File,
+        "same path, other project",
+        "vcs",
+    ));
+    snapshot
+        .edges
+        .push(KgEdge::new(&change, &foreign_artifact, P::ModifiesFile));
+    let graph = IntentGraph::from_snapshot(&snapshot, &root, 100).unwrap();
+    assert!(graph.recording_changes.is_empty());
+    assert!(!graph
+        .nodes
+        .iter()
+        .any(|node| node.id == change || node.id == foreign_artifact));
+}
+
+#[test]
+fn recording_evidence_limits_are_deterministic_and_primary_changes_are_not_dependency_only() {
+    let (mut snapshot, root, change, dependency, _, artifact) = recording_fixture();
+    snapshot
+        .edges
+        .push(KgEdge::new(&dependency, &artifact, P::ModifiesFile));
+    let graph = IntentGraph::from_snapshot(&snapshot, &root, 100).unwrap();
+    assert_eq!(graph.recording_changes.len(), 2);
+    assert!(graph.recording_changes.contains(&change));
+    assert!(graph.recording_changes.contains(&dependency));
+    assert!(graph.dependency_changes.is_empty());
+    for limit in [1, 2, 5, 10] {
+        let capped = IntentGraph::from_snapshot(&snapshot, &root, limit).unwrap();
+        assert!(capped.truncated);
+        assert_eq!(capped.nodes.len(), limit);
+        let mut reversed = snapshot.clone();
+        reversed.nodes.reverse();
+        reversed.edges.reverse();
+        assert_eq!(
+            serde_json::to_value(capped).unwrap(),
+            serde_json::to_value(IntentGraph::from_snapshot(&reversed, &root, limit).unwrap())
+                .unwrap()
+        );
+    }
+}
diff --git a/atomic-repository/tests/kg_change_availability.rs b/atomic-repository/tests/kg_change_availability.rs
new file mode 100644
index 00000000..3159b893
--- /dev/null
+++ b/atomic-repository/tests/kg_change_availability.rs
@@ -0,0 +1,127 @@
+use atomic_core::{
+    change::{Change, ChangeHeader, ProvenanceGraph},
+    kg::{iri, ontology::Predicate as P},
+    pristine::{KgMutTxnT, KgTxnT, MutTxnT},
+};
+use atomic_repository::Repository;
+
+#[test]
+fn ledger_reference_absent_then_arrived_preserves_identity_and_rebuilds() {
+    let directory = tempfile::tempdir().unwrap();
+    let repo = Repository::init(directory.path()).unwrap();
+    repo.init_kg().unwrap();
+    let change = Change::empty(ChangeHeader::new("arrived immutable header"));
+    let mut bytes = Vec::new();
+    let hash = change.serialize(&mut bytes).unwrap();
+    let id = iri::mint_change(&hash);
+    // Graph registration is not evidence that the immutable object exists.
+    {
+        let mut txn = repo.pristine().write_txn().unwrap();
+        txn.register_change(&hash).unwrap();
+        txn.commit().unwrap();
+    }
+    repo.save_provenance_graph(
+        &ProvenanceGraph::builder("availability-session", "opencode")
+            .add_change_explained(hash)
+            .build(),
+    )
+    .unwrap();
+
+    let assert_facts = |availability: &str, completeness: &str| {
+        let node = repo.vault_kg_node(&id).unwrap().unwrap();
+        let metadata = node.metadata.as_ref().unwrap();
+        assert_eq!(metadata["object_availability"], availability);
+        assert_eq!(metadata["projection_completeness"], completeness);
+        assert_eq!(node.id, id);
+        if completeness == "projected" {
+            assert_eq!(node.summary.as_deref(), Some("arrived immutable header"));
+            assert!(metadata.get("created_at").is_some());
+        } else {
+            assert!(metadata.get("created_at").is_none());
+        }
+        assert!(repo
+            .workflow_snapshot()
+            .unwrap()
+            .edges
+            .iter()
+            .any(|edge| edge.kind == P::GeneratedChange && edge.to_id == id));
+        assert!(!repo
+            .workflow_snapshot()
+            .unwrap()
+            .edges
+            .iter()
+            .any(|edge| edge.kind == P::DirectlyOnView && edge.from_id == id));
+    };
+    assert!(!repo.has_change(&hash));
+    assert_facts("absent", "reference-only");
+    repo.rebuild_kg(true).unwrap();
+    assert_facts("absent", "reference-only");
+
+    // The core ledger projector cannot inspect the object store. Replaying it
+    // alone must say unknown, rather than guessing absence from registration.
+    {
+        let mut txn = repo.pristine().write_txn().unwrap();
+        txn.del_kg_node(&id).unwrap();
+        for (record, turns) in txn.kg_session_rows().unwrap() {
+            txn.project_session_kg(&record, &turns).unwrap();
+        }
+        let node = txn.get_kg_node(&id).unwrap().unwrap();
+        assert_eq!(node.metadata.unwrap()["object_availability"], "unknown");
+        txn.commit().unwrap();
+    }
+    for _ in 0..2 {
+        repo.import_change_bytes(&hash, &bytes).unwrap();
+        assert_facts("available", "projected");
+    }
+    repo.rebuild_kg(true).unwrap();
+    assert_facts("available", "projected");
+
+    // Ledger replay must not downgrade an already verified projection.
+    repo.save_provenance_graph(
+        &ProvenanceGraph::builder("availability-session", "opencode")
+            .add_change_explained(hash)
+            .build(),
+    )
+    .unwrap();
+    assert_facts("available", "projected");
+}
+
+#[test]
+fn unverified_object_existence_cannot_upgrade_availability() {
+    use atomic_core::types::Base32;
+
+    let directory = tempfile::tempdir().unwrap();
+    let repo = Repository::init(directory.path()).unwrap();
+    repo.init_kg().unwrap();
+    let mut bytes = Vec::new();
+    let hash = Change::empty(ChangeHeader::new("expected object"))
+        .serialize(&mut bytes)
+        .unwrap();
+    repo.save_provenance_graph(
+        &ProvenanceGraph::builder("corrupt-availability", "opencode")
+            .add_change_explained(hash)
+            .build(),
+    )
+    .unwrap();
+    let id = iri::mint_change(&hash);
+    let encoded = hash.to_base32();
+    let path = repo
+        .changes_dir()
+        .join(&encoded[..2])
+        .join(format!("{encoded}.change"));
+    std::fs::create_dir_all(path.parent().unwrap()).unwrap();
+    let mut wrong_bytes = Vec::new();
+    Change::empty(ChangeHeader::new("different valid object"))
+        .serialize(&mut wrong_bytes)
+        .unwrap();
+    std::fs::write(path, wrong_bytes).unwrap();
+    assert!(repo.has_change(&hash));
+    assert!(repo.rebuild_kg(true).is_err());
+    let node = repo.vault_kg_node(&id).unwrap().unwrap();
+    assert_eq!(node.metadata.unwrap()["object_availability"], "absent");
+    // A failed rebuild is atomic; only verified arrival can replace the last
+    // successful store observation with available/projected.
+    repo.import_change_bytes(&hash, &bytes).unwrap();
+    let node = repo.vault_kg_node(&id).unwrap().unwrap();
+    assert_eq!(node.metadata.unwrap()["object_availability"], "available");
+}
diff --git a/atomic-repository/tests/kg_recorded_impact.rs b/atomic-repository/tests/kg_recorded_impact.rs
new file mode 100644
index 00000000..cd127249
--- /dev/null
+++ b/atomic-repository/tests/kg_recorded_impact.rs
@@ -0,0 +1,99 @@
+use std::collections::BTreeSet;
+
+use atomic_core::{
+    change::{ops::FileOps, Change, ChangeHeader},
+    crdt::TrunkId,
+    kg::{iri, ontology::Predicate as P},
+    types::{Hash, NodeId},
+};
+use atomic_repository::Repository;
+
+fn import_recorded_files(repo: &Repository, paths: &[&str]) -> Hash {
+    // Stored semantic operations are sufficient for impact projection; no
+    // working-copy files or view membership are required.
+    let file_ops = paths
+        .iter()
+        .enumerate()
+        .map(|(index, path)| {
+            FileOps::create(
+                TrunkId::new(NodeId::new(1), index as u32),
+                (*path).into(),
+                None,
+            )
+        })
+        .collect();
+    let change = Change::with_file_ops(
+        ChangeHeader::new("historically recorded files"),
+        vec![],
+        file_ops,
+        vec![],
+        vec![],
+    );
+    let mut bytes = Vec::new();
+    let hash = change.serialize(&mut bytes).unwrap();
+    repo.import_change_bytes(&hash, &bytes).unwrap();
+    hash
+}
+
+fn impact_facts(repo: &Repository, hash: &Hash) -> BTreeSet<(String, String)> {
+    let change = iri::mint_change(hash);
+    repo.workflow_snapshot()
+        .unwrap()
+        .edges
+        .into_iter()
+        .filter(|edge| edge.from_id == change && edge.kind == P::ModifiesFile)
+        .map(|edge| (edge.from_id, edge.to_id))
+        .collect()
+}
+
+#[test]
+fn recorded_ignored_file_still_projects_modifies_file() {
+    for ignore_file in [".atomicignore", ".gitignore", ".ignore"] {
+        let directory = tempfile::tempdir().unwrap();
+        let repo = Repository::init(directory.path()).unwrap();
+        repo.init_kg().unwrap();
+        let path = "historical/generated.rs";
+        std::fs::write(directory.path().join(ignore_file), "historical/\n").unwrap();
+
+        let hash = import_recorded_files(&repo, &[path]);
+        let file = iri::mint_file(&repo.project_id().unwrap(), path);
+        assert_eq!(
+            impact_facts(&repo, &hash),
+            BTreeSet::from([(iri::mint_change(&hash), file.clone())]),
+            "{ignore_file} must not hide recorded impact"
+        );
+        let node = repo.vault_kg_node(&file).unwrap().unwrap();
+        assert_eq!(node.metadata.unwrap()["path"], path);
+    }
+}
+
+#[test]
+fn rebuilding_after_ignore_changes_preserves_recorded_impact_facts() {
+    let directory = tempfile::tempdir().unwrap();
+    let repo = Repository::init(directory.path()).unwrap();
+    repo.init_kg().unwrap();
+    let paths = ["historical/first.rs", "historical/second.rs"];
+    let hash = import_recorded_files(&repo, &paths);
+    let expected: BTreeSet<_> = paths
+        .iter()
+        .map(|path| {
+            (
+                iri::mint_change(&hash),
+                iri::mint_file(&repo.project_id().unwrap(), path),
+            )
+        })
+        .collect();
+    assert_eq!(impact_facts(&repo, &hash), expected);
+
+    for ignore_file in [".atomicignore", ".gitignore", ".ignore"] {
+        for rules in ["historical/\n", "# no longer ignored\n"] {
+            std::fs::write(directory.path().join(ignore_file), rules).unwrap();
+            assert!(repo.rebuild_kg(true).unwrap());
+            assert_eq!(
+                impact_facts(&repo, &hash),
+                expected,
+                "rebuilding with {ignore_file} containing {rules:?} changed historical impact"
+            );
+        }
+    }
+}
diff --git a/atomic-repository/tests/patch_evidence.rs b/atomic-repository/tests/patch_evidence.rs
new file mode 100644
index 00000000..2d03c3cf
--- /dev/null
+++ b/atomic-repository/tests/patch_evidence.rs
@@ -0,0 +1,359 @@
+use atomic_core::change::{
+    Atom, Change, ChangeHeader, EdgeUpdate, FileOps, GraphOp, Insertion, Local, NewEdge,
+};
+use atomic_core::crdt::TrunkId;
+use atomic_core::kg::iri;
+use atomic_core::types::{Base32, ChangePosition, EdgeFlags, GraphNode, Hash, NodeId, Position};
+use atomic_repository::patch_evidence::{
+    PatchEvidenceLimits, PatchEvidenceStatus, RemovedPreimageStatus, MAX_EVIDENCE_CONTENT_BYTES,
+    MAX_EVIDENCE_OBJECT_BYTES, MAX_EVIDENCE_OPERATIONS, MAX_EVIDENCE_OPERATION_BYTES,
+};
+use atomic_repository::{execute_plan, parse_plan, QueryPlan, QueryStep, Repository};
+
+fn fixture(content: &[u8]) -> Change {
+    let insertion = Insertion {
+        predecessors: vec![],
+        successors: vec![],
+        flag: EdgeFlags::BLOCK,
+        start: ChangePosition::new(0),
+        end: ChangePosition::new(content.len() as u64),
+        inode: Position::new(None, ChangePosition::new(0)),
+    };
+    Change::with_file_ops(
+        ChangeHeader::builder().message("recorded evidence").build(),
+        vec![GraphOp::Edit {
+            change: Atom::Insertion(insertion),
+            local: Local::new("evidence.txt", 1),
+            encoding: None,
+        }],
+        vec![FileOps::create(
+            TrunkId::new(NodeId::new(1), 0),
+            "evidence.txt".into(),
+            None,
+        )],
+        content.to_vec(),
+        vec![],
+    )
+}
+
+#[test]
+fn patch_evidence_is_recorded_and_unchanged_after_working_copy_edits() {
+    let dir = tempfile::tempdir().unwrap();
+    let repo = Repository::init(dir.path()).unwrap();
+    let change = fixture(b"original\0\xff\n");
+    let hash = repo.save_change(&change).unwrap();
+    let change_iri = iri::mint_change(&hash);
+    let plan = parse_plan(
+        &serde_json::json!({
+            "schema_version": 1,
+            "steps": [{"type": "patch_evidence", "change_iri": change_iri}]
+        })
+        .to_string(),
+    )
+    .unwrap();
+    let before = execute_plan(&repo, &plan).unwrap();
+    std::fs::write(dir.path().join("evidence.txt"), b"MUTABLE UNRELATED BYTES").unwrap();
+    let after = execute_plan(&repo, &plan).unwrap();
+    assert_eq!(before.patch_evidence, after.patch_evidence);
+    assert!(after.nodes.is_empty());
+    assert!(after.edges.is_empty());
+    assert!(after.content.is_empty());
+    let evidence = &after.patch_evidence[0];
+    assert_eq!(evidence.status, PatchEvidenceStatus::Found);
+    assert_eq!(
+        evidence.graph_ops.as_ref().unwrap().operations,
+        change.hashed.hunks
+    );
+    assert_eq!(
+        evidence.file_ops.as_ref().unwrap().operations,
+        change.file_ops()
+    );
+    assert_eq!(
+        evidence.content_blob.as_ref().unwrap().bytes,
+        change.contents
+    );
+    assert!(!evidence.content_blob.as_ref().unwrap().truncated);
+    assert_eq!(
+        evidence.removed_preimage,
+        RemovedPreimageStatus::Unsupported
+    );
+    // Exercise the verified disk path too, not only the in-memory store cache.
+    drop(repo);
+    let reopened = Repository::open(dir.path()).unwrap();
+    assert_eq!(
+        evidence,
+        &reopened
+            .patch_evidence(&change_iri, PatchEvidenceLimits::default())
+            .unwrap()
+    );
+}
+
+#[test]
+fn patch_evidence_rejects_noncanonical_iris_and_hashes() {
+    let dir = tempfile::tempdir().unwrap();
+    let repo = Repository::init(dir.path()).unwrap();
+    let hash = Hash::of(b"identity");
+    let canonical = iri::mint_change(&hash);
+    for invalid in [
+        hash.to_base32(),
+        canonical.to_lowercase(),
+        canonical[..canonical.len() - 1].into(),
+        format!("{canonical}A"),
+        format!("{canonical}#fragment"),
+        iri::mint_provenance(&hash),
+        "change:ABC".into(),
+        "$results".into(),
+    ] {
+        assert!(
+            repo.patch_evidence(&invalid, PatchEvidenceLimits::default())
+                .is_err(),
+            "{invalid}"
+        );
+    }
+    // 52 Base32 characters with non-zero padding bits are not a canonical hash.
+    let mut noncanonical_padding = canonical.clone();
+    noncanonical_padding.pop();
+    noncanonical_padding.push('B');
+    assert!(repo
+        .patch_evidence(&noncanonical_padding, PatchEvidenceLimits::default())
+        .is_err());
+    assert_eq!(
+        repo.patch_evidence(&canonical, PatchEvidenceLimits::default())
+            .unwrap()
+            .status,
+        PatchEvidenceStatus::Missing
+    );
+}
+
+#[test]
+fn patch_evidence_missing_is_not_empty_or_unsupported_evidence() {
+    let dir = tempfile::tempdir().unwrap();
+    let repo = Repository::init(dir.path()).unwrap();
+    let missing = repo
+        .patch_evidence(
+            &iri::mint_change(&Hash::of(b"absent")),
+            PatchEvidenceLimits::default(),
+        )
+        .unwrap();
+    assert_eq!(missing.status, PatchEvidenceStatus::Missing);
+    assert!(missing.graph_ops.is_none());
+    assert!(missing.file_ops.is_none());
+    assert!(missing.content_blob.is_none());
+    let hash = repo
+        .save_change(&Change::empty(
+            ChangeHeader::builder().message("empty").build(),
+        ))
+        .unwrap();
+    let empty = repo
+        .patch_evidence(&iri::mint_change(&hash), PatchEvidenceLimits::default())
+        .unwrap();
+    assert_eq!(empty.status, PatchEvidenceStatus::Found);
+    let operations = empty.file_ops.unwrap();
+    assert_eq!(operations.total, 0);
+    assert!(!operations.truncated);
+    assert_eq!(empty.content_blob.unwrap().total_bytes, 0);
+}
+
+#[test]
+fn patch_evidence_preserves_removed_references_without_inventing_preimage_bytes() {
+    let dir = tempfile::tempdir().unwrap();
+    let repo = Repository::init(dir.path()).unwrap();
+    let dependency = repo.save_change(&fixture(b"historical preimage")).unwrap();
+    let deletion = EdgeUpdate {
+        inode: Position::new(Some(dependency), ChangePosition::new(0)),
+        edges: vec![NewEdge {
+            previous: EdgeFlags::BLOCK,
+            flag: EdgeFlags::BLOCK | EdgeFlags::DELETED,
+            from: Position::new(Some(dependency), ChangePosition::new(0)),
+            to: GraphNode::new(
+                Some(dependency),
+                ChangePosition::new(0),
+                ChangePosition::new(19),
+            ),
+            introduced_by: Some(dependency),
+        }],
+    };
+    let change = Change::with_file_ops(
+        ChangeHeader::builder().message("delete evidence").build(),
+        vec![GraphOp::FileDel {
+            del: deletion.clone(),
+            contents: Some(deletion),
+            path: "evidence.txt".into(),
+            encoding: None,
+        }],
+        vec![FileOps::delete(
+            TrunkId::new(NodeId::new(1), 0),
+            "evidence.txt".into(),
+        )],
+        vec![],
+        vec![dependency],
+    );
+    let hash = repo.save_change(&change).unwrap();
+    std::fs::write(dir.path().join("evidence.txt"), b"not historical evidence").unwrap();
+    let evidence = repo
+        .patch_evidence(&iri::mint_change(&hash), PatchEvidenceLimits::default())
+        .unwrap();
+    assert_eq!(evidence.graph_ops.unwrap().operations, change.hashed.hunks);
+    assert_eq!(evidence.file_ops.unwrap().operations, change.file_ops());
+    assert!(evidence.content_blob.unwrap().bytes.is_empty());
+    assert_eq!(
+        evidence.removed_preimage,
+        RemovedPreimageStatus::Unsupported
+    );
+}
+
+#[test]
+fn patch_evidence_enforces_count_byte_and_content_bounds() {
+    let dir = tempfile::tempdir().unwrap();
+    let repo = Repository::init(dir.path()).unwrap();
+    let change = fixture(&vec![0xff; MAX_EVIDENCE_CONTENT_BYTES + 1]);
+    let hash = repo.save_change(&change).unwrap();
+    let change_iri = iri::mint_change(&hash);
+    let limits = PatchEvidenceLimits {
+        max_operations: 1,
+        max_operation_bytes: MAX_EVIDENCE_OPERATION_BYTES,
+        max_content_bytes: 3,
+    };
+    let bounded = repo.patch_evidence(&change_iri, limits).unwrap();
+    assert_eq!(bounded.graph_ops.as_ref().unwrap().operations.len(), 1);
+    assert!(bounded.file_ops.as_ref().unwrap().truncated);
+    let blob = bounded.content_blob.unwrap();
+    assert_eq!(blob.bytes, vec![0xff; 3]);
+    assert_eq!(blob.total_bytes, change.contents.len());
+    assert!(blob.truncated);
+    let none = repo
+        .patch_evidence(
+            &change_iri,
+            PatchEvidenceLimits {
+                max_operations: 0,
+                max_operation_bytes: 0,
+                max_content_bytes: 0,
+            },
+        )
+        .unwrap();
+    assert!(none.graph_ops.unwrap().truncated);
+    assert!(none.file_ops.unwrap().truncated);
+    assert!(none.content_blob.unwrap().truncated);
+    // A single operation never leaks past the JSON byte budget or gets split.
+    let graph_bytes = serde_json::to_vec(&change.hashed.hunks[0]).unwrap().len();
+    let exact = repo
+        .patch_evidence(
+            &change_iri,
+            PatchEvidenceLimits {
+                max_operation_bytes: graph_bytes,
+                ..limits
+            },
+        )
+        .unwrap();
+    assert_eq!(exact.graph_ops.unwrap().operations.len(), 1);
+    let tight = repo
+        .patch_evidence(
+            &change_iri,
+            PatchEvidenceLimits {
+                max_operation_bytes: 1,
+                ..limits
+            },
+        )
+        .unwrap();
+    assert!(tight.graph_ops.unwrap().operations.is_empty());
+    assert!(tight.file_ops.unwrap().operations.is_empty());
+    for invalid in [
+        PatchEvidenceLimits {
+            max_operations: MAX_EVIDENCE_OPERATIONS + 1,
+            ..limits
+        },
+        PatchEvidenceLimits {
+            max_operation_bytes: MAX_EVIDENCE_OPERATION_BYTES + 1,
+            ..limits
+        },
+        PatchEvidenceLimits {
+            max_content_bytes: MAX_EVIDENCE_CONTENT_BYTES + 1,
+            ..limits
+        },
+    ] {
+        assert!(repo.patch_evidence(&change_iri, invalid).is_err());
+    }
+}
+
+#[test]
+fn patch_evidence_oversized_source_is_explicitly_unsupported() {
+    let dir = tempfile::tempdir().unwrap();
+    let repo = Repository::init(dir.path()).unwrap();
+    let hash = Hash::of(b"oversized");
+    let path = repo.change_store().change_path(&hash);
+    std::fs::create_dir_all(path.parent().unwrap()).unwrap();
+    std::fs::File::create(path)
+        .unwrap()
+        .set_len(MAX_EVIDENCE_OBJECT_BYTES + 1)
+        .unwrap();
+    let result = repo
+        .patch_evidence(&iri::mint_change(&hash), PatchEvidenceLimits::default())
+        .unwrap();
+    assert_eq!(result.status, PatchEvidenceStatus::Unsupported);
+    assert!(result.content_blob.is_none());
+}
+
+#[test]
+fn patch_evidence_corrupt_or_wrong_hash_object_fails_closed() {
+    let dir = tempfile::tempdir().unwrap();
+    let repo = Repository::init(dir.path()).unwrap();
+    let actual = repo.save_change(&fixture(b"authentic")).unwrap();
+    let wrong = Hash::of(b"wrong identity");
+    let wrong_path = repo.change_store().change_path(&wrong);
+    std::fs::create_dir_all(wrong_path.parent().unwrap()).unwrap();
+    std::fs::copy(repo.change_store().change_path(&actual), &wrong_path).unwrap();
+    assert!(repo
+        .patch_evidence(&iri::mint_change(&wrong), PatchEvidenceLimits::default())
+        .is_err());
+    std::fs::write(wrong_path, b"not an ATOM object").unwrap();
+    assert!(repo
+        .patch_evidence(&iri::mint_change(&wrong), PatchEvidenceLimits::default())
+        .is_err());
+}
+
+#[test]
+fn patch_evidence_query_step_has_strict_fields_missing_status_and_fanout_bound() {
+    let dir = tempfile::tempdir().unwrap();
+    let repo = Repository::init(dir.path()).unwrap();
+    let change_iri = iri::mint_change(&Hash::of(b"missing"));
+    let step = QueryStep::PatchEvidence {
+        change_iri: change_iri.clone(),
+        limits: PatchEvidenceLimits::default(),
+    };
+    let result = execute_plan(
+        &repo,
+        &QueryPlan {
+            schema_version: 1,
+            steps: vec![step.clone()],
+        },
+    )
+    .unwrap();
+    assert_eq!(
+        result.patch_evidence[0].status,
+        PatchEvidenceStatus::Missing
+    );
+    assert_eq!(result.step_stats[0].result_count, 0);
+    assert!(execute_plan(
+        &repo,
+        &QueryPlan {
+            schema_version: 1,
+            steps: vec![step; 9]
+        }
+    )
+    .is_err());
+    for extra in [
+        serde_json::json!({"path": "evidence.txt"}),
+        serde_json::json!({"bind": "x"}),
+    ] {
+        let mut value = serde_json::json!({"type": "patch_evidence", "change_iri": change_iri});
+        value
+            .as_object_mut()
+            .unwrap()
+            .extend(extra.as_object().unwrap().clone());
+        assert!(parse_plan(
+            &serde_json::json!({"schema_version": 1, "steps": [value]}).to_string()
+        )
+        .is_err());
+    }
+    assert!(parse_plan(&serde_json::json!({"schema_version":1,"steps":[{"type":"patch_evidence","change_iri":change_iri,"limits":{"unknown":1}}]}).to_string()).is_err());
+}
diff --git a/atomic-repository/tests/workflow_usage.rs b/atomic-repository/tests/workflow_usage.rs
new file mode 100644
index 00000000..0cea3307
--- /dev/null
+++ b/atomic-repository/tests/workflow_usage.rs
@@ -0,0 +1,118 @@
+use atomic_core::{
+    change::{AITool, AIVendor, Change, ChangeHeader, Provenance, ProvenanceGraph, TokenUsage},
+    kg::{iri, ontology::Predicate as P},
+    pristine::{GraphTxnT, KgMutTxnT, KgTxnT, MutTxnT, VaultEntryType, ViewTxnT},
+};
+use atomic_repository::{intent_graph::IntentGraph, Repository};
+
+#[test]
+fn legacy_projected_changes_gain_usage_in_read_models_without_rebuilding_or_writing() {
+    let directory = tempfile::tempdir().unwrap();
+    let root = directory.path();
+    let repo = Repository::init(root).unwrap();
+    repo.init_vault().unwrap();
+    let uid = "01ARZ3NDEKTSV4RRFFQ69G5FAV";
+    repo.vault_store(&format!("intents/{uid}/intent.md"), VaultEntryType::Intent,
+        b":::why\nInspect usage.\n:::\n".to_vec(),
+        serde_json::json!({"uid":uid,"id":"TEST::tester::1","title":"Inspect usage","kind":"feature","status":"in_progress"}).to_string()).unwrap();
+    repo.vault_reconcile_intent_manifest().unwrap();
+    let mut change = Change::empty(ChangeHeader::new("Real usage record"));
+    let mut provenance = Provenance::builder()
+        .vendor(AIVendor::Other("openrouter".into()))
+        .model("openai/gpt-6.1-sol")
+        .tool(AITool::cli("opencode"))
+        .request_id("usage-regression")
+        .cost_usd(0.086176)
+        .build();
+    provenance.tokens = TokenUsage::full(54, 2177, 105, 500, 7);
+    change.add_provenance(provenance);
+    let mut bytes = vec![];
+    let hash = change.serialize(&mut bytes).unwrap();
+    repo.import_change_bytes(&hash, &bytes).unwrap();
+    repo.save_provenance_graph(
+        &ProvenanceGraph::builder("usage-session", "opencode")
+            .plan_id(uid)
+            .add_change_explained(hash)
+            .build(),
+    )
+    .unwrap();
+    repo.rebuild_kg(true).unwrap();
+    let id = iri::mint_change(&hash);
+    let intent = iri::mint_intent(&repo.project_id().unwrap(), uid).unwrap();
+    let view = iri::mint_view(&repo.project_id().unwrap(), repo.current_view());
+    // Simulate an older KG, with canonical immutable provenance still present.
+    {
+        let mut txn = repo.pristine().write_txn().unwrap();
+        let mut node = txn.get_kg_node(&id).unwrap().unwrap();
+        for key in [
+            "ai_usage_records",
+            "ai_usage_record_version",
+            "ai_usage_state",
+            "ai_usage",
+        ] {
+            node.metadata
+                .as_mut()
+                .unwrap()
+                .as_object_mut()
+                .unwrap()
+                .remove(key);
+        }
+        txn.upsert_kg_node(&node).unwrap();
+        // Authoritative membership changes before its derived KG edges catch
+        // up, as in the real demo. Usage must follow the canonical view filter.
+        let mut current = txn.get_view(repo.current_view()).unwrap().unwrap();
+        let change_id = txn.get_internal(&hash).unwrap().unwrap();
+        txn.put_change(&mut current, change_id, &hash).unwrap();
+        txn.update_view(¤t).unwrap();
+        txn.commit().unwrap();
+    }
+    let database = root.join(".atomic/atomic.redb");
+    let before = std::fs::read(&database).unwrap();
+    let snapshot = repo.workflow_snapshot().unwrap();
+    assert!(
+        !snapshot
+            .edges
+            .iter()
+            .any(|edge| edge.from_id == id && edge.kind == P::DirectlyOnView),
+        "usage hydration must not fabricate missing KG membership edges"
+    );
+    for subject in [&id, &intent, &view] {
+        let node = snapshot
+            .nodes
+            .iter()
+            .find(|node| &node.id == subject)
+            .unwrap_or_else(|| panic!("missing usage subject {subject}"));
+        let usage = &node.metadata.as_ref().unwrap()["ai_usage"];
+        assert_eq!(usage["cost"]["amount_micros"], 86_176);
+        assert_eq!(usage["tokens"]["input"], 54);
+        assert_eq!(usage["tokens"]["output"], 2177);
+        assert_eq!(usage["tokens"]["reasoning"], 105);
+        assert_eq!(usage["tokens"]["total"], 2336);
+        assert_eq!(usage["tokens"]["cache_read"], 500);
+        assert_eq!(usage["tokens"]["cache_write"], 7);
+        assert_eq!(usage["contributors"][0]["vendor"], "openrouter");
+    }
+    let capped = IntentGraph::from_snapshot(&snapshot, &intent, 1).unwrap();
+    assert!(capped.truncated);
+    assert_eq!(
+        capped.nodes[0].metadata.as_ref().unwrap()["ai_usage"]["cost"]["amount"],
+        "0.086176"
+    );
+    let mut selected = vec![repo.vault_kg_node(&view).unwrap().unwrap()];
+    repo.enrich_workflow_usage(&mut selected).unwrap();
+    assert_eq!(
+        selected[0].metadata.as_ref().unwrap()["ai_usage"]["coverage"]["changes"],
+        1
+    );
+    assert_eq!(std::fs::read(database).unwrap(), before);
+    assert!(
+        repo.vault_kg_node(&id)
+            .unwrap()
+            .unwrap()
+            .metadata
+            .unwrap()
+            .get("ai_usage_records")
+            .is_none(),
+        "reading must not backfill stored rows"
+    );
+}
diff --git a/atomic-shacl/src/adapter.rs b/atomic-shacl/src/adapter.rs
index 4b30dd83..344f2d52 100644
--- a/atomic-shacl/src/adapter.rs
+++ b/atomic-shacl/src/adapter.rs
@@ -66,6 +66,16 @@ pub fn snapshot_ntriples(nodes: &[KgNode], edges: &[KgEdge]) -> Result"));
+        assert!(rdf.contains("workflow#projectionCompleteness>"));
+        let result =
+            validate_ntriples(&rdf, "availability", &ValidationRequest::default()).unwrap();
+        assert_eq!(
+            result.conforms, conforms,
+            "{availability}/{completeness}: {:?}",
+            result.results
+        );
+    }
+}
diff --git a/docs/intent-graph.md b/docs/intent-graph.md
index 4f9b4ff5..32ccb2b6 100644
--- a/docs/intent-graph.md
+++ b/docs/intent-graph.md
@@ -1,212 +1,218 @@
-# Intent Knowledge Graph
-
-Intents are authored as directive markdown, lifted into a typed canonical node,
-and **projected into the vault knowledge graph** so their structure — tasks,
-acceptance criteria, scope, constraints, and the links between them — is
-queryable alongside changes, files, and sessions.
-
-## From directives to a graph
-
-```
-intent.md (directives)
-        │  lift  (atomic-canonical)
-        ▼
-CanonicalNode  (typed: acceptance criteria, tasks, scope, constraints, refs)
-        ├── gate    (SHACL-style shapes: conform before attest / done / trust)
-        └── project (atomic-repository::vault_extract_kg)
-                 ▼
-         Knowledge graph nodes + edges  (redb KG tables)
+# Intent semantic change graphs
+
+```sh
+atomic intent graph ATOM::continuouslee::191
+atomic intent graph ATOM::continuouslee::191 --json
+atomic intent graph ATOM::continuouslee::191 --dot -o intent.dot
+atomic intent graph ATOM::continuouslee::191 --html
+atomic intent graph ATOM::continuouslee::191 --html -o intent.html
+atomic intent graph ATOM::continuouslee::191 --html --no-open -o intent.html
 ```
 
-The **gate** enforces a common, conformant shape (see
-[Conformance](#conformance-the-shacl-style-gate) below); the **projection**
-writes the structure into the queryable graph.
-
-The projection runs automatically whenever an intent is written or synced
-(`vault_store` → `vault_index_kg`). A body that does not lift (legacy prose, a
-missing id, malformed directives) is skipped gracefully — it never fails
-indexing.
-
-## What the graph contains
-
-For an intent `intent:`, the projection emits a node per child and typed
-edges linking them:
-
-```
-                    ┌───────────────────────────┐
-                    │   intent:           │
-                    └───────────────────────────┘
-        HAS_ACCEPTANCE_CRITERION │   │ HAS_TASK
-             ┌───────────────────┘   └───────────────────┐
-             ▼                                            ▼
-   ┌────────────────────┐                     ┌────────────────────┐
-   │ ac:-ac-1    │◀────  SATISFIES  ────│ task:-1     │
-   │ status, verified  │                     │ status            │
-   └────────────────────┘                     └─────────┬──────────┘
-                                                         │ TOUCHES
-                                                         ▼
-                                              ┌────────────────────┐
-                                              │ file:src/index.ts │  (shared)
-                                              └────────────────────┘
+Human keys, intent UIDs, and canonical project-qualified intent IRIs are accepted.
+Text is the default. `--html` saves the graph and opens it in the default browser:
+without `--output`/`-o`, it uses a unique temporary `.html` file retained after the
+command exits; with `-o`, it opens that output file. If browser launch fails, the
+saved graph remains available and a warning gives its file URL.
+
+Use `--html --no-open` for scripts and headless environments: HTML writes to stdout
+unless `-o` is supplied, in which case it only writes the file. HTML uses the
+existing D3 renderer and its CDN dependency.
+
+### Inspecting nodes
+
+Known semantic node types have distinct, stable colors in both the graph and its
+legend; newly encountered external types use a deterministic color derived from
+their full type IRI. Colors stay consistent between exports, and text labels
+remain available so identification does not depend on color alone.
+
+Click a node to pin its details in the right-hand pane. The selected node and
+its adjacent edges are highlighted; hovering still shows the lightweight tooltip.
+The pane displays the full label, canonical ID, all types, source, untruncated
+summary, structured metadata, incoming/outgoing relationships and raw exported
+node data. Click a related node in the pane to navigate to it.
+
+The node picker also supports keyboard selection. Use **Clear**, **Escape**, or
+click empty graph space to reset selection. Dragging a node does not change the
+selection. On narrow screens the inspector moves below the graph.
+
+Inspector data is embedded in the HTML and needs no running Atomic service.
+It reflects only the exported snapshot: file contents and historical patch bodies
+are not fetched. The shared query-graph HTML renderer includes the same inspector.
+
+### AI usage, costs and attribution
+
+Selecting a **change, intent or view** shows a dedicated AI usage section with
+input, output, reasoning, cache-read, cache-write and recorded-total tokens,
+precise USD costs, and coverage. Changes and intents also show their contributing
+models, tools and vendors; mixed attribution is listed separately rather than
+reduced to a single model. The same summary is exported as `metadata.ai_usage` in
+graph JSON.
+The summaries are also included by `atomic query neighbors 
+--json`, so the same usage data can be queried without opening HTML.
+
+- **Change:** its immutable recorded AI provenance.
+- **Intent:** the union of explicitly implementing changes and changes recording
+  its own intent artifact, with each change counted once. Dependency-only history
+  is excluded. Recording evidence remains distinct from implementation attribution.
+- **View:** authoritative repository membership, including ancestor-view membership
+  and validated transitive change dependencies, deduplicated by change identity.
+  This does not depend on derived KG membership edges being up to date. An
+  incomplete/unresolvable view filter is reported as incomplete, not a complete
+  zero-cost view.
+
+Summaries use the complete repository snapshot **before** display limits/filtering;
+a view's total does not shrink just because only one of its changes is visible in
+an intent graph. Do not add overlapping intent/view totals to calculate a repository
+total. Shared, identifiable provider requests or session-turn usage records are
+counted once; conflicting copies are excluded and reported. Unavailable changes and
+changes without AI provenance are counted in coverage, not assumed to be free.
+
+These are recorded AI costs, not development-cost estimates. Token categories may
+overlap according to the provider, so the recorded total is preserved rather than
+reconstructed by adding the rows. Older captures may store zero for fields that
+were not reported; the inspector discloses that limitation. All-zero token records
+and unmarked zero costs are treated as unreported, rather than assumed free.
+Prompts and reasoning
+text are not included in usage metadata.
+
+Older KG projections can read usage from verified immutable change headers without
+writing a backfill or rebuilding the repository. Newly projected changes store the
+normalized usage facts directly. Regenerate HTML exports to include these fields;
+already saved HTML files do not update themselves.
+
+Browser interaction regressions can be run explicitly with Chrome/Chromium:
+
+```sh
+ATOMIC_TEST_CHROME=/path/to/chrome \
+  cargo test -p atomic-cli --test intent_graph_test -- --ignored
 ```
 
-| Node kind | Example id | Source |
-|-----------|-----------|--------|
-| `intent` | `intent:` | the intent |
-| `acceptance_criterion` | `ac:-ac-1` | `:::acceptance-criterion` |
-| `task` | `task:-1` | `:::task` |
-| `scope` | `scope:-scope-in-1` | `:::scope-in` / `:::scope-out` |
-| `constraint` | `constraint:-constraint-1` | `:::constraint` |
-| `file` | `file:src/index.ts` | `::file-ref` (shared with changes) |
-
-| Edge | From → To | Meaning |
-|------|-----------|---------|
-| `HAS_ACCEPTANCE_CRITERION` | intent → ac | the intent declares this criterion |
-| `HAS_TASK` | intent → task | the intent decomposes into this task |
-| `HAS_SCOPE_IN` / `HAS_SCOPE_OUT` | intent → scope | scope boundaries |
-| `HAS_CONSTRAINT` | intent → constraint | a rule to respect |
-| `SATISFIES` | task → ac | the task fulfills the criterion |
-| `TOUCHES` | task → file | the task changes this file |
-
-A task's `TOUCHES` edge points at the **same** `file:` node that changes
-reference via `MODIFIES`, so "which intents touch this file" comes for free.
-
-Because a task may satisfy more than one criterion, `criteria=` (and the
-`satisfies=` alias) accept a comma-separated list — each entry becomes its own
-`SATISFIES` edge.
-
-## Lifecycle
-
-The projection stays correct across edits and deletes. When an intent is
-re-indexed or removed, its previously projected child nodes (and their
-`SATISFIES`/`TOUCHES` edges) are garbage-collected first, so a task removed from
-the body does not orphan its node. Shared `file:` nodes are never deleted.
-
-## Conformance: the SHACL-style gate
-
-Before an intent is trusted (attested, advanced to `done`, or relied on for
-queries), it is validated against a **SHACL-style policy engine** — the *gate*.
-This is what guarantees every intent shares a common, well-formed shape, so the
-projected graph is consistent and the queries above are reliable.
-
-M0 hand-codes the shapes in Rust (`atomic-canonical/src/gate.rs`); a full SHACL
-evaluator over Turtle shapes is a later milestone (see
-[shacl-oxirs-spike-findings.md](./shacl-oxirs-spike-findings.md)). The semantics
-are fixed regardless of implementation.
-
-### Shapes
-
-| Shape | Applies to | Enforces |
-|-------|-----------|----------|
-| `IntentShape` | the intent | closed `status`, author + proof present, a `why` exists, scope-out present when scope-in is |
-| `AcceptanceCriterionShape` | each `:::acceptance-criterion` | closed `acStatus`; a `met` criterion must carry `verifiedBy` + `evidence` |
-| `TaskShape` | each `:::task` | every `satisfies` target is a criterion the intent actually declares |
-| `MemoryShape` | memory nodes | closed `memoryKind` + `status`, author + proof present, non-empty text |
-
-### Governing principles
-
-- **Presence is enforced; content is left honest.** The gate requires that a
-  reason (`why`) *exists*, that authorship (`attributedTo`, a DID) is present,
-  and that a Data Integrity `proof` is present — it never grades the prose.
-- **Closed world.** Status and kind values come from fixed sets
-  (`backlog`/`todo`/`in_progress`/`needs-review`/`done`/`icebox`; `unmet`/`met`;
-  memory kinds/statuses). Anything outside the set is rejected.
-- **`status: done` is granted, not written.** Advancing an intent to `done`
-  must pass the gate first.
-- **A checked box needs proof.** `acStatus = met` without `verifiedBy` +
-  `evidence` fails.
-- **Referential integrity.** A task's `satisfies`/`criteria` must point at a
-  criterion the intent declares; a dangling or mistyped reference is a
-  violation, not a silent broken edge.
-- **Never auto-fixes.** The gate only *reports* — it returns a
-  `ValidationReport` of `Violation`s (each with the focus node, the shape, the
-  property path, and a message), leaving the load-bearing facts to the author.
-
-### Running it
-
-```bash
-# Validate a stored intent (by reference) or a markdown file directly.
-atomic intent validate 3
-atomic intent validate PIMO::lee-faus::3 --json
-atomic intent validate path/to/intent.md
+Set `ATOMIC_TEST_D3=/path/to/d3.min.js` to use a local D3 v7 bundle for the canvas
+test instead of its CDN URL. Normal tests do not launch a browser.
+
+The graph selects **asserted semantic lineage**, not a search neighborhood:
+
+- the intent, its tasks, acceptance criteria, scope, constraints, and references;
+- turns governed by the intent and directly implementing changes;
+- changes recording the selected intent's exact vault artifact, separately
+  classified as recording evidence rather than implementation attribution;
+- transitive **direct change-dependency** closure, separately classified from implementation;
+- recorded affected files, authors, direct view memberships and parent-view links;
+- generating turns/sessions, provenance, covering attestations, signer and verification links;
+- related review/memory/goal references and gap findings.
+
+Shared sessions, files, authors and attestations do not pull unrelated changes into
+the graph. View ancestry does not invent inherited direct membership. Related
+intents are boundary nodes: their implementations are not silently counted as this
+intent's implementation.
+
+JSON includes full canonical IRIs, asserted types/predicates, snapshot identity,
+ontology version, implementing/dependency-only change sets, missing resources and
+diagnostics. `--max-nodes` defaults to 5000, with a hard maximum of 10000. A cap
+always reports `truncated: true`; it never claims complete dependency closure.
+No linked changes means “none linked in this snapshot,” not “no work happened.”
+
+## Recorded change evidence without an implementation assertion
+
+A full change record may already be in the KG even when there is no explicit
+`implements` or governing-plan relationship. The default graph now also joins
+the intent's **stored `vault_path` metadata** to its exact canonical repository
+File node (`.vault/`) and follows existing incoming `modifiesFile`
+edges. It shows those changes, their recorded affected files, dependencies and
+available generating-turn/provenance relationships. It does not read working
+files, rebuild the KG, or require an agent session binding to expose this evidence.
+
+JSON reports the matched `intent_artifact` and its `recording_changes` separately
+from `implementing_changes`, `captured_changes` and `declared_changes`. Text, DOT
+and HTML label them “records intent artifact; implementation attribution not
+asserted.” A change can belong to both the recording and implementation sets
+when both kinds of evidence exist. Dependency-only changes exclude either kind
+of primary selected change.
+
+This is a join over existing snapshot facts, **not a new semantic edge or a
+retrospective declaration**. Recording an intent can be implementation work,
+documentation, or administration; it does not by itself prove implementation or
+governing-turn provenance. Shared task/source files, authors, sessions and
+attestations never seed additional changes. Historical renames whose paths are
+not represented by the current snapshot are not reconstructed.
+
+When recording evidence exists but implementation attribution does not, the
+diagnostic says so explicitly instead of presenting only an empty-implementation
+warning. An absent `plan_id` remains visible as a provenance gap; it does not
+hide the full recorded changes that the snapshot actually contains.
+
+This command reads through the shared local/reactor services and **does not rebuild**
+the KG. If an intent is not projected, use `atomic query enrich` explicitly. The
+scope is repository-wide and is not available as a remote sandbox query.
+
+“Full” means the predicate-scoped lineage present in the asserted snapshot. It
+does not mean inferred source behavior, token-level code graphs, a complete
+historical diff, or proof that all historical work has been ingested. For exact
+recorded operations use the `patch_evidence` query-plan step. A change's
+availability/projection fields remain separate from provenance/attestation coverage.
+
+## Capture governance when work starts
+
+Recording an intent file alongside code does **not** bind that intent to the
+generating turn. Declare the association while the hook turn is active:
+
+```sh
+atomic intent update ATOM::continuouslee::194 --status in_progress \
+  --session-id  --turn <1-indexed-hook-turn>
 ```
 
-`atomic intent attest` runs the same gate before signing, so an intent cannot
-be attested (or its criteria marked `met`) unless it conforms. A conforming
-report reads `conforms: yes`; otherwise each violation is listed:
+The CLI resolves the human key to the stored intent UID and persists a
+turn-scoped declaration before the status update returns. At checkpoint time,
+that UID becomes `plan_id` in both the provenance graph and `SessionTurn`, so the
+semantic graph can project the captured association. This works for ordinary,
+unmanaged sessions. Managed runs without an explicit declaration retain their
+existing work-item capture.
+
+Integrations can inject `ATOMIC_SESSION_ID` into each session's shell environment
+instead of passing `--session-id`. `--turn` is optional but recommended to reject
+delayed commands. The integration must complete `session-start` and `user-prompt`
+hooks **before** executing the work-start command, and must not use a process-wide
+session variable shared by concurrent sessions. Without a flag or environment
+context, an update remains a human/status-only operation; Atomic never chooses a
+session by recency or by scanning active sessions. The separately installed
+`atomic-opencode` integration needs to supply this context; changing the CLI does
+not update installed plugin code.
+
+Declarations are stored at
+`.atomic/sessions//intent-.json`, independently of mutable
+session JSON. They share the recorder's `turn-end.lock`, survive hook process
+restarts, are idempotent for the same UID, and reject replacement, stale/completed
+turns, recording contention, and conflicting managed work items. Governance is
+frozen before the first recording attempt, even if that attempt fails before
+checkpoint publication; retry cannot introduce a late declaration. They do not
+carry into the next turn. Re-declare governance explicitly on later turns that
+continue the same intent. This is a prospective work-start declaration, not a
+repair tool for an already recorded turn.
+
+## Repairing an implementation association without inventing provenance
+
+A failed recording hook can leave implementation in a mixed parent change with no
+captured intent ID. Do not change that historical provenance or guess a generating
+turn from shared files. An author can instead make an explicit declaration in the
+intent body, citing the **full canonical change IRI** and nonempty evidence:
 
 ```text
-conforms: no (1 violation(s))
-  ✗ [TaskShape] urn:atomic:task:01j8ze…-1 (satisfies): task satisfies
-    'urn:atomic:ac:01j8ze…-ac-9' which is not an acceptance criterion on this intent
-```
-
-## Querying it
-
-All of this is reachable through `atomic vault query`. Get an intent's ULID from
-`atomic intent show` or a search, then traverse.
-
-### Search
-
-```bash
-atomic vault query search "authentication"
-atomic vault query search "readline" -k 20 --json
-```
-
-### Neighbors (traverse the structure)
-
-```bash
-# The intent and everything it declares (tasks, criteria, scope, constraints).
-atomic vault query neighbors intent:01J8ZE7G2WABCDEFGHJKMNPQRS -d 1
-
-# Depth 2 reaches the files tasks touch and the criteria they satisfy.
-atomic vault query neighbors intent:01J8ZE7G2WABCDEFGHJKMNPQRS -d 2 --json
-
-# Which intents/tasks touch a given file (reverse traversal via the shared node).
-atomic vault query neighbors file:src/index.ts -d 1
-```
-
-### Natural-language (RAG)
-
-```bash
-# Uses the graph as context; requires ANTHROPIC_API_KEY or OPENAI_API_KEY.
-atomic vault query ask "which tasks still need to satisfy the strict-build criterion?"
-atomic vault query ask "what files does intent PIMO::lee-faus::3 touch?"
-```
-
-### Structured query plans
-
-```bash
-echo '{"steps":[
-  {"type":"kg_search","query":"interactive cli","bind":"hits"},
-  {"type":"kg_neighbors","node_id":"$hits","depth":2,"bind":"ctx"}
-]}' | atomic vault query plan --json
-```
-
-### Reindex
-
-The KG is maintained on every vault write. To rebuild from scratch:
-
-```bash
-atomic vault query reindex
+::ref{to=urn:atomic:change:blake3-256: edge=declaresImplementation declarationEvidence="Explain the explicit attribution, its evidence, and the historical gap."}
 ```
 
-## Relevant code
-
-- `atomic-canonical/src/{directive,lift,node}.rs` — parse, lift, and the typed
-  canonical node.
-- `atomic-canonical/src/gate.rs` — the SHACL-style gate: `validate_intent`,
-  `validate_memory`, `ValidationReport`, `Violation`, and the shape rules.
-- `atomic-canonical/src/vocab.rs` — the closed value sets (`INTENT_STATUS`,
-  `AC_STATUS`, `MEMORY_KIND`, `MEMORY_STATUS`, `DEPENDENCY_EDGES`).
-- `atomic-cli/src/commands/intent/{validate,attest}.rs` — CLI entry points
-  that run the gate.
-- `atomic-repository/src/repository/vault_triples.rs` — `project_intent_semantics`
-  (nodes + edges), `delete_intent_child_nodes` (lifecycle GC), `vault_extract_kg`.
-- `atomic-core/src/pristine/ontology.rs` — `HAS_TASK`, `HAS_ACCEPTANCE_CRITERION`,
-  `HAS_SCOPE_IN`/`OUT`, `HAS_CONSTRAINT`, `SATISFIES`, `TOUCHES`, and the
-  `TASK`/`ACCEPTANCE_CRITERION`/`SCOPE_ITEM`/`CONSTRAINT` entity types.
-- `atomic-cli/src/commands/query/mod.rs` — the `atomic vault query` subcommands.
-
-See also [Intent Identity](./intent-identity.md) for how `intent:` node
-ids and the `PROJECT::author::seq` references are formed.
+Then run `atomic vault sync` and `atomic intent attest `; verify the
+fresh attestation with `atomic intent verify `. This is a new,
+retrospective author claim, not a claim captured at the original turn. The
+declaration and evidence participate in the canonical signature. Unsigned
+authoring data can still be projected; verify the signed intent before trusting it.
+
+The graph emits `atomic:declaresImplementation` and lists the change under
+`declared_changes`. `implementing_changes` is the union of declared and captured
+implementation; `captured_changes` preserves the captured subset. Declarations
+never synthesize `atomic:implements`, `hasPlan`, or `generatedChange` edges, satisfy
+missing-provenance publication gates, or turn into dependency edges. Missing change
+objects produce unresolved-reference findings rather than fabricated targets. Old
+signed snapshots do not resurrect declarations removed from the current source.
+
+Automatic hook retry/intent capture is separate from this explicit repair path.
diff --git a/libatomic/src/daemon/services_agent.rs b/libatomic/src/daemon/services_agent.rs
index 9e5bb727..4a7b357f 100644
--- a/libatomic/src/daemon/services_agent.rs
+++ b/libatomic/src/daemon/services_agent.rs
@@ -3493,7 +3493,7 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl {
                 })
             }
             QueryOneof::Neighbors(neighbors) => {
-                let repo = handle.repository()?;
+                let repo = handle.repository_readonly()?;
                 let (nodes, edges) = crate::daemon::services_query::neighbors_impl(
                     &repo,
                     &neighbors.node_id,
@@ -3717,6 +3717,8 @@ impl knowledge_service_server::KnowledgeService for KnowledgeImpl {
                 let _ = &collected;
                 collected = nodes.len() as u32;
                 let _ = &collected;
+                repo.enrich_workflow_usage(&mut nodes)
+                    .map_err(repository_error)?;
                 Ok(QueryGraphResponse {
                     api_version: 1,
                     callers: Vec::new(),
diff --git a/libatomic/src/daemon/services_query.rs b/libatomic/src/daemon/services_query.rs
index 278b8fe1..ebb29094 100644
--- a/libatomic/src/daemon/services_query.rs
+++ b/libatomic/src/daemon/services_query.rs
@@ -1150,9 +1150,11 @@ pub fn neighbors_impl(
     node_id: &str,
     depth: u8,
 ) -> Result<(Vec, Vec), Status> {
-    let subgraph = repo
+    let mut subgraph = repo
         .vault_kg_neighbors(node_id, depth)
         .map_err(repository_error)?;
+    repo.enrich_workflow_usage(&mut subgraph.nodes)
+        .map_err(repository_error)?;
     let nodes = subgraph.nodes.into_iter().map(kg_node_proto).collect();
     let edges = subgraph.edges.into_iter().map(kg_edge_proto).collect();
     Ok((nodes, edges))
diff --git a/rust-toolchain.toml b/rust-toolchain.toml
new file mode 100644
index 00000000..a1598ccb
--- /dev/null
+++ b/rust-toolchain.toml
@@ -0,0 +1,4 @@
+[toolchain]
+channel = "1.98.0"
+profile = "minimal"
+components = ["rustfmt", "clippy"]

From 6847db73d0a12b134a9896b8edd5a00ce4bc5e70 Mon Sep 17 00:00:00 2001
From: Lee Faus 
Date: Fri, 23 Oct 2026 16:07:21 -0400
Subject: [PATCH 16/19] cleanup work

---
 atomic-cli/src/commands/query/graph_inspector.js | 1 +
 atomic-cli/tests/fixtures/graph_canvas_test.js   | 2 ++
 docs/intent-graph.md                             | 5 +++--
 3 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/atomic-cli/src/commands/query/graph_inspector.js b/atomic-cli/src/commands/query/graph_inspector.js
index 5193b989..623a094f 100644
--- a/atomic-cli/src/commands/query/graph_inspector.js
+++ b/atomic-cli/src/commands/query/graph_inspector.js
@@ -145,6 +145,7 @@ function createGraphInspector(container, graphData, onSelect) {
     clear.disabled = !node;
     container.dataset.selectedNode = node ? node.id : '';
     content.replaceChildren();
+    container.scrollTop = 0;
     if (!node) {
       status.textContent = 'No node selected.';
       content.appendChild(element('p', 'Click a node to pin its details here. Follow a relationship to inspect its other endpoint. Escape or Clear resets selection.', 'inspector-muted'));
diff --git a/atomic-cli/tests/fixtures/graph_canvas_test.js b/atomic-cli/tests/fixtures/graph_canvas_test.js
index 8fb317bf..bc4decd0 100644
--- a/atomic-cli/tests/fixtures/graph_canvas_test.js
+++ b/atomic-cli/tests/fixtures/graph_canvas_test.js
@@ -24,8 +24,10 @@ setTimeout(function() {
     ['https://atomic.dev/ns/workflow#Change', 'https://atomic.dev/ns/workflow#FeatureIntent'].forEach(function(kind) {
       var usageNode = graphData.nodes.find(function(node) { return node.kind === kind; });
       assert(usageNode, 'usage fixture node must be exported');
+      pane.scrollTop = 100;
       picker.value = usageNode.id;
       picker.dispatchEvent(new Event('change'));
+      assert(pane.scrollTop === 0, 'new selection must reveal usage details at the top of the pane');
       var usagePane = pane.querySelector('.inspector-ai-usage');
       assert(usagePane && usagePane.textContent.includes('$0.125000 USD'), 'change/intent must show recorded cost');
       assert(usagePane.textContent.includes('Cache read') && usagePane.textContent.includes('Recorded total'), 'token categories must be shown');
diff --git a/docs/intent-graph.md b/docs/intent-graph.md
index 32ccb2b6..abc02da9 100644
--- a/docs/intent-graph.md
+++ b/docs/intent-graph.md
@@ -48,8 +48,9 @@ precise USD costs, and coverage. Changes and intents also show their contributin
 models, tools and vendors; mixed attribution is listed separately rather than
 reduced to a single model. The same summary is exported as `metadata.ai_usage` in
 graph JSON.
-The summaries are also included by `atomic query neighbors 
---json`, so the same usage data can be queried without opening HTML.
+The summaries are also included by
+`atomic query neighbors  --json`, so the same usage data can be
+queried without opening HTML.
 
 - **Change:** its immutable recorded AI provenance.
 - **Intent:** the union of explicitly implementing changes and changes recording

From a88906c394c238cb3b793708abc775e43100ba23 Mon Sep 17 00:00:00 2001
From: Lee Faus 
Date: Sun, 25 Oct 2026 03:20:51 +0100
Subject: [PATCH 17/19] change to a tree view for change, intent show/graph

---
 atomic-canonical/src/render.rs                | 469 ++++++++++----
 atomic-canonical/tests/roundtrip.rs           |   2 +-
 atomic-cli/src/commands/change/command.rs     | 345 +---------
 atomic-cli/src/commands/change/mod.rs         |  48 +-
 atomic-cli/src/commands/change/tree.rs        | 601 ++++++++++++++++++
 atomic-cli/src/commands/change/types.rs       |   2 +-
 atomic-cli/src/commands/intent/graph.rs       | 515 ++++++++++++++-
 atomic-cli/src/commands/intent/mod.rs         |   4 +-
 atomic-cli/src/commands/intent/show.rs        |  21 +-
 atomic-cli/tests/change_tree_test.rs          | 111 ++++
 .../tests/clone_sidecar_integration_test.rs   |   2 +-
 atomic-cli/tests/intent_graph_test.rs         |  61 +-
 atomic-cli/tests/routing_parity_test.rs       |  21 +
 atomic-repository/src/workflow_usage.rs       |   6 +-
 14 files changed, 1666 insertions(+), 542 deletions(-)
 create mode 100644 atomic-cli/src/commands/change/tree.rs
 create mode 100644 atomic-cli/tests/change_tree_test.rs

diff --git a/atomic-canonical/src/render.rs b/atomic-canonical/src/render.rs
index b178f596..476c8f40 100644
--- a/atomic-canonical/src/render.rs
+++ b/atomic-canonical/src/render.rs
@@ -19,6 +19,8 @@ pub enum Target {
 }
 
 /// Project a canonical Intent node to a rendered string for the given target.
+/// The CLI view is one complete tree, including classification and review
+/// targets, so callers do not need to append their own metadata sections.
 pub fn render(node: &CanonicalNode, target: Target) -> String {
     match target {
         Target::Cli => render_cli(node),
@@ -38,141 +40,232 @@ pub fn render_memory(node: &MemoryNode, target: Target) -> String {
 }
 
 fn render_cli(node: &CanonicalNode) -> String {
-    let mut s = String::new();
-
-    // Header: human key + title.
-    s.push_str(&format!("{}  {}\n", node.human_key, node.title));
-    s.push_str(&"─".repeat(48));
-    s.push('\n');
-
-    // Spine fields — status is REGENERATED here from the spine, not authored.
-    s.push_str(&format!("Status:    {}\n", node.status));
-    if let Some(p) = &node.priority {
-        s.push_str(&format!("Priority:  {p}\n"));
-    }
-    if let Some(v) = &node.view {
-        s.push_str(&format!("View:      {v}\n"));
+    let mut root = Tree::new(format!("Intent {} — {}", node.human_key, node.title));
+    // Status is regenerated from the canonical spine, not authored prose.
+    let mut metadata = Tree::new("Metadata");
+    metadata.push(format!("Status: {}", node.status));
+    metadata.push(format!("Kind: {}", node.kind));
+    if let Some(priority) = &node.priority {
+        metadata.push(format!("Priority: {priority}"));
     }
-    if let Some(m) = &node.motivated_by {
-        s.push_str(&format!("Motivated: {m}\n"));
+    if let Some(view) = &node.view {
+        metadata.push(format!("View: {view}"));
     }
-    if !node.informed_by.is_empty() {
-        s.push_str(&format!("InformedBy: {}\n", node.informed_by.join(", ")));
+    if let Some(motivation) = &node.motivated_by {
+        metadata.push(format!("Motivated by: {motivation}"));
     }
+    metadata.group(
+        "Informed by",
+        node.informed_by.iter().map(Tree::new).collect(),
+    );
+    root.children.push(metadata);
 
-    // Why (unconstrained prose).
     if let Some(why) = &node.why {
-        s.push_str("\nWhy\n");
-        s.push_str(why);
-        s.push('\n');
+        root.group(
+            "Why",
+            why.lines()
+                .filter(|line| !line.trim().is_empty())
+                .map(Tree::new)
+                .collect(),
+        );
     }
-
-    // Acceptance criteria.
-    if !node.has_acceptance_criterion.is_empty() {
-        s.push_str("\nAcceptance Criteria\n");
-        for ac in &node.has_acceptance_criterion {
-            let mark = if ac.ac_status == "met" { "x" } else { " " };
-            s.push_str(&format!("  [{mark}] {}\n", ac.text));
-            if ac.ac_status == "met" {
-                if let (Some(by), Some(ev)) = (&ac.verified_by, &ac.evidence) {
-                    s.push_str(&format!("        verified by {by} · {ev}\n"));
+    root.group(
+        "Acceptance Criteria",
+        node.has_acceptance_criterion
+            .iter()
+            .map(|ac| {
+                let mark = if ac.ac_status == "met" { "x" } else { " " };
+                let mut criterion = Tree::new(format!("[{mark}] {}", ac.text));
+                if let Some(by) = &ac.verified_by {
+                    criterion.push(format!("Verified by: {by}"));
                 }
-            }
-        }
+                if let Some(evidence) = &ac.evidence {
+                    criterion.push(format!("Evidence: {evidence}"));
+                }
+                criterion.group(
+                    "Required checks",
+                    ac.required_kinds.iter().map(Tree::new).collect(),
+                );
+                criterion.group(
+                    "Verifications",
+                    ac.verifications
+                        .iter()
+                        .map(|record| {
+                            let mut verification = Tree::new(format!(
+                                "{}: {} ({})",
+                                record.kind, record.outcome, record.scope
+                            ));
+                            verification
+                                .push(format!("Observed at Merkle: {}", record.observed_at_merkle));
+                            if let Some(reference) = &record.reference {
+                                verification.push(format!("Reference: {reference}"));
+                            }
+                            if let Some(observation) = &record.observation {
+                                verification.push(format!("Observation: {observation}"));
+                            }
+                            verification
+                        })
+                        .collect(),
+                );
+                criterion
+            })
+            .collect(),
+    );
+    root.group(
+        "Tasks",
+        node.has_task
+            .iter()
+            .map(|task| {
+                let mark = if task.task_status == "done" { "x" } else { " " };
+                let mut item = Tree::new(format!("[{mark}] {}", task.text));
+                // Preserve the legacy scalar/list representation; rendering is read-only.
+                item.group(
+                    "Satisfies",
+                    task.satisfies.as_slice().iter().map(Tree::new).collect(),
+                );
+                item.group("Files", task.touches_file.iter().map(Tree::new).collect());
+                item
+            })
+            .collect(),
+    );
+    for (label, scope) in [
+        ("In Scope", &node.has_scope_in),
+        ("Out of Scope", &node.has_scope_out),
+    ] {
+        root.group(
+            label,
+            scope
+                .iter()
+                .map(|scope| {
+                    let mut item = Tree::new(&scope.text);
+                    item.group("Files", scope.files.iter().map(Tree::new).collect());
+                    item
+                })
+                .collect(),
+        );
     }
+    root.group(
+        "Constraints",
+        node.has_constraint
+            .iter()
+            .enumerate()
+            .map(|(i, constraint)| Tree::new(format!("{}. {}", i + 1, constraint.text)))
+            .collect(),
+    );
 
-    // Tasks.
-    if !node.has_task.is_empty() {
-        s.push_str("\nTasks\n");
-        for t in &node.has_task {
-            let mark = if t.task_status == "done" { "x" } else { " " };
-            s.push_str(&format!("  [{mark}] {}\n", t.text));
-            if !t.satisfies.is_empty() {
-                // `as_slice` so a legacy scalar renders identically to a list.
-                s.push_str(&format!(
-                    "        satisfies: {}\n",
-                    t.satisfies.as_slice().join(", ")
+    // These refs may be review, remediation or dependency relations, not all
+    // dependencies. Dedicated groups avoid duplicating review targets.
+    root.group(
+        "Relationships",
+        node.depends_on
+            .iter()
+            .filter(|reference| {
+                !matches!(
+                    reference.edge.as_str(),
+                    "declaresImplementation" | "reviews"
+                )
+            })
+            .map(|reference| Tree::new(format!("{} -> {}", reference.edge, reference.to)))
+            .collect(),
+    );
+    let reviews: Vec<_> = node
+        .depends_on
+        .iter()
+        .filter(|reference| reference.edge == "reviews")
+        .map(|reference| Tree::new(&reference.to))
+        .collect();
+    if node.kind == "review" && reviews.is_empty() {
+        root.push("Reviews: none declared");
+    } else {
+        root.group("Reviews", reviews);
+    }
+    root.group(
+        "Declared implementation (not captured turn provenance)",
+        node.depends_on
+            .iter()
+            .filter(|reference| reference.edge == "declaresImplementation")
+            .map(|reference| {
+                let mut declaration = Tree::new(&reference.to);
+                declaration.push(format!(
+                    "Declaration evidence: {}",
+                    reference
+                        .declaration_evidence
+                        .as_deref()
+                        .unwrap_or("missing evidence")
                 ));
-            }
-            if !t.touches_file.is_empty() {
-                s.push_str(&format!("        touches: {}\n", t.touches_file.join(", ")));
-            }
-        }
+                declaration
+            })
+            .collect(),
+    );
+
+    let mut provenance = Tree::new("Provenance");
+    provenance.push(format!("ID: {}", node.id));
+    if let Some(author) = &node.attributed_to {
+        provenance.push(format!("Author: {author}"));
+    }
+    if let Some(hash) = &node.content_hash {
+        provenance.push(format!("Hash: {hash}"));
     }
+    provenance.push(format!(
+        "Signed: {}",
+        if node.proof.is_some() { "yes" } else { "no" }
+    ));
+    root.children.push(provenance);
+    root.render()
+}
+
+/// A presentation tree, never a new source of canonical relationships.
+struct Tree {
+    label: String,
+    children: Vec,
+}
 
-    // Scope — a pure projection of the scope sub-nodes; no new authoring site.
-    if !node.has_scope_in.is_empty() {
-        s.push_str("\nIn Scope\n");
-        for item in &node.has_scope_in {
-            s.push_str(&format!("  • {}\n", item.text));
+impl Tree {
+    fn new(label: impl Into) -> Self {
+        Self {
+            label: label.into(),
+            children: Vec::new(),
         }
     }
-    if !node.has_scope_out.is_empty() {
-        s.push_str("\nOut of Scope\n");
-        for item in &node.has_scope_out {
-            s.push_str(&format!("  • {}\n", item.text));
-        }
+
+    fn push(&mut self, label: impl Into) {
+        self.children.push(Self::new(label));
     }
 
-    // Constraints — the checklist the doc describes (numbered).
-    if !node.has_constraint.is_empty() {
-        s.push_str("\nConstraints\n");
-        for (i, c) in node.has_constraint.iter().enumerate() {
-            s.push_str(&format!("  {}. {}\n", i + 1, c.text));
+    fn group(&mut self, label: &str, children: Vec) {
+        if !children.is_empty() {
+            self.children.push(Self {
+                label: label.into(),
+                children,
+            });
         }
     }
 
-    // Dependencies — each ref rendered as 'edge -> to'.
-    if node
-        .depends_on
-        .iter()
-        .any(|r| r.edge != "declaresImplementation")
-    {
-        s.push_str("\nDependencies\n");
-        for r in node
-            .depends_on
-            .iter()
-            .filter(|r| r.edge != "declaresImplementation")
-        {
-            s.push_str(&format!("  {} -> {}\n", r.edge, r.to));
+    fn render(&self) -> String {
+        fn clean(label: &str) -> String {
+            label
+                .chars()
+                .map(|c| if c.is_control() { ' ' } else { c })
+                .collect()
         }
-    }
-    if node
-        .depends_on
-        .iter()
-        .any(|r| r.edge == "declaresImplementation")
-    {
-        s.push_str("\nDeclared implementation (not captured turn provenance)\n");
-        for r in node
-            .depends_on
+        let mut out = format!("{}\n", clean(&self.label));
+        let mut pending: Vec<_> = self
+            .children
             .iter()
-            .filter(|r| r.edge == "declaresImplementation")
-        {
-            s.push_str(&format!(
-                "  {}\n    {}\n",
-                r.to,
-                r.declaration_evidence
-                    .as_deref()
-                    .unwrap_or("missing evidence")
-            ));
+            .enumerate()
+            .rev()
+            .map(|(i, child)| (child, String::new(), i + 1 == self.children.len()))
+            .collect();
+        while let Some((node, prefix, last)) = pending.pop() {
+            let branch = if last { "└──" } else { "├──" };
+            out.push_str(&format!("{prefix}{branch} {}\n", clean(&node.label)));
+            let prefix = format!("{prefix}{}", if last { "    " } else { "│   " });
+            for (i, child) in node.children.iter().enumerate().rev() {
+                pending.push((child, prefix.clone(), i + 1 == node.children.len()));
+            }
         }
+        out
     }
-
-    // Provenance footer.
-    s.push('\n');
-    s.push_str(&format!("id:        {}\n", node.id));
-    if let Some(a) = &node.attributed_to {
-        s.push_str(&format!("author:    {a}\n"));
-    }
-    if let Some(h) = &node.content_hash {
-        s.push_str(&format!("hash:      {h}\n"));
-    }
-    s.push_str(&format!(
-        "signed:    {}\n",
-        if node.proof.is_some() { "yes" } else { "no" }
-    ));
-
-    s
 }
 
 fn render_memory_cli(node: &MemoryNode) -> String {
@@ -240,6 +333,158 @@ mod tests {
     use crate::node::{default_kind, Constraint, Ref, ScopeItem, CONTEXT_URL};
     use crate::vocab::NodeType;
 
+    fn tree_fixture() -> CanonicalNode {
+        serde_json::from_value(serde_json::json!({
+            "@context": CONTEXT_URL, "@type": "Intent", "@id": "urn:atomic:intent:tree",
+            "humanKey": "TEST-1", "title": "A complete tree", "status": "in_progress",
+            "kind": "review", "priority": "high", "view": "dev", "createdAt": "2026-10-25T00:00:00Z",
+            "why": "Readable output\nKeep the evidence",
+            "hasAcceptanceCriterion": [{
+                "@type": "AcceptanceCriterion", "@id": "urn:atomic:ac:tree-1", "text": "Tree renders",
+                "acStatus": "met", "verifiedBy": "test-tree", "evidence": "cargo test passed"
+            }],
+            "hasTask": [{
+                "@type": "Task", "@id": "urn:atomic:task:tree-1", "text": "Render the tree",
+                "taskStatus": "done", "satisfies": "urn:atomic:ac:tree-1", "touchesFile": ["src/main.rs", "test/tree.rs"]
+            }],
+            "hasScopeIn": [{"@type": "ScopeItem", "@id": "urn:atomic:scope:tree-in", "text": "Terminal rendering", "files": ["src/main.rs"]}],
+            "hasScopeOut": [{"@type": "ScopeItem", "@id": "urn:atomic:scope:tree-out", "text": "JSON schema"}],
+            "hasConstraint": [{"@type": "Constraint", "@id": "urn:atomic:constraint:tree-1", "text": "Keep signatures intact"}],
+            "dependsOn": [
+                {"edge": "reviews", "to": "urn:atomic:intent:work"},
+                {"edge": "blockedBy", "to": "urn:atomic:intent:blocker"},
+                {"edge": "declaresImplementation", "to": "urn:atomic:change:example", "declarationEvidence": "Explicit author claim"}
+            ]
+        })).unwrap()
+    }
+
+    #[test]
+    fn intent_is_one_complete_tree_including_metadata_reviews_and_provenance() {
+        let text = render(&tree_fixture(), Target::Cli);
+        assert_eq!(
+            text,
+            concat!(
+                "Intent TEST-1 — A complete tree\n",
+                "├── Metadata\n",
+                "│   ├── Status: in_progress\n",
+                "│   ├── Kind: review\n",
+                "│   ├── Priority: high\n",
+                "│   └── View: dev\n",
+                "├── Why\n",
+                "│   ├── Readable output\n",
+                "│   └── Keep the evidence\n",
+                "├── Acceptance Criteria\n",
+                "│   └── [x] Tree renders\n",
+                "│       ├── Verified by: test-tree\n",
+                "│       └── Evidence: cargo test passed\n",
+                "├── Tasks\n",
+                "│   └── [x] Render the tree\n",
+                "│       ├── Satisfies\n",
+                "│       │   └── urn:atomic:ac:tree-1\n",
+                "│       └── Files\n",
+                "│           ├── src/main.rs\n",
+                "│           └── test/tree.rs\n",
+                "├── In Scope\n",
+                "│   └── Terminal rendering\n",
+                "│       └── Files\n",
+                "│           └── src/main.rs\n",
+                "├── Out of Scope\n",
+                "│   └── JSON schema\n",
+                "├── Constraints\n",
+                "│   └── 1. Keep signatures intact\n",
+                "├── Relationships\n",
+                "│   └── blockedBy -> urn:atomic:intent:blocker\n",
+                "├── Reviews\n",
+                "│   └── urn:atomic:intent:work\n",
+                "├── Declared implementation (not captured turn provenance)\n",
+                "│   └── urn:atomic:change:example\n",
+                "│       └── Declaration evidence: Explicit author claim\n",
+                "└── Provenance\n",
+                "    ├── ID: urn:atomic:intent:tree\n",
+                "    └── Signed: no\n",
+            )
+        );
+        assert_eq!(text.matches("urn:atomic:intent:work").count(), 1);
+    }
+
+    #[test]
+    fn tree_preserves_legacy_satisfies_shape_and_canonical_bytes() {
+        let scalar = tree_fixture();
+        let before = scalar.signing_bytes();
+        let text = render(&scalar, Target::Cli);
+        assert_eq!(scalar.signing_bytes(), before);
+        let mut list = scalar.to_value();
+        list["hasTask"][0]["satisfies"] = serde_json::json!(["urn:atomic:ac:tree-1"]);
+        let list: CanonicalNode = serde_json::from_value(list).unwrap();
+        assert_eq!(render(&list, Target::Cli), text);
+        assert!(scalar.to_value()["hasTask"][0]["satisfies"].is_string());
+        assert!(list.to_value()["hasTask"][0]["satisfies"].is_array());
+    }
+
+    #[test]
+    fn empty_optional_sections_are_omitted_but_missing_review_targets_remain_explicit() {
+        let mut node = tree_fixture();
+        node.why = Some(" \n ".into());
+        node.has_task.clear();
+        node.has_acceptance_criterion.clear();
+        node.has_scope_in.clear();
+        node.has_scope_out.clear();
+        node.has_constraint.clear();
+        node.depends_on.clear();
+        let text = render(&node, Target::Cli);
+        for label in [
+            "Why",
+            "Tasks",
+            "Acceptance Criteria",
+            "In Scope",
+            "Out of Scope",
+            "Constraints",
+            "Relationships",
+            "Declared implementation",
+        ] {
+            assert!(!text.contains(label));
+        }
+        assert!(text.contains("├── Reviews: none declared\n└── Provenance\n"));
+        node.kind = "feature".into();
+        assert!(!render(&node, Target::Cli).contains("Reviews"));
+    }
+
+    #[test]
+    fn verification_records_and_proof_state_are_preserved_without_tree_injection() {
+        let mut node = tree_fixture();
+        let criterion = &mut node.has_acceptance_criterion[0];
+        criterion.required_kinds.push("unit".into());
+        criterion
+            .verifications
+            .push(crate::node::VerificationRecord {
+                type_: "VerificationRecord".into(),
+                kind: "unit".into(),
+                outcome: "pass".into(),
+                scope: "task".into(),
+                observed_at_merkle: "state-hash".into(),
+                reference: Some("tree-test".into()),
+                observation: Some("Checked\n\u{1b}[31m".into()),
+            });
+        node.attributed_to = Some("did:atomic:tester".into());
+        node.content_hash = Some("content-hash".into());
+        node.proof = Some(crate::node::Proof {
+            type_: "DataIntegrityProof".into(),
+            cryptosuite: "eddsa-jcs-2022".into(),
+            verification_method: "did:atomic:tester#key".into(),
+            proof_purpose: "assertionMethod".into(),
+            proof_value: "test-proof".into(),
+        });
+        let text = render(&node, Target::Cli);
+        assert!(text.contains("Required checks\n"));
+        assert!(text.contains("unit: pass (task)"));
+        assert!(text.contains("Observed at Merkle: state-hash"));
+        assert!(text.contains("Reference: tree-test"));
+        assert!(text.contains("Observation: Checked  [31m"));
+        assert!(!text.contains('\u{1b}'));
+        assert!(text.contains("Author: did:atomic:tester"));
+        assert!(text.ends_with("    └── Signed: yes\n"));
+    }
+
     fn scope(id: &str, text: &str) -> ScopeItem {
         ScopeItem {
             type_: NodeType::ScopeItem.as_str().to_string(),
diff --git a/atomic-canonical/tests/roundtrip.rs b/atomic-canonical/tests/roundtrip.rs
index e748c14f..920a2302 100644
--- a/atomic-canonical/tests/roundtrip.rs
+++ b/atomic-canonical/tests/roundtrip.rs
@@ -81,7 +81,7 @@ fn full_round_trip_lifts_attests_gates_verifies_renders() {
 
     // Render regenerates the status line from the spine.
     let text = render(&node, Target::Cli);
-    assert!(text.contains("Status:    todo"), "render:\n{text}");
+    assert!(text.contains("Status: todo"), "render:\n{text}");
     assert!(text.contains("[x] On first load"));
 }
 
diff --git a/atomic-cli/src/commands/change/command.rs b/atomic-cli/src/commands/change/command.rs
index 9c9337a7..e541d4eb 100644
--- a/atomic-cli/src/commands/change/command.rs
+++ b/atomic-cli/src/commands/change/command.rs
@@ -52,7 +52,7 @@ pub struct ChangeCmd {
     /// Output format.
     ///
     /// Controls how change details are displayed:
-    /// - default: Full details with formatting
+    /// - default: Tree of metadata, changed paths and recorded provenance
     /// - short: Compact single-line format
     /// - json: Machine-readable JSON
     #[arg(short = 'f', long = "format", value_enum, default_value = "default")]
@@ -64,9 +64,9 @@ pub struct ChangeCmd {
     #[arg(long = "show-deps")]
     pub show_deps: bool,
 
-    /// Show graph_op details.
+    /// Expand per-path graph operation summaries in the tree.
     ///
-    /// When enabled, shows detailed information about each graph_op.
+    /// When enabled, includes coalesced graph operation details beneath each path.
     #[arg(long = "show-hunks")]
     pub show_hunks: bool,
 
@@ -313,192 +313,7 @@ impl ChangeCmd {
         sequence: Option,
         data: &ChangeRenderData,
     ) -> String {
-        let mut output = String::new();
-        let hash_len = self.get_hash_length();
-
-        // Change header
-        let hash_str = format_hash_with_length(hash, hash_len);
-        output.push_str(&format!(
-            "{} {}",
-            style_hash("change"),
-            style_hash(&hash_str)
-        ));
-        if let Some(seq) = sequence {
-            output.push_str(&format!(" {}", hint(&format!("(#{})", seq))));
-        }
-        output.push('\n');
-
-        // Authors
-        for author in &change.hashed.header.authors {
-            let author_line = format_author(author);
-            output.push_str(&format!("Author: {}\n", style_author(&author_line)));
-        }
-
-        // Timestamp
-        let time_str = format_timestamp(&change.hashed.header.timestamp);
-        output.push_str(&format!("Date:   {}\n", style_timestamp(&time_str)));
-
-        // Message
-        output.push('\n');
-        for line in change.hashed.header.message.lines() {
-            output.push_str(&format!("    {}\n", line));
-        }
-
-        // Description
-        if let Some(ref desc) = change.hashed.header.description {
-            output.push('\n');
-            for line in desc.lines() {
-                output.push_str(&format!("    {}\n", line));
-            }
-        }
-
-        // Dependencies
-        if !change.hashed.dependencies.is_empty() {
-            output.push('\n');
-            output.push_str(&format!(
-                "Dependencies: {}\n",
-                change.hashed.dependencies.len()
-            ));
-
-            for dep_hash in &change.hashed.dependencies {
-                let dep_hash_str = format_hash_with_length(dep_hash, 12);
-                let dep_msg = if self.show_deps {
-                    data.dep_messages
-                        .get(dep_hash)
-                        .cloned()
-                        .unwrap_or_else(|| "[unable to load]".to_string())
-                } else {
-                    String::new()
-                };
-
-                if self.show_deps {
-                    output.push_str(&format!("  {}... - {}\n", dep_hash_str, dep_msg));
-                } else {
-                    output.push_str(&format!("  {}...\n", dep_hash_str));
-                }
-            }
-        }
-
-        // Git import metadata (if present)
-        if let Some(ref unhashed) = change.unhashed {
-            if let Some(git) = unhashed.get("git") {
-                output.push('\n');
-                output.push_str("Git Import:\n");
-                if let Some(repo) = git.get("repository").and_then(|v| v.as_str()) {
-                    output.push_str(&format!("  Repository: {}\n", repo));
-                }
-                if let Some(sha) = git.get("sha").and_then(|v| v.as_str()) {
-                    output.push_str(&format!("  Commit: {}\n", &sha[..12.min(sha.len())]));
-                }
-                if git
-                    .get("empty_commit")
-                    .and_then(|v| v.as_bool())
-                    .unwrap_or(false)
-                {
-                    output.push_str(&format!("  {}\n", hint("(empty commit)")));
-                }
-                if git
-                    .get("empty_merge")
-                    .and_then(|v| v.as_bool())
-                    .unwrap_or(false)
-                {
-                    output.push_str(&format!("  {}\n", hint("(merge commit)")));
-                }
-            }
-        }
-
-        if let Ok(Some(evidence)) = extract_move_evidence(change) {
-            output.push('\n');
-            output.push_str("Move evidence:\n");
-            for moved in evidence.authoritative_moves {
-                let authority = match moved.authority {
-                    MoveAuthority::ExplicitAtomicMove => "explicit atomic move",
-                    MoveAuthority::StableInodeProjection => "stable inode projection",
-                };
-                output.push_str(&format!(
-                    "  authoritative: {} → {} ({})\n",
-                    moved.old_path, moved.new_path, authority
-                ));
-            }
-            for moved in evidence.probable_moves {
-                let basis = match moved.basis {
-                    MoveBasis::ByteIdentity => "byte identity",
-                    MoveBasis::ContentSimilarity => "content similarity",
-                };
-                output.push_str(&format!(
-                    "  probable: {} → {} ({}.{:02}%, {})\n",
-                    moved.old_path,
-                    moved.new_path,
-                    moved.score / 100,
-                    moved.score % 100,
-                    basis
-                ));
-            }
-            for loss in evidence.loss_notes {
-                match loss {
-                    LossNote::RenameUnresolved { candidates } => {
-                        output.push_str("  loss: rename unresolved; retained delete + add\n");
-                        for candidate in candidates {
-                            output.push_str(&format!(
-                                "    candidate: {} → {} ({}.{:02}%)\n",
-                                candidate.source_path,
-                                candidate.destination_path,
-                                candidate.score / 100,
-                                candidate.score % 100
-                            ));
-                        }
-                    }
-                    LossNote::EmptyDirectory { path } => {
-                        output.push_str(&format!(
-                            "  loss: empty directory '{}' omitted from Git tree projection\n",
-                            path
-                        ));
-                    }
-                }
-            }
-        }
-
-        // Graph statistics
-        output.push('\n');
-        let (vertices, edges) = count_atoms(&change.hashed.hunks);
-        let content_bytes = change.contents.len();
-        output.push_str(&format!(
-            "Graph: +{} vertices, ~{} edges, {} bytes\n",
-            vertices, edges, content_bytes
-        ));
-
-        // Hunks summary
-        if !change.hashed.hunks.is_empty() {
-            output.push_str(&format!(
-                "Files changed: {}\n",
-                count_unique_paths(&change.hashed.hunks)
-            ));
-
-            // Show one summary row per path. Git imports and large records can
-            // legitimately contain thousands of graph ops for a small number
-            // of files; printing each op makes the change view unusable.
-            for summary in hunk_display_summaries(&change.hashed.hunks) {
-                output.push_str(&format!(
-                    "  {} {} {}\n",
-                    summary.symbol,
-                    style_path(&summary.path),
-                    hint(&summary.info)
-                ));
-            }
-        }
-
-        // Attestation (inline AI metadata from change header)
-        if change.has_provenance() {
-            output.push('\n');
-            if let Some(prov) = change.hashed.provenance.first() {
-                output.push_str(&self.format_attestation(prov));
-            }
-        }
-
-        // Change ledger (causal decision DAG from .provenance file)
-        output.push_str(&self.format_change_ledger(&data.ledger_graphs));
-
-        output
+        super::tree::render(self, change, hash, sequence, data)
     }
 
     /// Format the change for short output.
@@ -549,152 +364,6 @@ impl ChangeCmd {
         )
     }
 
-    /// Format AI attestation information for display.
-    fn format_attestation(&self, prov: &Provenance) -> String {
-        let mut output = String::new();
-
-        output.push_str(&format!("{}\n", emphasis("=== Attestation ===")));
-        output.push_str(&format!(
-            "  Vendor:  {}\n",
-            info(&format!("{:?}", prov.vendor))
-        ));
-        output.push_str(&format!("  Model:   {}\n", info(&prov.model)));
-
-        if let Some(version) = &prov.model_version {
-            output.push_str(&format!("  Version: {}\n", hint(version)));
-        }
-
-        output.push_str(&format!(
-            "  Tool:    {}\n",
-            info(&format!("{:?}", prov.tool))
-        ));
-        output.push_str(&format!(
-            "  Type:    {}\n",
-            hint(&format!("{:?}", prov.suggestion_type))
-        ));
-
-        // Token usage
-        if !prov.tokens.is_empty() {
-            output.push_str("  Tokens:\n");
-            output.push_str(&format!("    Input:  {}\n", prov.tokens.input_tokens));
-            output.push_str(&format!("    Output: {}\n", prov.tokens.output_tokens));
-            output.push_str(&format!("    Total:  {}\n", prov.tokens.total_tokens));
-        }
-
-        // Cost
-        if !prov.cost.is_zero() {
-            output.push_str(&format!("  Cost:    ${:.6} USD\n", prov.cost.usd));
-        }
-
-        // Temperature
-        if let Some(temp) = prov.temperature {
-            let temp_f = temp as f64 / 1000.0;
-            output.push_str(&format!("  Temp:    {:.2}\n", temp_f));
-        }
-
-        // Request/Session IDs
-        if let Some(req_id) = &prov.request_id {
-            output.push_str(&format!("  Request: {}\n", hint(req_id)));
-        }
-        if let Some(sess_id) = &prov.session_id {
-            output.push_str(&format!("  Session: {}\n", hint(sess_id)));
-        }
-
-        // Additional metadata (key-value pairs from agent recording)
-        if !prov.metadata.is_empty() {
-            output.push_str("  Metadata:\n");
-            for (key, value) in &prov.metadata {
-                // turn_number is recorded as 1-indexed but session ledger
-                // turns are 0-indexed; display the 0-indexed value.
-                let display_value = if key == "turn_number" {
-                    value
-                        .parse::()
-                        .ok()
-                        .and_then(|n| n.checked_sub(1))
-                        .map(|n| n.to_string())
-                        .unwrap_or_else(|| value.clone())
-                } else {
-                    value.clone()
-                };
-                output.push_str(&format!("    {}: {}\n", hint(key), info(&display_value)));
-            }
-        }
-
-        output
-    }
-
-    /// Format the change ledger (causal decision DAG) for display.
-    ///
-    /// Displays the structured provenance graph stored in the `.provenance`
-    /// file: goals, tool executions, explorations, commitments, and patch
-    /// proposals that led to this change. The graphs arrive from the render
-    /// data (repository or wire — one render either way).
-    fn format_change_ledger(&self, graphs: &[(Hash, ProvenanceGraph)]) -> String {
-        let mut output = String::new();
-
-        if graphs.is_empty() {
-            output.push_str(&format!(
-                "{}\n",
-                hint("No provenance graph found for this change.")
-            ));
-            return output;
-        }
-
-        for (_graph_hash, graph) in graphs {
-            output.push_str(&format!("{}\n", emphasis("=== Change Ledger ===")));
-            output.push_str(&format!("  Session: {}\n", info(&graph.session_id)));
-            output.push_str(&format!(
-                "  Agent:   {} ({})\n",
-                info(&graph.agent_display_name),
-                hint(&graph.agent_vendor)
-            ));
-            output.push_str(&format!(
-                "  Nodes:   {}  Edges: {}  Changes: {}\n",
-                graph.node_count(),
-                graph.edge_count(),
-                graph.change_count()
-            ));
-            output.push('\n');
-
-            // Display nodes
-            for node in &graph.nodes {
-                let kind_str = format!("{}", node.kind);
-                let kind_styled = format!("{}", style(&kind_str).bold().cyan());
-                let duration = node
-                    .duration_ms
-                    .map(|ms| format!(" ({}ms)", ms))
-                    .unwrap_or_default();
-                let tool = node.tool_name.as_deref().unwrap_or("");
-                let tool_str = if tool.is_empty() {
-                    String::new()
-                } else {
-                    format!(" {}", hint(&format!("[{}]", tool)))
-                };
-
-                // Extract a one-line detail string from the structured detail JSON
-                let detail_str = node
-                    .detail
-                    .as_ref()
-                    .and_then(|d| format_node_detail(d))
-                    .map(|s| format!("\n      {}", hint(&s)))
-                    .unwrap_or_default();
-
-                output.push_str(&format!(
-                    "  {} {} {}{}{}{}
-",
-                    kind_styled,
-                    hint("\u{00bb}"),
-                    node.summary,
-                    tool_str,
-                    hint(&duration),
-                    detail_str,
-                ));
-            }
-        }
-
-        output
-    }
-
     /// Format the change for JSON output.
     pub(crate) fn format_json(
         &self,
@@ -710,8 +379,8 @@ impl ChangeCmd {
     }
 
     /// Load the change ledger(s) — the causal decision graph(s) from the
-    /// `.provenance` file — for JSON output. Mirrors the `=== Change Ledger ===`
-    /// section of the default text format. Missing/corrupt provenance is
+    /// `.provenance` file — for JSON output. Mirrors the `Change Ledger`
+    /// branch of the default text format. Missing/corrupt provenance is
     /// non-fatal: the ledger simply stays empty (and is omitted from the JSON).
     fn load_json_ledger(&self, graphs: &[(Hash, ProvenanceGraph)]) -> Vec {
         graphs
@@ -743,7 +412,7 @@ impl ChangeCmd {
 /// Extract a human-readable one-liner from a provenance node's detail JSON.
 ///
 /// Returns `None` if the detail is empty or has no interesting fields.
-fn format_node_detail(detail: &str) -> Option {
+pub(super) fn format_node_detail(detail: &str) -> Option {
     let v: serde_json::Value = serde_json::from_str(detail).ok()?;
     let obj = v.as_object()?;
 
diff --git a/atomic-cli/src/commands/change/mod.rs b/atomic-cli/src/commands/change/mod.rs
index ae554d86..a2c5d591 100644
--- a/atomic-cli/src/commands/change/mod.rs
+++ b/atomic-cli/src/commands/change/mod.rs
@@ -25,28 +25,31 @@
 //!
 //! ## Default Format
 //!
-//! The default format provides comprehensive information about the change:
+//! The default format groups change metadata and evidence in a terminal tree.
+//! Directory names end in `/`; textual badges describe recorded operations,
+//! without folder emoji or a dependence on color. `--show-hunks` expands the
+//! per-path graph summaries and `--show-deps` includes dependency messages.
 //!
 //! ```text
-//! change ABC123456789012345678901234567890123456789012345678901
-//! Author: Alice 
-//! Date:   Mon Jan 15 10:30:45 2024 -0500
-//!
-//!     Add authentication module
-//!
-//!     This implements JWT-based authentication for the API.
-//!     Includes token generation and validation.
-//!
-//! Dependencies: 2
-//!   DEF98765... - Initial project setup
-//!   GHI11111... - Add user model
-//!
-//! Files changed: 3
-//!   + src/auth/mod.rs
-//!   + src/auth/jwt.rs
-//!   ~ src/lib.rs
+//! change ABC123456789
+//! ├── Metadata
+//! │   ├── Author: Alice 
+//! │   └── Date: 2024-01-15 10:30:45
+//! ├── Message
+//! │   └── Add authentication module
+//! ├── Graph: +12 vertices, ~3 edges, 420 bytes
+//! └── Files changed: 3
+//!     └── src/
+//!         ├── auth/
+//!         │   ├── jwt.rs [added]
+//!         │   └── mod.rs [added]
+//!         └── lib.rs [modified]
 //! ```
 //!
+//! Other badges include `[deleted]`, `[moved]`, `[restored]`, `[attributes]`,
+//! `[resolved]` and `[conflicted]`. Unchanged parent directories are grouping
+//! nodes without a badge, not assertions that those directories were changed.
+//!
 //! ## Short Format (-f short)
 //!
 //! Compact single-line format:
@@ -84,8 +87,8 @@
 //! ```
 //!
 //! The `ledger` array carries the causal decision graph(s) explaining the
-//! change (the machine-readable form of the `=== Change Ledger ===` text
-//! section). It is omitted entirely when no provenance graph is recorded.
+//! change (the machine-readable form of the `Change Ledger` tree branch).
+//! It is omitted entirely when no provenance graph is recorded.
 //!
 //! # Change Identification
 //!
@@ -139,14 +142,13 @@ use crate::commands::{
 };
 use crate::error::{CliError, CliResult};
 use crate::output::{
-    author as style_author, emphasis, hash as style_hash, hint, info, path as style_path,
-    timestamp as style_timestamp, truncate_bytes,
+    author as style_author, hash as style_hash, hint, timestamp as style_timestamp, truncate_bytes,
 };
-use console::style;
 
 // Output Format
 
 pub(crate) mod command;
+mod tree;
 pub(crate) mod types;
 
 pub use command::*;
diff --git a/atomic-cli/src/commands/change/tree.rs b/atomic-cli/src/commands/change/tree.rs
new file mode 100644
index 00000000..5cf06c57
--- /dev/null
+++ b/atomic-cli/src/commands/change/tree.rs
@@ -0,0 +1,601 @@
+//! Presentation-only hierarchy; ledger entries remain in recorded order rather
+//! than inventing a causal tree from a DAG. Paths come from the change, not disk.
+
+use std::collections::{BTreeMap, BTreeSet};
+
+use super::*;
+
+struct Tree {
+    label: String,
+    children: Vec,
+}
+
+impl Tree {
+    fn new(label: impl Into) -> Self {
+        Self {
+            label: label.into(),
+            children: Vec::new(),
+        }
+    }
+
+    fn push(&mut self, label: impl Into) {
+        self.children.push(Self::new(label));
+    }
+
+    fn section(&mut self, section: Self) {
+        if !section.children.is_empty() {
+            self.children.push(section);
+        }
+    }
+
+    fn render(&self) -> String {
+        let mut out = format!("{}\n", clean(&self.label));
+        let mut pending: Vec<_> = self
+            .children
+            .iter()
+            .enumerate()
+            .rev()
+            .map(|(i, child)| (child, String::new(), i + 1 == self.children.len()))
+            .collect();
+        while let Some((node, prefix, last)) = pending.pop() {
+            let branch = if last { "└──" } else { "├──" };
+            out.push_str(&format!("{prefix}{branch} {}\n", clean(&node.label)));
+            let prefix = format!("{prefix}{}", if last { "    " } else { "│   " });
+            for (i, child) in node.children.iter().enumerate().rev() {
+                pending.push((child, prefix.clone(), i + 1 == node.children.len()));
+            }
+        }
+        out
+    }
+}
+
+fn clean(value: &str) -> String {
+    value
+        .chars()
+        .map(|c| if c.is_control() { ' ' } else { c })
+        .collect()
+}
+
+fn text_section(title: &str, text: &str) -> Tree {
+    let mut section = Tree::new(title);
+    for line in text.lines().filter(|line| !line.trim().is_empty()) {
+        section.push(line);
+    }
+    section
+}
+
+pub(super) fn render(
+    cmd: &ChangeCmd,
+    change: &Change,
+    hash: &Hash,
+    sequence: Option,
+    data: &ChangeRenderData,
+) -> String {
+    let hash_len = cmd.get_hash_length();
+    let seq = sequence.map(|seq| format!(" (#{seq})")).unwrap_or_default();
+    let mut root = Tree::new(format!(
+        "change {}{seq}",
+        format_hash_with_length(hash, hash_len)
+    ));
+    let mut metadata = Tree::new("Metadata");
+    for author in &change.hashed.header.authors {
+        metadata.push(format!("Author: {}", format_author(author)));
+    }
+    metadata.push(format!(
+        "Date: {}",
+        format_timestamp(&change.hashed.header.timestamp)
+    ));
+    root.section(metadata);
+    root.section(text_section("Message", &change.hashed.header.message));
+    if let Some(description) = &change.hashed.header.description {
+        root.section(text_section("Description", description));
+    }
+
+    let mut dependencies = Tree::new(format!(
+        "Dependencies ({})",
+        change.hashed.dependencies.len()
+    ));
+    for hash in &change.hashed.dependencies {
+        let id = format_hash_with_length(hash, hash_len);
+        let message = if cmd.show_deps {
+            format!(
+                " — {}",
+                data.dep_messages
+                    .get(hash)
+                    .map(String::as_str)
+                    .unwrap_or("[unable to load]")
+            )
+        } else {
+            String::new()
+        };
+        dependencies.push(format!("{id}{message}"));
+    }
+    root.section(dependencies);
+
+    if let Some(git) = change
+        .unhashed
+        .as_ref()
+        .and_then(|metadata| metadata.get("git"))
+    {
+        let mut section = Tree::new("Git Import");
+        if let Some(repository) = git.get("repository").and_then(|v| v.as_str()) {
+            section.push(format!("Repository: {repository}"));
+        }
+        if let Some(sha) = git.get("sha").and_then(|v| v.as_str()) {
+            section.push(format!(
+                "Git source revision: {}",
+                if cmd.full_hash {
+                    sha.to_owned()
+                } else {
+                    sha.chars().take(12).collect::()
+                }
+            ));
+        }
+        for (key, label) in [
+            ("empty_commit", "Empty Git source revision"),
+            ("empty_merge", "Git merge source revision"),
+        ] {
+            if git.get(key).and_then(|v| v.as_bool()).unwrap_or(false) {
+                section.push(label);
+            }
+        }
+        root.section(section);
+    }
+
+    if let Ok(Some(evidence)) = extract_move_evidence(change) {
+        let mut section = Tree::new("Move evidence");
+        for moved in evidence.authoritative_moves {
+            let authority = match moved.authority {
+                MoveAuthority::ExplicitAtomicMove => "explicit atomic move",
+                MoveAuthority::StableInodeProjection => "stable inode projection",
+            };
+            section.push(format!(
+                "authoritative: {} → {} ({authority})",
+                moved.old_path, moved.new_path
+            ));
+        }
+        for moved in evidence.probable_moves {
+            let basis = match moved.basis {
+                MoveBasis::ByteIdentity => "byte identity",
+                MoveBasis::ContentSimilarity => "content similarity",
+            };
+            section.push(format!(
+                "probable: {} → {} ({}.{:02}%, {basis})",
+                moved.old_path,
+                moved.new_path,
+                moved.score / 100,
+                moved.score % 100
+            ));
+        }
+        for loss in evidence.loss_notes {
+            match loss {
+                LossNote::RenameUnresolved { candidates } => {
+                    let mut loss = Tree::new("loss: rename unresolved; retained delete + add");
+                    for candidate in candidates {
+                        loss.push(format!(
+                            "candidate: {} → {} ({}.{:02}%)",
+                            candidate.source_path,
+                            candidate.destination_path,
+                            candidate.score / 100,
+                            candidate.score % 100
+                        ));
+                    }
+                    section.children.push(loss);
+                }
+                LossNote::EmptyDirectory { path } => section.push(format!(
+                    "loss: empty directory '{path}' omitted from Git tree projection"
+                )),
+            }
+        }
+        root.section(section);
+    }
+
+    let (vertices, edges) = count_atoms(&change.hashed.hunks);
+    root.push(format!(
+        "Graph: +{vertices} vertices, ~{edges} edges, {} bytes",
+        change.contents.len()
+    ));
+    root.section(files(change, cmd.show_hunks));
+    // Keep every recorded provenance entry, not just the first provider/model.
+    for prov in &change.hashed.provenance {
+        root.section(attestation(prov));
+    }
+    for (_, graph) in &data.ledger_graphs {
+        root.section(ledger(graph));
+    }
+    root.render()
+}
+
+#[derive(Default)]
+struct PathTree {
+    children: BTreeMap,
+    summary: Option,
+    directory: bool,
+}
+
+fn files(change: &Change, show_hunks: bool) -> Tree {
+    let directories: BTreeSet<_> = change
+        .hashed
+        .hunks
+        .iter()
+        .filter_map(|op| match op {
+            GraphOp::DirAdd { path, .. }
+            | GraphOp::DirDel { path, .. }
+            | GraphOp::DirUndel { path, .. } => Some(path.as_str()),
+            _ => None,
+        })
+        .collect();
+    let mut paths = PathTree::default();
+    for summary in hunk_display_summaries(&change.hashed.hunks) {
+        let mut node = &mut paths;
+        // Atomic paths use '/', including when the CLI runs on Windows. Do not
+        // interpret platform-specific separators or consult the working copy.
+        for part in summary.path.split('/') {
+            node = node.children.entry(part.to_owned()).or_default();
+        }
+        node.directory = directories.contains(summary.path.as_str());
+        node.summary = Some(summary);
+    }
+    let mut section = Tree::new(format!(
+        "Files changed: {}",
+        count_unique_paths(&change.hashed.hunks)
+    ));
+    section.children = path_nodes(paths, show_hunks);
+    section
+}
+
+fn path_nodes(paths: PathTree, show_hunks: bool) -> Vec {
+    paths
+        .children
+        .into_iter()
+        .map(|(name, path)| {
+            let slash = if path.directory || !path.children.is_empty() {
+                "/"
+            } else {
+                ""
+            };
+            let badge = path
+                .summary
+                .as_ref()
+                .map(|summary| format!(" [{}]", status(summary.symbol)))
+                .unwrap_or_default();
+            let mut node = Tree::new(format!("{name}{slash}{badge}"));
+            if show_hunks {
+                if let Some(summary) = &path.summary {
+                    node.push(summary.info.trim_start_matches('(').trim_end_matches(')'));
+                }
+            }
+            node.children.extend(path_nodes(path, show_hunks));
+            node
+        })
+        .collect()
+}
+
+fn status(symbol: &str) -> &str {
+    match symbol {
+        "+" | "📁+" => "added",
+        "-" | "📁-" => "deleted",
+        "~" | "±" => "modified",
+        "→" => "moved",
+        "↑" | "📁↑" => "restored",
+        "@" => "attributes",
+        "✓" => "resolved",
+        "!" => "conflicted",
+        "◉" => "root added",
+        "⊘" => "root deleted",
+        _ => "changed",
+    }
+}
+
+fn attestation(prov: &Provenance) -> Tree {
+    let mut section = Tree::new("Attestation");
+    section.push(format!("Vendor: {:?}", prov.vendor));
+    section.push(format!("Model: {}", prov.model));
+    if let Some(version) = &prov.model_version {
+        section.push(format!("Version: {version}"));
+    }
+    section.push(format!("Tool: {:?}", prov.tool));
+    section.push(format!("Type: {:?}", prov.suggestion_type));
+    if !prov.tokens.is_empty() {
+        let mut tokens = Tree::new("Tokens");
+        tokens.push(format!("Input: {}", prov.tokens.input_tokens));
+        tokens.push(format!("Output: {}", prov.tokens.output_tokens));
+        tokens.push(format!("Total: {}", prov.tokens.total_tokens));
+        section.section(tokens);
+    }
+    if !prov.cost.is_zero() {
+        section.push(format!("Cost: ${:.6} USD", prov.cost.usd));
+    }
+    if let Some(temp) = prov.temperature {
+        section.push(format!("Temp: {:.2}", temp as f64 / 1000.0));
+    }
+    if let Some(id) = &prov.request_id {
+        section.push(format!("Request: {id}"));
+    }
+    if let Some(id) = &prov.session_id {
+        section.push(format!("Session: {id}"));
+    }
+    let mut metadata = Tree::new("Metadata");
+    for (key, value) in &prov.metadata {
+        let value = if key == "turn_number" {
+            value
+                .parse::()
+                .ok()
+                .and_then(|n| n.checked_sub(1))
+                .map(|n| n.to_string())
+                .unwrap_or_else(|| value.clone())
+        } else {
+            value.clone()
+        };
+        metadata.push(format!("{key}: {value}"));
+    }
+    section.section(metadata);
+    section
+}
+
+fn ledger(graph: &ProvenanceGraph) -> Tree {
+    let mut section = Tree::new("Change Ledger");
+    section.push(format!("Session: {}", graph.session_id));
+    section.push(format!(
+        "Agent: {} ({})",
+        graph.agent_display_name, graph.agent_vendor
+    ));
+    section.push(format!(
+        "Nodes: {}  Edges: {}  Changes: {}",
+        graph.node_count(),
+        graph.edge_count(),
+        graph.change_count()
+    ));
+    for node in &graph.nodes {
+        let tool = node
+            .tool_name
+            .as_ref()
+            .map(|tool| format!(" [{tool}]"))
+            .unwrap_or_default();
+        let duration = node
+            .duration_ms
+            .map(|ms| format!(" ({ms}ms)"))
+            .unwrap_or_default();
+        let mut entry = Tree::new(format!("{}: {}{tool}{duration}", node.kind, node.summary));
+        if let Some(detail) = node
+            .detail
+            .as_ref()
+            .and_then(|detail| command::format_node_detail(detail))
+        {
+            entry.push(detail);
+        }
+        section.children.push(entry);
+    }
+    section
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use atomic_core::change::{Atom, ChangeHeader, EdgeUpdate, Insertion, Local};
+    use atomic_core::{ChangePosition, EdgeFlags, Position};
+
+    fn insertion() -> Insertion> {
+        Insertion {
+            predecessors: vec![],
+            successors: vec![],
+            flag: EdgeFlags::BLOCK,
+            start: ChangePosition::new(0),
+            end: ChangePosition::new(1),
+            inode: Position::new(None, ChangePosition::new(0)),
+        }
+    }
+
+    fn edges() -> EdgeUpdate> {
+        EdgeUpdate {
+            edges: vec![],
+            inode: Position::new(None, ChangePosition::new(0)),
+        }
+    }
+
+    fn fixture() -> Change {
+        Change::new(
+            ChangeHeader::new("Update CLI\nKeep it readable"),
+            vec![
+                GraphOp::DirAdd {
+                    path: "src".into(),
+                    add_name: insertion(),
+                    add_inode: insertion(),
+                },
+                GraphOp::FileAdd {
+                    path: "src/new.ts".into(),
+                    add_name: insertion(),
+                    add_inode: insertion(),
+                    contents: None,
+                    encoding: None,
+                },
+                GraphOp::FileDel {
+                    path: "src/old.ts".into(),
+                    del: edges(),
+                    contents: None,
+                    encoding: None,
+                },
+                GraphOp::Edit {
+                    local: Local::new("README.md", 1),
+                    change: Atom::Insertion(insertion()),
+                    encoding: None,
+                },
+                GraphOp::Edit {
+                    local: Local::new("README.md", 2),
+                    change: Atom::Insertion(insertion()),
+                    encoding: None,
+                },
+                GraphOp::FileMove {
+                    path: "src/renamed.ts".into(),
+                    del: edges(),
+                    add: insertion(),
+                },
+                GraphOp::DirDel {
+                    path: "obsolete".into(),
+                    del: edges(),
+                },
+                GraphOp::DirUndel {
+                    path: "restored".into(),
+                    undel: edges(),
+                },
+            ],
+            vec![b'x'],
+            vec![],
+        )
+    }
+
+    fn data() -> ChangeRenderData {
+        ChangeRenderData {
+            dep_messages: Default::default(),
+            ledger_graphs: vec![],
+        }
+    }
+
+    #[test]
+    fn paths_have_text_status_and_directories_have_slashes_not_emoji() {
+        let change = fixture();
+        assert_eq!(
+            files(&change, false).render(),
+            concat!(
+                "Files changed: 7\n",
+                "├── README.md [modified]\n",
+                "├── obsolete/ [deleted]\n",
+                "├── restored/ [restored]\n",
+                "└── src/ [added]\n",
+                "    ├── new.ts [added]\n",
+                "    ├── old.ts [deleted]\n",
+                "    └── renamed.ts [moved]\n",
+            )
+        );
+    }
+
+    #[test]
+    fn path_grouping_is_stable_and_hunk_details_expand_the_same_nodes() {
+        let mut change = fixture();
+        let compact = files(&change, false).render();
+        change.hashed.hunks.reverse();
+        assert_eq!(files(&change, false).render(), compact);
+        let detailed = files(&change, true).render();
+        assert_eq!(detailed.matches("README.md [modified]").count(), 1);
+        assert!(detailed.contains("│   └── 2 hunks:"));
+        assert!(!compact.contains("span"));
+        assert!(detailed.contains("span"));
+    }
+
+    #[test]
+    fn full_hash_and_dependency_messages_are_preserved() {
+        let mut change = fixture();
+        let dependency = Hash::of(b"dependency");
+        change.hashed.dependencies = vec![dependency];
+        let hash = Hash::of(b"change");
+        let mut data = data();
+        data.dep_messages.insert(dependency, "Initial setup".into());
+        let cmd = ChangeCmd::new().with_full_hash(true).with_show_deps(true);
+        let text = render(&cmd, &change, &hash, Some(42), &data);
+        assert!(text.starts_with(&format!("change {} (#42)\n", hash.to_base32())));
+        assert!(text.contains(&format!(
+            "│   └── {} — Initial setup\n",
+            dependency.to_base32()
+        )));
+        assert!(text.contains("├── Message\n│   ├── Update CLI\n│   └── Keep it readable\n"));
+    }
+
+    #[test]
+    fn empty_sections_are_omitted_and_controls_cannot_break_the_tree() {
+        let mut change = Change::empty(ChangeHeader::new("\n"));
+        change.hashed.header.description = Some(" \n ".into());
+        let text = render(
+            &ChangeCmd::new(),
+            &change,
+            &Hash::of(b"empty"),
+            None,
+            &data(),
+        );
+        for empty in [
+            "Dependencies",
+            "Files changed",
+            "Attestation",
+            "Change Ledger",
+            "No provenance",
+            "Message",
+            "Description",
+        ] {
+            assert!(!text.contains(empty));
+        }
+        assert!(text.contains("└── Graph:"));
+        let mut node = Tree::new("Files");
+        node.push("name\n\u{1b}[31m.ts");
+        assert_eq!(node.render(), "Files\n└── name  [31m.ts\n");
+    }
+
+    #[test]
+    fn git_import_labels_identify_source_revisions_not_atomic_commits() {
+        let mut change = fixture();
+        let sha = "abcdef0123456789".repeat(4);
+        change.unhashed = Some(serde_json::json!({"git": {
+            "repository": "source-repository", "sha": sha,
+            "empty_commit": true, "empty_merge": true
+        }}));
+        let hash = Hash::of(b"change");
+        let text = render(&ChangeCmd::new(), &change, &hash, None, &data());
+        assert!(text.contains("Git source revision: abcdef012345\n"));
+        assert!(text.contains("Empty Git source revision"));
+        assert!(text.contains("Git merge source revision"));
+        assert!(!text.to_lowercase().contains("commit"));
+        let text = render(
+            &ChangeCmd::new().with_full_hash(true),
+            &change,
+            &hash,
+            None,
+            &data(),
+        );
+        assert!(text.contains(&format!("Git source revision: {sha}\n")));
+    }
+
+    #[test]
+    fn all_attestations_and_recorded_ledger_nodes_are_nested() {
+        use atomic_core::change::provenance_graph::{ProvenanceNode, ProvenanceNodeKind};
+        let mut change = fixture();
+        for model in ["model-one", "model-two"] {
+            let mut prov = Provenance::builder().model(model).cost_usd(0.125).build();
+            prov.tokens = atomic_core::change::TokenUsage::full(100, 40, 10, 60, 20);
+            prov.metadata.push(("turn_number".into(), "2".into()));
+            change.add_provenance(prov);
+        }
+        let mut data = data();
+        data.ledger_graphs.push((
+            Hash::of(b"ledger"),
+            ProvenanceGraph::builder("session", "opencode")
+                .add_node(ProvenanceNode {
+                    id: "step-1".into(),
+                    kind: ProvenanceNodeKind::Execution,
+                    timestamp: 1000,
+                    summary: "Run tests".into(),
+                    detail: Some(r#"{"command":"cargo test"}"#.into()),
+                    change_hash: None,
+                    tool_name: Some("shell".into()),
+                    tool_call_id: None,
+                    duration_ms: Some(150),
+                    classified: false,
+                    confidence: None,
+                    consolidated_from: vec![],
+                })
+                .build(),
+        ));
+        let text = render(
+            &ChangeCmd::new(),
+            &change,
+            &Hash::of(b"change"),
+            None,
+            &data,
+        );
+        assert_eq!(text.matches("├── Attestation").count(), 2);
+        assert!(text.contains("Model: model-one") && text.contains("Model: model-two"));
+        assert!(text.contains("│   ├── Tokens\n│   │   ├── Input: 100"));
+        assert!(text.contains("Cost: $0.125000 USD"));
+        assert!(text.contains("turn_number: 1"));
+        assert!(text.contains("└── Change Ledger\n    ├── Session: session"));
+        assert!(text
+            .contains("    └── execution: Run tests [shell] (150ms)\n        └── $ cargo test\n"));
+    }
+}
diff --git a/atomic-cli/src/commands/change/types.rs b/atomic-cli/src/commands/change/types.rs
index 1ef28f10..1c7b069d 100644
--- a/atomic-cli/src/commands/change/types.rs
+++ b/atomic-cli/src/commands/change/types.rs
@@ -7,7 +7,7 @@ use super::*;
 /// or machine parsing.
 #[derive(Debug, Clone, Copy, PartialEq, Eq, ValueEnum, Default)]
 pub enum ChangeFormat {
-    /// Full detailed format with all information.
+    /// Tree format with metadata, changed paths and recorded provenance.
     ///
     /// Shows hash, author, date, message, description, dependencies, and hunks summary.
     #[default]
diff --git a/atomic-cli/src/commands/intent/graph.rs b/atomic-cli/src/commands/intent/graph.rs
index d45b4e53..93eb94df 100644
--- a/atomic-cli/src/commands/intent/graph.rs
+++ b/atomic-cli/src/commands/intent/graph.rs
@@ -1,10 +1,15 @@
 //! `atomic intent graph ` — semantic lineage, not a search neighborhood.
+//!
+//! Terminal output is a grouped tree with one line per selected node. Groups
+//! organize evidence; they do not assert parent/child relationships in the KG.
+//! Use `--details` for complete text and canonical relationships, `--json` for
+//! machine-readable facts, or `--html` for the interactive browser graph.
 
 use crate::{
     commands::Command,
     error::{CliError, CliResult},
 };
-use atomic_core::kg::ontology::compact;
+use atomic_core::kg::ontology::{compact, Class};
 use atomic_repository::intent_graph::{IntentGraph as Graph, DEFAULT_MAX_NODES};
 use clap::Parser;
 
@@ -21,6 +26,9 @@ pub struct IntentGraph {
     /// Save an interactive HTML graph and open it in the default browser.
     #[arg(long)]
     pub html: bool,
+    /// Expand the tree with full labels, canonical IDs, summaries and relationships.
+    #[arg(long, conflicts_with_all = ["json", "dot", "html"])]
+    pub details: bool,
     /// Do not open a browser; emit HTML to stdout or the --output file.
     #[arg(long, requires = "html")]
     pub no_open: bool,
@@ -76,11 +84,16 @@ impl Command for IntentGraph {
                 &format!("{notes}"),
                 1,
             )
+        } else if self.details {
+            render_details(&graph)
         } else {
             render_text(&graph)
         };
-        for diagnostic in &graph.diagnostics {
-            eprintln!("Note: {diagnostic}");
+        // Text already includes the notes; exports keep diagnostics off stdout.
+        if self.json || self.dot || self.html {
+            for diagnostic in &graph.diagnostics {
+                eprintln!("Note: {diagnostic}");
+            }
         }
         if self.html && !self.no_open {
             write_and_open_html(&output, self.output.as_deref(), |url| open::that(url))?;
@@ -172,47 +185,230 @@ fn annotate(graph: &Graph, nodes: &mut [atomic_core::KgNode]) {
 }
 
 pub(crate) fn render_text(graph: &Graph) -> String {
-    let mut out = format!("Intent semantic change graph\nRoot: {}\nSnapshot: {}\nScope: {}\n{} nodes, {} edges; {} implementing changes, {} intent-artifact recording changes, {} dependency-only changes{}\n\n", graph.root, graph.snapshot_identity, graph.scope, graph.nodes.len(), graph.edges.len(), graph.implementing_changes.len(), graph.recording_changes.len(), graph.dependency_changes.len(), if graph.truncated { " [TRUNCATED]" } else { "" });
-    let mut nodes = graph.nodes.clone();
-    annotate(graph, &mut nodes);
-    for node in &nodes {
-        out.push_str(&format!(
-            "[{}] {}\n  {}\n",
-            compact(node.primary_type()),
-            one_line(&node.label),
-            node.id
-        ));
-        if let Some(summary) = &node.summary {
-            out.push_str(&format!("  {}\n", one_line(summary)));
+    render_tree(graph, false)
+}
+
+fn render_details(graph: &Graph) -> String {
+    render_tree(graph, true)
+}
+
+fn render_tree(graph: &Graph, details: bool) -> String {
+    use std::collections::BTreeMap;
+
+    let root = graph.nodes.iter().find(|node| node.id == graph.root);
+    let mut out = match root {
+        Some(node) => {
+            let key = metadata_text(node, "human_key");
+            let label = tree_label(node);
+            format!(
+                "Intent{} — {label}\n",
+                key.map(|s| format!(" {s}")).unwrap_or_default()
+            )
+        }
+        None => "Intent [root missing]\n".into(),
+    };
+    out.push_str(&format!(
+        "{} nodes · {} implementing · {} recording · {} dependency-only{}\n",
+        graph.nodes.len(),
+        graph.implementing_changes.len(),
+        graph.recording_changes.len(),
+        graph.dependency_changes.len(),
+        if graph.truncated {
+            " · [TRUNCATED]"
+        } else {
+            ""
+        },
+    ));
+
+    // These are presentation groups, not newly asserted graph edges. Each
+    // selected node appears once, even when it has multiple incoming edges.
+    let mut groups: BTreeMap<(u8, String), Vec<&atomic_core::KgNode>> = BTreeMap::new();
+    for node in graph.nodes.iter().filter(|node| node.id != graph.root) {
+        let group = if graph.implementing_changes.contains(&node.id) {
+            (0, "Implementing changes".into())
+        } else if graph.recording_changes.contains(&node.id) {
+            (1, "Intent recording evidence (not implementation)".into())
+        } else if graph.dependency_changes.contains(&node.id) {
+            (2, "Dependency-only changes".into())
+        } else if graph.intent_artifact.as_ref() == Some(&node.id) {
+            (3, "Intent artifact".into())
+        } else if node.has_type(Class::Task) {
+            (4, "Tasks".into())
+        } else if node.has_type(Class::AcceptanceCriterion) {
+            (5, "Acceptance criteria".into())
+        } else if node.has_type(Class::ScopeItem) {
+            (6, "Scope".into())
+        } else if node.has_type(Class::Constraint) {
+            (7, "Constraints".into())
+        } else if node.has_type(Class::File) {
+            (8, "Files".into())
+        } else {
+            (
+                9,
+                compact(node.primary_type())
+                    .trim_start_matches("atomic:")
+                    .to_owned(),
+            )
+        };
+        groups.entry(group).or_default().push(node);
+    }
+    let count = groups.len();
+    if details {
+        let mut lines = vec![
+            format!("Snapshot: {}", one_line(&graph.snapshot_identity)),
+            format!("Scope: {}", one_line(graph.scope)),
+        ];
+        if let Some(node) = root {
+            lines.extend(node_details(graph, node));
+        } else {
+            lines.push(format!("ID: {}", one_line(&graph.root)));
         }
-        for edge in graph.edges.iter().filter(|e| e.from_id == node.id) {
+        write_tree_children(&mut out, "", &lines, count > 0);
+    }
+    for (index, ((_, title), mut nodes)) in groups.into_iter().enumerate() {
+        let last_group = index + 1 == count;
+        let branch = if last_group { "└──" } else { "├──" };
+        let indent = if last_group { "    " } else { "│   " };
+        nodes.sort_by(|a, b| (&a.label, &a.id).cmp(&(&b.label, &b.id)));
+        out.push_str(&format!("{branch} {title} ({})\n", nodes.len()));
+        for (i, node) in nodes.iter().enumerate() {
+            let branch = if i + 1 == nodes.len() {
+                "└──"
+            } else {
+                "├──"
+            };
+            let declaration = if graph.declared_changes.contains(&node.id)
+                && !graph.captured_changes.contains(&node.id)
+            {
+                " [declared; original turn unproven]"
+            } else {
+                ""
+            };
+            let recording = if graph.recording_changes.contains(&node.id)
+                && graph.implementing_changes.contains(&node.id)
+            {
+                " [also records intent]"
+            } else {
+                ""
+            };
             out.push_str(&format!(
-                "  --{}--> {}\n",
-                compact(edge.kind.iri()),
-                edge.to_id
+                "{indent}{branch} {}{declaration}{recording}\n",
+                tree_label(node)
             ));
-            if let Some(evidence) = edge
-                .metadata
-                .as_ref()
-                .and_then(|m| m.get("declarationEvidence"))
-                .and_then(|v| v.as_str())
-            {
-                out.push_str(&format!(
-                    "    Declaration evidence: {}\n",
-                    one_line(evidence)
-                ));
+            if details {
+                let child_indent = format!(
+                    "{indent}{}",
+                    if i + 1 == nodes.len() {
+                        "    "
+                    } else {
+                        "│   "
+                    }
+                );
+                write_tree_children(&mut out, &child_indent, &node_details(graph, node), false);
             }
         }
     }
     for id in &graph.missing_resources {
-        out.push_str(&format!("MISSING: {id}\n"));
+        out.push_str(&format!("Missing resource: {id}\n"));
     }
     for diagnostic in &graph.diagnostics {
-        out.push_str(&format!("Note: {diagnostic}\n"));
+        out.push_str(&format!("Note: {}\n", one_line(diagnostic)));
     }
+    out.push_str(if details {
+        "\nGrouped tree; relationships are references, not expanded subtrees.\n"
+    } else {
+        "\nGrouped tree; use --details for full labels, canonical IDs and relationships.\n"
+    });
     out
 }
 
+fn write_tree_children(out: &mut String, indent: &str, lines: &[String], has_following: bool) {
+    for (index, line) in lines.iter().enumerate() {
+        let branch = if index + 1 == lines.len() && !has_following {
+            "└──"
+        } else {
+            "├──"
+        };
+        out.push_str(&format!("{indent}{branch} {line}\n"));
+    }
+}
+
+fn node_details(graph: &Graph, node: &atomic_core::KgNode) -> Vec {
+    let mut lines = vec![
+        format!("ID: {}", one_line(&node.id)),
+        format!("Type: {}", compact(node.primary_type())),
+    ];
+    let label = if node.has_type(Class::File) {
+        metadata_text(node, "path").unwrap_or(&node.label)
+    } else {
+        &node.label
+    };
+    if console::measure_text_width(&one_line(label)) > 100 {
+        lines.push(format!("Label: {}", one_line(label)));
+    }
+    let mut annotated = [node.clone()];
+    annotate(graph, &mut annotated);
+    if let Some(summary) = &annotated[0].summary {
+        let summary = one_line(summary);
+        let summary = summary.trim();
+        // Context is the fallback role, not evidence or a useful summary.
+        let summary = summary.strip_prefix("[context]").unwrap_or(summary).trim();
+        if !summary.is_empty() {
+            lines.push(format!("Summary: {summary}"));
+        }
+    }
+    // References preserve graph direction without recursively expanding shared
+    // targets or cycles, so details cannot change the tree's grouping.
+    let mut edges: Vec<_> = graph
+        .edges
+        .iter()
+        .filter(|edge| edge.from_id == node.id)
+        .collect();
+    edges.sort_by(|a, b| (a.kind, &a.to_id).cmp(&(b.kind, &b.to_id)));
+    for edge in edges {
+        let mut line = format!("{} → {}", compact(edge.kind.iri()), one_line(&edge.to_id));
+        if let Some(evidence) = edge
+            .metadata
+            .as_ref()
+            .and_then(|m| m.get("declarationEvidence"))
+            .and_then(|v| v.as_str())
+        {
+            line.push_str(&format!(" [Declaration evidence: {}]", one_line(evidence)));
+        }
+        lines.push(line);
+    }
+    lines
+}
+
+fn metadata_text<'a>(node: &'a atomic_core::KgNode, key: &str) -> Option<&'a str> {
+    node.metadata.as_ref()?.get(key)?.as_str()
+}
+
+fn tree_label(node: &atomic_core::KgNode) -> String {
+    let label = if node.has_type(Class::File) {
+        metadata_text(node, "path").unwrap_or(&node.label)
+    } else {
+        &node.label
+    };
+    let label = one_line(label);
+    let status = metadata_text(node, "status")
+        .map(|value| format!(" [{}]", one_line(value)))
+        .unwrap_or_default();
+    if node.has_type(Class::Change) {
+        let id = node.id.rsplit(':').next().unwrap_or(&node.id);
+        let short: String = id.chars().take(12).collect();
+        if label.trim().is_empty() || label.trim() == short || label.trim() == id {
+            format!("{short}{status}")
+        } else {
+            let label = console::truncate_str(&label, 100, "…");
+            format!("{short} {label}{status}")
+        }
+    } else {
+        let label = console::truncate_str(&label, 100, "…");
+        format!("{label}{status}")
+    }
+}
+
 fn one_line(text: &str) -> String {
     text.chars()
         .map(|c| if c.is_control() { ' ' } else { c })
@@ -229,6 +425,263 @@ fn escape_html(text: &str) -> String {
 mod tests {
     use super::*;
 
+    fn fixture() -> Graph {
+        use atomic_core::{kg::ontology::Predicate, KgEdge, KgNode};
+        let root = KgNode::new(
+            "urn:intent:root",
+            Class::WorkIntent,
+            "Make graph readable",
+            "test",
+        )
+        .with_metadata(serde_json::json!({"human_key": "ATOM::test::1", "status": "in_progress"}));
+        let change = |id: &str, label: &str| KgNode::new(id, Class::Change, label, "test");
+        Graph {
+            schema_version: 1,
+            root: root.id.clone(),
+            snapshot_identity: "snapshot".into(),
+            generation: 2,
+            ontology_version: "test".into(),
+            scope: "test",
+            nodes: vec![
+                root,
+                change("urn:change:ABCDEFGHIJKLM", "Implement renderer"),
+                change("urn:change:RECORD", "Record intent"),
+                change("urn:change:DEPEND", "Earlier dependency"),
+                KgNode::new("urn:task:1", Class::Task, "Add tests", "test")
+                    .with_metadata(serde_json::json!({"status": "open"})),
+                KgNode::new("urn:file:1", Class::File, "graph.rs", "test").with_metadata(
+                    serde_json::json!({"path": "atomic-cli/src/commands/intent/graph.rs"}),
+                ),
+            ],
+            edges: vec![KgEdge::new(
+                "urn:change:ABCDEFGHIJKLM",
+                "urn:intent:root",
+                Predicate::Implements,
+            )],
+            implementing_changes: vec!["urn:change:ABCDEFGHIJKLM".into()],
+            captured_changes: vec![],
+            declared_changes: vec!["urn:change:ABCDEFGHIJKLM".into()],
+            intent_artifact: None,
+            recording_changes: vec!["urn:change:RECORD".into()],
+            dependency_changes: vec!["urn:change:DEPEND".into()],
+            truncated: false,
+            missing_resources: vec![],
+            diagnostics: vec![],
+        }
+    }
+
+    #[test]
+    fn compact_tree_separates_roles_and_uses_readable_labels() {
+        assert_eq!(
+            render_text(&fixture()),
+            concat!(
+                "Intent ATOM::test::1 — Make graph readable [in_progress]\n",
+                "6 nodes · 1 implementing · 1 recording · 1 dependency-only\n",
+                "├── Implementing changes (1)\n",
+                "│   └── ABCDEFGHIJKL Implement renderer [declared; original turn unproven]\n",
+                "├── Intent recording evidence (not implementation) (1)\n",
+                "│   └── RECORD Record intent\n",
+                "├── Dependency-only changes (1)\n",
+                "│   └── DEPEND Earlier dependency\n",
+                "├── Tasks (1)\n",
+                "│   └── Add tests [open]\n",
+                "└── Files (1)\n",
+                "    └── atomic-cli/src/commands/intent/graph.rs\n",
+                "\nGrouped tree; use --details for full labels, canonical IDs and relationships.\n",
+            )
+        );
+    }
+
+    #[test]
+    fn change_hash_labels_are_shown_only_once() {
+        use atomic_core::KgNode;
+        let hash = "T5RE2LAMC244ABCDEFGHIJKLMNOPQRSTUVWXYZ";
+        for label in ["T5RE2LAMC244", hash, " T5RE2LAMC244 ", ""] {
+            let node = KgNode::new(format!("urn:change:{hash}"), Class::Change, label, "test")
+                .with_metadata(serde_json::json!({"status": "available"}));
+            assert_eq!(tree_label(&node), "T5RE2LAMC244 [available]");
+        }
+        let node = KgNode::new(
+            "urn:change:UNJWGUMUN3IV",
+            Class::Change,
+            "UNJWGUMUN3IV",
+            "test",
+        );
+        assert_eq!(tree_label(&node), "UNJWGUMUN3IV");
+        let node = KgNode::new(
+            format!("urn:change:{hash}"),
+            Class::Change,
+            "Fix rendering",
+            "test",
+        );
+        assert_eq!(tree_label(&node), "T5RE2LAMC244 Fix rendering");
+    }
+
+    #[test]
+    fn tree_order_is_stable_and_shared_nodes_appear_once() {
+        let mut graph = fixture();
+        graph.nodes.push(atomic_core::KgNode::new(
+            "urn:task:2",
+            Class::Task,
+            "A first task",
+            "test",
+        ));
+        let expected = render_text(&graph);
+        graph.nodes.reverse();
+        graph.edges.reverse();
+        graph.edges.extend(graph.edges.clone());
+        assert_eq!(render_text(&graph), expected);
+        assert_eq!(expected.matches("Add tests").count(), 1);
+        assert!(expected.find("A first task").unwrap() < expected.find("Add tests").unwrap());
+    }
+
+    #[test]
+    fn overlapping_change_roles_are_not_hidden_or_duplicated() {
+        let mut graph = fixture();
+        graph
+            .recording_changes
+            .push(graph.implementing_changes[0].clone());
+        let text = render_text(&graph);
+        assert_eq!(text.matches("Implement renderer").count(), 1);
+        assert!(text.contains("[declared; original turn unproven] [also records intent]"));
+        graph.captured_changes = graph.implementing_changes.clone();
+        let text = render_text(&graph);
+        assert!(!text.contains("original turn unproven"));
+        assert!(text.contains("[also records intent]"));
+    }
+
+    #[test]
+    fn tree_keeps_completeness_warnings_and_bounds_unicode_labels() {
+        let mut graph = fixture();
+        graph.truncated = true;
+        graph.missing_resources.push("urn:missing".into());
+        graph
+            .diagnostics
+            .push("Dependency closure incomplete".into());
+        graph.nodes[4].label = format!("Task\n\u{1b}{}", "界".repeat(200));
+        let text = render_text(&graph);
+        assert!(text.contains("[TRUNCATED]"));
+        assert!(text.contains("Missing resource: urn:missing"));
+        assert!(text.contains("Note: Dependency closure incomplete"));
+        assert!(text.contains("… [open]"));
+        assert!(!text.contains('\u{1b}'));
+        assert!(!text.contains(&"界".repeat(200)));
+        let details = render_details(&graph);
+        assert!(details.contains(&graph.nodes[1].id));
+        assert!(details.contains("atomic:implements → urn:intent:root"));
+        assert!(details.contains(&"界".repeat(200)));
+    }
+
+    #[test]
+    fn tree_handles_root_only_and_missing_root() {
+        let mut graph = fixture();
+        graph.nodes.truncate(1);
+        assert!(!render_text(&graph).contains("──"));
+        graph.nodes.clear();
+        assert!(render_text(&graph).starts_with("Intent [root missing]"));
+    }
+
+    #[test]
+    fn details_expand_the_same_tree_with_nested_evidence() {
+        let mut graph = fixture();
+        graph.nodes[1].summary = Some("Implement the compact renderer".into());
+        graph.edges[0].metadata =
+            Some(serde_json::json!({"declarationEvidence": "Explicit author statement"}));
+        let compact = render_text(&graph);
+        let details = render_details(&graph);
+        let skeleton: Vec<_> = compact
+            .lines()
+            .take_while(|line| !line.is_empty())
+            .collect();
+        let detailed_skeleton: Vec<_> = details
+            .lines()
+            .filter(|line| skeleton.contains(line))
+            .collect();
+        assert_eq!(detailed_skeleton, skeleton);
+        assert!(
+            details.contains("├── Snapshot: snapshot\n├── Scope: test\n├── ID: urn:intent:root\n")
+        );
+        assert!(details.contains("│       ├── ID: urn:change:ABCDEFGHIJKLM\n"));
+        assert!(details.contains("│       └── atomic:implements → urn:intent:root [Declaration evidence: Explicit author statement]\n"));
+        assert!(details
+            .contains("object=unknown, projection=reference-only Implement the compact renderer"));
+        assert!(details.contains(
+            "    └── atomic-cli/src/commands/intent/graph.rs\n        ├── ID: urn:file:1\n"
+        ));
+        assert!(details.contains("        └── Type: atomic:File\n"));
+        assert!(!details.contains("Summary: [context]"));
+        assert!(!details.contains("use --details"));
+    }
+
+    #[test]
+    fn context_summaries_require_actual_content() {
+        let graph = fixture();
+        let mut node = graph.nodes[5].clone();
+        for summary in [None, Some(""), Some(" \n\t ")] {
+            node.summary = summary.map(str::to_owned);
+            assert!(!node_details(&graph, &node)
+                .iter()
+                .any(|line| line.starts_with("Summary:")));
+        }
+        node.summary = Some("Shared file used by the renderer".into());
+        let lines = node_details(&graph, &node);
+        assert!(lines.contains(&"Summary: Shared file used by the renderer".to_owned()));
+        assert!(!lines.iter().any(|line| line.contains("[context]")));
+        // Meaningful change roles and availability evidence must remain.
+        let lines = node_details(&graph, &graph.nodes[2]);
+        assert!(lines.iter().any(|line| line.contains("[records intent artifact; implementation attribution not asserted] object=unknown, projection=reference-only")));
+    }
+
+    #[test]
+    fn details_keep_cycles_as_references_and_sort_edges() {
+        use atomic_core::{kg::ontology::Predicate, KgEdge};
+        let mut graph = fixture();
+        graph.edges.push(KgEdge::new(
+            "urn:intent:root",
+            "urn:change:ABCDEFGHIJKLM",
+            Predicate::DeclaresImplementation,
+        ));
+        graph.edges.push(KgEdge::new(
+            "urn:change:ABCDEFGHIJKLM",
+            "urn:file:1",
+            Predicate::ModifiesFile,
+        ));
+        let text = render_details(&graph);
+        assert_eq!(text.matches("ID: urn:change:ABCDEFGHIJKLM").count(), 1);
+        assert_eq!(text.matches("ID: urn:intent:root").count(), 1);
+        assert_eq!(text.matches(" → ").count(), 3);
+        graph.edges.reverse();
+        graph.nodes.reverse();
+        assert_eq!(render_details(&graph), text);
+    }
+
+    #[test]
+    fn root_only_details_close_the_tree_and_keep_warnings() {
+        let mut graph = fixture();
+        graph.nodes.truncate(1);
+        graph.edges.clear();
+        graph.truncated = true;
+        graph.missing_resources.push("urn:missing".into());
+        graph.diagnostics.push("Selection incomplete".into());
+        let text = render_details(&graph);
+        assert!(text.contains("└── Summary: [root intent]"));
+        assert!(text.contains("[TRUNCATED]"));
+        assert!(text.contains("Missing resource: urn:missing"));
+        assert!(text.contains("Note: Selection incomplete"));
+    }
+
+    #[test]
+    fn details_is_a_text_only_option() {
+        assert!(
+            IntentGraph::try_parse_from(["graph", "ATOM-1", "--details"])
+                .unwrap()
+                .details
+        );
+        for export in ["--json", "--dot", "--html"] {
+            assert!(IntentGraph::try_parse_from(["graph", "ATOM-1", "--details", export]).is_err());
+        }
+    }
+
     #[test]
     fn html_is_complete_before_browser_opens_and_file_url_is_encoded() {
         let directory = tempfile::tempdir().unwrap();
diff --git a/atomic-cli/src/commands/intent/mod.rs b/atomic-cli/src/commands/intent/mod.rs
index 6da0acf7..c69d5a31 100644
--- a/atomic-cli/src/commands/intent/mod.rs
+++ b/atomic-cli/src/commands/intent/mod.rs
@@ -72,10 +72,10 @@ pub enum IntentCommands {
     /// Validate an intent against the canonical shapes (the authoring gate).
     Validate(IntentValidate),
 
-    /// Show an intent as a rendered read-time projection.
+    /// Show an intent as a complete terminal tree (--json for canonical JSON-LD).
     Show(IntentShow),
 
-    /// Show an intent-rooted semantic change graph, including dependencies and evidence links.
+    /// Show an intent tree (--details expands evidence; --html opens a browser graph).
     Graph(IntentGraph),
 
     /// Scaffold a new directive-based intent into the vault.
diff --git a/atomic-cli/src/commands/intent/show.rs b/atomic-cli/src/commands/intent/show.rs
index 7979a2ba..dda3568e 100644
--- a/atomic-cli/src/commands/intent/show.rs
+++ b/atomic-cli/src/commands/intent/show.rs
@@ -8,7 +8,7 @@ use crate::commands::intent::bridge;
 use crate::commands::{find_repository_root, Command};
 use crate::error::{CliError, CliResult};
 
-/// Show an intent as a rendered read-time projection.
+/// Show an intent as a complete terminal tree.
 ///
 /// Projects the lifted canonical node (as opposed to dumping the raw stored
 /// body).
@@ -57,25 +57,6 @@ pub(crate) fn project(
         );
     } else {
         print!("{}", render(&node, Target::Cli));
-        // The renderer has no `kind` field of its own, so surface the
-        // classification (and, for a review, what it reviews) here.
-        println!("Kind:      {}", node.kind);
-        if node.kind == "review" {
-            let targets: Vec<&str> = node
-                .depends_on
-                .iter()
-                .filter(|r| r.edge == "reviews")
-                .map(|r| r.to.as_str())
-                .collect();
-            if targets.is_empty() {
-                println!("Reviews:   (none declared)");
-            } else {
-                println!("Reviews:");
-                for t in targets {
-                    println!("  • {t}");
-                }
-            }
-        }
     }
     Ok(())
 }
diff --git a/atomic-cli/tests/change_tree_test.rs b/atomic-cli/tests/change_tree_test.rs
new file mode 100644
index 00000000..48e83448
--- /dev/null
+++ b/atomic-cli/tests/change_tree_test.rs
@@ -0,0 +1,111 @@
+use atomic_core::{
+    change::{Change, ChangeHeader, EdgeUpdate, GraphOp, Insertion},
+    types::{Base32, ChangePosition, Hash, Position},
+    EdgeFlags,
+};
+use atomic_repository::Repository;
+use std::{path::Path, process::Command};
+
+fn insertion() -> Insertion> {
+    Insertion {
+        predecessors: vec![],
+        successors: vec![],
+        flag: EdgeFlags::FOLDER,
+        start: ChangePosition::new(0),
+        end: ChangePosition::new(0),
+        inode: Position::new(None, ChangePosition::new(0)),
+    }
+}
+
+fn run(root: &Path, hash: &str, flags: &[&str]) -> String {
+    let output = Command::new(env!("CARGO_BIN_EXE_atomic"))
+        .current_dir(root)
+        .args(["--no-color", "change", hash])
+        .args(flags)
+        .env("ATOMIC_SERVICE", "local")
+        .env_remove("ATOMIC_RPC")
+        .output()
+        .unwrap();
+    assert!(
+        output.status.success(),
+        "{}",
+        String::from_utf8_lossy(&output.stderr)
+    );
+    String::from_utf8(output.stdout).unwrap()
+}
+
+#[test]
+fn change_tree_expands_paths_without_emoji_and_preserves_other_formats() {
+    let directory = tempfile::tempdir().unwrap();
+    let root = directory.path();
+    let repo = Repository::init(root).unwrap();
+    let mut bytes = Vec::new();
+    let dependency = Change::empty(ChangeHeader::new("Initial setup"))
+        .serialize(&mut bytes)
+        .unwrap();
+    repo.import_change_bytes(&dependency, &bytes).unwrap();
+    let change = Change::new(
+        ChangeHeader::new("Build a CLI"),
+        vec![
+            GraphOp::DirAdd {
+                path: "src".into(),
+                add_name: insertion(),
+                add_inode: insertion(),
+            },
+            GraphOp::FileAdd {
+                path: "src/hello.ts".into(),
+                add_name: insertion(),
+                add_inode: insertion(),
+                contents: None,
+                encoding: None,
+            },
+            GraphOp::FileDel {
+                path: "legacy.ts".into(),
+                del: EdgeUpdate {
+                    edges: vec![],
+                    inode: Position::new(None, ChangePosition::new(0)),
+                },
+                contents: None,
+                encoding: None,
+            },
+        ],
+        vec![],
+        vec![dependency],
+    );
+    bytes.clear();
+    let hash = change.serialize(&mut bytes).unwrap();
+    repo.import_change_bytes(&hash, &bytes).unwrap();
+    drop(repo);
+    let database = root.join(".atomic/atomic.redb");
+    let before = std::fs::read(&database).unwrap();
+    let id = hash.to_base32();
+    let text = run(root, &id, &[]);
+    assert!(text.starts_with(&format!("change {}\n", &id[..12])));
+    assert!(text.contains("├── Metadata\n"));
+    assert!(text.contains("│   └── Build a CLI\n"));
+    assert!(text.contains("└── Files changed: 3\n    ├── legacy.ts [deleted]\n    └── src/ [added]\n        └── hello.ts [added]\n"));
+    assert!(!text.contains('📁'));
+    assert!(!text.contains("vertices: name"));
+    assert!(!text.contains("No provenance"));
+
+    let expanded = run(root, &id, &["--show-hunks", "--show-deps", "--full-hash"]);
+    assert!(expanded.starts_with(&format!("change {id}\n")));
+    assert!(expanded.contains(&format!("{} — Initial setup", dependency.to_base32())));
+    assert!(expanded.contains("        ├── +2 vertices: name, inode\n"));
+    assert!(expanded.contains("            └── +2 vertices: name, inode\n"));
+    assert_eq!(expanded.matches("hello.ts [added]").count(), 1);
+
+    let short = run(root, &id, &["-f", "short"]);
+    assert_eq!(short.lines().count(), 1);
+    assert!(short.contains("Build a CLI"));
+    assert!(!short.contains("──"));
+    let json: serde_json::Value = serde_json::from_str(&run(root, &id, &["-f", "json"])).unwrap();
+    assert_eq!(json["hash"], id);
+    assert_eq!(json["message"], "Build a CLI");
+    assert_eq!(json["hunks"].as_array().unwrap().len(), 3);
+    assert_eq!(
+        std::fs::read(database).unwrap(),
+        before,
+        "change display must be read-only"
+    );
+}
diff --git a/atomic-cli/tests/clone_sidecar_integration_test.rs b/atomic-cli/tests/clone_sidecar_integration_test.rs
index 99951c8b..87dc03a6 100644
--- a/atomic-cli/tests/clone_sidecar_integration_test.rs
+++ b/atomic-cli/tests/clone_sidecar_integration_test.rs
@@ -227,7 +227,7 @@ fn fresh_clone_ingests_provenance_and_attestations() {
     );
     let stdout = String::from_utf8_lossy(&out.stdout);
     assert!(
-        stdout.contains("=== Change Ledger ==="),
+        stdout.contains("└── Change Ledger\n"),
         "change view lacks the ledger — sidecars were not ingested:\n{stdout}"
     );
     assert!(
diff --git a/atomic-cli/tests/intent_graph_test.rs b/atomic-cli/tests/intent_graph_test.rs
index 596f93ad..07c16a93 100644
--- a/atomic-cli/tests/intent_graph_test.rs
+++ b/atomic-cli/tests/intent_graph_test.rs
@@ -2,7 +2,6 @@ use atomic_core::{
     change::{Change, ChangeHeader, ProvenanceGraph},
     kg::iri,
     pristine::VaultEntryType,
-    types::Base32,
 };
 use atomic_repository::Repository;
 use std::{
@@ -71,6 +70,20 @@ fn graph_resolves_ids_renders_all_formats_and_preserves_database() {
     let canonical = fixture(root);
     let database = root.join(".atomic/atomic.redb");
     let before = std::fs::read(&database).unwrap();
+    let shown = success(command(root, &["intent", "show", KEY]).output().unwrap());
+    assert!(shown.starts_with(&format!("Intent {KEY} — ")));
+    assert!(shown.contains("├── Metadata\n│   ├── Status: in_progress\n│   └── Kind: feature\n"));
+    assert!(shown.contains("├── Why\n│   └── Inspect exact lineage.\n"));
+    assert!(shown.ends_with("    └── Signed: no\n"));
+    assert_eq!(shown.matches("Kind: feature").count(), 1);
+    let json_show: serde_json::Value = serde_json::from_str(&success(
+        command(root, &["intent", "show", KEY, "--json"])
+            .output()
+            .unwrap(),
+    ))
+    .unwrap();
+    assert_eq!(json_show["status"], "in_progress");
+    assert!(json_show["@id"].as_str().unwrap().contains(UID));
     let mut expected = None;
     for id in [KEY, UID, canonical.as_str()] {
         let output = success(
@@ -119,8 +132,17 @@ fn graph_resolves_ids_renders_all_formats_and_preserves_database() {
         expected = Some(json);
     }
     let text = success(command(root, &["intent", "graph", KEY]).output().unwrap());
-    assert!(text.contains("implementing change"));
-    assert!(text.contains("atomic:generatedChange"));
+    assert!(text.contains("├── Implementing changes (1)"));
+    assert!(!text.contains("atomic:generatedChange"));
+    assert!(!text.contains("Summary: [context]"));
+    let details = success(
+        command(root, &["intent", "graph", KEY, "--details"])
+            .output()
+            .unwrap(),
+    );
+    assert!(details.contains("├── Implementing changes (1)"));
+    assert!(details.contains("atomic:generatedChange →"));
+    assert!(details.contains(&format!("ID: {canonical}")));
     let dot = success(
         command(root, &["intent", "graph", KEY, "--dot"])
             .output()
@@ -300,8 +322,14 @@ fn graph_distinguishes_explicit_declaration_from_captured_implementation() {
             .output()
             .unwrap(),
     );
-    assert!(text.contains("declared implementation; original turn provenance not established"));
-    assert!(text.contains("Declaration evidence: Author explicitly repairs"));
+    assert!(text.contains("[declared; original turn unproven]"));
+    let details = success(
+        command(directory.path(), &["intent", "graph", KEY, "--details"])
+            .output()
+            .unwrap(),
+    );
+    assert!(details.contains("declared implementation; original turn provenance not established"));
+    assert!(details.contains("Declaration evidence: Author explicitly repairs"));
 }
 
 /// The demo-2 failure: real, fully recorded source and intent artifacts exist,
@@ -309,6 +337,7 @@ fn graph_distinguishes_explicit_declaration_from_captured_implementation() {
 #[cfg(unix)]
 #[test]
 fn graph_exposes_real_recording_evidence_without_plan_id_or_database_mutation() {
+    use atomic_core::types::Base32;
     use std::fs;
     let directory = tempfile::tempdir().unwrap();
     let home = tempfile::tempdir().unwrap();
@@ -374,8 +403,10 @@ fn graph_exposes_real_recording_evidence_without_plan_id_or_database_mutation()
         .iter()
         .any(|n| n["metadata"]["path"] == "src/cli.ts"));
     let text = run(&["intent", "graph", KEY]);
-    assert!(text.contains("1 intent-artifact recording changes"));
-    assert!(text.contains("records intent artifact; implementation attribution not asserted"));
+    assert!(text.contains("0 implementing · 1 recording"));
+    assert!(text.contains("Intent recording evidence (not implementation) (1)"));
+    let details = run(&["intent", "graph", KEY, "--details"]);
+    assert!(details.contains("records intent artifact; implementation attribution not asserted"));
     assert!(!text.contains("No implementing changes are linked"));
     let dot = run(&["intent", "graph", KEY, "--dot"]);
     assert!(dot.contains("records intent artifact"));
@@ -409,6 +440,7 @@ async fn graph_has_local_reactor_parity_through_shared_read_endpoints() {
             .output()
             .unwrap(),
     );
+    let local_show = success(command(&root, &["intent", "show", KEY]).output().unwrap());
     // Keep the Unix socket short enough for sockaddr_un on macOS.
     let socket_dir = tempfile::Builder::new()
         .prefix("ig-")
@@ -434,12 +466,18 @@ async fn graph_has_local_reactor_parity_through_shared_read_endpoints() {
                 },
             ),
     );
-    let output = tokio::task::spawn_blocking(move || {
-        command(&root, &["intent", "graph", KEY, "--json"])
+    let (output, show_output) = tokio::task::spawn_blocking(move || {
+        let graph = command(&root, &["intent", "graph", KEY, "--json"])
             .env("ATOMIC_SERVICE", "reactor")
-            .env("ATOMIC_DAEMON_SOCKET", socket)
+            .env("ATOMIC_DAEMON_SOCKET", &socket)
             .output()
-            .unwrap()
+            .unwrap();
+        let show = command(&root, &["intent", "show", KEY])
+            .env("ATOMIC_SERVICE", "reactor")
+            .env("ATOMIC_DAEMON_SOCKET", &socket)
+            .output()
+            .unwrap();
+        (graph, show)
     })
     .await
     .unwrap();
@@ -450,4 +488,5 @@ async fn graph_has_local_reactor_parity_through_shared_read_endpoints() {
         serde_json::from_str::(&local).unwrap(),
         serde_json::from_str::(&reactor).unwrap()
     );
+    assert_eq!(success(show_output), local_show);
 }
diff --git a/atomic-cli/tests/routing_parity_test.rs b/atomic-cli/tests/routing_parity_test.rs
index 7ac2d3e4..107a1398 100644
--- a/atomic-cli/tests/routing_parity_test.rs
+++ b/atomic-cli/tests/routing_parity_test.rs
@@ -284,6 +284,27 @@ fn intent_show_json_projects_the_attested_node() {
         node.get("proof").is_some(),
         "the signed proof rides the node"
     );
+    let text = run(repo_dir, home_dir, &["intent", "show", &id]);
+    assert!(text.starts_with(&format!("Intent {id} — Projected intent\n")));
+    assert!(text.contains("├── Metadata\n"));
+    assert!(text.contains("└── Provenance\n"));
+    assert!(text.ends_with("    └── Signed: yes\n"));
+    assert_eq!(text.matches("Kind: feature").count(), 1);
+
+    // A changed spine must warn and render the current unsigned tree, never
+    // reuse the proof on the old content or append a second metadata footer.
+    run(
+        repo_dir,
+        home_dir,
+        &["intent", "update", &id, "--title", "Revised intent"],
+    );
+    let stale = atomic(repo_dir, home_dir, &["intent", "show", &id]);
+    assert!(stale.status.success());
+    assert!(String::from_utf8_lossy(&stale.stderr).contains("stale"));
+    let text = String::from_utf8(stale.stdout).unwrap();
+    assert!(text.starts_with(&format!("Intent {id} — Revised intent\n")));
+    assert!(text.ends_with("    └── Signed: no\n"));
+    assert_eq!(text.matches("Kind: feature").count(), 1);
 }
 
 #[test]
diff --git a/atomic-repository/src/workflow_usage.rs b/atomic-repository/src/workflow_usage.rs
index 9bc1f335..1508ddff 100644
--- a/atomic-repository/src/workflow_usage.rs
+++ b/atomic-repository/src/workflow_usage.rs
@@ -57,10 +57,12 @@ pub(crate) fn view_scopes(
                     if !scope.changes.insert(iri::mint_change(&hash)) {
                         continue;
                     }
-                    if !dependency_cache.contains_key(&hash) {
+                    if let std::collections::btree_map::Entry::Vacant(entry) =
+                        dependency_cache.entry(hash)
+                    {
                         match repo.load_change(&hash) {
                             Ok(change) => {
-                                dependency_cache.insert(hash, change.dependencies().to_vec());
+                                entry.insert(change.dependencies().to_vec());
                             }
                             Err(RepositoryError::ChangeNotFound { .. }) => {
                                 scope.error = Some(format!("View change {} is unavailable; dependency closure may be incomplete", hash.to_base32()));

From f689785949d076ecd559a4eb928a926334cf2365 Mon Sep 17 00:00:00 2001
From: Lee Faus 
Date: Sun, 25 Oct 2026 06:56:35 +0100
Subject: [PATCH 18/19] fix windows errors

---
 atomic-cli/src/commands/intent/graph.rs   | 5 ++++-
 atomic-repository/tests/workflow_usage.rs | 8 +++++++-
 2 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/atomic-cli/src/commands/intent/graph.rs b/atomic-cli/src/commands/intent/graph.rs
index 93eb94df..d8bc7f69 100644
--- a/atomic-cli/src/commands/intent/graph.rs
+++ b/atomic-cli/src/commands/intent/graph.rs
@@ -694,7 +694,10 @@ mod tests {
             assert_eq!(url.scheme(), "file");
             assert!(target.contains("%23") && target.contains("%25") && target.contains("%20"));
             assert_eq!(
-                url.to_file_path().unwrap(),
+                // Windows file URLs decode without the verbatim `\\?\`
+                // prefix returned by canonicalize. Compare canonical paths on
+                // both sides, not two valid spellings of the same file.
+                std::fs::canonicalize(url.to_file_path().unwrap()).unwrap(),
                 std::fs::canonicalize(&path).unwrap()
             );
             assert_eq!(std::fs::read_to_string(&path).unwrap(), html);
diff --git a/atomic-repository/tests/workflow_usage.rs b/atomic-repository/tests/workflow_usage.rs
index 0cea3307..dfe529c2 100644
--- a/atomic-repository/tests/workflow_usage.rs
+++ b/atomic-repository/tests/workflow_usage.rs
@@ -66,8 +66,13 @@ fn legacy_projected_changes_gain_usage_in_read_models_without_rebuilding_or_writ
         txn.update_view(¤t).unwrap();
         txn.commit().unwrap();
     }
+    // Windows denies raw reads of redb's locked file while a writable handle
+    // is alive. Close the writer before taking the byte-for-byte baseline,
+    // then exercise hydration through a genuinely read-only repository.
+    drop(repo);
     let database = root.join(".atomic/atomic.redb");
     let before = std::fs::read(&database).unwrap();
+    let repo = Repository::open_readonly(root).unwrap();
     let snapshot = repo.workflow_snapshot().unwrap();
     assert!(
         !snapshot
@@ -104,7 +109,6 @@ fn legacy_projected_changes_gain_usage_in_read_models_without_rebuilding_or_writ
         selected[0].metadata.as_ref().unwrap()["ai_usage"]["coverage"]["changes"],
         1
     );
-    assert_eq!(std::fs::read(database).unwrap(), before);
     assert!(
         repo.vault_kg_node(&id)
             .unwrap()
@@ -115,4 +119,6 @@ fn legacy_projected_changes_gain_usage_in_read_models_without_rebuilding_or_writ
             .is_none(),
         "reading must not backfill stored rows"
     );
+    drop(repo);
+    assert_eq!(std::fs::read(database).unwrap(), before);
 }

From c903362a7e87ed29878017e5cfd97921972505ec Mon Sep 17 00:00:00 2001
From: Bradley Hilton 
Date: Sun, 25 Oct 2026 09:35:50 +0100
Subject: [PATCH 19/19] chore: update KG docs and filter Markdown-only
 workflows

---
 .github/workflows/ci.yml      |   3 +
 .github/workflows/release.yml |   3 +
 README.md                     | 138 +++++++++++++++++++++++++---------
 3 files changed, 109 insertions(+), 35 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 7266b1a2..98866349 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -3,6 +3,9 @@ name: CI
 on:
   pull_request:
     branches: [dev]
+    paths-ignore:
+      - '**.md'
+      - '**.markdown'
 
 env:
   CARGO_TERM_COLOR: always
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
index ad3c5724..c6318375 100644
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -5,6 +5,9 @@ on:
   push:
     branches: [release]
     tags: ['v*']
+    paths-ignore:
+      - '**.md'
+      - '**.markdown'
   # Callable from the Prepare Release dispatch workflow, which bumps the
   # version, merges dev into release, and then invokes this build directly.
   # Invoking via workflow_call runs in the same workflow run, so it works
diff --git a/README.md b/README.md
index af55b426..b3793864 100644
--- a/README.md
+++ b/README.md
@@ -292,57 +292,123 @@ does not track `.vault/` keeps the vault private and ignored by Git.
 
 ### Knowledge Graph — Code Intelligence at Scale
 
-Atomic builds a **knowledge graph** from your entire codebase — source code content, file structure, tree-sitter entities (functions, classes, types), module hierarchy, change history, and their relationships. Search it, traverse it, visualize it, or let an LLM explore it with tools.
+Atomic's **workflow knowledge graph** connects intents, sessions, turns, provenance,
+attestations, changes, files, and views, alongside goals and memories. It is a
+deterministic, rebuildable read model: recorded changes, session ledgers, provenance
+sidecars, attestations, and vault objects remain authoritative.
+
+Workflow relationships use the canonical `atomic:` namespace
+(`https://atomic.dev/ns/workflow#`). For example:
+
+```text
+Session     ──atomic:hasTurn───────────▶ Turn
+Turn        ──atomic:hasPlan───────────▶ Intent
+Turn        ──atomic:generatedChange───▶ Change
+Turn        ──atomic:turnExplainedBy───▶ ProvenanceGraph
+Change      ──atomic:implements───────▶ Intent
+Change      ──atomic:changeExplainedBy─▶ ProvenanceGraph
+Change      ──atomic:attestedBy────────▶ Attestation
+Attestation ──atomic:coversChange──────▶ Change
+Change      ──atomic:modifiesFile──────▶ File
+Change      ──atomic:directlyOnView────▶ View
+```
 
-```bash
-# Search the knowledge graph — unified structural + content search
-atomic query search "replication"
+These links reflect captured facts, not guessed historical associations. Missing
+facts produce explicit gap findings. `atomic:directlyOnView` means a view's own
+change membership, not visibility inherited from ancestors or dependencies.
 
-# Search source code content (powered by syntext trigram index)
-atomic query code "replication" -t cpp -g src/mongo/db/repl/
+Resources have canonical IRIs: local workflow resources are qualified by the
+repository's stable ProjectId, while changes, provenance graphs, and session
+attestations use full, algorithm-qualified content hashes. JSON nodes carry
+asserted class IRIs in `types`, and edge `kind` values are absolute predicate IRIs.
+Copy resource IDs from search or graph JSON rather than constructing legacy
+`file:…`, `entity:…`, or abbreviated `change:…` IDs.
+
+```bash
+# Refresh workflow facts from recorded changes, files, and views
+atomic query enrich
 
-# List tree-sitter entities in a file (functions, classes, types)
-atomic query entities src/mongo/db/repl/replication_coordinator_impl.cpp
+# Search workflow resources, with indexed source-content matches
+atomic query search "replication"
 
-# Explore relationships around a node (1-2 hops)
-atomic query neighbors "module:src/mongo/db/repl"
+# Explore a resource's asserted relationships (1–2 hops)
+atomic query neighbors "" --json
 
-# Visualize the graph — opens interactive D3 force-directed layout
+# Visualize a search neighborhood in an interactive HTML graph
 atomic query graph "replication" -k 20 --depth 2
+
+# Inspect an intent's lineage and dependency closure, not a generic neighborhood
+atomic intent graph ATOM::alice::1 --html
 ```
 
-The KG contains seven node types connected by typed edges:
+Intent graphs distinguish captured and explicitly declared implementation from
+changes that merely record the intent artifact, and report truncation and missing
+resources. Change, intent, and view nodes also expose recorded AI usage summaries,
+with cost/token coverage and model attribution; missing usage is not assumed to be
+free. See [Intent semantic change graphs](docs/intent-graph.md) for scope and
+accounting details.
 
+**Code structure is queried live, not persisted in the workflow KG.** Tree-sitter
+provides entities and caller relationships on demand; module, call, definition,
+and import edges are not stored as a parallel code graph.
+
+```bash
+# Build/update the separate syntext source-content index
+atomic query index
+
+# Search indexed source text; -t filters language and -g filters paths
+atomic query code "replication" -t rs -g atomic-core/src/
+
+# Parse a file's functions, classes, and types with line ranges
+atomic query entities atomic-core/src/kg/iri.rs
+
+# Find callers by path-qualified symbol or bare name using live parsing
+atomic query callers "atomic-core/src/kg/iri.rs#mint_change" --json
 ```
-module:src/mongo/db/repl          ← directory-level grouping
-  ├─ PART_OF ← file:replication_coordinator.h    ← tracked files
-  │              ├─ DEFINES → entity:ReplicationCoordinator (class, L42-890)
-  │              └─ INCLUDES → file:replication_process.h
-  ├─ PART_OF ← file:bgsync.cpp
-  │              └─ MODIFIES ← change:R4YQUAS2 ("fix replication lag")
-  │                              ├─ AUTHORED_BY → identity:alice
-  │                              └─ DEPENDS_ON → change:XMJZ3IPF
-  └─ PART_OF → module:src/mongo/db  (parent module)
-```
 
-Search is powered by two indexes that work together:
-- **KG FTS** — searches node labels, summaries, and metadata (structural search)
-- **syntext** — trigram-indexed content search across all source files, ~20x faster than grep
+Search combines **KG full-text search** over resource IDs, labels, and summaries
+with **syntext** indexed source-content matches. The content index can be updated
+with `atomic query index`; it is separate from the workflow graph.
+
+**SHACL validation** runs locally in the CLI using PurRDF. Builtin profiles and
+enterprise-authored `.atomic/shapes/*.ttl` files use the same engine:
+
+```bash
+# Validate the asserted workflow snapshot (default projection profile)
+atomic query validate --format json
+
+# Apply the stricter managed-plan, provenance, and verified-attestation checks
+atomic query validate --profile publication --format json
+
+# Explicitly rebuild the complete derived workflow graph, including vault entries
+atomic query reindex --full
+```
 
-Results are ranked: `src/` files above tests, modules above flat files, content match counts as a signal. A bounded min-heap ensures the top N results surface from thousands of matches without sorting the full set.
+The projection profile permits a missing shortcut only with its matching gap
+finding. Reports include snapshot/profile identity, target coverage, and standard
+validation results. Conformance with no applicable targets is vacuous, not proof of
+complete history. Exit codes are **0** for conformance, **1** for violations, and
+**3** for shape or engine errors. Imports resolve only among explicitly loaded
+shape bundles, without implicit network fetching.
 
-For agents (Claude Code, Gemini, etc.), the CLI commands ARE the tools. An agent calls `atomic query search`, `atomic query neighbors`, `atomic query entities`, and its own `read_file` — the same path a human follows. No grep, no find, no LLM-calling-LLM overhead.
+Phase 1 does **not** enable OWL inference or mandatory validation hooks on sync or
+publication. Signature verification remains separate from SHACL. See the
+[KG upgrade guide](docs/knowledge-graph-phase1-upgrade.md) for the breaking API
+changes and migration behavior.
 
-For humans on the terminal, `ask` provides an agentic loop with tool use:
+Agents can use the same CLI queries directly. For interactive natural-language
+questions, `ask` provides an LLM tool-use loop:
 
 ```bash
-# LLM explores the KG with tools and answers your question
 atomic query ask "who fixed the auth bug?"
 ```
 
-The `ask` command gives the LLM six tools (`kg_search`, `kg_neighbors`, `read_file`, `list_entities`, `code_search`, `vault_read`) and lets it decide what to look up. Provider-agnostic — works with Anthropic or OpenAI. Set `ANTHROPIC_API_KEY` or `OPENAI_API_KEY` to enable.
+The model can search workflow facts, inspect source files, parse entities, and read
+vault entries. Set `ANTHROPIC_API_KEY` or `OPENAI_API_KEY` to enable it. Answers are
+model-generated interpretations, not machine-validated historical audit evidence.
 
-No API key required for `search`, `code`, `entities`, `neighbors`, `graph`, or `enrich`.
+No LLM API key is required for `search`, `code`, `entities`, `callers`, `neighbors`,
+`graph`, `enrich`, `index`, `reindex`, or `validate`.
 
 ## Quick Start
 
@@ -409,12 +475,14 @@ Every change stores two parallel representations:
 | `atomic query search ` | Search the knowledge graph (structural + content) |
 | `atomic query code ` | Search source code content (trigram-indexed, `-t` for file type, `-g` for path filter) |
 | `atomic query entities ` | List tree-sitter entities in a file (functions, classes, types with line ranges) |
-| `atomic query neighbors ` | Explore the graph around a node (DEFINES, MODIFIES, PART_OF, INCLUDES edges) |
-| `atomic query graph ` | Visualize the knowledge graph as interactive HTML (D3 force-directed, `-k` seeds, `--depth`, `--kinds`) |
+| `atomic query callers ` | Find callers using live source parsing, without stored KG code edges |
+| `atomic query neighbors ` | Explore asserted workflow relationships around a canonical resource |
+| `atomic query graph ` | Visualize the workflow graph as interactive HTML (`-k` seeds, `--depth`, `--kinds` with class IRIs) |
 | `atomic query ask ` | Natural-language Q&A with agentic tool loop (requires API key) |
-| `atomic query enrich` | Build/rebuild KG from VCS data + content index (modules, entities, includes) |
+| `atomic query enrich` | Refresh VCS workflow facts (changes, files, views) |
 | `atomic query index` | Build/update the content search index (`--rebuild` for full, `--stats` for info) |
-| `atomic query reindex` | Rebuild the KG index from vault entries |
+| `atomic query reindex [--full]` | Reproject vault entries; `--full` rebuilds the complete derived workflow graph |
+| `atomic query validate` | Run builtin and local enterprise SHACL shapes (`--profile projection\|publication`, `--format json`) |
 
 ### View Commands