Skip to content

feat: user-defined vault kinds (reference implementation for #276) - #278

Draft
Zaba505 wants to merge 36 commits into
atomicdotdev:devfrom
Zaba505:feat/user-defined-vault-kinds
Draft

Zaba505 wants to merge 36 commits into
atomicdotdev:devfrom
Zaba505:feat/user-defined-vault-kinds

Conversation

@Zaba505

@Zaba505 Zaba505 commented Oct 22, 2026 •

Copy link
Copy Markdown

Draft / reference implementation for discussion #276, user-defined vault kinds. It covers tier A, milestones M0–M2, with Revision 1 (schema in YAML frontmatter, read by a real YAML 1.2 parser) and Revision 2 (the KG half redesigned after #265; evidence and errata).

It's a draft so the proposal can be evaluated against working code; it isn't asking to be merged as is. Every open decision takes the proposal's default (see Assumed decisions). If a ruling goes the other way, the affected code is listed there.

What changed since the previous description

Rebased onto dev 8053c02, which adds #319, #322, #325 and #332. There were no conflicts, and git range-diff shows all 36 patches unchanged, so only the commit SHAs changed: the previous tip e8ee156 is now 094334e. Every SHA in this description is the rebased one, apart from the pre-#265 commits and the dev commits that earlier runs were made against. The checks in Testing were re-run on 094334e. The cross-version runs in Compatibility were not, and still name the 5c31fae binary they used.

The rebuild after #265

History was rewritten. The old tip was 16e6b33, on d968586. This branch was rebuilt on dev at 5c31fae, with 36 commits. #265 closed the KG vocabulary and removed what the old KG commits used (KgNode.kind, free-string edge kinds, entry_subject, delete_intent_child_nodes), so those commits do not compile against dev and could not be kept. Of the 19 old commits:

Old commits What happened
12 Picked, with their messages unchanged apart from a cherry-pick line
1 (7b5c539) Picked, with one fixture literal amended to keep the vocabulary lint green (a5b1ee1)
5 (9bc650a, 8bf2c9b, 4361e76, fda76b8, 2ee7188) Rewritten: d9ca76f, 425ab6a, fbddd5e, 4804469, 285dacf. Each message names its source and what it left out
1 (4cdfde7) Dropped. Dev's whole-vault re-resolve replaces its mechanism. Its behaviour tests are ported in 6c951b6

Then 6 new commits adapt to #265, and 12 more come from a review pass against the real binary.

Two things to know when reading the history:

  • 01deb36 and 089c609 keep their old messages, which describe reservations that d697cc6 replaces.
  • instance_change_depends_on_definition_change is red from 88b3fe7 until d697cc6, on a stale literal from feat: Phase 1 knowledge graph closure and SHACL validation #265's renderer. Kinds do not project into the KG until 6c951b6.

The KG half is new. Instances and definitions are now atomic:Concept nodes, custom relations are atomic:references with labels in edge metadata, and conformance is an atomic:Finding. atomic-core, atomic-shacl and libatomic have no diff. The costs of this are in Revision 2: an ADR's type does not say "ADR", custom values do not reach RDF, and the validator is hand-coded against the merged plan's direction.

Withdrawn from the previous description:

  • <k>:<ULID> nodes, <k>.<EDGE> edges and kind:<k> nodes.
  • query neighbors adr:<ULID> as a working query. It finds nothing and exits 0.
  • "A closed construct list maps 1:1 to SHACL-Core", and the mapping in kind show.
  • "Every value the CLI prints from repository data is sanitized", which was too broad.
  • "A custom edge: HAS_TASK can't delete an intent's children". The mechanism is gone, and HAS_TASK no longer passes the edge grammar.
  • The 51-name reserved list, and kind, conforms, vault_path, violation_codes and urn as reserved field names.
  • "As memory does". memory tries RPC first and kind does not.
  • "Data is compatible both ways", which is now narrower. See Compatibility.
  • 10,876 tests and the 10 git_shim_cb13d failures. Those counts were on d968586.

What it does

A repository defines a kind in .vault/kinds/<k>/_kind.md. The frontmatter is the schema and the body is guidance prose. Instances are stored as .vault/kinds/<k>/<ULID>.md. Atomic then:

  • validates instances fail-closed, with atomic kind validate [<k>] [--view V] [--json], which exits 2 on violations;
  • projects them into the KG as atomic:Concept nodes at urn:atomic:project:<pid>:concept:<k>.<ULID>, each with an atomic:references edge to a hub at concept:kind.<k>, and with declared fields under metadata.field.<name>;
  • gives a non-conforming instance one atomic:Finding with code KindViolation, and an invalid definition one with KindDefinitionInvalid;
  • authors instances with atomic kind new, which applies defaults at write time and runs the gate before writing;
  • exposes guidance through atomic kind list/show --guidance, rendered as delimited, length-capped data.

There is no new storage, transport, redb schema or proto change, and the ontology gains no terms. Rows stay VaultEntryType::Scratch.

Run on this branch, with urn:atomic:project:<pid>: shown as …, and ... for lines left out:

$ atomic kind validate adr
✗ .vault/kinds/adr/_kind.md: fields.status: unknown key "pattern"
  (adr stays unregistered until this is fixed)
...
$ atomic kind new adr --title "Sign changes with Ed25519" --set status=maybe
✗ status: "maybe" is not one of: proposed, accepted, superseded
Nothing was written.
...
$ atomic kind new adr --title "Sign changes with Ed25519" --set intent=1 --body-file d1.md
Created .vault/kinds/adr/01M6PW2H50KT41W46Y1G37FVQ5.md  (adr:01M6PW2H50KT41W46Y1G37FVQ5)
  KG id: …concept:adr.01M6PW2H50KT41W46Y1G37FVQ5
$ atomic query neighbors …concept:adr.01M6PW2HH3GHBW1TX7BQ8SFP3K
...
Edges (2):
  …concept:adr.01M6PW2HH3GHBW1TX7BQ8SFP3K →[https://atomic.dev/ns/workflow#references supersedes]→ …concept:adr.01M6PW2HE6VNDFMRCZYHHF95XT
  …concept:adr.01M6PW2HH3GHBW1TX7BQ8SFP3K →[https://atomic.dev/ns/workflow#references @kind]→ …concept:kind.adr

Revision 2, part 2 has the full walkthrough.

Commits, by layer

Layer Commits Notes
Spec language a5b1ee1, 8f742f7, 01deb36, 089c609, d697cc6 atomic-canonical/src/kind_spec.rs. Pure: no repository or YAML dependency. d697cc6 isolates the #265 delta here: identifiers removed, edge: becomes a label (^[a-z][a-zA-Z0-9_]*$), reserved lists rebuilt.
YAML parser (Rev. 1) 22a31ed, 35a9545 saphyr-parser event API with the YAML 1.2 core schema. Rejects anchors, aliases, tags, <<, duplicate keys and multi-doc, and caps size and depth. kinds/** parses strict-only. Everything else runs both parsers and keeps the legacy result if they disagree (one-release fallback).
Registry and validator (M1) d9ca76f, 890c850, fbddd5e, 80e94bf, 41614fe, efd5591 Resolved from files, from the working copy or a named view, and never from redb. View registries are cached by the view's Merkle folded with its ancestors'.
CLI and docs (M2) abea88d, 1ac0087, e7bef49, dc1c34f, 62f2a0d, 0c1e033, ccf9e65, 2c6c1fb, 9bee96c, 5be81eb, 263985e list/show/init/new/validate, plus a "Custom kinds" section in the vault skill. Reads open read-only and writes create the vault on first write.
Embed policy and materialize 425ab6a, 4804469 Embed policy applies in both embed paths, with vault_reembed_changed_kinds for a definition change. materialize_all skips kinds/**, and single-path materialize refuses it.
KG projection (M0) 6c951b6, 6c5dd8e, c6fd368, d292265 One Scratch arm in vault_projection.rs. vault_triples.rs and kg_rebuild.rs have no diff. extract_kind_entry returns no Result, so repository content cannot fail writable open.
Query printers 59c5c4a, with 0c1e033 Labels from metadata.edges on query neighbors, and terminal-safe node text in query search, neighbors and plan, for every row type.
Tests 88b3fe7, 0584d2e, 285dacf, 6ff12f6, 094334e End-to-end across views, the move into the it binary, atomic query, the SHACL mapping, and the hub's last-deflate behaviour.

Hardening from review

Carried over, and re-run on this base:

  • Fail-open ref: a ref to an instance of an unregistered or invalid kind used to conform. It now gives UNRESOLVED_REF, in the gate and in the KG.
  • Identity-dependent intent refs: stored refs accept only a full ULID or an exact full human key. kind new rewrites what the user typed into the canonical form.
  • Projection source: projection, the hub and the embed policy read the stored kinds/<k>/_kind.md entry, not the file on disk. The per-view gate still resolves from files.
  • Cross-kind edges: the old re-index hooks are gone. Dev re-resolves the whole vault on every store, so an rfc → adr edge survives adr's spec going invalid and coming back with no hook. The behaviour tests were ported.
  • Terminal and prompt hygiene: spec strings are rejected if they contain Unicode Cc/Cf characters or exceed 1024 characters. kind commands print repository data sanitized. JSON stays raw.
  • Also: unreadable instances are reported as UNREADABLE, the dual-run parser warning is deduplicated per file and content hash, and written is on every kind new result.

New in this pass, each with a test that failed before the fix except where noted:

  • Deflate followed symlinks under kinds/ (6c5dd8e, major). A link's target became a vault row and, with a uid, a KG node, so a file outside the repository could reach the index and, by reading, with embed: true the embedding provider. A link to /dev/zero or a FIFO never finished. Under kinds/ the scan now skips anything that is not a regular file. One non-UTF-8 or unreadable file there no longer stops every other vault file from deflating. Scope: kinds/ only. Elsewhere the scan behaves as on dev.
  • A linked or unreadable definition (41614fe). A symlinked _kind.md with no instances disappeared and kind validate exited 0. One with no read permission failed every kind command. Both now make that one kind invalid, with the reason.
  • Repeated links (c6fd368). A 1.2MB instance with 200,000 [[x]] made every later vault write slow. Debug build: deflate 103s to 2.1s, a later unrelated store 103s to 2.2s, a full rebuild 186s to 3.7s. The "before" figures are from a review run. Measured before and after on one machine at 20,000 links, those went from 11.0s, 10.9s and 19.9s to 0.27s, 0.28s and 0.46s. Kind rows only. A built-in row with the same body keeps dev's cost.
  • Gate weaker than the KG (efd5591). A ref to a file with no uid, no parseable frontmatter or a conflict marker passed kind validate and kind new --set, while the KG marked it UNRESOLVED_REF. The gate now applies the projection's rule.
  • Label collision (6c951b6). labels: [kind.adr] mints the hub's exact IRI. An instance going inert while an intents/ row labels it lost both node and edge incrementally, while a rebuild kept them. An inert row now owns a bare node while a label names it. With the fix disabled at 6c951b6, 2 of the 28 kg_kinds tests then in the file failed. A label spelling a kind node still links to it: see decision 4.
  • Query printers (59c5c4a). query search, neighbors and plan printed raw terminal escapes and injected lines from node text.
  • Refused writes (ccf9e65). A rejected kind new, or kind init over an existing file, initialized a vault and then said Nothing was written. Both now check first. A writable open still happens first and can touch .atomic/, as it does for every writing command.
  • Smaller:
    • a blank spec title gave the hub an empty label (d292265);
    • kind new printed only a handle that looks queryable and is not (2c6c1fb);
    • kind list <k> aborted on one unreadable instance (9bee96c);
    • vault materialize -p kinds/... showed its refusal as an internal error (5be81eb, now exit 2);
    • the skill doc's --set intent=PIMO-12 did not resolve (263985e).
  • Pinned with no code change (094334e). A characterization test: the hub answers for the last deflate, not for every view.

Retired on purpose: field-name reservation for kind, conforms, vault_path, violation_codes and urn. The field. prefix makes them unnecessary, and they are legal names now.

What the end-to-end tests established

These are the proposal's patch-theory claims, run against the real binary (atomic-cli/tests/it/kind_view_test.rs):

  • Instance → definition dependency: confirmed. An instance change depends on the change that created kinds/<k>/.
  • Insert carries the definition: confirmed, and only the definition. Inserting only the instance into a sibling view brings the definition along, and kind validate --view is ok there. New here: a ref adds no patch dependency, so inserting an ADR that supersedes another leaves the target behind, and kind validate --view exits 2 with does not resolve to an existing adr.
  • Unrecord guard: confirmed, via Repository::unrecord's view-closure check, not UnrecordError::HasDependents. The CLI shows it as an "Internal error … appears to be a bug" with exit 128. That handling predates this branch.
  • Two views adding the same _kind.md: name conflict, not merge. Each view creates its own kinds/, kinds/adr/ and _kind.md, so after an insert all three are name conflicts. kind validate fails closed in the working copy and with --view.
  • The registry is per view: confirmed.

And for the KG (atomic-repository/tests/it/kg_kinds.rs, atomic-cli/tests/it/kind_query_test.rs):

  • No repository content fails projection or writable open. no_repository_content_can_fail_projection_or_writable_open runs malformed rows, invalid specs, dangling refs, cycles and unparsable stored rows through a writable open with an expired generation.
  • Incremental projection matches a rebuild for kinds' own life cycle. It does not hold for every operation in a repository that has kinds. On dev, removing a label, an assignee or a file mention from a built-in row leaves an orphan node until a rebuild (run on an export of 5c31fae). Kinds do not project file mentions, to stay out of that class.
  • A repository with no kinds is unchanged. a_repository_with_no_kinds_projects_as_it_did_before_kinds pins a digest computed by running the same fixture on dev.
  • query validate is unaffected. It exits 0 on broken definitions, non-conforming instances, label collisions, and fields named after every key the adapter reads. With the field. prefix removed, it exits 3, so that test detects a real leak.
  • The KG is last-deflate-wins. A record on a sibling view empties the hub, and switching back does not restore it until a deflate. kind validate --view still answers for the view.

And against PurRDF (atomic-cli/tests/it/kind_shacl_mapping_test.rs, 9 tests, purrdf-shapes 3.0.1): the spec language is not 1:1 with SHACL-Core. Most constructs pair a code with a component. Five rows have no triple at all. sh:defaultValue masks sh:minCount. A list default cannot be written as sh:defaultValue. The shape is a hand translation and nothing compiles a spec to a shape. Revision 2, part 2 has the table.

Timing. Whole-vault re-projection is fine at a few hundred vault rows and a real problem from about 1,000. Memory rows show the same shape at about half the cost, so this is dev's pipeline, not kinds. AMD Ryzen 9 5950X, tmpfs, single-threaded, one run per size, release with LTO off and 16 codegen units (not the shipping profile):

ADR rows in the vault Store one row Whole-vault pass rebuild_kg Hub neighbours
100 64ms 45ms 17ms 0.5ms
1,000 3.5s 2.1s 127ms 4.4ms
5,000 86s 56s 674ms 25ms

A first deflate of 100 ADRs took 2.6s. A first deflate of 1,000 did not finish in 15 minutes (at 6ff12f6), while 1,000 memory rows took 622s. About 45 hours for 5,000 ADRs is an estimate from a fit, not a measurement. On NVMe with btrfs, a store at 1,000 ADR rows took 4.0s and 3.7s against 3.6s and 3.9s on tmpfs, and a 100-row deflate took 4.2s against 2.3s (both at 6ff12f6). The probes are kg_kinds::scale_probe and kg_kinds::scale_probe_deflate, both #[ignore].

Assumed decisions

The numbers are Revision 2's list.

# Default taken Where it lives if overruled
1 No new ontology terms: atomic:Concept, atomic:references to a hub, metadata.kind_name. Closed per view relative to registered specs vault_projection.rs (Scratch arm, extract_kind_entry). A real class also touches atomic-core/src/kg/, the TTL and atomic-shacl/src/adapter.rs
2 A small data-driven field validator runs in the binary, for custom kinds only, pinned against PurRDF kind_spec.rs (validate_instance), kinds/validate.rs. Compiling to shapes makes validate_instance a test oracle and removes KindViolation
3 Registry per view. KG repository-wide and last-deflate-wins kinds/mod.rs (KindSource), or QueryView::project_node
4 concept:<k>.<ULID> and concept:kind.<k>. kind as the word. Reserved rule, 33 names kind_hub_name, kind_instance_name, ProjectionRefs, RESERVED_KIND_NAMES, atomic-cli/src/commands/kind/. A :kind: segment adds a mint and parse pair in iri.rs and removes inert_kind_row
5 Not exercised by M0–M2 M3 scope
6 Built-in value sets stay binary-defined untouched
7 Not a prerequisite schedule only
8 YAML 1.2 via saphyr-parser; one-release legacy fallback outside kinds/**; lift::parse_markdown out of scope vault_frontmatter.rs (the legacy parser is marked TODO: remove)
9 atomic:references with fields and edges in edge metadata; edge: is a label, lowerCamel by convention extract_kind_entry, merge_kind_edges, the printers in query/mod.rs and rpc.rs, kind_query_test
10 atomic:Finding with KindViolation and KindDefinitionInvalid; no conforms key extract_kind_entry
11 No wire parity: kind opens the repository directly and stays out of the remote-sandbox allowlist atomic-cli/src/commands/kind/, plus new proto arms and handlers
12 Whole-vault re-projection accepted for tier A up to a few hundred vault rows; pipeline unchanged vault_auto_index, project_vault_kg
13 Tier-B identity deferred no code in M0–M2

Compatibility with existing repositories

Tested with a real 5c31fae binary against this branch's binary, both in throwaway repositories. Dev created the repository: an attested intent, an attested memory, and a hand-written memory with YAML-ambiguous forms (title: a: b, empty:, hex: 0x1F, note: x # y, a block list), all recorded.

  • No kinds: dev → branch → dev holds. status, vault list, vault show, intent list, intent verify and memory verify give byte-identical output on the branch, and query validate gives the same conforms, snapshot_identity, profile_identity and engine. After branch writes the same comparisons stay identical except status and the file tree, and dev reopening matches byte for byte and then syncs and records normally.
  • Substance hashes hold. Both attestations verify under both binaries at every stage. The attestation files and .atomic/canonical sidecars are byte-identical at the end of the run. atomic-core has no diff, so there is no redb schema change and no new VaultEntryType variant.
  • A new stderr warning. It is printed once per command, during vault deflate (record, vault sync), for each hand-written vault file that YAML 1.2 rejects or reads differently, e.g. title: a: b. The legacy reading is still what gets stored.
  • vault materialize can rewrite bytes for some values. The difference from dev's output is quoting only: empty: "", hex: "0x1F", list: "". Stored values are identical, but such an entry shows as modified after re-materializing.
  • Unchanged from dev: outside kinds/**, the legacy parser still loses YAML block lists during the one-release fallback.
  • Dev on a repository that has kinds is not clean. "Compatible both ways" does not hold here.
    • Dev still reads kinds and gives the same snapshot_identity.
    • Any dev vault store removes every kind node, and query validate still conforms. Kinds stay absent until the branch's next vault store or query reindex --full.
    • Dev's deflate flattens a nested _kind.md. A branch query reindex --full then shows the hub invalid until a branch deflate repairs it.
    • Dev's vault materialize writes kinds/. After a branch deflate it writes a JSON-flow fields: that still validates. After dev's own deflate it writes the flat form, kind validate fails with unknown key, and instances report KIND_INVALID.
    • atomic restore recovers a recorded definition. An unrecorded definition overwritten this way cannot be recovered.
  • Below feat: Phase 1 knowledge graph closure and SHACL validation #265. A 0.20.0 binary (d968586) is refused on a dev-created repository with repository requires unsupported capabilities: 'kg-semweb-v1', by feat: Phase 1 knowledge graph closure and SHACL validation #265's fence. Starting from a 0.20.0-created repository, dev and this branch behave the same at each step, including a first status failing with Table 'pending_untracks' does not exist, which a later write clears.

Known limitations / follow-ups

  • Scale. See Timing and decision 12. The cost lands on every vault write in a repository, not only kind writes.
  • A ref is not a patch dependency. Insert closure does not carry an ADR's ref targets.
  • A label spelling a kind node links to it, with no Finding. An intent labelled kind.adr appears among the hub's neighbours through hasLabel.
  • An unrecorded instance blocks view switch to a view without that kind: exit 128, reported as an internal error. Plain dev does the same for any tracked directory holding an untracked child (run on both), so this is not specific to kinds, but kind new makes it easy to hit because it writes the file without recording it.
  • Selecting one kind or one relation is client side. -t and --kinds cannot separate adr from rfc. A filter plan only counts: its "input" takes the bare binding name, "$adrs" silently returns 0, and nodes keeps every ADR and the hub. read_content on the binding returns nothing.
  • Search scope. Titles, uids and kind names only. A word only in a body or field value returns nothing, as for memory bodies. test and ab are legal kind names that search cannot find by kind name.
  • Outside kinds/**, the working-copy scan is dev's: it follows symlinks and stops on the first unreadable .md. Run on this branch: one .md with no read permission under .vault/memory/ makes vault sync exit 128 with nothing synced, and with a link there to /dev/zero, vault sync had not finished after 25 seconds.
  • Sanitization is scoped to kind commands plus query search, neighbors and plan. Other printers of KG text, such as query ask, are not sanitized.
  • kg_id can name a node that does not exist yet. kind new validates against the file but stores only the instance row, so with a hand-written _kind.md that was never synced, query neighbors on the printed kg_id finds nothing until the next deflate (run).
  • Reads on a checkout with no working_copy_id exit non-zero with requires writable migration, as other read-only commands do. They used to migrate it silently. A legacy pristine.redb plus changes.redb layout is merged on first read, which is dev's behaviour for every read-only command.
  • A rejected kind new still does a writable open, which can touch .atomic/.
  • Intent refs resolve through redb even for --view (KindRefResolver). This is part of decision 7.
  • Gate and projection differ on intent refs. The gate is strict and projection uses case-folded aliases, so UNRESOLVED_REF can appear with no KindViolation. A test pins this.
  • Bodies that document conflict markers fail closed. A column-0 <<<<<<< inside a code fence, for example, can't conform.
  • Agent context: context: true is recorded on the hub, but nothing consumes it. By reading, instances are not Memory, so they do not enter context.
  • Guidance author is always unknown, because there's no per-path history API.
  • Legacy parser warnings: hand-written built-in files that use forms YAML reads differently log a dual-run warning until they're rewritten.
  • No daemon/proto arm for kind (M4). A remote sandbox cannot use it. query works through the existing service path.
  • M3 isn't included: extra_dependencies frontier pinning, widen-only/DEFINITION_NARROWED, kind migrate and the body conflict-marker deflate guard. The orphan sweep is dropped: GC is owner-based, and query reindex --full rebuilds.

Testing

172 tests added and none removed from dev: 170 in the normal suite and 2 #[ignore] timing probes. Size: 41 files, +13,990 / -256. Of that, 5,310 added lines are outside tests and 8,680 are in tests (inline test modules counted by a heuristic).

Tip 094334e, on dev 8053c02, Linux. The whole suite: 11,094 passed, 0 failed, 19 ignored. Dev was not re-measured at 8053c02. At 5c31fae, dev ran 10,753 tests with 17 ignored, and the branch ran 10,923 with 19 ignored.

All 21 git_shim_cb13d tests pass, so the old note about 10 failures does not apply in this environment.

Check Result
cargo fmt --all -- --check pass
cargo clippy --locked --workspace -- -D warnings, with RUSTFLAGS=-Dwarnings pass
cargo clippy --locked --no-deps -p atomic-canonical -p atomic-cli --features dsse-export -- -D warnings pass
cargo clippy --locked --all-targets -p atomic-canonical -p atomic-repository -- -D warnings pass
node --test libatomic/scripts/package-runtime.test.mjs 11 passed
cargo nextest run --locked --workspace --features atomic-cli/adoption-test-injection --profile ci, except diagnostics 11,054 passed, 0 failed
Same, diagnostics_integration_test only 40 passed, 0 failed
Doctests 489 passed, 241 ignored
Harness 09 (55 passed), harness 46 pass
RUSTDOCFLAGS=-Dwarnings cargo doc --locked --workspace --no-deps pass
Conformance vectors, at the commit CI pins (e7508ce, v0.25.0) 1 passed
Export consumer, API and real CLI, with cryptography==50.0.0 221 passed, 1 ignored; 6 passed

Before this rebase, each of the 12 newest commits passed fmt, clippy on the three crates, --all-targets clippy on atomic-canonical and atomic-repository, and the vocabulary lint. That per-commit pass was not repeated on the rebased commits. The tip passes all of them, and kg_vocabulary_lint runs inside the nextest suite.

Environment differences from CI: 8 test threads rather than 2 per CPU; diagnostics run separately at 2 threads with TMPDIR on disk, because each of those tests copies the roughly 2G debug binary; the line-tables-only debug settings not applied; Fedora rather than ubuntu-latest; Python 3.14 rather than 3.12.

What was not run

  • macOS and Windows. This PR shows no CI checks. The new symlink, FIFO and permission tests are Unix-only, and the regular-file check was not exercised on Windows.
  • Pull, clone or push across versions, a daemon (ATOMIC_SERVICE=reactor), and a branch-created repository first opened by dev.
  • repo identity adopt. repo identity fork was run on a hostile corpus, and every hub neighbour's IRI used the new id. adopt was only read.
  • kind itself on a legacy repository. read_only_open_merges_a_legacy_repository covers the opener. The redb format-upgrade case is from reading only.
  • A case-insensitive filesystem.
  • The shipping release profile for timing (LTO, 1 codegen unit, panic=abort). The 1,000 and 5,000-row deflate probes and the 5,000 memory-row probe were not repeated at the tip; those numbers are from 6ff12f6.
  • cargo clippy --all-targets on atomic-cli. CI does not run it. At 6ff12f6 it failed under -Dwarnings on two lints in files this branch does not change (vault_auto_init_test.rs, real_import_sample.rs). Its test targets compiled under -Dwarnings through nextest at the tip.
  • atomic-cli test targets at intermediate commits. Only library and binary targets were gated at each of the first 17 commits, so an intermediate commit could have a CLI test-only build problem.
  • Built-in intent and memory ref targets through the SHACL mapping table.
  • Engine cost per kind new under decision 2(b). Not measured.
  • Release workflows. They do not run on a pull request.

🤖 Generated with Claude Code

@geekgonecrazy

Copy link
Copy Markdown
Contributor

@Zaba505 as mentioned in #276 take a look at #265 this definitely changes things a bit

@Zaba505

Zaba505 commented Oct 27, 2026

Copy link
Copy Markdown
Author

@geekgonecrazy yeh I'm planning to rebase this later today and go back through it to see how it needs to change

@Zaba505
Zaba505 force-pushed the feat/user-defined-vault-kinds branch from 16e6b33 to e8ee156 Compare November 6, 2026 03:13
Zaba505 and others added 27 commits November 11, 2026 17:01
Add `kind_spec`, the open-world extension tier beside the closed `vocab`
registry. A kind's definition frontmatter (`kinds/<k>/_kind.md`) is parsed
into a `KindSpec` with every problem collected and path-qualified, and
instances are validated against it with stable violation codes, ref
resolution through a caller-supplied `RefResolver`, and namespaced edges
(`<kind>.<EDGE>`) for projection. Also adds write-time helpers
(`apply_defaults`, `parse_set_value`) and a SHACL-Core summary for display.

Picked from 7b5c539. The self-reference test's edge fixture changes from
`CHILD_OF`, a term atomicdotdev#265 retired, to `PARENT_OF_ADR`, so the KG vocabulary
lint stays green.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A `=======` line now counts as a conflict marker only when the text also
has an opening or closing marker, so a setext heading underlined with
seven `=` is not flagged. Fields gain an optional `description` string,
shown in `summary_rows` as `sh:description`.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1
(cherry picked from commit 3dec282)
Add `parse_frontmatter_strict`, an event-level YAML 1.2 reader built on
saphyr-parser that resolves plain scalars through the core schema and
refuses anchors, aliases, tags, merge keys, duplicate and non-string keys,
multiple documents, a non-mapping top level, more than 64 KiB, and nesting
deeper than 32 levels.

Every vault reader now goes through `parse_vault_frontmatter_at`. Files
under `kinds/` are read strictly (an invalid block is logged and stored as
`{}`); every other file is read by both parsers for one release, and the
legacy line reading is kept, with a warning naming the file and key,
whenever the two disagree. The materializer's output is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1
(cherry picked from commit 9bf209f)
The materializer wrote strings such as "", "~", "True", "0x1F", "x:" and
"- x" bare, which YAML 1.2 reads as null, a bool, a number or an error.
It now JSON-quotes any string whose strict YAML reading (or legacy
reading) is not that same string. Stored values are unchanged; only the
materialized bytes for those keys differ.

Add `render_frontmatter_document`, the writer for `kinds/` files: sorted
keys, no system keys, flow-style JSON for collections, the same quoting,
and keys quoted when YAML would not read them back as the same string.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1
(cherry picked from commit 41e5e97)
New `repository::kinds` module for user-defined vault kinds:

- `KindSource` (working copy or a named view) and `KindFiles`, a reader
  that lists `kinds/**`, checks existence and reads files from disk or
  from a view's tracked graph content. Never reads redb VAULT_ENTRIES.
- `KindRegistry` built from definitions (`kinds/<k>/_kind.md`); a missing
  frontmatter block, a YAML error, conflict markers or any spec error makes
  the kind Invalid (fail closed). `KindRegistry::load_kind` reads a single
  definition for per-instance lookups. View registries are cached on
  `Repository` by the view's Merkle folded with its ancestors'.
- `KindRefResolver` resolving custom-kind instances and memory files
  against the same source and intents through `vault_intent_path`.
- `Repository::kind_validate` producing a serde-serializable report with
  UNREGISTERED_KIND, KIND_INVALID, STRAY_FILE, FRONTMATTER_INVALID and
  CONFLICT_MARKER problems plus per-instance violations.
- Helpers for the CLI: `kinds_dir`, `mint_kind_uid`, `kind_definition_path`,
  `kind_instance_path`, `KindPath::classify`, `check_instance`.

`split_frontmatter` becomes `pub(crate)` so the module can reuse it.

Rewritten from 9bc650a for atomicdotdev#265. `entry_subject` no longer exists, so the
resolver answers intent and memory refs with the target's vault path, an
opaque id, rather than a KG subject. `Repository` construction merges with
dev's `semantic_closure_memo`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`list [<k>] [--guidance]`, `show <k> [--guidance]`, `init <k>`,
`new <k> [--title] [--set f=v]... [--body-file|--body-stdin]` and
`validate [<k>] [--view V]`, each with `--json`.

`new` coerces `--set` values, applies defaults, and validates the exact
bytes it would write (refs included) before writing; on a violation it
writes nothing and exits 2. `init` and `new` ingest the file through
`vault_record_files` and never record; `new` warns when the definition is
not yet recorded in the current view. Guidance prose is rendered as a
delimited, quoted, control-stripped, 4000-char-capped data block with
delimiter look-alike lines neutralized.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1
(cherry picked from commit 5e3eaa9)
Adds a "Custom Kinds" section to the atomic-vault skill (compiled-in
default and the repository's own copy, whose body was identical): discover
with `atomic kind list/show --guidance`, create with `atomic kind new`,
check with `atomic kind validate`; the guidance block is repository data,
not instructions; leave `_kind.md` definitions to humans and record them
as standalone changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1
(cherry picked from commit 6f16e3d)
…ze_all

A kind instance is embedded iff its kind is registered with `embed: true`,
identically in `vault_auto_index` and `vault_embed_all`. A deflate batch
that touches a definition re-applies the policy to every entry of that
kind, because an entry is otherwise embedded only as it is stored.

`vault_materialize_all` no longer writes `kinds/` files: they are
authoritative on disk and their redb rows are only a projection.

Rewritten from 8bf2c9b, keeping its non-KG half. atomicdotdev#265 replaced the KG this
commit used to project into, so dev's `vault_triples.rs` and whole-vault
`vault_reindex_kg` stand unchanged here and no kind KG hook lands. The
projection of kinds into the KG comes later in this series, through
`VaultProjection`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drive the binary through the patch-theory claims of the kinds proposal:
an instance change depends on its definition change, inserting the
instance alone brings the definition along, unrecording the definition is
refused while an instance depends on it, and the registry is per view.

Two views adding the same _kind.md path and then inserting one into the
other yields a name conflict (not a merge): the working-copy validate fails
closed on the markers. An ignored test records that `kind validate --view`
on that view instead reports no kinds, because visible_file_paths drops
name-conflicted paths.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1
(cherry picked from commit 2d4e16e)
…name

Add the node-id prefixes and node kinds that were missing (provenance,
session_turn, content, layer, reasoning), the ontology CURIE prefixes
(atom, prov, skos, dc, rdf) and the URI schemes used as node ids (urn,
did), so a custom kind's `<k>:<ULID>` id cannot land in a built-in
namespace.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1
(cherry picked from commit 5b4a212)
…f dropping them

A view's tree projection holds back every path that two inodes claim
(visible_file_paths omits it, get_file_content_on_view refuses it), so the
view-backed kind reader never saw a conflicted _kind.md and `kind validate
--view` reported the kind as absent with exit 0.

The view reader now also takes the projection's name conflicts under
kinds/. A conflicted definition makes its kind Invalid (shown as
`conflicted`; its instances get KIND_INVALID), and a conflicted instance
or kinds/ directory is a new NAME_CONFLICT problem, so the view fails
closed like the working copy does on conflict markers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1
(cherry picked from commit 707bb4b)
… names

A kind definition's title, field descriptions, enum values, ref targets
and string defaults reach terminals and agents through `atomic kind show`.
Reject any of them that holds a control character (Cc, newlines included)
or a format character (Cf: bidi overrides, zero-width characters, tag
characters), or more than MAX_SPEC_TEXT_CHARS characters, with a
path-qualified problem that names the code point instead of echoing it.

Reserve `kind`, `conforms`, `vault_path`, `violation_codes` and `urn` as
field names: the KG projection writes them as instance-node metadata, so
a same-named field would be silently shadowed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1
(cherry picked from commit 5d880f6)
…tent

Route every value a kind command prints from repository files through one
sanitizer, `display_text`: instance titles and field values in
`kind list <k>`, spec strings and SHACL rows in `kind show`, and problem
and violation messages (which can echo user values) in `show`, `list`,
`new` and `validate`. It drops control and format characters and collapses
whitespace runs, newlines included, to one space, so a value can neither
emit a terminal escape nor start a line of its own. The guidance sanitizer
now drops format characters too. JSON output stays raw.

JSON shapes:
- `kind list --json` and `kind show --json` report the count as
  `instance_count`; `instances` stays the array in `kind list <k> --json`.
- `kind new --json` prints `"written": true` on success, and on every
  failure one object with `"written": false` and either `violations` or
  `error` (`message`, `exit_code`, and `problems` for an invalid kind);
  exit codes are unchanged.

`kind validate` no longer loads the registry a second time just to count
fields, so the summary line drops the field count.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1
(cherry picked from commit 3c0baf2)
… stored entries

Ref resolution:
- A custom-kind ref resolves only when the target kind is registered in
  the same source and the target instance exists; an instance of an
  unregistered or invalid kind is not a node.
- Intent refs resolve only by a full ULID (any case) or the exact full
  human key from the manifest. `3`, `alice::3` and ULID prefixes depended
  on who ran validation or deflate and on which other intents existed.
- `Repository::canonicalize_kind_refs` rewrites user-typed intent ids to
  that stored form; `atomic kind new` calls it before the gate.

Stored definitions:
- The embedding policy reads a kind's spec from the stored
  `kinds/<k>/_kind.md` entry (`stored_kind_definition`), not from the
  working copy. The KG is a redb-derived, last-deflate-wins projection, so
  instance and spec must come from the same layer; this also covers sandbox
  deflates of a view's files that are not on disk, and case-insensitive
  filesystems where `_KIND.md` would stand in for `_kind.md`. The stored
  definition fails closed on conflict markers or `{}` frontmatter. The
  per-view gate still reads files.

Rewritten from 4361e76 without its KG half: the `vault_triples.rs` hunk,
`KindRefResolver::stored` and its `Backend::Stored` are gone, and
`KindDefinition::from_stored` stays for projection. The resolver test keeps
every assertion against the one remaining resolver.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An instance path that was not a regular file (a symlink, which is never
followed) came back from `kind validate` as `conforms: false` with no
violation and no problem, and a read error failed the whole report. Both
are now a repository-level `UNREADABLE` problem on the instance.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1
(cherry picked from commit 7f12917)
- The strict/legacy frontmatter warnings fire once per process for each
  (file, frontmatter content) pair, not on every scan of an unchanged
  file.
- `vault_materialize` refuses a `kinds/**` path, as
  `vault_materialize_all` already skips them: the file on disk is
  authoritative and must not be overwritten from its redb projection.
- Drop `Repository::kinds_dir`, which only a test used.

Rewritten from fda76b8 without its `vault_triples.rs` hunk and the `urn`
metadata on kind instance nodes: `urn:atomic:<k>:<ULID>` is retired, and
`VaultProjection` will own those nodes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Loading the registry once dropped the count from the summary line. Carry
it on `KindReport` instead, so the line reads as in the proposal
("✓ adr: 3 fields, edges …").

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1
(cherry picked from commit 16e6b33)
…rebuild the reserved lists

The kind spec language still spoke the graph vocabulary atomicdotdev#265 retired:
`<k>.<EDGE>` edge kinds, `<k>:<ULID>` node ids and `urn:atomic:<k>:<ULID>`.
None of them survive. `namespaced_edge`, `instance_kg_id`, `instance_urn`,
`KindSpec::summary_rows` and `ShaclRow` are gone, `lift::as_urn` is private
again, and `kind show` no longer prints a SHACL table its rows could not back
up. Its behaviour against SHACL gets a test of its own later in this series.

`edge:` now names a label on a relation, so its grammar becomes
`^[a-z][a-zA-Z0-9_]*$`, and `edges()` and `ResolvedRef.edge` carry the bare
label. Upper-case terms can no longer be spelled in a spec.

A kind name is now reserved when, lowercased with `-` and `_` dropped, it is
the local name of an ontology class or one of `intent`, `kind`, `kinds`, `urn`
and `did`. That is 33 names, and it catches `tool-result`, `finding` and
`project`, which the old 51-entry list missed. `module`, `entity`, `layer`,
`content`, `test` and `agent` become legal. `atomic-canonical` cannot depend
on `atomic-core`, so a test in `atomic-repository/tests/it/kg_kinds.rs` holds
the list equal to `Class::ALL`. A second test there replaces the old
identifier test: generated instance and hub names round-trip through
`parse_concept`, and no other `iri::parse_*` accepts them.

Only `uid`, `title`, `entry_type`, `content_hash` and the timestamps stay
reserved field names, since the KG metadata names no longer need protecting.

`KindRefResolver` now answers every target with a vault path, which is what
it already did for intents and memories. `kind new` keeps printing the
`<k>:<ULID>` handle but drops `kg_id` from its JSON until a real IRI exists.

Also update `kind_view_test`'s `Dependencies: 1` to `Dependencies (1)`, the
form atomicdotdev#265's `change --show-deps` renderer prints. The test had been red since
it was replayed onto dev.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pure move: `kind_cli_test.rs` and `kind_view_test.rs` join the other CLI
integration tests under `tests/it/`, registered with two `mod` lines, so the
workspace is not linked twice more for them. Neither file changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`kind list`, `kind show` and `kind validate` opened the repository writable,
so a read could migrate it, mint a project identity and rebuild the knowledge
graph. They now use the read-only opener. Kinds are read from files, or from a
view through read transactions, so nothing they do needs a write: a test runs
them on a repository rolled back to no identity and an older KG generation,
with working-copy, view and intent-ref reads, and finds every byte under the
checkout unchanged. A writable open of the same repository does change it,
which shows the fixture is the state the reads are meant to spare.

A checkout from before working-copy identities cannot be opened read-only.
There the reads fail with the same typed migration error as every other
read-only command and change nothing. A test pins that, and that the next
write migrates it.

`kind init` and `kind new` now create the vault on first use, like `memory
new`, and the `has_vault` skip in `ingest` is gone. It left a file on disk
that the vault index never saw.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This is a rewrite, not a port. It replaces the KG half of 8bf2c9b, 4361e76
and fda76b8 from the pre-atomicdotdev#265 branch: `vault_extract_kind_kg`,
`KindRefResolver::stored`, `KindBatchContext`, `kg_incoming_edge_owners`,
`kind_instances_referring_to`, `kind_instance_nonconforming` and the KG half
of `vault_reindex_kind`, none of which were replayed. Dev's whole-vault
re-resolve on every store makes those hooks unnecessary: spec edits, target
arrival and target deletion now converge with no kind-specific code. Only
`vault_reembed_changed_kinds` survives, because `vault_auto_index` embeds only
the stored path. `vault_triples.rs` and `kg_rebuild.rs` have no diff from dev.

`rebuild_kg`, `project_vault_kg` and `vault_extract_kg` all reach kinds through
one new `Scratch` arm in `VaultProjection`, as every other vault row does:

- An instance is one `atomic:Concept` at `concept:<k>.<ULID>`, a definition one
  at `concept:kind.<k>`. Membership is an `atomic:references` edge to the hub
  labelled `@kind`, plus `metadata.kind_name`.
- A resolved ref with an `edge:` is an `atomic:references` edge with `fields`
  and `edges` in its metadata, merged per target as the last step, because edge
  identity is `(from, to, predicate)` and two fields on one target would
  otherwise overwrite each other. A `[[memory]]` link in the body is labelled
  `@body`.
- Declared field values go under `field.<name>`, which no adapter key or GC key
  can collide with, so a field cannot reach RDF or claim ownership of a row.
- A non-conforming instance stays visible with a `KindViolation` finding, and an
  invalid definition keeps its hub with a `KindDefinitionInvalid` one. A row
  failing the arm predicate stays inert, for `kind validate` to report.

Projection cannot fail on content. `extract_kind_entry` returns no `Result` and
has no `expect` or `unwrap`, and a ref resolves only to a subject this projection
emits a node for, so `rebuild_kg`'s closure check holds. `rebuild_kg` runs under
`?` in writable open, so anything less could lock a user out of a repository.
`no_repository_content_can_fail_projection_or_writable_open` runs malformed
instances, invalid specs, dangling refs, refs to malformed instances and
instances of unregistered kinds through an expired-generation writable open.

Labels share this namespace: `iri::mint_concept(project, label)` means an intent
labelled `kind.adr` or `adr.<ULID>` names a kind node's exact IRI. The owned node
wins in every write order, since a label's stub has no owner and lands only while
the IRI is free. The one failure was an instance going inert under a label on a
row that sorts before `kinds/`: the stub did not land, then the row deleted its
old node, and `del_kg_node` took the label's edge with it, which a rebuild keeps.
Such a row now owns a bare node while a label names it. No repository without
`kinds/` rows changes: `a_repository_with_no_kinds_projects_as_it_did_before_kinds`
pins a digest computed by running that same fixture on dev at 5c31fae.

Unlike built-ins, kinds do not project file mentions from the body. A File stub
has no owner, so it would outlive the last row that mentions it until a rebuild.
Every unowned stub on dev does this today (a removed label, assignee or file
mention leaves an orphan node), but this keeps kinds out of that class, so an
instance's life cycle always ends where a rebuild does.

The behaviour tests of 4cdfde7 and 4361e76 are ported to `tests/it/kg_kinds.rs`
with no hook behind them, along with an `#[ignore]` timing probe. Dropped because
what they guard is gone: `test_kind_custom_has_task_edge_cannot_delete_intent_children`,
`test_vault_reindex_kind_rejects_invalid_name`, `test_kind_subjects`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`kind new --json` and `kind show --json` now carry `kg_id`, the IRI of the
instance or of the kind's hub, through the new `Repository::kind_instance_iri`
and `Repository::kind_hub_iri`. It is null while a clone has no project identity,
so a read-only `kind show` on an unmigrated repository still answers. The IRI is
local to the clone, since adopting another clone's identity rewrites it, and the
skill doc says to use it in a query and never write it into a file.

Every custom relation shares `atomic:references`, so both `query neighbors` text
printers now follow the predicate with any labels in `metadata.edges`:
`→[https://atomic.dev/ns/workflow#references implements]→`. No other edge has that
key, so built-in edges print as they did. Labels go through `display_text`, like
everything else a kind command prints from repository data.

`tests/it/kind_query_test.rs` starts here and drives this through the binary. It
pins that the concept IRI passes oxrdf through `query validate`, and that a field
named after any key the SHACL adapter reads stays off that key: with the `field.`
prefix removed, `query validate` fails with `status must be a string` on the first
such instance. It also pins that a repository of broken kinds and labels spelling
kind nodes still validates, before and after `query reindex --full`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drive the binary from `kind init`/`record`/`kind new` straight into `query
neighbors`, `query search`, `query graph` and `query plan`, with no manual
reindex: instance nodes, the hub, membership and labelled relations must
all be visible through the service path.

Rewrites 2ee7188 against the IRIs and metadata of the projection commit.
The per-kind ids, namespaced edges and `--kind adr` searches it asserted no
longer exist, so a search now selects a kind by filtering on
`metadata.kind_name`, and a test pins that `-t` and `--kinds` cannot do it.

The plan test pins what the query surface actually gives for "accepted
ADRs": `filter` takes the binding's bare name and only binds, so the answer
is its step count and `nodes` keeps every step's output.

Extends the file the previous commit started, which carries the adapter-key
fixture and the `query validate` tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The proposal said each spec construct maps one-to-one onto SHACL-Core. It
does not, and this replaces the claim with a table that runs.

One ADR spec goes through `validate_instance`, and the same rules, written
by hand as a shape, go through `atomic_shacl::validate_ntriples`. Each row
pins the exact violation codes and constraint components for one edit:

- Both reject, value for value: required (absent, null, empty list), enum,
  datatype, closed, a second scalar value, unresolved refs.
- Only the validator rejects, because the objection has no triple: a
  one-item list on a scalar, a scalar on a list, uid against filename, a bad
  filename, a conflict marker.
- Both reject under different names: a non-string enum or ref value, and a
  ref naming a kind outside `target`.

`sh:defaultValue` is the opposite of `default`: it masks `sh:minCount`, the
other constraints run on it, and a second value fails the whole shape load.
`sh:closed` needs `rdf:type` listed, and the index's own system keys, which
the validator skips.

Test-only. No production code compiles a spec to a shape.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`kind validate` reported a symlink under `kinds/` as never followed, while
the working-copy scan read straight through it. The target's bytes became a
vault row and, with a uid, a KG node, so a file outside the repository could
reach the index, and with `embed: true` the embedding provider. A link to a
device or a FIFO never finished.

Under `kinds/`, the scan now skips anything that is not a regular file and
counts a row as present only when its path is a regular file reached through
no symlink, so swapping a stored file or its directory for a link removes the
row. A kind file that cannot be read (not UTF-8, no permission) is logged and
skipped instead of failing the scan, which used to stop every other vault file
from deflating, in `record` included. Its row stays until the file is readable
again, and `kind validate` already names the file.

The scan outside `kinds/` is unchanged: memory, intents and the rest behave as
on dev, which reads through links and fails on the first unreadable file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A `_kind.md` that was a symlink disappeared: with no instance beside it, `kind
validate` said there were no kinds and exited 0. One with no read permission
failed every kind command in the repository. Both now make that one kind
invalid, with the reason, so the others keep working and the file is reported.

`KindFiles::exists` also stops at a linked directory. It checked only the last
path component, so `kinds/adr -> elsewhere` still read a definition and resolved
refs through the link, although the walk would not descend into it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A body repeating `[[x]]` pushed one node per occurrence, each upserted again
on every vault write, because every store re-projects the whole vault. One
1.2 MB instance with 200,000 links made each later write take 103s and a full
rebuild 186s in a debug build, for 31 distinct rows; those now take 2.2s and
3.7s. The merge keeps what storing every copy would leave: the last node the
row owns, or else the first stub.

Only kind rows change. A built-in row with the same body costs what it does on
dev, so I4 holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Zaba505 and others added 9 commits November 11, 2026 17:01
The custom arm of `KindRefResolver` only checked that the target file existed
and its kind registered. Projection also requires the target to be able to
project, so a ref to a file with no `uid`, no parseable frontmatter or a
conflict marker conformed under `kind validate` and `kind new --set`, while the
KG gave the referrer a `KindViolation` with `UNRESOLVED_REF` and no edge.

The resolver now reads the target and applies the same rule, so the gate is the
stricter side in this direction too, as the design says it is. `kind new` refuses
such a ref before writing. The cost is one more file read per custom ref.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`check_spec` accepts `title: ""` and `title: "   "`, and the hub used the value
as it was, so it printed blank in `query` and graph output. A blank title now
falls back to the kind name, as a blank instance title falls back to the uid.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`query search`, `query neighbors` and `query plan` printed node labels and
summaries as stored. A title can hold escape sequences and newlines, and anyone
who can record a file writes one, so `kind` commands sanitized this data while
the query commands they send readers to did not. Those printers now pass node
text and plan content previews through the same `display_text`. This covers
every row type, because memory titles took the same path. `--json` output is
unchanged.

Other printers of KG text, such as `query ask`, are not touched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both opened with the vault-ensuring opener before any check, so on a repository
with no vault, a rejected instance or an unknown kind still created `.vault/`
with its starter files. `kind new` then said `Nothing was written.` over the
files it had just left untracked. Both now run their checks first and create the
vault immediately before writing.

A writable open still happens first, and it can touch `.atomic/`, as it does for
every writing command. A check against a read-only handle would refuse a checkout
that needs migration, which a write must be able to perform.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`kind new` printed only the handle `adr:<ULID>`, which looks queryable and is
not: `query neighbors adr:<ULID>` reports no neighbors and exits 0. The summary
now carries the IRI that does answer, when the clone has one. `--help` listed the
JSON keys without the `kg_id` the skill doc tells agents to use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One unreadable instance made `kind list <k>` exit 3 with no output, while `kind
validate` reported it and carried on, and a symlink or non-UTF-8 instance already
came out as a row. It is now a row with `conforms: false` too, with a warning on
stderr pointing at `kind validate`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`vault materialize -p kinds/...` refused correctly but printed an internal error
asking for a bug report, with exit 128. The refusal is the design working, so the
command now makes it itself and exits 2. The repository call keeps its own check
for every other caller.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`--set intent=PIMO-12` is rejected: intent keys look like `PIMO::alice::3`. The
example now uses `--set intent=1`, which `kind new` rewrites to the full key, and
a test runs what both copies of the skill show.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ry view

A comment here called the hub's answer repository-wide. It is whatever the last
deflate saw, as decision 3 says: a record in a sibling view drops an instance that
view's working copy does not hold, and switching back does not restore it until a
deflate. `kind validate --view` still answers for the view. This pins that, so
nobody reads the hub as a union of views. It passes before and after; no code
changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Zaba505
Zaba505 force-pushed the feat/user-defined-vault-kinds branch from e8ee156 to 094334e Compare November 12, 2026 02:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants