Repository navigation
Conversation
Contributor
Author
|
@geekgonecrazy yeh I'm planning to rebase this later today and go back through it to see how it needs to change |
Zaba505
force-pushed
the
feat/user-defined-vault-kinds
branch
from
November 6, 2026 03:13
16e6b33 to
e8ee156
Compare
Add `kind_spec`, the open-world extension tier beside the closed `vocab` registry. A kind's definition frontmatter (`kinds/<k>/_kind.md`) is parsed into a `KindSpec` with every problem collected and path-qualified, and instances are validated against it with stable violation codes, ref resolution through a caller-supplied `RefResolver`, and namespaced edges (`<kind>.<EDGE>`) for projection. Also adds write-time helpers (`apply_defaults`, `parse_set_value`) and a SHACL-Core summary for display. Picked from 7b5c539. The self-reference test's edge fixture changes from `CHILD_OF`, a term atomicdotdev#265 retired, to `PARENT_OF_ADR`, so the KG vocabulary lint stays green. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A `=======` line now counts as a conflict marker only when the text also has an opening or closing marker, so a setext heading underlined with seven `=` is not flagged. Fields gain an optional `description` string, shown in `summary_rows` as `sh:description`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1 (cherry picked from commit 3dec282)
Add `parse_frontmatter_strict`, an event-level YAML 1.2 reader built on
saphyr-parser that resolves plain scalars through the core schema and
refuses anchors, aliases, tags, merge keys, duplicate and non-string keys,
multiple documents, a non-mapping top level, more than 64 KiB, and nesting
deeper than 32 levels.
Every vault reader now goes through `parse_vault_frontmatter_at`. Files
under `kinds/` are read strictly (an invalid block is logged and stored as
`{}`); every other file is read by both parsers for one release, and the
legacy line reading is kept, with a warning naming the file and key,
whenever the two disagree. The materializer's output is unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1
(cherry picked from commit 9bf209f)
The materializer wrote strings such as "", "~", "True", "0x1F", "x:" and "- x" bare, which YAML 1.2 reads as null, a bool, a number or an error. It now JSON-quotes any string whose strict YAML reading (or legacy reading) is not that same string. Stored values are unchanged; only the materialized bytes for those keys differ. Add `render_frontmatter_document`, the writer for `kinds/` files: sorted keys, no system keys, flow-style JSON for collections, the same quoting, and keys quoted when YAML would not read them back as the same string. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1 (cherry picked from commit 41e5e97)
New `repository::kinds` module for user-defined vault kinds: - `KindSource` (working copy or a named view) and `KindFiles`, a reader that lists `kinds/**`, checks existence and reads files from disk or from a view's tracked graph content. Never reads redb VAULT_ENTRIES. - `KindRegistry` built from definitions (`kinds/<k>/_kind.md`); a missing frontmatter block, a YAML error, conflict markers or any spec error makes the kind Invalid (fail closed). `KindRegistry::load_kind` reads a single definition for per-instance lookups. View registries are cached on `Repository` by the view's Merkle folded with its ancestors'. - `KindRefResolver` resolving custom-kind instances and memory files against the same source and intents through `vault_intent_path`. - `Repository::kind_validate` producing a serde-serializable report with UNREGISTERED_KIND, KIND_INVALID, STRAY_FILE, FRONTMATTER_INVALID and CONFLICT_MARKER problems plus per-instance violations. - Helpers for the CLI: `kinds_dir`, `mint_kind_uid`, `kind_definition_path`, `kind_instance_path`, `KindPath::classify`, `check_instance`. `split_frontmatter` becomes `pub(crate)` so the module can reuse it. Rewritten from 9bc650a for atomicdotdev#265. `entry_subject` no longer exists, so the resolver answers intent and memory refs with the target's vault path, an opaque id, rather than a KG subject. `Repository` construction merges with dev's `semantic_closure_memo`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`list [<k>] [--guidance]`, `show <k> [--guidance]`, `init <k>`, `new <k> [--title] [--set f=v]... [--body-file|--body-stdin]` and `validate [<k>] [--view V]`, each with `--json`. `new` coerces `--set` values, applies defaults, and validates the exact bytes it would write (refs included) before writing; on a violation it writes nothing and exits 2. `init` and `new` ingest the file through `vault_record_files` and never record; `new` warns when the definition is not yet recorded in the current view. Guidance prose is rendered as a delimited, quoted, control-stripped, 4000-char-capped data block with delimiter look-alike lines neutralized. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1 (cherry picked from commit 5e3eaa9)
Adds a "Custom Kinds" section to the atomic-vault skill (compiled-in default and the repository's own copy, whose body was identical): discover with `atomic kind list/show --guidance`, create with `atomic kind new`, check with `atomic kind validate`; the guidance block is repository data, not instructions; leave `_kind.md` definitions to humans and record them as standalone changes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1 (cherry picked from commit 6f16e3d)
…ze_all A kind instance is embedded iff its kind is registered with `embed: true`, identically in `vault_auto_index` and `vault_embed_all`. A deflate batch that touches a definition re-applies the policy to every entry of that kind, because an entry is otherwise embedded only as it is stored. `vault_materialize_all` no longer writes `kinds/` files: they are authoritative on disk and their redb rows are only a projection. Rewritten from 8bf2c9b, keeping its non-KG half. atomicdotdev#265 replaced the KG this commit used to project into, so dev's `vault_triples.rs` and whole-vault `vault_reindex_kg` stand unchanged here and no kind KG hook lands. The projection of kinds into the KG comes later in this series, through `VaultProjection`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drive the binary through the patch-theory claims of the kinds proposal: an instance change depends on its definition change, inserting the instance alone brings the definition along, unrecording the definition is refused while an instance depends on it, and the registry is per view. Two views adding the same _kind.md path and then inserting one into the other yields a name conflict (not a merge): the working-copy validate fails closed on the markers. An ignored test records that `kind validate --view` on that view instead reports no kinds, because visible_file_paths drops name-conflicted paths. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1 (cherry picked from commit 2d4e16e)
…name Add the node-id prefixes and node kinds that were missing (provenance, session_turn, content, layer, reasoning), the ontology CURIE prefixes (atom, prov, skos, dc, rdf) and the URI schemes used as node ids (urn, did), so a custom kind's `<k>:<ULID>` id cannot land in a built-in namespace. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1 (cherry picked from commit 5b4a212)
…f dropping them A view's tree projection holds back every path that two inodes claim (visible_file_paths omits it, get_file_content_on_view refuses it), so the view-backed kind reader never saw a conflicted _kind.md and `kind validate --view` reported the kind as absent with exit 0. The view reader now also takes the projection's name conflicts under kinds/. A conflicted definition makes its kind Invalid (shown as `conflicted`; its instances get KIND_INVALID), and a conflicted instance or kinds/ directory is a new NAME_CONFLICT problem, so the view fails closed like the working copy does on conflict markers. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1 (cherry picked from commit 707bb4b)
… names A kind definition's title, field descriptions, enum values, ref targets and string defaults reach terminals and agents through `atomic kind show`. Reject any of them that holds a control character (Cc, newlines included) or a format character (Cf: bidi overrides, zero-width characters, tag characters), or more than MAX_SPEC_TEXT_CHARS characters, with a path-qualified problem that names the code point instead of echoing it. Reserve `kind`, `conforms`, `vault_path`, `violation_codes` and `urn` as field names: the KG projection writes them as instance-node metadata, so a same-named field would be silently shadowed. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1 (cherry picked from commit 5d880f6)
…tent Route every value a kind command prints from repository files through one sanitizer, `display_text`: instance titles and field values in `kind list <k>`, spec strings and SHACL rows in `kind show`, and problem and violation messages (which can echo user values) in `show`, `list`, `new` and `validate`. It drops control and format characters and collapses whitespace runs, newlines included, to one space, so a value can neither emit a terminal escape nor start a line of its own. The guidance sanitizer now drops format characters too. JSON output stays raw. JSON shapes: - `kind list --json` and `kind show --json` report the count as `instance_count`; `instances` stays the array in `kind list <k> --json`. - `kind new --json` prints `"written": true` on success, and on every failure one object with `"written": false` and either `violations` or `error` (`message`, `exit_code`, and `problems` for an invalid kind); exit codes are unchanged. `kind validate` no longer loads the registry a second time just to count fields, so the summary line drops the field count. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1 (cherry picked from commit 3c0baf2)
… stored entries
Ref resolution:
- A custom-kind ref resolves only when the target kind is registered in
the same source and the target instance exists; an instance of an
unregistered or invalid kind is not a node.
- Intent refs resolve only by a full ULID (any case) or the exact full
human key from the manifest. `3`, `alice::3` and ULID prefixes depended
on who ran validation or deflate and on which other intents existed.
- `Repository::canonicalize_kind_refs` rewrites user-typed intent ids to
that stored form; `atomic kind new` calls it before the gate.
Stored definitions:
- The embedding policy reads a kind's spec from the stored
`kinds/<k>/_kind.md` entry (`stored_kind_definition`), not from the
working copy. The KG is a redb-derived, last-deflate-wins projection, so
instance and spec must come from the same layer; this also covers sandbox
deflates of a view's files that are not on disk, and case-insensitive
filesystems where `_KIND.md` would stand in for `_kind.md`. The stored
definition fails closed on conflict markers or `{}` frontmatter. The
per-view gate still reads files.
Rewritten from 4361e76 without its KG half: the `vault_triples.rs` hunk,
`KindRefResolver::stored` and its `Backend::Stored` are gone, and
`KindDefinition::from_stored` stays for projection. The resolver test keeps
every assertion against the one remaining resolver.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An instance path that was not a regular file (a symlink, which is never followed) came back from `kind validate` as `conforms: false` with no violation and no problem, and a read error failed the whole report. Both are now a repository-level `UNREADABLE` problem on the instance. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1 (cherry picked from commit 7f12917)
- The strict/legacy frontmatter warnings fire once per process for each (file, frontmatter content) pair, not on every scan of an unchanged file. - `vault_materialize` refuses a `kinds/**` path, as `vault_materialize_all` already skips them: the file on disk is authoritative and must not be overwritten from its redb projection. - Drop `Repository::kinds_dir`, which only a test used. Rewritten from fda76b8 without its `vault_triples.rs` hunk and the `urn` metadata on kind instance nodes: `urn:atomic:<k>:<ULID>` is retired, and `VaultProjection` will own those nodes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Loading the registry once dropped the count from the summary line. Carry
it on `KindReport` instead, so the line reads as in the proposal
("✓ adr: 3 fields, edges …").
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BSsGJb7SDBhUKCUKowh8n1
(cherry picked from commit 16e6b33)
…rebuild the reserved lists The kind spec language still spoke the graph vocabulary atomicdotdev#265 retired: `<k>.<EDGE>` edge kinds, `<k>:<ULID>` node ids and `urn:atomic:<k>:<ULID>`. None of them survive. `namespaced_edge`, `instance_kg_id`, `instance_urn`, `KindSpec::summary_rows` and `ShaclRow` are gone, `lift::as_urn` is private again, and `kind show` no longer prints a SHACL table its rows could not back up. Its behaviour against SHACL gets a test of its own later in this series. `edge:` now names a label on a relation, so its grammar becomes `^[a-z][a-zA-Z0-9_]*$`, and `edges()` and `ResolvedRef.edge` carry the bare label. Upper-case terms can no longer be spelled in a spec. A kind name is now reserved when, lowercased with `-` and `_` dropped, it is the local name of an ontology class or one of `intent`, `kind`, `kinds`, `urn` and `did`. That is 33 names, and it catches `tool-result`, `finding` and `project`, which the old 51-entry list missed. `module`, `entity`, `layer`, `content`, `test` and `agent` become legal. `atomic-canonical` cannot depend on `atomic-core`, so a test in `atomic-repository/tests/it/kg_kinds.rs` holds the list equal to `Class::ALL`. A second test there replaces the old identifier test: generated instance and hub names round-trip through `parse_concept`, and no other `iri::parse_*` accepts them. Only `uid`, `title`, `entry_type`, `content_hash` and the timestamps stay reserved field names, since the KG metadata names no longer need protecting. `KindRefResolver` now answers every target with a vault path, which is what it already did for intents and memories. `kind new` keeps printing the `<k>:<ULID>` handle but drops `kg_id` from its JSON until a real IRI exists. Also update `kind_view_test`'s `Dependencies: 1` to `Dependencies (1)`, the form atomicdotdev#265's `change --show-deps` renderer prints. The test had been red since it was replayed onto dev. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pure move: `kind_cli_test.rs` and `kind_view_test.rs` join the other CLI integration tests under `tests/it/`, registered with two `mod` lines, so the workspace is not linked twice more for them. Neither file changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`kind list`, `kind show` and `kind validate` opened the repository writable, so a read could migrate it, mint a project identity and rebuild the knowledge graph. They now use the read-only opener. Kinds are read from files, or from a view through read transactions, so nothing they do needs a write: a test runs them on a repository rolled back to no identity and an older KG generation, with working-copy, view and intent-ref reads, and finds every byte under the checkout unchanged. A writable open of the same repository does change it, which shows the fixture is the state the reads are meant to spare. A checkout from before working-copy identities cannot be opened read-only. There the reads fail with the same typed migration error as every other read-only command and change nothing. A test pins that, and that the next write migrates it. `kind init` and `kind new` now create the vault on first use, like `memory new`, and the `has_vault` skip in `ingest` is gone. It left a file on disk that the vault index never saw. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This is a rewrite, not a port. It replaces the KG half of 8bf2c9b, 4361e76 and fda76b8 from the pre-atomicdotdev#265 branch: `vault_extract_kind_kg`, `KindRefResolver::stored`, `KindBatchContext`, `kg_incoming_edge_owners`, `kind_instances_referring_to`, `kind_instance_nonconforming` and the KG half of `vault_reindex_kind`, none of which were replayed. Dev's whole-vault re-resolve on every store makes those hooks unnecessary: spec edits, target arrival and target deletion now converge with no kind-specific code. Only `vault_reembed_changed_kinds` survives, because `vault_auto_index` embeds only the stored path. `vault_triples.rs` and `kg_rebuild.rs` have no diff from dev. `rebuild_kg`, `project_vault_kg` and `vault_extract_kg` all reach kinds through one new `Scratch` arm in `VaultProjection`, as every other vault row does: - An instance is one `atomic:Concept` at `concept:<k>.<ULID>`, a definition one at `concept:kind.<k>`. Membership is an `atomic:references` edge to the hub labelled `@kind`, plus `metadata.kind_name`. - A resolved ref with an `edge:` is an `atomic:references` edge with `fields` and `edges` in its metadata, merged per target as the last step, because edge identity is `(from, to, predicate)` and two fields on one target would otherwise overwrite each other. A `[[memory]]` link in the body is labelled `@body`. - Declared field values go under `field.<name>`, which no adapter key or GC key can collide with, so a field cannot reach RDF or claim ownership of a row. - A non-conforming instance stays visible with a `KindViolation` finding, and an invalid definition keeps its hub with a `KindDefinitionInvalid` one. A row failing the arm predicate stays inert, for `kind validate` to report. Projection cannot fail on content. `extract_kind_entry` returns no `Result` and has no `expect` or `unwrap`, and a ref resolves only to a subject this projection emits a node for, so `rebuild_kg`'s closure check holds. `rebuild_kg` runs under `?` in writable open, so anything less could lock a user out of a repository. `no_repository_content_can_fail_projection_or_writable_open` runs malformed instances, invalid specs, dangling refs, refs to malformed instances and instances of unregistered kinds through an expired-generation writable open. Labels share this namespace: `iri::mint_concept(project, label)` means an intent labelled `kind.adr` or `adr.<ULID>` names a kind node's exact IRI. The owned node wins in every write order, since a label's stub has no owner and lands only while the IRI is free. The one failure was an instance going inert under a label on a row that sorts before `kinds/`: the stub did not land, then the row deleted its old node, and `del_kg_node` took the label's edge with it, which a rebuild keeps. Such a row now owns a bare node while a label names it. No repository without `kinds/` rows changes: `a_repository_with_no_kinds_projects_as_it_did_before_kinds` pins a digest computed by running that same fixture on dev at 5c31fae. Unlike built-ins, kinds do not project file mentions from the body. A File stub has no owner, so it would outlive the last row that mentions it until a rebuild. Every unowned stub on dev does this today (a removed label, assignee or file mention leaves an orphan node), but this keeps kinds out of that class, so an instance's life cycle always ends where a rebuild does. The behaviour tests of 4cdfde7 and 4361e76 are ported to `tests/it/kg_kinds.rs` with no hook behind them, along with an `#[ignore]` timing probe. Dropped because what they guard is gone: `test_kind_custom_has_task_edge_cannot_delete_intent_children`, `test_vault_reindex_kind_rejects_invalid_name`, `test_kind_subjects`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`kind new --json` and `kind show --json` now carry `kg_id`, the IRI of the instance or of the kind's hub, through the new `Repository::kind_instance_iri` and `Repository::kind_hub_iri`. It is null while a clone has no project identity, so a read-only `kind show` on an unmigrated repository still answers. The IRI is local to the clone, since adopting another clone's identity rewrites it, and the skill doc says to use it in a query and never write it into a file. Every custom relation shares `atomic:references`, so both `query neighbors` text printers now follow the predicate with any labels in `metadata.edges`: `→[https://atomic.dev/ns/workflow#references implements]→`. No other edge has that key, so built-in edges print as they did. Labels go through `display_text`, like everything else a kind command prints from repository data. `tests/it/kind_query_test.rs` starts here and drives this through the binary. It pins that the concept IRI passes oxrdf through `query validate`, and that a field named after any key the SHACL adapter reads stays off that key: with the `field.` prefix removed, `query validate` fails with `status must be a string` on the first such instance. It also pins that a repository of broken kinds and labels spelling kind nodes still validates, before and after `query reindex --full`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drive the binary from `kind init`/`record`/`kind new` straight into `query neighbors`, `query search`, `query graph` and `query plan`, with no manual reindex: instance nodes, the hub, membership and labelled relations must all be visible through the service path. Rewrites 2ee7188 against the IRIs and metadata of the projection commit. The per-kind ids, namespaced edges and `--kind adr` searches it asserted no longer exist, so a search now selects a kind by filtering on `metadata.kind_name`, and a test pins that `-t` and `--kinds` cannot do it. The plan test pins what the query surface actually gives for "accepted ADRs": `filter` takes the binding's bare name and only binds, so the answer is its step count and `nodes` keeps every step's output. Extends the file the previous commit started, which carries the adapter-key fixture and the `query validate` tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The proposal said each spec construct maps one-to-one onto SHACL-Core. It does not, and this replaces the claim with a table that runs. One ADR spec goes through `validate_instance`, and the same rules, written by hand as a shape, go through `atomic_shacl::validate_ntriples`. Each row pins the exact violation codes and constraint components for one edit: - Both reject, value for value: required (absent, null, empty list), enum, datatype, closed, a second scalar value, unresolved refs. - Only the validator rejects, because the objection has no triple: a one-item list on a scalar, a scalar on a list, uid against filename, a bad filename, a conflict marker. - Both reject under different names: a non-string enum or ref value, and a ref naming a kind outside `target`. `sh:defaultValue` is the opposite of `default`: it masks `sh:minCount`, the other constraints run on it, and a second value fails the whole shape load. `sh:closed` needs `rdf:type` listed, and the index's own system keys, which the validator skips. Test-only. No production code compiles a spec to a shape. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`kind validate` reported a symlink under `kinds/` as never followed, while the working-copy scan read straight through it. The target's bytes became a vault row and, with a uid, a KG node, so a file outside the repository could reach the index, and with `embed: true` the embedding provider. A link to a device or a FIFO never finished. Under `kinds/`, the scan now skips anything that is not a regular file and counts a row as present only when its path is a regular file reached through no symlink, so swapping a stored file or its directory for a link removes the row. A kind file that cannot be read (not UTF-8, no permission) is logged and skipped instead of failing the scan, which used to stop every other vault file from deflating, in `record` included. Its row stays until the file is readable again, and `kind validate` already names the file. The scan outside `kinds/` is unchanged: memory, intents and the rest behave as on dev, which reads through links and fails on the first unreadable file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A `_kind.md` that was a symlink disappeared: with no instance beside it, `kind validate` said there were no kinds and exited 0. One with no read permission failed every kind command in the repository. Both now make that one kind invalid, with the reason, so the others keep working and the file is reported. `KindFiles::exists` also stops at a linked directory. It checked only the last path component, so `kinds/adr -> elsewhere` still read a definition and resolved refs through the link, although the walk would not descend into it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A body repeating `[[x]]` pushed one node per occurrence, each upserted again on every vault write, because every store re-projects the whole vault. One 1.2 MB instance with 200,000 links made each later write take 103s and a full rebuild 186s in a debug build, for 31 distinct rows; those now take 2.2s and 3.7s. The merge keeps what storing every copy would leave: the last node the row owns, or else the first stub. Only kind rows change. A built-in row with the same body costs what it does on dev, so I4 holds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The custom arm of `KindRefResolver` only checked that the target file existed and its kind registered. Projection also requires the target to be able to project, so a ref to a file with no `uid`, no parseable frontmatter or a conflict marker conformed under `kind validate` and `kind new --set`, while the KG gave the referrer a `KindViolation` with `UNRESOLVED_REF` and no edge. The resolver now reads the target and applies the same rule, so the gate is the stricter side in this direction too, as the design says it is. `kind new` refuses such a ref before writing. The cost is one more file read per custom ref. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`check_spec` accepts `title: ""` and `title: " "`, and the hub used the value as it was, so it printed blank in `query` and graph output. A blank title now falls back to the kind name, as a blank instance title falls back to the uid. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`query search`, `query neighbors` and `query plan` printed node labels and summaries as stored. A title can hold escape sequences and newlines, and anyone who can record a file writes one, so `kind` commands sanitized this data while the query commands they send readers to did not. Those printers now pass node text and plan content previews through the same `display_text`. This covers every row type, because memory titles took the same path. `--json` output is unchanged. Other printers of KG text, such as `query ask`, are not touched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both opened with the vault-ensuring opener before any check, so on a repository with no vault, a rejected instance or an unknown kind still created `.vault/` with its starter files. `kind new` then said `Nothing was written.` over the files it had just left untracked. Both now run their checks first and create the vault immediately before writing. A writable open still happens first, and it can touch `.atomic/`, as it does for every writing command. A check against a read-only handle would refuse a checkout that needs migration, which a write must be able to perform. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`kind new` printed only the handle `adr:<ULID>`, which looks queryable and is not: `query neighbors adr:<ULID>` reports no neighbors and exits 0. The summary now carries the IRI that does answer, when the clone has one. `--help` listed the JSON keys without the `kg_id` the skill doc tells agents to use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One unreadable instance made `kind list <k>` exit 3 with no output, while `kind validate` reported it and carried on, and a symlink or non-UTF-8 instance already came out as a row. It is now a row with `conforms: false` too, with a warning on stderr pointing at `kind validate`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`vault materialize -p kinds/...` refused correctly but printed an internal error asking for a bug report, with exit 128. The refusal is the design working, so the command now makes it itself and exits 2. The repository call keeps its own check for every other caller. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`--set intent=PIMO-12` is rejected: intent keys look like `PIMO::alice::3`. The example now uses `--set intent=1`, which `kind new` rewrites to the full key, and a test runs what both copies of the skill show. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ry view A comment here called the hub's answer repository-wide. It is whatever the last deflate saw, as decision 3 says: a record in a sibling view drops an instance that view's working copy does not hold, and switching back does not restore it until a deflate. `kind validate --view` still answers for the view. This pins that, so nobody reads the hub as a union of views. It passes before and after; no code changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Zaba505
force-pushed
the
feat/user-defined-vault-kinds
branch
from
November 12, 2026 02:49
e8ee156 to
094334e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft / reference implementation for discussion #276, user-defined vault kinds. It covers tier A, milestones M0–M2, with Revision 1 (schema in YAML frontmatter, read by a real YAML 1.2 parser) and Revision 2 (the KG half redesigned after #265; evidence and errata).
It's a draft so the proposal can be evaluated against working code; it isn't asking to be merged as is. Every open decision takes the proposal's default (see Assumed decisions). If a ruling goes the other way, the affected code is listed there.
What changed since the previous description
Rebased onto dev
8053c02, which adds #319, #322, #325 and #332. There were no conflicts, andgit range-diffshows all 36 patches unchanged, so only the commit SHAs changed: the previous tipe8ee156is now094334e. Every SHA in this description is the rebased one, apart from the pre-#265 commits and the dev commits that earlier runs were made against. The checks in Testing were re-run on094334e. The cross-version runs in Compatibility were not, and still name the5c31faebinary they used.The rebuild after #265
History was rewritten. The old tip was
16e6b33, ond968586. This branch was rebuilt on dev at5c31fae, with 36 commits. #265 closed the KG vocabulary and removed what the old KG commits used (KgNode.kind, free-string edge kinds,entry_subject,delete_intent_child_nodes), so those commits do not compile against dev and could not be kept. Of the 19 old commits:7b5c539)a5b1ee1)9bc650a,8bf2c9b,4361e76,fda76b8,2ee7188)d9ca76f,425ab6a,fbddd5e,4804469,285dacf. Each message names its source and what it left out4cdfde7)6c951b6Then 6 new commits adapt to #265, and 12 more come from a review pass against the real binary.
Two things to know when reading the history:
01deb36and089c609keep their old messages, which describe reservations thatd697cc6replaces.instance_change_depends_on_definition_changeis red from88b3fe7untild697cc6, on a stale literal from feat: Phase 1 knowledge graph closure and SHACL validation #265's renderer. Kinds do not project into the KG until6c951b6.The KG half is new. Instances and definitions are now
atomic:Conceptnodes, custom relations areatomic:referenceswith labels in edge metadata, and conformance is anatomic:Finding.atomic-core,atomic-shaclandlibatomichave no diff. The costs of this are in Revision 2: an ADR's type does not say "ADR", custom values do not reach RDF, and the validator is hand-coded against the merged plan's direction.Withdrawn from the previous description:
<k>:<ULID>nodes,<k>.<EDGE>edges andkind:<k>nodes.query neighbors adr:<ULID>as a working query. It finds nothing and exits 0.kind show.edge: HAS_TASKcan't delete an intent's children". The mechanism is gone, andHAS_TASKno longer passes the edge grammar.kind,conforms,vault_path,violation_codesandurnas reserved field names.memorydoes".memorytries RPC first andkinddoes not.git_shim_cb13dfailures. Those counts were ond968586.What it does
A repository defines a kind in
.vault/kinds/<k>/_kind.md. The frontmatter is the schema and the body is guidance prose. Instances are stored as.vault/kinds/<k>/<ULID>.md. Atomic then:atomic kind validate [<k>] [--view V] [--json], which exits 2 on violations;atomic:Conceptnodes aturn:atomic:project:<pid>:concept:<k>.<ULID>, each with anatomic:referencesedge to a hub atconcept:kind.<k>, and with declared fields undermetadata.field.<name>;atomic:Findingwith codeKindViolation, and an invalid definition one withKindDefinitionInvalid;atomic kind new, which applies defaults at write time and runs the gate before writing;atomic kind list/show --guidance, rendered as delimited, length-capped data.There is no new storage, transport, redb schema or proto change, and the ontology gains no terms. Rows stay
VaultEntryType::Scratch.Run on this branch, with
urn:atomic:project:<pid>:shown as…, and...for lines left out:Revision 2, part 2 has the full walkthrough.
Commits, by layer
a5b1ee1,8f742f7,01deb36,089c609,d697cc6atomic-canonical/src/kind_spec.rs. Pure: no repository or YAML dependency.d697cc6isolates the #265 delta here: identifiers removed,edge:becomes a label (^[a-z][a-zA-Z0-9_]*$), reserved lists rebuilt.22a31ed,35a9545saphyr-parserevent API with the YAML 1.2 core schema. Rejects anchors, aliases, tags,<<, duplicate keys and multi-doc, and caps size and depth.kinds/**parses strict-only. Everything else runs both parsers and keeps the legacy result if they disagree (one-release fallback).d9ca76f,890c850,fbddd5e,80e94bf,41614fe,efd5591abea88d,1ac0087,e7bef49,dc1c34f,62f2a0d,0c1e033,ccf9e65,2c6c1fb,9bee96c,5be81eb,263985elist/show/init/new/validate, plus a "Custom kinds" section in the vault skill. Reads open read-only and writes create the vault on first write.425ab6a,4804469vault_reembed_changed_kindsfor a definition change.materialize_allskipskinds/**, and single-path materialize refuses it.6c951b6,6c5dd8e,c6fd368,d292265Scratcharm invault_projection.rs.vault_triples.rsandkg_rebuild.rshave no diff.extract_kind_entryreturns noResult, so repository content cannot fail writable open.59c5c4a, with0c1e033metadata.edgesonquery neighbors, and terminal-safe node text inquery search,neighborsandplan, for every row type.88b3fe7,0584d2e,285dacf,6ff12f6,094334eitbinary,atomic query, the SHACL mapping, and the hub's last-deflate behaviour.Hardening from review
Carried over, and re-run on this base:
UNRESOLVED_REF, in the gate and in the KG.kind newrewrites what the user typed into the canonical form.kinds/<k>/_kind.mdentry, not the file on disk. The per-view gate still resolves from files.rfc → adredge survivesadr's spec going invalid and coming back with no hook. The behaviour tests were ported.kindcommands print repository data sanitized. JSON stays raw.UNREADABLE, the dual-run parser warning is deduplicated per file and content hash, andwrittenis on everykind newresult.New in this pass, each with a test that failed before the fix except where noted:
kinds/(6c5dd8e, major). A link's target became a vault row and, with a uid, a KG node, so a file outside the repository could reach the index and, by reading, withembed: truethe embedding provider. A link to/dev/zeroor a FIFO never finished. Underkinds/the scan now skips anything that is not a regular file. One non-UTF-8 or unreadable file there no longer stops every other vault file from deflating. Scope:kinds/only. Elsewhere the scan behaves as on dev.41614fe). A symlinked_kind.mdwith no instances disappeared andkind validateexited 0. One with no read permission failed every kind command. Both now make that one kind invalid, with the reason.c6fd368). A 1.2MB instance with 200,000[[x]]made every later vault write slow. Debug build: deflate 103s to 2.1s, a later unrelated store 103s to 2.2s, a full rebuild 186s to 3.7s. The "before" figures are from a review run. Measured before and after on one machine at 20,000 links, those went from 11.0s, 10.9s and 19.9s to 0.27s, 0.28s and 0.46s. Kind rows only. A built-in row with the same body keeps dev's cost.efd5591). A ref to a file with nouid, no parseable frontmatter or a conflict marker passedkind validateandkind new --set, while the KG marked itUNRESOLVED_REF. The gate now applies the projection's rule.6c951b6).labels: [kind.adr]mints the hub's exact IRI. An instance going inert while anintents/row labels it lost both node and edge incrementally, while a rebuild kept them. An inert row now owns a bare node while a label names it. With the fix disabled at6c951b6, 2 of the 28kg_kindstests then in the file failed. A label spelling a kind node still links to it: see decision 4.59c5c4a).query search,neighborsandplanprinted raw terminal escapes and injected lines from node text.ccf9e65). A rejectedkind new, orkind initover an existing file, initialized a vault and then saidNothing was written.Both now check first. A writable open still happens first and can touch.atomic/, as it does for every writing command.d292265);kind newprinted only a handle that looks queryable and is not (2c6c1fb);kind list <k>aborted on one unreadable instance (9bee96c);vault materialize -p kinds/...showed its refusal as an internal error (5be81eb, now exit 2);--set intent=PIMO-12did not resolve (263985e).094334e). A characterization test: the hub answers for the last deflate, not for every view.Retired on purpose: field-name reservation for
kind,conforms,vault_path,violation_codesandurn. Thefield.prefix makes them unnecessary, and they are legal names now.What the end-to-end tests established
These are the proposal's patch-theory claims, run against the real binary (
atomic-cli/tests/it/kind_view_test.rs):kinds/<k>/.kind validate --viewis ok there. New here: arefadds no patch dependency, so inserting an ADR that supersedes another leaves the target behind, andkind validate --viewexits 2 withdoes not resolve to an existing adr.Repository::unrecord's view-closure check, notUnrecordError::HasDependents. The CLI shows it as an "Internal error … appears to be a bug" with exit 128. That handling predates this branch._kind.md: name conflict, not merge. Each view creates its ownkinds/,kinds/adr/and_kind.md, so after an insert all three are name conflicts.kind validatefails closed in the working copy and with--view.And for the KG (
atomic-repository/tests/it/kg_kinds.rs,atomic-cli/tests/it/kind_query_test.rs):no_repository_content_can_fail_projection_or_writable_openruns malformed rows, invalid specs, dangling refs, cycles and unparsable stored rows through a writable open with an expired generation.5c31fae). Kinds do not project file mentions, to stay out of that class.a_repository_with_no_kinds_projects_as_it_did_before_kindspins a digest computed by running the same fixture on dev.query validateis unaffected. It exits 0 on broken definitions, non-conforming instances, label collisions, and fields named after every key the adapter reads. With thefield.prefix removed, it exits 3, so that test detects a real leak.kind validate --viewstill answers for the view.And against PurRDF (
atomic-cli/tests/it/kind_shacl_mapping_test.rs, 9 tests,purrdf-shapes3.0.1): the spec language is not 1:1 with SHACL-Core. Most constructs pair a code with a component. Five rows have no triple at all.sh:defaultValuemaskssh:minCount. A list default cannot be written assh:defaultValue. The shape is a hand translation and nothing compiles a spec to a shape. Revision 2, part 2 has the table.Timing. Whole-vault re-projection is fine at a few hundred vault rows and a real problem from about 1,000. Memory rows show the same shape at about half the cost, so this is dev's pipeline, not kinds. AMD Ryzen 9 5950X, tmpfs, single-threaded, one run per size, release with LTO off and 16 codegen units (not the shipping profile):
rebuild_kgA first deflate of 100 ADRs took 2.6s. A first deflate of 1,000 did not finish in 15 minutes (at
6ff12f6), while 1,000 memory rows took 622s. About 45 hours for 5,000 ADRs is an estimate from a fit, not a measurement. On NVMe with btrfs, a store at 1,000 ADR rows took 4.0s and 3.7s against 3.6s and 3.9s on tmpfs, and a 100-row deflate took 4.2s against 2.3s (both at6ff12f6). The probes arekg_kinds::scale_probeandkg_kinds::scale_probe_deflate, both#[ignore].Assumed decisions
The numbers are Revision 2's list.
atomic:Concept,atomic:referencesto a hub,metadata.kind_name. Closed per view relative to registered specsvault_projection.rs(Scratcharm,extract_kind_entry). A real class also touchesatomic-core/src/kg/, the TTL andatomic-shacl/src/adapter.rskind_spec.rs(validate_instance),kinds/validate.rs. Compiling to shapes makesvalidate_instancea test oracle and removesKindViolationkinds/mod.rs(KindSource), orQueryView::project_nodeconcept:<k>.<ULID>andconcept:kind.<k>.kindas the word. Reserved rule, 33 nameskind_hub_name,kind_instance_name,ProjectionRefs,RESERVED_KIND_NAMES,atomic-cli/src/commands/kind/. A:kind:segment adds a mint and parse pair iniri.rsand removesinert_kind_rowsaphyr-parser; one-release legacy fallback outsidekinds/**;lift::parse_markdownout of scopevault_frontmatter.rs(the legacy parser is markedTODO: remove)atomic:referenceswithfieldsandedgesin edge metadata;edge:is a label, lowerCamel by conventionextract_kind_entry,merge_kind_edges, the printers inquery/mod.rsandrpc.rs,kind_query_testatomic:FindingwithKindViolationandKindDefinitionInvalid; noconformskeyextract_kind_entrykindopens the repository directly and stays out of the remote-sandbox allowlistatomic-cli/src/commands/kind/, plus new proto arms and handlersvault_auto_index,project_vault_kgCompatibility with existing repositories
Tested with a real
5c31faebinary against this branch's binary, both in throwaway repositories. Dev created the repository: an attested intent, an attested memory, and a hand-written memory with YAML-ambiguous forms (title: a: b,empty:,hex: 0x1F,note: x # y, a block list), all recorded.status,vault list,vault show,intent list,intent verifyandmemory verifygive byte-identical output on the branch, andquery validategives the sameconforms,snapshot_identity,profile_identityandengine. After branch writes the same comparisons stay identical exceptstatusand the file tree, and dev reopening matches byte for byte and then syncs and records normally..atomic/canonicalsidecars are byte-identical at the end of the run.atomic-corehas no diff, so there is no redb schema change and no newVaultEntryTypevariant.record,vault sync), for each hand-written vault file that YAML 1.2 rejects or reads differently, e.g.title: a: b. The legacy reading is still what gets stored.vault materializecan rewrite bytes for some values. The difference from dev's output is quoting only:empty: "",hex: "0x1F",list: "". Stored values are identical, but such an entry shows as modified after re-materializing.kinds/**, the legacy parser still loses YAML block lists during the one-release fallback.snapshot_identity.query validatestill conforms. Kinds stay absent until the branch's next vault store orquery reindex --full._kind.md. A branchquery reindex --fullthen shows the hub invalid until a branch deflate repairs it.vault materializewriteskinds/. After a branch deflate it writes a JSON-flowfields:that still validates. After dev's own deflate it writes the flat form,kind validatefails withunknown key, and instances reportKIND_INVALID.atomic restorerecovers a recorded definition. An unrecorded definition overwritten this way cannot be recovered.d968586) is refused on a dev-created repository withrepository requires unsupported capabilities: 'kg-semweb-v1', by feat: Phase 1 knowledge graph closure and SHACL validation #265's fence. Starting from a 0.20.0-created repository, dev and this branch behave the same at each step, including a firststatusfailing withTable 'pending_untracks' does not exist, which a later write clears.Known limitations / follow-ups
refis not a patch dependency. Insert closure does not carry an ADR's ref targets.kind.adrappears among the hub's neighbours throughhasLabel.view switchto a view without that kind: exit 128, reported as an internal error. Plain dev does the same for any tracked directory holding an untracked child (run on both), so this is not specific to kinds, butkind newmakes it easy to hit because it writes the file without recording it.-tand--kindscannot separateadrfromrfc. Afilterplan only counts: its"input"takes the bare binding name,"$adrs"silently returns 0, andnodeskeeps every ADR and the hub.read_contenton the binding returns nothing.testandabare legal kind names that search cannot find by kind name.kinds/**, the working-copy scan is dev's: it follows symlinks and stops on the first unreadable.md. Run on this branch: one.mdwith no read permission under.vault/memory/makesvault syncexit 128 with nothing synced, and with a link there to/dev/zero,vault synchad not finished after 25 seconds.kindcommands plusquery search,neighborsandplan. Other printers of KG text, such asquery ask, are not sanitized.kg_idcan name a node that does not exist yet.kind newvalidates against the file but stores only the instance row, so with a hand-written_kind.mdthat was never synced,query neighborson the printedkg_idfinds nothing until the next deflate (run).working_copy_idexit non-zero withrequires writable migration, as other read-only commands do. They used to migrate it silently. A legacypristine.redbpluschanges.redblayout is merged on first read, which is dev's behaviour for every read-only command.kind newstill does a writable open, which can touch.atomic/.--view(KindRefResolver). This is part of decision 7.UNRESOLVED_REFcan appear with noKindViolation. A test pins this.<<<<<<<inside a code fence, for example, can't conform.context: trueis recorded on the hub, but nothing consumes it. By reading, instances are notMemory, so they do not enter context.unknown, because there's no per-path history API.kind(M4). A remote sandbox cannot use it.queryworks through the existing service path.extra_dependenciesfrontier pinning, widen-only/DEFINITION_NARROWED,kind migrateand the body conflict-marker deflate guard. The orphan sweep is dropped: GC is owner-based, andquery reindex --fullrebuilds.Testing
172 tests added and none removed from dev: 170 in the normal suite and 2
#[ignore]timing probes. Size: 41 files, +13,990 / -256. Of that, 5,310 added lines are outside tests and 8,680 are in tests (inline test modules counted by a heuristic).Tip
094334e, on dev8053c02, Linux. The whole suite: 11,094 passed, 0 failed, 19 ignored. Dev was not re-measured at8053c02. At5c31fae, dev ran 10,753 tests with 17 ignored, and the branch ran 10,923 with 19 ignored.All 21
git_shim_cb13dtests pass, so the old note about 10 failures does not apply in this environment.cargo fmt --all -- --checkcargo clippy --locked --workspace -- -D warnings, withRUSTFLAGS=-Dwarningscargo clippy --locked --no-deps -p atomic-canonical -p atomic-cli --features dsse-export -- -D warningscargo clippy --locked --all-targets -p atomic-canonical -p atomic-repository -- -D warningsnode --test libatomic/scripts/package-runtime.test.mjscargo nextest run --locked --workspace --features atomic-cli/adoption-test-injection --profile ci, except diagnosticsdiagnostics_integration_testonlyRUSTDOCFLAGS=-Dwarnings cargo doc --locked --workspace --no-depse7508ce, v0.25.0)cryptography==50.0.0Before this rebase, each of the 12 newest commits passed fmt, clippy on the three crates,
--all-targetsclippy onatomic-canonicalandatomic-repository, and the vocabulary lint. That per-commit pass was not repeated on the rebased commits. The tip passes all of them, andkg_vocabulary_lintruns inside the nextest suite.Environment differences from CI: 8 test threads rather than 2 per CPU; diagnostics run separately at 2 threads with
TMPDIRon disk, because each of those tests copies the roughly 2G debug binary; theline-tables-onlydebug settings not applied; Fedora rather than ubuntu-latest; Python 3.14 rather than 3.12.What was not run
ATOMIC_SERVICE=reactor), and a branch-created repository first opened by dev.repo identity adopt.repo identity forkwas run on a hostile corpus, and every hub neighbour's IRI used the new id.adoptwas only read.kinditself on a legacy repository.read_only_open_merges_a_legacy_repositorycovers the opener. The redb format-upgrade case is from reading only.panic=abort). The 1,000 and 5,000-row deflate probes and the 5,000 memory-row probe were not repeated at the tip; those numbers are from6ff12f6.cargo clippy --all-targetsonatomic-cli. CI does not run it. At6ff12f6it failed under-Dwarningson two lints in files this branch does not change (vault_auto_init_test.rs,real_import_sample.rs). Its test targets compiled under-Dwarningsthrough nextest at the tip.atomic-clitest targets at intermediate commits. Only library and binary targets were gated at each of the first 17 commits, so an intermediate commit could have a CLI test-only build problem.intentandmemoryref targets through the SHACL mapping table.kind newunder decision 2(b). Not measured.🤖 Generated with Claude Code