diff --git a/AGENTS.md b/AGENTS.md index 0a187c8..9cdb449 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -32,9 +32,21 @@ There are several places that need reviewing and updating with any new feature o - README.md - overview, kept brief; detail belongs in docs/. Shown by `sctx docs` - AGENTS.md - this file -- docs/* - `sctx docs ` shows concepts.md, config.md, schemas.md (`schema`) and rules.md; also architecture.md, api.md +- docs/* - `sctx docs ` shows concepts.md, config.md, schemas.md (`schema`), rules.md and semantic-rules.md; also architecture.md, api.md - plugin/* - skills, commands, hooks, and scripts; update any affected parts +"Affected" includes files the change does not touch. Search for old names, then reread each file below for meaning, not only for renamed words: + +| Change | Check | +|---|---| +| CLI command, flag or output | `docs/*` for the command; `plugin/skills/structured-context/references/commands.md`; the command list in `plugin/skills/structured-context/SKILL.md`; `plugin/commands/*` | +| Rule, schema or `$metadata` shape | `docs/schemas.md`, `docs/rules.md`, `docs/semantic-rules.md`; `references/schema-authoring.md`, `references/schema-design.md`; bundled `schemas/` | +| User-visible error or warning message | the troubleshooting table in `SKILL.md` | +| Config key or space config | `docs/config.md`; the config example in `references/commands.md` | +| `validate --json` output | `plugin/scripts/state.ts` (the hooks parse it) and `bun run test:hook` | +| Parsing or node content | `docs/concepts.md`; the embedded nodes section in `SKILL.md` | +| New term | `docs/concepts.md` | + ## Planning & Design Principles - **Simplicity first:** Strive for the simplest design that completely solves the problem. Prefer conventions and established filesystem patterns over adding new configuration, programmatic APIs, or abstractions. diff --git a/plugin/.claude-plugin/plugin.json b/plugin/.claude-plugin/plugin.json index 7079195..0c87b2a 100644 --- a/plugin/.claude-plugin/plugin.json +++ b/plugin/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "structured-context", - "version": "0.12.20", + "version": "0.12.21", "description": "Structured content authoring for Obsidian — automatic validation hooks, slash commands, and agent skills to keep markdown files compliant with your schema.", "homepage": "https://github.com/mindsocket/structured-context", "repository": "https://github.com/mindsocket/structured-context", diff --git a/plugin/skills/structured-context/SKILL.md b/plugin/skills/structured-context/SKILL.md index 8e049b7..997d1d2 100644 --- a/plugin/skills/structured-context/SKILL.md +++ b/plugin/skills/structured-context/SKILL.md @@ -11,6 +11,7 @@ description: > `structured-context` validates Obsidian markdown frontmatter against JSON schemas. Content lives in **spaces** (directories of `.md` files or a single "on-a-page" format); schemas define entity types, properties, and rules. +Rules are JSONata expressions, semantic rules evaluated by a model provider (only with `--semantic`), or both. ## Version compatibility @@ -48,6 +49,8 @@ sctx schemas show # inspect a bundled partial (e sctx docs # full README sctx docs config # plugin config reference (fieldMap, typeInference, etc.) sctx docs concepts # terminology reference +sctx docs semantic-rules # semantic rule reference +sctx providers --config # model providers for --semantic ``` `spaces` is the starting point — it shows each space as a block with its schema name, `fieldMap` @@ -65,15 +68,17 @@ partials. ``` validate Validate space content against schema (--watch for live) +validate-file Validate one file within its space (after a single-file edit) show Output the node hierarchy dump Output parsed node data as JSON diagram Generate Mermaid diagram (--output ) +render Render a space in a given format miro-sync Sync to Miro board (requires MIRO_TOKEN env var + miro plugin config with boardId) template-sync Sync Obsidian templates from schema examples plugins List available plugins ``` -All commands require a registered space name. Run `sctx --help` or `sctx --help` for flags. +All commands except `validate-file` require a registered space name. Run `sctx --help` or `sctx --help` for flags. **`dump` is the key debugging tool.** Use it to verify `fieldMap` remapping is working or to inspect exactly what JSONata rules see when a rule fires unexpectedly. @@ -85,6 +90,7 @@ Embedded nodes are nodes that live physically inside another node's file (via ta **Adjacent Relationships** determine how these are parsed: - **Heading matching:** When a heading matches a relationship `matcher` (case-insensitive or `/regex/`), following tables/lists are parsed as that child type. - **Agnostic parsing:** The parser uses the semantic grandparent as the parent for items matched via a relationship heading. +- **Node content:** an embedded node's `content` is the markdown it owns; the containing page's `content` is the body outside its embedded nodes. Run `sctx docs concepts` (Node content) for the exact rules. - **`fieldOn` direction:** Relationships support two link directions. `fieldOn: "child"` (default) sets the relationship field on each child node. `fieldOn: "parent"` instead populates the parent node's array field with `[[Child]]` wikilinks — use this when the content model lists children on the parent (e.g. `activity.tasks`). When `fieldOn: "parent"`, child nodes do not get a `parent` field from the relationship. **When to use sub-entities:** @@ -102,8 +108,8 @@ When writing or editing Obsidian markdown frontmatter: ## Non-obvious issues -**All files appear as "Non-space (no type field)"** — the space uses a different field name for the entity discriminator -(e.g., `record_type`). Configure `fieldMap` in the space config to remap it to `type`. +**Files missing from results (low `Valid` count)** — files without a `type` field are not nodes and are skipped silently. The space may use a different field name for the entity discriminator +(e.g., `record_type`): configure `fieldMap` in the space config to remap it to `type`, or `typeInference` to infer types from folders. **Rule violations on every node of a type** — the rule may be too strict or misconfigured. Use `dump` to verify what the rule actually sees in the `current` object, then adjust the rule in the schema. @@ -114,10 +120,11 @@ what the rule actually sees in the `current` object, then adjust the rule in the | Error Message | Likely Cause | Solution | |---------------|--------------|----------| -| `File has no type field` | Discriminator field missing or named differently | Check `fieldMap` in config or add `type` to frontmatter | -| `must have property 'X'` | Required schema property missing | Check `schemas show --space` to see required properties | -| `could not find node '[[Title]]'` | Broken wikilink | Fix the title in the link or ensure the target file exists and has that title | -| `JSONata error: ...` | Syntax error in schema `$metadata.rules` | Verify the expression with `dump` and a JSONata tester | +| `Invalid type "X". Valid types are: ...` | Type not defined in the schema (or a typo) | Check `schemas show --space`; fix the type or add an alias | +| `must have required property 'X'` | Required schema property missing | Check `schemas show --space` to see required properties | +| `Link target "X" in field "parent" not found` | Broken wikilink | Fix the title in the link or ensure the target file exists and has that title | +| `[rule-id] ... (Rule error: ...)` | A rule's `expression` failed to evaluate (syntax or runtime error) | Verify the expression with `dump` and a JSONata tester | +| `[rule-id] ... (Semantic evaluation error: ...)` | The model provider call failed, or input is still over the per-node input cap after trimming | Check `sctx providers` and the API key; narrow `semantic.scope` or drop `includeContent` | ## Plugins @@ -131,7 +138,7 @@ sctx docs config ## References -- **`references/schema-authoring.md`** — schema file structure, `$metadata`, JSONata rules (run `sctx docs schema` for schema dialect reference) +- **`references/schema-authoring.md`** — schema file structure, `$metadata`, rules (expression and semantic) (run `sctx docs schema` for schema dialect reference) - **`references/schema-design.md`** — process for designing a schema from existing content For CLI and config reference, use `sctx docs ` (topics: `concepts`, `config`, `schema`, `rules`, `semantic-rules`). diff --git a/plugin/skills/structured-context/references/commands.md b/plugin/skills/structured-context/references/commands.md index da9c67d..aa44c6b 100644 --- a/plugin/skills/structured-context/references/commands.md +++ b/plugin/skills/structured-context/references/commands.md @@ -19,7 +19,8 @@ Validates all `.md` files in the space against the JSON schema. For each file: - Skips files with no frontmatter or no `type` field (after `fieldMap` remapping) - Runs JSON schema validation - Runs reference checks (wikilinks → known node titles) -- Runs executable rules (JSONata expressions in `$metadata.rules`), with cascade gating: a +- Runs rules from `$metadata.rules` (each rule's `expression`; its `semantic` part only with + `--semantic`), with cascade gating: a schema, hierarchy, or blocking-rule failure on a node suppresses rule evaluation for its hierarchy subtree. Gated nodes are reported in a coverage section (and `coverageGates` in `--json` output) so you know what wasn't checked and why. @@ -29,9 +30,11 @@ Validates all `.md` files in the space against the JSON schema. For each file: the top — useful for fixing a space level by level (fix the top, re-run with a deeper `--depth`, repeat). `--include-gated` evaluates rules on gated nodes anyway, still reporting them as gated — use it to surface cheap bottom-up checks even when an upstream blocker fired. -`--semantic` also evaluates the `semantic` part of rules through a configured model provider (costly; off by -default; `--semantic ` uses that configured model provider instead), and `--rule-filter =` (repeatable; category, severity, type, id) -narrows which rules run. `validate-file` accepts the same two flags. +`--semantic` also evaluates the `semantic` part of rules through the configured model provider +(costly; off by default; `--semantic ` uses that configured provider instead; `sctx providers` +lists them). When input for a node exceeds the per-node input cap, node bodies are trimmed and the run +summary says which rules were affected. `--rule-filter =` (repeatable; category, +severity, type, id) narrows which rules run. `--json` prints machine-readable results. Exit codes: `0` = clean (warning and info rule violations don't fail), `1` = validation errors found. @@ -45,6 +48,18 @@ sctx validate sctx validate --watch ``` +## validate-file + +```bash +sctx validate-file [--json] [--include-gated] [--semantic [provider]] [--rule-filter ] +``` + +Validates one file within its space: the space is found from the configured space paths, and +the whole space is read so links, hierarchy and gating resolve as in `validate`. Reports only +that file's issues; with `--semantic`, only that file's nodes are sent to the model provider. +Takes a file path, not a space name. A file outside every space reports nothing and exits `0`. +Use it after editing a single file. + ## show ```bash @@ -208,6 +223,6 @@ sctx template-sync --create-missing } ``` -All commands require a registered space name (not an arbitrary path). +All commands except `validate-file` require a registered space name (not an arbitrary path). Schema resolution order: space `schema` > global `schema`. Schema resolution fails if none is configured. diff --git a/plugin/skills/structured-context/references/schema-authoring.md b/plugin/skills/structured-context/references/schema-authoring.md index 27640cf..15d3bc7 100644 --- a/plugin/skills/structured-context/references/schema-authoring.md +++ b/plugin/skills/structured-context/references/schema-authoring.md @@ -64,9 +64,42 @@ Relationships define how sub-entities (nodes inside other files) are parsed and **`fieldOn: "parent"` pattern** — use when the content model lists children on the parent (e.g. `activity.tasks: ["[[Task A]]"]`). Embedded parsing appends child wikilinks to the parent's `field` array rather than setting a `parent` field on each child. Validation checks each array entry resolves to a node of `type`. +### Rules (`$metadata.rules`) + Rules are a flat array. Each rule's `category` (`validation`, `coherence`, `workflow`, `best-practice`) supplies its default severity: `validation` → `error`, `coherence` and `workflow` → `warning`, `best-practice` → `info`. Only `error` violations fail validation; warnings and info are reported only. An optional per-rule `severity` (`error` | `warning` | `info`) overrides the category default — prefer picking the right category over overriding. -A rule has an `expression` (a JSONata test that must return `true`), a `semantic` object (a model evaluates the node against a `rubric`, which falls back to `description`), or both; a rule with neither is rejected. `global: true` runs an `expression` once for the space (no `when`, `type` or `semantic`). `semantic: {}` takes every default; it may declare `verdicts` (verdict → description, or `{ description, guidance }`; default `pass`/`fail`) with a `semantic.severity` map saying which verdicts are `error`/`warning`/`info`/`ignore` (required with custom verdicts). `semantic.scope` (a list of `parent`, `siblings`, `children`, `ancestors`, `descendants`, `branch` (ancestors plus descendants) along the main hierarchy) sends related nodes with the evaluated node; `semantic.includeContent` (`nodeOnly` or `fullScope`/`true`) sends node bodies, and without it no body is sent; such a rule may return `missing-context` (default `warning`) when its input was trimmed to fit. The top-level `severity` is that of a failed `expression` and of the default `fail` verdict. A rule's `expression` always runs and, when it fails, the `semantic` part is skipped; the `semantic` part runs only with `--semantic`. Run `sctx docs semantic-rules` for the full semantic rule reference. A `when` JSONata test (the applicability condition, any rule) limits a rule to the nodes where it returns true. An optional `gatesDescendants` controls whether a failure suppresses rules on the node's subtree (default: true for `error` severity). +What a rule checks depends on which parts it has (a rule with neither is rejected): + +- `expression` — a JSONata test that must return `true` (see [JSONata rules](#jsonata-rules)). Always runs. +- `semantic` — a model provider evaluates the node against `semantic.rubric` (falls back to `description`). Runs only with `--semantic`. `semantic: {}` takes every default. +- Both — the `expression` is a cheap precondition: when it fails, the `semantic` part is skipped. + +Other rule fields: + +- `type` and `when` (a JSONata applicability condition) limit which nodes a rule applies to. +- `global: true` runs an `expression` once for the whole space; it takes no `type`, `when` or `semantic`. +- `gatesDescendants` controls whether a failure suppresses rules on the node's subtree (default: true for `error` severity). + +```json5 +{ + "id": "solution-addresses-opportunity", + "category": "coherence", + "type": "solution", + "description": "A solution addresses the opportunity it sits under", + "expression": "$exists(current.summary)", // optional precondition + "semantic": { + "scope": ["parent"], // send the parent opportunity too + "includeContent": "nodeOnly" // send this node's body; omit to send no bodies + } +} +``` + +`semantic` options, in brief (run `sctx docs semantic-rules` for the full reference): + +- `verdicts` (verdict → description, or `{ description, guidance }`; default `pass`/`fail`) with a `severity` map giving each verdict `error`/`warning`/`info`/`ignore` (required with custom verdicts). The rule's top-level `severity` applies to a failed `expression` and to the default `fail` verdict. +- `scope` — related nodes to send along the main hierarchy: `parent`, `siblings`, `children`, `ancestors`, `descendants`, `branch` (ancestors plus descendants). Prefer the narrowest scope that answers the rubric. +- `includeContent` — `nodeOnly` (the evaluated node's body) or `fullScope`/`true` (every body in scope). Without it, no body is sent. +- A rule with `scope` or `includeContent` may return `missing-context` (default `warning`) when its input was trimmed to fit. ## Metadata composition semantics