Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 36 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,20 +183,43 @@ Bare wikilink items (`- [[Existing Node]]`) in any embedding section populate a

With `fieldOn: "parent"`, embedded child nodes (parsed from a matching heading's list or table) are appended as wikilinks to the parent's `field` array, rather than receiving a `parent` field. This matches schemas where the content model naturally lists children on the parent (e.g. `activity.tasks: ["[[Task A]]"]`).

Metadata is composable across `$ref` graphs:
- `hierarchy`: multiple schema files may define a hierarchy; **last one wins** (root schema overrides partials)
- `aliases` are shallow-merged (later wins)
- `rules` merge by `id`; conflicts error unless the later rule sets `override: true`
- each rule's `category` supplies its default severity (`validation` → `error`, `coherence`/`workflow` → `warning`, `best-practice` → `info`); an optional `severity` field overrides it
- `$metadata.rules` supports `$ref` imports for reusable rule packs
- `relationships` are collected from all files
`$ref` reuses JSON Schema definitions for validation only; it brings no metadata. Metadata is composed only through explicit imports:

```json5
"$metadata": { "imports": ["sctx://_strategy_general"] }
```

- An import brings the whole schema's `$metadata`, and is transitive (the imported schema's own imports come too).
- Merge order: imports in order, depth-first, then the schema's own metadata last.
- One rule for every key: a duplicate rule `id`, alias, relationship (same `parent`/`type`/`field`) or hierarchy `name` is an error unless the later entry sets `override: true` (for an alias, write the value as `{ "type": "goal", "override": true }`).
- `hierarchy`: each schema file declares at most one hierarchy, and every imported hierarchy is kept as a separate named hierarchy (see [Multiple hierarchies](#multiple-hierarchies))
- Each rule's `category` supplies its default severity (`validation` → `error`, `coherence`/`workflow` → `warning`, `best-practice` → `info`); an optional `severity` field overrides it.

If no file defines `hierarchy`, hierarchy-specific checks are skipped.

#### Multiple hierarchies

A space can have several hierarchies by importing schemas that each declare one. For example, a root schema might import a work hierarchy (`goal → task`) and a skills hierarchy (`skill_area → skill`).

- **Naming**: a hierarchy's `name` defaults to the declaring schema's file stem (e.g. `_skills` for `_skills.json`). Set `"name"` in `$metadata.hierarchy` to override it.
- **Main hierarchy**: if the root schema declares a hierarchy, it is the main one. If the root declares none and exactly one is imported, that one is the main hierarchy. If several are imported, the root must select one:

```json5
"$metadata": {
"imports": ["sctx://_work", "sctx://_skills"],
"hierarchy": { "$ref": "sctx://_work#/$metadata/hierarchy" }
}
```

A root may either declare a hierarchy or select one with `$ref`, not both.
- **Each type belongs to at most one hierarchy.** A type in two hierarchies is an error. To change an imported hierarchy's levels, declare a hierarchy with the same `name` and `"override": true`.
- **Validation** (layering, parent types, skip levels, orphans) runs per hierarchy. Relationships can connect types across hierarchies.
- The main hierarchy drives `space_on_a_page` parsing, hierarchy embedding, `template-sync`, and the default `show`/`diagram`/`render` output. Use `--hierarchy <name>` to render another one.

**Customizing Schemas:**
- **Partial schemas**: Files starting with an underscore (like `_sctx_base.json`, `_strategy_general.json`, `_knowledge_wiki.json`) are loaded and used to resolve references (using `$ref`).
- **Partials as entity libraries**: Partials can define reusable entity types in `$defs` that composing schemas reference via `$ref`. Bundled partials like `_strategy_general` and `_knowledge_wiki` provide common entity sets for strategy and wiki content.
- **Partials can carry metadata**: Unlike plain JSON Schema, partials may include `$metadata` (hierarchy, aliases, relationships, rules). This makes them self-contained units that bundle both type definitions and behavioral metadata.
- **Partials can carry metadata**: Unlike plain JSON Schema, partials may include `$metadata` (hierarchy, aliases, relationships, rules). A schema gets that metadata only by listing the partial in `$metadata.imports`.
- **No-metadata partials**: If a partial has no `$metadata`, prefer `$schema: "http://json-schema.org/draft-07/schema#"` so it validates standalone as plain JSON Schema.
- **Loading priority**: Partial schemas are loaded from both the default schema directory and the directory of your specified target schema.
- **Transitive resolution**: `$ref` chains are resolved recursively across files/schemas (including nested `allOf` usage in partials).
Expand Down Expand Up @@ -224,10 +247,10 @@ Validates markdown files against the JSON schema:
### Show space tree

```bash
sctx show <space> [--filter <view-or-expression>]
sctx show <space> [--filter <view-or-expression>] [--hierarchy <name>]
```

Prints the space as an indented hierarchy tree. Hierarchy roots are listed first, followed by orphans (nodes in the hierarchy but with no resolved parent) and non-hierarchy nodes.
Prints the space as an indented hierarchy tree. Hierarchy roots are listed first, followed by orphans (nodes in the hierarchy but with no resolved parent) and non-hierarchy nodes. The main hierarchy is shown by default; `--hierarchy <name>` selects another (see [Multiple hierarchies](#multiple-hierarchies)). `diagram` and `render` accept the same option.

When a node appears under multiple parents (DAG hierarchy), it is printed in full under its first parent. Subsequent appearances with children show a `(*)` marker indicating the subtree is omitted.

Expand Down Expand Up @@ -259,6 +282,7 @@ The WHERE predicate is a [JSONata](https://docs.jsonata.org/overview) expression
- **`ancestors[]`** — flat array of ancestor nodes, nearest first, deduplicated. Each entry includes all schema fields of the ancestor node, plus:
- `_field` — the edge field name that connects to the ancestor
- `_source` — `'hierarchy'` or `'relationship'`
- `_hierarchy` — the hierarchy name, for hierarchy edges
- `_selfRef` — whether the edge is a same-type (self-referential) link
- **`descendants[]`** — same structure, for descendant nodes

Expand Down Expand Up @@ -309,7 +333,7 @@ SELECT relationships(assumption) WHERE resolvedType='opportunity'
### Generate Mermaid diagram

```bash
sctx diagram <space> [--output path/to/output.mmd]
sctx diagram <space> [--output path/to/output.mmd] [--hierarchy <name>]
```

Generates a Mermaid `graph TD` diagram from validated space nodes:
Expand All @@ -326,7 +350,7 @@ sctx schemas show <schema-file> [--mermaid-erd] [--space <name>]

Generates a Mermaid Entity Relationship Diagram from a schema:
- Shows all entity types and their properties
- Displays parent-child relationships based on hierarchy metadata
- Displays parent-child relationships based on the levels of each hierarchy
- Useful for visualizing schema structure during development

Example:
Expand Down
2 changes: 1 addition & 1 deletion docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ see [architecture.md](architecture.md#information-flow) for that flow.
| `loadConfig()` | Load the resolved config (spaces, schemas) from the standard config locations. |
| `createSpaceContext(spaceName, config)` | Build a `SpaceContext` for a named space. Throws `SpaceNotFoundError` on an unknown name. |
| `readSpace(context)` | Read a space into a flat `ReadSpaceResult` (`nodes`, `parseIssues`, `diagnostics`, `source`). |
| `loadSpaceGraph(spaceName, config, options?)` | Read, assemble, and optionally filter a space into a navigable `SpaceGraph`. |
| `loadSpaceGraph(spaceName, config, options?)` | Read, assemble, and optionally filter a space into a navigable `SpaceGraph`. `options.hierarchy` names the hierarchy that structures the graph (default: the main hierarchy). |
| `assembleSpaceGraph(context, options?)` | Assemble (and optionally filter) a graph from a context you already hold — reuses the context instead of rebuilding it. |
| `validateSpace(context, options?)` | Run full validation (schema, references, rules, duplicates) over a space. |
| `validateFile(filePath, config)` | Validate a single file within its space. |
Expand Down
12 changes: 9 additions & 3 deletions docs/concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ A **schema** defines the valid structure for nodes in a `space`: the fields, typ

The schema handles structural validation. Cross-node and workflow checks are handled by executable `rules` defined in `$metadata.rules`.

Schemas are composable: structural definitions and metadata can be sourced across `$ref` graphs, then merged deterministically (root metadata applied last, single hierarchy provider, aliases merged, rules merged by `id` with explicit override semantics).
Schemas are composable. Structural definitions are reused with `$ref` (validation only). Metadata is reused only by listing schema `$id`s in `$metadata.imports`: imports merge transitively, in order, with the schema's own metadata last. Hierarchies, relationships, aliases and rules all follow one rule — a duplicate is an error unless the later entry sets `override: true` — and each declared hierarchy is kept as a separate named hierarchy.

### Rules

Expand Down Expand Up @@ -162,6 +162,10 @@ The **hierarchy** is the ordered list of node types in a space, from root to lea

The hierarchy is modelled as a layered DAG: a non-root node may have zero parents (orphaned), one parent, or multiple parents. The `show` command renders this as an indented tree, marking repeated nodes with `(*)` where the subtree is already shown elsewhere.

A space may have **several hierarchies**, one for each imported schema that declares one. Each hierarchy has a **name** (defaulting to the declaring schema's file stem), and each type belongs to at most one hierarchy. Structural validation runs per hierarchy; relationships may connect types across hierarchies.

The **main hierarchy** is the one declared by the root schema; failing that, the only imported hierarchy; failing that, the one the root selects with `"hierarchy": { "$ref": ... }`. The main hierarchy drives `space on a page` parsing, hierarchy embedding, template sync, and default rendering. `show`, `diagram` and `render` take `--hierarchy <name>` to render another. See [docs/schemas.md](schemas.md#multiple-hierarchies).

Each non-root level uses the shared `field`, `fieldOn`, and `multiple` edge options (see [Graph edges](#graph-edges)). Hierarchy-specific options:

| Option | Default | Meaning |
Expand Down Expand Up @@ -252,9 +256,10 @@ Each entry is a `ResolvedParentRef` object:
| `title` | `string` | The parent node's title |
| `field` | `string` | The frontmatter field that held the wikilink |
| `source` | `'hierarchy' \| 'relationship'` | Whether the edge came from a hierarchy level or a relationship |
| `hierarchy` | `string` | Name of the hierarchy the edge belongs to; present only when `source` is `'hierarchy'` |
| `selfRef` | `boolean` | Whether the edge is a same-type (self-referential) parent link |

The `source` label lets downstream consumers distinguish edge types without re-inspecting the schema. Validation routes `hierarchy` edges to structural checks (parent-type rules, skip-level detection) and `relationship` edges to field reference checks (type-match, missing-target). Tree rendering and rule evaluation use the full set.
The `source` and `hierarchy` labels let downstream consumers distinguish edge types and hierarchies without re-inspecting the schema. Validation routes `hierarchy` edges to structural checks within their own hierarchy (parent-type rules, skip-level detection) and `relationship` edges to field reference checks (type-match, missing-target). Tree rendering and rule evaluation use the full set.

### Content links

Expand Down Expand Up @@ -317,7 +322,8 @@ Each entry in `ancestors[]` or `descendants[]` includes all schema fields of the
| Metadata field | Type | Description |
|----------------|------|-------------|
| `_field` | `string` | The edge field name that connects to this ancestor/descendant |
| `_source` | `'hierarchy' \| 'relationship'` | Whether the edge came from the hierarchy or a relationship |
| `_source` | `'hierarchy' \| 'relationship'` | Whether the edge came from a hierarchy or a relationship |
| `_hierarchy` | `string` | Hierarchy name; present only for hierarchy edges |
| `_selfRef` | `boolean` | Whether the edge is a same-type (self-referential) link |

### SELECT spec
Expand Down
21 changes: 7 additions & 14 deletions docs/rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,28 +107,21 @@ Inside `nodes[...]`, bare names refer to each candidate node. Use `$$` to refere
$count(nodes[resolvedParentTitle=$$.current.title and resolvedType='solution'])
```

## Rule imports (`$ref`)
## Reusing rules

`$metadata.rules` can include `$ref` entries that import:
- one rule
- a rule-set object with `rules: []`

Example:
To share rules, put them in a schema's `$metadata.rules` and list that schema's `$id` in `$metadata.imports`. An import brings the whole schema's metadata, including its own imports:

```json5
"rules": [
{ "$ref": "sctx://rule-pack#/$defs/workflowRule" },
{ "$ref": "sctx://rule-pack#/$defs/coreRuleSet" }
]
"$metadata": {
"imports": ["sctx://_rule-pack"]
}
```

Imported entries are normalized into the same flat runtime list.

## Merge and conflict behavior

When metadata is composed across `$ref`:
When metadata is merged from imports (imports in order, then the schema's own metadata):
- Rules are merged by `id`.
- Different payloads for the same `id` are an error by default.
- A duplicate `id` is an error.
- A later rule may replace an earlier one only with `override: true`.

Example override:
Expand Down
8 changes: 4 additions & 4 deletions docs/schema-dialect-update-proposal.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@ Issue #28 opened the door to reorganize schema metadata, rules, and composition
- Top-level `"$metadata"` is the shipped keyword.
- Hierarchy shape is `"$metadata.hierarchy.levels"` with optional `allowSkipLevels`.
- Rules are a flat `"$metadata.rules"` array with per-rule `category`.
- Metadata is composed across `$ref` graph (DFS, root last) with deterministic merge semantics.
- Metadata is composed across `$ref` graph (DFS, root last) with deterministic merge semantics. *(Superseded by #121: metadata is composed only through explicit `$metadata.imports`; `$ref` is validation-only. See [schemas.md](schemas.md#composition-and-merge-semantics).)*
- Rule conflict policy is error by default; explicit `override: true` enables replacement.
- `$metadata.rules` supports `$ref` imports for specific rules and rule sets.
- `$metadata.rules` supports `$ref` imports for specific rules and rule sets. *(Removed by #121: import a rules-only schema via `$metadata.imports` instead.)*
- Metadata contract source is `src/metadata-contract.ts` (`json-schema-to-ts` types), with generated metaschema artifact in `schemas/generated/_ost_tools_schema_meta.json`.

## Decision Areas and Recommendations
Expand Down Expand Up @@ -167,7 +167,7 @@ Metadata merge behavior is implicit and fragile.
Define deterministic merge rules in this update:

- Metadata sources: traverse `$ref` graph from root schema (DFS), then apply root schema metadata last.
- `hierarchy`: exactly one provider allowed; error if multiple providers define it.
- `hierarchy`: exactly one provider allowed; error if multiple providers define it. *(Superseded: as implemented, the last provider won; since #121 each provider's hierarchy is kept as a separate named hierarchy. See [schemas.md](schemas.md#multiple-hierarchies).)*
- `aliases`: shallow merge by key; later provider wins.
- `rules`: concatenate, then dedupe by `id`; later provider wins on conflict.
- Validation: fail fast on duplicate rule ids with incompatible payload unless override is explicit.
Expand Down Expand Up @@ -443,7 +443,7 @@ Stage B acceptance criteria:
### Stage C - Composability and merge semantics

- [x] Implement metadata collection across `$ref` graph (DFS) with root metadata applied last.
- [x] Enforce single-provider hierarchy (error on multiple providers).
- [x] ~~Enforce single-provider hierarchy (error on multiple providers).~~ Superseded by multiple named hierarchies (#121).
- [x] Implement aliases merge (shallow merge, later provider wins).
- [x] Implement rules merge by `id` with conflict policy:
- [x] default conflict is error;
Expand Down
Loading
Loading