Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,9 @@ There are several places that need reviewing and updating with any new feature o
- `schemas/` — Bundled default schema files (JSON5) using the structured-context schema dialect and top-level `$metadata`. Files starting with `_` are "partials" (fragments for `$ref`).
- `src/schema/metadata-contract.ts` — Single source of truth for the `$metadata` contract
- `schemas/generated/latest/_structured_context_schema_meta.json` — Generated metaschema for the newest dialect (generated on build or with `bun run generate:schema-meta`). `schemas/generated/<version>/` folders are frozen copies, created by the `version` script (`generate:schema-meta --release`) only when a release changes the dialect; never edit them. `src/schema/meta-schemas.ts` loads them at runtime.
- `src/schema/gating.ts` — Cascade gating: computes hierarchy depth and evaluates rules with subtree gating (schema/hierarchy/blocking-rule failures suppress downstream rule checks). See [docs/rules.md](docs/rules.md#cascade-gating).
- `src/judge/` — Semantic rule judges: provider interface and types (`types.ts`), verdict resolution (`verdicts.ts`), per-node and grouped dispatch (`judge-node.ts`), and providers (`provider/anthropic.ts`, `provider/openai.ts`, `provider/jev.ts`, `provider/fake.ts`), provider hydration and the bundled catalogue (`factory.ts`), and `sctx providers` (`src/commands/providers.ts`). See [docs/semantic-rules.md](docs/semantic-rules.md).
- `src/schema/rule-filter.ts` — `--rule-filter` parsing and rule filtering by mode, category, severity, type or id.
- `src/schema/gating.ts` — Cascade gating: computes hierarchy depth and evaluates rules with subtree gating (schema/hierarchy/blocking-rule failures suppress downstream rule evaluation). See [docs/rules.md](docs/rules.md#cascade-gating).
- `src/judge/` — Semantic rule evaluation (the "judge" is its internal name): provider interface and types (`types.ts`), verdict resolution (`verdicts.ts`), per-node and grouped dispatch (`judge-node.ts`), and providers (`provider/anthropic.ts`, `provider/openai.ts`, `provider/jev.ts`, `provider/fake.ts`), provider hydration and the bundled catalogue (`factory.ts`), and `sctx providers` (`src/commands/providers.ts`). See [docs/semantic-rules.md](docs/semantic-rules.md).
- `src/schema/rule-filter.ts` — `--rule-filter` parsing and rule filtering by category, severity, type or id.

## Testing
For most development only the main unit tests need re-running regularly.
Expand Down
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,8 +115,8 @@ Schemas are JSON Schema 2020-12 plus a top-level `$metadata` block for hierarchy
"id": "active-outcome-count",
"category": "workflow",
"description": "Only one outcome should be active at a time",
"scope": "global",
"check": "$count(nodes[resolvedType='outcome' and status='active']) <= 1"
"global": true,
"expression": "$count(nodes[resolvedType='outcome' and status='active']) <= 1"
}
]
}
Expand All @@ -131,17 +131,17 @@ See [docs/schemas.md](docs/schemas.md) (`sctx docs schema`) for the dialect, com
### Validate nodes

```bash
sctx validate <space> [--watch] [--depth <n>] [--include-gated] [--semantic] [--judge <provider>] [--rule-filter <dim=values>]
sctx validate <space> [--watch] [--depth <n>] [--include-gated] [--semantic [provider]] [--rule-filter <dim=values>]
```

Validates markdown files against the JSON schema:
- Extracts YAML frontmatter from each `.md` file
- Skips files without frontmatter or without a `type` field
- Reports validation results with counts and per-file errors
- Runs executable rules (`$metadata.rules`) with **cascade gating**: a schema, hierarchy, or blocking-rule failure on a node suppresses rule evaluation for that node's hierarchy subtree. Gated checks are reported in a Coverage section so you know what wasn't evaluated and why. See [docs/rules.md](docs/rules.md#cascade-gating) for details.
- Runs executable rules (`$metadata.rules`) with **cascade gating**: a schema, hierarchy, or blocking-rule failure on a node suppresses rule evaluation for that node's hierarchy subtree. Gated rules are reported in a Coverage section so you know what wasn't evaluated and why. See [docs/rules.md](docs/rules.md#cascade-gating) for details.
- `--depth <n>` limits evaluation to hierarchy nodes within `n` breadth-first levels from the top — useful for staged agentic fix loops (fix the top level, then request the next).
- `--include-gated` evaluates rules even on gated nodes (still reported as gated) — for surfacing cheap bottom-up checks on request.
- `--semantic` also evaluates `mode: semantic` rules with the configured judge (Jev, Anthropic or OpenAI-compatible); `--judge <provider>` uses that configured model provider instead and implies `--semantic`; `--rule-filter` narrows which rules run. See [docs/semantic-rules.md](docs/semantic-rules.md) (`sctx docs semantic-rules`).
- `--semantic` also evaluates the `semantic` part of rules with the configured model provider (Jev, Anthropic or OpenAI-compatible); `--semantic <provider>` uses that configured model provider instead; `--rule-filter` narrows which rules run. See [docs/semantic-rules.md](docs/semantic-rules.md) (`sctx docs semantic-rules`).

### Render space

Expand Down
2 changes: 1 addition & 1 deletion config.example.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,5 @@
}
],
"schema": "/path/to/custom/schemas/my-general-schema.json",
"judge": { "provider": "openrouter-jev" }
"semantic": { "provider": "openrouter-jev" }
}
10 changes: 6 additions & 4 deletions docs/concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,17 +163,19 @@ Rules may be:

Rules are distinct from schema validation: the schema checks structure; rules check meaning and quality.

A rule's `when` is its **applicability condition**: a test that decides whether the rule applies to a node. Its `expression` is the test the node must pass.

Each executable rule has a **category** (`validation`, `coherence`, `workflow`, `best-practice`) that supplies its default **severity** (`error`, `warning`, `info`). A rule may override its category's severity, but categories are the preferred grouping. Only `error` severity violations fail validation.

See [docs/rules.md](rules.md) for the rules reference, including JSONata expression syntax and the full `$metadata` field reference.

### Semantic rules

A **semantic rule** (`mode: semantic`) asks a model to judge a node, for example whether an opportunity is a customer need. It has a `rubric` and may have a `check`, an expression precondition that runs before the judge.
A **semantic rule** is a rule with a `semantic` part. It asks a model to evaluate a node, for example whether an opportunity is a customer need. The `semantic` object can carry a `rubric`, `verdicts` and a verdict `severity` map. A rule can also have an `expression`, a JSONata test that runs before the model.

A **judge** is the model that judges semantic rules: a decision model (Jev) or a chat model (Claude, OpenAI-compatible). For each rule, the judge gives one **verdict** from the rule's `verdicts`, for example `customer-need`, `vague` or `solution`. The rule's `severity` says which verdicts report a violation. Semantic rules cost money, so they are judged only when a run asks for it (`--semantic` or `--judge`); a rule's `check` always runs. The judge is a configured **model provider**: a bundled provider or an entry in `modelProviders`. See [docs/semantic-rules.md](semantic-rules.md).
A **model provider** evaluates semantic rules: a decision model (Jev) or a chat model (Claude, OpenAI-compatible). It is a bundled provider or an entry in `modelProviders`. For each rule, the model gives one **verdict** from the rule's `semantic.verdicts`, for example `customer-need`, `vague` or `solution`. The rule's `semantic.severity` says which verdicts report a violation. Semantic rules cost money, so they are evaluated only when a run asks for it (`--semantic`); a rule's `expression` always runs. See [docs/semantic-rules.md](semantic-rules.md).

A **node ruleset group** is the set of nodes that share the same semantic rules. It forms within one evaluation step of [cascade gating](#cascade-gating): one hierarchy depth, or the set of non-hierarchy nodes. Nodes are keyed on the rules left after `when` and `check` filtering. A judge on the `openrouter-decisions` or `typesafe-systemone` API judges one group in as few decision requests as it can.
A **node ruleset group** is the set of nodes that share the same semantic rules. It forms within one evaluation step of [cascade gating](#cascade-gating): one hierarchy depth, or the set of non-hierarchy nodes. Nodes are keyed on the rules left after `when` and `expression` filtering. A model provider on the `openrouter-decisions` or `typesafe-systemone` API evaluates one group in as few decision requests as it can.

A **decision request** is one call to a decisions API. It carries the state of one or more nodes and a list of decision questions.

Expand All @@ -183,7 +185,7 @@ A **decision question** is one choice question about one node under one semantic

**Cascade gating** is the mechanism by which a schema failure, hierarchy structure violation, or blocking rule violation on a node suppresses rule evaluation for that node's `hierarchy` subtree. It exists so downstream checks — especially costly semantic (model-judged) rules — aren't run against inputs already known to be invalid.

A **coverage gate** records one instance of this: the gating node, what kind of failure triggered it (`schema`, `hierarchy`, or a specific rule `id`), and the set of nodes whose rule checks were suppressed as a result. `sctx validate` reports coverage gates so both humans and agents can see the validation "edge" — where evaluation stopped and why. See [docs/rules.md](rules.md#cascade-gating) for the full model, including the per-rule `gatesDescendants` field and the `--include-gated` / `--depth` CLI options.
A **coverage gate** records one instance of this: the gating node, what kind of failure triggered it (`schema`, `hierarchy`, or a specific rule `id`), and the set of nodes whose rules were suppressed as a result. `sctx validate` reports coverage gates so both humans and agents can see the validation "edge" — where evaluation stopped and why. See [docs/rules.md](rules.md#cascade-gating) for the full model, including the per-rule `gatesDescendants` field and the `--include-gated` / `--depth` CLI options.

---

Expand Down
14 changes: 7 additions & 7 deletions docs/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,9 @@ All plugin names must start with `sctx-` (the prefix is optional in config and n

Plugins can also contribute schemas and partials via a `<pluginRoot>/schemas/` directory (or `"sctx": { "schemas": "..." }` in `package.json`). If a space specifies a plugin-prefixed schema (e.g. `"sctx-wardley-mapping/wardley_map.json"`), schemas are resolved statically without executing plugin code, and the plugin's code hooks are loaded when parsing or rendering.

## Model providers and judge
## Model providers and semantic rules

structured-context judges [semantic rules](semantic-rules.md) with a **model provider**: a named definition that says which wire API to speak, which model to use and where. A provider definition combines an `api`, explicit values or environment-variable names, and the key source.
structured-context evaluates [semantic rules](semantic-rules.md) with a **model provider**: a named definition that says which wire API to speak, which model to use and where. A provider definition combines an `api`, explicit values or environment-variable names, and the key source.

**Bundled providers** work without any configuration: a fixed model per vendor chosen by this project, plus `-env` variants that also read the vendor's usual model and endpoint variables (`OPENAI_MODEL`, `OPENAI_BASE_URL`, `ANTHROPIC_MODEL`), falling back to the bundle's defaults where a variable is unset. Bundles without `-env` read no environment beyond the key: they behave the same on every machine.

Expand Down Expand Up @@ -105,7 +105,7 @@ Bundled provider names follow the pattern:

| Field | Description |
|---|---|
| `api` | **Required.** The wire API (see the table above and [Choosing a judge](semantic-rules.md#choosing-a-judge)), or `fake` (for tests). |
| `api` | **Required.** The wire API (see the table above and [Choosing a model provider](semantic-rules.md#choosing-a-model-provider)), or `fake` (for tests). |
| `model` | A fixed model. Its default depends on the API. |
| `modelEnv` | Environment variable holding a model override. An explicit `model` wins; a variable set but empty is an error. |
| `endpoint` | A fixed API URL. |
Expand All @@ -117,18 +117,18 @@ Resolution per field: the explicit value wins, then the named environment variab

The environment holds only keys and the variables a `-env` bundle names. No environment variable selects a provider.

The top-level `judge` names the model provider that judges semantic rules: a bundled provider or an entry in `modelProviders`. A space can set its own `judge`, which replaces the top-level one for that space, for example to keep a private space on a local model:
The top-level `semantic.provider` names the model provider that evaluates semantic rules: a bundled provider or an entry in `modelProviders`. A space can set its own `semantic`, which replaces the top-level one for that space, for example to keep a private space on a local model:

```json
{
"judge": { "provider": "openrouter-jev" },
"semantic": { "provider": "openrouter-jev" },
"spaces": [
{ "name": "private-notes", "path": "/path/to/notes", "judge": { "provider": "local" } }
{ "name": "private-notes", "path": "/path/to/notes", "semantic": { "provider": "local" } }
]
}
```

Config does not turn judging on. Each run asks for it with `--semantic`, or with `--judge <provider>`, which also selects a different provider for that run.
Config does not turn semantic evaluation on. Each run asks for it with `--semantic`. `--semantic <provider>` also selects a different provider for that run.

## Markdown plugin config

Expand Down
Loading
Loading