Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ There are several places that need reviewing and updating with any new feature o
- `src/schema/metadata-contract.ts` — Single source of truth for the `$metadata` contract
- `schemas/generated/latest/_structured_context_schema_meta.json` — Generated metaschema for the newest dialect (generated on build or with `bun run generate:schema-meta`). `schemas/generated/<version>/` folders are frozen copies, created by the `version` script (`generate:schema-meta --release`) only when a release changes the dialect; never edit them. `src/schema/meta-schemas.ts` loads them at runtime.
- `src/schema/gating.ts` — Cascade gating: computes hierarchy depth and evaluates rules with subtree gating (schema/hierarchy/blocking-rule failures suppress downstream rule evaluation). See [docs/rules.md](docs/rules.md#cascade-gating).
- `src/judge/` — Semantic rule evaluation (the "judge" is its internal name): provider interface and types (`types.ts`), verdict resolution (`verdicts.ts`), per-node and grouped dispatch (`judge-node.ts`), related-node selection for `semantic.scope` (`context.ts`), and providers (`provider/anthropic.ts`, `provider/openai.ts`, `provider/jev.ts`, `provider/fake.ts`), provider hydration and the bundled catalogue (`factory.ts`), and `sctx providers` (`src/commands/providers.ts`). See [docs/semantic-rules.md](docs/semantic-rules.md).
- `src/judge/` — Semantic rule evaluation (the "judge" is its internal name): provider interface and types (`types.ts`), verdict resolution (`verdicts.ts`), per-node and grouped dispatch (`judge-node.ts`), related-node selection for `semantic.scope` (`context.ts`), the per-node input cap and body trimming (`trim.ts`), and providers (`provider/anthropic.ts`, `provider/openai.ts`, `provider/jev.ts`, `provider/fake.ts`), provider hydration and the bundled catalogue (`factory.ts`), and `sctx providers` (`src/commands/providers.ts`). See [docs/semantic-rules.md](docs/semantic-rules.md).
- `src/schema/rule-filter.ts` — `--rule-filter` parsing and rule filtering by category, severity, type or id.

## Testing
Expand Down
2 changes: 2 additions & 0 deletions docs/concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,8 @@ A **decision request** is one call to a decisions API. It carries the state of o

A **decision question** is one choice question about one node under one semantic rule, inside a decision request.

The **per-node input cap** limits the input for one evaluated node and its related nodes. See [Input size and trimming](semantic-rules.md#input-size-and-trimming).

### Cascade gating

**Cascade gating** is the mechanism by which a schema failure, hierarchy structure violation, or blocking rule violation on a node suppresses rule evaluation for that node's `hierarchy` subtree. It exists so downstream checks — especially costly semantic (model-judged) rules — aren't run against inputs already known to be invalid.
Expand Down
8 changes: 4 additions & 4 deletions docs/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,10 +79,10 @@ A provider's `api` names the **request shape**, potentially supported by multipl

| API | Shape | Bundled providers |
|---|---|---|
| `openrouter-decisions` | Decisions API, closed-set `choice` questions, answers with confidence and probabilities; nodes with the same rules share a decision request | `openrouter-jev`, `openrouter-openai-decisions` |
| `typesafe-systemone` | Same decisions shape (currently), including shared decision requests | `typesafe-jev` |
| `openai-decisions` | OpenAI's Decisions API: same concept, its own request/response shape; one node per request | `openai-decisions` |
| `anthropic-messages` | Chat: one request per rule, structured JSON verdict | `anthropic`, `anthropic-haiku-5-5`, `anthropic-env` |
| `openrouter-decisions` | Decisions API, closed-set `choice` questions, answers with confidence and probabilities | `openrouter-jev`, `openrouter-openai-decisions` |
| `typesafe-systemone` | Same decisions shape (currently) | `typesafe-jev` |
| `openai-decisions` | OpenAI's Decisions API: same concept, its own request/response shape | `openai-decisions` |
| `anthropic-messages` | Chat: structured JSON verdict | `anthropic`, `anthropic-haiku-5-5`, `anthropic-env` |
| `openai-completions` | Chat: OpenAI-compatible; works with Ollama, vLLM and OpenRouter | `openai`, `openai-env`, `openrouter-openai` |

Bundled provider names follow the pattern:
Expand Down
8 changes: 4 additions & 4 deletions docs/schema-dialect-update-proposal.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Issue #28 opened the door to reorganize schema metadata, rules, and composition
- Metadata is composed across `$ref` graph (DFS, root last) with deterministic merge semantics. *(Superseded by #121: metadata is composed only through explicit `$metadata.imports`; `$ref` is validation-only. See [schemas.md](schemas.md#composition-and-merge-semantics).)*
- Rule conflict policy is error by default; explicit `override: true` enables replacement.
- `$metadata.rules` supports `$ref` imports for specific rules and rule sets. *(Removed by #121: import a rules-only schema via `$metadata.imports` instead.)*
- Metadata contract source is `src/metadata-contract.ts` (`json-schema-to-ts` types), with generated metaschema artifact in `schemas/generated/_ost_tools_schema_meta.json`.
- Metadata contract source is `src/schema/metadata-contract.ts` (`json-schema-to-ts` types), with generated metaschema artifact in `schemas/generated/latest/_structured_context_schema_meta.json`.

## Decision Areas and Recommendations

Expand Down Expand Up @@ -322,10 +322,10 @@ Use TS-authored metadata contract as the source of truth (`as const`) and infer

Shipped workflow:

- Author metadata schema contract in `src/metadata-contract.ts`.
- Author metadata schema contract in `src/schema/metadata-contract.ts`.
- Infer TS types from that contract via `json-schema-to-ts`.
- Validate runtime metadata via AJV using the same contract object.
- Generate `schemas/generated/_ost_tools_schema_meta.json` from the in-code contract (`bun run generate:schema-meta`).
- Generate `schemas/generated/latest/_structured_context_schema_meta.json` from the in-code contract (`bun run generate:schema-meta`).

This keeps one authoritative contract while still producing a portable schema artifact for tooling.

Expand Down Expand Up @@ -485,7 +485,7 @@ Stage D acceptance criteria:
- **Breaking**: rule declarations migrated to flat `rules[]` with per-rule `category`.
- **New**: metadata composition across `$ref` graph with deterministic merge semantics.
- **New**: rule imports via `$metadata.rules` `$ref` entries and explicit rule override (`override: true`) behavior.
- **Internal**: metadata contract source consolidated in `src/metadata-contract.ts` with generated metaschema artifact.
- **Internal**: metadata contract source consolidated in `src/schema/metadata-contract.ts` with generated metaschema artifact.

## Open questions for decision

Expand Down
12 changes: 9 additions & 3 deletions docs/semantic-rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,13 +146,19 @@ Scope sends node data only, never the verdicts of other rules, and [gating](rule

A global rule cannot have a `semantic` part, so it has no scope.

## Input size and trimming

The input for one evaluated node is capped at about 32,000 estimated tokens. Over the cap, structured-context cuts node bodies, longest related bodies first, and marks each cut with `[RULE_EVALUATION_CONTENT_TRIMMED n chars]`. If the input still does not fit, the rule reports a semantic evaluation error. A run that trims content says so in its summary.

A rule with `semantic.scope` or `semantic.includeContent` can return `missing-context` when content it needs was cut. It is a `warning` by default; change it in `semantic.severity`.

## How semantic evaluation works

- **Order.** On each node, structured-context runs each rule's [`expression`](#combining-an-expression) first. It sends the semantic parts of the rules that pass to the model provider.
- **Grouping.** A model provider on the `openrouter-decisions` or `typesafe-systemone` API evaluates each node ruleset group (see [concepts](concepts.md#semantic-rules)) in as few decision requests as it can, with one decision question per node and rule. This includes the bundled `openrouter-openai-decisions` provider. A decision request is limited to about 32,000 estimated input tokens and 40 decision questions, so a larger group is split into several decision requests, never within a node. A node that fills a decision request on tokens is sent whole as its own decision request. A node with more than 40 rules has its rules split across decision requests. If a node's questions are split across several requests and one fails, all of that node's rules are reported as errors. Other providers, including `openai-decisions` and chat models, evaluate one node per request. Within a space, the nodes at one hierarchy depth are evaluated together, because their verdicts decide whether the next depth is evaluated. Grouping can shift verdicts and probabilities slightly compared with evaluating each node alone.
- **Grouping.** A model provider on a decisions API (`openrouter-decisions` or `typesafe-systemone`) evaluates nodes that share the same rules together, in as few requests as it can, to save cost. Grouping can shift verdicts and probabilities slightly compared with evaluating each node alone.
- **Gating.** structured-context does not evaluate the semantic part on a gated node. [Coverage output](rules.md#coverage-output) shows which nodes were gated.
- **Evaluation errors.** If the model provider fails (for example, a network error or output that structured-context cannot read), structured-context reports a violation with the suffix `(Semantic evaluation error: ...)`. An evaluation error does not gate. If one decision request fails, only the rules on the nodes in that decision request report the error.
- **Input.** The model provider gets the node's title, type, parent titles and schema data, including frontmatter. It gets the node's body, and any [related nodes](#related-nodes-and-content), only as the rule's `semantic.scope` and `semantic.includeContent` ask.
- **Evaluation errors.** If the model provider fails (for example, a network error or output that structured-context cannot read), structured-context reports a violation with the suffix `(Semantic evaluation error: ...)`. An evaluation error does not gate.
- **Input.** The model provider gets the node's title, type, parent titles and schema data, including frontmatter. It gets the node's body, and any [related nodes](#related-nodes-and-content), only as the rule's `semantic.scope` and `semantic.includeContent` ask ([trimmed](#input-size-and-trimming) to fit).
- **Output.** A violation from a semantic verdict also includes the verdict, the model's evidence and any guidance.

`sctx validate-file` evaluates the semantic part only for the file and the rules on its ancestors that can gate it. Thus one file costs at most one model call for each ancestor level, plus one.
Expand Down
2 changes: 1 addition & 1 deletion docs/torres-alignment-research.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ My thoughts so far:
"significant gap if the tool intends to model a complete OST per Torres"
* I think we should also explore alternative schemas. We could `mkdir schemas && mv schema.json schemas/general.json` then add a `schemas/strict_ost.json` for the Torres version (limited
parentage, opinionated names from Torres' work). We have '--schema' scafolding already. We could promote the schema in config.json to space level, and update the default so
`schemas/general.json` is used - partly for backward compatibility but mainly because I have stronger opinions about structure. Put common `refs` in a shared file to avoid repetition.
`schemas/strategy_general.json` is used - partly for backward compatibility but mainly because I have stronger opinions about structure. Put common `refs` in a shared file to avoid repetition.
* Side note - there doesn't appear to be an IP (trademark) issue with "OST". We should be respectful and attribute Torres' work, while calling out these tools' applicability to a broader range of frameworks. The project already wants to explore adjacent and
composite models anyway.
* Taking that idea one further "general.json" might really be strategy_plus_ost.json or something.
Expand Down
2 changes: 1 addition & 1 deletion plugin/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "structured-context",
"version": "0.12.17",
"version": "0.12.20",
"description": "Structured content authoring for Obsidian — automatic validation hooks, slash commands, and agent skills to keep markdown files compliant with your schema.",
"homepage": "https://github.com/mindsocket/structured-context",
"repository": "https://github.com/mindsocket/structured-context",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Schema Authoring Reference

Schema files use a 2020-12-based dialect with top-level `$metadata`.
See `~/src/structured-context/schemas/` for examples (`strategy_general.json`, `strict_ost.json`, `_ost_strict.json`).
See the bundled schemas shipped with `sctx` for examples (`strategy_general.json`, `strict_ost.json`, `_ost_strict.json`).

## `$metadata` (top-level)

Expand Down Expand Up @@ -66,7 +66,7 @@ Relationships define how sub-entities (nodes inside other files) are parsed and

Rules are a flat array. Each rule's `category` (`validation`, `coherence`, `workflow`, `best-practice`) supplies its default severity: `validation` → `error`, `coherence` and `workflow` → `warning`, `best-practice` → `info`. Only `error` violations fail validation; warnings and info are reported only. An optional per-rule `severity` (`error` | `warning` | `info`) overrides the category default — prefer picking the right category over overriding.

A rule has an `expression` (a JSONata test that must return `true`), a `semantic` object (a model evaluates the node against a `rubric`, which falls back to `description`), or both; a rule with neither is rejected. `global: true` runs an `expression` once for the space (no `when`, `type` or `semantic`). `semantic: {}` takes every default; it may declare `verdicts` (verdict → description, or `{ description, guidance }`; default `pass`/`fail`) with a `semantic.severity` map saying which verdicts are `error`/`warning`/`info`/`ignore` (required with custom verdicts). `semantic.scope` (a list of `parent`, `siblings`, `children`, `ancestors`, `descendants`, `branch` (ancestors plus descendants) along the main hierarchy) sends related nodes with the evaluated node; `semantic.includeContent` (`nodeOnly` or `fullScope`/`true`) sends node bodies, and without it no body is sent. The top-level `severity` is that of a failed `expression` and of the default `fail` verdict. A rule's `expression` always runs and, when it fails, the `semantic` part is skipped; the `semantic` part runs only with `--semantic`. Run `sctx docs semantic-rules` for the full semantic rule reference. A `when` JSONata test (the applicability condition, any rule) limits a rule to the nodes where it returns true. An optional `gatesDescendants` controls whether a failure suppresses rules on the node's subtree (default: true for `error` severity).
A rule has an `expression` (a JSONata test that must return `true`), a `semantic` object (a model evaluates the node against a `rubric`, which falls back to `description`), or both; a rule with neither is rejected. `global: true` runs an `expression` once for the space (no `when`, `type` or `semantic`). `semantic: {}` takes every default; it may declare `verdicts` (verdict → description, or `{ description, guidance }`; default `pass`/`fail`) with a `semantic.severity` map saying which verdicts are `error`/`warning`/`info`/`ignore` (required with custom verdicts). `semantic.scope` (a list of `parent`, `siblings`, `children`, `ancestors`, `descendants`, `branch` (ancestors plus descendants) along the main hierarchy) sends related nodes with the evaluated node; `semantic.includeContent` (`nodeOnly` or `fullScope`/`true`) sends node bodies, and without it no body is sent; such a rule may return `missing-context` (default `warning`) when its input was trimmed to fit. The top-level `severity` is that of a failed `expression` and of the default `fail` verdict. A rule's `expression` always runs and, when it fails, the `semantic` part is skipped; the `semantic` part runs only with `--semantic`. Run `sctx docs semantic-rules` for the full semantic rule reference. A `when` JSONata test (the applicability condition, any rule) limits a rule to the nodes where it returns true. An optional `gatesDescendants` controls whether a failure suppresses rules on the node's subtree (default: true for `error` severity).

## Metadata composition semantics

Expand Down
7 changes: 7 additions & 0 deletions src/commands/validate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ export async function validate(
parseIssues: result.parseIssues,
unresolvedContentLinks: result.unresolvedContentLinks,
coverageGates: result.coverageGates,
contextTrimmed: result.contextTrimmed,
skippedByDepth: result.skippedByDepth,
depthLimit: result.depthLimit,
},
Expand Down Expand Up @@ -237,6 +238,12 @@ export async function validate(
for (const { file } of result.skippedByDepth) console.log(` ${file}`);
}

if (result.contextTrimmed.nodeCount > 0) {
console.log(
`\nContext trimmed for ${result.contextTrimmed.nodeCount} node(s) across rules ${result.contextTrimmed.ruleIds.join(', ')}.`,
);
}

console.log(`\n`);

if (
Expand Down
Loading
Loading