diff --git a/docs/concepts.md b/docs/concepts.md index f14b5e02..f18c7639 100644 --- a/docs/concepts.md +++ b/docs/concepts.md @@ -169,9 +169,9 @@ See [docs/rules.md](rules.md) for the rules reference, including JSONata express ### Semantic rules -A **semantic rule** (`mode: semantic`) asks a model to judge a node, for example whether an opportunity is a customer need. It has a `rubric` instead of a `check`. +A **semantic rule** (`mode: semantic`) asks a model to judge a node, for example whether an opportunity is a customer need. It has a `rubric` and may have a `check`, an expression precondition that runs before the judge. -A **judge** is the model that judges semantic rules: a decision model (Jev) or a chat model (Claude, OpenAI-compatible). For each rule, the judge gives one **verdict** from the rule's `verdicts`, for example `customer-need`, `vague` or `solution`. The rule's `severity` says which verdicts report a violation. Semantic rules cost money, so they run only when a run asks for them (`--semantic` or `--judge`). The judge is a configured **model provider**: a bundled provider or an entry in `modelProviders`. See [docs/semantic-rules.md](semantic-rules.md). +A **judge** is the model that judges semantic rules: a decision model (Jev) or a chat model (Claude, OpenAI-compatible). For each rule, the judge gives one **verdict** from the rule's `verdicts`, for example `customer-need`, `vague` or `solution`. The rule's `severity` says which verdicts report a violation. Semantic rules cost money, so they are judged only when a run asks for it (`--semantic` or `--judge`); a rule's `check` always runs. The judge is a configured **model provider**: a bundled provider or an entry in `modelProviders`. See [docs/semantic-rules.md](semantic-rules.md). ### Cascade gating diff --git a/docs/config.md b/docs/config.md index 240989fd..3245a3f3 100644 --- a/docs/config.md +++ b/docs/config.md @@ -128,7 +128,7 @@ The top-level `judge` names the model provider that judges semantic rules: a bun } ``` -Config does not turn semantic rules on. Each run asks for them with `--semantic`, or with `--judge `, which also selects a different provider for that run. +Config does not turn judging on. Each run asks for it with `--semantic`, or with `--judge `, which also selects a different provider for that run. ## Markdown plugin config diff --git a/docs/rules.md b/docs/rules.md index d7c18908..ab0afdfe 100644 --- a/docs/rules.md +++ b/docs/rules.md @@ -78,7 +78,7 @@ Prefer choosing the right category and relying on its default. Set `severity` on "when": "current.status = 'ready'", "check": "$exists(current.owner)" } ``` -`when` costs almost nothing, so use it to stop an expensive rule from running where it does not apply. For a semantic rule, this saves a model call on each node that `when` excludes. +`when` costs almost nothing, so use it to stop an expensive rule from running where it does not apply. For a semantic rule, this saves a model call on each node that `when` excludes. A semantic rule can also carry a `check`; see [Preconditions](semantic-rules.md#preconditions-check) for the order. An error in a `when` or `check` is reported as a non-gating `error` violation. diff --git a/docs/semantic-rules.md b/docs/semantic-rules.md index 99110456..fbe3c8e6 100644 --- a/docs/semantic-rules.md +++ b/docs/semantic-rules.md @@ -35,16 +35,41 @@ Semantic rules cost money and send node content to the judge's provider. Thus th - `--semantic` runs them with the configured judge. - `--judge ` runs them with a specific model provider instead. +A semantic rule's [`check`](#preconditions-check) costs no model call, so it runs on every validation. Without a judge, only that expression runs. + ## Rule fields | Field | Required | Description | |---|---|---| | `mode` | yes | `"semantic"` | +| `check` | no | An expression that a node must pass before the judge sees it. Runs even without a judge. See [Preconditions](#preconditions-check) | | `rubric` | no | Tells the judge what to assess. Default: `description` | | `verdicts` | no | The answers the judge can give. Default: `pass` and `fail`. See [Verdicts](#verdicts) | | `severity` | if `verdicts` is set | The severity of each verdict. See [Severity](#severity) | -A semantic rule cannot have `check` or `scope: "global"`. +A semantic rule cannot have `scope: "global"`. + +## Preconditions (`check`) + +A `check` costs no model call. It uses the same [expression syntax and context](rules.md#writing-expressions) as an expression rule. Use it for a requirement that an expression can verify, so the judge only sees nodes worth the cost: + +```json +{ "id": "opportunity-customer-voice", "category": "coherence", "type": "opportunity", + "mode": "semantic", "description": "Opportunity is framed in customer voice", + "check": "$exists(current.source)" } +``` + +The order is `when`, then `check`, then the judge. Without a judge, the order is `when`, then `check`. + +- If `check` returns `true`, the judge runs. Without a judge, nothing is reported. The verdict maps to a severity as usual. +- If `check` does not return `true`, structured-context reports a violation and does not call the judge. The violation has no verdict, and its evidence reads `expression check failed: `. +- If `check` cannot be evaluated, structured-context reports a non-gating `error` violation and does not call the judge. + +A failing `check` reports the severity that an expression rule would report: the rule's `severity`, or the default for its category. With a [severity map](#severity), it reports the most severe severity in the map. If every entry in the map is `ignore`, it reports `info`. + +A failing `check` [gates](rules.md#cascade-gating) the node's descendants as an expression rule does. It gates by default when it reports `error` severity. Set `gatesDescendants` to override that either way. + +`--rule-filter mode=semantic` selects these rules with or without a judge. ## Verdicts @@ -83,7 +108,7 @@ Gating uses the severity of the verdict. A verdict with `error` severity gates t ## How judging works -- **Order.** On each node, expression rules run first. Then structured-context sends all of the node's semantic rules to the judge in one call. +- **Order.** On each node, expression rules run first. Then structured-context applies each semantic rule's [`check`](#preconditions-check) and sends the rules that pass to the judge in one call. - **Gating.** structured-context does not judge a gated node. [Coverage output](rules.md#coverage-output) shows which nodes were gated. - **Judge errors.** If the judge fails (for example, a network error or output that structured-context cannot read), structured-context reports a violation. A judge error does not gate. - **Input.** The judge gets the node's title, type, parent titles and schema data, including frontmatter. diff --git a/plugin/.claude-plugin/plugin.json b/plugin/.claude-plugin/plugin.json index 09abbb97..21c2d184 100644 --- a/plugin/.claude-plugin/plugin.json +++ b/plugin/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "structured-context", - "version": "0.12.13", + "version": "0.12.14", "description": "Structured content authoring for Obsidian — automatic validation hooks, slash commands, and agent skills to keep markdown files compliant with your schema.", "homepage": "https://github.com/mindsocket/structured-context", "repository": "https://github.com/mindsocket/structured-context", diff --git a/plugin/skills/structured-context/references/schema-authoring.md b/plugin/skills/structured-context/references/schema-authoring.md index 5a6095b4..129ec4e7 100644 --- a/plugin/skills/structured-context/references/schema-authoring.md +++ b/plugin/skills/structured-context/references/schema-authoring.md @@ -66,7 +66,7 @@ Relationships define how sub-entities (nodes inside other files) are parsed and Rules are a flat array. Each rule's `category` (`validation`, `coherence`, `workflow`, `best-practice`) supplies its default severity: `validation` → `error`, `coherence` and `workflow` → `warning`, `best-practice` → `info`. Only `error` violations fail validation; warnings and info are reported only. An optional per-rule `severity` (`error` | `warning` | `info`) overrides the category default — prefer picking the right category over overriding. -Rules default to `mode: "expression"` (a JSONata `check`). Set `mode: "semantic"` for rules an LLM or decision model judges against a `rubric` (falls back to `description`): they take no `check` or `scope`, may declare `verdicts` (verdict → description, or `{ description, guidance }`; default `pass`/`fail`) with a `severity` map saying which verdicts are `error`/`warning`/`info`/`ignore`, and only run with `--semantic`. Run `sctx docs semantic-rules` for the full semantic rule reference. A `when` JSONata test (any rule) limits a rule to the nodes where it returns true. An optional `gatesDescendants` controls whether a violation suppresses rule checks on the node's subtree (default: true for `error` severity). +Rules default to `mode: "expression"` (a JSONata `check`). Set `mode: "semantic"` for rules an LLM or decision model judges against a `rubric` (falls back to `description`): they take no `scope` and may take a `check` (a JSONata precondition that runs before the judge, and even without `--semantic`; a failing `check` reports a violation without calling the judge), may declare `verdicts` (verdict → description, or `{ description, guidance }`; default `pass`/`fail`) with a `severity` map saying which verdicts are `error`/`warning`/`info`/`ignore`, and are judged only with `--semantic` or `--judge`. Run `sctx docs semantic-rules` for the full semantic rule reference. A `when` JSONata test (any rule) limits a rule to the nodes where it returns true. An optional `gatesDescendants` controls whether a violation suppresses rule checks on the node's subtree (default: true for `error` severity). ## Metadata composition semantics diff --git a/schemas/generated/latest/_structured_context_schema_meta.json b/schemas/generated/latest/_structured_context_schema_meta.json index feb8506f..c1fa17d7 100644 --- a/schemas/generated/latest/_structured_context_schema_meta.json +++ b/schemas/generated/latest/_structured_context_schema_meta.json @@ -284,14 +284,7 @@ }, "then": { "not": { - "anyOf": [ - { - "required": ["check"] - }, - { - "required": ["scope"] - } - ] + "required": ["scope"] }, "if": { "required": ["verdicts"] diff --git a/src/schema/evaluate-rule.ts b/src/schema/evaluate-rule.ts index 4db55548..aaf2c584 100644 --- a/src/schema/evaluate-rule.ts +++ b/src/schema/evaluate-rule.ts @@ -1,5 +1,6 @@ import jsonata from 'jsonata'; import type { Rule, RuleViolation, SpaceNode } from '../types'; +import { ruleSeverity } from './rule-severity'; const expressionCache = new Map>(); @@ -139,3 +140,43 @@ export function expressionErrorViolation(rule: Rule, file: string, error: string description: `${rule.description} (Rule error: ${error})`, }; } + +/** + * Applies the `check` precondition of semantic rules on a node. A rule whose `check` fails or errors reports a + * violation (at its own severity, or `error` for an evaluation failure) and is left out of `judgeable`, so the + * judge never sees it. A failing check's violation names the expression in its evidence, and its rule is listed in + * `failed` so the caller can honour `gatesDescendants`; an evaluation error is never listed. + */ +export async function applySemanticChecks( + rules: Rule[], + node: SpaceNode, + allNodes: SpaceNode[], + nodeIndex: Map, +): Promise<{ judgeable: Rule[]; violations: RuleViolation[]; failed: Rule[] }> { + const judgeable: Rule[] = []; + const violations: RuleViolation[] = []; + const failed: Rule[] = []; + for (const rule of rules) { + if (!rule.check) { + judgeable.push(rule); + continue; + } + const outcome = await tryEvaluateExpression(rule.check, buildEvalContext(node, allNodes, nodeIndex)); + if ('error' in outcome) { + violations.push(expressionErrorViolation(rule, node.label, outcome.error)); + } else if (outcome.value !== true) { + violations.push({ + file: node.label, + ruleId: rule.id, + category: rule.category, + severity: ruleSeverity(rule), + description: rule.description, + evidence: `expression check failed: ${rule.check}`, + }); + failed.push(rule); + } else { + judgeable.push(rule); + } + } + return { judgeable, violations, failed }; +} diff --git a/src/schema/gating.ts b/src/schema/gating.ts index 0c9588f7..b09b4e84 100644 --- a/src/schema/gating.ts +++ b/src/schema/gating.ts @@ -2,7 +2,12 @@ import { judgeNode } from '../judge/judge-node'; import type { SemanticJudgeProvider } from '../judge/types'; import type { SpaceGraph } from '../space-graph'; import type { Rule, RuleSeverity, RuleViolation, SpaceNode } from '../types'; -import { buildEvalContext, expressionErrorViolation, tryEvaluateExpression } from './evaluate-rule'; +import { + applySemanticChecks, + buildEvalContext, + expressionErrorViolation, + tryEvaluateExpression, +} from './evaluate-rule'; import { ruleSeverities, ruleSeverity } from './rule-severity'; /** @@ -157,7 +162,7 @@ export async function evaluateRulesWithGating( } } - const perNodeRules = rules.filter((r) => r.scope !== 'global' && (r.mode !== 'semantic' || judge)); + const perNodeRules = rules.filter((r) => r.scope !== 'global' && (r.mode !== 'semantic' || judge || r.check)); // With a judge target, only the target and its hierarchy ancestors can need judging. const targetAncestors = new Set(); @@ -232,7 +237,17 @@ export async function evaluateRulesWithGating( else if (outcome.value !== true) report(rule); } - const semantic = applicable.filter((r) => r.mode === 'semantic'); + // A semantic rule's `check` fails fast before the judge. Its violation gates like an expression rule's, at the severity it reports. + const prechecked = await applySemanticChecks( + applicable.filter((r) => r.mode === 'semantic'), + node, + nodes, + nodeIndex, + ); + violations.push(...prechecked.violations); + const gatingRule = prechecked.failed.find((r) => ruleGatesDescendants(r)); + if (gatingRule && !blockingRuleId) blockingRuleId = gatingRule.id; + const semantic = prechecked.judgeable; if (judge && semantic.length > 0) { for (const j of await judgeNode(judge, node, semantic)) { if (j.failed) { diff --git a/src/schema/metadata-contract.ts b/src/schema/metadata-contract.ts index 08ac5378..46586f65 100644 --- a/src/schema/metadata-contract.ts +++ b/src/schema/metadata-contract.ts @@ -77,6 +77,7 @@ const RULE_SCHEMA = { ], }, description: { type: 'string', minLength: 1 }, + /** Expression rules: the test a node must pass. Semantic rules: a precondition evaluated after `when` and before the judge; a failing node is reported without judging. */ check: { type: 'string', minLength: 1 }, /** Optional JSONata applicability test, evaluated per node before the rule runs. The rule applies only when it returns true. */ when: { type: 'string', minLength: 1 }, @@ -111,12 +112,12 @@ const RULE_SCHEMA = { required: ['id', 'category', 'description'], // A global rule has no node to test, so `when` needs a per-node rule. not: { required: ['when', 'scope'] }, - // Semantic rules are judged per node (no `check`, not global) and with custom verdicts must map each failing + // Semantic rules are judged per node (not global, `check` optional) and with custom verdicts must map each failing // verdict to a severity. Every other rule needs a `check` and takes none of the semantic-only fields. if: { properties: { mode: { const: 'semantic' } }, required: ['mode'] }, // biome-ignore lint/suspicious/noThenProperty: JSON Schema `if`/`then`/`else` keyword, not a thenable then: { - not: { anyOf: [{ required: ['check'] }, { required: ['scope'] }] }, + not: { required: ['scope'] }, if: { required: ['verdicts'] }, // biome-ignore lint/suspicious/noThenProperty: JSON Schema `if`/`then`/`else` keyword, not a thenable then: { required: ['severity'], properties: { severity: { type: 'object' } } }, diff --git a/src/schema/validate-rules.ts b/src/schema/validate-rules.ts index 30afe3f3..342b7012 100644 --- a/src/schema/validate-rules.ts +++ b/src/schema/validate-rules.ts @@ -1,13 +1,18 @@ import { judgeNode } from '../judge/judge-node'; import type { SemanticJudgeProvider } from '../judge/types'; import type { Rule, RuleViolation, SpaceNode } from '../types'; -import { buildEvalContext, expressionErrorViolation, tryEvaluateExpression } from './evaluate-rule'; +import { + applySemanticChecks, + buildEvalContext, + expressionErrorViolation, + tryEvaluateExpression, +} from './evaluate-rule'; import { ruleSeverity } from './rule-severity'; export { DEFAULT_SEVERITY_BY_CATEGORY, ruleSeverity } from './rule-severity'; export interface ValidateRulesOptions { - /** Judge for semantic rules. Without one, semantic rules are not evaluated. */ + /** Judge for semantic rules. Without one, a semantic rule runs only its `check`. */ judge?: SemanticJudgeProvider; /** Label of the only node to judge (`validate-file`); other nodes' semantic rules are skipped. */ judgeTarget?: string; @@ -108,13 +113,14 @@ export async function validateRules( } } - // 2. Evaluate semantic rules only when a judge is supplied + // 2. Evaluate semantic rules. Without a judge only their `check` runs. const judge = options.judge; - if (judge) { + const runnableSemanticRules = judge ? semanticRules : semanticRules.filter((r) => r.check); + if (runnableSemanticRules.length > 0) { for (const node of nodes) { if (options.judgeTarget !== undefined && node.label !== options.judgeTarget) continue; const applicable: Rule[] = []; - for (const r of semanticRules) { + for (const r of runnableSemanticRules) { if (r.type && r.type !== node.resolvedType) continue; if (r.when) { const when = await tryEvaluateExpression(r.when, buildEvalContext(node, nodes, nodeIndex)); @@ -131,7 +137,12 @@ export async function validateRules( // Safe cascading default: gate semantic evaluation if node failed upstream checks if (failedNodeLabels.has(node.label)) continue; - for (const j of await judgeNode(judge, node, applicable)) { + // A semantic rule's `check` fails fast before the judge. + const prechecked = await applySemanticChecks(applicable, node, nodes, nodeIndex); + violations.push(...prechecked.violations); + if (!judge) continue; + + for (const j of await judgeNode(judge, node, prechecked.judgeable)) { if (!j.failed) continue; violations.push({ file: node.label, diff --git a/src/validate.ts b/src/validate.ts index 451b6d24..af79e2f6 100644 --- a/src/validate.ts +++ b/src/validate.ts @@ -169,7 +169,7 @@ export interface ValidateSpaceOptions { includeGated?: boolean; /** Evaluate only rules matching this filter (see `parseRuleFilter`). */ ruleFilter?: RuleFilter; - /** Judge for `mode: semantic` rules. Without one, semantic rules are not evaluated. */ + /** Judge for `mode: semantic` rules. Without one, a semantic rule runs only its `check`. */ judge?: SemanticJudgeProvider; } @@ -440,7 +440,7 @@ export interface ValidateFileOptions { includeGated?: boolean; /** Evaluate only rules matching this filter (see `parseRuleFilter`). */ ruleFilter?: RuleFilter; - /** Judge for `mode: semantic` rules. Without one, semantic rules are not evaluated. */ + /** Judge for `mode: semantic` rules. Without one, a semantic rule runs only its `check`. */ judge?: SemanticJudgeProvider; } diff --git a/tests/schema/semantic-rule-contract.test.ts b/tests/schema/semantic-rule-contract.test.ts index 9c349a1b..6d923f72 100644 --- a/tests/schema/semantic-rule-contract.test.ts +++ b/tests/schema/semantic-rule-contract.test.ts @@ -37,8 +37,8 @@ describe('rule contract: check and semantic mode', () => { expect(load('semantic-ok', { ...base, mode: 'semantic' }).rules).toHaveLength(1); }); - it('rejects a semantic rule that also declares check', () => { - expect(() => load('semantic-check', { ...base, mode: 'semantic', check: 'true' })).toThrow(); + it('accepts a semantic rule that declares a check precondition', () => { + expect(load('semantic-check', { ...base, mode: 'semantic', check: 'true' }).rules).toHaveLength(1); }); it('rejects a global semantic rule', () => { diff --git a/tests/schema/validate-rules-semantic.test.ts b/tests/schema/validate-rules-semantic.test.ts index 7b29f56d..22753984 100644 --- a/tests/schema/validate-rules-semantic.test.ts +++ b/tests/schema/validate-rules-semantic.test.ts @@ -152,4 +152,75 @@ describe('validateRules - semantic evaluation', () => { expect(fake.callHistory).toHaveLength(1); expect(fake.callHistory[0]?.node.label).toBe('valid-opp.md'); }); + + it('applies a semantic rule check before the judge', async () => { + const checked: Rule[] = [ + { + id: 'opportunity-customer-voice', + category: 'best-practice', + mode: 'semantic', + type: 'opportunity', + check: 'current.source = "User interview with Jane"', + description: 'Must be in customer voice', + }, + ]; + const fake = new FakeJudgeProvider({ pass: false, evidence: 'Not customer voice' }); + + const violations = await validateRules(sampleNodes, checked, { judge: fake }); + + // bad-opp.md fails the check: reported at the rule severity, never sent to the judge. + expect(fake.callHistory.map((c) => c.node.label)).toEqual(['valid-opp.md']); + expect(violations).toEqual([ + expect.objectContaining({ file: 'valid-opp.md', evidence: 'Not customer voice' }), + { + file: 'bad-opp.md', + ruleId: 'opportunity-customer-voice', + category: 'best-practice', + severity: 'info', + description: 'Must be in customer voice', + evidence: 'expression check failed: current.source = "User interview with Jane"', + }, + ]); + }); + + describe('check precondition (no hierarchy)', () => { + const checked: Rule = { + id: 'opportunity-source-interview', + category: 'coherence', + mode: 'semantic', + type: 'opportunity', + description: 'Opportunity comes from an interview', + check: '$contains(current.source, "interview")', + }; + + it('runs the check without a judge: failing reports, passing is silent', async () => { + const violations = await validateRules(sampleNodes, [checked]); + expect(violations).toEqual([ + expect.objectContaining({ + file: 'bad-opp.md', + ruleId: checked.id, + severity: 'warning', + evidence: 'expression check failed: $contains(current.source, "interview")', + }), + ]); + }); + + it('reports an unevaluable check as an error', async () => { + const fake = new FakeJudgeProvider(); + const violations = await validateRules(sampleNodes, [{ ...checked, check: '$notAFunction(' }], { judge: fake }); + expect(fake.callHistory).toHaveLength(0); + expect(violations.map((v) => v.severity)).toEqual(['error', 'error']); + expect(violations[0]?.description).toContain('Rule error'); + }); + + it('evaluates when before check, and judges only nodes whose check passes', async () => { + const fake = new FakeJudgeProvider(); + const whenFalse = await validateRules(sampleNodes, [{ ...checked, when: 'false' }], { judge: fake }); + expect(whenFalse).toEqual([]); + expect(fake.callHistory).toHaveLength(0); + + await validateRules(sampleNodes, [checked], { judge: fake }); + expect(fake.callHistory.map((c) => c.node.label)).toEqual(['valid-opp.md']); + }); + }); }); diff --git a/tests/validate/semantic-rule-shape.test.ts b/tests/validate/semantic-rule-shape.test.ts index 829c85f4..bee6abab 100644 --- a/tests/validate/semantic-rule-shape.test.ts +++ b/tests/validate/semantic-rule-shape.test.ts @@ -167,3 +167,151 @@ describe('rule expression errors', () => { expect(byId('bad-when')).toEqual([expect.objectContaining({ file: 'T.md', severity: 'error' })]); }); }); + +describe('semantic rule check precondition', () => { + const gated = { + id: 'step-fit', + type: 'step', + mode: 'semantic', + category: 'coherence', + description: 'Step fits its task', + check: '$exists(current.owner)', + gatesDescendants: true, + }; + const judgedLabels = (judge: FakeJudgeProvider) => judge.callHistory.map((c) => c.node.label); + + it('judges when the check passes', async () => { + const { context } = setup([gated], { frontmatter: { S: 'owner: sam' } }); + const judge = new FakeJudgeProvider({ 'step-fit': { verdict: 'fail' } }); + const result = await validateSpace(context(), { judge }); + expect(judgedLabels(judge)).toEqual(['S.md']); + expect(result.ruleViolations).toEqual([expect.objectContaining({ ruleId: 'step-fit', verdict: 'fail' })]); + }); + + it('reports a failing check at the rule severity without calling the judge', async () => { + const { context } = setup([gated]); + const judge = new FakeJudgeProvider({ 'step-fit': { verdict: 'fail' } }); + const result = await validateSpace(context(), { judge }); + expect(judge.callHistory).toHaveLength(0); + expect(result.ruleViolations).toEqual([ + { + file: 'S.md', + ruleId: 'step-fit', + category: 'coherence', + severity: 'warning', + description: 'Step fits its task', + evidence: 'expression check failed: $exists(current.owner)', + }, + ]); + }); + + it('reports a failing check at the most severe mapped severity', async () => { + const { context } = setup([{ ...stepRule, check: '$exists(current.owner)' }]); + const judge = new FakeJudgeProvider({ 'step-fit': { verdict: 'minor' } }); + const result = await validateSpace(context(), { judge }); + expect(judge.callHistory).toHaveLength(0); + expect(result.ruleViolations).toEqual([expect.objectContaining({ ruleId: 'step-fit', severity: 'error' })]); + expect(result.ruleViolations[0]?.verdict).toBeUndefined(); + }); + + it('gates descendants of a failing check like an expression rule: by severity, unless overridden', async () => { + const taskRule = { ...gated, id: 'task-fit', type: 'task', gatesDescendants: undefined }; + const run = async (rule: Record) => { + const judge = new FakeJudgeProvider({ 'step-fit': { verdict: 'minor' } }); + const result = await validateSpace(setup([rule, stepRule]).context(), { judge }); + return { result, judged: judgedLabels(judge) }; + }; + + // error severity gates by default + const byDefault = await run({ ...taskRule, severity: 'error' }); + expect(byDefault.judged).toEqual([]); + expect(byDefault.result.coverageGates[0]).toMatchObject({ reason: 'rule', ruleId: 'task-fit' }); + expect(byDefault.result.ruleViolations.map((v) => v.ruleId)).toEqual(['task-fit']); + + // explicit false turns it off + const optedOut = await run({ ...taskRule, severity: 'error', gatesDescendants: false }); + expect(optedOut.judged).toEqual(['S.md']); + expect(optedOut.result.coverageGates).toEqual([]); + expect(optedOut.result.ruleViolations.map((v) => v.ruleId).sort()).toEqual(['step-fit', 'task-fit']); + + // warning severity does not gate by default + const warning = await run({ ...taskRule, severity: 'warning' }); + expect(warning.judged).toEqual(['S.md']); + expect(warning.result.coverageGates).toEqual([]); + + // explicit true gates a warning + const forced = await run({ ...taskRule, severity: 'warning', gatesDescendants: true }); + expect(forced.judged).toEqual([]); + expect(forced.result.coverageGates[0]).toMatchObject({ reason: 'rule', ruleId: 'task-fit' }); + }); + + it('gates the file when an ancestor check fails (single-file validation)', async () => { + const { space } = setup([{ ...gated, id: 'task-fit', type: 'task' }, stepRule]); + const judge = new FakeJudgeProvider({ 'step-fit': { verdict: 'minor' } }); + const result = (await validateFile(space.file('S.md'), space.config, { + judge, + configDir: space.dir, + })) as FileValidationResult; + // T.md's failing check gates S.md, so S.md is not judged and only reports the gate. + expect(judgedLabels(judge)).toEqual([]); + expect(Object.keys(result.warnings)).toEqual(['gated:T.md']); + expect(result.warnings['gated:T.md']?.message).toContain('task-fit'); + }); + + describe('without a judge', () => { + it('reports a failing check, marked as an expression check failure', async () => { + const { context } = setup([gated]); + const result = await validateSpace(context(), {}); + expect(result.ruleViolations).toEqual([ + expect.objectContaining({ + ruleId: 'step-fit', + file: 'S.md', + evidence: 'expression check failed: $exists(current.owner)', + }), + ]); + }); + + it('reports nothing when the check passes', async () => { + const { context } = setup([gated], { frontmatter: { S: 'owner: sam' } }); + const result = await validateSpace(context(), {}); + expect(result.ruleViolations).toEqual([]); + }); + + it('still skips a semantic rule that has no check', async () => { + const { context } = setup([stepRule]); + expect((await validateSpace(context(), {})).ruleViolations).toEqual([]); + }); + + it('honours --rule-filter mode=semantic', async () => { + const { context } = setup([gated]); + const kept = await validateSpace(context(), { ruleFilter: { mode: ['semantic'] } }); + expect(kept.ruleViolations).toHaveLength(1); + const dropped = await validateSpace(context(), { ruleFilter: { mode: ['expression'] } }); + expect(dropped.ruleViolations).toEqual([]); + }); + }); + + it('reports a failing check with an all-ignore severity map at info', async () => { + const rule = { ...stepRule, severity: { minor: 'ignore', major: 'ignore' }, check: '$exists(current.owner)' }; + const { context } = setup([rule]); + const result = await validateSpace(context(), {}); + expect(result.ruleViolations).toEqual([expect.objectContaining({ ruleId: 'step-fit', severity: 'info' })]); + }); + + it('evaluates when before check', async () => { + const { context } = setup([{ ...gated, when: 'false' }]); + const judge = new FakeJudgeProvider(); + const result = await validateSpace(context(), { judge }); + expect(judge.callHistory).toHaveLength(0); + expect(result.ruleViolations).toEqual([]); + }); + + it('reports a check that cannot be evaluated as an error without judging', async () => { + const { context } = setup([{ ...gated, check: '$notAFunction(' }]); + const judge = new FakeJudgeProvider(); + const result = await validateSpace(context(), { judge }); + expect(judge.callHistory).toHaveLength(0); + expect(result.ruleViolations[0]).toMatchObject({ file: 'S.md', severity: 'error' }); + expect(result.ruleViolations[0]?.description).toContain('Rule error'); + }); +});