Skip to content

feat(rules)!: rule shape — expression and semantic decide the kind - #164

Merged
mindsocket merged 3 commits into
mainfrom
rule-shape
Oct 16, 2026
Merged

mindsocket merged 3 commits into
mainfrom
rule-shape

Conversation

@mindsocket

@mindsocket mindsocket commented Oct 16, 2026 •

Copy link
Copy Markdown
Owner

Stacked on #161.

Summary

  • The rule's fields decide its kind. A rule has an expression (was check), a semantic object, or both. mode, rubric and verdicts move off the top level. A rule with neither part is rejected when the schema loads.
  • The semantic object holds rubric (falls back to description), verdicts and a per-verdict severity map. semantic: {} takes every default.
  • Top-level severity is the severity of a failed expression and of the default fail verdict.
  • global: true replaces scope: "global", so Semantic rules: rule scope and judge input (what the model sees) #152 can use semantic.scope for related nodes. A global rule rejects when, type and semantic.
  • One flag: --semantic [provider] replaces --judge. The config key is semantic: { provider } (was judge). --rule-filter loses its mode dimension.
  • No more "judge" in user-facing text. Errors read (Semantic evaluation error: …) and "Model provider returned …".
  • Clean break, pre-1.0: old fields fail as ordinary unknown properties. There are no migration hints.

Behaviour changes

  • --judge-target: expressions on ancestor nodes now always run. Semantic rules on nodes outside the target are skipped only when a model provider is in use.
  • expression failed: … evidence appears only when the rule also has a semantic part.
  • Ordering: in a space with no hierarchy, a failed expression is now listed before semantic violations.

Tests

lint and typecheck clean; bun run test 739 pass, 15 skip, 0 fail. test:smoke: 2 known failures from an external schema still in the old shape, which gets migrated after merge.

@mindsocket
mindsocket added this pull request to stack #167 October 16, 2026 03:55
@mindsocket mindsocket changed the title rule shape feat(rules)!: rule shape — expression and semantic decide the kind Oct 16, 2026
@mindsocket
mindsocket marked this pull request as ready for review October 16, 2026 04:04
Comment thread src/schema/gating.ts
Comment on lines +183 to 191
if (
judge &&
r.semantic !== undefined &&
options.judgeTarget !== undefined &&
node.label !== options.judgeTarget
) {
// Ancestors only matter through the gates they could raise.
if (!(targetAncestors.has(node.title) && ruleCanGate(r))) continue;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With --judge-target, this continue drops the whole rule for non-target nodes, including its expression, whenever the rule has a semantic part. That contradicts the PR description ("expressions on ancestor nodes now always run"; semantic rules outside the target are skipped only when a model provider is in use).

Example: a rule has both expression and semantic, gatesDescendants is unset, and semantic.severity maps only to warnings. On an ancestor of the target, ruleCanGate(r) is false, so the rule is skipped. Its failing expression never runs and never gates, although a plain validate would have gated the target. On non-ancestor, non-target nodes the expression is skipped too, unlike an expression-only rule.

Fix: keep the rule in applicable and withhold only its semantic part when the node is outside the target or cannot gate. For example, mark the rule as expression-only for that node instead of continue.

@claude

claude Bot commented Oct 16, 2026

Copy link
Copy Markdown

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

@mindsocket
mindsocket force-pushed the rule-shape branch 2 times, most recently from 6eaf54e to 86e98fa Compare October 16, 2026 04:34
Base automatically changed from issue-155 to main October 16, 2026 04:36
Replace rule fields check/mode/rubric/verdicts/scope with expression, a
semantic object (rubric, verdicts, severity), and global: true. A rule has an
expression, a semantic part, or both; the semantic part is skipped when the
rule's own expression fails. Severity is top-level for the expression and the
default fail verdict; semantic.severity maps the other verdicts. Old-shape
schemas fail to load with a pointer to the replacement field.

Regenerate the latest metaschema, migrate bundled schemas and fixtures, and
drop the mode dimension from --rule-filter.
Update rules, semantic rules, config, concepts, schemas docs, README,
AGENTS.md and plugin skill text for expression, semantic, global,
semantic.severity, --semantic [provider] and the semantic config key.
@mindsocket
mindsocket merged commit 420ad32 into main Oct 16, 2026
2 checks passed
@mindsocket
mindsocket deleted the rule-shape branch October 16, 2026 06:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant