Skip to content

Rule severity levels with category defaults - #120

Merged
mindsocket merged 3 commits into
mainfrom
feat/60-rule-severity
Sep 27, 2026
Merged

mindsocket merged 3 commits into
mainfrom
feat/60-rule-severity

Conversation

@mindsocket

Copy link
Copy Markdown
Owner

Closes #60.

  • Rules take an optional severity (error | warning | info). If it's omitted, the category default applies: validation → error, coherence/workflow → warning, best-practice → info.
  • Only error violations fail validate (exit 1). validate-file reports warning and info violations under warnings.
  • validate --json adds severity to rule errors and a ruleViolations array covering every violation. errorCount now includes global-scope rule errors.
  • schemas show shows category defaults and per-rule overrides.
  • The single place severity is decided is DEFAULT_SEVERITY_BY_CATEGORY / ruleSeverity() in src/schema/validate-rules.ts. It's the hook point for future CLI/config overrides and cascade gating (Layered rule evaluation: cascade gating and coverage annotation #115).

Notes:

  • The pre-commit hook bumped the plugin version to 0.11.1.
  • No bundled rule has the validation category, so none of them is an error by default any more.

https://claude.ai/code/session_01EJgagmpxDYVYr7fp9C6cZi

@claude

claude Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

Claude encountered an error after 3m 19s —— View job


I'll analyze this and get back to you.

@claude

claude Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

Claude encountered an error after 3m 8s —— View job


I'll analyze this and get back to you.

@mindsocket
mindsocket force-pushed the feat/60-rule-severity branch from df4c010 to e04e647 Compare September 26, 2026 23:55
Comment thread src/validate.ts
Comment on lines 376 to 379
if (v.file === label) {
errors[`rule:${v.ruleId}`] = { kind: 'rule', message: `[${v.ruleId}] ${v.description}` };
const target = v.severity === 'error' ? errors : warnings;
target[`rule:${v.ruleId}`] = { kind: 'rule', message: `[${v.ruleId}] ${v.description}`, severity: v.severity };
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CLAUDE.md compliance: plugin/* not updated for this behavior change.

if (v.file === label) {
const target = v.severity === 'error' ? errors : warnings;
target[`rule:${v.ruleId}`] = { kind: 'rule', message: `[${v.ruleId}] ${v.description}`, severity: v.severity };
}

With this change, only severity === 'error' rule violations land in errors; the rest go to warnings. Since no bundled schema rule currently uses the validation category (all are workflow/best-practice/coherence, which default to warning/info), every bundled-rule violation that used to appear in errors now lands in warnings instead.

plugin/scripts/on-stop.ts (line 105: const FRESH_ERRORS = result.errors ?? {};) and plugin/scripts/pre-edit.ts (line 86: const ERRORS = result.errors ?? {};) both read only result.errors to detect newly-introduced rule violations and block the session on new ones. Neither file is touched by this PR, so these hooks will now silently stop flagging newly-introduced bundled-rule violations as session-blocking errors — an unreviewed regression in plugin/*.

CLAUDE.md's "Definition of done" states: "There are several places that need reviewing and updating with any new feature or change added: ... plugin/* - includes skills, commands, hooks, and scripts; update any affected parts"

https://github.com/mindsocket/structured-context/blob/e04e6473070a23ad20b9e37ba5b8848d6cb9fdd7/CLAUDE.md

@claude

claude Bot commented Sep 27, 2026

Copy link
Copy Markdown

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

1 similar comment
@claude

claude Bot commented Sep 27, 2026

Copy link
Copy Markdown

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

Rule violations now carry a severity of error, warning or info. Each
rule category supplies a default (validation -> error, coherence and
workflow -> warning, best-practice -> info), and a rule may set an
optional `severity` field to override it.

Only error severity violations fail validation: `validate` and
`validate-file` exit non-zero and JSON output reports `valid: false`
only for errors. Warning and info violations are still reported.

- Add `severity` to the rule metadata contract and generated metaschema
- Attach effective severity to RuleViolation; export severity types from the API
- validate: per-severity summary counts, details grouped by severity,
  `ruleViolations` array in JSON output
- validate-file: warning/info rule violations go under `warnings`
- schemas show: list category default severity and per-rule overrides
- Update docs, README and plugin skill references

Claude-Session: https://claude.ai/code/session_01EJgagmpxDYVYr7fp9C6cZi
With rule severity levels, rule violations below 'error' go to `warnings`,
which the hooks ignored. The pre-edit hook now also records a warnings
baseline. The Stop hook still blocks with exit 2 for new errors. For new
warnings (except info-level), it returns them as
hookSpecificOutput.additionalContext, so Claude sees them without a hook
error.

Also set a schema in the hook-test fixture config: the hook tests failed
after the default schema fallback was removed.

Claude-Session: https://claude.ai/code/session_01EJgagmpxDYVYr7fp9C6cZi
@mindsocket
mindsocket force-pushed the feat/60-rule-severity branch from e122bce to 14802df Compare September 27, 2026 02:34
@mindsocket
mindsocket merged commit 91b31ef into main Sep 27, 2026
2 checks passed
@mindsocket
mindsocket deleted the feat/60-rule-severity branch September 27, 2026 04:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Rule categories and severity

1 participant