Skip to content

feat(judge): per-node input cap, body trimming and missing-context - #166

Merged
mindsocket merged 6 commits into
mainfrom
issue-152-trim
Oct 16, 2026
Merged

mindsocket merged 6 commits into
mainfrom
issue-152-trim

Conversation

@mindsocket

@mindsocket mindsocket commented Oct 16, 2026 •

Copy link
Copy Markdown
Owner

Closes #152. Stacked on #165.

Summary

  • Per-node input cap: 32k estimated tokens per evaluated node, counting the node, its related nodes and bodies (characters ÷ 4). It is the same for every model provider, and fixed.
  • Trimming cuts bodies only, never fields, and never drops a node.
    • Related bodies come down to one shared length, largest first, while shorter bodies stay whole.
    • The evaluated node's own body is cut only after all related bodies are used up.
    • Each cut ends at a whitespace and leaves [RULE_EVALUATION_CONTENT_TRIMMED <n> chars].
  • Still over the cap after trimming: the provider is not called. The rule reports (Semantic evaluation error: input over the per-node cap after trimming content) for that node.
  • missing-context: a reserved verdict, offered on every rule with semantic.scope or includeContent. The model is told to use it only where a trim marker shows content was cut. Its default severity is warning, and semantic.severity can override it.
  • Reporting: sctx validate prints "Context trimmed for N node(s) across rules X, Y." The result carries contextTrimmed too. This is an additive public API field on ValidationResult, agreed.
  • Naming: "per-node input cap" and "request packing limit" are added to docs/concepts.md. The Jev constants are renamed to JEV_REQUEST_PACKING_LIMIT_*.

Tests

lint and typecheck clean; bun run test 779 pass, 15 skip, 0 fail. One shared test checks that all three renderers show the relation, the trim marker and missing-context. No live test yet sends a scoped, over-cap node to a real provider.

@mindsocket
mindsocket added this pull request to stack #167 October 16, 2026 03:55
@mindsocket mindsocket changed the title issue 152 trim feat(judge): per-node input cap, body trimming and missing-context Oct 16, 2026
@mindsocket
mindsocket force-pushed the issue-152-trim branch 4 times, most recently from 1644ab5 to 3c68fbd Compare October 16, 2026 04:20
@mindsocket
mindsocket marked this pull request as ready for review October 16, 2026 05:31
@claude

claude Bot commented Oct 16, 2026

Copy link
Copy Markdown

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

…rdict

Cap one node's input (node plus related nodes) at 32k estimated tokens for
every provider. Over the cap, cut related bodies by water-filling, then the
evaluated node's body, ending each cut at whitespace with a
[RULE_EVALUATION_CONTENT_TRIMMED n chars] marker. Input still over the cap
reports a semantic evaluation error and skips the provider. Rules with
semantic.scope or includeContent offer missing-context (warning by default).
The validate summary and JSON report trimmed nodes and rules.

Rename the Jev packing constants to request packing limit.
Adds a test that scans README, AGENTS, CLAUDE, top-level docs and plugin
markdown for backticked repo paths and fails on any that do not resolve.
Lines containing an example marker (e.g., eg, example) are skipped, as are
placeholders and gitignored paths. Fixes the four stale paths it found.
Drop request grouping, packing and trimming mechanics from config.md and
semantic-rules.md. Keep what a user acts on: that grouping saves cost and can
shift verdicts, the per-node input cap, the trim marker and the errors.
Base automatically changed from issue-152-scope to main October 16, 2026 06:03
@mindsocket
mindsocket merged commit b52b5d4 into main Oct 16, 2026
2 checks passed
@mindsocket
mindsocket deleted the issue-152-trim branch October 16, 2026 06:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Semantic rules: rule scope and judge input (what the model sees)

1 participant