Skip to content

Hooks: whole-space baseline at prompt submit, compared at Stop (#124, #127) - #126

Merged
mindsocket merged 9 commits into
mainfrom
fix/124-hook-tests
Sep 27, 2026
Merged

mindsocket merged 9 commits into
mainfrom
fix/124-hook-tests

Conversation

@mindsocket

@mindsocket mindsocket commented Sep 27, 2026 •

Copy link
Copy Markdown
Owner

Closes #124. Closes #127.

Failing tests and CI (#124)

These were already fixed on main shortly after the issue was filed:

  • 6dc8c6d fixed the failures: the fixture config had no schema after the default-schema fallback was removed.
  • 49d390c added bun run test:hook to CI (test.yml) and to pre-push (check-pushed-tree.sh).

Whole-space baseline at prompt submit (#127)

The hooks used to validate per file: one full-space read per edited file before the edit and again at Stop, reporting only issues in the edited file. The first commit fixed two baseline bugs in that design (a second edit hid the first edit's errors; Write over an existing file reported every existing issue as new). The second commit replaces it:

  • UserPromptSubmit (on-prompt.ts): baselines every space inside or containing the session directory. When cwd is the vault, that's every space in it.
  • Stop (on-stop.ts): validates each space that changed since its baseline and reports issues missing from the baseline, in any file of the space. That catches breakage in files that weren't edited (e.g. renaming a note breaks another file's parent link) and edits made through Bash. Errors block and warnings go to additionalContext, as before. Info-level rule violations are still ignored.
  • PreToolUse (pre-edit.ts): now a fallback for spaces outside the session directory. The first edit to such a space in a turn baselines the whole space.
  • SessionEnd (on-session-end.ts): removes the session's state file.

Details:

  • Baselines store issue keys only. Messages are only needed for new issues at Stop.
  • Each space has a hash of the paths, sizes and modification times of its .md files, schema-directory files and config files. Stop skips spaces whose hash matches their baseline, and the next prompt reuses the last Stop's keys when the hash still matches.
  • If a turn is interrupted before Stop, the next prompt keeps the existing baseline, so those changes are still compared.
  • Overlapping spaces report each issue in a file once.
  • The hooks call sctx validate <space> --json and a new sctx spaces --json (config files, and each space's path and schema). There's no new validation command.

Measured with the eight spaces under the vault:

Time
Prompt, first in the session 3.3s
Prompt, unchanged since last Stop 0.13s
Stop, nothing changed 0.13s

The state file is 60KB, almost all of it unresolved content-link keys (628 of 635 issues). #128 covers letting links outside a space validate without warnings.

CLI: piped output past 64KB

validate --json for a large space was cut off at 65,536 bytes when the reader was slow, which would have broken the hooks.

  • measured: once node:process is imported (here through unified → vfile), Bun's console.log drops piped output past 64KB. A bare script printing the same output doesn't lose any, and process.stdout.write delivers all of it.
  • src/index.ts now routes console.log through process.stdout.write, and validate / validate-file set process.exitCode instead of calling process.exit().

Tests

  • Hook unit tests (29) drive the real flow: a prompt baseline, file changes, then Stop. They cover edits in files that weren't edited, Bash-style renames, interrupted turns, reuse of cached results, overlapping spaces and warnings.
  • e2e 4/4 (with the UserPromptSubmit hook in the harness), bun run test 461, lint and typecheck all pass.

https://claude.ai/code/session_01EJgagmpxDYVYr7fp9C6cZi

The Stop hook kept the latest pre-edit entry per file, so a second edit to
the same file took a baseline that already contained errors the first edit
introduced, and they went unreported. It now keeps the earliest entry, and
the pre-edit hook skips files it has already recorded, which avoids a
full-space validation on every repeat edit.

A Write over an existing file was recorded with no baseline, so all of the
file's existing issues were reported as new. It now records a baseline like
an Edit.

Claude-Session: https://claude.ai/code/session_01EJgagmpxDYVYr7fp9C6cZi
The UserPromptSubmit hook baselines every space inside or containing the
session directory; the Stop hook validates changed spaces and reports
issues missing from the baseline in any file, so breakage in files that
weren't edited and edits made through Bash are caught. The pre-edit hook
remains as a fallback for spaces outside the session directory.

Baselines store issue keys only. A hash of each space's markdown, schema
and config file stats skips validation when nothing changed, and the last
Stop's keys are reused at the next prompt. A baseline that survives an
interrupted turn is kept.

The hooks use `sctx validate --json` and the new `sctx spaces --json`.
The CLI now routes console.log through process.stdout.write: once
node:process is loaded, Bun dropped piped output past 64KB for slow
readers.

Claude-Session: https://claude.ai/code/session_01EJgagmpxDYVYr7fp9C6cZi
@mindsocket mindsocket changed the title Hooks: compare against each file's first baseline; git-snapshot findings (#124) Hooks: whole-space baseline at prompt submit, compared at Stop (#124, #127) Sep 27, 2026
The Agent SDK loads ~/.claude settings and CLAUDE.md when settingSources is omitted.

Claude-Session: https://claude.ai/code/session_01EJgagmpxDYVYr7fp9C6cZi
Comment thread docs/use-cases.md Resolved
On Linux the fixture space lives directly under tmpdir(), so it counted as
related and a baseline was written. Use a fresh sibling directory instead.

Claude-Session: https://claude.ai/code/session_01EJgagmpxDYVYr7fp9C6cZi
Repository owner deleted a comment from claude Bot Sep 27, 2026
Repository owner deleted a comment from claude Bot Sep 27, 2026
@claude

claude Bot commented Sep 27, 2026

Copy link
Copy Markdown

Code review

1 issue found (CLAUDE.md documentation compliance):

  • docs/use-cases.md#L63 and docs/use-cases.md#L94 still describe the old per-file "hook on .md save" / "hooks for save-time checks" design (e.g. Validation hook on \.md` saveandhooks for save-time checks). This PR replaces that with whole-space baselining at UserPromptSubmitand comparison atStop (plugin/scripts/pre-edit.tsno longer validates on save, only baselines;plugin/scripts/on-stop.ts` does the compare). The equivalent wording elsewhere in the same file (line 21) was already updated to "plugin hooks report issues introduced during a Claude turn," but these two rows were missed.

    Per CLAUDE.md's "Definition of done": "docs/ - includes architecture, concepts etc"* should be reviewed/updated with any new feature or change — these two rows are outside the diff's hunk range so they can't be flagged as inline suggestions, but should be updated to match, e.g.:

    • Line 63: Validation hook on \.md` save→Plugin hooks report issues introduced during a Claude turn`
    • Line 94: hooks for save-time checks → plugin hooks report issues introduced during a Claude turn

No bugs found in the diff.

Two rows still described the old per-file save-time validation hook;
align with the whole-space baseline/compare design already reflected
elsewhere in the file.

Claude-Session: https://claude.ai/code/session_01Vu78JfVBQXdqVtBZUPdu2E
# Conflicts:
#	plugin/.claude-plugin/plugin.json
@mindsocket
mindsocket merged commit a6c0ac9 into main Sep 27, 2026
2 checks passed
@mindsocket
mindsocket deleted the fix/124-hook-tests branch September 27, 2026 10:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Hooks: whole-space validation baseline at prompt submit, compared at Stop Fix failing hook tests and explore git-snapshot baseline for plugin hooks

1 participant