Skip to content

Fix failing hook tests and explore git-snapshot baseline for plugin hooks #124

Closed

Description

@mindsocket

Problem

bun run test:hook has 7 failing tests, in hook-test/unit/pre-edit.test.ts, hook-test/unit/on-stop.test.ts and hook-test/hooks.test.ts. They also fail on main. CI doesn't catch this because bun run test only covers tests/.

Hypothesis (not checked): the test fixture isn't matched to a space through configPath, so validate-file comes back with inSpace: false and no errors. Cheapest check: log sctx's stdout and stderr inside runPreEdit during a failing test.

Tasks

  • Find the cause and fix the 7 failures
  • Add bun run test:hook to CI, and possibly to the pre-push lefthook check
  • Explore the git-snapshot approach described below

Explore: how the security-guidance plugin tracks changes

Anthropic's security-guidance plugin (claude-plugins-official, v2.x) doesn't use our PreToolUse approach, where the hook validates each file before editing it and keeps a JSONL baseline. Instead it works out what changed from git:

  • It records a baseline HEAD SHA for each session, plus the paths touched in that session.
  • At Stop, it diffs against that baseline. It uses a temporary index with git add --intent-to-add, so untracked files show up in the diff without touching the user's real index (hooks/gitutil.py: _temp_index, get_git_diff).
  • Its Stop and PostToolUse hooks use asyncRewake: true with a rewakeMessage. The review runs in the background and wakes Claude when it finds something, rather than making Claude wait at Stop.
  • It keeps the Stop-loop count and already-reported findings in state, with a TTL, so it doesn't report the same finding twice.

Questions to answer:

  • Could the baseline for an edited file come from git (git show <baseline>:path, validated in a temp dir) instead of running validate-file before every Edit? That would remove the PreToolUse hook and its cost on each edit.
  • Would it also catch changes made outside Write/Edit, such as Bash sed or scripts?
  • How should it behave outside a git repo, for example in Obsidian vaults that aren't under git?
  • Does asyncRewake suit validation, given that validation is fast and deterministic?

Activity

  1. added a commit that references this issue on Sep 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions