Skip to content

perf(resolution): a full index detects frameworks once, and remembers which paths are tests - #3151

Merged
colbymchenry merged 1 commit into
mainfrom
claude/spd-detect-once
Oct 31, 2026
Merged

colbymchenry merged 1 commit into
mainfrom
claude/spd-detect-once

Conversation

@colbymchenry

Copy link
Copy Markdown
Owner

Every resolver worker detected the project's frameworks for itself as it
started, and a detector may read every source file: Spring's reads each .java
until it finds an annotation, so in a Java project without Spring, six workers
each read the whole project. A full index has just detected them on the same
files in the main thread, so it now hands the workers the names. A sync, where
the main thread's answer may be older, leaves them to detect as before.

isTestPath runs seven patterns over a path, and resolving asks about the same
paths for every reference and candidate. It now remembers the answer, up to
100,000 paths.

elasticsearch subset, 1,710 files: self time in detection 4.4 s to 2.3 s, in
isTestPath 1.4 s to under 0.1 s, out of about 50 s of CPU.

Validation

Adds resolver-shared-frameworks.test.ts: the names give what detection gives, in the same order. A sync passes nothing, so workers detect as before there. Framework resolvers keep no state of their own (detect only reads, and their memos are per context).

  • Graphs: scripts/dump-graph.mjs dumps byte-identical to main on 14 repos: tokio, gin, flask, jsoup, excalidraw, serilog, ripgrep, taxonomy, and subsets of django, rocksdb, elasticsearch (two), rails and laravel. That's with all of this stack's commits in, on one build.
  • Full suite on the stack tip 38c97bd0, which holds all nine commits of this work on a0174ce (kernel built from that tree, CODEGRAPH_KERNEL_EXPECT=1, 4 workers, Vitest 4): 848 files passed, 6 skipped; 10,515 tests passed, 2 expected fails, 87 skipped, 0 failed.
  • Numbers are interleaved on a machine other agents were using, so read the ratios. REG's quiet-machine run gives the final ones.

Part of the work to bring main back to 1.6.2's speed. Independent of the other PRs of it: it compiles alone on main.

🤖 Generated with Claude Code

… which paths are tests

Every resolver worker detected the project's frameworks for itself as it
started, and a detector may read every source file: Spring's reads each `.java`
until it finds an annotation, so in a Java project without Spring, six workers
each read the whole project. A full index has just detected them on the same
files in the main thread, so it now hands the workers the names. A sync, where
the main thread's answer may be older, leaves them to detect as before.

isTestPath runs seven patterns over a path, and resolving asks about the same
paths for every reference and candidate. It now remembers the answer, up to
100,000 paths.

elasticsearch subset, 1,710 files: self time in detection 4.4 s to 2.3 s, in
isTestPath 1.4 s to under 0.1 s, out of about 50 s of CPU.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@colbymchenry
colbymchenry merged commit 1b0e351 into main Oct 31, 2026
@colbymchenry
colbymchenry deleted the claude/spd-detect-once branch October 31, 2026 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant