Skip to content

perf(explore): cover name lookups with (kind, name); size budgets from cheap counts - #258

Merged
bompus merged 1 commit into
fork/consolidatedfrom
perf/explore-sql
Sep 28, 2026
Merged

bompus merged 1 commit into
fork/consolidatedfrom
perf/explore-sql

Conversation

@bompus

@bompus bompus commented Sep 28, 2026

Copy link
Copy Markdown
Owner

Follow-up to the ledger §5.98 profile, which found most of an explore call's time in two statements: the name LIKE lookups and the per-call edge count.

  • idx_nodes_kind becomes (kind, name) (schema v17, migration guarded for fixtures without a nodes table). The substring lookups already walked this index for kind IN (...) but read every full row to test name; now the scan stays in the index.
  • findNodesByNameSubstring orders ties by length(name), kind, rowid. That was the implicit order under the old index. Leaving it to the plan changed one of ten trezor-suite answers (two RPC files replaced by UI files), so it is now explicit. A new test in db-perf.test.ts fails without it.
  • Explore, its budget note and the tool list read getSizeStats() (node and file counts, files per language) instead of getStats(), which also grouped every node and edge by kind on each call.

Measured on trezor-suite (151k nodes, 387k edges), recorded as ledger §5.99:

Before After
One LIKE lookup 7–10 ms 1–2.4 ms
10-question explore round median, paired runs 5,884 / 6,551 ms 5,033 / 5,070 ms
Explore output byte-identical, v16 and v17 index
Index build / size 35–38 ms 66–70 ms, 5.3 MB of 513 MB

Checks: full suite 6,125 passed, 32 skipped; tsc --noEmit; check:agent-docs; readme-sync.

README rows checked: the "About this fork" table and measured sections. No row quotes single-call explore latency, so none changed. The Bun concurrent-explore line (3.87 s vs 2.65 s) will shift a little for both runtimes; it was not re-measured and is due for a re-run with the pending Bun fixes anyway.

…m cheap counts

idx_nodes_kind becomes (kind, name) in schema v17, so explore's name LIKE
scans read the index instead of every row of the kind. The substring query
now states its old tie order (length, kind, rowid) so answers do not move
with the index shape. Explore and the tool list size their budgets from
getSizeStats() instead of getStats(), which grouped every edge by kind.

trezor-suite: one lookup 7-10 ms -> 1-2.4 ms; 10-question explore round
about 15% faster; output byte-identical.
@bompus
bompus merged commit bbc622a into fork/consolidated Sep 28, 2026
@bompus
bompus deleted the perf/explore-sql branch September 28, 2026 07:54
bompus added a commit that referenced this pull request Oct 29, 2026
…debt + explore fixes (#585)

* fix(kernel): port upstream's remaining JS binding-scope and ambient resolution debt

Three pre-existing resolver suites go green:

- js-receiver-binding-scope (colbymchenry#2589): a bound receiver whose binding says
  nothing falls through to the name strategies — an unannotated parameter
  (`(root)`, `{ root }: { root: any }`) or a `new C()` whose C is no class.
  Annotated params (`store: any`) and typed/call-result declarations stay
  terminal under the claim model. Strategy 3 now also filters React
  lifecycle methods on Component-derived classes for non-this receivers.
- dynamic-import-binding (colbymchenry#2491): isOutOfRepoBinding + innerScopeShadows —
  the narrowest in-scope import binding (block-scoped dynamic rows win over
  file-wide statics) decides the name's module; when it's outside the repo
  only same-file declarations nested in a function still shadow it.
- ambient-module-imports: the resolveOne ambient gate — a declaration
  inside `declare module 'x'` is reached only through an import of `x`,
  never by bare name from a file that reaches no `x`.

Also in the batch: MEMBER_TYPE_NON_TYPES verbatim (Java `permits`/`throws`),
and a `fn_ref` outcome channel so paren-less Pascal method pointers keep
their fnRef edge metadata.

Precision: vite 8/8, svelte 1/1, jq clean; golden dumps unchanged.

* fix(explore): honor the fork's explicit substring ordering and bound long covering-test ranges

- name-substring-index: the fork (#258) made result order explicit —
  (length, kind, rowid) — and reshaped idx_nodes_kind for it. Keep
  upstream's memoized substring index but order its hits by that contract
  instead of simulating whichever plan SQLite picked; the plan-detection
  machinery goes with it.
- explore-source-ranges/tools: a covering test longer than
  WHOLE_TEST_MAX_LINES emitted its whole span and truncated before the
  requested call+assertion. Long callbacks now emit their header plus the
  statement ranges, and sourceMinimums reserves every no-nodeId range of a
  covering test, not just the first.

* perf(kernel): share Go declaration walks across a ref's resolution

The Go receiver path re-ran the same ancestor scans at the same site:
go_written_qualifier and resolve_go_cross_package each shadow-checked the
import root with a full local_at walk; go_asserted_local_type ran twice per
ref (once in go_asserted_member, again inside go_named_local_written); and
go_named_local_written repeated the local_declaration_in ancestor walk
go_local_declaration_at had just done for the same name+site.

- resolve_go_cross_package skips its is_shadowed_import when
  go_ref_qualifier already proved the receiver is the written qualifier
- go_local_decl_memo (keyed file+name+site byte) shares the local_decl_at
  walk — its value keeps the target identifier's byte range so
  go_named_local_written re-derives the node instead of re-walking
- go_asserted_type_memo dedupes the asserted-type scan

kernel-prof on the gin corpus, calls|go nonbare refs: ~261 -> ~206 ms
(-21%), bound_receiver_claim 208 -> ~165 ms. Precision identical
(8417 edges, 3/3 held); golden dumps and all 459 Go tests green.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant