diff --git a/CHANGELOG.md b/CHANGELOG.md index 9f3a240908..816efbab64 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -217,7 +217,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). - In a JavaScript or TypeScript monorepo, imports between packages now reach the right file. Each package's own `tsconfig.json` or `jsconfig.json` `paths` apply to its files, so `@/logger` in one app no longer lands in another app's `src/`. A workspace package whose `main`, `module` or `exports` point at a `dist/` folder that isn't committed now resolves to the source file that builds it. Calls through those imports used to fall back to matching the name anywhere in the project. -- `codegraph_explore` answers about 15% faster on large projects. Its name lookups read a narrower index instead of every matching row, and it no longer counts every edge by kind on each call to size its answer. An existing index is upgraded the next time it is opened for writing, such as by `codegraph sync`. +- `codegraph_explore` answers about 15% faster on large projects. Its name lookups read a narrower index instead of every matching row, and it no longer counts every edge by kind on each call to size its answer. Its full-text search now ranks matches before reading their rows, which takes less than half the time it did. An existing index is upgraded the next time it is opened for writing, such as by `codegraph sync`. - A one-file sync is faster on large JavaScript and TypeScript projects: codegraph remembers which files each inference step and the framework check found nothing in and skips them until they change, and reads only the symbols each step can use. Checking the project's files against `.gitignore` and preparing source text for the inference steps also take less time. Syncs also use less memory. - Nuxt page routes now link to the page component they render, and `index` folders no longer leak into the URL (`pages/account/index/[page].vue` is `/account/:page`). diff --git a/docs/design/metrics-ledger.md b/docs/design/metrics-ledger.md index 41a01ce94c..9f5bffe41c 100644 --- a/docs/design/metrics-ledger.md +++ b/docs/design/metrics-ledger.md @@ -829,6 +829,17 @@ Now two programming languages that cannot name each other's symbols never bind b | `eval:precision` javalin | 1/1 absent, 1/1 present held (Kotlin → Java member import kept) | | Kernel/TS resolve parity, bridge suites (RN, Expo, Swift/ObjC, cross-tier) | pass | +### 5.100 Explore full-text search: rank first, read rows for the page (2026-09-28) + +Follow-up to §5.99. `searchNodesFTS` joined `nodes` on `nodes_fts.id = nodes.id` and sorted every match with all its columns by `bm25`. `id` in an external-content FTS table is read back from `nodes`, and the join then looked it up again through the `id` index. The query now ranks `(rowid, score)` in a subquery, joined by rowid for the kind and language filters, and reads full rows only for the page it returns. Ties order by rowid, which the single-query form also gave. + +| trezor-suite (151k nodes) | Before | After | +|---|---|---| +| 8 FTS queries, explore's 21-kind filter, limit 100 | 84.3 ms | 36.7 ms | +| Same, 8 definition kinds | 69.9 ms | 24.3 ms | +| 10-question explore round median, paired runs | 4,776 / 4,797 ms | 4,469 / 4,558 ms (about −6%) | +| Explore output | | byte-identical on all 10 questions | + ### 5.99 Explore SQL: covering index for name lookups, cheap size counts (2026-09-28) Follow-up to §5.98's per-statement profile. Two changes: diff --git a/src/db/queries.ts b/src/db/queries.ts index 8b30f2ebff..5da97fcab5 100644 --- a/src/db/queries.ts +++ b/src/db/queries.ts @@ -1777,11 +1777,17 @@ export class QueryBuilder { // nameMatchBonus) can promote results that BM25 alone undervalues. const ftsLimit = Math.max(limit * 5, 100); + // Rank on (rowid, score) alone and read full rows for the page only: sorting + // every match with all its columns cost 2-3x as much on a 151k-node index. + // Joining by rowid skips the content-table read of `id` and its index lookup. + // Ties keep FTS rowid order, as the single-query form gave implicitly. let sql = ` - SELECT nodes.*, bm25(nodes_fts, 0, 20, 5, 1, 2) as score - FROM nodes_fts - JOIN nodes ON nodes_fts.id = nodes.id - WHERE nodes_fts MATCH ? + SELECT nodes.*, ranked.score + FROM ( + SELECT nodes_fts.rowid AS rid, bm25(nodes_fts, 0, 20, 5, 1, 2) as score + FROM nodes_fts + JOIN nodes ON nodes.rowid = nodes_fts.rowid + WHERE nodes_fts MATCH ? `; const params: (string | number)[] = [ftsQuery]; @@ -1796,7 +1802,12 @@ export class QueryBuilder { params.push(...languages); } - sql += ' ORDER BY score LIMIT ? OFFSET ?'; + sql += ` + ORDER BY score, rid LIMIT ? OFFSET ? + ) ranked + JOIN nodes ON nodes.rowid = ranked.rid + ORDER BY ranked.score, ranked.rid + `; params.push(ftsLimit, offset); try {