Repository navigation
fix(graph): impact reads what the budget can keep, and reads it once (#3248) - #3285
Open
Marcos-199-GitHub wants to merge 2 commits into
Open
Marcos-199-GitHub wants to merge 2 commits into
Marcos-199-GitHub wants to merge 2 commits into
Conversation
…olbymchenry#3248) colbymchenry#3125 gave impact traversal a node and edge budget, and fetched each visited node's incoming edges as `kind IN (<every kind but contains>) ... LIMIT ?`. Both halves slowed the read the walk makes once per visited node: the kind list searches the target index once per kind, and the LIMIT under the read's ORDER BY made it slower still. getImpactRadius took 2-3x as long for the same subgraph. The walk now reads `WHERE target = ? AND kind != 'contains'` with no LIMIT, through a cached statement, and the slice applies the budget as before. A container's members are read without a LIMIT too, for the same reason. A contains edge still never takes a budget slot or marks the result truncated, and the output is unchanged: the same nodes, edges, order and truncation as main on 20,000 random graphs and 500 traversals of codegraph's own index. Fixes colbymchenry#3248
…olbymchenry#3248) The walk read every dependent a node has and sliced the list afterwards, so the read was not bounded by the budget: a walk whose budget ran out still paid a full read for every node it visited, and a node reached again from a nearer depth was read again. An edge of a kind the graph does not know also counted as a dependency, and the limit the query builder took for its two edge reads had no caller left. An index that carries the order the read asks for -- (target, kind, source, line, col) -- makes a bounded read as cheap as a whole one, because the rows come back in that order and no sort stands between the read and the rows. The read asks for the budget's share and the one row that tells a full result from a cut one; a spent budget asks for a single row rather than a list; what a node's expansion read is kept, so a node reached again from a nearer depth is read once; and an edge of a kind the graph does not know is not a dependency, read through the same list of kinds as before. The two edge reads that answer the same question share one order, so they cannot drift apart. The output does not change: the same nodes, edges, order and truncation on 2,640 walks of this repository's own index, on 300 random graphs of up to 13 nodes and 59 edges with cycles and self-loops, and on 116,256 walks of those graphs under seven budgets. Measured (codegraph's own source indexed by each build, Linux, Node 22): | build | impact, default budget | impact, maxEdges 10 | | --- | --- | --- | | main | 9.0-10.0 ms | 0.1-0.2 ms | | the branch above | 5.0-6.2 ms | 0.8-1.3 ms | | this change | 5.5-5.9 ms | 0.1 ms | The index costs 8.6 MB on an index of 37k nodes and 86k edges (6% of the database), 741 ms to build once, and 2.6-3.1% of an index run.
This was referenced Nov 17, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #3249 by @inth3shadows (Eric Minish): that pull request restored the read #3125 broke, and this branch keeps its commit unchanged and adds the rest — the read bounded by the budget, the index that makes a bounded read as cheap as a whole one, and the limits #3249 left as known ones.
The two pull requests overlap by one commit. Whichever the maintainer merges first, the other follows as a clean single-commit change.
What
Impact traversal reads each visited node's dependents in one query, bounded by what the budget can keep, and reads a node once however many depths reach it.
#3249 restored the read #3125 broke — one index search per visited node instead of one per kind — but it left the read unbounded: a walk whose budget ran out still paid a full read for every node it visited, and a node reached again from a nearer depth was read again.
How
src/db/schema.sql,src/db/migrations.ts— a new index,idx_edges_target_kind_source ON edges(target, kind, source, line, col), carries the order the read asks for. The rows come back in that order, and no sort stands between the read and the rows — which is what makes a bounded read as cheap as a whole one: 10.4 µs against 10.3 µs over 2,000 reads of small nodes, and 24.6 ms against 41.3 ms over the 200 busiest nodes. Migration v17 adds it on existing databases.src/db/queries.tsgetIncomingEdgesExceptContains(targetId, limit)reads the dependents of a node, at mostlimitof them.getFirstDependentEdge(targetId)andhasContainedMembers(sourceId)answer "does anything depend on this?" with one row, for a walk whose budget is already spent.INCOMING_EDGE_ORDER, so they cannot drift apart.limitparametergetIncomingEdgesandgetOutgoingEdgestook is gone: no caller passed it. It never shipped — fix(graph): impact stops at a safety budget and says it was truncated #3125, which added it, is unreleased.src/graph/traversal.ts__tests__/graph.test.ts— the read is bounded by the budget; a spent budget probes instead of reading; a node reached from two depths is read once; an edge of a kind the graph does not know is not a dependency; the index that carries the order is the one the read uses.The output does not change
The same nodes, edges, order and truncation as
main(f72c9d2e):maxEdges: 0andmaxNodes: 1— byte for byte identical (the same SHA over the whole digest).Measured (codegraph's own source indexed by each build, Linux, Node 22)
maxEdges: 10f72c9d2e(main)The whole suite passes: 867 test files, 10,526 tests, 0 failures.
tsc --noEmitexits 0.What the index costs
Compatibility
No re-index needed: the migration adds the index on the next writable open. Until then the read is correct, and slower than with the index.