Repository navigation
Look up parents in a map when computing metrics - #1259
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1259 +/- ##
==========================================
+ Coverage 70.25% 70.34% +0.09%
==========================================
Files 62 62
Lines 16325 16414 +89
==========================================
+ Hits 11469 11547 +78
- Misses 4856 4867 +11 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
tree-sitter doesn't store parent pointers: `tree_sitter::Node::parent` walks down from the root of the tree, so it costs O(depth). Some metrics walk up through all the ancestors of a node (e.g. the C++ LOC metric for every declaration), which then costs O(depth²) per node. On very deep trees, like the ones of some generated files in mozilla-central, computing the metrics of a single file took hours. `metrics` now computes the parent of each node once, with a single cursor pass over the tree, and the nodes it visits look their parent up in that map. Nodes created elsewhere keep using tree-sitter's lookup. Since `Node` now has a private field, it can no longer be constructed directly outside of the crate; `From<tree_sitter::Node>` is provided instead. Fixes #1258.
This reverts commit ac94f0c. With parents looked up in a map, computing the metrics of security/manager/ssl/nsSTSPreloadList.inc and third_party/wasm2c/src/prebuilt/lexer-keywords.cc takes seconds instead of hours, so they no longer need to be excluded.
marco-c
force-pushed
the
fast-parent-lookup
branch
from
November 20, 2026 00:58
8b112d0 to
21f1c20
Compare
The parent map only contains the nodes below the root it was computed from. A ParserTrait implementation can return a subtree from get_root(), and then the ancestors above it were no longer found: e.g. with the arrow function of `const f = () => 1` as root, the JavaScript checker didn't see the enclosing variable declarator and counted a closure instead of a function. On a miss, look the parent up with tree-sitter instead. For the root of the whole tree, that returns immediately.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1258.
Problem
tree-sitter doesn't store parent pointers:
tree_sitter::Node::parentwalks down from the root of the tree, so it costs O(depth). Some metrics walk up through all the ancestors of a node with repeatedparent()calls (e.g. the C++ LOC metric, for every declaration), which costs O(depth²) per node. On very deep trees, computing the metrics of a single file takes hours. That's what made the "check tree-sitter-mozcpp" CI task always hit its deadline: two generated files in mozilla-central,security/manager/ssl/nsSTSPreloadList.inc(tree ~6,900 levels deep) andthird_party/wasm2c/src/prebuilt/lexer-keywords.cc, never finished.Fix
metrics()now computes the parent of each node once, with a single cursor pass over the tree (Parents).Nodegets a private, optional reference to that map, propagated to every node reached from it (children, siblings, cursors), andNode::parent()looks the parent up there. Nodes created without it, outside ofmetrics(), keep using tree-sitter's lookup, and so do nodes missing from the map: aParserTraitimplementation can return a subtree fromget_root(), and its ancestors must still be found (for the root of a whole tree, tree-sitter's lookup returns immediately). Soparent()always returns the same node as before.Node's public field (node.0) is unchanged, but sinceNodenow also has a private field, it can't be constructed directly outside of the crate anymore.From<tree_sitter::Node>is provided instead.Results
Measured with release builds on a local mozilla-central checkout (20 cores):
third_party/wasm2c/src/prebuilt/lexer-keywords.ccsecurity/manager/ssl/nsSTSPreloadList.incdom/C/C++ files (5,964), CPU timedom/, peak RSSThe outputs are byte-identical to master's for all 41,970 files that master can process, and the two files above now produce their metrics too.
Tests
node::tests::parents_match_tree_sitter: the map gives the same parent as tree-sitter for every node of a C++ sample.node::tests::parents_above_subtree_root: with the arrow function ofconst f = () => 1;as root, its parent is still the variable declarator, and the JavaScript checker still counts it as a function rather than a closure.node::tests::metrics_on_deep_tree: 500 nested namespaces, each with a declaration (25 s with master in release, 0.2 s with this PR). The test doesn't assert on time, to avoid flakiness, but without the fix it takes ~10× longer even at this size.Notes
Node::has_siblingiterates over the children of the node itself rather than those of its parent, so it behaves likeNode::is_child. This PR keeps that behavior unchanged; it probably deserves a separate look.