Skip to content

gh-158860: Make minidom Node.normalize() linear in adjacent text nodes - #158861

Open
jonbaldie wants to merge 1 commit into
python:mainfrom
jonbaldie:perf-minidom-normalize
Open

jonbaldie wants to merge 1 commit into
python:mainfrom
jonbaldie:perf-minidom-normalize

Conversation

@jonbaldie

@jonbaldie jonbaldie commented Oct 5, 2026 •

Copy link
Copy Markdown

Summary

Node.normalize() merges adjacent text nodes with node.data = node.data + child.data, once per absorbed node. A run of n text nodes copies the growing string n times, so it's quadratic.

This collects the pieces for each run and joins them once at the end:

 for child in self.childNodes:
     ...
     elif L and L[-1].nodeType == child.nodeType:
-        node.data = node.data + child.data
+        runs[-1][1].append(child.data)   # or start a new run for node
     ...
+for node, data in runs:
+    node.data = ''.join(data)
 self.childNodes[:] = L

Sibling links are still fixed up inline, as before. runs is only created when something actually merges.

Parsing doesn't hit this, because expatbuilder already merges character data into the previous text node. You only get long runs from documents built through the DOM API.

Evidence

One element with n text-node children, 3.16 dev build, macOS arm64:

case before after
n=8,000, 48 chars each 24–26 ms 1.9 ms
n=16,000 89–105 ms 3.8 ms
n=32,000 427–469 ms 7.9 ms
n=100,000, 16 chars each 1,054–1,243 ms 24 ms

New test testNormalizeManyTextNodes (100k nodes, with an empty one every third node) takes 1.35 s before and 0.18 s after. Like the other complexity tests, it's slow-before / fast-after, not a hard timing assertion.

Differential check against the old normalize(): 20,000 random DOMs (text, empty text, elements, comments, nested). toxml() output and every parentNode / previousSibling / nextSibling link are identical.

test_minidom, test_pulldom, test_xml_dom_minicompat: 179 run, all pass.

Merge Danger

Door: two-way

Same result and same links. Only how the merged string gets built changes.

Blast Radius: a small slowdown when nothing merges

Normalizing an already-normal parsed document (200 items, 601 normalize() calls) went from about 102 µs to about 106 µs, measured over 10 alternating runs in both orders. That's the cost of the extra runs is None check, roughly 4 ns per call.

AI disclosure: the profiling and the patch were done with AI tools (Codex, Claude).

🤖 Generated with Claude Code

…t nodes

Merging each text node with `node.data + child.data` copies the growing
string once per absorbed node. Collect the pieces and join each run once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread Lib/test/test_minidom.py
, "testNormalizeDeleteAndCombine -- result")
doc.unlink()

def testNormalizeManyTextNodes(self):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test doesn't assert anything, so it doesn't really verify the change. I think it's hard to write a test for this, so it may be fine to just remove it.

I've run the benchmark locally (debug build, one element with n text nodes of 48 chars each):

n before after
8,000 2.8 s 18 ms
16,000 11.3 s 37 ms
32,000 45.6 s 75 ms
64,000 181 s 148 ms

Comment thread Lib/xml/dom/minidom.py
L = []
# (text node, [data, ...]) for each text node that absorbs others.
# Join each run once at the end; concatenating as we go is quadratic.
runs = None

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can initialize it here, then the None guards at L214 and L230 can be removed:

Suggested change
runs = None
runs = []

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants