Repository navigation
Neo4j transaction too big #116
Description
Describe the bug
The analyzer creates a huge Neo4J transaction which may conflict with the Neo4J configuration.
To Reproduce
Steps to reproduce the behavior:
- Ran on superset (frontend)
bun run start -- -i ../superset/superset-frontend/ --app-name superset-frontend --emit neo4j --neo4j-uri bolt://localhost:7687 --neo4j-user neo4j --neo4j-password somepassword
Expected behavior
The graph should be ingested without any issues.
Logs
[codeanalyzer-ts] FATAL Neo4jError: The allocation of an extra 2.0 MiB would use more than the limit 2.7 GiB. Currently using 2.7 GiB. dbms.memory.transaction.total.max threshold reached
at captureStacktrace (/root/projects/python/codeanalyzer-typescript/node_modules/neo4j-driver-core/lib/result.js:624:21)
at new Result (/root/projects/python/codeanalyzer-typescript/node_modules/neo4j-driver-core/lib/result.js:112:23)
at <anonymous> (/root/projects/python/codeanalyzer-typescript/node_modules/neo4j-driver-core/lib/session.js:224:29)
at <anonymous> (/root/projects/python/codeanalyzer-typescript/node_modules/neo4j-driver-core/lib/session.js:188:27)
at <anonymous> (/root/projects/python/codeanalyzer-typescript/src/build/neo4j/bolt.ts:120:31)
at withSession (/root/projects/python/codeanalyzer-typescript/src/build/neo4j/bolt.ts:249:18)
at boltWriter (/root/projects/python/codeanalyzer-typescript/src/build/neo4j/bolt.ts:118:13)
at async emitNeo4j (/root/projects/python/codeanalyzer-typescript/src/utils/serialize.ts:51:11)
at async emit (/root/projects/python/codeanalyzer-typescript/src/utils/serialize.ts:17:11)
at async main (unknown)
at processTicksAndRejections (native:7:39)
Additional context
No
Activity
rahlk commentedon Sep 2, 2026
Looked into this. Short version: the crash is real, but it is probably a symptom of something worse, and the crash may well have saved you.
What the analyzer does before it writes anything
When it connects to Neo4j, it first checks what schema version the database was written with. If that does not match its own, it assumes the database holds leftovers from an old version of this analyzer and clears them out before writing. That cleanup is the first thing it runs, and it is the line your stack trace points at (
bolt.ts:120).The cleanup statement is:
In English: delete every node that belongs to a source file but is not labelled
:CanNode.:CanNodeis a label only the current TypeScript analyzer applies, so anything without it was assumed to be old TypeScript output.Why that assumption is wrong
codeanalyzer-pythonandcodeanalyzer-javaboth tag their nodes with_module, and neither of them uses:CanNode— I checked:_moduleis on four node types (codeanalyzer/neo4j/schema.py:91,107,131,160);CanNodeappears nowhere in the repo_moduleis set inGraphProjector.java:256,273;CanNodeappears in zero filesSo that predicate is not a description of "old TypeScript nodes". It is an exact description of every python and java node in the database. If you pointed this at a Neo4j that already held a python or java graph, the TypeScript analyzer tried to delete all of it before writing anything of its own.
Why the error is the good outcome
The delete ran as one big transaction. It hit the memory ceiling, failed, and rolled back — so nothing was actually deleted. Had the foreign graph been smaller, or the server given more memory, the delete would have succeeded silently and you would have lost the python/java graph with no error at all.
So the memory error is not really the bug. The bug is that the cleanup is not scoped to this analyzer's own nodes.
Is that what happened to you?
I could not reproduce the crash from a TypeScript-only database. I ran superset-frontend into a clean Neo4j capped at the same 2.7 GiB you hit (124,866 nodes / 314,675 edges): it succeeded, twice, in about a minute. I also tried the cleanup against 200,000 and then 300,000 synthetic leftover nodes with relationships — Neo4j deleted them without complaint. Nothing blew up until I looked at what a shared database would contain.
To confirm, could you run this against the database you pushed to and say what it returns?
That is exactly the set the failing statement tried to delete. A large number, especially with python or java nodes in that database, confirms it. Zero would mean I am looking in the wrong place.
Also useful: which commit you were on (your line 120 matches v1.1.0), and whether that Neo4j had a python or java graph in it.
What needs fixing
:Module:TSModule,:Symbol:TSCallable), so it can identify them without matching anything a sibling analyzer wrote.MATCH (a:Application) WHERE a.id IS NULL DETACH DELETE a.:Applicationis a deliberately shared, unprefixed label, so that can delete another analyzer's application node.Unrelated thing found while digging
content_hashis never actually written to the graph — all 1,841 module nodes in my test database have it asNULL. The projection strips it (project.ts:231even says so), but the incremental push still reads it to decide which modules changed. It always reads nothing, so it always concludes everything changed, so every push re-uploads the whole graph. That is not your crash, but on a memory-constrained server it guarantees every push does the maximum possible work. Worth its own issue.fix(neo4j)!: stop deleting other analyzers' graphs; artifact text who…