Repository navigation
Description
Problem
The Bolt writer's incremental path never works. Every push is a full re-upsert of every module.
bolt.ts reads each module's stored content_hash to decide which modules changed:
const res = await s.run("MATCH (m:TSModule) RETURN m._module AS k, m.content_hash AS h");but nothing ever writes it. finalizeAnalysis strips the field from the wire envelope
(emit.ts:61, delete mod["content_hash"]), the Neo4j projection is built from that same
envelope, and project.ts:231 says so in a comment before omitting it:
// The wire strips the internal fields (module_name, content_hash) — the graph name is the
// file key, exactly as the historical projection of the stripped tree produced.So the diff compares against NULL for every module and concludes everything changed, every time.
Confirmed on a live database after pushing superset-frontend:
MATCH (m:TSModule) RETURN count(m) AS modules, count(m.content_hash) AS with_hash;
modules, with_hash
1841, 0
and the log line on a re-push of unchanged source: 1841 modules (1841 changed).
Three parts of the system disagree: schema.ts:97 declares content_hash: "string" on
:TSModule, bolt.ts reads it, and project.ts never writes it.
Why the tests did not catch it
test/neo4j-bolt.test.ts:176 hand-writes the property into a seed node
(CREATE (:Module:TSModule {... content_hash:'stale'})), so the diff logic is exercised against
data the real projection cannot produce.
Parity
codeanalyzer-python does this correctly and is the reference: content_hash is wire payload
(schema/py_schema.py:450, not stripped), projected onto the node (neo4j/project.py:689), and
read back for the diff (neo4j/bolt.py:105). Its incremental path works.
Scope boundary
Make content_hash reach the :TSModule node so the diff functions. Not in scope: the cache's own
content hashing, which is unrelated and works.
Goals
-
content_hashis present on every projected:TSModule, matching python - A re-push of unchanged source reports 0 changed modules, not all of them
- A test that fails if the property stops being written — not one that seeds it by hand
- Decide whether it becomes wire payload in
analysis.json(python's choice) or is threaded to
the projection some other way
Caveats and known risks
- If it becomes wire payload,
analysis.jsongains a field per module. Additive, and it restores
parity with python rather than diverging further. - Enabling a diff that has never actually run may expose latent bugs in the incremental path
(the per-module purge, edge ownership) that a permanent full re-upsert has been masking. - Interacts with Neo4j transaction too big #116: with deletion now gated on
--eager,
a working diff means a default push touches only changed modules instead of rewriting everything.
Definition of done
Push a project twice with no source changes; the second run reports 0 changed modules and writes no
node updates, verified against a live Neo4j rather than a hand-seeded fixture.
Activity
fix(neo4j): write content_hash, so the incremental push actually diffs (