Skip to content

Neo4j: never delete by default; gate purge on --eager and scope it to this analyzer #171

Closed

Description

@rahlk

Problem

codeanalyzer/neo4j/bolt.py:127-131 purges a changed module with two unlabelled statements:

tx.run("MATCH (x {_module: $m})-[r]->() DELETE r", m=module)
tx.run("MATCH (x {_module: $m}) ... DETACH DELETE x", ...)

MATCH (x {_module: $m}) matches any node in the database with that _module value, not just
python's. codeanalyzer-typescript and codeanalyzer-java both set _module to the file key on
their nodes, so where a python module and a sibling analyzer's module share a file key, a python
push deletes that sibling's nodes and their relationships.

It also has no index behind it: INDEXES covers py_callable_name, py_class_name, and a
fulltext, but nothing on _module. An unlabelled match on a non-indexed property is a full store
scan, run once per changed module.

Found while fixing the same class of bug in codeanalyzer-typescript
(codellm-devkit/codeanalyzer-typescript#116). There the equivalent statement was
MATCH (n) WHERE n._module IS NOT NULL AND NOT n:CanNode DETACH DELETE n, which described python's
and java's nodes exactly — both set _module, neither applies :CanNode. It surfaced only as an
out-of-memory error, because the delete was large enough to exhaust
dbms.memory.transaction.total.max and roll back. A smaller foreign graph would have been deleted
silently.

Worth noting python's orphan prune (bolt.py:154-156) is anchored correctly —
MATCH (:PyApplication {name: $app})-[:PY_HAS_MODULE]->(m:PyModule). It is the per-module purge
that is exposed. That anchored pattern is the better model, and typescript is adopting it.

Scope boundary

The two unlabelled statements in the per-module purge. Not in scope: the orphan prune, which is
already scoped to the application.

Goals

  • Anchor both statements on python-owned labels so a sibling analyzer's nodes cannot match
  • A test asserting a foreign node with a colliding _module survives a push
  • Consider an index on the anchoring label's _module — the current match is a full scan per module

Caveats and known risks

  • PySymbol is already a shared merge label across PyClass/PyCallable/PyExternal and could
    serve as part of the anchor, but it does not cover PyModule/PyAttribute/PyVariable/
    PyBodyNode, so the predicate likely has to enumerate labels.
  • Silent data loss: no error, no log line when this fires. Any occurrence to date would have gone
    unnoticed, so shipped graphs may warrant an audit.

Definition of done

A python push into a database holding a typescript or java graph with an overlapping _module
value leaves that graph byte-identical.

Activity

  1. rahlk commented on Sep 2, 2026

    @rahlk
    ContributorAuthor

    Widening this: deletion should be --eager-only

    codeanalyzer-typescript has taken a broader fix than "anchor the purge", and the same shape applies
    here. Rather than making the deletes safer, it stopped doing them by default:

    • Default push: MERGE current nodes over whatever is in the graph, and delete nothing. The
      per-module purge and the orphan prune do not run. A schema-version mismatch forces a full
      re-UPSERT rather than a wipe.
    • --eager: purge this application's own nodes, then rebuild from scratch.

    The reasoning is that database lifetime is the operator's call, not the analyzer's. An analyzer
    that only ever adds and updates cannot corrupt anything it did not write, whatever else shares the
    database. That property is worth more than the tidiness of automatically removing declarations that
    vanished from a source file.

    --eager already means "ignore the cache and rebuild everything" in all three analyzers
    (codeanalyzer/__main__.py:177 here, CodeAnalyzer.java:149 in java), so extending it to mean
    "rebuild the graph too" is consistent rather than a new concept.

    The purge still needs anchoring

    Gating on --eager shrinks the exposure; it does not remove it. An --eager run must still not
    delete a sibling analyzer's nodes. TypeScript's purge is scoped twice:

    MATCH (n:CanNode) WHERE n.id STARTS WITH $prefix
    CALL { WITH n DETACH DELETE n } IN TRANSACTIONS OF 5000 ROWS
    
    • a label only that analyzer applies → a sibling's graph cannot match
    • id STARTS WITH <application id> → a second application in the same database survives
    • batched → deleting a whole application in one transaction is what exhausted
      dbms.memory.transaction.total.max in the original report

    Both scopes matter independently, and the batching is not optional at real sizes.

    Consequence to weigh

    Without a default purge, declarations removed from a source file — a deleted function, a renamed
    class — linger in the graph until someone runs --eager. That is the accepted tradeoff: a stale
    node is visible and recoverable, whereas another tool's silently deleted edges are neither.
    codeanalyzer-java already reaches this exact conclusion for its snapshot path in
    CypherWriter.java:40-52.

    Verified

    TypeScript's version was tested against a live Neo4j: a python graph planted alongside a 1,841-module
    TypeScript graph survived both a default push and an --eager push, where the previous code would
    have deleted it in either case.

  2. changed the title [-]Per-module purge is unlabelled — it deletes sibling analyzers' nodes[/-] [+]Neo4j: never delete by default; gate purge on --eager and scope it to this analyzer[/+] on Sep 2, 2026
  3. added a commit that references this issue on Sep 2, 2026
    d055c0b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions