Repository navigation
Description
Problem
The two projections of this analyzer disagree about the primary text, and the graph is the one that is wrong.
Canonical decision D1 (.claude/SCHEMA_DECISIONS.md:15) makes module.source the primary text and every narrower node's text a byte-slice of it — it dropped per-callable code from analysis.json explicitly "(SDK slices module.source[span.bytes])". The JSON projection obeys it: module.source is required, and callable has no code property at all.
The Neo4j projection inverted it. :JModule carried id, file_key, package, content_hash and no source, while :JCallable kept the one derivation D1 deleted (code). The source was in hand at the moment the graph was written — V2GraphProjector.java:312 calls module.getSource() to slice code out of it — so this was an omission, not missing data.
The flattened span is the other half. putLines emitted start_line / end_line only, dropping the columns and the UTF-8 byte offsets, even though the JSON schema defines span.bytes as "[from, to) UTF-8 offsets into module.source, so node text is an O(1) slice".
Together those two gaps mean nothing narrower than a callable can be resolved to text at all. python-sdk documents the consequences it is forced to live with: a reconstructed JCompilationUnit gets source="" (cldk/analysis/java/neo4j/reconstruct.py:35-36,343), and _body_source is hardwired to return None because ":JBodyNode carries a line range and no text, and :JModule carries no source to slice one out of" (neo4j_backend.py:721-723). A required pydantic source: str cannot express "the projection did not carry this", so it reports "".
The invariant this restores: analysis.json and the graph carry the same facts. Where the graph re-expresses a JSON field as an edge or a flattened property that is fine; silently dropping one is not.
Scope boundary
Restores the text-resolution facts on the Java graph projection only: source on :JModule, and columns plus byte offsets on the flattened span.
Does NOT drop JCallable.code, even though this change proves it derivable — python-sdk's reconstruction reads it today, so removing it is a separate breaking change. Does NOT add body_span to the graph. Does NOT touch analysis.json, which was already correct. Does NOT move schema_version, which stays at the held 2.0.0 baseline (see codellm-devkit/.github#50). Does NOT change codeanalyzer-python or codeanalyzer-typescript, which have the identical inversion and are filed separately.
Goals
-
:JModulecarriessource, the whole file, matchinganalysis.json's requiredmodule.source -
sourceis always present, never absent — coalesce null to"", sinceRowBuilder.prunedrops nulls and the two projections would then disagree about whether the property exists - the flattened span carries
start_column,end_column,start_byte,end_bytealongside the existing line pair -
V2SchemaCatalogdeclares all of the above, and the checked-inschema.neo4j.jsonsnapshot is regenerated - a conformance test asserts every
:JModule'ssourceequals theanalysis.jsonmodule source, per module rather than by presence, so the inversion cannot return as a truncated or placeholder value
Caveats
- The four span property names are new shared vocabulary. All three analyzers flatten spans to exactly
{start_line, end_line}today (codeanalyzer-python/codeanalyzer/neo4j/schema.py:64,codeanalyzer-typescript/src/build/neo4j/schema.ts:55). Java coiningstart_column/end_column/start_byte/end_bytefirst means the siblings must adopt these names, not invent their own — a term coined twice is permanently wrong under the cross-language parity clause. - Node payloads grow by the size of the source tree. Measured on daytrader8: 653 KiB across 141 modules. The graph already redundantly stored 295 KiB of that as
JCallable.codeslices, so the marginal cost is smaller than it looks, and it becomes negative ifcodeis ever dropped. - No opt-out flag. The artifact layer has
--no-artifact-textfor the same concern; this deliberately ships without a counterpart rather than adding a knob nobody has asked for. If graph size becomes a real problem that is the precedent to follow. schema_versiondoes not move, so this is not detectable from the payload — the same cost every addition on the held 2.0.0 baseline carries. Detection is by presence until the coordinated re-baseline.
Definition of done
- Every
:JModulerow carriessource; on daytrader8 that is 141 of 141, and each one's SHA-256 equals thecontent_hashthe same row declares (proves the value survives Cypher escaping intact). source[start_byte:end_byte]equals thecodeproperty already on the node, byte-for-byte, for every callable where both are present — 662 on daytrader8, zero mismatches. This is the check that the byte offsets are real rather than merely present.--emit schemaoutput matches the checked-inschema.neo4j.json.- Full suite green apart from the pre-existing Docker-only
CodeAnalyzerIntegrationTestinitialization failure.
Activity