Skip to content

feat(graph): rank questionable call edges for review - #3207

Open
bob-codedaptive wants to merge 2 commits into
colbymchenry:mainfrom
codedaptive:pr/suspects
Open

bob-codedaptive wants to merge 2 commits into
colbymchenry:mainfrom
codedaptive:pr/suspects

Conversation

@bob-codedaptive

@bob-codedaptive bob-codedaptive commented Nov 1, 2026 •

Copy link
Copy Markdown

Stacked on fix(swift): a call on a standard-library value no longer links by name to a project method (#3199). The standard Swift method predicate is shared with resolution rather than copied into the report.

What changes

  • Add codegraph suspects and the opt-in codegraph_suspects MCP tool to review name-only and low-confidence call edges.
  • Rank targets by standard-library collisions, missing imports, multiple definitions and low confidence, with bounded sample callers and versioned JSON.
  • Read the candidate edges one row at a time and keep only each target's best sample callers, so the candidate rows and edge samples the report holds follow the number of targets and the sample cap, not the number of candidate edges.
  • Keep same-file exact matches out of the candidate set and expose equivalent SQL and JavaScript classifiers for consistent report behavior.
  • Skip an edge whose stored metadata is not valid JSON, as reading an edge does, instead of failing the whole report.
  • Preserve the explore-only default MCP listing and read-only annotations.

Verification

Run on Node 26.12.0.

  • npx vitest run __tests__/suspects.test.ts: 43 tests pass.
  • npx vitest run __tests__/suspects-surfaces.test.ts: 10 tests pass. Its CLI test spawns the built CLI, which on Node 26 was run with the change from fix(cli): CodeGraph runs on Node 25 and later instead of refusing to start applied locally and not part of this branch.
  • npx vitest run __tests__/mcp-tool-allowlist.test.ts __tests__/mcp-tool-annotations.test.ts: 2 files, 11 tests pass.
  • npx tsc --noEmit -p . passes.

Failing before:

  • With the whole source change removed and the tests kept, the two suspect test files cannot load their modules, because the report does not exist.
  • A behavioural case, with the new modules kept and only src/bin/codegraph.ts and src/mcp/tools.ts returned to the parent: npx vitest run __tests__/suspects-surfaces.test.ts fails 9 of its 10 tests. On the input codegraph suspects -p <a project with caller.ts calling helper.ts> --json --limit all, the CLI exits 1 with error: unknown command 'suspects' instead of printing the JSON report, and codegraph_suspects called through the tool handler returns the unknown-tool error instead of the same report. With the change both return the ranked JSON, and the two agree.
  • The previous revision of this PR read every candidate edge into memory before ranking: streams the candidate rows instead of reading them all at once in __tests__/suspects.test.ts fails on it. Over 200,000 candidate edges to one target with limit: 1, maxEdgesPerTarget: 0, the sampled heap grew by about 330 MiB with that revision and about 36 MiB with this one, for the same report.

The full suite and Linux/Windows validation have not been run. Swift module evidence follows conventional SwiftPM paths and does not inspect custom target paths in Package.swift; Cargo import aliases and globs can still require human review. Candidates are not proven bugs.

Known limit: on a project with fewer than 500 indexed files, a session that serves tools itself (CODEGRAPH_NO_DAEMON=1) lists only codegraph_explore, codegraph_search and codegraph_node, even when CODEGRAPH_MCP_TOOLS names suspects: the existing small-project gate in ToolHandler.getTools() applies to every tool outside those three, upstream's opt-in codegraph_callers included. codegraph_suspects can still be called by name there.

Testing

  • Platform: tested on macOS only; not run on Linux or Windows.

🤖 Generated with Claude Code

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-04T17:47:16.697294Z 0ee807b New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ad0e18b5c0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/mcp/tools.ts
Comment on lines +1697 to +1698
name: 'codegraph_suspects',
description: 'Audit name-only and low-confidence call edges, ranked by standard-library collisions (Rust and Swift), missing imports, ambiguous definitions and confidence. Returns targets and sample callers; these are review candidates, not proven errors.',

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Update the canonical MCP instructions alongside the new tool

This adds a new agent-facing MCP workflow but leaves src/mcp/server-instructions.ts unchanged. The root AGENTS.md explicitly requires MCP behavior or usage changes to update that canonical guidance; opted-in sessions otherwise still receive the unconditional “There is a single tool, codegraph_explore” playbook without guidance about the new audit workflow. Update the initialization guidance alongside the tool while retaining the explore-only default.

AGENTS.md reference: AGENTS.md:L284-L284

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for flagging this. We checked it against the file's own contract: the comment at the top of src/mcp/server-instructions.ts says the default surface is codegraph_explore alone and that the other tools "stay defined and are re-enablable via CODEGRAPH_MCP_TOOLS, but they are NOT listed to agents, so don't name them", and DEFAULT_MCP_TOOLS stays ['explore'] here. The new tool is opt-in in exactly the way codegraph_node, codegraph_search and codegraph_callers are, so its usage guidance lives in its tool description and input schema, as theirs does; the "single tool" wording for an allowlist that leaves out explore is how every opt-in tool behaves today. We have left the instructions alone for that reason. If you would rather have a short opt-in note there, tell us the wording you want and we will add it.

case 'scala': return SCALA_STD_METHODS;
case 'java': return JAVA_STD_METHODS;
case 'ruby': return RUBY_STD_METHODS;
case 'swift': return SWIFT_STD_METHODS;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve Swift extension overloads in the standard-name fallback

For xs.sorted(limit: 1) where xs: [Int] and the project defines extension Array { func sorted(limit: Int) ... }, the parent resolver finds the extension method, but the new resolver returns no target. Adding Swift here activates the generic single-candidate veto in matchMethodCall, whose receiverNamesOwner check does not preserve Swift's external-extension or argument-label exceptions. Re-indexing therefore removes legitimate calls from callers, impact, and explore; honor that Swift-specific evidence before rejecting the candidate.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the same issue as the one raised on the parent change. The fix is in that parent commit, which sits under this one on the branch and has been updated. The extension Array { func sorted(limit:) } call on an [Int], on both a parameter and a local annotated [Int], now resolves again, and tests on that change cover it.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correction: the parent PR (#3199) now keeps only the extension case. The extension Array { func sorted(limit:) } call on an [Int] still resolves; a labelled call on a factory result such as x.contains(jpeg: 1) no longer links by its label. Details are in the reply on #3199.

Comment thread src/graph/suspects.ts Outdated
* identifier, never input.
*/
export function candidateEdgeSql(alias: string): string {
const by = `json_extract(${alias}.metadata, '$.resolvedBy')`;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Guard candidate metadata extraction against malformed JSON

Existing indexes can contain malformed edge metadata: __tests__/migrations-synthesis-json.test.ts explicitly preserves it, and ordinary edge reads treat it as absent metadata. With even one such calls row, this unguarded json_extract makes the entire suspect report throw malformed JSON, so neither new surface returns the remaining valid candidates. This reproduces against real SQLite through both report entry points. Guard JSON access with CASE WHEN json_valid(...) so malformed rows are excluded rather than aborting the audit.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, confirmed. A calls edge whose stored metadata is not valid JSON made the whole report throw malformed JSON. The candidate predicate now reads the metadata through CASE WHEN json_valid(...), so such an edge reads as having no metadata and is skipped, the same way an ordinary edge read treats it. The valid candidates are still listed. suspects: an edge whose stored metadata is not valid JSON pins it for two kinds of bad metadata (non-JSON text and the empty string).

Comment thread src/mcp/tools.ts Outdated
type: 'object',
properties: {
projectPath: projectPathProperty,
limit: { type: 'string', description: 'Positive integer (string or number) or "all"; default 25 targets.' },

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Advertise numeric limits in the MCP input schema

For schema-validating MCP clients, { "limit": 10 } is invalid because this property only permits strings, even though its description and parseSuspectCount explicitly support numbers. Such clients reject the advertised numeric form before handleSuspects can accept it; the direct-handler tests bypass this boundary. Declare both string and integer representations in the published schema, or make the documented contract consistently string-only.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed: a client that validates arguments against the published schema would have rejected { limit: 10 } even though the handler accepts it. limit is now published as type: ['string', 'integer'], so both 10 and "10" (and "all") validate. publishes a limit schema that admits both the numeric and the string form pins the schema and calls the handler with a numeric limit.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a4f8544d84

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/resolution/name-matcher.ts Outdated
if (owner !== null && !swiftDeclOf(owner, context).projectType) return true;
// The method-call ref is named by its whole chain (`x.contains`); the
// label follows the member's own name on the line.
const shape = swiftCallShape({ ...ref, referenceName: n.name }, context);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Anchor Swift label lookup to the current member call

With two same-named calls on one line, such as let a = xs.contains(1); return x.contains(jpeg: 1), this check reads the first call's label instead of the current one's. Changing referenceName to contains leaves column pointing at the receiver, so swiftCallShape misses its column-based match and takes the first regex match on the line. For an [Int] receiver xs and a factory-derived x whose Bag declares contains(jpeg:), a parent/current matchReference probe resolves the labeled call in the parent but returns null here, removing a legitimate caller/impact edge. Locate the current member token before parsing its label.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed. This is the same defect as the one reported on the parent PR (#3199), and it is fixed there. The label lookup now starts at the current call's own column, so xs.contains(1); return x.contains(jpeg: 1) links the second call to Bag.contains(jpeg:). The test x.contains(jpeg: 1) after xs.contains(1) on the same line reaches Bag.contains(jpeg:) by its own label in __tests__/swift-std-method-calls.test.ts pins it, and this PR picks it up from its parent.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correction: the parent PR (#3199) no longer links a labelled call on a factory result by its label, so x.contains(jpeg: 1) after xs.contains(1) does not link to Bag.contains(jpeg:) there or here; that test is now an it.todo. Details are in the reply on #3199.

Comment thread src/graph/suspects.ts Outdated
interface Group { suspect: Suspect; scored: Array<{ edge: SuspectEdge; score: number }> }
const groups = new Map<string, Group>();

for (const row of db.prepare(CANDIDATE_SQL).all() as CandidateRow[]) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Bound retained edge samples while scanning the index

On large indexes, report limits do not bound memory: this materializes every candidate row, then retains every edge again in group.scored before applying the sample cap. A real-SQLite probe with 200,000 candidate edges and { limit: 1, maxEdgesPerTarget: 0 } grew the JS heap from roughly 35 MiB to 362 MiB while returning one target with no samples. Memory therefore scales with the entire candidate graph and can exhaust the CLI or MCP worker heap on dense indexes despite a bounded request. Iterate the rows and retain only each target's best requested samples while updating counts, scores, and flag unions incrementally.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed. The report now reads the candidate rows one at a time and updates each target's count, score and flags as rows arrive. Per target it keeps only enough edges to fill the requested sample cap, so a cap of 0 keeps none. In a 200,000-edge probe with limit: 1, maxEdgesPerTarget: 0, sampled heap growth fell from about 330 MiB to about 36 MiB, with the same output. The test streams the candidate rows instead of reading them all at once pins the streaming; the existing ranking and cap tests pin the output.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c8c8798465

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/resolution/name-matcher.ts Outdated
Comment on lines +7488 to +7489
const call = member.exec(line);
if (!call) return false;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Follow continued Swift calls when checking argument labels

For a factory-derived x whose Bag defines contains(jpeg:), a valid call with return x on one line and .contains(jpeg: 1) on the next now loses its edge. The extractor records x.contains at the receiver's line, but this lookup searches only that line and returns false before seeing the label on the continuation. A parent/current matchReference probe resolves Bag::contains only in the parent, confirming a new multiline failure despite the corrected same-line anchoring. Re-indexing therefore drops a legitimate caller/impact link; locate the member and label across the continued call before applying the standard-name veto.

Useful? React with 👍 / 👎.

Comment thread src/mcp/tools.ts
const root = this.getCodeGraph(args.projectPath as string | undefined).getProjectRoot();
const { findSuspects, formatSuspects, suspectsToJson } = await import('../graph/suspects');
const list = findSuspects(root, { limit, maxEdgesPerTarget: edges });
return this.textResult(args.format === 'json' ? JSON.stringify(suspectsToJson(list), null, 2) : formatSuspects(list));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep MCP notices outside the JSON report payload

If this tool is a session's first successful query of an older index, format: "json" returns a warning followed by JSON rather than a JSON document. MCPSession passes an IndexVersionWarningState to execute(), whose withIndexVersionNotice() prepends prose to this ordinary textResult; worktree and staleness notices can modify the same payload. This reproduces through the handler with real SQLite and makes JSON.parse(result.content[0].text) fail, whereas the new tests omit the session warning state. Expose the report through structuredContent or otherwise keep notices separate from the serialized JSON.

Useful? React with 👍 / 👎.

Comment thread src/graph/suspects.ts
Comment on lines +290 to +294
for (const [alias, member] of getCargoWorkspaceCrateMap(context)) {
let dir = member.replace(/^(\.\/)+/, '');
if (dir === '.') dir = '';
byDir.set(dir, alias.replace(/-/g, '_'));
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include the implicit root package in Cargo workspace membership

For a non-virtual Cargo workspace with a root [package] and members = ["crates/kit"], the root package is implicitly a workspace member, but this map omits it because getCargoWorkspaceCrateMap() enumerates only explicit members. Consequently, rustCrate('src/lib.rs') returns null, and missingImport() suppresses every cross-crate warning involving the root package even though its name is available in the root manifest. A real-SQLite probe gives the same edge score 1 without an explicit "." member and score 4 after adding that redundant entry. Include the root package as dir: '' when the manifest contains both [package] and [workspace].

Useful? React with 👍 / 👎.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 70b49b00a4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +31616 to +31617
const args = cppParenListAfter(ref.filePath, ref.line, Math.max(0, ref.column), member, context);
if (args === null) return null;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Ignore string-literal parentheses when scanning Swift arguments

For a [String] receiver and a project Array.joined(glue:) extension, texts.joined(glue: "(") now loses its call edge. The newly used cppParenListAfter balances raw characters, including the literal's (, so it cannot find the argument-list end and isSwiftStdNameKept rejects the extension. A parent/current matchReference probe resolves this call only in the parent; changing the literal to "ok" resolves in both. Use a quote-aware balanced scan before treating parsing failure as grounds to discard a member, otherwise valid calls disappear from callers and impact results.

Useful? React with 👍 / 👎.

const args = cppParenListAfter(ref.filePath, ref.line, Math.max(0, ref.column), member, context);
if (args === null) return null;
const params = cppParenListAfter(decl.filePath, decl.startLine, 0, decl.name, context);
return params === null ? null : swiftLabelsFit(swiftArgumentLabels(args), params);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Consume all arguments belonging to a Swift variadic parameter

For an [Int] receiver and an Array.sorted(limit:extra:) extension with extra: Int..., xs.sorted(limit: 1, extra: 1, 2) now returns no target, while the single-element variadic call still resolves. This new guard delegates to swiftLabelsFit, which treats a variadic parameter as optional but consumes only one label, leaving the additional _ label unmatched. A parent/current matchReference probe resolves the multi-element call only in the parent, demonstrating a separate failure from the fixed single-argument sorted(limit:) case. Consume the remaining unlabeled variadic arguments instead of vetoing these legitimate caller/impact edges.

Useful? React with 👍 / 👎.

…e to a project method

arr.sorted(), xs.map { ... } and values.filter { ... }.sorted() now need receiver
evidence before they resolve to a project member of that name: a receiver whose name
names the owner, an owner the project only extends, or, in a chain, a label the member
declares. On a receiver that names nothing, a label alone is not enough: the member
must extend an outside type and the call's argument labels must fit it.
A chain whose receiver ends in a trailing closure has nothing to name the owner.
The method-name list moves to swift-builtins.ts.
`codegraph suspects` and the opt-in `codegraph_suspects` MCP tool list name-only and low-confidence call edges, ranked for review by a read-only pass over the index. Both use the same candidate classifiers.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0ee807b647

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

*/
function swiftCallFitsLabels(ref: UnresolvedRef, decl: Node, context: ResolutionContext): boolean | null {
const member = ref.referenceName.slice(ref.referenceName.lastIndexOf('.') + 1);
const args = cppParenListAfter(ref.filePath, ref.line, Math.max(0, ref.column), member, context);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Account for trailing closures in Swift extension matching

For an [Int] receiver and an Array.sorted(limit:by:) extension whose by parameter is a required comparator closure, xs.sorted(limit: 1) { $0 < $1 } now loses its call edge. Only the parenthesized limit: 1 reaches swiftLabelsFit, so the closure parameter is incorrectly considered missing. The fixture typechecks with Swift, and a parent/current matchReference probe resolves Array::sorted only in the parent, while the equivalent xs.sorted(limit: 1, by: { $0 < $1 }) resolves in both. Include trailing-closure arguments in this check so re-indexing does not remove valid callers and impact edges.

Useful? React with 👍 / 👎.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant