Repository navigation
fix: --json carries the coverage verdict on incomplete scans - #7
Merged
Merged
Conversation
`flagrix scan --json` printed scanner-core's raw result, so a scan that ran
out of GitHub budget over a low result said "riskLevel": "low" and
"safeToClone": true while the exit code was 4 and the human output said
"SCAN INCOMPLETE — no verdict".
The JSON (and the MCP tool's JSON block) now adds:
- `verdict`: "low" | "medium" | "high" | "incomplete", matching exit 0/2/3/4
- `coverage`: { complete, unreliableClear, filesRead, filesUnread, reason? }
and forces `safeToClone` to false when the low result is unreliable. All
existing engine fields are kept, so consumers don't break; `riskLevel` still
reports the engine's reading of the files it did see.
scan-user needs no change: a profile scan fetches everything or fails with
exit 1, so it never returns a partial result.
A scanner reduces risk; it can't certify a repository, so the CLI stops saying it can: - A complete low result reads "LOW RISK — No blocking findings" instead of "Safe to clone", matching what the MCP tool already said. - Exit 4 is explained as "not enough of the repository was read to give a verdict" instead of "too little was read to call it safe", which implied that a complete scan would. Changed in --help, the human output, the MCP guidance, the clone-gate hook, the README and docs/agent-gating.md. The `safeToClone` JSON field is unchanged: scanner-core sets it and the browser extension reads it.
Bumps the package, the lockfile, `flagrix --version`, the MCP server's reported version and the MCP registry manifest (server.json, which was still at 0.1.0) to 0.5.0. Minor rather than patch: `--json` gains `verdict` and `coverage`, `safeToClone` turns false on incomplete scans, and the human-readable verdict wording changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
flagrix scan --jsonprinted scanner-core's raw result. When a scan ran out of GitHub budget over a low result, the JSON still said"riskLevel": "low"and"safeToClone": true, while the exit code was 4 and the human output said "SCAN INCOMPLETE — no verdict". An agent or script reading the JSON instead of the exit code would pass a repository that was barely read.Change
scanReport()insrc/lib/coverage.tswraps the engine result for both--jsonand thescan_github_repoMCP tool's JSON block:verdict:"low" | "medium" | "high" | "incomplete". It matches exit 0 / 2 / 3 / 4.coverage:{ complete, unreliableClear, filesRead, filesUnread, reason? }, the same objectcoverageOf()already computed for the exit code and the human output.safeToCloneis forced tofalsewhencoverage.unreliableClearis true.All existing engine fields are kept, so consumers don't break.
riskLevelstill holds the engine's reading of the files it did see, so it can still say"low"on an incomplete scan. The docs now say to gate onverdictor the exit code instead.Behaviour change for consumers:
safeToCloneused to betrueon an incomplete low scan and is nowfalse. It is unchanged in every other case.scan-userneeds no change. scanner-core's user scanner throws on any failed GitHub call (exit 1), so it never returns a partial result.Wording: no "safe" claims
A scanner reduces risk; it can't certify a repository. The second commit changes the wording to match:
LOW RISK — No blocking findingsinstead ofSafe to clone, which is what the MCP tool already said.--help, the human output, the MCP guidance (now "only part of this repository was read"), the clone-gate hook, the README anddocs/agent-gating.md.safeToCloneJSON field name is unchanged, because scanner-core sets it and the browser extension reads it.Docs: README "Incomplete scans (exit 4)",
docs/agent-gating.md(the exit-code table now includes code 4, plus a note on gating on the JSON) and the--helptext.Tests
New
JSON verdict and coverageblock intests/scan.test.ts:verdict: "low",safeToClone: true, full coverageverdict: "incomplete",safeToClone: false,riskLevelstill"low"reason: "fetch-errors"verdict: "medium",coverage.complete: false,unreliableClear: falseflagrix scan --jsonnpm test(37/37),npm run typecheckandnpm run buildall pass.Release
Version is 0.5.0 in
package.json,package-lock.json,flagrix --version, the MCP server's reported version (src/commands/mcp.ts) and the MCP registry manifest (server.json, previously still at 0.1.0). It's a minor bump because the JSON output, thesafeToClonebehaviour on incomplete scans and the verdict wording all change. This PR doesn't publish to npm or the MCP registry.