Skip to content

[SCR-262] Make a throttled mine resumable - #264

Closed
itsryanarmani wants to merge 2 commits into
scr-250-mine-confirmed-defectsfrom
scr-262-resumable-mine
Closed

itsryanarmani wants to merge 2 commits into
scr-250-mine-confirmed-defectsfrom
scr-262-resumable-mine

Conversation

@itsryanarmani

@itsryanarmani itsryanarmani commented Sep 3, 2026 •

Copy link
Copy Markdown
Owner

Closes #262

Stacked on #257 — review that first. This branch's diff against #257's head is the cache plus the council's pagination and line-attribution fix.

What

Caches gh api responses on disk and turns a rate-limit refusal into a pause rather than a fatal error, so a throttled mine emits what it gathered and a re-run resumes. Also carries the council's --slurp pagination fix and its hunk-accurate defect-line attribution, which belong here rather than on #257 for size reasons.

Why

The default invocation could not finish. A full fleet mine costs about two API calls per merged pull request, which across the eight default repos exceeds the 5,000/hour limit shared with every other tool on the machine.

The first real run died on content-rabbit after walking seven repos and wrote no output file at all:

mining pooriaarab/skills
mining pooriaarab/vibecodereview
gh: API rate limit exceeded for user ID 42897904 (HTTP 403)
error: gh api --paginate repos/pooriaarab/content-rabbit/pulls/3/files failed (exit 1)
error: cannot read /tmp/confirmed-defects.json: No such file or directory

An hour of paid-for calls discarded by a failure mode certain to recur.

How I verified

All 13 tests pass, including the nine pre-existing ones:

$ ./mine-confirmed-defects.test.sh
...
ok - re-run writes an identical file
ok - validator rejects an incomplete case
ok - a cached re-run makes no new upstream calls
ok - a corrupt cache entry is discarded and refetched
ok - a different query gets its own cache entry
ok - a non-throttle API failure still exits non-zero
---
13 passed, 0 failed

Live, and a cached re-run reproduces the dataset byte for byte:

$ MINE_CACHE_DIR=$CC ./mine-confirmed-defects pooriaarab/vibecodereview --out /tmp/n1.json
by_lens: correctness=6, security=2
cases: 3
errors: 0

$ MINE_CACHE_DIR=$CC ./mine-confirmed-defects pooriaarab/vibecodereview --out /tmp/n2.json
errors: 0

$ diff -q /tmp/n1.json /tmp/n2.json && echo identical
identical

No dead pagination path. The council first added a manual multi-page JSON decoder, then replaced it with gh api --slurp. Only --slurp ships — three call sites, and grep -c raw_decode returns 0. Keeping both would have left a dead path for the next agent to copy.

Defect lines are now real, not placeholders. They used to be line 1 on every defect, which capped how precisely a judge could score a finding against a label:

  action.yml line 14

Two corrections made while verifying

My first cache test passed for the wrong reason. It removed the fake gh from PATH and asserted the run still worked — which it did, because the real gh was still on PATH, not because the cache served it. The test now counts fake-gh invocations: n1 > 0 on the first run, n2 == 0 on the second. That is the property that actually matters.

A fully-cached run still needs gh installed. The binary preflight runs before the cache is consulted, so PATH=/usr/bin:/bin gives error: gh CLI not found even with a complete cache. Acceptable — the criterion is "no upstream API calls", not "runs without gh" — but it is why the removal-based test was misleading. Moving the preflight would be a separate concern.

Note on why this is a second PR

Combined, the work exceeded the 500-line cap. It splits along a real seam: #257 is the miner and its label rules, this is surviving a throttle and the pagination and line-attribution robustness that came with it.

Assisted-by: claude-personal:claude-opus-5

@coderabbitai

coderabbitai Bot commented Sep 3, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: f37884df-ae49-4d7a-9449-848584f4efd4

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@itsryanarmani
itsryanarmani force-pushed the scr-250-mine-confirmed-defects branch from acb51e5 to 2749268 Compare September 3, 2026 04:47
A full fleet mine costs about two API calls per merged pull request,
which is more than an hour's rate limit allows. Without a cache a
throttle part-way through loses every call already paid for and the
re-run starts from zero. That happened on the first real run: it died
on content-rabbit after walking seven repos and wrote no output at all.

Responses are now cached on disk, keyed on the exact argument list, and
a rate-limit refusal raises rather than exiting, so the run emits the
repos it already mined and names the one to resume from. Any other API
failure still exits non-zero.

The tests count fake-gh invocations rather than hiding gh from PATH.
Removing the binary only proves the preflight check fires; counting
calls proves the cache actually served the second run. Each invocation
also gets its own cache directory, because the cases swap fixtures
under one fake gh and would otherwise serve stale data to the next.
…hunk

gh api --paginate without --slurp prints one JSON document per page, so
json.loads breaks with "Extra data" once any repo or PR needs a second
page. Separately, defect line numbers were always overwritten with the
first hunk's start line, mislabeling defects that live in a later hunk
of a multi-hunk patch.
@itsryanarmani
itsryanarmani deleted the branch scr-250-mine-confirmed-defects September 3, 2026 05:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant