Repository navigation
[SCR-310] Verify provider readiness before dispatch - #312
itsryanarmani wants to merge 5 commits into
Conversation
Gate the implementation sentinel on a live provider probe on the recorded Box: actual binary, canonical personal credential file, and a bounded unique arithmetic/file-write probe whose reply and file must both equal the expected value. Failures report provider bootstrap status and leave the sentinel unreached.
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧑⚖️ Council review
⚠️ Posted by the fallback chair: every Claude subscription token failed, so this
review came from a single OpenRouter completion with no repo access. It saw the diff
and the council's findings only — it could not open other files, run anything, or push
fixes. Treat it as weaker than a normal review.
Solid, well-documented gate script with good test coverage for the muse happy/failure paths, but a few diff-introduced correctness gaps survive review, and the evidence in the PR body doesn't reach several code paths this diff actually changes.
- The anti-ambient-fallback guarantee (a core acceptance criterion, and one of the two bugs #310 cites) is only enforced for
gemini/cursor/devin(which explicitlyunset/env -uthe shared API-key vars before invoking the CLI).muse,pi,kimi, andcodexget no such isolation, so a shared/ambient credential on the Box can make the probe succeed even when the checked canonical personal file is stale/wrong — formusein particular, this is the exact provider exercised by this PR's own evidence, yet the isolation gap is untested. cursor's probe command passes--output-format json, but the reply parser expects the literal digits as the last non-empty stdout line. Ifcursor-agent --output-format jsonwraps its answer in a JSON envelope (as the flag name implies), every real cursor probe will fail the exact-match check regardless of correctness — and there is zero test coverage exercising this path to catch it.- The initial reachability poll (
"$BOX" exec --cwd "$CWD" "$BOXID" -- true) has no per-call--timeout, unlike every otherbox execinvocation in the file. A single hung/slow transport call blocks the loop indefinitely, which defeats the documented "bounded retries" behavior for the very first gate. - Evidence gap: the
## How I verifiedsection only runs the muse path (both the real preflight and the full test suite are muse-only). The five other provider branches this diff adds — including the likely-broken cursor path above — have no test or manual run cited, so the diff's untested surface is large relative to what's claimed verified.
None of these rise to Critical (no crash/security/data-loss trigger visible in the diff), so this is a comment, not a blocking review, but the ambient-fallback and cursor-parsing issues should be fixed or at least explicitly scoped out before other providers are wired in.
Reviewed head:
903d4c668d78502580cbabbedb5930835cf5eb38.
Findings
- 🟠 Major
box-provider-readiness:101— Only thegemini,cursor, anddevinprobe commandsunset/env -utheir ambient API-key variables before invoking the CLI.muse,pi,kimi, andcodexdo not, so if the Box has a shared/ambient credential exported in the exec environment, the probe can succeed via that ambient credential even though the canonical personal file (test -schecked earlier) is stale, wrong, or belongs to a different account — directly undermining the PR's 'no ambient/shared fallback' guarantee. This is concerning specifically formuse, since it's the only provider exercised by this PR's tests and real-world evidence, so the gap is unverified. - 🟠 Major
box-provider-readiness:106— The cursor probe command passes--output-format jsontocursor-agent, but the reply is parsed as the last non-empty raw stdout line and compared literally to the expected integer. Ifcursor-agent's JSON output format wraps the answer (e.g.{"result":"..."}) rather than emitting the bare digits, every cursor readiness probe fails the exact-match check even when the CLI answered correctly. No test inbox-provider-readiness.test.shexercises this branch, so the bug (if present) would ship undetected. - 🟠 Major
box-provider-readiness:71— The initial reachability poll ("$BOX" exec --cwd "$CWD" "$BOXID" -- true) is issued without any--timeout, unlike every otherbox execcall in this script (which useexec_jsonwith an explicit bound). If a single invocation of this call hangs on a slow/stalled transport, the surroundingREADY_SECSwall-clock retry loop never gets to re-check its deadline, so the 'bounded retries' behavior documented for this gate is not actually bounded for its first check. - 🟡 Minor
box-provider-readiness.test.sh:92— The test suite (and the PR's cited manual run) exercises only themuseprovider. The other five provider branches added inbox-provider-readiness—pi,gemini,cursor,devin,kimi,codex— each have distinct credential-export semantics and CLI invocation forms that are entirely untested, including the likely cursor JSON-parsing issue noted above. The 'How I verified' section should either add fixtures for these paths or explicitly scope the claim to muse-only until they are covered. - 🟡 Minor
box-provider-readiness.test.sh:163— No fixture exercises 'correct file, wrong reply' (only 'wrong file, correct reply' viawrong-writeis tested), so the reply-mismatch fail branch ([ "$REPLY" = "$EXPECTED" ] || fail ...) is never directly exercised by the suite, leaving one of the two acceptance-criteria checks ('both reply and file must equal exactly') partially unverified.
Fallback chair: anthropic/claude-sonnet-5 via OpenRouter.
Add synthetic gh auth-precedence tests (fake HOME, synthetic-only values): the canonical token file is active when env is scrubbed, a stale inherited GH_TOKEN shadows the file, and an expired inherited token fails closed with HTTP 401. No real secrets, no Box, no state changes.
There was a problem hiding this comment.
🧑⚖️ Council review
⚠️ Posted by the fallback chair: every Claude subscription token failed, so this
review came from a single OpenRouter completion with no repo access. It saw the diff
and the council's findings only — it could not open other files, run anything, or push
fixes. Treat it as weaker than a normal review.
Solid, well-tested gate for the muse and pi paths, but the diff introduces five more provider probe recipes (gemini, cursor, devin, kimi, codex) that no test in this PR exercises, and that gap hides at least two real defects: the cursor probe requests JSON output but the parser expects bare digits, and four of the seven providers (muse, pi, kimi, codex) never scrub ambient credential env vars before the probe runs, unlike gemini/cursor/devin, which contradicts the explicit 'no ambient/shared fallback' requirement in the linked issue. A new, unrelated gh-auth-precedence.test.sh file is included in the diff but is never invoked in the ## How I verified section and tests gh credential precedence directly rather than the readiness gate — it should either be run and reported or dropped/split out per the PR's own stated scope ('touches only box-agent... disjoint from PR309-owned files'). None of these rise to a crash/security/data-loss level (the gate still fails closed on error), so nothing here is Critical, but the cursor bug and the credential-scrub gap are real, diff-introduced correctness/policy violations that should be fixed or at least covered by a test before merge.
Reviewed head:
c36601593d5e0a0011bcbf4e738df0d733241bc8.
Findings
- 🟠 Major
box-provider-readiness:123— The cursor probe command invokescursor-agent -p --trust --force --model composer-2.5 --output-format json "$P", but the readiness check right after (REPLY= last non-empty stdout line, compared to the plain integer$EXPECTED) assumes bare-digit output. With--output-format json, the CLI's reply is a JSON envelope, so the comparison[ "$REPLY" = "$EXPECTED" ]will fail even for a fully working, correctly authenticated Cursor Box — permanently blocking dispatch for the cursor provider. No test in this PR exercises the cursor probe path, so this would only surface against a real Box. Either drop--output-format jsonfor the probe or extract the numeric field from the JSON before comparing. - 🟠 Major
box-provider-readiness:109— The linked issue requires 'Load only its canonical personal credential file... no ambient/shared fallback.' The gemini, cursor, and devin probe commands explicitlyunset/env -uthe corresponding ambient API-key env vars before sourcing the canonical file, but the muse, pi, kimi, and codex probe commands do none of this. If any of these four ambient env vars happen to be set in the Box's exec environment, the probe (and hence readiness) can pass — and the actualbox-workrunner could later authenticate — via ambient credentials even when the canonical personal credential file is invalid, contradicting the stated acceptance criterion and the file-existence check's intent. Only the file's presence/non-emptiness is verified, not that it is what actually authenticates the call. - 🟠 Major
gh-auth-precedence.test.sh:1— This new test file exercises livegh auth status/gh api usercredential-precedence behavior, which is unrelated to the box-provider-readiness gate itself and not mentioned anywhere in the PR's## How I verifiedsection — it is never run or reported. This both violates the PR's own stated scope ('The hook touches only box-agent... disjoint from PR309-owned files') and leaves a diff-introduced file with zero verification evidence. Either run this file and report its result, or remove/split it into its own change. - 🟡 Minor
box-provider-readiness:96— Whenbox execfor thecommand -v/test -spreflight steps suffers a transport failure or returns malformed JSON, the python parse fails and the script falls through toHAVE=""/CRED_CODE=1, which is then reported as 'binary not found' or 'credential missing or empty.' This misreports the actual failure mode (infra/transport vs. genuine absence), which cuts against the acceptance criterion to 'report bootstrap status accurately.' The gate still fails closed correctly, so this is a diagnostic-accuracy issue rather than a functional one. - 🟡 Minor
box-provider-readiness:141— The probe-file comparison strips all whitespace from the file contents before comparing to$EXPECTED("".join(...split())), rather than requiring the documented one-line exact-digit contract. This would let a malformed write (e.g., digits split across lines/spaces) pass the check. Low real-world likelihood, but it weakens the exactness the probe is meant to guarantee. - 🟡 Minor
box-provider-readiness:98— The gemini, cursor, devin, kimi, and codex probe command strings (only muse and pi are exercised by the offline unit tests, and only pi by the box-agent dispatch-gate tests) are entirely untested by anything in this PR. Any quoting/env/flag error in those five recipes — such as the--output-format jsonissue above — would only be caught against a real Box. Given the PR explicitly claims 'Box CLI is unauthorized in this environment... no fleet readiness is claimed,' this gap should at least be called out per-provider in the verification section rather than only asserted as 'flags validated against--help.'
Fallback chair: anthropic/claude-sonnet-5 via OpenRouter.
Drive the real helper across cursor JSON replies, pinned-model validation, strict timeout bounds, and canonical-token precedence: cursor --output-format json parses via its result field, every pinned route forwards its validated model, 0/00/over-bound timeouts fail before any probe, and stale inherited GH_TOKEN/GITHUB_TOKEN lose to the canonical token file (expired keys fail as bootstrap failures). Replace the standalone gh precedence test (live calls, repurposed HOME) with fake-gh/fake-git tests through the real selector code. Probe directories are cleaned on success and failure, and replies are scrubbed before any mismatch is reported.
…check - box-agent now forwards --repo to box-provider-readiness so the ls-remote repo-access check actually runs during real dispatch, not just in tests that pass --repo explicitly. - The Box-ready polling loop bounds each `box exec ... -- true` call with --timeout 20 instead of leaving it unbounded, so a hung call can no longer defeat --ready-secs. - The probe result-file check now compares the stripped file content exactly instead of concatenating all whitespace-split tokens, so malformed multi-token output (e.g. "12 34") no longer passes as "1234".
There was a problem hiding this comment.
🧑⚖️ Council
| Model | Lens | Result |
|---|---|---|
| GPT-5.6 (Codex) | correctness | mixed: 1 confirmed (ready-secs bound), 2 rejected |
| Gemini 3 Pro | performance | no findings (openrouter timed out) |
| Kimi K3 | security | no findings (openrouter timed out) |
| Grok 4.5 | maintainability | no findings (timed out) |
| GPT-5.6 (scope) | scope | confirmed (untested provider branches) |
Reviewed head: 406fe4dae1ff518ebd022a823a9b46ab08b5d36f
No pushes this cycle — both confirmed findings below are real but shouldn't be auto-fixed right now, for two different reasons (loop guard on one, the fleet 500-line cap on the other). Details inline; no Critical is open.
Discarded
box-provider-readiness:143(muse ambient credential fallback) — the correctness lens claimed a nonempty-but-invalid muse credential file plus an ambientMUSE_API_KEYcould let the probe pass on ambient auth, unlike gemini/cursor/devin which explicitly scrub their ambient key. Butbox-setup/install-agent-clis.shdocuments muse's auth as OAuth via~/.config/muse/auth.jsononly — there's noMUSE_API_KEYanywhere else in this repo (gemini/cursor/devin's env-var names, by contrast, are documented in that same file). The premise doesn't hold here; discarding as unconfirmed.box-provider-readiness:172(result-file whitespace tolerance) — claimed.strip()could let a malformed result wrongly pass..strip()only trims incidental whitespace before an exact-integer-string equality check; there's no input where a wrong answer strips down to equal the expected one. This is also the exact spot 406fe4d already hardened (stopped"12 34"→"1234"concatenation). No concrete failure trigger survives.
Open (not fixed)
-
Major — readiness poll can overshoot
--ready-secsby ~20s (box-provider-readiness:101-107). Eachbox exec --timeout 20 -- trueattempt uses a hardcoded 20s regardless of the remaining budget, and only checks the deadline after an attempt returns. With--ready-secsunder ~20 (a valid input — the script only requires1 ≤ n ≤ 1800), an unresponsive Box can make the loop run the full 20s-plus-5s-sleep before the deadline check fires and fails it, well past what the caller asked for.
This is the same polling loop cycle 1 already patched (406fe4d: "bounds eachbox exec ... truecall with--timeout 20... so a hung call can no longer defeat--ready-secs") — that fix closed the unbounded case but not this overshoot. Per the loop guard, I'm not taking a third pass at this mechanism myself; flagging for a human to decide whether each attempt's timeout/sleep should be clamped to the remaining deadline. -
Major — gemini/devin/kimi/codex dispatch branches ship with zero passing tests (
box-provider-readiness.test.sh:55mkcli()). Only muse, cursor-agent, and pi get fake-binary coverage; the other four providers' credential-export logic (unset GEMINI_API_KEY; . "$HOME/.gemini-personal/.gemini/.env", devin'senv -u DEVIN_API_KEY -u DEVIN_TOKEN -u WINDSURF_API_KEY, etc.) — the most security-sensitive new code in this file — is completely unexercised. The PR body attributes this to "no devin binary in this environment," but the existing muse/cursor/pi tests already run entirely against fake stub binaries inmkcli(), so the same technique would cover the other four without needing real CLIs; the stated justification doesn't actually explain the gap.
I'd normally add these tests myself, but this PR is already at 493 of the fleet's 500-line cap (per its own "Counted size vs main" note), and four more fake-CLI branches plus test functions would very likely push it over — which I'm told not to do with my own commits. This needs to be a follow-up split by the author: either landbox-provider-readiness.test.shgemini/devin/kimi/codex coverage as its own immediate follow-up PR, or trim this PR's existing test additions to make room before adding it here.
FYI
- Repo state has an untracked
probe.4file at the working-tree root, unrelated to this diff — left alone.
| echo "box-provider-readiness: box=$BOXID provider=$PROVIDER model=${MODEL:-default} cwd=$CWD" | ||
| start="$(date +%s)" | ||
| while :; do | ||
| if "$BOX" exec --cwd "$CWD" "$BOXID" --timeout 20 -- true >/dev/null 2>&1; then break; fi |
There was a problem hiding this comment.
Major: this attempt is hardcoded to --timeout 20 and the loop only checks the READY_SECS deadline after it returns (plus a flat 5s sleep). With --ready-secs below ~20 — a value this script's own validation allows (1–1800) — an unresponsive Box can make this loop run the full ~20-25s before failing, well past the caller's requested bound.
This is the same loop 406fe4d already bounded (fixed the unbounded-hang case). This is a residual, narrower overshoot on the same mechanism — per the loop guard I'm not pushing a third patch here myself; flagging for a human to decide whether the per-attempt timeout/sleep should clamp to the remaining READY_SECS - elapsed budget.
| STUB | ||
| chmod +x "$T/stubbin/box" | ||
| } | ||
| mkcli() { # <dir>: one generic fake provider CLI (muse/cursor-agent/pi) asserting its pinned route |
There was a problem hiding this comment.
Major: mkcli() only stubs muse/cursor-agent/pi. The gemini/devin/kimi/codex dispatch branches in box-provider-readiness (their per-provider credential export/scrub logic — the most security-sensitive new code here) have no passing test at all. The PR body says this is because there's "no devin binary in this environment," but the muse/cursor/pi tests already pass entirely against fake stub binaries added right here in mkcli() — no real CLI needed for those either, so the same approach would cover the rest.
I'd add this myself, but the PR is already at 493 of the fleet's 500-line cap, and four more fake-CLI branches + tests would likely tip it over from my own commit — which I'm told to avoid. This needs to land as an author follow-up (own PR, or trim existing tests to make room first).
Prove canonical GitHub identity and repo access from an existing cwd before cloning, then clone with the canonical personal token instead of ambient inheritance. Require the login to equal pooriaarab, select the codex personal profile via CODEX_HOME, and use the established devin -p route since no devin binary exists here to verify --prompt-file. Replace HOME reassignment in tests with BOX_BIN, BOX_CLI, BOX_PROBE_BASE, and GATE_T injectors, and add clone-ordering and wrong-identity regressions.
There was a problem hiding this comment.
Reviewed head: 912b4d4af3ac93fd1d0b1b412c386678ac659296
Council findings verified against the code; no fixes pushed this cycle (see rationale below — the three confirmed issues are all explicitly excluded from being auto-fixed by our own review policy).
Confirmed findings
1. Major — PR is over the 500-counted-line cap
gh pr view --json files reports box-agent 37+6, box-provider-readiness 219+0, box-provider-readiness.test.sh 392+0 → 654 counted lines (additions+deletions, no excluded globs apply), against the repo's hard cap of 500 (AGENTS.md, .github/pr-standards.json). The body's own "Counted size vs main: 493 lines, 3 files" was accurate only through commit 6153f57; the two subsequent commits (406fe4d, 912b4d4) added ~198 more insertions and pushed the PR over the cap. This is not something I can fix by editing (my own commits would count against the same cap and make it worse). Concrete split: land box-provider-readiness + its core test coverage (muse/cursor/codex/github-gate paths) as the SCR-310 PR, and move the later box-agent clone-identity rework (canonical-token clone gate, BOX_BIN/BOX_CLI injection, the devin/codex route fixes) to a follow-up PR — that second slice is also its own reviewable concern (GitHub clone-credential policy vs. provider-probe readiness).
2. Major — proof is stale
The body's ## How I verified reports ./box-provider-readiness.test.sh → pass=31 fail=0. Counting every pass "..." call site actually reachable in the current suite (21 test functions run in the trailing for t in ... loop) totals 40, not 31 — the recorded result predates commits 406fe4d/912b4d4, which added test_gate_clone_ordering, test_gate_github_failure_no_clone, stale-env clone-ordering assertions, etc. Evidence is the author's to produce, so I'm not rerunning and pasting a number in — please rerun ./box-provider-readiness.test.sh and update the recorded result to match the current suite.
3. Major — three new provider routes ship with zero test coverage
box-provider-readiness:186-189 add real dispatch commands for gemini, devin, and kimi (env-scrubbing, config sourcing, model routes). The test file's mkcli() only stubs muse, cursor-agent, pi, and codex — there is no fake gemini, devin, or kimi binary anywhere in box-provider-readiness.test.sh, so none of those three probe branches ever executes in CI. The PR body only owns up to the devin gap ("no devin binary in this environment to re-check"); gemini and kimi aren't mentioned at all. I'm not adding tests myself here since the PR is already over the line cap (finding #1) and more test code would make that worse — this belongs in the same follow-up split.
Discarded claims (verified against the code, not real)
- box-agent:177 (command -v pre-check races ahead of the readiness retry loop) — real behavior, but the three lines in question (
CLI=...;have=...;[[ -z "$have" ]]...) are unchanged context in the diff, i.e. pre-existing code this PR didn't touch. - box-provider-readiness:216 (file check should validate raw bytes, not
.strip()) — the.strip()normalization is consistent with the same leniency already applied to the reply-parsing path elsewhere in this file; a provider adding a trailing newline around a byte-exact correct value isn't a real failure mode, and the numeric equality is still checked exactly. - box-agent:183 (hardcoded
pooriaarabidentity should be configurable / split out) —VISION.mdstates the single-owner scope is deliberate ("One developer: the owner, pooriaarab... The single-user scope is deliberate"), andpooriaarabis already hardcoded elsewhere in the repo (launchd plists, etc). Not a scope violation. - box-provider-readiness:196 (ambient
MUSE_API_KEY/pi/kimi env vars could falsely certify an invalid canonical credential) — plausible in the abstract, butbox-setup/install-agent-clis.shshows muse authenticates via OAuth in~/.config/muse/auth.json, pi viapi auth print-api-keyreading its own stored config, and kimi via~/.kimi-code/config.toml— none of these three tools is documented in this repo as reading an ambient API-key env var the waygemini/cursordo, so the specific failure trigger named isn't grounded in evidence here. Worth a second look if any of those CLIs turn out to support such an override, but not confirmed.
🧑⚖️ Council
| Model | Lens | Result |
|---|---|---|
| GPT-5.6 (Codex) (via OpenRouter) | correctness | 1 rejected (pre-existing code), 1 unconfirmed/speculative (ambient credential fallback for muse/pi/kimi) |
| GPT-5.6 (scope) | scope | 2 confirmed (stale proof, untested provider routes), 1 rejected (hardcoded identity — by design per VISION.md) |
| Gemini 3 Pro | performance | no findings (timed out) |
| Kimi K3 | security | no findings (timed out) |
| Grok 4.5 | maintainability | no findings (timed out) |
| READINESS_GATE="$(cd "$(dirname "$0")" && pwd)/box-provider-readiness" | ||
| [[ -x "$READINESS_GATE" ]] || { echo "box-agent: box-provider-readiness not found next to box-agent; refusing unwired dispatch" >&2; exit 3; } | ||
| box exec "$BOX" -- mkdir -p "$BOX_WORK" >/dev/null 2>&1 || true | ||
| "$READINESS_GATE" --box "$BOX" --cwd "$BOX_WORK" --repo "$REPO" --github-only || exit 3 |
There was a problem hiding this comment.
Major: this PR is 654 counted lines (37+6 box-agent, 219+0 box-provider-readiness, 392+0 box-provider-readiness.test.sh) against the repo's 500-line cap. The PR body's "493 lines, 3 files" predates this and the previous commit, which added ~198 more lines. Please split: land the readiness-gate + its core test coverage here, move the box-agent clone-identity rework (this hunk) to a follow-up PR.
| test_gate_success test_gate_bootstrap_failure \ | ||
| test_gate_clone_ordering test_gate_github_failure_no_clone; do "$t"; done | ||
| echo | ||
| echo "pass=$PASS fail=$FAIL" |
There was a problem hiding this comment.
Major: the PR body's "## How I verified" claims pass=31 fail=0 for this suite. Counting every pass "..." call reachable across the 21 test functions run in the trailing loop gives 40, not 31 — the recorded result predates commits 406fe4d/912b4d4 (which added test_gate_clone_ordering, test_gate_github_failure_no_clone, stale-env assertions, etc). Please rerun the suite and update the recorded result.
| gemini) CMD="unset GEMINI_API_KEY; set -a; . \"$PROBE_BASE/.gemini-personal/.gemini/.env\"; set +a; [ -n \"\${GEMINI_API_KEY:-}\" ] || exit 1; env -u GOOGLE_GENAI_USE_VERTEXAI -u GOOGLE_APPLICATION_CREDENTIALS -u GOOGLE_CLOUD_PROJECT -u GOOGLE_CLOUD_LOCATION GEMINI_CLI_HOME=\"$PROBE_BASE/.gemini-personal\" GEMINI_API_KEY=\"\$GEMINI_API_KEY\" GEMINI_CLI_TRUST_WORKSPACE=true gemini --skip-trust --yolo -m \"$MODEL\" -p \"\$P\"" ;; | ||
| cursor) CMD="unset CURSOR_API_KEY; set -a; . \"$PROBE_BASE/.agents/cursor.env\"; set +a; [ -n \"\${CURSOR_API_KEY:-}\" ] || exit 1; cursor-agent -p --trust --force --model \"$MODEL\" --output-format json \"\$P\"" ;; | ||
| devin) CMD="env -u DEVIN_API_KEY -u DEVIN_TOKEN -u WINDSURF_API_KEY devin --model \"$MODEL\" --respect-workspace-trust false --permission-mode dangerous -p \"\$P\"" ;; | ||
| kimi) CMD="kimi -m \"$MODEL\" -p \"\$P\"" ;; |
There was a problem hiding this comment.
Major: gemini (line 186), devin (188), and kimi (189) each get a real dispatch command here, but box-provider-readiness.test.sh's mkcli() only stubs muse/cursor-agent/pi/codex — there's no fake gemini, devin, or kimi binary anywhere in the suite, so none of these three branches ever executes in CI. The PR body only owns the devin gap ("no devin binary in this environment to re-check"); gemini and kimi aren't mentioned. Please add stub coverage for all three, or explicitly scope them out.
|
Stale-PR sweep: leaving this draft as-is. Per #319's body this branch stays untouched until its replacement is reviewable, and #319 is still open — itself blocked on the 500-line size cap (details in its thread). This is the earlier, narrower version of the same gate; merging it now would collide with #319. Not marking ready. |
912b4d4 to
218fb78
Compare
218fb78 to
836aab4
Compare
Closes #310
What
Add
box-provider-readiness, a gate that proves a provider can run on the recorded target Box — actual binary, canonical personal credential, pinned model route, and a bounded unique arithmetic/file-write probe — before the implementation sentinel runs, with bootstrap failures exiting nonzero, the sentinel unreached, and no ambient credential fallback. Wire the gate into thebox-agenton-Box dispatch path (after clone, before the runner ships) so a bootstrap failure exits 3 with the runner never copied and the Box stopped by the existing EXIT trap. The hook touches onlybox-agent, disjoint from PR309-owned files, with no manifest, installer, provisioning, network, reaper, heartbeat, or global-settings changes.Why
Issue #310: worker bootstraps failed on Boxes with valid manifests but a missing CLI or personal credential (Cursor auth failure on a reused Box, Devin absent despite environment credentials). The
box-agentpreflight only checkscommand -v, which cannot detect an unauthed provider, so this closes that gap at one real dispatch boundary.How I verified
The readiness tests drive the real helper and a real sentinel through a stub Box: fresh and resumed-after-transient successes reach it (
box newnever called), while missing binary, missing provider credential, invalid auth (leak redacted), wrong probe file, and probe timeout all fail with it unreached. Cursor coverage uses a faithful single-line{"result":"<n>",...}reply and asserts the pinnedcomposer-2.5JSON route end to end;0,00, and over-bound timeouts, wrong muse/cursor models,--modelon codex, and an unknown provider all fail before any probe runs (probe log empty). GitHub precedence is proven through the real selector code with fakegh/git: stale inheritedGH_TOKEN+GITHUB_TOKENstill yield the canonical identity, an expired canonical key and anls-remotefailure both fail as bootstrap failures, and a missing token file fails explicitly. NoHOMEis repurposed and no live network is touched (the earlier standalone live-ghtest is deleted). Two dispatch-gate tests drive the realbox-agentplus the real helper: a ready Box passes the live pi probe and the runner ships exactly once with onebox new; a missing credential exits 3 with the runner never shipped and the Box stopped. Probe invocation flags were validated against the real local CLIs; devin matches the canonicalbox-work/install-agent-clis.shtext (no devin binary in this environment to re-check). Live local Muse probe (muse exec --yolo --model muse-spark-1.3-contributor,23*47) returned1081in both reply and file, exit 0. Cursor has no local auth so its live path is covered by the faithful-shape fake; DeepSeek was never invoked (paused campaign route; the unknown-provider test only proves rejection without a probe). TheboxCLI is unauthorized in this environment (401), so Box-side proof runs coordinator-side with the exact commands in the SCR-310 result report; no fleet readiness is claimed here. Counted size vsmain: 493 lines, 3 files (box-agent10,box-provider-readiness178,box-provider-readiness.test.sh305).Assisted-by: muse-code:muse-spark