Repository navigation
zapier: document the S001 3-users-with-live-Zaps floor - #10
Merged
Merged
Conversation
Clearing every per-action task still leaves S001 (3 distinct Zapier accounts with turned-on live Zaps), which editor tests can never clear and which gates submission. Solo automation can't reach a submittable state — document the invite-link and embed paths to get real users.
claude Bot
pushed a commit
that referenced
this pull request
Aug 30, 2026
…po collision, argv limit - reverts-#N matching used bare substring search, so PR #1 was falsely flagged as reverted by a commit that actually reverts #10 (or any #1X). Require the number not be followed by another digit. - fetch_default_branch swallowed all gh failures and silently fell back to "main", contradicting the fix in fetch_merged_prs that this same PR makes for the same class of bug. - the per-repo branch cache file flattened "owner/repo" by replacing '/' with '_', so repos like "a_b/c" and "a/b_c" collided on one filename. Nest by the real path components instead. - prs_json was passed to python via argv, which can exceed the OS argument-size limit for a repo with many closed PRs; read it from a file.
itsryanarmani
added a commit
that referenced
this pull request
Aug 31, 2026
* Measure the defect rate of each agent Every merged pull request in this fleet ends with an Assisted-by trailer naming the agent and model that wrote it. Nothing read it. At fleet volume the first sign that one agent produces junk is a person noticing, months late, that a certain kind of change keeps coming back. Report, per agent, how often its merged work was reverted or hot-fixed inside a window after it landed. Show the merged count beside every rate: a zero percent rate from one merged PR is a sample size, not a quality signal, and a report that hides the denominator invites exactly that reading. * fix: agent-defect-rate paginate/parse correctly and stop swallowing gh failures gh api --paginate emits one JSON array per page when the --jq filter itself wraps output in [...], so any repo with more than one page of merged PRs, commits, or PR files broke json.loads() and silently collapsed to "no data" / 0% rates. Also stop treating gh api failures (auth, rate limit, network) as empty results, since a working fallback that hides a real failure is worse than a loud one. Fix a cross-repo PR-number collision that silently dropped a defect PR from the TOTAL row when analyzing multiple repos with colliding PR numbers, and tighten the hotfix-title match so "Fixture ..." titles no longer count as a fix of an unrelated PR. * fix: agent-defect-rate false revert match, silent branch fallback, repo collision, argv limit - reverts-#N matching used bare substring search, so PR #1 was falsely flagged as reverted by a commit that actually reverts #10 (or any #1X). Require the number not be followed by another digit. - fetch_default_branch swallowed all gh failures and silently fell back to "main", contradicting the fix in fetch_merged_prs that this same PR makes for the same class of bug. - the per-repo branch cache file flattened "owner/repo" by replacing '/' with '_', so repos like "a_b/c" and "a/b_c" collided on one filename. Nest by the real path components instead. - prs_json was passed to python via argv, which can exceed the OS argument-size limit for a repo with many closed PRs; read it from a file. * fix: agent-defect-rate non-default-branch merges, GitHub revert suffix, cross-repo issue collision - Filter PRs to the repo's default branch before counting them: revert commits are only ever fetched from that branch, so a PR merged elsewhere could never be flagged as a defect and just diluted the denominator. - Match GitHub's own "Revert" button commit subject, which appends " (#N)" for the revert PR's own number, not just an exact string. - Stop letting a cross-repo "Closes other/repo#100" collide with an unrelated same-numbered issue in a later same-repo "Fixes #100". Adds test coverage for all three. --------- Co-authored-by: vibecodereview[bot] <vibecodereview@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Clearing every per-action task still leaves S001 — 3 distinct Zapier accounts each with a turned-on live Zap. Editor tests can't touch it, the CLI
promotelists it as a failure, so it gates submission. Solo automation can't reach a submittable state. Documents the invite-link (3 accounts) and embed paths to get the real users.