Skip to content

zapier: document the S001 3-users-with-live-Zaps floor - #10

Merged
itsryanarmani merged 1 commit into
mainfrom
zapier-s001-floor
Aug 23, 2026
Merged

itsryanarmani merged 1 commit into
mainfrom
zapier-s001-floor

Conversation

@itsryanarmani

Copy link
Copy Markdown
Owner

Clearing every per-action task still leaves S001 — 3 distinct Zapier accounts each with a turned-on live Zap. Editor tests can't touch it, the CLI promote lists it as a failure, so it gates submission. Solo automation can't reach a submittable state. Documents the invite-link (3 accounts) and embed paths to get the real users.

Clearing every per-action task still leaves S001 (3 distinct Zapier
accounts with turned-on live Zaps), which editor tests can never clear
and which gates submission. Solo automation can't reach a submittable
state — document the invite-link and embed paths to get real users.
@itsryanarmani
itsryanarmani merged commit 3435227 into main Aug 23, 2026
@itsryanarmani
itsryanarmani deleted the zapier-s001-floor branch August 23, 2026 18:40
claude Bot pushed a commit that referenced this pull request Aug 30, 2026
…po collision, argv limit

- reverts-#N matching used bare substring search, so PR #1 was falsely
  flagged as reverted by a commit that actually reverts #10 (or any #1X).
  Require the number not be followed by another digit.
- fetch_default_branch swallowed all gh failures and silently fell back to
  "main", contradicting the fix in fetch_merged_prs that this same PR makes
  for the same class of bug.
- the per-repo branch cache file flattened "owner/repo" by replacing '/'
  with '_', so repos like "a_b/c" and "a/b_c" collided on one filename.
  Nest by the real path components instead.
- prs_json was passed to python via argv, which can exceed the OS
  argument-size limit for a repo with many closed PRs; read it from a file.
itsryanarmani added a commit that referenced this pull request Aug 31, 2026
* Measure the defect rate of each agent

Every merged pull request in this fleet ends with an Assisted-by
trailer naming the agent and model that wrote it. Nothing read it. At
fleet volume the first sign that one agent produces junk is a person
noticing, months late, that a certain kind of change keeps coming back.

Report, per agent, how often its merged work was reverted or hot-fixed
inside a window after it landed. Show the merged count beside every
rate: a zero percent rate from one merged PR is a sample size, not a
quality signal, and a report that hides the denominator invites exactly
that reading.

* fix: agent-defect-rate paginate/parse correctly and stop swallowing gh failures

gh api --paginate emits one JSON array per page when the --jq filter itself
wraps output in [...], so any repo with more than one page of merged PRs,
commits, or PR files broke json.loads() and silently collapsed to "no data" /
0% rates. Also stop treating gh api failures (auth, rate limit, network) as
empty results, since a working fallback that hides a real failure is worse
than a loud one. Fix a cross-repo PR-number collision that silently dropped
a defect PR from the TOTAL row when analyzing multiple repos with colliding
PR numbers, and tighten the hotfix-title match so "Fixture ..." titles no
longer count as a fix of an unrelated PR.

* fix: agent-defect-rate false revert match, silent branch fallback, repo collision, argv limit

- reverts-#N matching used bare substring search, so PR #1 was falsely
  flagged as reverted by a commit that actually reverts #10 (or any #1X).
  Require the number not be followed by another digit.
- fetch_default_branch swallowed all gh failures and silently fell back to
  "main", contradicting the fix in fetch_merged_prs that this same PR makes
  for the same class of bug.
- the per-repo branch cache file flattened "owner/repo" by replacing '/'
  with '_', so repos like "a_b/c" and "a/b_c" collided on one filename.
  Nest by the real path components instead.
- prs_json was passed to python via argv, which can exceed the OS
  argument-size limit for a repo with many closed PRs; read it from a file.

* fix: agent-defect-rate non-default-branch merges, GitHub revert suffix, cross-repo issue collision

- Filter PRs to the repo's default branch before counting them: revert
  commits are only ever fetched from that branch, so a PR merged
  elsewhere could never be flagged as a defect and just diluted the
  denominator.
- Match GitHub's own "Revert" button commit subject, which appends
  " (#N)" for the revert PR's own number, not just an exact string.
- Stop letting a cross-repo "Closes other/repo#100" collide with an
  unrelated same-numbered issue in a later same-repo "Fixes #100".

Adds test coverage for all three.

---------

Co-authored-by: vibecodereview[bot] <vibecodereview@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant