Fix Poetry matrix flakes on PyPI/patch API transport blips

[Mikola Lysenko (mikolalysenko)]

Merged current main (045d7ec) at 10c7b083 to re-validate against the updated CLI API retry path. The merge was clean and touches none of this PR's files; the Python suite passes locally on the merged tree (148 tests). CI and Bugbot are re-running on the new head. The previous head ef3beaf0 was fully green (241 checks), with no unresolved review threads.

Problem

Transient PyPI and patch API failures can interrupt the native Poetry compatibility matrix. A retry must distinguish those failures from functional regressions: for example, an informational relock timeout must not cause a failed rescan byte invariant to be retried and replaced by a passing result.

Change

Retry a case from a fresh directory at most three times, with 10s/20s backoff, only when every failed required check has transport evidence from its own operation. Typed command failures and HTTP/transport exceptions retain partial check results. Any earlier independent functional failure vetoes a retry. Successful recovered warnings, informational checks, and expected negative probes cannot trigger one.

CLI transport errors returned in structured output are recognized even when the process exits successfully. Lock and pyproject byte invariants remain independent. A required setup transport failure stops dependent probes after recording those invariants. Persistent transport failures remain red after the attempt limit.

Failed attempts retain their logs under attempts/<case>/<n>/, which the workflow uploads, and the final row records transportRetries. Job output includes failed check details, including checks retained when a later operation raises. Production CLI behavior and other ecosystem harnesses are unchanged.

Validation

• Full Python suite on the fix: 148 tests run, 147 passed and one platform-specific skip.
• Six false-green production-flow regressions fail on the original PR head and pass with the fix. They cover unrelated informational/expected-negative/recovered transport messages, later exceptions, and independent file churn.
• Independent review of the exact fix commit: nine flow scenarios and 14 classifier assertions pass.
• Existing fresh-case recreation, retry limit, backoff, evidence retention, and failure-detail tests remain covered. Syntax parsing and diff whitespace checks pass.
• The native Poetry matrix, repository workflows, and Bugbot completed successfully on ef3beaf0. Local flow regressions additionally verify retry causality with external commands stubbed.

The original flake sample established a transport cause for only one of 12 inspected failures; other failures may be functional and must remain red.

Original implementation: Claude Code session. Review correction: ef3beaf0.

🤖 Generated with Claude Code

---

Note

Medium Risk
Retry eligibility logic is subtle; incorrect classification could hide functional failures or over-retry flaky CI, though scope is limited to the Poetry backtest harness and production CLI behavior is unchanged.

Overview
Adds transport-aware retries to the native Poetry compatibility backtest so transient PyPI/patch API blips do not fail the matrix, without turning real regressions green.

backtest-poetry.py classifies terminal transport errors (Poetry/pip connection failures, CLI JSON errors on exit 0, 5xx/429) per failed required check via record_check / transportFailures. Cases re-run from a clean capture directory at most three times with backoff only when every failed required check has causal transport evidence; informational checks, recovered retry warnings, independent byte invariants, and earlier functional failures block retries. Failed attempt logs land under attempts/<case>/<n>/, appear in transportRetries, and are uploaded by CI; job output prints per-check failure notes.

Docs describe the policy. New unit tests (PoetryTransportRetryTests, test_poetry_retry.py) cover classification, false-green guards, and full case flow with stubbed commands.

^{Reviewed by Cursor Bugbot for commit 722582c. Configure here.}

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

[burn-down agent] Labeled Ready for review at af5ecde4be7f6bd4a274fd95ef497e855826faa3.

• CI: 241/245 check runs green on this head (4 skipped by path/matrix filters), 0 failing, including the Poetry compatibility matrix. Mergeable. 7 commits behind main, but none of them touch this PR's files (scripts/backtest-poetry.py, its tests, the Poetry workflow and doc), so no re-sync was needed.
• Bugbot: reviewed af5ecde and found no issues. No unresolved review threads.
• What to look at: which exceptions the backtest now treats as transient and retries, versus real failures that should still fail the case. Retrying too broadly could hide real regressions in the matrix.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Review updated for ef3beaf0dd076247b82f37b8613aedc3e961cff0: Ready to merge as-is from this review. Final-head CI is complete: 241 successful checks, 4 skipped; no failures or pending checks. Bugbot passed, there are no unresolved review threads, and the PR is mergeable.

The original retry classifier let unrelated informational errors, expected negative probes, recovered warnings, or a later exception erase an earlier required functional failure. The fix associates transport evidence with the failing operation and preserves partial check results. Every failed required check must have causal transport evidence; an independent functional invariant vetoes retry. Structured CLI transport errors still qualify, and required setup failures stop dependent probes after preserving file invariants.

Validation: the Python suite ran 148 tests: 147 passed and one platform-specific skip. Six false-green flow scenarios fail on the original PR head and pass with this fix. A separate exact-commit review passed nine flow scenarios and 14 classifier assertions. Fresh directory recreation, three-attempt bound, backoff, and archived attempt evidence remain covered. The commit merges cleanly with current main.

No remaining code finding from this review. The Ready label has been restored after all checks completed on the fixed commit.

[Mikola Lysenko (mikolalysenko)]

Agreed, this is a real gap. The classifier is case-wide: it scans every log, including steps that succeed after a retried request (pip prints Retrying ... NewConnectionError warnings) and informational steps such as relock and warm install. So a functional failure in rollbackRestoresLockBytes can be retried and then pass, which contradicts "functional failures are never retried".

The fix you describe is the right one: tie the transport evidence to the failing required check's own operation (its log, or the structured CLI output), and never retry when an independent functional failure is present. I won't push to this branch while you prepare the correction, so we don't collide. I'll pick up CI and Bugbot on your push.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 722582c. Configure here.}

[Mikola Lysenko (mikolalysenko)]

[burn-down agent] Re-verified at the current head 722582c (the earlier ready-for-review comment cited an older commit). The Ready for review label stays.

• CI: every GitHub Actions workflow run on 722582c passed.
• Bugbot: reviewed 722582c with no new issues. The finding on 10c7b08 is fixed and its thread is resolved.
• Merges cleanly into main, and no review threads are open.

Slack announcement not sent: this run has no Slack send tool.

---

Generated by Claude Code