[zwanski-start]

Closes #20

Problem

Testing agents picked passing features randomly (ORDER BY random()) or from an in-memory _recently_tested set held by the orchestrator. That set was lost every time the orchestrator restarted, so the same features got tested again and again while others were never tested. That wastes tokens and time.

Changes

• api/database.py: new Feature.regression_count column (INTEGER NOT NULL DEFAULT 0, indexed) counts how many times a feature has been picked for regression testing. A new migration _migrate_add_regression_count_column adds it to existing features.db files, and to_dict() now includes it.
• parallel_orchestrator.py:
  • _get_test_batch() always picks the passing, not-in-progress features with the lowest regression_count. Ties are broken by impact (number of dependents, then the feature's own dependency count, capped at 3), then randomly. The counter goes up in the DB as soon as a feature is picked, so a feature is only tested again after every other passing feature has caught up. This carries over across orchestrator restarts.
  • _get_random_passing_feature() (legacy single mode) also picks least-tested first and increments the counter.
  • Removed the in-memory _recently_tested set. Each testing agent's full batch is now tracked (_testing_batch_ids), so concurrent testing agents skip every feature already handed to another agent, not just that agent's primary feature.
• test_regression_selection.py (new, 6 tests) covers: full coverage before any repeat, persistence across orchestrator instances, skipping failing and in-progress features, impact ordering within the same count, the legacy single-feature path, and the legacy DB migration.
• CLAUDE.md: added the new test file to the testing commands.

Verification

• python -m pytest test_regression_selection.py: 6 passed
• ruff check .: clean
• mypy on the changed files: no new errors (the remaining errors are in temp_cleanup.py and already exist on master)
• python test_security.py, pytest test_dependency_resolver.py test_client.py: pass
• No UI changes, so I did not check anything in a browser.

Assumptions

• The counter goes up when a feature is selected for a test batch, not when the test finishes. This keeps concurrent agents from picking the same feature. If a testing agent crashes, its features count as tested once. That is acceptable because they just move to the back of the rotation.
• regression_count is not reset when a feature regresses and is fixed again. The fix itself checks the feature, so leaving it later in the rotation is reasonable.

🤖 Generated with Claude Code

[zwanski-start]

🔍 QA report: ✅ Passed

Tester: Dennis (cubefarm agent, in a fresh session) · Round: 1 · Author: Ada

Adds a persisted, indexed Feature.regression_count column (auto-migrated on existing features.db files) and makes the orchestrator always pick the least-tested passing features, so every feature is covered before any repeat, even across orchestrator restarts. Concurrent testing agents now skip every feature already handed to another agent. New tests, ruff, mypy (no new errors), the security suites and the existing unit tests all pass; a master-created DB migrates cleanly and the app runs without errors.

	Check	Details
✅	Issue #20: no duplicate testing until all passing features are covered	Probe with 7 passing features and 1 failing, batch size 3: the batches covered all 7 before any repeat. After the counts evened out, a batch of 10 returned the count-1 features first, then the count-2 features ordered by impact. The failing feature stayed at regression_count 0.
✅	Persistence across orchestrator restart	A fresh ParallelOrchestrator on the same DB continued from the stored counts; the PR's test_selection_persists_across_orchestrator_restarts also passes.
✅	Concurrent testing agents get disjoint batches	Registered fake running testing agents with their batches in _testing_batch_ids: b1=[1,2,3], b2=[7,6,5], b3=[4]. No overlap between any of them.
✅	Backward-compatible migration of a master-created features.db	Created the DB with origin/master code (no regression_count column), then opened it with the PR code: the column was added as INTEGER NOT NULL DEFAULT 0 along with ix_features_regression_count, and existing rows read as 0.
✅	Legacy _get_random_passing_feature path	Returns the least-tested feature and increments its counter (unit test passes; the probe returned feature 5 at the lowest count).
✅	Edge cases: batch size 0, no candidates	_get_test_batch(0) returns [] without error; a batch size larger than the number of candidates returns all candidates.
✅	Code review	The counter is updated atomically with UPDATE ... SET coalesce(regression_count,0)+1, and _testing_batch_ids is cleaned up when an agent completes and on stop_all. Minor: counts are bumped before the spawn re-check, so a refused spawn still counts as one test (the PR states this trade-off). There is no leftover debug code and the security allowlist is untouched.
✅	FastAPI boot and REST routes	uvicorn server.main:app on 127.0.0.1:5247: /api/health 200; project registration 200; features list 200; feature 1 200; feature 999 404; graph 200. A feature created over the API stored regression_count=0. No errors in the server log.
✅	UI smoke test in a real browser (Playwright)	Dashboard (Live WebSocket), Kanban and Graph render for the migrated project; 0 console errors or warnings. At phone width (390px) the page renders; the cramped sidebar is pre-existing and not touched by this PR.
⏭️	UI build, lint and Playwright end-to-end tests	The PR has no UI changes (only api/database.py, parallel_orchestrator.py, a new test file and CLAUDE.md).

🧪 Commands run

Command	Result
venv/bin/python -m pytest -q test_regression_selection.py	6 passed
venv/bin/python -m ruff check .	All checks passed
venv/bin/python -m mypy . (PR vs origin/master worktree)	24 errors in 9 files on both; diff of error lists identical (no new errors)
venv/bin/python test_security.py	ALL TESTS PASSED
venv/bin/python test_security_integration.py	ALL INTEGRATION TESTS PASSED
venv/bin/python -m pytest -q test_client.py test_dependency_resolver.py test_rate_limit_utils.py	55 passed
venv/bin/python -m pytest -q test_devserver_security.py test_ws_security.py	21 failed, 67 passed, identical on origin/master (pre-existing, unrelated)
Throwaway probe scripts (master-created DB -> PR migration, concurrent batches, restart, batch 0/10)	All behaved as expected
HOME=<scratch> venv/bin/python -m uvicorn server.main:app --host 127.0.0.1 --port 5247 + curl of /api routes	Health 200, features 200, missing feature 404, graph 200

No browser screenshots were taken in this round.

_{Posted by cubefarm · merges automatically once GitHub's checks pass}