[zwanski-start]

Closes #69

What changed

Agents can now save test evidence on a feature: a summary of findings, a log of how they tested it, and screenshots. It shows up on the feature card so you can review it quickly.

Backend / MCP

• New nullable features.test_evidence JSON column, with a migration (_migrate_add_test_evidence_column). Shape: {result, summary, test_log[], screenshots[], recorded_at}.
• feature_mark_passing and feature_mark_failing take optional summary, test_log and screenshots arguments. Calls without them behave exactly as before, and existing evidence is kept (COALESCE).
• New feature_record_evidence tool for testing agents. A regression check that finds the feature still working shouldn't call mark_passing again, so this records the re-verification instead. It is in the coding and testing agent tool lists and disallowed for the read-only assistant.
• api/test_evidence.py handles normalization and limits: summary up to 4000 chars, up to 50 log entries, up to 10 screenshots. Screenshots must be image files inside the project directory, and rejected ones come back as warnings in the tool result. Accepted screenshots are copied into .autoforge/evidence/feature_<id>/ so later sessions can't overwrite or clean them up in .playwright-cli/. The folder is gitignored.
• Evidence is deliberately left out of Feature.to_dict() so it doesn't bloat agent context (feature_get_by_id etc.).
• REST: FeatureResponse.test_evidence is added, plus GET /api/projects/{name}/features/{id}/evidence/screenshots/{index}. The endpoint looks screenshots up by index and never takes a path from the client. Before serving, it re-checks that the stored path is an image inside the project (traversal, absolute paths outside the project and non-image files all return 404).

Prompts

• The coding prompt (STEP 6 and the tool rules), the testing prompt and the batch prompt now tell agents to include evidence, with an example call.

UI

• New TestEvidencePanel in the feature modal: a "Verified passing" or "Regression found" badge, the time it was recorded, findings, a numbered test log, and a screenshot thumbnail grid. Clicking a thumbnail opens the full image; if the file is missing, the path is shown instead.
• Completed feature cards show a small "Evidence" marker when evidence exists.

How I verified it

• python -m pytest test_test_evidence.py: 25 new tests pass. They cover path validation and traversal, evidence building and copying, all three MCP tools (including the already-passing guard leaving evidence untouched), the API response, and the screenshot endpoint (serving plus a tampered path giving 404).
• ruff check . is clean and python test_security.py passes. mypy reports no errors in the files I touched; the existing mypy errors are elsewhere.
• test_client.py, test_dependency_resolver.py, test_rate_limit_utils.py, test_devserver_security.py and test_ws_security.py give 21 failed / 122 passed, the same count as on untouched origin/master, so those failures were already there.
• npm run lint shows no errors (only the existing warnings) and npm run build succeeds.
• Manual test: I ran the server on port 5345 with a throwaway HOME and demo project. I called the real MCP tool functions to mark one feature passing (with log and screenshot) and one failing, then checked in the browser that the Kanban card shows the Evidence marker and the modal renders both the passing and the regression evidence, including the screenshot thumbnail.

Assumptions / notes

• Evidence is optional at the tool level, so older prompts and custom project prompts keep working. Prompts ask agents to provide it rather than requiring it.
• Each new record replaces the previous one (latest verification wins); there is no history. Screenshots from older records stay on disk under .autoforge/evidence/ and are not cleaned up.
• I couldn't push to AutoForgeAI/autoforge (403 for the available accounts), so this PR comes from the fork zwanski-start/autoforge.

🤖 Generated with Claude Code

[zwanski-start]

🔍 QA report: ✅ Passed

Tester: Barbara (cubefarm agent, in a fresh session) · Round: 1 · Author: Ken

The PR meets issue #69: agents can attach a summary, a test log and screenshots when marking features passing or failing, or through the new feature_record_evidence tool, and the UI shows them on the feature card and in the modal. Backend checks pass: 25 new tests, ruff, security tests, migration of an old-schema DB, and screenshot path validation that rejects traversal, outside paths, symlinks and non-images on both write and serve. The UI build, lint and real-browser checks (desktop and phone) pass. Minor notes: extra screenshot copies are left when mark_passing is rejected, and stale 'Verified passing' evidence stays on a feature later marked failing without new evidence. The e2e suite could not run because the Playwright browser binary is missing on this machine.

	Check	Details
✅	Issue acceptance: evidence saved on a feature and reviewable on the card	Called the real MCP tool functions on a demo project. Feature #1 was marked passing with summary, 3-step log and 2 screenshots; #2 marked passing then failing with regression evidence; #3 passing without evidence; #4 with a screenshot later deleted. The Kanban 'Evidence' marker appears only on #1 and #4. The modal shows the badge, recorded time, findings, numbered test log and screenshot thumbnails (both images loaded, naturalWidth 640).
✅	Regression evidence display	Feature #2 (pending) shows a red 'Regression found' badge with findings and log.
✅	Missing screenshot fallback	After deleting the copied file for #4, the thumbnail falls back to the ImageOff icon and path; the only console error is that expected 404.
✅	Screenshot path security (MCP side)	A symlink to /etc/hostname named .png, ../../etc/passwd.png, /etc/x.png, secret.txt and a missing file were all rejected with warnings; a duplicate was de-duplicated; accepted files were copied to .autoforge/evidence/feature_1/.
✅	Screenshot endpoint status codes	Valid indexes return 200 image/png. Index out of range, -1, deleted file, feature without evidence, unknown feature, unknown project and ..%2F path all return 404; a non-numeric index returns 422.
✅	Backward-compatible migration	A features.db created with master's code opened with the PR code: the test_evidence column was added, existing rows were intact, and to_dict() does not include evidence.
✅	Tool semantics	Calls without evidence keep existing evidence (COALESCE). The already-passing guard still returns an error. feature_record_evidence with no input returns an error, and an unknown ID returns 'not found'.
✅	Security not weakened	security.py and the localhost middleware are unchanged; feature_record_evidence is added to DISALLOWED_ASSISTANT_TOOLS; no secrets are logged.
✅	Responsive layout	At 390x844 the modal evidence panel fits (scrollWidth == innerWidth == 390) and the 2-column thumbnail grid renders.
✅	Kanban, Graph and WebSocket	The Kanban and DAG graph render; /ws/projects/demo connects and streams agent_status and dev_server_status.
✅	Process cleanup	The uvicorn server on 5443 was stopped; no listeners remain on 5443 or 5173.
⏭️	UI Playwright e2e suite	Every spec failed at browserType.launch because chrome-headless-shell is not installed on this machine; npx playwright install chromium-headless-shell timed out after 300s. This is an environment issue unrelated to the PR, and the existing specs cover the assistant panel and tooltips, not this feature.
✅	Minor: extra screenshot copies when mark_passing is rejected	Not blocking. Evidence is built (screenshots copied) before the 'already passing' guard, so a rejected call leaves an extra copy under .autoforge/evidence/ (that folder is gitignored).
✅	Minor: stale passing evidence on a failing feature	Not blocking. feature_mark_failing without evidence keeps the earlier result='passing' record, so the modal can show 'Verified passing' on a pending feature. The PR describes keeping old evidence as intended.

🧪 Commands run

Command	Result
python -m pytest test_test_evidence.py -q	25 passed
ruff check .	All checks passed
python test_security.py	ALL TESTS PASSED
mypy . (PR vs origin/master)	Only difference is 1 error in the local venv/bin/activate_this.py; no new errors in changed files
python -m pytest test_client.py test_dependency_resolver.py test_rate_limit_utils.py test_devserver_security.py test_ws_security.py test_security_integration.py	21 failed / 131 passed, identical on origin/master (failures already there)
npm --prefix ui run lint	0 errors, 6 warnings (already there)
npm --prefix ui run build	tsc -b && vite build succeeded
HOME=<tmp> uvicorn server.main:app --port 5443 + curl probes of /api/projects/demo/features and the evidence screenshot endpoint	Status codes as expected (200/404/422)
python websockets client to ws://127.0.0.1:5443/ws/projects/demo	Connected and received agent_status and dev_server_status
VITE_API_PORT=5443 CI=1 npm --prefix ui run test:e2e	Not run: chromium-headless-shell missing and install timed out (environment)

No browser screenshots were taken in this round.

_{Posted by cubefarm · merges automatically once GitHub's checks pass}