[zwanski-start]

Summary

Agents can now save test evidence: a summary of how they tested and what they found, an optional raw log, and optional screenshots. The evidence is stored on the feature and shown on its card, so a person can quickly review how each feature was verified.

Backend

• DB: new features.test_evidence JSON column, with an auto-migration for existing databases. Each entry has {recorded_at, agent_type, result: pass|fail, summary, log, screenshots[]}. Only the 10 most recent entries are kept. Summaries are capped at 4k chars, logs at 20k chars and screenshots at 10 per entry. The column is deliberately left out of Feature.to_dict() so it doesn't make agent and orchestrator payloads bigger.
• MCP:
  • feature_mark_passing and feature_mark_failing take optional test_summary, test_log and screenshots. The evidence is written in the same transaction as the status change. Calls without evidence behave exactly as before.
  • New feature_record_evidence(feature_id, result, test_summary, ...) records evidence without changing the status. Testing agents need this because they are told not to call feature_mark_passing again on a feature that is already passing.
  • Screenshot paths are turned into project-relative POSIX paths. Paths outside the project are dropped.
  • client.py passes AGENT_TYPE to the MCP server so each entry records whether a coding or testing agent captured it. The new tool is added to the coding and testing agent allowlists and is explicitly disallowed for the read-only assistant.
• API:
  • The feature list and detail responses now include evidence_count.
  • GET /api/projects/{p}/features/{id}/evidence returns the full entries. They are fetched only when a feature is opened, so the list poll that runs every 5s stays small.
  • GET /api/projects/{p}/features/{id}/evidence/screenshot?path= serves a screenshot only when that exact path appears in the feature's evidence, the path resolves inside the project directory, and it is an image (png/jpg/gif/webp).

UI

• The feature modal has a Test Evidence section directly under the description, newest entry first. Each entry shows pass/fail, which agent recorded it, a timestamp, the summary, screenshot thumbnails (with a fallback when the file is gone) and a collapsible raw log.
• Feature cards show an "Evidence" indicator when evidence exists.

Prompts

• The coding and testing templates and the batch prompt now tell agents to attach test_summary (plus a log and screenshots when available) when they mark features. Testing agents call feature_record_evidence when a regression check passes.
• I checked that the YOLO prompt stripping still works.

Verification

• New test_feature_evidence.py (7 tests) covers the MCP tools (record, cap, truncate, validation, path normalisation) and the API endpoints, including the screenshot path restrictions.
• Existing tests: test_security.py passes, and pytest test_client.py test_dependency_resolver.py test_rate_limit_utils.py gives 55 passed.
• ruff check . is clean, and mypy reports no new errors in the changed files.
• npm run lint shows 0 errors (the 6 warnings were already there), and npm run build succeeds.
• Checked by hand in the browser: I ran the backend with an isolated HOME and the Vite dev server, seeded a demo project through the MCP functions, then confirmed the card indicator, the evidence list, the screenshot thumbnail, the missing-file fallback and the log toggle all work.

Assumptions / notes

• Evidence is stored as JSON in features.db instead of a separate table. That keeps the change small and matches how human_input_request is stored.
• Screenshots are referenced by path and not copied. Playwright CLI saves them to .playwright-cli/, so if someone deletes that folder the UI shows "file unavailable".
• Existing projects keep their own copies of the prompts, so they won't get the updated prompt text automatically. The MCP tool descriptions still tell agents to attach evidence.

Closes #69

🤖 Generated with Claude Code

[zwanski-start]

🔍 QA report: ✅ Passed

Tester: Ada (cubefarm agent, in a fresh session) · Round: 1 · Author: Dennis

PR #319 does what issue #69 asks: agents can attach a test summary, raw log and screenshots when they mark a feature passing or failing, or through the new feature_record_evidence tool, and reviewers see it on the card and in the feature modal. The migration from an old features.db works, the screenshot endpoint only serves listed image files inside the project (traversal attempts get 404), and calls without evidence behave as before. ruff, all Python tests, the UI build and lint pass, mypy shows no new errors, and the UI works in the browser at desktop and phone sizes. One small non-blocking issue: an open modal doesn't show evidence recorded after it was opened until it is reopened, which is how the modal already handled feature status before this PR.

	Check	Details
✅	Acceptance: evidence saved on the feature and viewable on its card	Recorded evidence through the MCP functions (mark_passing with summary, log and screenshots; mark_failing; record_evidence). The Kanban cards show an 'Evidence' indicator only for features #1 and #3, and the modal shows a Test Evidence section under the description, newest first, with Passed/Failed and Coding/Testing agent badges and timestamps.
✅	Acceptance: log of how it tested and what it found	The summary is shown with line breaks kept. The raw log is behind a Show/Hide test log toggle, and a <script> payload in it renders as plain text (no XSS).
✅	Backward-compatible DB migration	Created features.db with master's code (no test_evidence column). Opening it with the PR code added test_evidence TEXT NULL; existing rows and the list API still work, and evidence_count is 0 where there is no evidence.
✅	MCP tool behaviour and validation	mark_passing without evidence returns evidence_recorded:false and stores nothing; a second mark_passing still returns an 'already passing' error; record_evidence on an unknown id or with an empty summary returns an error; an invalid result value is rejected by the MCP argument validation; screenshot paths outside the project (/etc/passwd) are dropped and absolute paths inside it become project-relative POSIX paths; AGENT_TYPE is recorded.
✅	REST endpoints, status codes and security	/evidence returns 200 with the entries and skips a malformed entry with a logged warning; 404 for an unknown feature or project, 400 for an invalid project name, 422 for a non-numeric id. The screenshot endpoint returns 200 image/png for the listed file; 404 for a listed but missing file, unlisted paths, ../../../etc/passwd and .autoforge/features.db; 422 without a path.
✅	Screenshot thumbnail and missing-file fallback	The listed PNG renders as a thumbnail linked to the endpoint; the deleted file shows '.playwright-cli/gone.png (file unavailable)'. The only console error is the expected 404 for that file.
✅	Graph view and responsive layout	The dependency graph renders, and clicking a node opens the modal with the failing evidence and its screenshot. At 390x844 the modal and evidence list fit without overflow.
✅	Live refresh while the modal is open	Minor, not blocking: evidence recorded while the modal was open did not appear until it was reopened, because the modal gets a snapshot of the feature when it opens (it already worked this way before this PR). After reopening, the new Failed entry is shown first.
✅	Security allowlist, assistant restrictions and YOLO prompt	test_security and the integration tests pass; feature_record_evidence is added to the coding and testing tool lists and blocked for the assistant; YOLO stripping of the coding prompt still applies.
✅	WebSocket /ws/projects/demo	Connected and received agent_status, dev_server_status and progress messages.
⏭️	UI e2e suite	Ran with the system Chromium because Playwright's own browser was missing and the download timed out. All 15 tests skipped themselves: they look for an old .neo-dropdown-item class that the current UI doesn't render. The ui/e2e files are identical to master, so this is not caused by the PR.

🧪 Commands run

Command	Result
ruff check .	All checks passed
mypy . (PR vs origin/master worktree)	24 errors on both; the only extra line on the PR side is in venv/, which is not repo code; no new errors
python test_security.py	ALL TESTS PASSED
python test_security_integration.py	ALL INTEGRATION TESTS PASSED
python -m pytest test_feature_evidence.py test_client.py test_dependency_resolver.py test_rate_limit_utils.py	62 passed
npm --prefix ui run build	tsc -b && vite build succeeded
npm --prefix ui run lint	0 errors, 6 warnings (already on master)
HOME=<isolated> uvicorn server.main:app --port 5571 + curl probes of the evidence and screenshot endpoints	Status codes as expected (200/400/404/422); no tracebacks in the server log
npx playwright test (system Chromium, VITE_API_PORT=5571)	15 skipped: the tests use an old selector; ui/e2e is identical to master
python websockets client to ws://127.0.0.1:5571/ws/projects/demo	Connected; received agent_status, dev_server_status and progress

No browser screenshots were taken in this round.

_{Posted by cubefarm · merges automatically once GitHub's checks pass}