Repository navigation
Save agent validation screenshots to .autoforge/validation/ - #289
zwanski-start wants to merge 1 commit into
Conversation
Coding/testing/auto-improve prompt templates now tell agents to pass --filename=.autoforge/validation/<name>.png to playwright-cli screenshot. The directory is created on scaffold, added to .autoforge/.gitignore, and existing projects are migrated (v1 -> v2) by rewriting the default screenshot wording in their copied prompts. Closes AutoForgeAI#170 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🔍 QA report: ✅ PassedTester: Barbara (cubefarm agent, in a fresh session) · Round: 1 · Author: Ada Agents are now told to save screenshots under .autoforge/validation/, which is gitignored and documented. A new migration step (version 1 to 2) brings existing projects onto the new prompt wording. I tested that migration on version 0 and version 1 projects: the prompts came out byte-identical to the new templates, the gitignore and folder were added, a second run did nothing, and customized prompts were left alone. Running the real playwright-cli screenshot --filename command wrote the file into the gitignored folder, and ruff, mypy (no new errors) and the unit tests are clean.
🧪 Commands run
No browser screenshots were taken in this round. Posted by cubefarm · merges automatically once GitHub's checks pass |
Adaptations: the prompt migration lands as v3 (_migrate_v2_to_v3, CURRENT_MIGRATION_VERSION = 3) after AutoForgeAI#292's v2. Fixed a non-idempotent replacement (the testing-prompt note was appended again on every run, since its new text starts with the old one) and skip symlinked prompt files. New test_prompt_migrations.py: v3 converges pre-v3 template wording on the current templates, second run changes nothing, custom prompts untouched, validation/ + .gitignore entry created, v1 chain and new-project stamp; v2 idempotency and untouched prompts. Integrates AutoForgeAI#289 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AeH26pcsqoxPb3CDs8g1VN
…utoForgeAI#136) Adaptations: the testing-prompt migration lands as v4 (_migrate_v3_to_v4, CURRENT_MIGRATION_VERSION = 4) after AutoForgeAI#289's v3. It now recognizes the default STEP 2 as v3 leaves it, with or without the browser-unavailable note (the original only matched raw v1 text, which v3 rewrites first), skips symlinked prompt files and shares _apply_replacements with v3. Restored the return of _migrate_v2_to_v3 lost at the merge boundary. AutoForgeAI#283's regression-script step and testing header now defer to the bash scenario script for non-UI features, and STEP 2 names .autoforge/prompts/app_spec.txt so new projects get no root spec copy. test_prompts.py merges AutoForgeAI#283's and AutoForgeAI#333's tests; v4 idempotency, untouched custom prompts and the full v1->v4 chain are in test_prompt_migrations.py. Integrates AutoForgeAI#333 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AeH26pcsqoxPb3CDs8g1VN
Adaptations: merged with AutoForgeAI#334 quality gates (evidence is built only after the gates pass, so a rejected call copies nothing), AutoForgeAI#245 manual testing and AutoForgeAI#300 in database.py/feature_mcp.py/client.py/templates; screenshot examples point at .autoforge/validation/ (Phase 3, AutoForgeAI#289). The unsandboxed MCP server no longer copies screenshots through a symlinked .autoforge/evidence path (agent-plantable), and copies that the stored evidence no longer references are deleted after each evidence write (re-tests and rejected calls used to accumulate copies). Tests RED->GREEN. Integrates AutoForgeAI#264 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AeH26pcsqoxPb3CDs8g1VN
Closes #170
What changed
coding_prompt,testing_prompt,auto_improve_prompt): screenshot instructions now useplaywright-cli screenshot --filename=.autoforge/validation/<name>.png. They also tell the agent never to save screenshots anywhere else in the project and never to commit them.autoforge_paths.py:validation/added to the generated.autoforge/.gitignore. New helperget_validation_dir().prompts.py:scaffold_project_promptscreates.autoforge/validation/.CURRENT_MIGRATION_VERSION = 2). It refreshes.autoforge/.gitignore, creates the directory, and rewrites the default screenshot wording in existing projects' copiedcoding_prompt.md/testing_prompt.md.CLAUDE.md:.autoforge/validation/documented in the generated-project structure.Verification
ruff check,mypyon the changed modules,test_security.py, and the pytest suites (test_client,test_dependency_resolver,test_rate_limit_utils) all pass.migrate_project_to_current. Both prompts came out byte-identical to the new templates,.autoforge/.gitignorecontainedvalidation/, and the directory was created. A second run was a no-op..autoforge/validation/. In a git repo, a file placed there does not appear ingit status.playwright-cli screenshot --filename=.autoforge/validation/x.pngwrites the file relative to the cwd, and creates the directory if it is missing.Assumptions
playwright-cli snapshot) still go to.playwright-cli/. They are element-ref lookups rather than validation evidence, and that directory is already gitignored.auto_improve_promptis not copied into projects (it is loaded from the template), so it needs no migration.🤖 Generated with Claude Code