Skip to content

Add feature_split tool and stop agents passing unverified features (#68) - #318

Open
zwanski-start wants to merge 1 commit into
AutoForgeAI:masterfrom
zwanski-start:swarm/issue-68-guido
Open

zwanski-start wants to merge 1 commit into
AutoForgeAI:masterfrom
zwanski-start:swarm/issue-68-guido

Conversation

@zwanski-start

Copy link
Copy Markdown

Closes #68

Problem

Coding agents sometimes (a) skip a feature, or pass it partially done, when it's too large, and (b) mark a feature passing after browser automation failed to launch, assuming it works.

Changes

New MCP tool feature_split (mcp_server/feature_mcp.py, exposed to coding agents in client.py)

  • Takes a non-passing feature and 2–10 sub-features (name, description, steps, optional category).
  • Creates the sub-features atomically. They inherit the parent's category, dependencies and priority, so the orchestrator schedules them next, possibly on other agents. That is how this PR does the issue's "trigger another agent to break it down" idea.
  • The parent is released (in_progress=0) and made to depend on all its sub-features. It becomes ready again only after every sub-feature passes, and then it acts as the end-to-end check of the whole original feature. A cycle can't form because sub-features only depend on the parent's existing dependencies.
  • The tool validates its input, enforces MAX_DEPENDENCIES_PER_FEATURE, and changes nothing on error.

Prompt and tool-description hardening

  • coding_prompt.template.md:
    • New section "When a Feature Is Too Large: SPLIT IT", plus a rule that "too large" is never a valid skip reason.
    • New "If Browser Automation Fails" subsection: troubleshoot, and if verification is still impossible, clear in-progress and document the error. Never mark the feature passing.
    • Stronger "only mark passing when every step was implemented and verification actually ran" rule.
    • feature_split added to the allowed-tools list.
  • testing_prompt.template.md: same rule about browser failures.
  • The feature_mark_passing and feature_skip docstrings, which agents see as tool descriptions, now state these rules.
  • The parallel-mode single and batch assignment headers in prompts.py now mention feature_split and the never-pass-unverified rule. These headers are generated in code, so they also reach existing projects whose copied coding_prompt.md predates this change.
  • _CLI_STEP5_CONTENT and _CLI_TESTING_STEP2 (the v0→v1 migration text) are kept in sync with the templates.
  • In YOLO mode the browser-failure subsection is stripped together with STEP 5, as intended.

Verification

  • New test_feature_split.py (6 tests): creating and inheriting fields, the parent staying blocked until its sub-features pass (via feature_get_ready), rejecting passing or missing features, rejecting invalid input with no DB changes, and the dependency limit.
  • Checked through mcp.list_tools() that the tool registers with the expected schema.
  • ruff check . is clean.
  • mypy reports no new errors in the changed files. Errors that already existed in other files are unchanged.
  • test_security.py, test_client.py, test_dependency_resolver.py and test_feature_split.py all pass.
  • I didn't run a UI check: the UI is unchanged.

Assumptions / notes

  • I didn't add a hard server-side gate on feature_mark_passing, such as requiring verification evidence, because it would break the testing agent, the auto-improve agent and existing project prompts. Enforcement happens through prompts and tool descriptions.
  • Existing projects keep their copied coding_prompt.md. They get the new behaviour through the generated assignment headers and the tool descriptions, but not the full new template sections.
  • I didn't add a UI indicator for split features. They appear as normal features, and the parent shows as blocked by its sub-features in the dependency graph.

🤖 Generated with Claude Code

…AutoForgeAI#68)

Add a feature_split MCP tool so a coding agent that finds a feature too
large breaks it into 2-10 sub-features instead of skipping it or marking
it passing partially done. Sub-features inherit the parent's category,
dependencies and priority; the parent is released and made to depend on
them so it is re-verified end-to-end once they all pass.

Harden coding/testing prompts, the parallel-mode assignment headers and
the mark_passing/skip tool descriptions: a feature whose verification
could not run (e.g. the browser failed to open) must never be marked
passing, and 'too large' is not a valid skip reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zwanski-start

Copy link
Copy Markdown
Author

🔍 QA report: ✅ Passed

Tester: Dennis (cubefarm agent, in a fresh session) · Round: 1 · Author: Guido

The new feature_split MCP tool works end to end through a real MCP stdio client: sub-features inherit the parent's category, priority and dependencies; the parent is released and stays blocked until every sub-feature passes, then becomes ready again; nested splits also work; and invalid input is rejected without changing the database. The prompt hardening is consistent: the template and the v0→v1 migration text are identical, YOLO mode strips the browser-failure section, and the parallel single and batch headers carry the new rules. ruff is clean, mypy has the same 24 errors as master, and the security tests and the 6 new tests pass. The full pytest run has 21 failures that are identical on master (pre-existing, not caused by this PR). The UI is unchanged; it builds, and the Kanban board and dependency graph render a split project with no console errors.

Check Details
✅ feature_split over real MCP stdio (happy path) Started mcp_server.feature_mcp with PROJECT_DIR set to a temporary project. Created features Login, Checkout (depends on Login) and Receipt (depends on Checkout), then split Checkout into Cart and Pay. The result was sub_feature_ids [4,5] and parent_dependencies [1,4,5]. Before Login passed, only Login was ready; after it passed, Cart and Pay were ready and Checkout was blocked by [4,5]. Once the sub-features passed, Checkout was ready again.
✅ Nested split Split sub-feature Cart into Cart1 and Cart2. Cart then depended on [1,6,7]; the graph had no cycle and resolved correctly once the subtree passed.
✅ Input validation, no partial writes Called the tool with 1 part, 11 parts, steps given as a string, a non-string category, non-dict items, and feature_id=0. Each returned a pydantic or tool error and nothing was written to the database. The tests also cover a passing parent, a missing parent and the dependency limit.
✅ Tool exposure feature_split is in CODING_AGENT_TOOLS only. Its listed input schema has minItems 2, maxItems 10 and feature_id minimum 1.
✅ Prompt changes and migration sync STEP 5 of the template is identical to _CLI_STEP5_CONTENT, and STEP 2 of the testing template is identical to _CLI_TESTING_STEP2. A migrated v0 prompt contains the 'If Browser Automation Fails' section. YOLO mode strips that section but keeps the split guidance. get_single_feature_prompt and get_batch_feature_prompt include the feature_split rule and the rule against passing unverified features.
✅ Parallel orchestrator interaction (code review) _on_agent_complete clears in_progress and counts a failure only when the return code is non-zero. A split parent has in_progress=0 and is blocked by its sub-features, so it is not rescheduled until they pass.
✅ Regression vs master The full pytest run had the same 21 failures (test_devserver_security and others) on this branch and on an origin/master worktree, so none are introduced by this PR. mypy reported 24 errors in 9 files on both.
✅ Backend boot and REST Started uvicorn on 127.0.0.1:5247 with an isolated HOME and no API key. GET /api/projects returned 200, /api/projects/splitdemo/features returned 200, and an unknown project returned 404. The server log had no errors or tracebacks.
✅ UI render (Kanban and graph of a split project) The Kanban board shows Checkout pending with a 3/3 dependency badge, Receipt blocked, and the sub-features done. The dependency graph shows Login→Cart1/Cart2→Cart→Checkout→Receipt and Pay→Checkout. No console errors. Sub-features show the same '#2' badge as their parent because the badge shows priority; this is existing behaviour.
✅ Mobile layout At 390x844 the app renders, but the sidebar does not collapse and the graph is cramped. This is existing behaviour; the PR does not touch the UI.
⏭️ Playwright e2e Not run because the PR does not change the UI.
🧪 Commands run
Command Result
venv/bin/python -m ruff check . All checks passed
venv/bin/python -m pytest -q test_feature_split.py test_client.py test_dependency_resolver.py test_rate_limit_utils.py 61 passed
venv/bin/python test_security.py ALL TESTS PASSED
venv/bin/python test_security_integration.py ALL INTEGRATION TESTS PASSED
venv/bin/python -m pytest -q (PR branch, then origin/master worktree) 150 passed, 21 failed on the PR branch; the same 21 failures on master (pre-existing)
venv/bin/python -m mypy . (PR branch and master) 24 errors in 9 files on both; none in the changed lines
MCP stdio probe script (feature_create_bulk, feature_split, feature_get_ready, feature_get_blocked, feature_get_graph, feature_mark_passing) All behaviour as expected; invalid input rejected
prompts probe (template/migration sync, YOLO strip, single/batch headers) All checks true
npm --prefix ui run build Succeeded
npm --prefix ui run lint 0 errors, 6 warnings
uvicorn server.main:app --host 127.0.0.1 --port 5247 + curl /api/projects, /api/projects/splitdemo/features, /api/projects/nope/features 200, 200, 404

No browser screenshots were taken in this round.

Posted by cubefarm · merges automatically once GitHub's checks pass

cabana8471-arch pushed a commit to cabana8471-arch/autoforge that referenced this pull request Nov 8, 2026
…ied features

Adaptations: split-depth limit (MAX_SPLIT_DEPTH = 2): each sub-feature's description ends with 'Split from feature #N, split depth d.', which feature_split reads back to refuse a deeper split (no schema change). Ported AutoForgeAI#318's test that the parent becomes ready only after its sub-features pass. The coding prompt's tool list keeps my-features' quality gate tools (items 8-9); feature_split is item 10.

Integrates AutoForgeAI#263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AeH26pcsqoxPb3CDs8g1VN
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agent skips and passes features without testing

1 participant