Skip to content

Fix #14: Remove shell access from web-UI spec chat and tell the model why - #298

Open
zwanski-start wants to merge 1 commit into
AutoForgeAI:masterfrom
zwanski-start:swarm/issue-14-ada
Open

zwanski-start wants to merge 1 commit into
AutoForgeAI:masterfrom
zwanski-start:swarm/issue-14-ada

Conversation

@zwanski-start

Copy link
Copy Markdown

Closes #14

Problem

The web UI's spec creation chat (SpecChatSession) only pre-approved Read/Write/Edit/Glob/Web tools and never removed Bash. When a user dropped a binary file (e.g. .parquet) into the project, the model tried to read it with Python/shell, each call failed with This command requires approval, and the model then asked the user to approve — which the web UI has no way to do.

Changes

  • server/services/chat_constants.py: new NO_SHELL_TOOLS and NO_SHELL_PROMPT_NOTE. The note tells the model it has no shell, that there is no approval prompt (so never ask for one), and that for formats it can't read (parquet, xlsx, sqlite, archives…) it should ask the user to describe the structure or paste/attach an excerpt.
  • server/services/spec_chat_session.py: Bash is now in disallowed_tools and in the settings deny list, and the note is appended to the system prompt (written to the project CLAUDE.md).
  • server/services/expand_chat_session.py: already disallowed Bash; now also appends the note so the model doesn't ask for approvals there either.
  • test_chat_sessions_no_shell.py: tests that capture the ClaudeAgentOptions built by both sessions and check Bash is removed/denied and the note is in the prompt.

The assistant chat already disallows Bash and states it in its prompt, so it is unchanged.

Verification

  • pytest test_chat_sessions_no_shell.py — 4 passed
  • pytest test_client.py test_ws_security.py test_security.py — 44 passed
  • ruff check . — clean
  • mypy . — no new errors (the remaining unused type: ignore errors on the effort= lines exist on master)
  • No UI changes, so no browser check was needed.

Assumptions

  • The fix is to make the limitation explicit rather than give spec creation a shell: spec chat runs with the sandbox disabled, so letting it run arbitrary commands would be a meaningful security change. Reading binary data files can be revisited separately (e.g. server-side extraction like the existing document attachment support).
  • The note goes only into the server-side sessions, not .claude/commands/create-spec.md, because that file is also used by the CLI flow, where Claude Code can prompt for approval.

🤖 Generated with Claude Code

…l the model why

The spec creation chat never removed the Bash tool, so the model tried to
run commands (e.g. Python to read a .parquet file) that failed with
'This command requires approval' and then asked the user to approve them,
which is impossible in the web UI.

- Deny/disallow Bash in the spec chat session
- Append an environment note to spec and expand chat system prompts: no
  shell, no approval prompts, and ask the user to describe binary files
- Add tests covering both sessions

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zwanski-start

Copy link
Copy Markdown
Author

🔍 QA report: ✅ Passed

Tester: Dennis (cubefarm agent, in a fresh session) · Round: 1 · Author: Ada

The web-UI spec chat now removes Bash (disallowed_tools) and denies it in its settings file. Both spec and expand chats append a note telling the model it has no shell and no approval prompt, and to ask the user to describe or paste unreadable files such as .parquet. I confirmed the SDK passes this as --disallowedTools Bash and that Claude Code 2.1.303 then leaves Bash out of its tool list. A live spec WebSocket session wrote the note to CLAUDE.md and the deny list to settings, and degraded cleanly with no credentials. Lint and new tests pass; the 21 pytest failures and 24 mypy errors are the same on master.

Check Details
✅ Issue #14: Bash unavailable in web-UI spec chat In the diff, spec_chat_session passes disallowed_tools=['Bash'] and adds deny ['Bash'] to its settings file. SDK 0.1.81 maps disallowed_tools to --disallowedTools. A Claude Code 2.1.303 stream-json init with a fake key and empty HOME lists Bash without the flag and leaves it out with --disallowedTools Bash.
✅ Issue #14: model told it cannot ask for approval Live spec WebSocket 'start' on a registered project containing a fake data.parquet wrote CLAUDE.md ending with the 'Environment Constraints (AutoForge Web UI)' note (no shell, never ask for approval, ask the user to describe or paste binary files like .parquet). .autoforge/.claude_settings.json contained deny: ["Bash"].
✅ Expand chat gets the same note expand_chat_session appends NO_SHELL_PROMPT_NOTE to system_prompt. Bash was already disallowed there. Covered by the new unit test.
✅ No dependence on shell in create-spec skill Grepped .claude/commands/create-spec.md and expand-project.md: no Bash or shell steps that would break with Bash removed.
✅ Graceful degradation without API key or login Spec chat in the browser showed 'Not logged in · Please run /login' and stayed connected, with 0 console errors.
✅ Regression vs master (pytest, mypy) Full pytest: 148 passed, 21 failed on the PR. The failing list is identical to an origin/master worktree run (all test_devserver_security, pre-existing). mypy reports 24 errors on both branches, and the unused-ignore errors on the effort= lines exist on master too.
✅ Security not weakened security.py and the localhost middleware are untouched; the spec chat is more restricted than before. Non-blocking: the CLI also offers a Monitor tool (it can run commands) that is not explicitly disallowed; the prompt note tells the model it has no shell.
✅ UI loads and spec chat renders, incl. phone width Dashboard 'Project Setup Required' and the spec chat render at 1280x720 and 390x844. The phone-width header overflow is pre-existing and not touched by this PR.
⏭️ UI Playwright e2e PR has no UI changes.
✅ Process cleanup After stopping uvicorn, no leftover processes on 5247 or the temporary project; git worktree clean.
🧪 Commands run
Command Result
python -m pytest test_chat_sessions_no_shell.py -q 4 passed
python -m pytest -q (PR vs origin/master worktree) 148 passed, 21 failed on PR; identical 21 failures on master (test_devserver_security)
ruff check . All checks passed
mypy . (PR vs master) 24 errors in both, all pre-existing
python test_security.py ALL TESTS PASSED
claude -p hi --output-format stream-json --verbose [--disallowedTools Bash] (fake key, empty HOME) Bash in init tools without flag; absent with flag
npm --prefix ui run build success
npm --prefix ui run lint 0 errors, 6 warnings
uvicorn server.main:app --port 5247 (isolated HOME, no API key) + POST /api/projects + spec WS start 200; WS returned 'Not logged in' text + response_done; CLAUDE.md and settings verified

No browser screenshots were taken in this round.

Posted by cubefarm · merges automatically once GitHub's checks pass

cabana8471-arch pushed a commit to cabana8471-arch/autoforge that referenced this pull request Nov 7, 2026
…l the model why

Adaptations: with AutoForgeAI#316 in place the spec and expand prompts reach the CLI through the session's ephemeral --system-prompt-file, so the 'no Bash/shell tool' note stays scoped to that conversation and never lands in a CLAUDE.md the coding agents read. The PR's tests now read the note from the system prompt file and assert that no CLAUDE.md is created (they asserted claude_md.endswith(NO_SHELL_PROMPT_NOTE)).

Integrates AutoForgeAI#298

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AeH26pcsqoxPb3CDs8g1VN
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The agent needs approval to launch shell commands

1 participant