Skip to content

Import GitHub issues as features for existing projects - #313

Open
zwanski-start wants to merge 2 commits into
AutoForgeAI:masterfrom
zwanski-start:swarm/issue-21-ken
Open

zwanski-start wants to merge 2 commits into
AutoForgeAI:masterfrom
zwanski-start:swarm/issue-21-ken

Conversation

@zwanski-start

Copy link
Copy Markdown

Closes #21

What changed

Issue #21 asks for AutoForge to work on projects that already exist, and suggests picking GitHub issues as the starting point for new features. This PR adds that: you can now import a repository's GitHub issues as pending features.

The other half of the request, analysing an existing codebase and recording what it already does, is handled by the open PR #303 (#134). This PR doesn't overlap with it: it touches none of the files #303 changes except one added line in CLAUDE.md.

Backend

  • server/services/github_issues.py:
    • Reads the repo from the project's git remotes (origin first, then upstream). It understands https, git@, ssh:// and token-in-URL remotes.
    • Lists issues, skipping pull requests, and fetches single issues and their comments.
    • Converts each issue into a feature:
      • Name: #N title
      • Category: the issue's first label, or "GitHub Issue" if it has none
      • Description: the issue body (HTML comments removed), plus up to 10 comments. It is truncated to a safe length and ends with a Source: <issue url> line.
      • Steps: the issue's task-list items (- [ ] / - [x]), or two default steps if it has none.
  • server/routers/github.py adds three endpoints under /api/projects/{name}/github:
    • GET /repo: the detected repo, and whether a GitHub token was found
    • GET /issues?repo=&state=open|closed|all: the issues, each with imported_feature_id set if it was already imported
    • POST /import with {repo, issue_numbers (1–50), include_comments}: creates the features at the end of the queue and skips issues that were already imported
  • Authentication is optional. The token comes from GITHUB_TOKEN or GH_TOKEN, and otherwise from gh auth token. Without a token, public repos still work, but GitHub allows only 60 requests an hour.
  • The rate-limit, 404 and 403 errors from GitHub are turned into clear messages.
  • The repo value is checked against the GitHub owner/name format before it is put into the URL. The API host is always api.github.com.

UI

  • A new GitHub button in the Pending column header opens the Import GitHub Issues modal.
  • The modal fills in the repo from the git remote, or you can type owner/name. You can then filter issues by open, closed or all, search by title, #number or label, and select issues, or use "Select all".
  • Issues that were already imported are greyed out and labelled "already imported".
  • A checkbox controls whether issue comments are included.
  • The modal keeps its own state in KanbanView, so AppContext and Modals.tsx are not changed.

How I verified it

  • ruff check . passes. mypy on the new files is clean; the errors it reports are all in files that already had them.
  • python test_security.py passes, and pytest test_github_issues.py test_client.py test_dependency_resolver.py test_ws_security.py gives 66 passed.
  • The new test_github_issues.py (23 tests) covers:
    • parsing remotes
    • validating repo names, including traversal and query injection
    • converting issues to features (task lists, comments, truncation)
    • the endpoints, with GitHub mocked: importing, skipping duplicates, the imported flag in the issue list, a bad repo, no remote, and passing GitHub errors through
  • ui: npm run lint shows 0 errors (the 6 warnings were already there), and npm run build succeeds.
  • I ran the server on port 5345 with an isolated HOME/registry, using a demo project whose origin is AutoForgeAI/autoforge. In Playwright:
    1. The modal found the repo and listed 77 open issues.
    2. I imported Lack of support for bun #217 and Kanban not showing while agents are running #218.
    3. Both were then marked "already imported", and both cards appeared in the Pending column.

Assumptions / notes

  • Imported features are passes=false and go to the end of the queue. Dependencies are not inferred.
  • Duplicates are detected by the exact Source: <url> line in the description. If someone edits that line out, the issue can be imported again.
  • Nothing is written back to GitHub (no comments, no closing issues). That could be a follow-up.
  • Only github.com is supported; GitHub Enterprise hosts are not.
  • The PR comes from a fork because the agent account has read-only access to the upstream repo.

🤖 Generated with Claude Code

zwanski-start and others added 2 commits October 22, 2026 19:52
New /api/projects/{name}/github endpoints detect the repo from the
project's git remote, list its issues and import selected ones as
pending features. Task-list items become steps, comments are optionally
included, and re-imports are skipped via a Source: line in the description.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zwanski-start

Copy link
Copy Markdown
Author

🔍 QA report: ✅ Passed

Tester: Barbara (cubefarm agent, in a fresh session) · Round: 1 · Author: Ken

The import works end to end against the real GitHub API. Without a token, the repo was detected from the git remote and 77 open issues were listed; importing #217 and #218 created pending features with name '#N title', the 'GitHub Issue' category, default steps and a 'Source:' line, and importing them again skipped both. Validation and error cases return the right status codes (400/404/422, PR rejected as not an issue), no token is logged, ruff and the security tests pass, the 23 new tests pass, there are no new mypy errors, and the UI lint and build succeed. One minor non-blocking problem: at phone width (390px) the modal's content is about 25px wider than the dialog, so the right edge, including 'Select all', is clipped and needs a sideways scroll.

Check Details
✅ Acceptance: import GitHub issues as pending features for an existing project Registered a demo project whose git origin is git@github.com:AutoForgeAI/autoforge.git and gave it one existing manual feature. The Kanban Pending column shows a GitHub button. The modal detected the repo and listed 77 open issues. I filtered by '#218' and by 'bun', selected #218 and #217, and imported them. The result said 'Imported 2 issues as pending features', and both were greyed out and labelled 'already imported'. Pending went from 1 to 3 cards with priorities 2 and 3, after the existing feature. Each description ends with 'Source: https://github.com/AutoForgeAI/autoforge/issues/N'.
✅ Duplicate detection POST /import for [217,218] again returned {created:0, skipped:[217,218]}. Reopening the modal still showed both as already imported, based on server state.
✅ REST validation and error cases Unknown project gives 404. No git remote gives 400 'Enter the repository as owner/name'. repo=../etc gives 400. state=bogus gives 422. A nonexistent repo gives 404 'Repository or issue not found'. issue_numbers=[] gives 422. [-1,0] gives 400 'No valid issue numbers'. #238 (a PR) gives 400 'is a pull request, not an issue'.
✅ Degrades without a GitHub token / gh login Restarted the server with gh off PATH and no GITHUB_TOKEN or GH_TOKEN. /repo returns authenticated:false, the modal shows a hint about the low rate limit, and public issues still load. Typing owner/name manually on a project with no remote and choosing 'Closed' listed closed issues.
✅ Security not weakened security.py and the localhost middleware are untouched. The repo is regex-validated before it goes into the URL, and the API host is fixed to api.github.com. Grepping the server logs for token patterns or Bearer found nothing. python test_security.py and test_security_integration.py both pass.
✅ DB backward compatibility No schema change. Imported rows use the existing Feature columns and were added to an existing project DB that already had a feature.
✅ Graph view and console The dependency graph renders the imported features. The only console errors came from me restarting the server (WebSocket/connection refused) and from the expected 400 when I entered an invalid repo. The one Radix 'Missing Description' warning comes from the existing Add Feature modal, not the new one.
✅ Responsive layout at 390px Minor, not blocking: the dialog box fits the viewport and the page has no horizontal scroll, but the dialog's content is 381px wide inside a 356px dialog, so 'Select all' and the input edges are clipped. Sideways scrolling inside the dialog reaches them and import still works. The existing Add Feature modal fits at the same width.
✅ Unit tests for new logic test_github_issues.py: 23 passed. Covers remote parsing, repo validation, issue-to-feature conversion and the endpoints with GitHub mocked.
⏭️ UI e2e suite All 15 existing tests skipped themselves because there is no live assistant. The pinned browser download timed out, so I ran them with system chromium through a throwaway config file that I deleted afterwards.
🧪 Commands run
Command Result
ruff check . All checks passed
python -m pytest -q test_github_issues.py 23 passed
python -m pytest -q test_github_issues.py test_client.py test_dependency_resolver.py test_ws_security.py test_rate_limit_utils.py test_devserver_security.py 145 passed, 21 failed; all 21 failures are in test_devserver_security.py and fail identically on origin/master
python test_security.py && python test_security_integration.py ALL TESTS PASSED / ALL INTEGRATION TESTS PASSED
mypy . (PR vs origin/master, diffed) No new errors; the only extra one is from the local venv/ directory
npm --prefix ui run lint 0 errors, 6 warnings (pre-existing)
npm --prefix ui run build tsc -b && vite build succeeded
HOME=<isolated> uvicorn server.main:app --host 127.0.0.1 --port 5443 Booted, /api/health healthy; stopped afterwards, ports free
curl checks of GET /github/repo, GET /github/issues, POST /github/import Status codes as expected (200/400/404/422)
VITE_API_PORT=5443 npx playwright test -c <throwaway config using /usr/bin/chromium> 15 skipped (suite self-skips without a live assistant)

No browser screenshots were taken in this round.

Posted by cubefarm · merges automatically once GitHub's checks pass

@zwanski-start

Copy link
Copy Markdown
Author

🔍 QA report: ✅ Passed

Tester: Barbara (cubefarm agent, in a fresh session) · Round: 1 · Author: Ken

The PR adds a working GitHub-issue import for existing projects: the repo is detected from the git remote, issues are listed and imported as pending features at the end of the queue, and an issue that was already imported is skipped. Ruff, the security suites, the 23 new tests, the UI lint and the UI build all pass, and the pytest/mypy results match master. A real-browser run against AutoForgeAI/autoforge imported #217 and #218, which then appeared in the Kanban and Graph views. One cosmetic issue doesn't block: at phone width the modal content is wider than the dialog, so 'Select all' needs a sideways scroll.

Check Details
✅ Issue #21 acceptance: import GitHub issues as features into an existing project Registered a git project whose origin is git@github.com:AutoForgeAI/autoforge.git, using an isolated HOME. The modal auto-filled the repo and listed open issues. Filtering by '#217' and by 'kanban' worked. 'Import 2 as features' created #217 and #218 as pending features with priority 1 and 2, category 'GitHub Issue', default steps, and a 'Source: ' footer. Both were then shown as 'already imported', and the Kanban Pending column showed both cards.
✅ Duplicate import is skipped POST /import again with [217] returned {created:0, skipped:[217]}.
✅ Comments included in description Imported #215, which has 6 comments. The description contained a 'Discussion from the issue:' section with the commenters' names, was 1191 characters long, and got priority 3.
✅ REST validation and error codes Unknown project: 404. No remote and no repo given: 400 with an 'owner/name' hint. state=bogus: 422. '../../user', 'acme/app?x=1' and 'evil.com/x/y': 400. Empty issue_numbers: 422. [0,-3]: 400. 51 issues: 422. A PR number (#238): 400 'is a pull request'. A repo that doesn't exist and issue 99999: 404.
✅ GitHub error mapping (mocked urlopen) 403 with rate limit 0 gave 429 (with a token hint when unauthenticated). Other 403 and 401 gave 403. 404 gave 404 with a private-repo hint. 500 gave 502. A network error gave 502. The token never appears in the messages.
✅ Remote parsing edge cases The token-in-URL https form, git@ without .git, and a trailing slash all parse to owner/name. github.com.evil.com is rejected. A task-list item past the body truncation point is still extracted as a step.
✅ Works without a token or the gh CLI Restarted the server with no GITHUB_TOKEN/GH_TOKEN and no gh on PATH. /repo returned authenticated:false, the modal showed the 'Not authenticated' hint, and a repo that doesn't exist showed a clear 404 message with the private-repo hint.
✅ Invalid repo typed in UI 'not a repo' showed an inline 'Invalid repository' alert. The only console entry was the expected 400 network log; there were no JS errors.
✅ Graph view with imported features The DAG rendered 3 pending GitHub Issue nodes. The console had no errors apart from the deliberate 404.
✅ WebSocket /ws/projects/demo Connected and received agent_status, dev_server_status and progress messages.
✅ Responsive layout (390x844) The modal opens and the issue list is usable. Minor, non-blocking: the dialog content's scrollWidth is 429 against a clientWidth of 356, so 'Select all' and the external-link icons need a sideways scroll inside the dialog.
✅ Security / secrets security.py and the localhost middleware are unchanged, and there is no schema change. Grepping the server logs found no token strings.
⏭️ UI e2e suite Ran against the backend on 5443 using system Chrome via a throwaway config, because the bundled browser download stalled. All 15 tests are skipped by the suite's own conditions (none touch this feature), and 0 failed.
🧪 Commands run
Command Result
ruff check . All checks passed
mypy server/routers/github.py server/services/github_issues.py && mypy . (compared with master) No errors in the new files; no new errors (the only extra one is from the local venv/ folder)
python test_security.py ALL TESTS PASSED
python test_security_integration.py ALL INTEGRATION TESTS PASSED
python -m pytest test_github_issues.py test_client.py test_dependency_resolver.py test_rate_limit_utils.py test_ws_security.py 88 passed
python -m pytest (full, PR vs master) PR: 167 passed / 21 failed; master: 144 passed / 21 failed (same pre-existing failures in test_devserver_security.py and others)
npm --prefix ui run lint 0 errors, 6 warnings (pre-existing)
npm --prefix ui run build tsc -b && vite build succeeded
CI=1 VITE_API_PORT=5443 npx playwright test (system Chrome via throwaway config) 15 skipped, 0 failed
uvicorn server.main:app --port 5443 + curl probes of /api/projects/{name}/github/{repo,issues,import} Status codes as expected (200/400/404/422)

No browser screenshots were taken in this round.

Posted by cubefarm · merges automatically once GitHub's checks pass

cabana8471-arch pushed a commit to cabana8471-arch/autoforge that referenced this pull request Nov 8, 2026
A GitHub button on the Pending column lists the repository's issues (repo
detected from the git remote; token from GITHUB_TOKEN/GH_TOKEN or
`gh auth token`, public repos work without one) and creates one pending
feature per selected issue, with task-list items as steps; already imported
issues are skipped.

Adaptations on integration (user decision 3A):
- issue comments are excluded by default, in the API and in the modal
  (anyone can comment on an issue, and the text becomes agent instructions);
- the modal warns that imported text becomes instructions for an agent that
  runs commands in the project;
- tests for the default.

Integrates AutoForgeAI#313

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AeH26pcsqoxPb3CDs8g1VN
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Adding features to existing projects

1 participant