Repository navigation
feat(server): index worktrees for active agents, outside the repo - #67
Merged
Merged
Conversation
A checkout only indexed when something asked it a code question, the file cap could leave a repo's own source out of its index, and the index folders sat inside the user's repo. T3 now indexes a project when it is added and a worktree when an agent starts in it. An idle worktree is not rebuilt, and its index goes when its thread settles, by the rules that let a worktree be deleted. Settings > Labs has one switch for all of this; while it is off, only a code question builds an index. Indexes now live in T3's data directory. The SDK only takes a data directory inside its project root, so each checkout gets a private root that links back to its top-level entries. Folders an older version left in a checkout are removed the next time it indexes, and a sweep drops indexes for checkouts that are gone or no longer known. Listings over the cap now rank first-party source first, builds run at low priority with at most two at once, and the badge reports first builds and failures. The Windows link and copy paths are untested. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Preserve build lock inodes during cleanup and reject linked legacy lock paths. Rebuild when checkout indexing configuration changes, and follow settled ancestors when releasing subagent worktree indexes. Add regression coverage for cleanup locking, config changes, and parent-settlement sweeps; drain background refreshes in generation assertions and document retained legacy lock files.
Maintenance kept every worktree whose agent had switched branches, because nothing wrote HEAD back to the thread. A new WorktreeBranches service reads HEAD and rebinds the threads on that worktree. Cleanup keeps for one pass after a rebind, then applies the existing rules. A detached HEAD stays kept unless a ref outside refs/t3 holds its commit. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A thread kept its old branch name after its agent switched branches, so the pull request refresh skipped it and cleanup had to repair it later. Run finalization now syncs the worktree branch before that refresh. A sync failure only logs, so it cannot block the next turn. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Launch carried its own copy of the loop that writes a branch to every thread on a worktree. It now calls WorktreeBranches.sync after the rename, so one path writes thread.branch for a worktree. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Pins the resume half of the cleanup change. Once a thread is rebound and its worktree is removed, the next turn recreates the checkout on the branch the agent switched to, at that commit, and not on the stale name. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A deleted owner's record cannot be rewritten, so only its surviving subagents are rebound. This pins that the group comparison still lets removal proceed on the next pass, and that the branch ref survives. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
Devin Review found 4 potential issues.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
Avoid defensive SQLite lock journals, share checkout identity across path casing, and reject UNC checkout junctions instead of publishing partial indexes. Model: Devin built by Cognition AI`nHarness: Devin Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…data Cleanup kept `.codegraph-t3/` in users' repos, could add new untracked lock files there, deleted any file a user had put in that folder, and left a lock-only folder per worktree in T3's data directory that every hourly sweep locked again. Old folders in a checkout that no build reached stayed forever. Removal now deletes the lock files and their folders too. So a builder cannot trust a lock on a file that was deleted before its lock landed, it compares the file's identity across taking the lock and backs off if it changed. Lock files go while held where the OS allows it, and on release where an open file cannot be deleted. Old folders lose only the files T3 and the SDK wrote, and a generation holding anything else is left alone. Startup and hourly sweeps clear them from every project and worktree T3 knows, even with automatic indexing off. Also from review: a pinned or explicitly active thread keeps a shared worktree's index, source in a language outside the SDK's catalog makes coverage partial once it is over 5% of the code, and an unreadable codegraph.json or .gitignore no longer fails builds and reads. Model: Opus 5.5 Harness: Claude Code in T3 Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two code index tests sent SIGABRT, which Windows rejects with ENOSYS, so they failed before reaching the behaviour under test. They now use SIGKILL on Windows and keep SIGABRT elsewhere; production treats any unexpected exit the same. Model: Opus 5.5 Harness: Claude Code in T3 Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lone An independent check of the cleanup change found three ways to lose or strand work. A different repository at the worktree path had its branch written to the thread, so the thread forgot its own branch. A branch another checkout holds was recorded even though the thread could not recreate its worktree there. And the detached HEAD check ran before later steps that gave a holding ref time to go. Sync now compares the worktree's repository with the project's before it writes, and treats a branch another checkout holds like the default branch. Cleanup checks the holding ref last, immediately before removal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cleanup counted any remote-tracking ref as holding a detached commit. A routine `git fetch --prune` removes a remote branch that was deleted upstream, so the commit could become unreachable after its worktree was gone. Only local branches, tags, and the remote's default branch count now. A detached worktree held by any other remote branch stays kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y thread Adversarial review reproduced more ways for this change to lose work or never finish. Removing a detached or default-branch worktree could lose a commit only its reflog held, or strand a thread whose recorded branch another checkout held. A remote-tracking ref counted as protecting a commit even though a routine prune or a stale origin/HEAD could remove that protection. A run end skipped a sibling whose write had been deferred to it. Threads with a trailing slash in their path or no recorded branch never rebound, and mismatched records on a detached or default-branch worktree were kept forever. A held index lock blocked rebinding, and main counted as the default branch when the remote's default was trunk. Cleanup now checks HEAD and the reflog against local branches and tags only, and keeps a worktree while another checkout holds a recorded branch. Sync compares HEAD with every thread on the worktree instead of the caller's record, reads HEAD without depending on the index lock, finds threads by the path each one stored, and asks the remote when no local HEAD ref names its default. Model: Opus 5.5 Harness: Claude Code in T3 Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # apps/server/src/orchestration-v2/RunFinalizationService.test.ts
ScriptedAlchemy
marked this pull request as ready for review
November 5, 2026 21:34
…vant Code indexes outlived their use. When Maintenance removed a worktree its index stayed until the next hourly sweep, "Delete now" never touched indexes, and an unsettled worktree that Maintenance kept held its index forever. Maintenance now removes a worktree's index just before the folder, while the path still resolves and a running build can stop first. The index sweep follows every Maintenance run instead of keeping its own timer, so both share one schedule. A worktree's index also goes after 7 days with no activity on its threads, unless one is pinned or marked active. The next agent run rebuilds it. Model: Opus 5.5 Harness: Claude Code in T3 Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ScriptedAlchemy
changed the base branch from
main
to
fix/worktree-cleanup-branch-sync
November 5, 2026 21:47
An adversarial review of this branch reproduced commit loss that main prevents. A rebound worktree counted as bound on its next pass, which skipped the reflog check, so cleanup removed it and gc then pruned commits only its reflog held. A deleted owner with no surviving threads skipped the same check. Another repository's linked worktree at the path skipped the repository check and lost its untracked files to `git clean`. Two detached worktrees recording one branch were both removed, and with no local origin/HEAD a default other than main or master could become a thread's recorded branch. The reflog check now runs on every worktree. Rewritten commits land in the reflog too, so an amended or rebased worktree waits until gc expires those entries. Cleanup compares the worktree's repository with the project's before it reads the checkout, by file identity so a differently cased path still matches. It keeps a detached or default-branch worktree while a thread elsewhere records the same branch. The service asks the remote for its default whatever the name, once per checkout. Model: Opus 5.5 Harness: Claude Code in T3 Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # apps/server/src/storageCleanup.ts
Cleanup keeps a detached or default-branch worktree while a thread in another worktree records the same branch, because only one checkout can hold it. The rule compared branch names across all projects, so a same-named branch in an unrelated repository kept the worktree with an untrue reason. It now compares threads in the same project. Two projects sharing one repository are not compared, and resume there still reports Git's error. Model: Opus 5.5 Harness: Claude Code in T3 Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # apps/server/src/orchestration-v2/RunFinalizationService.test.ts # apps/server/src/server.ts # apps/server/src/storageCleanup.test.ts # apps/server/src/storageCleanup.ts
ScriptedAlchemy
changed the base branch from
fix/worktree-cleanup-branch-sync
to
main
November 6, 2026 07:11
ScriptedAlchemy
enabled auto-merge (squash)
November 6, 2026 07:12
Wait for index shutdown before the final activity, branch, local-change, submodule, and recovery-reference checks. This preserves late tracked edits when submodule removal requires force and retains detached history if its last holding ref disappears during shutdown. Keep worktrees when reflog or worktree-list inspection is truncated. Use Effect Path for native cleanup fixture paths and preserve the Android test-platform environment variable.
Assert report accounting against filesystem-provided allocated bytes instead of assuming tiny files allocate nonzero space. Move and restore the Git-created hidden file for the invalid-pointer case.
The test decided its expected result by asking whether the upper-cased path exists, to detect the Windows runner. That is also true on a Mac, where the linux row still compares case-sensitively, so it failed on every Mac. It now checks the real host. Model: Opus 5.5 Harness: Claude Code in T3 Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A checkout indexed only when something asked it a code question, so a new project or worktree had no code graph when an agent needed one. The 10,000 file cap could also fill up on hidden folders and leave out all of a repo's own source. The index wrote
.codegraph-t3*folders into the user's repo as well, and those folders showed up on disk and in the Files panel.Change
When it indexes. T3 indexes a project when it is added and a worktree when an agent starts in it. T3 does not rebuild an idle worktree until an agent is active in it again. Builds run at low priority, with at most two at once. Listings over the cap rank first-party source first.
When it cleans up. T3 deletes a worktree's index when its thread settles, by the rules that let a worktree be deleted. A pinned or explicitly active thread that shares the worktree keeps the index. A worktree's index also goes after 7 days with no activity on its threads, unless one is pinned or marked active, and the next agent run rebuilds it. A project's own checkout keeps its index.
Maintenance. Maintenance removes a worktree's index just before it removes the folder. The index sweep follows every Maintenance run, including "Delete now" and runs with no worktree rules on, instead of keeping its own timer. Each sweep removes a settled or long-unused worktree's index, including a subagent's whose settled parent has no worktree or uses another one, and an index for a checkout that is gone or that T3 no longer knows about. The Last run report does not list index removals, because a new report type would break older clients.
Worktree cleanup. Cleanup's rule for a branch recorded by a thread in another worktree now compares threads in the same project, so a same-named branch in an unrelated repository no longer keeps a worktree.
The switch. It is at Settings > Labs > Automatic code indexing, on web and mobile. Turning it off stops running builds, and after that only a code question builds an index. T3 still removes leftovers while the switch is off.
Where it lives. Indexes are in
<state>/code-index/<hash of checkout path>/, never in the repo. The SDK accepts a data directory only inside its project root, so each checkout gets a private root that holds the index and links back to the checkout's top-level entries. POSIX uses symlinks. Windows uses junctions for folders, with a copy per build when file links need a privilege. Removal deletes the whole root. It unlinks those entries and never removes recursively from the root.Old folders in repos. A checkout's next build and every sweep remove
.codegraph-t3*folders that an older version left there. This applies to all projects and worktrees T3 knows, even with the switch off. T3 deletes only files that T3 and the SDK wrote. Anything else in the control folder stays. T3 leaves alone a generation folder that holds anything else, and any folder with symlinked or multiply linked entries.Lock safety. Removing lock files could let two servers each hold a lock on a different file. A builder now compares the file's identity before and after it takes the lock, and backs off if the identity changed. T3 deletes lock files while it holds them where the OS allows it, and on release where the OS cannot delete an open file. T3 therefore never deletes a lock that is still in use.
Configuration changes. Index metadata records a fingerprint of the contents of the checkout's
codegraph.jsonand root.gitignore. Freshness checks detect a change or a deletion. The next refresh then rebuilds the graph, reconsiders excluded files, and removes newly excluded entries. A config path that T3 cannot read, such as a folder with that name, no longer fails builds or reads.Badge. The Not indexed, Indexing, and Index failed states now report first builds. The badge reads the state again when an index is removed or a build stops. Code queries ignore that signal, so the signal cannot restart a build T3 just stopped. The badge shows Partial once source in a language outside the SDK's catalog is over 5% of the code. Docs, assets, and an occasional script stay Ready, and coverage still lists the unsupported count.
Original implementation verification (before this audit)
Pre-audit head on macOS
vp run check:changedpasses against the latestmain: format, lint, knip, generated GraphQL, native addon, typecheck of 36 packages, 14 test files (412 passed, 5 Windows-only skipped), and the React compiler checks..codegraph-t3-g7/notes.txt. Repos stayed clean undergit status --ignored.root.ts.Windows, before the Maintenance change
codeIndexLocks,CodeIntelligence, andCodeIndexReactortest files pass on awindows-2025runner, 93 of 93. They cover lock removal, deleting index roots with their junctions, and old-folder cleanup: run.Earlier checks, not repeated on the final head
2bb2e5bfixed path-case identity, UNC rejection, and the lock journal.Earlier verification limits (before this audit)
d75ea81.Devin wrote the Windows fixes. GPT-6 Astra Pro in ChatGPT did the follow-up review and fixes.
DeepSeek-V4.1-Flash via OpenCode did the implementation. Opus 5.5 via Claude Code in T3 Code did the design, review, and fixes.
🤖 Generated with Claude Code
Audit follow-up — 2026-11-06
Index shutdown now happens before the complete final cleanup safety sequence. Regression tests cover a recovery ref disappearing during shutdown and a tracked edit arriving in an initialized submodule worktree that requires forced removal. Both cases retain the worktree.
Cleanup also retains a worktree when Git truncates reflog or worktree-list inspection output. Windows fixtures use native paths, filesystem casing, exact allocated-byte accounting, and safe preservation of Git's hidden worktree metadata file. The shared test environment retains the Android platform flag.
Local validation during this audit: 399 selected tests passed with 6 expected platform/permissions skips; 13 selected React compiler checks passed; all 41 affected typecheck tasks passed through the supported uncached task runner. The final cleanup safety ordering passed its 32-case focused run. The three final Windows fixture corrections passed their focused tests, file format/lint, and a server typecheck with explicit exit 0.
The full Windows cleanup suite passed on
6fe77ad584d27bc3d2cc9b4084cfcf893c09919b: 209 passed, 1 existing permissions case skipped, 1 file passed. The job completed successfully in 5m 14s. This covers the two cleanup safety regressions and the three corrected Windows fixture cases.Local browser tests remained unavailable because the Chromium installer returned a truncated archive. Live UI verification could not run because the repository's
test-t3-appworkflow requires the T3 Browser panel, which is unavailable in this harness.Model: GPT-6 Astra Pro. Harness: ChatGPT.