Skip to content

Weeding: Architect runs that follow each other as soon as the last plan is merged #311

Closed
5 of 5 issues completed

Description

@JacobStephens2

Problem Statement

I run thirdshift architect from cron on my server with merge.always, so each Architect run reviews the codebase, builds its Architect plan and merges it. Then nothing happens until the next night. I want Weeding: as soon as one Architect run's plan is merged into the Base branch, the next Architect run starts, so thirdshift keeps clearing the detritus that features leave behind as they land.

A crontab line that fires every few minutes nearly does this today. An Architect run is already skipped while another one, or a Pickup run, is running on the repository, and while an Architect plan is still open. That covers a failed run, whose plan stays open. Three things stop it working in practice:

  • Every skipped pass sends a Run notification when notifications are on, so a */5 line sends about twelve "skipped" emails an hour.
  • When the Architecture review finds no Strong candidate, nothing blocks the next pass. Every few minutes another full review runs and files the next-weaker idea as needs-triage, flooding the Day shift's triage queue and burning tokens. When the top idea is already covered by an open issue someone triaged long ago, every pass runs a full review only to land on that same issue again.
  • Weeding competes with work a human shaped. With both an Architect line and a Pickup line on the repository, whichever fires first after a run ends takes the repository, so self-improvement can push a Ready issue aside.

Solution

Weeding is a practice, not a new command. thirdshift still never schedules itself (ADR 0009), and Weeding is one crontab line running thirdshift architect base <branch> every few minutes with merge.always. The changes make that line safe and quiet:

  • A skipped Architect run sends no Run notification, the same as a skipped Pickup run. It still exits 0, and the log keeps its line.
  • An Architect idea pauses Weeding. When the review ends on an issue rather than a plan, either the idea it filed or the open issue that already covers its top recommendation, thirdshift labels that issue architect-idea and needs-triage. needs-triage goes back on a covering issue that was already triaged, because the factory again takes it for the best next move. While any Architect idea is open and still labelled needs-triage, every Architect run on the repository is skipped. Any triage decision releases it: needs-triage taken off, whatever replaces it, or the issue closed.
  • A Ready issue goes first. While the repository has a Ready issue, by the Pickup run's own definition, an Architect run is skipped.
  • A skip says what blocks it. stdout prints one URL per blocking issue, and stderr prints one line naming the reason.

Order of checks, where the first one that applies is reported: the lock (another run still running), then open Architect plans, then Architect ideas waiting for triage, then a Ready issue.

User Stories

  1. As an operator, I want to run thirdshift architect from a crontab line every few minutes, so that a new Architect run starts soon after the last one's plan is merged.
  2. As an operator, I want Weeding to need no new command, daemon or flag, so that it is the same command I would type and each pass keeps its own exit code and log line.
  3. As an operator, I want a skipped Architect run to send no Run notification, so that a frequent crontab line doesn't flood my inbox.
  4. As an operator, I want a skipped Architect run to still exit 0, so that cron sees no failure.
  5. As an operator, I want a skipped Architect run's reason in the command log, so that I can see why nothing happened without an email.
  6. As an operator, I want a broken notification setup to still fail the pass before any skip check, so that a misconfiguration shows up on the first pass and not only when a run does work.
  7. As an operator, I want an Architect run that did work to still send exactly one Run notification when asked, so that each plan, idea or failure reaches me.
  8. As a Day shift member, I want the issue a no-Strong review files labelled architect-idea, so that I can tell the factory's ideas from other needs-triage issues.
  9. As a Day shift member, I want Weeding to pause while an Architect idea waits for my triage, so that the factory doesn't keep filing weaker and weaker ideas once it has run out of Strong ones.
  10. As a Day shift member, I want any triage decision on an Architect idea (ready-for-agent, ready-for-human, wontfix, needs-info, or closing it) to release the pause, so that I don't have to learn a separate ritual.
  11. As a Day shift member, I want an idea I triaged as ready-for-human and left for a month not to stop Weeding for that month, so that triage, not closing the issue, is what releases the pause.
  12. As a Day shift member, I want the covering issue of an "idea already filed" review labelled architect-idea and put back in needs-triage, so that I learn the factory still thinks this is the best next move, and Weeding pauses instead of reviewing the same thing every few minutes.
  13. As a Day shift member, I want the covering issue's other labels kept when architect-idea and needs-triage are added, so that my earlier triage stays visible.
  14. As an operator, I want an Architect run that can't label its Architect idea to fail, sending its Run notification with the cause, so that a broken pause never lets the loop spin unseen.
  15. As an operator, I want an Architect idea triaged to ready-for-agent to be built by a Pickup run before Weeding resumes, since it is then a Ready issue, so that the idea I approved goes first.
  16. As an operator running both an Architect line and a Pickup line, I want an Architect run skipped while the repository has a Ready issue, so that work a human shaped goes ahead of self-improvement.
  17. As an operator, I want an issue labelled ready-for-agent that is not a Ready issue (claimed, blocked, unsettled, a Ticket inside a Spec, a Base fix's issue) not to skip an Architect run, so that Weeding pauses only for work a Pickup run could actually take now.
  18. As an operator, I want the Architect run to use the very same definition of a Ready issue as the Pickup run, so that the two can never disagree about whether work is waiting.
  19. As an operator, I want an Architect run skipped for a Ready issue to print that issue's URL on stdout and name it on stderr, so that I know what Weeding is waiting for.
  20. As an operator, I want an Architect run skipped for an Architect idea to print the idea's URL on stdout and name it on stderr, so that I know what to triage to resume Weeding.
  21. As an operator, I want a skip for open Architect plans to keep printing each plan's URL, so that existing scripts keep working.
  22. As an operator, I want the lock checked before any GitHub search, so that a pass fired while a run is going does as little as possible.
  23. As an operator, I want a failed Run or Spec run on an Architect plan to keep pausing Weeding until the plan is closed or loses architect-plan, so that the factory never builds on top of its own failure.
  24. As an operator, I want no cap on Architect runs per day, so that Weeding goes on for as long as it keeps finding Strong ideas and merging them.
  25. As a nightly Architect user, I want the README to say a skipped night now sends no email, so that I don't take a missing email for a broken setup.
  26. As an operator with ready-for-agent issues I plan to run by hand and no Pickup line, I want the README to warn that those issues keep Architect runs from starting, so that I am not surprised when my nightly run never happens.
  27. As a new user, I want a README "Weeding" section with a sample */5 crontab line using merge.always, so that I can set it up by copying.
  28. As a user who also runs a Pickup line, I want the README to mention the variant of thirdshift architect --plan-only plus a Pickup run that builds the plan as a Ready issue, so that I can route Weeding's work through the Claim limit.
  29. As a reader of ADR 0009, I want its Consequences no longer to say a skipped Architect run sends a notification, so that the ADR matches the behaviour.
  30. As a Pickup run user, I want Pickup runs unchanged, so that adding Weeding doesn't alter which issue a Pickup run takes.

Implementation Decisions

  • No new command or mode. ADR 0009 stands. Weeding is the existing architect command started often. Only the skip rules, labels, notifications and docs change.
  • New label architect-idea, defined in the labels module next to architect-plan. thirdshift adds it, never the agent, and creates it in the repository if missing, as it does for architect-plan.
  • Concluding the review. For the "idea filed" and "idea already filed" outcomes, thirdshift now puts architect-idea and needs-triage on the issue the review ended on, in one request through the labels module's swap, keeping the issue's other labels. Today it changes no label. A failure to label makes the Architect run a failed review, and its Run notification carries the cause. The plan path is unchanged.
  • New skip reasons on the Architect run's skipped outcome, next to "already running" and "open plans": Architect ideas waiting for triage (open, labelled architect-idea and needs-triage, compared ignoring case as every label rule now is), and a Ready issue. Each carries the blocking issues, so that stdout can list their URLs and stderr can name the reason.
  • Order of checks: lock, open Architect plans, waiting Architect ideas, Ready issue. All of them come after the notification checks and the preflight checks, as today.
  • Reuse the Pickup run's Ready issue search. The search that decides whether an open ready-for-agent issue is a Ready issue (Unready labels, Claim, Ticket-in-Spec, Base fix issue, blockers, started, Spec with all Tickets closed, settle time) is currently private to the Pickup run. It is exposed as one function both runs call: "the lowest-numbered Ready issue in this repository, if any". The Architect run calls it with no Sweep and no Claim limit, because those belong to the Pickup run.
  • Run notification: a skipped Architect run sends none, whatever the flag or email.always says. The notification checks still run first on every pass, so a broken setup fails before the skip.
  • No state between passes. Every pause is read from labels on GitHub, which keeps ADR 0009's rule that no process outlives a pass to remember anything.
  • Docs: a README "Weeding" section beside "On a schedule", covering the */5 line with merge.always, the variant of --plan-only plus a Pickup line, and what pauses Weeding. Update the one-at-a-time, one-plan-at-a-time and notification sections. Edit ADR 0009's Consequences line about skipped Architect runs. The sample crontab picks up a Weeding line. The glossary is already updated in Weeding and the Architect idea in the glossary #320.

Testing Decisions

  • One seam: the end-to-end tests that run the real thirdshift architect binary against the fake gh and claude and the Resend stand-in. Every rule here shows up as a label read or written, a line on stdout or stderr, an exit code, or an email sent or not. Test only that external behaviour.
  • New tests at that seam:
    • An open Architect idea labelled needs-triage skips the run before any review, prints its URL, and names it on stderr.
    • Once triaged (needs-triage off, in any case, whatever replaces it) or closed, it no longer skips.
    • "Idea filed" leaves the idea labelled architect-idea and needs-triage.
    • "Idea already filed" puts both on the covering issue, puts needs-triage back on one that was triaged, and keeps its other labels.
    • A failure to label fails the run and sends one notification with the cause.
    • A Ready issue skips the run and prints its URL. An issue labelled ready-for-agent that is not a Ready issue (claimed, unsettled, blocked) does not skip it.
    • The order of checks: with both an open plan and a waiting idea, the plan is reported.
    • A skipped run sends no email, even with email.always or email on the line. A broken notification setup still fails the pass before the skip.
  • Tests that flip: a_skipped_architect_run_sends_one_notification_with_skipped_in_its_subject_and_why_in_its_body, an_architect_run_skipped_for_open_plans_sends_one_notification_naming_each_and_its_command, and the idea half of an_open_idea_issue_or_a_plan_left_needing_triage_does_not_skip_the_architect_run. Rewrite them to assert the new behaviour.
  • Prior art: the existing open-plan skip tests and the already-running skip tests in the Architect run's suite, the Pickup run's suite for Ready issue cases, and the notification tests for "exactly one, or none".
  • The Ready issue rules themselves stay covered by the Pickup run's existing suite. The search is shared, not duplicated.

Out of Scope

  • A watch, --every, --continuous or daemon mode, which ADR 0009 rejects.
  • A cap on Architect runs per day or per hour.
  • Remembering past passes on the machine, e.g. a cooldown, or skipping only when the Base branch has moved.
  • Telling whether a Pickup line exists for the repository. The Ready issue skip applies either way, and the README warns about it.
  • Changes to the Pickup run, the Claim limit or the Sweep.
  • Labelling Architect ideas filed before this change. Only ideas from Architect runs after it get architect-idea.
  • Changes to the Architecture review prompt. The review still files ideas as needs-triage, and thirdshift adds architect-idea.

Further Notes

  • The framing is Pocock's "Upkeep: keep the codebase and issue list healthy; generates work for the flow". The glossary term is Weeding, with "upkeep" listed under Avoid, because Weeding is the narrower practice.
  • The glossary entries (Weeding, Architect idea, the Architect run's skips, the Run notification) landed in Weeding and the Architect idea in the glossary #320.
  • Machine capacity: a Weeding line and a Pickup line on one repository never build at once, because of the shared lock. Lines on different repositories should still be staggered.

Activity

  1. changed the title [-]identify way to keep architect runs continually going - to have one start as soon as the prior one finishes[/-] [+]Weeding: Architect runs that follow each other as soon as the last plan is merged[/+] on Oct 3, 2026
  2. added
    ready-for-agentFully specified, ready for an AFK agent
    in-progressA Claim: thirdshift has taken this issue
    and removed
    needs-triageMaintainer needs to evaluate this issue
    ready-for-agentFully specified, ready for an AFK agent
    on Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions