Skip to content

Generate a top-level prompts/ directory showing every Session prompt #176

Closed
2 of 2 issues completed
Closed

Description

@JacobStephens2

Problem Statement

An engineer who wants to understand or evaluate thirdshift will most want to look at what the agents receive. Half of that is easy to find: the Factory skills sit at the top of the repository in skills/, as plain Markdown. The other half, the Session prompts that start or resume every agent session, is hard to find. They are format! strings full of \n\ escapes inside a Rust source file, and nothing at the top level points to them. The Prompts and skills page on the site shows them, but only people browsing the site will see it, not people browsing the repository. So a reviewer reading a pull request that changes a Session prompt has to read Rust string literals to see what the agent will now be told.

Solution

Add a top-level prompts/ directory, next to skills/, that shows every Session prompt as readable Markdown: one file per prompt, plus a README that lists the prompts and the claude command line every session runs with. The directory is generated from the same code the binary uses, by the golden-file test that already keeps the Prompts and skills page in sync, so it cannot drift from what thirdshift actually sends. The prompts' source stays in Rust. Any change to a Session prompt then also shows up as a prose diff in prompts/, in the same pull request.

User Stories

  1. As an engineer evaluating thirdshift, I want a top-level prompts/ directory, so that I can find every input to the agents without knowing the Rust source layout.
  2. As an engineer evaluating thirdshift, I want prompts/ to sit next to skills/, so that the Session prompts and the Factory skills are found the same way.
  3. As an engineer evaluating thirdshift, I want each Session prompt in its own Markdown file, so that I can open and read one prompt at a time.
  4. As an engineer evaluating thirdshift, I want each file to say when a Run sends that prompt, so that I understand its role without reading the orchestration code.
  5. As an engineer evaluating thirdshift, I want the prompt text shown byte-for-byte as the agent receives it, apart from placeholders, so that what I read is what the agent reads.
  6. As an engineer evaluating thirdshift, I want the parts that change per Run shown as clear placeholders such as <Issue URL>, <base> and <branch>, so that I can tell fixed wording from Run facts.
  7. As an engineer evaluating thirdshift, I want the prompt text in a fenced code block, so that Markdown rendering doesn't change quotes, #, or line breaks.
  8. As an engineer evaluating thirdshift, I want a prompts/README.md, so that I have one entry point that lists every Session prompt.
  9. As an engineer evaluating thirdshift, I want the README to list each prompt with its title, when it is sent, and a link to its file, so that I can scan them all at a glance.
  10. As an engineer evaluating thirdshift, I want the README to show the claude command line a new session and a Resume run with, so that I can see the permission mode and plugin loading as well as the text.
  11. As an engineer evaluating thirdshift, I want the README to point to skills/ for the Factory skills, so that I know where the rest of the agents' inputs are.
  12. As an engineer evaluating thirdshift, I want the file names to match the prompts' names (fresh.md, continuation.md, continuation-pr.md, spec-review.md, conflict-repair.md, foreign-conflict-repair.md, review-repair.md, ci-fix-repair.md, resume.md), so that I can find a prompt by name.
  13. As a reviewer of a pull request that changes a Session prompt, I want the change to also show as a prose diff in prompts/, so that I can judge the new wording without reading Rust string literals.
  14. As a maintainer, I want the prompts' source to stay in src/prompt.rs, so that the prompts keep being built with ordinary Rust (conditionals, lists of failed checks) and need no templating engine.
  15. As a maintainer, I want cargo test to fail when prompts/ is out of date with the code, so that the directory can never mislead a reader.
  16. As a maintainer, I want the failure to show which file differs and a line diff, so that I can see what changed without guessing.
  17. As a maintainer, I want the failure to tell me the exact command to regenerate, so that fixing it takes one step.
  18. As a maintainer, I want one command, UPDATE_PROMPTS=1 cargo test prompts_page, to regenerate both the Prompts and skills page and prompts/, so that I never have to remember two.
  19. As a maintainer, I want cargo test to fail when prompts/ holds a file no Session prompt produces, so that a removed or renamed prompt doesn't leave a stale file behind.
  20. As a maintainer, I want regenerating to delete stale files in prompts/, so that renaming a prompt is one regenerate, not a regenerate plus a manual delete.
  21. As a maintainer, I want to add a new Session prompt in one place, the list the page already renders from, so that it reaches the page and prompts/ together.
  22. As a maintainer, I want every generated file to begin with a note that it is generated from src/prompt.rs and how to regenerate it, so that nobody edits it by hand.
  23. As an agent working in this repository, I want AGENTS.md to say that prompts/ is generated and that changes belong in src/prompt.rs, so that I don't edit a generated file and fail CI.
  24. As a reader of README.md, I want prompts/ mentioned next to skills/, so that I learn both exist.
  25. As a reader of README.md, I want the regeneration instructions to name UPDATE_PROMPTS and cover both outputs, so that the documented command matches the real one.
  26. As someone installing thirdshift from crates.io, I want the package to stay as it is, without prompts/, so that the published crate carries only what the binary needs.
  27. As a maintainer running the tests from the crates.io package, I want the test to skip when prompts/ is absent, as it already does for site/, so that the packaged tests still pass.
  28. As a reader of the domain glossary, I want Session prompt defined, so that "prompt" in the code, the docs and prompts/ means one thing.

Implementation Decisions

  • Session prompt is now a term in CONTEXT.md: the message thirdshift starts or resumes an agent session with. It carries the Run's facts and names the Factory skills to use. The Resume message counts as one.
  • Session prompts and Factory skills stay separate. A Session prompt is the orchestration contract (a Run's facts plus which skills to call), and a Factory skill is reusable know-how. The recipe in the Fresh prompt is not folded into a skill.
  • The prompts' source stays in the prompt module (src/prompt.rs). Nothing in the Run's code changes, and no templating engine or prompt files are read at runtime. prompts/ is a generated view only, and the binary never reads it.
  • The prompts page module (src/prompts_page.rs) is extended to render the prompts/ directory as well as the site fragment, from the same inputs: the existing list of prompt entries (id, title, when, text) and the same claude arguments function. The directory can't show anything the page doesn't, and the reverse holds too.
  • One file per prompt entry, nine in all, named after the entry's id with the prompt- prefix removed. Each file has:
    • a generated-file note naming src/prompt.rs and the regenerate command
    • the entry's title as a heading
    • its "when" sentence
    • the prompt text in a fenced code block, rendered with the same placeholders the page uses, including the issue-number sentinel shown as #<n>.
      The page's "sent by unit" links are omitted (or given as plain text), because they point to anchors on the site's home page.
  • prompts/README.md is generated too. It contains the generated-file note, a short intro, the claude command line for a new session and for a Resume (as on the page), a table of the nine prompts (title, when, link to file), and a pointer to skills/ for the Factory skills.
  • The fence used for prompt text must not appear inside any prompt. If a prompt ever contains triple backticks, the renderer uses a longer fence.
  • The regeneration environment variable is renamed from UPDATE_PROMPTS_PAGE to UPDATE_PROMPTS. It regenerates both the page fragment and prompts/. The old name is dropped, not kept as an alias.
  • Stale files: in check mode, any file in prompts/ that the renderer doesn't produce is a failure. In update mode, those files are deleted.
  • prompts/ is not added to the crates.io package's include list.
  • Documentation: README.md gets one line mentioning prompts/ next to where skills/ is mentioned, and its regeneration paragraph is updated to the new variable name and to cover both outputs. AGENTS.md gets one line saying prompts/ is generated, so edit src/prompt.rs and regenerate with UPDATE_PROMPTS=1 cargo test prompts_page.
  • No ADR: the directory is cheap to reverse, and the reasoning is recorded in the generated files' notes and in AGENTS.md.

Testing Decisions

  • A good test here checks what a reader sees, the generated files on disk, against what the binary would send. It does not check how the renderer builds the strings.
  • There is one seam: the existing golden-file test in the prompts page module. It is extended, not duplicated:
    • It compares every rendered prompts/ file with the checked-in file, the same way it compares the page fragment.
    • It fails when a file is missing, differs, or is stale. The message names the file, shows a line diff using the existing diff helper, and gives the UPDATE_PROMPTS=1 cargo test prompts_page command.
    • With UPDATE_PROMPTS set, it writes the page fragment and every prompts/ file and deletes stale ones.
    • It skips when prompts/ is absent, matching the existing skip for site/ in the crates.io package.
  • Small unit tests in the same module, alongside the existing ones for the diff and for placeholder highlighting, may cover pure helpers the renderer adds, such as choosing a fence longer than any run of backticks in the text.
  • Prior art: prompts_page_shows_what_the_binary_sends, diff_shows_only_the_changed_lines and placeholders_are_highlighted_and_fixed_text_is_escaped in the prompts page module.
  • No tests are added at the Run or session level. The prompts' text and how they are sent don't change.

Out of Scope

  • Moving the prompts' source out of Rust into template files, or adding a templating engine.
  • Turning any Session prompt, such as Fresh's implement → review → PR recipe, into a Factory skill.
  • Changing the wording of any Session prompt.
  • Changing the Prompts and skills page's content or design beyond the renamed regeneration variable.
  • Shipping prompts/ in the crates.io package or in the binary.
  • Generating a copy of the Factory skills into prompts/: they already live in skills/.

Further Notes

  • The motivating goal is discoverability for engineers evaluating the system: "what do the agents get?" should be answered by reading prompts/ and skills/.
  • CONTEXT.md already has the Session prompt entry from the design discussion. The implementing session only needs to use the term consistently, for example in the README intro and the generated-file notes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-agentFully specified, ready for an AFK agent

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions