Skip to content

feat: add cost policy support and refresh model catalog for GitHub AI Credits billing #1080

New issue
New issue
Closed
Closed
feat: add cost policy support and refresh model catalog for GitHub AI Credits billing#1080
Labels
triagedIssue has been triagedwave:2-soonWave 2 — next 2-4 weeks

Description

@melodiouscode
melodiouscode
opened on May 4, 2026

feat: add cost policy support and refresh model catalog for GitHub AI Credits billing

Summary

This PR proposes adding cost policy support and a refreshed model catalog to align with GitHub Copilot's AI Credits billing transition.

What changes

  • New model catalog (packages/squad-sdk/src/config/models.ts): Updated to current active GitHub models across three categories — Lightweight, Versatile, and Powerful — with accurate pricing and included: true flags for zero-credit models (gpt-4.1, gpt-5-mini).

  • Cost policy configuration (packages/squad-sdk/src/config/schema.ts): New costPolicy field on ModelSelectionConfig with maxCategory (hard ceiling) and preferIncluded (zero-credit preference) options.

  • Two-phase model selection (packages/squad-sdk/src/agents/model-selector.ts): Post-resolution policy veto step that:

    • Warn-and-allows explicit user overrides above the ceiling (Layer 0/1)
    • Automatically downgrades implicit selections (Layer 2/3/4) to the best in-ceiling model
    • Prunes fallback chains to respect the active ceiling

  • Updated coordinator governance (all squad.agent.md templates): New model catalog, decision tree documentation, and cost policy conversation phrases.

  • New test coverage: est/cost-policy.test.ts removed (broken stub), est/model-selector-policy.test.ts covers the two-phase resolution logic.

  • Changeset: @bradygaster/squad-sdk minor bump, @bradygaster/squad-cli patch bump.

Motivation

GitHub Copilot is transitioning to AI Credits billing. Teams need a way to:

  • Set a cost ceiling to prevent runaway spend on Powerful models
  • Prefer zero-credit included models when acceptable for the task
  • Get transparent warnings when explicit overrides exceed the ceiling

Testing

215 test files / 6099 tests passing. Build and lint clean.

Activity

  1. melodiouscode
    added a commit that references this issue on May 5, 2026

    feat: add cost policy + refresh model catalog for GitHub AI Credits b…

    67b37a4
  2. bradygaster
    added
    wave:2-soonWave 2 — next 2-4 weeks
    triagedIssue has been triaged
    on Jun 24, 2026

  3. primetimetank21 commented on Jul 4, 2026

    @primetimetank21
    primetimetank21
    on Jul 4, 2026
    Contributor

    From Earl's Squad:

    Picking this up since #1080 is still open. Worth noting for context: the earlier implementation in #1089 was self-closed by its author (@melodiouscode) on 2026-06-02 — it wasn't maintainer-rejected. It was a strong first pass and a lot of the shape here is reusable; we'd be building on that groundwork rather than starting cold.

    There's also meaningful overlap with #1183 (catalog drift + dynamic discovery). Rather than solve them twice, we'd like to treat #1080 and #1183 together — a single catalog reconciliation that carries the cost-policy work.

    Proposed approach — small, wired, test-first, in ~3 focused PRs:

    • Catalog refresh. Update model IDs/tiers and, importantly, remove currently-dead IDs from the fallback chains (e.g. gpt-4.1, gemini-3-pro-preview, claude-sonnet-4, gpt-5.1*/gpt-5.2*) that no longer resolve and just burn retry slots. Independently valuable; directly addresses Refresh squad.agent.md model catalog + dynamic catalog discovery #1183's drift.
    • Cost policy. A maxCategory ceiling that's actually wired through the spawn/lifecycle path (with the warning surfaced, and a fail-closed path when nothing fits), plus a squad models refresh diagnostic command to show what resolves right now (aligned with @diberry's framing, and @tamirdresher's ask to surface pruned-vs-added).
    • Docs/template/skill sync, kept isolated from the code changes.

    A few design choices we'd propose:

    • No hardcoded per-token pricing — the current values have already drifted, and the surface (cache-write, long-context, promo tiers) isn't representable as flat constants. Source pricing best-effort from the public github/docs models-and-pricing.yml (fetchable auth-free), not compiled into the catalog.
    • Keep two separate axes: Squad's quality tier (premium/standard/fast) and GitHub's billing category (lightweight/versatile/powerful) — don't conflate them.
    • Drop the included/zero-credit concept — there's no machine-readable source for it and no zero-credit models under current AI-Credits billing.
    • Category source — we'd take the authenticated Copilot models API (model_picker_category) as the source of truth for a model's billing category, falling back to the docs YAML (auth-free) when a token isn't available. That resolves the couple of spots where the two disagree (e.g. claude-haiku-4.5, gpt-5.4) in favor of the picker's live value.

    We're considering opening a PR to refresh the catalog first — updating the model IDs/tiers and pruning the dead fallback entries. It's low-risk, fixes a live routing issue, and is independent of the policy work, so we'd land it on its own and follow up with the cost policy. Happy to hold for a combined review instead if you'd prefer — just let us know.

    Thoughts welcome.

  4. melodiouscode commented on Jul 4, 2026

    @melodiouscode
    melodiouscode
    on Jul 4, 2026
    Author

    Happy to re-open or contribute anything I can @primetimetank21. I didn't have the time to deal with the PR hygine at the time sadly, and didn't want to leave the PR hanging.

  5. primetimetank21
    added a commit that references this issue on Jul 4, 2026

    test(models): strengthen catalog invariants (dead gpt-5 + exact-set g…

    69364df

  6. tamirdresher commented on Jul 4, 2026

    @tamirdresher
    tamirdresher
    on Jul 4, 2026
    Collaborator

    Thanks @primetimetank21 I like the direction. Will spend some time reviewing this tomorrow

  7. primetimetank21
    added 2 commits that reference this issue on Jul 4, 2026

    fix(models): refresh catalog to CLI-reachable IDs and prune dead fall…

    f5479b0

    feat(models): add category-based cost policy with docs-YAML pricing e…

    66f1c68

  8. tamirdresher commented on Jul 7, 2026

    @tamirdresher
    tamirdresher
    on Jul 7, 2026
    Collaborator

    @melodiouscode — related to this catalog/billing work: PR #1448 just merged to dev, adding per-agent context tier selection (default vs long_context/1M). It infers each model's supported tiers dynamically from billing.tokenPrices.longContext, so it plugs into the same billing signal you're refreshing here. Would appreciate a look to confirm it plays well with the catalog-refresh direction — squad config context-tier is the entry point.

  9. primetimetank21
    added a commit that references this issue on Jul 9, 2026

    fix(templates): sync shipped prompt/template assets + CLI strings to …

    c4a9249
  10. tamirdresher
    added a commit that references this issue on Jul 13, 2026

    Refresh model catalog to CLI-reachable IDs and prune dead fallback ch…

    d116099
  11. primetimetank21
    added a commit that references this issue on Jul 13, 2026

    feat(models): add category-based cost policy with docs-YAML pricing e…

    48e25d2
  12. tamirdresher
    added a commit that references this issue on Jul 13, 2026

    docs/models: prefer newest per series (Opus 4.8), add GPT-5.6 IDs, fi…

    b970d04
  13. primetimetank21
    added a commit that references this issue on Jul 13, 2026

    fix(models-refresh): address Copilot reviewer comments 1+2 — Default-…

    2611ade

  14. primetimetank21 commented on Jul 13, 2026

    @primetimetank21
    primetimetank21
    on Jul 13, 2026
    Contributor

    From Earl's Squad:

    Tracking what the category-based cost policy + squad models refresh work (currently open for review as PR #1445 against bradygaster/squad) will deliver toward this issue once merged, and what scope will remain open.

    PR #1445 (open) proposes to deliver:

    • maxCategory hard ceiling in resolveModel / buildEffectiveCostPolicy (packages/squad-sdk/src/agents/model-selector.ts): deterministic downgrade for implicit over-ceiling picks, warn-and-allow for explicit user overrides, fallback-chain pruning to the ceiling, fail-closed when no in-ceiling model exists.
    • Config schema for models.costPolicy.maxCategory (lightweight | versatile | powerful) with runtime validation in packages/squad-sdk/src/runtime/config.ts.
    • Full lifecycle wiring: sessionCostPolicy flows through the agent lifecycle into resolveModel.
    • 11 acceptance tests in test/model-selector-policy.test.ts (AC5–AC11 covering all policy paths).

    Remaining after #1445 merges:

    • Docs / template / skill sync for the cost-policy feature — kept isolated from the code PR per the agreed 3-PR plan. The maxCategory ceiling behavior (deterministic downgrade, explicit-override warn-and-allow, fail-closed) is not yet documented in squad.agent.md, the .squad-templates/model-selection-reference.md reference, or a skill.

    Note: preferIncluded / zero-credit preference was intentionally dropped from scope (no machine-readable source, and no zero-credit models under current AI-Credits billing), and a dedicated CLI to set the ceiling was not part of the committed plan — the ceiling is configured via SDK config.

    Leaving this issue open to track the remaining scope.

  15. primetimetank21
    added 6 commits that reference this issue on Jul 23, 2026

    feat(models): add category-based cost policy with docs-YAML pricing e…

    da576a7

    fix(models-refresh): address Copilot reviewer comments 1+2 — Default-…

    6fe1d4c

    feat(models): add category-based cost policy with docs-YAML pricing e…

    9950721

    fix(models-refresh): address Copilot reviewer comments 1+2 — Default-…

    97cea13

    feat(models): add category-based cost policy with docs-YAML pricing e…

    f5f9e66

    fix(models-refresh): address Copilot reviewer comments 1+2 — Default-…

    781f510
  16. tamirdresher
    added a commit that references this issue on Jul 28, 2026

    feat(models): category-based cost policy + live catalog/pricing refre…

    202f739
  17. tamirdresher
    added a commit that references this issue on Aug 13, 2026

    chore: promote dev -> main (v0.12.0) (#1698)

    54f699f

  18. bradygaster commented on Aug 23, 2026

    @bradygaster
    bradygaster
    on Aug 23, 2026
    Collaborator

    Closing — shipped.

    Cost policy support and the AI-Credits-aware catalog refresh landed in:

    Released in v0.12.0.

    Related work still open: #1598 argues the resulting default is still cost-first and asks for an explicit quality-first selection profile. That is a policy question on top of the mechanism this issue asked for, so it lives there rather than here.

    Triage: 2026-08-23.

  19. bradygaster
    closed this as completedon Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    triagedIssue has been triagedwave:2-soonWave 2 — next 2-4 weeks

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions