Skip to content

Research: Economy Mode E2E gaps (#585) #685

Closed

Description

@tamirdresher

Research for #585. 34 unit tests exist but NO E2E validation.

Gaps: No CLI invocation test, no spawn flow test proving economy propagates, no CostTracker integration test, no baseline vs economy comparison API, selectModelForTask is internal-only.

Key files: models.ts (ECONOMY_MODEL_MAP + estimateCost), model-selector.ts (selectModelForTask), cost-tracker.ts (per-agent metrics), economy.ts (CLI toggle), economy-mode.test.ts (34 unit tests), economy-mode/SKILL.md (policy).

Key insight: Per-token pricing IS cheaper but total cost depends on total tokens consumed. If economy model generates 15x more tokens, savings zeroed out. E2E must measure total tokens.

Proposed tests: CLI toggle round-trip, model resolution under economy, CostTracker comparison, full pipeline with real session.

Refs #585

Activity

  1. added
    squadSquad triage inbox — Lead will assign to a member
    type:spikeResearch/investigation — produces a plan, not code
    on Mar 29, 2026
  2. github-actions commented on Mar 29, 2026

    @github-actions
    Contributor

    📋 Assigned to FIDO (Quality Owner)

    Issue: #685 — Research: Economy Mode E2E gaps (#585)

    FIDO will pick this up in the next Copilot session.

    For Copilot coding agent: If enabled, this issue will be worked automatically.
    Otherwise, start a Copilot session and say:
    FIDO, work on issue #685

  3. github-actions commented on Mar 29, 2026

    @github-actions
    Contributor

    🏗️ Squad Triage — Flight (Lead)

    Issue: #685 — Research: Economy Mode E2E gaps (#585)
    Assigned to: FIDO (Quality Owner)
    Reason: Issue relates to testing/quality work

    @copilot evaluation: 🔴 Not suitable — issue involves work outside the coding agent's capability profile.

    Team roster:

    • Flight (Lead) → label: squad:flight
    • Procedures (Prompt Engineer) → label: squad:procedures
    • EECOM (Core Dev) → label: squad:eecom
    • FIDO (Quality Owner) → label: squad:fido
    • PAO (DevRel) → label: squad:pao
    • CAPCOM (SDK Expert) → label: squad:capcom
    • CONTROL (TypeScript Engineer) → label: squad:control
    • Surgeon (Release Manager) → label: squad:surgeon
    • Booster (CI/CD Engineer) → label: squad:booster
    • GNC (Node.js Runtime) → label: squad:gnc
    • Network (Distribution) → label: squad:network
    • RETRO (Security) → label: squad:retro
    • INCO (CLI UX & Visual Design) → label: squad:inco
    • GUIDO (VS Code Extension) → label: squad:guido
    • Telemetry (Aspire & Observability) → label: squad:telemetry
    • VOX (REPL & Interactive Shell) → label: squad:vox
    • DSKY (TUI Engineer) → label: squad:dsky
    • Sims (E2E Test Engineer) → label: squad:sims
    • Handbook (SDK Usability) → label: squad:handbook
    • Ralph (Work Monitor) → label: squad:ralph
    • @copilot (Coding Agent) → label: squad:copilot

    To reassign, remove the current squad:* label and add the correct one.

  4. diberry commented on Mar 31, 2026

    @diberry
    Contributor

    🔄 Ralph status: Research complete — 34 unit tests exist, 0 E2E. Gaps documented. Proposed tests identified. No implementation PR yet.

  5. bradygaster commented on Apr 25, 2026

    @bradygaster
    Collaborator

    Closed by #theSquadsquad triage — verified fixed/superseded in v0.9.4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    go:needs-researchNeeds investigationsquadSquad triage inbox — Lead will assign to a membertype:spikeResearch/investigation — produces a plan, not code

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions