Skip to content

ROADMAP-12 — P1 — No runs.model column or cost attribution; cached-token detail ignored #104

Open

Description

@getdownandcode

Series: ROADMAP — filed from the 2026-10-27 whole-project assessment · Backlog task: M15-T05 · Priority: P1 · Effort: medium

Problem

Runs cannot be attributed by model or cost:

  • runs has token totals but no model column, despite LLM_PROVIDER/LLM_MODEL being configurable per deployment and per client
  • Cached-token accounting is ignored: OpenAI prompt_tokens_details and Gemini cached_token_count are not read (only prompt_tokens / candidates_token_count), so cost models cannot distinguish cached vs fresh input
  • The unrestricted provider payload is persisted into raw_metadata with no field filtering or size cap

Token placement correctness is ROADMAP-06; this issue is the data model + attribution.

Evidence

  • agentkit/openai extraction agentkit/llm/openai.py:199–200, metadata :202–207
  • agentkit/llm/gemini.py:252–255 usage extraction, :270–277 metadata dump
  • agentkit/db/models.py:38–47 — runs columns

Suggested fix / acceptance criteria

  • runs.model persisted at run creation (and recorded in traces)
  • Cached-token counts captured in TokenUsage and surfaces in metadata
  • raw_metadata filtered to a bounded, documented field list
  • Docs describe cost reading; (optional) aggregated /metrics-style sums arrive with ROADMAP-25

Suggested branch / commit

feat/run-model-cost · feat(db): record run model and cached-token/cost attribution

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions