Skip to content

Add the Kimi Code Harness - #1923

Merged
JacobStephens2 merged 2 commits into
mainfrom
issue-1888
Nov 10, 2026
Merged

JacobStephens2 merged 2 commits into
mainfrom
issue-1888

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Nov 10, 2026 •

Copy link
Copy Markdown
Owner

Summary

A new Harness, kimi, runs a Command's sessions on Kimi Code CLI (@moonshot-ai/kimi-code 2.5.2), wherever the other Harnesses can be chosen: harness kimi, [harness.kimi], harness.default and Setup.

session
  kimi --output-format stream-json [-m <alias>] [--session <id>] -p "Load thirdshift-<skill> with your skill tool. …"
    stdin null; KIMI_CODE_HOME set (yours, else ~/.kimi-code, else <private HOME>/.kimi-code)
    KIMI_CODE_TRUST_WORKSPACE=1, KIMI_CODE_NO_AUTO_UPDATE=1, KIMI_DISABLE_TELEMETRY=1
  stream (stdout)          -> progress, final message, Skill-load warning, errors
    session.resume_hint    -> session id (printed only after a completed turn)
  wire.jsonl (after exit)  -> usage
    <KIMI_CODE_HOME>/sessions/<workspace>/<id>/agents/<agent>/wire.jsonl
    sum step.end usage by Model alias, cache reads and writes kept apart,
    counting only the turn this session's prompt started
    no id printed (failed turn) -> find the session folder holding the prompt
    no step.end usage           -> usage missing, not zero
 src/harness/
+├── kimi.rs              # adapter: invocation, Model check, Setup, release summary agent, AGENTS.md link
+└── kimi/
+    ├── catalog.rs       # kimi provider list --json: aliases, provider base URL, api_key_env
+    ├── stream.rs        # stream-json decoder
+    └── wire.rs          # wire.jsonl usage reader (Retained)
 src/proxy.rs             # + Fireworks hosts; a Kimi Trial allows only its alias's provider hosts
 src/sandbox.rs           # + Kimi config/sign-in copy, provider key from its api_key_env, npm package bind
 src/sandbox/triage.rs    # + Kimi home files in the private home
 src/harness/adapter.rs   # + Adapter::prepare (sets KIMI_CODE_HOME on every launch)
  • Model: an alias in Kimi Code's own config, checked against kimi provider list --json before any work (no tokens); a bad alias fails naming the configured ones. Kimi has no Effort flag, so any Effort fails before work.
  • Fireworks: configured as an OpenAI-compatible provider in Kimi's config.toml with api_key_env = "FIREWORKS_API_KEY", so the key stays in the environment, never in a repository (documented in the guide). A sandboxed kimi Trial on that Model gets api.fireworks.ai and nothing else; api.fireworks.ai is also in the shared provider table as fireworks-ai, so Allow Fireworks and Baseten through opencode, with a key per provider #1883 finds it there.
  • Release summary: Kimi's print mode approves every tool, so the summary runs --agent-file with an agent whose tools: [] and whose instructions are the release input.
  • Docs: guide (and regenerated manual), README, GLOSSARY, ADR-0013, docs/trial.md, docs/eval-tasks.md, site/index.html, site/architecture/, site/llms.txt.

Evidence

Execution evidence

  • cargo test --test kimi_sessions: 7 passed. A Run on harness kimi model kimi-k3.1-pro runs kimi --output-format stream-json -m kimi-k3.1-pro -p … with stdin null and KIMI_CODE_HOME set; usage across three step.end events is 16713 input tokens (14592 cache read, 576 cache write), 250 output tokens; a session with no step.end usage shows no usage line; model kimi-k9 fails before any Claim or worktree naming kimi-code/kimi-for-coding, kimi-k3.1-pro; a failed turn still reports its usage.
  • cargo test --test process_tree kimi: interrupting a kimi Run stops a command in its own session, and interrupting the live catalog check stops before work.
  • cargo test --bin thirdshift harness::kimi: 20 passed, including the Resume (--session <id> with the hinted id, same flags, null stdin) and proxy::tests::each_harness_allows_only_its_provider_hosts covering kimi on a Fireworks Model (["api.fireworks.ai"]).
  • Review findings, run as written before fixing:
    • Before: cargo test --bin thirdshift harness::kimi → test result: FAILED. 17 passed; 3 failed (claude_only_instructions_are_linked_as_agents: NotFound; a_turn_without_the_resume_hint_still_reports_its_wire_usage; wire::tests::a_resume_counts_only_its_own_steps)
    • After: cargo test --bin thirdshift harness::kimi → test result: ok. 20 passed; 0 failed
  • Full validation on c6c623f: scripts/affected-tests main → PASS npm test --prefix site, PASS cargo build --locked, PASS npm test --prefix console; its cargo test failed three tests/process.rs findings::* Security tests under load, which pass alone on this branch (cargo test --test process findings → 19 passed) and on main. The whole suite then passed with cargo test --no-fail-fast (exit 0, 99 test binaries ok). cargo clippy --all-targets clean; npm run docs:check --prefix site clean.
  • Not done: a real kimi run. No Fireworks or Kimi credentials were available here, and running the downloaded Kimi Code CLI against a local stub provider was refused by this session's permission policy. The step.end usage shape and stream lines come from reading Kimi Code 2.5.2's published bundle (finishMachineStepProjection writes step.end with usage inside context.append_loop_event; PromptJsonWriter; writeResumeHint); tests/fixtures/kimi-wire.jsonl is reconstructed from that source and says so. See Unaddressed findings, P2.

Visual evidence

Architecture diagram, desktop 1440×900 (Agent session box now lists kimi):

Before
After

Architecture diagram, mobile 390×844:

Before
After

Home page Requirements, desktop 1440×900:

Before
After

Home page Requirements, mobile 390×844:

Before
After

Merge Danger

Door: two-way

A new, opt-in Harness. Nothing runs on kimi unless a command or the User config chooses it.

Blast Radius: small

Shared edits: Adapter::prepare (a no-op default) in the Harness launch helpers, a fireworks-ai entry in the proxy's provider table (OpenCode Trials naming that provider now pass the host check), and a [harness.kimi] section in Setup's written User config.

Review

Standards: 10 findings; Spec: 7. Both reviewers read every changed file except site/docs/index.html, generated from docs/guide.md and confirmed by both with npm run docs:check --prefix site (clean), so it was left unread on purpose. Addressed: S1 (AGENTS.md link, ADR-0013), S2/P6 (duplicate fixture removed), S3 (shared HOME_FILES), S5 (accumulate, input_including_cache), S6 and S7 (doc fixes), S9 (panic message), S10 (stale Harness lists), P1 (failed-turn usage via the session folder), P5 in part (Fireworks in the shared table), P7 (Resume counts only its turn), and P2's fixture metadata (protocol_version: "1.5").

Unaddressed findings

Standards

  • S4 (Duplicated Code: a Kimi Trial reads kimi provider list twice). Declined: proxy::allowed_hosts(harness, model) is the shared per-Harness entry the sandbox calls for every Harness, and kimi_into runs earlier in sandbox::prepare's home step; sharing one read would thread Kimi state through the shared Request. The extra read is a free local CLI call (src/proxy.rs Harness::Kimi => kimi_trial_hosts(model, Catalog::read)?).
  • S7 (Primitive Obsession: Catalog keeps raw JSON). The doc-comment part is fixed; the typed model is declined: provider() is the only reader, and Kimi's config schema has many optional, aliased fields (provider/providerId, model- or provider-level baseUrl, oauth) that it reads in one place (src/harness/kimi/catalog.rs fn provider).
  • S8 (side effect in Kimi::summary). Declined: the Adapter::summary contract requires the input travel by stdin or a file beside prompt_file, which "remains available until the CLI exits" (src/harness/adapter.rs); Kimi's -p reads neither, and an agent file is its only way to remove every tool. A failed write fails the session, and the release falls back to generated notes (src/harness/summary.rs).
  • S3 (Kimi Trial knowledge spread across proxy.rs/sandbox.rs/triage.rs). The duplicated list is fixed; moving Trial logic behind the adapter is declined here: every Harness's Trial hosts and homes already live in those three matches, and changing that is an architecture change beyond this Ticket.

Spec

  • P2 (a real kimi run's wire.jsonl not checked). Not done in this session: no credentials, and running the downloaded CLI against a local stub was refused by the session's permission policy. The shape follows Kimi Code 2.5.2's source as above. Filed for the Day shift: Check a real kimi run's wire.jsonl against the Kimi Harness's usage parser #1925
  • P3 ([harness.kimi] has no provider or API-key-variable keys). Declined, because the Spec's Key interfaces must decide: the provider and its api_key_env live in Kimi Code's own config.toml, which Kimi already reads and validates; thirdshift reads them from kimi provider list --json (src/harness/kimi/catalog.rs Provider { base_url, api_key_env }). Adding them to [harness.kimi] would make two sources for one setting. Filed: Should [harness.kimi] carry the Kimi provider and its API-key variable? #1924
  • P4 (Resume tested only at the adapter seam). Declined: a Resume starts only after killed background work (GLOSSARY Resume; src/session.rs if let (false, Some(session_id)) = (ended.killed.is_empty(), …)), and Kimi's print mode waits for or steers its own background work and reports none killed (applyPrintBackgroundPolicy, default steer), as with Grok (guide). harness::kimi::tests::a_resume_uses_the_hinted_session_id_with_the_same_flags_and_null_stdin passes.
  • P5, remainder (kimi_into key forwarding and the npm package bind untested). Declined here: both need a real kimi on PATH (Catalog::read) and bwrap; the host decision they feed is covered by proxy::tests::each_harness_allows_only_its_provider_hosts and a_kimi_trial_allows_only_its_models_provider_hosts, which pass.

Closes #1888

Built with claude 2.1.314 · opus · default effort

Security review outcome

No unaddressed introduced findings.

CI notes

  • console-test (run 42729950160): console/tests/logs.test.cjs "a live Session appends without redraw…" timed out after 5 s waiting for the final unterminated line to drain. This branch changes neither console/ nor the Console's log streaming. The test passed 3 times on its own and the full suite passed 168/168 locally on this branch, so the failure looks intermittent.

Sessions run kimi -p --output-format stream-json with stdin null and a set
KIMI_CODE_HOME; usage is the total of the session's wire.jsonl step.end
events by Model alias, with cache reads and writes kept apart. Models are
checked against kimi provider list --json, a Trial's proxy allows only the
alias's provider hosts (api.fireworks.ai for Fireworks), and Setup offers
kimi when it is installed.
- Link AGENTS.md to a CLAUDE.md-only worktree, as for OpenCode (ADR-0013).
- Find a failed turn's session folder by its prompt, and count only the
  turn this session started when a Resume appends to the same record.
- Add Fireworks to the shared provider host table.
- Share Kimi's home files, drop the duplicate stream fixture, and fix stale
  Harness lists in the guide and eval docs.
@JacobStephens2
JacobStephens2 merged commit 6488790 into main Nov 10, 2026
19 of 20 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-1888 branch November 10, 2026 01:07
@JacobStephens2 JacobStephens2 mentioned this pull request Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a Kimi Code CLI Harness

1 participant