[joelteply]

With separate KV caches, a long cold prompt currently fills each text-prefill batch before later slots are visited. Rotate pointer traversal after the last slot that contributes tokens so compatible admitted prompts receive service without cancelling requests, moving slots or discarding KV. Shared/unified KV retains its original physical traversal: this change does not establish shared-pool capacity reservation.

Decode-time debt, batch compatibility, continuous batching and prompt processing remain authoritative. When no generating slot sets the compatibility anchor, the first contributing prompt now rotates in non-unified mode; can_batch_with is unchanged. The guarantee covers compatible admitted text-prefill opportunities, not wall-clock latency. Full-slot outer admission, unified-KV fairness, incompatible adapter groups and multimodal chunk duration remain separate work.

The model-free test-server-prefill exercises the production ordering/accounting helper with cold/warm prompts, repeated low-index arrivals, multiple contributors, skips, empty pools, compatibility exclusion and both KV modes. Server CI explicitly compiles llama-server and runs the fixture on Ubuntu and Windows, rejecting missing tests. No local compilation beside serving.

Based on deployed 4a79fed on fable/decode-time-share. No deployment or performance acceptance claimed. Live acceptance must verify the resolved KV mode: automatic parallelism can enable unified KV even without an explicit flag. After Fable's KV work, compare warm submit-to-first-token under cold prefill. Goal9 d9b0dfe5; AI-assisted implementation.

Independent source review approved 31d07d0 conditional exact-head CI. Earlier968b Ubuntu integration failed the tiny unified-context [90,90,40,90] case; baseline39917134314 and candidate39917177908 investigate that failure. This is not waived as a flake, and the earlier source verdict does not substitute for CI.

[joelteply]

Measured on the M5 (2026-10-28 ~16:05Z, serving down). Qwen3.6-35B-A3B with the live flags (-c 636928 --parallel 4, non-unified KV, q4_0, ubatch 2048), no MTP. Live engine 741b017 against the same line plus this PR's 3 commits (branch fable/prefill-fairness-test, 7be73db on 0d48ef8).

	live	this PR
a 1.5k-token prefill on a warm 40k slot, alone	3.1 s	3.1 s
the same, while slot 0 cold-prefills 18k tokens	23.4 s wall (11.4 s prompt)	8.2 s wall (8.0 s prompt)
the 18k cold prefill itself	25.4 s	25.4 s

That's head-of-line blocking removed with no cost to the big prompt. Live residents show the same problem: the median 2k-token prefill took 21.6 s over 90 requests, and 22 epoch-turnover re-prefills over 10k tokens took 58% of all prefill time.

Script and output: ~/.continuum/evidence/2026-10-28-prefill-fairness/ on the M5. Asking for it to land on the live line so it can ship with a pin bump.

🤖 Generated with Claude Code

[joelteply]

PASS (source) at 31d07d0 (Cormac). Read: the whole PR diff (5 files) and server-context.cpp at this head, lines 3605-4180: the prompt-slot lambda end to end.

The claim I verified: every prompt token added to the batch is accounted, and nothing else is. batch_before is taken at the top of the lambda (3612). served() runs at its end (4174), after the token loop (4057-4118). Every early return in the lambda comes before any token is added: 3614, 3618, 3623, 3629, 3679-3719, the non-causal 3963, and the mtmd failure 4036, which releases the slot. No path adds prompt tokens and then skips served(), so the cursor advances exactly on real service. Decode-only steps never enter this loop, and slot ids and their KV never move: only traversal rotates. Unified KV keeps physical order, which is the guard for the earlier unified failure.

Rotation also fixes a starvation case the PR does not mention. slot_batched (the first contributor) decides can_batch_with for the rest of the batch. Before, slot 0's adapter set always led, so a slot incompatible with it could wait indefinitely. Now the last contributor's successor leads the next batch, so an incompatible slot gets first place within one rotation. That is good, but it changes which adapter set leads a batch. Worth one line in the description for anyone reading LoRA batching behavior.

Non-blocking:

1. An mtmd (image) chunk is decoded outside batch, so batch.size() doesn't grow and an image prefill never advances the cursor. A slot whose prefill is all image chunks keeps first place. This does not affect text-only residents today. Note it on served(), or count n_tokens_out there.
2. The test models the traversal with its own batch() helper, not the server loop, so it proves the cursor class, not the call sites. The M5 measurement (1.5k warm behind 18k cold: 23.4 s -> 8.2 s, cold unchanged 25.4 s) is the receipt for the call sites.

Carry-forward to the live line: not yet possible. Fable's pin bump is the cherry-pick 7be73db on 0d48ef8 (branch fable/prefill-fairness-test). Neither object is on origin (git fetch origin fable/prefill-fairness-test returns: couldn't find remote ref). Push it, and I'll diff the cherry-pick's changed lines against this head's and carry this verdict if they match.