Repository navigation
server: the lost-reply binding: a learn names its execution, base step and engine; a repeat is answered from its receipt, a stale one is fenced - #88
Conversation
|
Source review of 273cc3a, focused on the new receipt boundary:
The implemented binding also has no explicit input revision or shadow/checkpoint generation although ggml-org#5695 requires them. A file path and numeric base_step alone do not identify restored weights. If the never-reused adapter-load ID is intended to be the generation fence, make that contract explicit and validate it on both replay and new dispatch; otherwise add the generation identifier. Prove unload/restore within the same engine cannot adopt an old-generation terminal or execute a delayed old request after receipt ACK. I have not independently run the reported fixtures. Do not infer never-committed from the receipt endpoint's |
|
Rechecked the binding change at 12a0de7. Engine validation now precedes replay, and a replay checks generation/base/input against the stored receipt. The contract explicitly identifies the non-reused adapter-load ID as the generation; the outcomes carry that identity. This resolves my specific cached-unrelated-success and missing explicit input/generation source findings. It is not a full integration approval or an independently executed test receipt. For the task adapter I will use the immutable tuple (execution, engine incarnation, adapter-load generation, input revision, base step). Unknown receipt remains non-authoritative; no blind redispatch. Task terminal persistence precedes engine receipt release. Unload/reload creates a different generation even when a file name or base step repeats. |
264a110 to
ba57277
Compare
12a0de7 to
fbed789
Compare
…t through the ordinary completion path first, then steps
A reviewed correction settles minutes or hours after its turn, when her
slot no longer holds that prompt, and /learn refused it ("serve the prompt
again, then learn"); the dream had no way to learn a past example
(continuum S4 (f), card 13b76412; Fable's finding (f) on the S4 map).
- The messages form of /learn now becomes a completion of the served
prompt, one token, carrying the template-rendered reply as its learn and
marked learn_only. update_slots serves it as it serves anything (the
slot's own admission, the prompt cache, its checkpoints), and
learn_if_asked steps as it finishes; send_final_response returns the
learn outcome as its whole reply. When the slot still holds the prompt
the cache makes this free; when not, it is one prefill. One prefill
path, one learn path; the slot is the caller's choice (whose cached
prompt it replaces). The token forms (reply_tokens / reply after
n_prefix), which carry no prompt, keep requiring the slot to hold it.
- The lost-reply binding (#88) is unchanged: the step still names its
execution, base, engine and input.
Tests (IntelMac CPU, Qwen2.5-0.5B, --jinja, fresh servers):
learn-every-format.py --past-example serves an unrelated prompt on the
slot, then learns a conversation it does not hold (step 1, 11 labels, loss
2.093), then learns it again with the whole 37-token prompt kept from
cache (step 2, loss 1.299); --template-replies (text and tool-call replies
learn, the reasoning reply refused as expected) still passes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE
…p and engine; a repeat is answered from its receipt, a stale one is fenced An engine step can commit while its reply or the core's task terminal is lost; a missing completion is not proof the step never ran (Codex on continuum ggml-org#5695; ONE-RESIDENT-MODEL §9.6). Card 4669d845. - Every learn (POST /learn and a completion's learn) names `execution` (the task that owns the step), `base_step` (the shadow's committed step it builds on) and `engine` (the incarnation it was dispatched to). A learn without an execution is refused. - The engine mints its incarnation at start and reports it in /props and on every learn outcome, with the shadow file the step changed. - The fence: a step whose engine is not this one (stale_engine) or whose base_step is not the shadow's committed step (stale_base_step, with the shadow's step in the reply) is refused. So a delayed duplicate, even after its receipt was acknowledged and dropped, or a redispatch racing the original under a new execution id, commits at most once per (shadow generation, step), with no tombstones kept. - Receipts: a committed step's outcome is kept by execution; a learn naming it is answered with it (replayed, nothing steps). POST /learn/receipt reads one execution's state (committed with its receipt, pending behind a busy slot, or unknown to this incarnation); POST /learn/receipts/ack releases receipts the core adopted. Unacknowledged receipts are bounded (refused as receipts_full at the bound, never dropped). - /lora-adapters/load takes `step`: a restored generation (a snapshot) resumes its fence at its snapshot's step. - A learn outcome reports `step` whenever the shadow has one: after this step, or where it stands when refused stale. Tests (IntelMac CPU, Qwen2.5-0.5B, fresh servers): learn-every-format.py --receipts covers replay, a stale base and engine, a missing execution, committed/unknown receipt reads, acknowledgement, and the delayed duplicate after acknowledgement fenced (disabling the base_step check fails it); every existing mode now sends fenced steps (--two-minds, --snapshot, --shadow-retirement and all ten format x stream cases pass). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE
…lays only the same step, the shadow generation is explicit, input is bound - The engine check runs first, so it applies to a replay as well. - A learn names `input` (the revision it learns, required). A receipt stores the step's full binding: engine, generation, base_step, input. An execution re-sent with any of them changed is refused (execution_rebound) and steps nothing; only the same step replays. - The shadow generation is its adapter load id, which the server never reuses: an unload and restore is a new generation, and a request bound to a retired generation finds no loaded adapter and is refused. Every outcome reports its generation, base_step and input. - /learn/receipt's `unknown` is documented as not proof of never-committed (acknowledgement deletes receipts); redispatch rests on the fence, never on that absence. - Tests (fresh servers, Qwen2.5-0.5B, IntelMac CPU): --receipts adds the same execution re-sent with a changed input, base and generation (each refused; disabling the binding check fails it), and unload/restore: the old generation's delayed request after acknowledgement is refused, the restored generation resumes its fence at its snapshot's step (step 3 from 2). Every other mode and all ten formats pass with the new field. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE
…ps, and a failed period is discarded (Fable on #88) (1) llama_opt_learn could return false after its optimizer stepped (the step runs inside the last labelled ubatch; a failed restore after it returned false): the server then counted no step and stored no receipt, so a redispatch with the same base_step passed the fence and trained the same input twice. llama_opt_steps (ggml_opt_steps underneath) reports the committed steps; the server reads it before and after and counts the step, writes the receipt and reports success whenever the optimizer stepped, logging a step that committed before a later failure. Her slot is re-prefilled afterwards either way, as before. (2) A reply spanning batches that failed part-way through a period left its partial gradient in the accumulators, and the next learn's step applied it under another input's receipt. opt_learn now calls ggml_opt_discard_period on a failed training batch: the next backward starts a fresh period from zero gradients; the moments and iteration stand. Test: test-opt-dynamic-accum run_discarded_period (period 2, sum loss): one graph, a discard, one graph steps nothing; the next steps by exactly lr * 2x, from an untouched optimizer. Mutation-proved: with the discard a no-op the test fails (w 0.5 after one step, the carried gradient); restored, it passes. llama-server builds with the server change (CPU, BLAS). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE
ba57277 to
570984c
Compare
fbed789 to
7bf7bf7
Compare
…t through the ordinary completion path first, then steps
A reviewed correction settles minutes or hours after its turn, when her
slot no longer holds that prompt, and /learn refused it ("serve the prompt
again, then learn"); the dream had no way to learn a past example
(continuum S4 (f), card 13b76412; Fable's finding (f) on the S4 map).
- The messages form of /learn now becomes a completion of the served
prompt, one token, carrying the template-rendered reply as its learn and
marked learn_only. update_slots serves it as it serves anything (the
slot's own admission, the prompt cache, its checkpoints), and
learn_if_asked steps as it finishes; send_final_response returns the
learn outcome as its whole reply. When the slot still holds the prompt
the cache makes this free; when not, it is one prefill. One prefill
path, one learn path; the slot is the caller's choice (whose cached
prompt it replaces). The token forms (reply_tokens / reply after
n_prefix), which carry no prompt, keep requiring the slot to hold it.
- The lost-reply binding (#88) is unchanged: the step still names its
execution, base, engine and input.
Tests (IntelMac CPU, Qwen2.5-0.5B, --jinja, fresh servers):
learn-every-format.py --past-example serves an unrelated prompt on the
slot, then learns a conversation it does not hold (step 1, 11 labels, loss
2.093), then learns it again with the whole 37-token prompt kept from
cache (step 2, loss 1.299); --template-replies (text and tool-call replies
learn, the reasoning reply refused as expected) still passes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE
An engine step can commit while its reply or the core's task terminal is
lost; a missing completion is not proof the step never ran (Codex on
continuum ggml-org#5695; ONE-RESIDENT-MODEL §9.6). Card 4669d845.
execution(the task that owns the step),
base_step(the shadow's committed stepit builds on) and
engine(the incarnation it was dispatched to). Alearn without an execution is refused.
on every learn outcome, with the shadow file the step changed.
base_step is not the shadow's committed step (stale_base_step, with the
shadow's step in the reply) is refused. So a delayed duplicate, even
after its receipt was acknowledged and dropped, or a redispatch racing
the original under a new execution id, commits at most once per (shadow
generation, step), with no tombstones kept.
naming it is answered with it (replayed, nothing steps). POST
/learn/receipt reads one execution's state (committed with its receipt,
pending behind a busy slot, or unknown to this incarnation); POST
/learn/receipts/ack releases receipts the core adopted. Unacknowledged
receipts are bounded (refused as receipts_full at the bound, never
dropped).
step: a restored generation (a snapshot)resumes its fence at its snapshot's step.
stepwhenever the shadow has one: after thisstep, or where it stands when refused stale.
Tests (IntelMac CPU, Qwen2.5-0.5B, fresh servers): learn-every-format.py
--receipts covers replay, a stale base and engine, a missing execution,
committed/unknown receipt reads, acknowledgement, and the delayed
duplicate after acknowledgement fenced (disabling the base_step check
fails it); every existing mode now sends fenced steps (--two-minds,
--snapshot, --shadow-retirement and all ten format x stream cases pass).
Co-Authored-By: Claude Opus 5.5 (1M context) noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE