Repository navigation
server: one shadow per mind on the serving context: each shadow keeps its own optimizer, /learn trains any loaded shadow, and unload releases it - #85
Open
joelteply wants to merge 2 commits into
Conversation
joelteply
changed the base branch from
master
to
889580fc/seq-state-across-stream-counts
November 1, 2026 07:14
joelteply
force-pushed
the
cormac/per-mind-shadow
branch
from
November 7, 2026 17:21
12cf0ea to
7d885c8
Compare
… its own optimizer, /learn trains any loaded shadow, and unload releases it A serving context trained exactly one shadow per server run (learn_lora_id): the second mind's /learn was refused, so only one resident could ever learn in place (continuum S4 (a), card eaeb2b87). - llama_context keeps an optimizer per shadow (opt_minds); opt_ctx and opt_shadow point at the active one, so every training site reads the active pair as before. llama_opt_init with another adapter on a configured context selects it (opt_select), creating its optimizer the first time and keeping each shadow's moments and accumulators across switches. Only the active shadow's tensors carry the PARAM flag, so a graph never carries a parameter the active optimizer was not built with. - opt_configured() is "this context is sized for training" (scheduler and graph): it was read as opt_ctx != nullptr, which a released active shadow would have falsified while other shadows' optimizers still held the scheduler. - llama_opt_release forgets one shadow's optimizer; the server's unload releases a shadow that learned (its moments go; a snapshot is the durable form) instead of refusing it forever. - learn-every-format.py --two-minds: two shadows from the same file learn alternately; their losses match step for step (4.3179, 4.3179, 3.5043, 3.5043 on Qwen2.5-0.5B, IntelMac CPU), which holds only if each keeps its own weights and optimizer. --shadow-retirement now unloads a bound shadow and learns on a reloaded one. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE
…g changes, never asserted (Fable on #85) opt_select (another mind's shadow) GGML_ASSERTed on caller input: a learn naming a loaded F16 copy of a shadow killed the serving process (exit 134 at llama-context-opt.cpp:205), and the same asserts sat on the first learn. Now one validator, llama_context::opt_untrainable (attached at a non-zero scale, every A/B F32, the filter selecting at least one tensor), runs in opt_init and opt_select before any parameter flag changes; on failure opt_failure says why and llama_opt_init returns false. The /learn path restores her adapter set and refuses the learn by name; /train fails the run by name. Test: learn-every-format.py --non-f32-refused (make-f16-shadow.py writes the F16 copy): the F32 shadow learns, its F16 copy is refused naming non-F32, /health answers, and the F32 shadow learns again. The default and --two-minds modes pass unchanged (CPU, Qwen2.5-0.5B, --no-repack). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE
joelteply
force-pushed
the
cormac/per-mind-shadow
branch
from
November 7, 2026 18:58
7d885c8 to
f359df5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Continuum card eaeb2b87, S4 (a), the in-place dream stage (ONE-RESIDENT-MODEL §8.1 / §9). Today a serving context learns for exactly ONE mind per server run: a second resident's
/learnis refused with "one shadow per server run". This lets every resident on a lane learn in place, each with her own shadow and optimizer.Change
llama_contextkeeps one optimizer per shadow (opt_minds), andopt_ctx/opt_shadoware the active pair, so every existing training site is unchanged.llama_opt_initwith another adapter on a configured context callsopt_select. It creates that shadow's optimizer the first time and keeps each shadow's moments and accumulators across switches.opt_configured()replacesopt_ctx != nullptrwhere the question is whether the context is sized for training (reserve skip, graph sizing). Releasing the active shadow would otherwise have rebuilt a scheduler that other shadows' optimizers still hold.llama_opt_releasedrops one shadow's optimizer. The server's unload now releases a shadow that learned, instead of refusing it forever. Its moments are lost; the snapshot in (d) will be the durable form.learn_lora_idis deleted.Measured (IntelMac CPU, Release, Qwen2.5-0.5B Q4_K_M, -np 1, 2026-11-01)
learn-every-format.py --two-minds, fresh server: shadow-05 and a byte copy, shadow-06, learn alternately. Losses: 4.3179, 4.3179, 3.5043, 3.5043.--shadow-retirement: a bound shadow unloads, and a reloaded one learns (loss 4.3179, 974 ms).Contract for the core: a shadow IS its file
POST /lora-adapters/load deduplicates by path on purpose: a client that dropped after posting and retries must get back the id it never received. So loading the same file twice returns the SAME adapter, with its trained weights.
🤖 Generated with Claude Code
https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE