vh
300ecc1276
Live on vllm-voices (fv-ml1 GPU0 :8027) beside voices-base, lv-yarros and lv-bronte.
Healthy 190 s after recreate, four models served, GPU0 96,092 -> 96,090 MiB. The adapter
was verified byte-identical to checkpoint-850 by sha256 across both transfer hops, and the
seat was verified by generating, not by reading its config: base emits 170 words of <think>
planning and never writes the passage, lv-hemingway writes the scene.
Gate design was pre-registered before any generation existed (0bb4938). Three arms, 60
held-out beats, 4 seeds, 240 generations per arm.
A. VOICE PASS 6.4x +0.413 delta_cb, pairwise floor 0.064 -- and it clears the OLD
all-arms floor (0.113) too, so this verdict does not lean on the
rule change. Closes 73.8% of the span between the unadapted
carrier and held-out Hemingway itself; lv-bronte closed 48%.
B. NOT COPIED see below
C. NO DAMAGE PASS ran-on +0.08, on-beat -0.14, both inside a 0.217 floor
AXIS B: THE NEGATIVE CONTROL WAS THE WRONG ONE, AND FIXING IT MADE THE RESULT WORSE, NOT
BETTER. memorization_check.py uses the base-unadapted arm as its control. Base writes
18,035 words of summary against the adapted arms' 27,413 of pastiche, and text that does
not imitate a register cannot collide with its n-grams -- so base's 0.00 measures "different
register", not "did not memorise". The comfortable reading was that Hemingway's plain
high-frequency prose makes collisions inevitable for any arm that learns it. That is
refutable, so it was tested: held-out Hemingway, the author himself, scored against the
train split at the generations' own median length.
HELD-OUT HEMINGWAY (never trained) 370 chunks 0.01 hit-rate mean-longest 0.1 max 10
base-unadapted 240 gens 0.00 0.0 0
ckpt850 (shipped) 240 gens 0.07 0.6 9
positive control (train vs train) 160
The hypothesis is false: the adapter reproduces train n-grams ~7x more often than the
author reproduces himself. That is real and is on the record. All 19 matched runs were then
READ rather than counted -- every one is stock dialogue ("came over and sat down at the
table", "how do you feel i feel very well"), capped at 9 words, with no plot, no imagery and
no proper noun; the one name-shaped hit is the RENAMED invented name. Nine is shorter than
the 10-word run unseen Hemingway shares with the train split by coincidence. Elevated rate,
zero protectable content. Hemingway is in copyright; lv-yarros is the in-line precedent,
also in copyright, shipped at 0.10 against a 0.07 control. Unload is 0.003 s.
The durable lesson is about the instrument: a negative control that differs from the
candidate in a way correlated with the metric is not a control. memorization_selfsim.py and
memorization_dump_matches.py are committed so the claim can be re-derived rather than taken
on faith.
SHIPPED ckpt850, NOT the loss minimum at step 1750. The two are indistinguishable on voice
-- 0.072 apart against a 0.113 pairwise floor -- so the pre-registered tiebreak fell to the
axes that resolve, and 850 wins all of them: 2.3x tighter seed spread (0.050 vs 0.113),
lower memorisation, less ran-on, half an epoch less overfit. ckpt1750's spread is one seed
(0.491, 0.449, 0.468, then 0.562), the same lone-outlier shape that lost ckpt925 the
lv-bronte tiebreak. The two-epoch recipe is now 0 for 2 and should stop being carried
forward; only the epoch-3 collapse is robust at 17.4x jitter.
servers/fv-ml1/ssh-target was a bare IP, so deploy-stack.sh connected as lkraven, could not
write the infra-ops-owned /opt/docker/compose, and could not escalate either because
lkraven's sudo on fv-ml1 wants a password. Now infra-ops@10.251.50.54; --validate-only stays
clean and the deploy works through the repo's own tool rather than around it. Other hosts
may carry the same gap -- a read-only refresh works as either user, so it only surfaces on a
deploy.
voices-seat — author voice adapters on one carrier
fv-ml1 GPU 0, port 8027. One Qwen3-4B-Instruct base; each author is a LoRA adapter
named lv-<author> (lv = lang-voice). Switching voices is a request field, not a
deployment.
curl http://10.251.50.54:8027/v1/chat/completions -H 'Content-Type: application/json' \
-d '{"model":"lv-yarros","messages":[{"role":"user","content":"Beat: ..."}]}'
# ^^^^^^^^^ the only thing that changes between voices
voices-base serves the unadapted carrier from the same process, which is what makes an
adapter-on / adapter-off comparison harness-matched.
Adding a voice
Two paths, and they are not interchangeable.
Try one now — no restart, ~0.24 s:
curl -X POST http://10.251.50.54:8027/v1/load_lora_adapter -H 'Content-Type: application/json' \
-d '{"lora_name":"lv-hemingway","lora_path":"/adapters/lv-hemingway-4b-v1"}'
curl -X POST http://10.251.50.54:8027/v1/unload_lora_adapter -H 'Content-Type: application/json' \
-d '{"lora_name":"lv-hemingway"}'
⚠ A runtime-loaded adapter is GONE on the next compose up -d. To make it survive, add
it to --lora-modules in compose.yaml — which costs a container recreate and a full model
reload (~3 min). Runtime load is for trying a voice; the compose list is what persists.
Adapters live on the host at /tank/aimodels/voice-adapters/<name>/, mounted read-only at
/adapters. A rename on the host is visible inside immediately.
Reaching it through LiteLLM
Each voice is one alias entry pointing at this seat with its own model value — no new
deployment, no new container, no VRAM:
- model_name: lv-yarros
litellm_params:
model: openai/lv-yarros
api_base: http://10.251.50.54:8027/v1
⚠ Retiring a voice orphans scoped keys. Any LiteLLM key whose allowlist names a removed
alias starts returning silent per-endpoint 403s. Audit /key/list + /key/info on every
repoint — that failure has cost this fleet two and a half months before.
What it costs, measured
| decode, base | 143.0 tok/s (median, n=30) |
| decode, LoRA | 108.2 tok/s (median, n=30) |
| LoRA overhead | −24.3%, against an A-vs-A noise floor of 0.1% |
| resident VRAM | 10,740 MiB |
| KV cache | 10,912 tokens → 1.33x concurrency at 8k context |
Arms were interleaved rather than blocked, because the co-tenants on that card take traffic this seat does not control and a block design would alias their load onto one arm.
The 24.3% is accepted deliberately. Three authors cost 8.4 GB as adapters and ~23 GB merged; six cost 9.2 GB versus ~46 GB. On a card with 1.8 GB free after this seat, that is the whole argument. If a voice ever lands on a latency path, merge that one and serve it separately.
Placement warnings
⚠ GPU 0 is now at 96.0 of 97.9 GB. This seat's 10.7 GB went into the last real gap on
fv-ml1: GPU 1 has ~5.7 GB, GPU 2 has ~2.4 GB, and GPU 3 is a held reserve for a future
full-card seat (flash-next alone needs 93 of 96 GiB). There is no room for another seat
here without a placement decision.
⚠ --gpu-memory-utilization is a request against TOTAL VRAM that the card must already be
able to honour — not a share of what is free. The first bring-up refused at 0.12 with
11.16 GiB free, and refusing was correct: it protected cyberprev, gen-small and the
Parakeet STT seat rather than squeezing them.
⚠ Pin --kv-cache-memory in bytes. With it, the requested fraction predicted residency
to within 40 MiB. Without it, this box has been wrong by 8–10 GB in both directions.
Support was checked, not assumed
vllm/model_executor/models/qwen3.py declares Qwen3ForCausalLM with SupportsLoRA plus
packed_modules_mapping and embedding_modules. The training playbook records a LoRA
refusal on a Qwen3 MoE architecture, and its lesson is that support is per-architecture,
not per-family. Do not transplant this compose onto an MoE carrier without re-running that
check.