2 Commits

Include renames

Author	SHA1	Message	Date
vh	300ecc1276	voices-seat: ship lv-hemingway (ckpt850), and replace the memorisation control that passed it ... Live on vllm-voices (fv-ml1 GPU0 :8027) beside voices-base, lv-yarros and lv-bronte. Healthy 190 s after recreate, four models served, GPU0 96,092 -> 96,090 MiB. The adapter was verified byte-identical to checkpoint-850 by sha256 across both transfer hops, and the seat was verified by generating, not by reading its config: base emits 170 words of <think> planning and never writes the passage, lv-hemingway writes the scene. Gate design was pre-registered before any generation existed (0bb4938). Three arms, 60 held-out beats, 4 seeds, 240 generations per arm. A. VOICE PASS 6.4x +0.413 delta_cb, pairwise floor 0.064 -- and it clears the OLD all-arms floor (0.113) too, so this verdict does not lean on the rule change. Closes 73.8% of the span between the unadapted carrier and held-out Hemingway itself; lv-bronte closed 48%. B. NOT COPIED see below C. NO DAMAGE PASS ran-on +0.08, on-beat -0.14, both inside a 0.217 floor AXIS B: THE NEGATIVE CONTROL WAS THE WRONG ONE, AND FIXING IT MADE THE RESULT WORSE, NOT BETTER. memorization_check.py uses the base-unadapted arm as its control. Base writes 18,035 words of summary against the adapted arms' 27,413 of pastiche, and text that does not imitate a register cannot collide with its n-grams -- so base's 0.00 measures "different register", not "did not memorise". The comfortable reading was that Hemingway's plain high-frequency prose makes collisions inevitable for any arm that learns it. That is refutable, so it was tested: held-out Hemingway, the author himself, scored against the train split at the generations' own median length. HELD-OUT HEMINGWAY (never trained) 370 chunks 0.01 hit-rate mean-longest 0.1 max 10 base-unadapted 240 gens 0.00 0.0 0 ckpt850 (shipped) 240 gens 0.07 0.6 9 positive control (train vs train) 160 The hypothesis is false: the adapter reproduces train n-grams ~7x more often than the author reproduces himself. That is real and is on the record. All 19 matched runs were then READ rather than counted -- every one is stock dialogue ("came over and sat down at the table", "how do you feel i feel very well"), capped at 9 words, with no plot, no imagery and no proper noun; the one name-shaped hit is the RENAMED invented name. Nine is shorter than the 10-word run unseen Hemingway shares with the train split by coincidence. Elevated rate, zero protectable content. Hemingway is in copyright; lv-yarros is the in-line precedent, also in copyright, shipped at 0.10 against a 0.07 control. Unload is 0.003 s. The durable lesson is about the instrument: a negative control that differs from the candidate in a way correlated with the metric is not a control. memorization_selfsim.py and memorization_dump_matches.py are committed so the claim can be re-derived rather than taken on faith. SHIPPED ckpt850, NOT the loss minimum at step 1750. The two are indistinguishable on voice -- 0.072 apart against a 0.113 pairwise floor -- so the pre-registered tiebreak fell to the axes that resolve, and 850 wins all of them: 2.3x tighter seed spread (0.050 vs 0.113), lower memorisation, less ran-on, half an epoch less overfit. ckpt1750's spread is one seed (0.491, 0.449, 0.468, then 0.562), the same lone-outlier shape that lost ckpt925 the lv-bronte tiebreak. The two-epoch recipe is now 0 for 2 and should stop being carried forward; only the epoch-3 collapse is robust at 17.4x jitter. servers/fv-ml1/ssh-target was a bare IP, so deploy-stack.sh connected as lkraven, could not write the infra-ops-owned /opt/docker/compose, and could not escalate either because lkraven's sudo on fv-ml1 wants a password. Now infra-ops@10.251.50.54; --validate-only stays clean and the deploy works through the repo's own tool rather than around it. Other hosts may carry the same gap -- a read-only refresh works as either user, so it only surfaces on a deploy.	2026-09-17 03:36:29 -07:00
vh	91bda3c480	fv-ml1: complete the cutover — rename, renumber, DNS, and the LiteLLM repoint ... The box is physically at Fountain Valley, renamed, renumbered onto 10.251/16, and serving inference again. This lands the repo half of that. Host: hostname ana-ml2 -> fv-ml1, pinned to 10.251.50.54 by a dnsmasq reservation so the address the runbook, DNS and LiteLLM all assume is the address it actually has. Its headscale node is renamed too. The sweep ran from scripts/fv-ml1-rename-sweep.sh, whose allowlist is the reason this diff touches current-state files and not the record. Dated persistent-memory entries, archival-memory and incident notes still say ana-ml2 in 31 and 62 places respectively, because that is what the box was when those things happened. Rewriting them would make the history lie. LiteLLM was the load-bearing piece and needed more than the api_base sed the runbook describes. Twenty api_base entries repointed, but a grep-and-verify pass also caught a LIVE pass_through_endpoints target for the scalar-judge reward route still on the old address -- an api_base-only substitution would have left it dead. Four prose references describing current state were repointed as well; one historical note recording where a hand-test was run is deliberately left pointing at 10.250.50.54. Two facts in the server tables were wrong and are corrected here. The site is Fountain Valley, not Anaheim. And the box has FOUR RTX PRO 6000 Blackwell Max-Q, not two -- verified by nvidia-smi -L and independently by PCI enumeration of four GB202GL devices. That is 391 GB of VRAM rather than 196, which changes what fits on it. DNS: fv-ml1, fv-ml1-bmc and fv-gw added under the fv site via the piggyback approach, scriberr re-homed, and the ana-ml2 records removed. Applied to all three resolvers. The BMC record carries a warning that its 802.1q VLAN tag must stay disabled -- it shipped tagging VLAN 250 into an untagged port, which made it invisible to every network-side diagnostic and is the reason it appeared dead through several cable changes. Verified end to end: summarizer and sec both answer through the Anaheim gateway across the mesh to FV seats on different ports.	2026-09-12 22:00:50 -07:00