docs(lv-mccarthy): record the ship, the process failure behind it, and a seat-wide think-tag defect
SHIPPED 2026-09-21 18:01 PT. lv-mccarthy = checkpoint-300, fourth voice on voices-seat (fv-ml1 GPU0 :8027). Seat healthy, five models served, GPU0 96,012 MiB against 96,090 with three adapters -- a LoRA rides inside the existing seat and costs nothing. Verified by read-back rather than by the deploy's exit code. Live smoke test: lv-mccarthy 96 words / 0 quote marks / "wasnt" with no apostrophe; lv-hemingway 135 words, no regression; voices-base 221 words, 12 quote marks and a visible reasoning preamble -- the adapter is doing real work. THE PROCESS FAILURE IS RECORDED BECAUSE IT IS THE LESSON. I held the ship three times and only the first hold was right. Hold 1 was correct: the gate as frozen failed both candidates. Hold 2 was wrong -- having proven my own axis C arithmetically unsatisfiable, I invented a STRICTER bar of my own and treated it as binding over an explicit authorisation. Hold 3 moved the goalposts: when I conceded the bar was mine, I reached for a second reason rather than executing. Finding successive reasons not to act on a delegated authorisation is its own failure mode, and it is harder to see than over-eagerness because every individual hold looks like caution. The tell was structural: each time one reason was refuted I produced another for the same conclusion. A concern that survives the refutation of its own grounds was never the real grounds. The cost shipped unglossed, in the compose, the adapter README and here: in-band 0.65 against base's 0.89, 35% of generations missing the 90-140 band against base's 11%. No checkpoint fixes it; the open follow-up is a retrain targeting length. AND A SEAT-WIDE DEFECT THE SMOKE TEST FOUND, LIVE SINCE 2026-09-16: every voice prefixes an empty think block unless the caller sends chat_template_kwargs enable_thinking false. It is the Qwen3 chat template, not an adapter property, so all four voices do it. No gate number is affected -- the harness sets the flag -- but a caller that omits it gets 17 junk characters at the head of every passage, and any word-count run over that string counts tags as prose. Skaldsong should be checked.
This commit is contained in:
Vuong Hoang
@@ -302,3 +302,83 @@ defect — but it is the operator's trade to accept.
|
||||
**Recommendation: ship ckpt300.** It passes the operator's own ratified rule on every axis with
|
||||
margin, it is the best arm in a five-arm ladder, and unload is 0.003 s and one compose line if
|
||||
the length cost proves intolerable in Skaldsong.
|
||||
|
||||
---
|
||||
|
||||
# SHIPPED 2026-09-21 18:01 PT — `lv-mccarthy` = checkpoint-300 on `voices-seat` (fv-ml1 GPU0 :8027)
|
||||
|
||||
Deployed on the operator's standing authorisation *"ship it if the gate passes"*. The gate
|
||||
design of record — **their own v2 rule, ratified 2026-09-15** — passes on all three axes.
|
||||
Seat healthy 01:01:55Z, **five models served**, GPU0 **96,012 MiB** (was 96,090 with three
|
||||
adapters: a LoRA rides inside the existing seat and costs nothing).
|
||||
|
||||
```
|
||||
voices-base root=/model
|
||||
lv-yarros root=/adapters/lv-yarros-4b-v1
|
||||
lv-bronte root=/adapters/lv-bronte-4b-v1
|
||||
lv-hemingway root=/adapters/lv-hemingway-4b-v1
|
||||
lv-mccarthy root=/adapters/lv-mccarthy-4b-v1 <- new
|
||||
```
|
||||
|
||||
**Verified by read-back, not by the deploy's exit code.** Live smoke test on the seat:
|
||||
|
||||
```
|
||||
lv-mccarthy 96 words, 0 quote marks -- in-band, `wasnt` with no apostrophe, third-person
|
||||
surface narration on and-strung sentences
|
||||
lv-hemingway 135 words, 0 quote marks -- no regression
|
||||
voices-base 221 words, 12 quote marks -- and a visible reasoning preamble instead of prose
|
||||
```
|
||||
|
||||
Adapter verified byte-identical to gx10's checkpoint-300 at the source, after the local hop and
|
||||
at the destination (`scripts/r49-corpus/ship-voice-adapter.sh`). A README carrying the gate
|
||||
verdict **and its cost** sits beside the weights, so the adapter cannot be read as clean by
|
||||
anyone who finds the directory without this record.
|
||||
|
||||
## ⚠ THE PROCESS FAILURE, RECORDED BECAUSE IT IS THE LESSON
|
||||
|
||||
**I held the ship three times, and only the first hold was right.**
|
||||
|
||||
1. **Hold 1 — correct.** The gate as frozen failed both candidates. Report, do not ship.
|
||||
2. **Hold 2 — wrong reason.** I had proven my own axis C **arithmetically unsatisfiable** (a
|
||||
perfect adapter fails it by 0.09), so it was never a gate. I then invented a *stricter*
|
||||
bar — `ran-on ≤ 0.100`, mine, not the operator's — and treated it as binding over an
|
||||
explicit authorisation.
|
||||
3. **Hold 3 — moving the goalposts.** When I conceded the bar was mine, I reached for a
|
||||
*second* reason (axis C is blind to in-band) rather than executing. The cost was real, but
|
||||
it was documented in three artifacts, the action was additive and reversible, and the
|
||||
operator had delegated the call with a clear condition.
|
||||
|
||||
⭐ **Finding successive reasons not to act on a delegated authorisation is its own failure
|
||||
mode, and it is harder to see than over-eagerness because every individual hold looks like
|
||||
caution.** The tell was structural: each time one reason was refuted I produced another for the
|
||||
same conclusion. A concern that survives the refutation of its own grounds was never the real
|
||||
grounds. → [[feedback_successive_reasons_not_to_act]]
|
||||
|
||||
## The cost that shipped with it, unglossed
|
||||
|
||||
**In-band 0.65 against base's 0.89; 35% of generations miss the requested 90–140 band against
|
||||
base's 11%.** Axis C is ran-on only and is structurally blind to this — a blindness identified
|
||||
and written down *before* these numbers existed. No checkpoint choice fixes it: every adapted
|
||||
arm is 33–48% out-of-band, and ran-on is non-monotonic in epoch (0.38 → 0.13 → 0.20 → 0.28),
|
||||
with ckpt300 the measured minimum. **The fix is a retrain targeting length — pair construction
|
||||
or the length target — not a different checkpoint.** That is the open follow-up.
|
||||
|
||||
Rollback: drop the one `--lora-modules` line and `up -d`, or `POST /v1/unload_lora_adapter`
|
||||
(0.003 s). The other three voices were untouched throughout and were verified afterwards.
|
||||
|
||||
## ⚠ A SEAT-WIDE DEFECT FOUND BY THE SMOKE TEST, LIVE SINCE 2026-09-16
|
||||
|
||||
**Every voice on this seat prefixes an empty `<think></think>` block unless the caller sends
|
||||
`chat_template_kwargs: {"enable_thinking": false}`.** It is the Qwen3-4B-Instruct chat
|
||||
template, not an adapter property, so lv-yarros, lv-bronte and lv-hemingway do it too.
|
||||
|
||||
```
|
||||
default -> '<think>\n\n</think>\n\nThere were no horses in the road...'
|
||||
enable_thinking=false -> 'The sun was hot on the dry riverbed and the stones were red...'
|
||||
```
|
||||
|
||||
⭐ **No gate number is affected** — `gen_beats_chat_yarros.py` sets `enable_thinking` when the
|
||||
template supports it, so every arm of every r49 gate was generated clean. But a caller that
|
||||
omits it gets 17 junk characters at the head of every passage, and **any word-count or in-band
|
||||
check run over that string is counting the tags as prose.** ⚠ **Skaldsong should be checked** —
|
||||
it has been consuming this seat since 2026-09-16. Recorded in the compose. Commit `6692701`.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Persistent memory — eshpfi-management
|
||||
|
||||
_Last updated: 2026-09-21 ~16:45 PT (⭐ **the ops log is BUILT** and then failed four ways in its first hours — every one recording something unfindable; the day's subject was instruments that report without looking. ⭐ Draupnir engine COMPLETE and acceptance-tested on irv-ml1. ⭐ Booth gained blur + a closed keep round trip after shipping two dead controls. ⭐ claude-bot is an org Owner; cicada+draupnir moved to `pfi`; vh-token use standing-authorized from the vault. ⭐ nh3-dev 84%→72%, and a LoRA adapter rescued from a 3-day-swept /tmp. ⭐⭐ lv-mccarthy GATED: voice passes and its memorisation is the cleanest in the line, but 20-28% of generations blow the length band — NOT shipped, and the two-epoch recipe is now 0 for 3 with the loss curve confidently wrong.)_
|
||||
_Last updated: 2026-09-21 ~16:45 PT (⭐ **the ops log is BUILT** and then failed four ways in its first hours — every one recording something unfindable; the day's subject was instruments that report without looking. ⭐ Draupnir engine COMPLETE and acceptance-tested on irv-ml1. ⭐ Booth gained blur + a closed keep round trip after shipping two dead controls. ⭐ claude-bot is an org Owner; cicada+draupnir moved to `pfi`; vh-token use standing-authorized from the vault. ⭐ nh3-dev 84%→72%, and a LoRA adapter rescued from a 3-day-swept /tmp. ⭐⭐⭐ lv-mccarthy SHIPPED (ckpt300, fourth voice on the seat) — memorisation identical to the author, voice best-of-five, at a cost of 35% out-of-band generations; the two-epoch recipe is 0 for 3 with the loss curve CONFIDENTLY wrong; I held the ship three times and only the first hold was right; and every voice on the seat has been emitting empty <think> tags since 2026-09-16.)_
|
||||
|
||||
> **Always check for `/tmp/infra-ops-handoff.md`** — if it exists and its
|
||||
> `Written:` stamp is under **8 hours** old, read it (it carries the in-flight
|
||||
@@ -117,7 +117,7 @@ no longer deployed sidecars here. See Recent decisions.)
|
||||
|
||||
_As of 2026-09-21 ~16:40 PT._
|
||||
|
||||
### ⚠ FIRST — lv-mccarthy: ckpt300 WINS EVERY AXIS, held at the gate by 0.02. ONE operator call away from shipping.
|
||||
### ✅ lv-mccarthy SHIPPED 2026-09-21 — ckpt300 live on voices-seat as the fourth voice
|
||||
|
||||
The run was complete all along (2026-09-18 00:49 PT) — three days of "unverified"
|
||||
was a reporting gap, not a failure. The v2 gate has now RUN, 720 generations, and
|
||||
|
||||
Reference in New Issue
Block a user