memory: retract the MTP-head degeneration hypothesis; n=1 was never evidence

Operator ruling: the multi-turn degeneration lives in the un-fixed vLLM, not
in the weights. The hypothesis that sec's stock-graft MTP head causes it is
withdrawn.

Two failures produced it. First, a false dichotomy treated as a deduction:
having verified gen and sec run an identical engine, I concluded config was
eliminated and therefore the weights were responsible. That does not follow.
An engine bug present in both seats is not exonerated by the seats being
identical; it only means the engine cannot explain a difference between
them. It can still explain the failure.

Second, and more instructive, the difference being explained may not exist.
The premise was a single operator observation made during a session with
many concurrent changes. That cannot carry a causal claim, and it became the
load-bearing support for a root-cause narrative it could not hold.

The same caveat now attaches to the coherent-to-10k observation on the new
build: same n, same uncontrolled conditions, opposite direction. The
comparison is weak at both ends, so the file no longer presents either
sighting as a result.

What survives as measured fact is unchanged and still recorded: sec's MTP
head is byte-identical to the uncensored base across all 15 tensors, gen's
was abliterated in-band, and acceptance differs slightly. None of that is
shown to cause degeneration.

Adds the generalisable lesson: an observation made while many things are
changing cannot support a causal conclusion. It is the inverse of the
warning already in the gen-seat compose file, which guards against trusting
a negative result from a synthetic probe; this guards against trusting a
positive sighting from an uncontrolled session.

persistent-memory.d/2026-08-22-dflash2-spec-decode.md

@@ -70,31 +70,60 @@ plausible (the drafter reads quantized hidden states at its five taps). Second c
 prompt distribution (ours general-purpose, theirs GSM8K/MATH/HumanEval/MBPP/MT-Bench).
 **Neither is confirmed.** Settling it needs a BF16 target seat (~56 GB) — a real GPU window.
 
-## 🔶 HYPOTHESIS — why sec degenerated and gen did not
+## ❌ RETRACTED — the "MTP head mismatch causes the degeneration" hypothesis
 
-Operator confirmed **sec degenerates at ~2k tokens as served**, gen does not, on identical
-engines. Config was eliminated as a variable:
+**Operator ruling, 2026-08-22: this hypothesis is WRONG. The degeneration lives in the un-fixed
+vLLM, not in the weights.** Recorded here rather than deleted, because it was reasoned to
+confidently enough that a future session could re-derive it.
 
-- **Same vLLM image ID** `sha256:bd3236cff208…`, same live version `0.27.2rc1.dev150+g311b3513a`
-  read from inside both running processes (tag equality alone would not prove this).
-- Same `qwen3_5_mtp` k=3, same `--enable-prefix-caching`, same `fp8` KV, same `float32` mamba
-  cache. Only deltas were `gpu-memory-utilization` 0.43 vs 0.44 and the served name.
+**Two independent failures produced it, and the second is the instructive one:**
 
-**The standing hypothesis is the MTP head.** Per our own provenance (verified with
-`compare_mtp_head.py`): sec's head is **byte-identical to `qwen38-27b-uncensored-bf16`, all 15
-tensors** — a stock head against a security-finetuned body, because the
-`Qwen3_5ForConditionalGeneration` wrapper never loads the head so the finetuning could not reach
-it. gen's orcarouter head was **abliterated in-band by the author**, matched to its body.
-Acceptance corroborates (gen 58.4% vs sec 55.9%).
+1. **I treated a false dichotomy as a deduction.** Having verified gen and sec run an identical
+   engine (same image ID `sha256:bd3236cff208…`, same live version
+   `0.27.2rc1.dev150+g311b3513a` read from inside both processes, same flags bar
+   `gpu-memory-utilization` 0.43 vs 0.44), I concluded "config is eliminated, therefore it is the
+   weights." That does not follow. **An engine bug present in BOTH seats is not exonerated by the
+   two seats being identical** — it just means the engine cannot explain a *difference*. It can
+   still explain the *failure*.
+2. **The difference I was explaining may not exist.** The premise was a single operator
+   observation of sec degenerating at ~2k, made during a session with many concurrent changes.
+   **n=1 under heavy concurrent modification is not evidence** — see the meta-lesson below.
 
-**⚠ This is consistent with everything measured but is NOT proven.** Nobody has shown the head
-mismatch *causes* the degeneration.
+**What survives as fact** (measured, still true, just not causal): sec's MTP head *is*
+byte-identical to `qwen38-27b-uncensored-bf16` across all 15 tensors — a stock head on a
+security-finetuned body, because the `Qwen3_5ForConditionalGeneration` wrapper never loads the
+head, so the finetuning could not reach it. gen's orcarouter head *was* abliterated in-band by
+its author. Acceptance differs slightly (gen 58.4%, sec 55.9%). **All true. None of it shown to
+cause multi-turn degeneration.**
 
-## ⚠️ CONFOUNDED — what fixed sec is not yet known
+**Current standing explanation: the degeneration is an engine bug in the un-fixed vLLM.** Both
+production seats run `311b3513`, which is **172 commits behind GDN spec-decode fix #53077**
+(merged 2026-08-20). `#51113` is present in that build and is therefore **necessary but
+insufficient** on its own.
+
+## ⭐⭐ META-LESSON — n=1 during a busy session is not evidence
+
+The operator's own framing, and it generalises past this incident: **an observation made while
+many things are being changed at once cannot carry a causal claim, no matter how confidently it
+is reported.** Tonight that single observation became the load-bearing premise for a weights-side
+hypothesis, a root-cause narrative, and very nearly a recommendation.
+
+This is the same failure the gen-seat compose file already warns about in different words — *"a
+passing probe is NOT sufficient evidence"* — inverted. That note guards against trusting a
+**negative** result from a synthetic test. This one guards against trusting a **positive**
+sighting from an uncontrolled session. Both reduce to: **hold the system still, or do not draw
+causal conclusions from it.**
+
+Applies equally to the "coherent to 10k" observation below — same n, same conditions, opposite
+direction. Neither observation is worth more than the other.
+
+## ⚠️ CONFOUNDED — and the "before" state is itself unreliable
 
 sec now runs DFlash2 on a newer build and the operator reports **coherent to 10k tokens with
-adversarial nonsense prompts**, where it degenerated at 2k before. **Two variables changed at
-once:**
+adversarial nonsense prompts**. ⚠ Treat this the same way as the 2k sighting it is being compared
+against: **n=1, uncontrolled session, not evidence.** The comparison is weak on *both* ends.
+
+**Two variables changed at once:**
 
 1. **Engine**: `311b3513` → `e9d1398d`, **+259 commits, `behind_by=0`** (a strict superset),
    including GDN spec-decode fix **#53077** (merged 2026-08-20) that production is **172 commits

persistent-memory.md

@@ -123,7 +123,7 @@ _As of 2026-08-21 (late) — **the big AI-seat overhaul session; three seats set
 - **FLEET CLEANUP:** LFM2.5 retired; reranker consolidated (**nevermore had been silently failing 8 days** — pinned to the retired `granite-4.1-8b` alias; repointed to `summarizer`/`reranker`); `:8002`/`:8014`/granite service retired; A3 promoted to `stacks/vllm`.
 - ⏳ **OPEN:** file the upstream vLLM issue (operator); OWUI image-tag drift (`:main` vs pinned v0.11.0); Cold-Fusion NVFP4 quants delete/keep; Brokkr duplicate `reranker-a3-bge-v2-m3` alias; `/tank` DEGRADED **70+ days** (parked, operator→colo w/ cold spare); **MANY commits unpushed** (push is operator's call)._
 
-- **🟡 DFLASH2 SPEC-DECODE — MEASURED, and `sec` IS CURRENTLY RUNNING ON IT (2026-08-22, experimental).** `incoai/Qwen3.8-27B-DFlash2` (2B block-diffusion drafter, Apache-2.0) merged into vLLM as **#52816** on 2026-08-21; method string is **`"dflash"`**. **✅ MEASURED on our stack** (the card only claims stock BF16 on H200): works with an **abliterated + NVFP4 `compressed-tensors` target on Blackwell sm_120**. gen: MTP k=3 **2.753** tok/forward @ 114.9 tok/s → DFlash2 k=7 **3.254** @ 131.9. sec: **2.676** @ 110.5 → **3.252** @ 130.0. **⭐ THE k=7 MTP CONTROL INVERTED THE OBVIOUS READ — do not "just raise num_speculative_tokens":** MTP k=7 improves acceptance (3.041) but **collapses throughput to 74.0 tok/s**, because our head is a single module run autoregressively so k tokens cost k sequential passes. DFlash2 wins by making depth cheap, NOT by drafting better (our MTP is *better* at position 0, 79.6% vs 75.4%). **⭐ The drafter is model-agnostic — 3.254 vs 3.252 across two different finetunes (0.06%)** — so ONE weights file serves both seats, but it is **EAGLE3-style coupled** (`load_model(target_model)`, taps target hidden states at layers 5/19/33/47/61) so **VRAM is 3.85 GB PER SEAT, never shared**. **⚠️ `sec` NOW RUNS FROM A STANDALONE CONTAINER `vllm-sec-dflash2`, NOT its compose stack** (stopped, unmodified) — on `nightly-e9d1398d`, DFlash2 k=7, **480,000 ctx / 526,617 KV**, 2048² vision. **ROLLBACK = `docker rm -f vllm-sec-dflash2` + `compose up -d`.** Operator reports **coherent to 10k** where it degenerated at 2k — but **⚠️ CONFOUNDED: engine (+259 commits, incl. GDN fix #53077 that production is 172 behind) and drafter both changed; isolate by running MTP k=3 on the same build.** **#51113 is in BOTH builds — necessary but INSUFFICIENT.** 🔶 Hypotheses NOT proven: that sec's stock-graft MTP head (byte-identical to the uncensored base, vs gen's in-band-abliterated one) *causes* the degeneration; that NVFP4 explains our ~3.25 vs the card's 4.10–5.46. ❌ Wrong turns recorded: version strings ≠ lineage (use compare API `behind_by`), Docker Hub push timestamps ≠ source freshness (grep the image), and **"1M needs YaRN, absent from config" is FALSE** — YaRN is fully present on the sec quant (`factor 4.0`, orig 262144). Full arc + epistemic labels → `persistent-memory.d/2026-08-22-dflash2-spec-decode.md`
+- **🟡 DFLASH2 SPEC-DECODE — MEASURED, and `sec` IS CURRENTLY RUNNING ON IT (2026-08-22, experimental).** `incoai/Qwen3.8-27B-DFlash2` (2B block-diffusion drafter, Apache-2.0) merged into vLLM as **#52816** on 2026-08-21; method string is **`"dflash"`**. **✅ MEASURED on our stack** (the card only claims stock BF16 on H200): works with an **abliterated + NVFP4 `compressed-tensors` target on Blackwell sm_120**. gen: MTP k=3 **2.753** tok/forward @ 114.9 tok/s → DFlash2 k=7 **3.254** @ 131.9. sec: **2.676** @ 110.5 → **3.252** @ 130.0. **⭐ THE k=7 MTP CONTROL INVERTED THE OBVIOUS READ — do not "just raise num_speculative_tokens":** MTP k=7 improves acceptance (3.041) but **collapses throughput to 74.0 tok/s**, because our head is a single module run autoregressively so k tokens cost k sequential passes. DFlash2 wins by making depth cheap, NOT by drafting better (our MTP is *better* at position 0, 79.6% vs 75.4%). **⭐ The drafter is model-agnostic — 3.254 vs 3.252 across two different finetunes (0.06%)** — so ONE weights file serves both seats, but it is **EAGLE3-style coupled** (`load_model(target_model)`, taps target hidden states at layers 5/19/33/47/61) so **VRAM is 3.85 GB PER SEAT, never shared**. **⚠️ `sec` NOW RUNS FROM A STANDALONE CONTAINER `vllm-sec-dflash2`, NOT its compose stack** (stopped, unmodified) — on `nightly-e9d1398d`, DFlash2 k=7, **480,000 ctx / 526,617 KV**, 2048² vision. **ROLLBACK = `docker rm -f vllm-sec-dflash2` + `compose up -d`.** **⚠️ DEGENERATION: operator ruling 2026-08-22 — it lives in the UN-FIXED vLLM, not the weights. My "sec's stock-graft MTP head causes it" hypothesis is RETRACTED** (false dichotomy: an engine bug in BOTH seats is not exonerated by the seats being identical — it just can't explain a *difference*). **⭐⭐ AND THE UNDERLYING OBSERVATION IS NOT EVIDENCE: n=1 taken during a session with many concurrent changes.** Same caveat applies to the "coherent to 10k" sighting — same n, same conditions, opposite direction; the comparison is weak at BOTH ends. Production is **172 commits behind GDN spec-decode fix #53077**; **#51113 is in both builds — necessary but INSUFFICIENT.** Any real conclusion needs a held-still system: MTP k=3 on the new build, controlled. 🔶 Still unproven: that NVFP4 explains our ~3.25 vs the card's 4.10–5.46. ❌ Wrong turns recorded: version strings ≠ lineage (use compare API `behind_by`), Docker Hub push timestamps ≠ source freshness (grep the image), and **"1M needs YaRN, absent from config" is FALSE** — YaRN is fully present on the sec quant (`factor 4.0`, orig 262144). Full arc + epistemic labels → `persistent-memory.d/2026-08-22-dflash2-spec-decode.md`
 
 - **🟢 SPEACHES ASR — LIVE on irv-ml1 A6000 :8204 (2026-08-21, operator-approved).** OpenAI-compatible faster-whisper for **Eyra** (meeting recorder, `eyra-dev`). `Systran/faster-whisper-large-v3` + `Systran/faster-distil-whisper-large-v3`, fp16, both resident (`STT_MODEL_TTL=-1`), ~5.9 GB VRAM / 20 GB still free. **Deliberately co-exists with `parakeet` (:8765)** — parakeet is a TDT/transducer returning bare `{"text":…}` with **no `no_speech_prob` concept**, so it structurally cannot serve this consumer. **★ THE MEASURED FINDING IS WORTH MORE THAN THE DEPLOY: `no_speech_prob` ALONE IS A WEAK HALLUCINATION GATE.** Silence and pink room tone both produced the classic Whisper `"Thank you."` hallucination while `no_speech_prob` stayed **under 0.11** — a conventional `>0.6` threshold passes both through. `avg_logprob` separates ~6× better (−0.114 speech vs −0.650/−0.724 non-speech); `compression_ratio` 1.141 vs 0.556. **Consumers must gate on a composite.** (Synthetic inputs — shape of the separation, not calibration constants.) **VAD pinned OFF** (`_UNSTABLE_VAD_FILTER=False`) at consumer request — they VAD-gate upstream; consequence is the service will transcribe silence and does not defend itself. **★ IMAGE PINNED BY DIGEST, not `:latest-cuda`** — the VAD flag is an upstream-declared *unstable* var name, so a floating bump could rename it, restore VAD, and move gate semantics with **no error and no log line**; bumping = deliberate + re-run the checks. ⚠ Two gotchas: `PRELOAD_MODELS` **does not download** (only loads already-cached — use `POST /v1/models/{id}`), and the bind-mounted cache needs a `hub/` subdir or **every** `/v1/models` 500s `CacheNotFound` **while `/health` still returns 200**. ⚠ docker `device_ids: ["1"]` = A6000 (native CUDA inverts this — sees A6000 as `cuda:0`). ⏳ **NOT DONE by design: no LiteLLM gateway alias** — agreed sequencing was direct-first; adding it means editing the shared gateway + re-running fidelity. ⏳ Eyra's **diarization workload** (GPU pyannote, gated weights, own HF token) is a **future VRAM claim on this same A6000** — arrives at their diarize milestone. Canonical `stacks/speaches/`, commit `aa5863c`.