memory: DFlash2 spec-decode measured; sec running on it (experimental, confounded)

Records the 2026-08-22 session with measured results, hypotheses, and wrong
turns explicitly separated -- the operator held this back while it was in
flight specifically so conjecture would not enter the record as fact.

MEASURED. DFlash2 works on an abliterated + NVFP4 compressed-tensors target
on Blackwell sm_120, which the model card does not claim (it tests stock
BF16 on H200). gen 2.753 -> 3.254 accepted tok/forward and 114.9 -> 131.9
tok/s; sec 2.676 -> 3.252 and 110.5 -> 130.0. The drafter is model-agnostic
across two different finetunes to 0.06%, but is EAGLE3-style coupled to its
target's hidden states, so the weights file is shareable while the 3.85 GB
of VRAM is per-seat.

The k=7 MTP control is the load-bearing result: raising MTP depth improves
acceptance and collapses throughput to 74.0 tok/s, because a single-module
head run autoregressively costs one forward pass per draft token. Without
that control the obvious recommendation would have been wrong.

CONFOUNDED. sec no longer degenerates at 2k, but the engine advanced 259
commits and the drafter changed at the same time. Isolating it means running
MTP k=3 on the new build. Also recorded: #51113 is present in both builds
and is therefore necessary but insufficient, since sec ran it and still
degenerated.

HYPOTHESES, labelled as such: that sec's stock-graft MTP head causes the
degeneration, and that NVFP4 explains the gap against published acceptance
figures. Neither is proven.

WRONG TURNS, recorded so they are not re-derived: version strings are not
lineage, Docker Hub push timestamps are not source freshness, and the claim
that 1M context needs YaRN absent from config is false for the sec quant.

Operationally important: sec is serving from a standalone container rather
than its compose stack, which is stopped but unmodified. Rollback is two
commands and is written down.

persistent-memory.md

@@ -123,6 +123,8 @@ _As of 2026-08-21 (late) — **the big AI-seat overhaul session; three seats set
 - **FLEET CLEANUP:** LFM2.5 retired; reranker consolidated (**nevermore had been silently failing 8 days** — pinned to the retired `granite-4.1-8b` alias; repointed to `summarizer`/`reranker`); `:8002`/`:8014`/granite service retired; A3 promoted to `stacks/vllm`.
 - ⏳ **OPEN:** file the upstream vLLM issue (operator); OWUI image-tag drift (`:main` vs pinned v0.11.0); Cold-Fusion NVFP4 quants delete/keep; Brokkr duplicate `reranker-a3-bge-v2-m3` alias; `/tank` DEGRADED **70+ days** (parked, operator→colo w/ cold spare); **MANY commits unpushed** (push is operator's call)._
 
+- **🟡 DFLASH2 SPEC-DECODE — MEASURED, and `sec` IS CURRENTLY RUNNING ON IT (2026-08-22, experimental).** `incoai/Qwen3.8-27B-DFlash2` (2B block-diffusion drafter, Apache-2.0) merged into vLLM as **#52816** on 2026-08-21; method string is **`"dflash"`**. **✅ MEASURED on our stack** (the card only claims stock BF16 on H200): works with an **abliterated + NVFP4 `compressed-tensors` target on Blackwell sm_120**. gen: MTP k=3 **2.753** tok/forward @ 114.9 tok/s → DFlash2 k=7 **3.254** @ 131.9. sec: **2.676** @ 110.5 → **3.252** @ 130.0. **⭐ THE k=7 MTP CONTROL INVERTED THE OBVIOUS READ — do not "just raise num_speculative_tokens":** MTP k=7 improves acceptance (3.041) but **collapses throughput to 74.0 tok/s**, because our head is a single module run autoregressively so k tokens cost k sequential passes. DFlash2 wins by making depth cheap, NOT by drafting better (our MTP is *better* at position 0, 79.6% vs 75.4%). **⭐ The drafter is model-agnostic — 3.254 vs 3.252 across two different finetunes (0.06%)** — so ONE weights file serves both seats, but it is **EAGLE3-style coupled** (`load_model(target_model)`, taps target hidden states at layers 5/19/33/47/61) so **VRAM is 3.85 GB PER SEAT, never shared**. **⚠️ `sec` NOW RUNS FROM A STANDALONE CONTAINER `vllm-sec-dflash2`, NOT its compose stack** (stopped, unmodified) — on `nightly-e9d1398d`, DFlash2 k=7, **480,000 ctx / 526,617 KV**, 2048² vision. **ROLLBACK = `docker rm -f vllm-sec-dflash2` + `compose up -d`.** Operator reports **coherent to 10k** where it degenerated at 2k — but **⚠️ CONFOUNDED: engine (+259 commits, incl. GDN fix #53077 that production is 172 behind) and drafter both changed; isolate by running MTP k=3 on the same build.** **#51113 is in BOTH builds — necessary but INSUFFICIENT.** 🔶 Hypotheses NOT proven: that sec's stock-graft MTP head (byte-identical to the uncensored base, vs gen's in-band-abliterated one) *causes* the degeneration; that NVFP4 explains our ~3.25 vs the card's 4.10–5.46. ❌ Wrong turns recorded: version strings ≠ lineage (use compare API `behind_by`), Docker Hub push timestamps ≠ source freshness (grep the image), and **"1M needs YaRN, absent from config" is FALSE** — YaRN is fully present on the sec quant (`factor 4.0`, orig 262144). Full arc + epistemic labels → `persistent-memory.d/2026-08-22-dflash2-spec-decode.md`
+
 - **🟢 SPEACHES ASR — LIVE on irv-ml1 A6000 :8204 (2026-08-21, operator-approved).** OpenAI-compatible faster-whisper for **Eyra** (meeting recorder, `eyra-dev`). `Systran/faster-whisper-large-v3` + `Systran/faster-distil-whisper-large-v3`, fp16, both resident (`STT_MODEL_TTL=-1`), ~5.9 GB VRAM / 20 GB still free. **Deliberately co-exists with `parakeet` (:8765)** — parakeet is a TDT/transducer returning bare `{"text":…}` with **no `no_speech_prob` concept**, so it structurally cannot serve this consumer. **★ THE MEASURED FINDING IS WORTH MORE THAN THE DEPLOY: `no_speech_prob` ALONE IS A WEAK HALLUCINATION GATE.** Silence and pink room tone both produced the classic Whisper `"Thank you."` hallucination while `no_speech_prob` stayed **under 0.11** — a conventional `>0.6` threshold passes both through. `avg_logprob` separates ~6× better (−0.114 speech vs −0.650/−0.724 non-speech); `compression_ratio` 1.141 vs 0.556. **Consumers must gate on a composite.** (Synthetic inputs — shape of the separation, not calibration constants.) **VAD pinned OFF** (`_UNSTABLE_VAD_FILTER=False`) at consumer request — they VAD-gate upstream; consequence is the service will transcribe silence and does not defend itself. **★ IMAGE PINNED BY DIGEST, not `:latest-cuda`** — the VAD flag is an upstream-declared *unstable* var name, so a floating bump could rename it, restore VAD, and move gate semantics with **no error and no log line**; bumping = deliberate + re-run the checks. ⚠ Two gotchas: `PRELOAD_MODELS` **does not download** (only loads already-cached — use `POST /v1/models/{id}`), and the bind-mounted cache needs a `hub/` subdir or **every** `/v1/models` 500s `CacheNotFound` **while `/health` still returns 200**. ⚠ docker `device_ids: ["1"]` = A6000 (native CUDA inverts this — sees A6000 as `cuda:0`). ⏳ **NOT DONE by design: no LiteLLM gateway alias** — agreed sequencing was direct-first; adding it means editing the shared gateway + re-running fidelity. ⏳ Eyra's **diarization workload** (GPU pyannote, gated weights, own HF token) is a **future VRAM claim on this same A6000** — arrives at their diarize milestone. Canonical `stacks/speaches/`, commit `aa5863c`.
 
 - **🟢 ESH INTERNAL IPv6 — LIVE on two LANs 2026-08-21 (operator-directed), and the CGNAT static is a CARRIER problem.** Enabled `ipv6_interface_type: none → pd` + `ipv6_pd_interface: wan` on **`esh-userland`** (VLAN 10, SSID `PVC`) and **`esh-server`** (10.0.50.0/24) on ESH-UDMPM. Cityside delegates **`2607:73c0:402:1d00::/56`**; per-VLAN /64s userland `:1d00`/`:1d01`, esh-server `:1d02`. **Verified: `esh-docker-vm` egresses over v6 as its OWN address (`2607:73c0:402:1d02:be24:11ff:fe13:3a8`) — no NAT** — vs v4 which egresses as the shared CGNAT pool `154.50.58.1`. Zero disruption through both writes (all subsystems ok, 54 wireless clients, Open WebUI 200, DNS resolving). Originals backed up. **⚠ NH3 stays v6-OFF deliberately — its single `/64` is RESERVED FOR NETWORK MESHING** (operator). **★ THE v4 STATIC IS NOT PROVISIONED — proven, stop testing it.** A full gateway **reboot** (verified by observed down-transition + uptime reset 140,846s→183s) forced a fresh DHCP **DISCOVER** and Cityside returned the *identical* `100.104.3.250`. Also tried: physical cable pull (missed — WAN is **`eth8` = Port 9**, 2.5GE copper; ports 1–7 have **never carried a byte**), `renew-dhcp` (**not a valid devmgr verb**), `force-provision` (no WAN effect). **A DHCP *renew* could never have worked anyway — renewal PRESERVES the lease by design; only release/rebind or link-down forces a new DISCOVER.** Next step is a **Cityside ticket**: is the static applied or merely billed, and will it arrive as a DHCP reservation (a bounce picks it up) or a manual assignment (needs IP/mask/gw → `wan_type=static`)? **v6 firewall AUDITED and CORRECT** — default-deny WAN→internal for BOTH versions, only 4 hand-written policies (all IoT-scoped, none WAN-facing); my earlier "v6 strips CGNAT's accidental shielding" warning was **overstated and is retracted**. ⚠ audit was config-based, **never probed from outside** (no v6 vantage exists — NH3 has none by design). Flat-zone lateral-movement finding **parked, id 44** `esh-split-cameras-mgmt-out-of-the-flat-internal`. Cleanup: the dead `Cable Internet` UCI modem (last seen 08-18, replaced by Cityside; failover is the 5G on WAN2) was **deleted from the controller**, which cleared the lingering `wlan`/`lan` **error** states to ok. `vpn: error` persists and predates all of this. → auto-memory `reference_fleet_vpn_topology_ipv6`