fix(gen-seat): diagnose the unterminated-<think> leak — model defect, temp-triggered

Operator reported the new Heretic-300 gen seat "sends CoT but never completes
the turn" through Lobe. Diagnosed; not yet fixed (the fix changes gen's
semantics, so it is the operator's call).

The Qwen3.8 chat template appends a pre-closed <think>\n\n</think>\n\n when
enable_thinking is false. The h300 model opens a fresh <think> anyway and never
closes it. Because the prompt already closed the block, vLLM's qwen3 reasoning
parser is not in reasoning state, so the tag passes through as ordinary text --
reasoning_content empty, reasoning_tokens 0, and the whole reasoning-plus-answer
blob lands in content. Lobe then correctly treats the unterminated tag as
still-thinking and renders no answer. The client and the serving stack are both
behaving correctly; the model is not.

The trigger is TEMPERATURE, not presence_penalty (n=12 per arm):

  temp 0.7, pp 1.5  (current gen)   4/12
  temp 0.7, pp 0.0                  4/12
  temp 0.7, pp 0.5                  3/12
  temp 0,   pp 1.5                  0/12

That falsifies the standing hypothesis, recorded in the litellm config comment
and in the operator's own 2026-08-16 note, that presence_penalty 1.5 is the
first dial to move. It is not this bug's cause.

It also explains the blast radius: only the two temp-0.7 aliases leak, `gen`
and `summarizer-large`. summarizer, classifier, image-judge and qwen-image-bench
all run at temp 0 and are clean, so nevermore's summarizer path is unaffected.

Candidate fix, validated n=30 over 4 prompt types plus a 3-turn conversation:
chat_template_kwargs {enable_thinking: true, reasoning_effort: low} takes 8/30
leaks to 0/30, at ~+27% completion tokens and a ~3% empty-content residual.

The tell appears in eval_coldfusion_h300.json and in none of the aeon, heresy,
mixed or w4a16 evals, so it is new with this build -- but L35 was never evaled,
so this does not separate a Cold-Fusion base trait from a Heretic-300
abliteration artifact.

Reproducers and the full method land in bench/think-leak/. Note in particular
that the 7/7 alias smoke test run at cutover structurally could not catch this:
trivial prompts never invite reasoning, so they never sample the leaking token.

persistent-memory.md

@@ -122,6 +122,14 @@ _As of 2026-08-20 23:30 — **the Heretic-300 session** (see the 🔴 entry abov
 
 - **🟢 HOMEPAGE — cleaned + themed (Australis Skyfall).** Three real defects fixed (UltraSeedbox on all tabs, Uptime Kuma double-rendered, fiction column counts), AI tab reordered by clickability, then themed from the operator's Skyfall handoff bundle with a background generated by **Arbo** (`t2i-ui-background`, job `13f0891f4e42`). ⚠ **After any recreate the tab bar/wallpaper/i18n vanish for up to ~an hour and then return on their own — do not chase it.** ⚠ CSS is served per-request: a theme change needs a **reload**, not a recreate, and candidate CSS can be injected live via Playwright for seconds-long iteration. → `persistent-memory.d/2026-08-19-homepage-skyfall-theme.md`
 
+- **🔴 GEN SEAT DEFECT 2026-08-21 — the h300 build emits an UNTERMINATED `<think>` into `content`, ~27% of the time, on any temp>0 alias. Operator-reported via Lobe ("sends CoT, never completes the turn").**
+  - **Mechanism.** With `enable_thinking:false` the chat template appends a **pre-closed** `<think>\n\n</think>\n\n` to the prompt (jinja L165-166). The h300 model **opens a fresh `<think>` anyway and never closes it** — verified raw: `has </think>: False`, `finish_reason: stop`, reasoning *and* answer in one `content` blob starting `<think>Ok, let's figure this out:`. vLLM's `qwen3` reasoning parser can't catch it: the prompt already closed the block, so the parser isn't in reasoning state and the tag is just text (`reasoning_content` empty, `reasoning_tokens: 0`). **The client is blameless** — Lobe correctly treats an unterminated `<think>` as still-thinking, so it renders an endless thought bubble and never shows the answer.
+  - **★ It is a SAMPLING event, and the trigger is TEMPERATURE — not presence_penalty.** n=12 per arm on the reproducer: `pp 1.5` → 4 leaks, `pp 0.0` → 4, `pp 0.5` → 3 (all the same), **`temperature 0` → 0**. ⚠️ **This FALSIFIES the standing "presence_penalty 1.5 is the first dial to move" hypothesis** recorded in the litellm config comment and by the operator 2026-08-16 — it is not this bug's cause. Leave that dial alone for this symptom.
+  - **Blast radius = exactly the two temp-0.7 aliases.** `gen` and `summarizer-large` leak (~17-27%); `summarizer`, `classifier`, `image-judge`, `qwen-image-bench` are all **temp=0 and clean at 0/12** — so **nevermore's `summarizer` path is NOT affected**. `gen-reasoning` doesn't leak (its think block is legitimately open) but shows the *other* symptom, empty `content`, at ~1/12.
+  - **New with this build, probably.** The `<think>`/"figure this out" tell appears in `bench/eval_coldfusion_h300.json` (x2) and in **none** of `eval_aeon`, `eval_heresy`, `eval_mixed`, `eval_w4a16`. ⚠ **L35 was never evaled**, so this does NOT separate "a Cold-Fusion base trait" from "a Heretic-300 abliteration artifact" — both are Cold-Fusion. Abliteration optimizes refusal count under a KL cap and has **no notion of format compliance**, so degrading the pre-closed-think-block convention is a plausible cost of the 8/100 result.
+  - **FIX, validated n=30 over 4 prompt types + a 3-turn conversation:** `chat_template_kwargs: {enable_thinking: true, reasoning_effort: low}` on `gen` → **0/30 leaks** (current config: **8/30**, worst on prose 4/6). Give the model a legitimately open `<think>` and it closes it properly, the parser does its job, `content` comes out clean. Cost ~+27% completion tokens (257 vs 202 avg) and a residual **1/30 empty-content**. Semantic change: `gen` stops being a non-thinking deployment — **operator's call, not applied.**
+  - **★ PROCESS LESSON: the 7/7 alias smoke test structurally CANNOT catch this.** Trivial prompts ("Reply with exactly: OK-gen") never invite reasoning, so they never sample the leaking token. Same shape as the compose file's own warning that single-turn probes missed the xhigh budget bug. **Probe with a reasoning-inviting prompt at n≥12, and grep the raw `content` for `<think>` — never just check HTTP 200.** The tell was sitting in my own `eval_coldfusion_h300.json` output the night of the cutover and I read past it.
+
 - **🟢 GEN SEAT — LIVE = COLD-FUSION HERETIC-300 (cut over 2026-08-20 23:05).** `GEN_MODEL=/tank/aimodels/qwen38-27b-coldfusion-h300-nvfp4-mixed` on `gen-seat`/`vllm-gen`, ana-ml2 GPU0 `:8015`, served-name unchanged (`qwen3.8-27b-uncensored` / `-thinking`) so all 7 LiteLLM aliases route without a gateway edit. **Verified end to end:** healthy in 5.5 min; KV **401,550 tok / 1.53×** (baseline 403k/1.54× — within noise); **7/7 aliases green** through LiteLLM; **VISION INTACT** (correctly enumerated colour/form/position of 3 shapes — the surface that had never been exercised after abliteration → MTP-dropping export → graft → quant); **MTP acceptance 59.7% median @ 118.37 tok/s** (`bench/mtp_coldfusion_h300.json`) — statistically identical to L35's **59.1% @ 118.71** on the same instrument, so **the roadmap's "~47% for a pristine graft" prediction was WRONG — a pristine graft accepts as well as an in-band one.** ⚠ A single long-prose sample read 47.5%; the 8-run spread is 49.0–65.4%, so **one sample cannot characterize acceptance** — always use `quickbench.py`. Deterministic quality gens all correct (heat-pump, primes=77, 14:20→17:05, prose); abliteration survival **4/4 compliance**. **⏳ PPL NOT MEASURED** — `eval_quality.py` aborts with "prompt_logprobs look uniform" because the seat runs `--speculative-config`; the documented workaround is a spec-decode-free probe seat (`bench/serve_probe.sh`, :8017), and **there is no VRAM for one** (GPU0 5.2 GB free, GPU1 2.4 GB free). Comparison target = heresy's **6.910 mean / 5.625 median**. **ROLLBACK (one line):** `sudo cp /opt/docker/compose/gen-seat/.env.bak-pre-h300-20260820 /opt/docker/compose/gen-seat/.env && cd /opt/docker/compose/gen-seat && sudo docker compose up -d vllm-gen` → back to `qwen38-27b-coldfusion-L35-nvfp4-mixed`. **Do NOT delete** `-L35-nvfp4-mixed` or `qwen38-27b-heresy-nvfp4-mixed`. ⚠ **The self-harm guardrail is GONE on this build** (operator's own next work item). Also normalized the quant dir from root:0600 to `llmuser:llmuser` 0664 to match every other model dir. **Confirmed the right weights are mounted on TWO discriminating views** — mtime and a 64 MB head-hash both match h300 and differ from L35; `config.json` sha256 is **identical** across both builds and therefore useless as a discriminator (it carries no weight-specific content — don't reach for it again).
 
 - **⚪ PRIOR GEN SEAT — `absolute-heresy` 2026-08-17 (validated, promoted; superseded by L35 then h300 on 2026-08-20).** Live gen = `/tank/aimodels/qwen38-27b-heresy-nvfp4-mixed` — **MuXodious/Qwen3.8-27B-absolute-heresy** (Heretic v1.4.0 + **SOMPOA**, trial T377, pin `c2374593`) put through our own mixed NVFP4+FP8 recipe. Chosen because it beats the incumbent on **both** axes at once: author refusals 2/101 vs 12/100, first-token KL 0.0759 vs 0.1191. **Gate (probe :8017, pinned nightly, seat-matched flags): MTP 47.2% (inc. 48.2%), decode 103.5 tok/s (96.4), prefill 6618/5403 @6.7k/27k (6334/5085), PPL 6.910 (7.059 — 2.1% BETTER), surface 6/6, abliteration 4/4, and 0/55 refusals on our battery-instruct arm with ZERO EMPTY (no catatonia).** ⚠ speed deltas are **image-confounded** (probe on the pinned nightly, incumbent numbers from an earlier image) — read as "not worse", not a clean win. All 7 LiteLLM aliases verified end-to-end; GPU0 at 91.3/97.9 GB with meromero healthy (more headroom than the old build's 96.8). ⚠ **RC1, 2 days old, ~348 downloads.** **Operator-confirmed "working very well" in real use 2026-08-17**, same evening as the cutover — the signal the synthetic gates structurally cannot give (multi-turn degeneration is stochastic; four synthetic tests once validated three non-fixes). Not yet the 60k-token bar the prior seat cleared, so **keep watching and do NOT delete the rollback weights yet**. **ROLLBACK:** `sudo cp /opt/docker/compose/gen-seat/.env.bak-heresy-20260817 /opt/docker/compose/gen-seat/.env && cd /opt/docker/compose/gen-seat && sudo docker compose up -d vllm-gen`; incumbent weights UNTOUCHED at `qwen38-27b-uncensored-nvfp4-mixed` — **do NOT delete** until this holds. Runbook `services/gen-seat-mixed-quant/RUNBOOK-heresy-swap.md`.