From 7bd38b33b5c759344cd26bb1340c8d9a936ad941 Mon Sep 17 00:00:00 2001
From: Vuong Hoang <vh@phasefinal.com>
Date: Sun, 16 Aug 2026 22:22:58 -0700
Subject: [PATCH] =?UTF-8?q?fix(gen-seat):=20disable=20MTP=20=E2=80=94=20qw?=
 =?UTF-8?q?en3=5F5=5Fmtp=20corrupts=20Qwen3.8-27B=20multi-turn=20past=20~2?=
 =?UTF-8?q?k=20tokens?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Root cause of the long-hunted 'gen goes degenerate in conversation',
isolated 2026-08-16 and operator-confirmed. qwen3_5_mtp speculative
decoding corrupts Qwen3.8-27B output once cumulative multi-turn context
passes ~2,000 tokens: the draft head's bad tokens get accepted and the
reply degenerates into CONTEXT-BLEEDING (a 'describe durian' answer that
contained the Krebs-cycle and winter replies from earlier turns), then
collapses to a few words.

Isolation, each step measured on the varied 7-turn probe:
  - not the gateway (identical input -> gateway == direct; echo intact)
  - not presence_penalty (1.5/0.5/0.0 all collapse), not temperature
    (1.0 collapses harder), not repetition (varied unrelated topics
    collapse identically -> it is context length, not template-lock)
  - model-INDEPENDENT across all three Qwen3.8-27B quants we serve
    (AEON W4A4, unsloth FP8-attn, in-house mixed)
  - Qwen3.6 (char-rp-reasoning) and Gemma-4 (char-rp) are CLEAN
  - DECISIVE: same Qwen3.8 model + same conversation, MTP OFF -> coherent
    through 4k+ tokens, no bleed. MTP is the cause.

Qwen3.6 runs the same qwen3_5_mtp method and is clean, so the 3.6 MTP
head/graft is fine and the 3.8 one is not (suspects: the bf16 MTP graft,
or spec depth 3). COST: ~half decode tok/s without spec decoding.
Accepted as known-good until the 3.8 MTP is fixed; first thing to try on
re-enable is num_speculative_tokens=1. Seat restored to AEON W4A4 (the
production choice); verified clean on the varied series after this change.
---
 stacks/gen-seat/compose.yaml | 30 ++++++++++++++++++++++++++++--
 1 file changed, 28 insertions(+), 2 deletions(-)

diff --git a/stacks/gen-seat/compose.yaml b/stacks/gen-seat/compose.yaml
index cf248e4..fa060e0 100644
--- a/stacks/gen-seat/compose.yaml
+++ b/stacks/gen-seat/compose.yaml
@@ -95,8 +95,34 @@ services:
       - --enable-auto-tool-choice
       - --tool-call-parser
       - qwen3_coder
-      - --speculative-config
-      - '{"method": "${GEN_SPEC_METHOD:-qwen3_5_mtp}", "num_speculative_tokens": ${GEN_SPEC_TOKENS:-3}}'
+      # ⚠️ MTP / SPECULATIVE DECODING DISABLED 2026-08-17 — DO NOT re-enable
+      # without re-testing multi-turn coherence past ~2k tokens.
+      #
+      # ROOT CAUSE (isolated 2026-08-16, operator-confirmed): `qwen3_5_mtp`
+      # speculative decoding CORRUPTS Qwen3.8-27B output in multi-turn
+      # conversations once cumulative context passes ~2,000 tokens. The draft
+      # head emits bad tokens that get accepted, and the response degenerates
+      # into CONTEXT-BLEEDING — stitching prior turns' content into the current
+      # answer (a "describe durian" reply that contained the Krebs-cycle and
+      # winter answers from earlier turns), then collapsing to a few words.
+      #
+      # Proven model-INDEPENDENT across every Qwen3.8-27B quant we serve (AEON
+      # W4A4, unsloth FP8-attn, our in-house mixed) and FIXED by disabling MTP:
+      # the identical model stays coherent through 4k+ tokens with this off.
+      # It is MTP-on-Qwen3.8 specifically — Qwen3.6 (char-rp-reasoning) runs the
+      # same qwen3_5_mtp method and is CLEAN, so the 3.6 MTP head/graft is fine
+      # and the 3.8 one is not (suspects: the bf16 MTP graft, or spec depth 3).
+      #
+      # COST: no speculative decoding = ~half decode tok/s. Accepted as the
+      # known-good until the 3.8 MTP is fixed.
+      #
+      # BEFORE RE-ENABLING: run the multi-turn probe in
+      # services/gen-seat-mixed-quant/bench/ (varied 7-turn series, watch t5-t7
+      # past 2k tokens for the collapse). First thing worth trying is
+      # num_speculative_tokens=1 (shallowest draft, least divergence). Restore by
+      # uncommenting the two lines below.
+      # - --speculative-config
+      # - '{"method": "${GEN_SPEC_METHOD:-qwen3_5_mtp}", "num_speculative_tokens": ${GEN_SPEC_TOKENS:-3}}'
     deploy:
       resources:
         reservations: