revert(gen-seat): back to MTP-off known-good — APC-off mitigation failed in real use

The MTP-on + prefix-caching-off mitigation (63a3cb2) passed synthetic
7-turn probes but the operator still saw severe degeneration in real use.
A passing synthetic probe is NOT sufficient evidence -- it under-covers
real workloads (content distribution, conversation depth). Reverted to the
verified known-good: MTP off, prefix caching on (the 7bd38b3 state), ~half
decode speed but coherent. Operator is driving it to re-confirm.

Lesson reinforced (the recurring one this session): do not trust a
synthetic reproduction to VALIDATE a fix for a bug that only manifests in
the operator's real usage -- it validated a non-fix twice tonight.
This commit is contained in:
vh
2026-08-16 23:05:20 -07:00
parent 63a3cb2d86
commit 1f5b2cbcb0
+22 -22
View File
@@ -56,14 +56,11 @@ services:
- float32
- --kv-cache-dtype
- ${GEN_KV_CACHE_DTYPE:-fp8}
# ⚠️ PREFIX CACHING DISABLED 2026-08-17 as the MTP mitigation. vLLM V1
# defaults prefix caching ON, so the explicit --no- form is required to
# turn it off. With MTP re-enabled below, APC-off forces the GDN cache
# into a mode where the partial-accept align-path bug is inert
# (vllm#43559 / #47194): the corruption is gated on MTP × prefix-caching
# together. Trade vs the alternative (MTP off): this keeps MTP's ~2x decode
# and loses only prefix-cache reuse. GATED ON the multi-turn probe passing.
- --no-enable-prefix-caching
# Prefix caching ON (vLLM V1 default). The APC-off MTP mitigation was tried
# 2026-08-17 and did NOT hold — synthetic 7-turn probes passed but the
# operator still saw severe degeneration in real use, so it was reverted.
# See the MTP block below.
- --enable-prefix-caching
- --enable-chunked-prefill
- --limit-mm-per-prompt
- '{"image": 4}'
@@ -102,7 +99,14 @@ services:
- --enable-auto-tool-choice
- --tool-call-parser
- qwen3_coder
# MTP RE-ENABLED 2026-08-17, PAIRED WITH --no-enable-prefix-caching above.
# ⚠️ MTP / SPECULATIVE DECODING DISABLED — the verified known-good.
# Re-enable ONLY after real-use multi-turn testing, not a synthetic probe.
#
# History: MTP-off is clean. The APC-off mitigation (keep MTP, disable
# prefix caching) passed synthetic 7-turn probes on 2026-08-17 but the
# operator STILL saw severe degeneration in real use — so a passing probe
# is NOT sufficient evidence; reverted to MTP-off. The synthetic probes
# structurally under-cover real workloads (content distribution, depth).
#
# ROOT CAUSE (isolated 2026-08-16, operator-confirmed): `qwen3_5_mtp`
# speculative decoding CORRUPTS Qwen3.8-27B output in multi-turn
@@ -116,20 +120,16 @@ services:
# NOT depth-dependent (n=1 and n=2 both corrupt, so num_speculative_tokens=1
# is NOT a fix).
#
# THE MITIGATION (this config): the corruption is gated on MTP × prefix-
# caching TOGETHER (#43559/#47194). Disabling prefix caching forces the GDN
# cache into a mode where the buggy align-path is inert, so MTP can stay on.
# Keeps MTP's ~2x decode; costs only prefix-cache reuse. The proper upstream
# fix (#51113) is in vLLM main / v0.27.2rc0 only — not in a stable release
# as of this date, so we do not jump the fleet gateway to an RC.
# The APC-off mitigation (#43559/#47194) was supposed to make the GDN
# align-path bug inert while keeping MTP. It did not survive real use.
# The proper upstream fix (#51113) is in vLLM main / v0.27.2rc0 only — not
# in a stable release — so we do not jump the fleet gateway to an RC.
# Cost of MTP-off: ~half decode tok/s. Accepted as the known-good.
#
# ⚠️ GATED ON THE MULTI-TURN PROBE. If the 7-turn varied series
# (services/gen-seat-mixed-quant/bench/) still collapses past ~2k tokens
# with this config, APC-off did not hold on our stack — revert to MTP OFF
# (comment these two lines, restore --enable-prefix-caching) which is the
# verified known-good.
- --speculative-config
- '{"method": "${GEN_SPEC_METHOD:-qwen3_5_mtp}", "num_speculative_tokens": ${GEN_SPEC_TOKENS:-3}}'
# TO RE-ENABLE (only with real-use verification): uncomment the two lines
# below. Do NOT trust a synthetic probe alone — it passed and was wrong.
# - --speculative-config
# - '{"method": "${GEN_SPEC_METHOD:-qwen3_5_mtp}", "num_speculative_tokens": ${GEN_SPEC_TOKENS:-3}}'
deploy:
resources:
reservations: