Measured 2026-08-22 on one target with one instrument: raising MTP
num_speculative_tokens from 3 to 7 improved accepted length from 2.753 to
3.041 per forward pass while throughput fell from 114.9 to 74.0 tok/s.
Reporting acceptance alone would have recommended a 36% regression.
The cause is architectural rather than model-specific. A single-module MTP
head has no depth of its own, so vLLM runs it autoregressively and k draft
tokens cost k sequential forward passes. Past a shallow depth the drafting
cost exceeds what the extra accepted tokens save.
Records the comparison rule that follows: match k when comparing two
speculative methods, or the measurement is of depth rather than method. A
parallel-drafting drafter at k=7 against an autoregressive MTP at k=3 is not
a method comparison. In the case that produced this, the depth control
showed most of the apparent acceptance advantage was depth, while the
throughput advantage was real and came from parallel drafting -- our MTP was
better at position 0 and still lost overall.
Only the measured, model-agnostic result is recorded here. The
DFlash2-specific findings, the hypotheses that remain unproven, and the
wrong turns taken along the way live in
persistent-memory.d/2026-08-22-dflash2-spec-decode.md with explicit
epistemic labels, deliberately kept out of the playbook.
vh
ab3a0ca5bc