docs(quant): consolidate quantization lessons into a durable playbook

Quants are hard-fought and we keep re-paying for the same lessons. A survey
found quant knowledge scattered across 18 files in four trees, with three
documents having independently discovered and recorded overlapping
"landmines" sections — and one of them now actively misleading.

Adds docs/pfi/model-quantization-playbook.md as the single home for the
TRANSFERABLE lessons, with per-model artifacts demoted to worked examples
that link up to it. Contents:

- scheme decision table, incl. that a literal "W4A8" NVFP4 checkpoint is
  unservable on vLLM (two legal activation settings, FP8 is not one)
- the reference mixed-precision recipe and the three parts of it that are
  load-bearing and easy to drop
- the recurring landmines, ordered by cost: the loader-class trap
  (rediscovered THREE times), the three separate ways to lose the MTP head,
  toolchain deadlocks, vision configs, memory/device placement
- pipeline shape: prove targets before spending GPU time; mandatory
  post-steps that verify rather than assume
- the acceptance gate, and the three ways measurement has lied to us —
  prefix caching faking both speed metrics, prompt_logprobs going uniform
  under speculative decoding, and a 0600 .env making compose silently no-op
- hardware/co-residency, including that a SMALLER model can starve its
  neighbour because gpu-memory-utilization is a fraction of the whole card
- a superseded-claims table, and measured negatives not to re-chase

The superseded table earns its place immediately: the heretic2 runbook tells
readers to use modelopt because "compressed-tensors can't load the BF16 MTP
head, 0% acceptance". That symptom was real but the cause was not the format
-- it was the missing re:^mtp.* ignore entry. compressed-tensors gives
47.7-83.2% acceptance in production. A fresh session following that doc would
be sent down the modelopt path that current memory calls dependency hell, so
the runbook now carries a stale-warning header pointing here.

Wires discovery: an orientation.md "Where to look for what" row, pointers
from the gen-seat / heretic2 / mistral artifacts, and a CLAUDE.md maintenance
rule so the playbook gets fed instead of going stale -- model-agnostic
lessons land in the playbook, model-specific ones stay put, and a wrong
claim earns a dated superseded row rather than a silent edit.

Motivated by Qwen3.8 having just released: the next model swap will need a
requant, and this is what that session should read first.
This commit is contained in:
vh
2026-08-15 10:02:12 -07:00
parent b9da05aeb2
commit a91cc3fb38
6 changed files with 320 additions and 0 deletions
+3
View File
@@ -212,6 +212,9 @@ These caught us once; don't let them catch you twice.
| What's currently open / in-flight? | `STATUS.md` |
| What do I need to know that isn't in current code? | `MEMORY.md` + the `.md` files it links |
| Why did we do X? | Check memory files + `STATUS.md` session milestones at the bottom |
| **I need to quantize / requant a model** | **`docs/pfi/model-quantization-playbook.md` — READ IT FIRST.** Consolidated hard-won lessons (scheme choice, the recurring landmines, the acceptance gate, superseded claims). Per-model runbooks are worked examples, not the general guide. |
| What sampler/serve settings for model X? | `docs/pfi/recommended-model-settings.md` |
| Which model is on which GPU seat? | `servers/ana-ml2/README.md` + `stacks/<seat>/README.md` |
## Inventory + automation scripts