vh
a91cc3fb38
Quants are hard-fought and we keep re-paying for the same lessons. A survey found quant knowledge scattered across 18 files in four trees, with three documents having independently discovered and recorded overlapping "landmines" sections — and one of them now actively misleading. Adds docs/pfi/model-quantization-playbook.md as the single home for the TRANSFERABLE lessons, with per-model artifacts demoted to worked examples that link up to it. Contents: - scheme decision table, incl. that a literal "W4A8" NVFP4 checkpoint is unservable on vLLM (two legal activation settings, FP8 is not one) - the reference mixed-precision recipe and the three parts of it that are load-bearing and easy to drop - the recurring landmines, ordered by cost: the loader-class trap (rediscovered THREE times), the three separate ways to lose the MTP head, toolchain deadlocks, vision configs, memory/device placement - pipeline shape: prove targets before spending GPU time; mandatory post-steps that verify rather than assume - the acceptance gate, and the three ways measurement has lied to us — prefix caching faking both speed metrics, prompt_logprobs going uniform under speculative decoding, and a 0600 .env making compose silently no-op - hardware/co-residency, including that a SMALLER model can starve its neighbour because gpu-memory-utilization is a fraction of the whole card - a superseded-claims table, and measured negatives not to re-chase The superseded table earns its place immediately: the heretic2 runbook tells readers to use modelopt because "compressed-tensors can't load the BF16 MTP head, 0% acceptance". That symptom was real but the cause was not the format -- it was the missing re:^mtp.* ignore entry. compressed-tensors gives 47.7-83.2% acceptance in production. A fresh session following that doc would be sent down the modelopt path that current memory calls dependency hell, so the runbook now carries a stale-warning header pointing here. Wires discovery: an orientation.md "Where to look for what" row, pointers from the gen-seat / heretic2 / mistral artifacts, and a CLAUDE.md maintenance rule so the playbook gets fed instead of going stale -- model-agnostic lessons land in the playbook, model-specific ones stay put, and a wrong claim earns a dated superseded row rather than a silent edit. Motivated by Qwen3.8 having just released: the next model swap will need a requant, and this is what that session should read first.
docs/
Navigation map for the documentation tree. New session? Read
orientation.md first — it's the narrative overview
of the fleet, backup architecture, governing principles, and gotchas,
and it points at everything else.
Tree
docs/
├── orientation.md # start here — fleet overview + where-to-look guide
├── runbooks/ # ops runbooks (recovery, deployment phases)
│ ├── disaster-recovery.md
│ ├── nh3-prune-ritual.md
│ └── pbs-deployment.md
└── pfi/ # PFI-specific reference (services, models, VMs)
├── docker-stack.md
├── model-list.md
├── proxmox-vms.md
├── recommended-model-settings.md
├── vm-102-matrix-appservice.md
└── vm-102-matrix-synapse.md
What goes where
runbooks/— step-by-step ops procedures. Anything you'd reach for during an incident or while standing up new infrastructure. Examples: disaster recovery (blast-radius tiers + restoration steps), PBS deployment (9-phase rollout). New runbook → new file here.pfi/— PFI-specific reference material that's too narrow for the top-level CLAUDE.md but doesn't change incident response. AI model inventory, recommended inference settings, Matrix bridge config, Proxmox VM map. New stable reference → new file here.- Top-level (
docs/orientation.md,docs/README.md) — narrative guides about the workspace itself, not about specific infra.
Cross-references
- Fleet topology + servers table: top-level
CLAUDE.md. - Open work + recent milestones: top-level
STATUS.md. - Durable cross-session facts:
~/.claude/projects/-home-lkraven-development-eshpfi-management/memory/.
Conventions
- Markdown, GitHub-flavored. CommonMark renders fine in most viewers.
- File names are lowercase-kebab-case, descriptive. No dates in filenames — git history covers that.
- One topic per file. If a file grows past ~500 lines, look for a natural split before adding more.
- No checked-in binaries or checksums. Build/release artifacts belong
in a build pipeline or
tools/, notdocs/.