vh
982c319d9f
The fast char-rp-reasoning seat works: ~77 tok/s (vs GGUF ~59.5, base NVFP4 ~53), MTP draft-acceptance 32-40%, mean acceptance length 2.19. Same Heretic2/NEO-CODE model, NVFP4 + native qwen3_5_mtp spec-decode. Full end-to-end recipe + the four landmines in docs/runbooks/heretic2-nvfp4-mtp-seat.md: (1) load as AutoModelForImageTextToText not AutoModelForCausalLM (namespace/gibberish); (2) modelopt format not compressed-tensors (compressed-tensors MTP = 0% accept); (3) modelopt 0.45 <-> transformers 5.12.1 FusedMoE crash (guarded in quant_modelopt.py); (4) vLLM 0.24.0 does NOT propagate modelopt exclude_modules to the spec-decode draft model -> BF16 mtp head gets quantized -> shape crash; no checkpoint config fixes it (is_layer_skipped is exact-membership not glob) -> fix is a mounted sitecustomize that force-skips mtp.* in is_layer_skipped (upstream vLLM bug to report). Scripts: quant_modelopt.py (FusedMoE guard + single-shard export + multimodal load), finalize_modelopt_mtp.py (splice bf16 mtp), serve_modelopt_mtp.sh, run_quant_modelopt.sh, sitecustomize-mtp-workaround.py.
docs/
Navigation map for the documentation tree. New session? Read
orientation.md first — it's the narrative overview
of the fleet, backup architecture, governing principles, and gotchas,
and it points at everything else.
Tree
docs/
├── orientation.md # start here — fleet overview + where-to-look guide
├── runbooks/ # ops runbooks (recovery, deployment phases)
│ ├── disaster-recovery.md
│ ├── nh3-prune-ritual.md
│ └── pbs-deployment.md
└── pfi/ # PFI-specific reference (services, models, VMs)
├── docker-stack.md
├── model-list.md
├── proxmox-vms.md
├── recommended-model-settings.md
├── vm-102-matrix-appservice.md
└── vm-102-matrix-synapse.md
What goes where
runbooks/— step-by-step ops procedures. Anything you'd reach for during an incident or while standing up new infrastructure. Examples: disaster recovery (blast-radius tiers + restoration steps), PBS deployment (9-phase rollout). New runbook → new file here.pfi/— PFI-specific reference material that's too narrow for the top-level CLAUDE.md but doesn't change incident response. AI model inventory, recommended inference settings, Matrix bridge config, Proxmox VM map. New stable reference → new file here.- Top-level (
docs/orientation.md,docs/README.md) — narrative guides about the workspace itself, not about specific infra.
Cross-references
- Fleet topology + servers table: top-level
CLAUDE.md. - Open work + recent milestones: top-level
STATUS.md. - Durable cross-session facts:
~/.claude/projects/-home-lkraven-development-eshpfi-management/memory/.
Conventions
- Markdown, GitHub-flavored. CommonMark renders fine in most viewers.
- File names are lowercase-kebab-case, descriptive. No dates in filenames — git history covers that.
- One topic per file. If a file grows past ~500 lines, look for a natural split before adding more.
- No checked-in binaries or checksums. Build/release artifacts belong
in a build pipeline or
tools/, notdocs/.