vh
ab3a0ca5bc
Measured 2026-08-22 on one target with one instrument: raising MTP num_speculative_tokens from 3 to 7 improved accepted length from 2.753 to 3.041 per forward pass while throughput fell from 114.9 to 74.0 tok/s. Reporting acceptance alone would have recommended a 36% regression. The cause is architectural rather than model-specific. A single-module MTP head has no depth of its own, so vLLM runs it autoregressively and k draft tokens cost k sequential forward passes. Past a shallow depth the drafting cost exceeds what the extra accepted tokens save. Records the comparison rule that follows: match k when comparing two speculative methods, or the measurement is of depth rather than method. A parallel-drafting drafter at k=7 against an autoregressive MTP at k=3 is not a method comparison. In the case that produced this, the depth control showed most of the apparent acceptance advantage was depth, while the throughput advantage was real and came from parallel drafting -- our MTP was better at position 0 and still lost overall. Only the measured, model-agnostic result is recorded here. The DFlash2-specific findings, the hypotheses that remain unproven, and the wrong turns taken along the way live in persistent-memory.d/2026-08-22-dflash2-spec-decode.md with explicit epistemic labels, deliberately kept out of the playbook.
docs/
Navigation map for the documentation tree. New session? Read
orientation.md first — it's the narrative overview
of the fleet, backup architecture, governing principles, and gotchas,
and it points at everything else.
Tree
docs/
├── orientation.md # start here — fleet overview + where-to-look guide
├── runbooks/ # ops runbooks (recovery, deployment phases)
│ ├── disaster-recovery.md
│ ├── nh3-prune-ritual.md
│ └── pbs-deployment.md
└── pfi/ # PFI-specific reference (services, models, VMs)
├── docker-stack.md
├── model-list.md
├── proxmox-vms.md
├── recommended-model-settings.md
├── vm-102-matrix-appservice.md
└── vm-102-matrix-synapse.md
What goes where
runbooks/— step-by-step ops procedures. Anything you'd reach for during an incident or while standing up new infrastructure. Examples: disaster recovery (blast-radius tiers + restoration steps), PBS deployment (9-phase rollout). New runbook → new file here.pfi/— PFI-specific reference material that's too narrow for the top-level CLAUDE.md but doesn't change incident response. AI model inventory, recommended inference settings, Matrix bridge config, Proxmox VM map. New stable reference → new file here.- Top-level (
docs/orientation.md,docs/README.md) — narrative guides about the workspace itself, not about specific infra.
Cross-references
- Fleet topology + servers table: top-level
CLAUDE.md. - Open work + recent milestones: top-level
STATUS.md. - Durable cross-session facts:
~/.claude/projects/-home-lkraven-development-eshpfi-management/memory/.
Conventions
- Markdown, GitHub-flavored. CommonMark renders fine in most viewers.
- File names are lowercase-kebab-case, descriptive. No dates in filenames — git history covers that.
- One topic per file. If a file grows past ~500 lines, look for a natural split before adding more.
- No checked-in binaries or checksums. Build/release artifacts belong
in a build pipeline or
tools/, notdocs/.