esh-pfi-infrastructure/persistent-memory.d/2026-07-14-nvfp4-mtp-fast-char-rp-reasoning-seat-landed.md at 9b2c47602d2e89feee176945e28ded316162f6d3

Files

T

vh d710e56aca memory: migrate persistent-memory.md to two-tier index (53 detail files)

Split the 53 over-threshold dated log entries (Recent decisions, Tried
and abandoned) into per-entry persistent-memory.d/<slug>.md detail files,
leaving one-line pointers in the index; the 10 short entries stay inline.
Startup index drops 60,527 -> 25,256 bytes (492 -> 270 lines); entry
bodies move verbatim to on-demand detail files, so a fresh session loads
~25 KB instead of ~60 KB and pulls a detail file only when its pointer is
relevant. Top matter (Repo purpose, Tools & conventions, Current state)
is unchanged; both archival back-references preserved.

CLAUDE.md persistent-memory section now documents the index<->detail read
discipline (read the index, pull details on demand, never bulk-read the
dir, commit both together).

Auto-archival still held every dated entry back (all <30 days old); the
July burst begins aging past the 30-day guard ~2026-07-31.

2026-07-15 13:33:42 -07:00

1.3 KiB

Raw Blame History

[2026-07-14] NVFP4+MTP fast char-rp-reasoning seat LANDED + LIVE + gateway-repointed + VRAM-tuned. Modelopt-format re-quant made MTP work. The load-crash root cause = vLLM 0.24 does NOT propagate modelopt exclude_modules to the spec-decode DRAFT model → the bf16 mtp head gets quantized → shape crash; NO checkpoint config fixes it (is_layer_skipped is exact-membership, and the drafter never sees the mtp excludes) → workaround = a mounted sitecustomize that force-skips mtp.* in is_layer_skipped (upstream vLLM bug to file). Productionized as compose stack heretic2-charrp-reasoning (:8018, workaround baked in). Gateway char-rp-reasoning alias fixed: repointed off the stale GGUF served-name deckard-pkd-27b, added enable_thinking:true, dropped min_p (MTP-incompatible), canonical samplers temp1.0/top_p0.95/top_k20. Rebalanced GPU0 (gen 0.37→0.30/16-seq/256K + reasoning 0.39/16-seq/192K+MTP + char-rp 128K, 2.7GB free). All 4 gateway roles verified; vLLM reasoning-parser confirmed leak-free (unlike the GGUF budget-forcing). Full record + the 4 quant landmines in docs/runbooks/heretic2-nvfp4-mtp-seat.md; committed 982c319. Open (non-blocking): brokkr P00 (seat is live ahead of it), retire the stopped GGUF reasoning seat, file the vLLM bug.

1.3 KiB Raw Blame History

1.3 KiB

Raw Blame History