vh
d710e56aca
Split the 53 over-threshold dated log entries (Recent decisions, Tried and abandoned) into per-entry persistent-memory.d/<slug>.md detail files, leaving one-line pointers in the index; the 10 short entries stay inline. Startup index drops 60,527 -> 25,256 bytes (492 -> 270 lines); entry bodies move verbatim to on-demand detail files, so a fresh session loads ~25 KB instead of ~60 KB and pulls a detail file only when its pointer is relevant. Top matter (Repo purpose, Tools & conventions, Current state) is unchanged; both archival back-references preserved. CLAUDE.md persistent-memory section now documents the index<->detail read discipline (read the index, pull details on demand, never bulk-read the dir, commit both together). Auto-archival still held every dated entry back (all <30 days old); the July burst begins aging past the 30-day guard ~2026-07-31.
2 lines
944 B
Markdown
2 lines
944 B
Markdown
- `[2026-07-14]` **NVFP4 (llm-compressor / compressed-tensors) gives NO batch-1 speedup over GGUF for the Qwen3.5 GDN-hybrid, and its MTP is 0%-accept.** Measured base NVFP4 no-MTP ≈53 tok/s decode vs the GGUF NEO-CODE seat ~59.5 (llama.cpp wins single-stream; NVFP4's edge is concurrency, and this hybrid is bandwidth-bound at batch-1 with the BF16 linear_attn/GDN layers dominating). MTP spec-decode = 0% acceptance (vLLM's `Qwen3_5MTP` drafter won't load the bf16 mtp weights off a compressed-tensors main model → `Parameter … not found in params_dict`, `Avg Draft acceptance rate: 0.0%`). Pantheon is identical — its "working NVFP4+MTP" was working *structure*, never real acceleration. Working native MTP needs the **modelopt** main-model format (AEON, ~3.3/3 accept). LESSON: don't expect a faster single-stream seat from an llm-compressor NVFP4 quant of this arch; the MTP multiplier is the whole point and it requires modelopt.
|