vh
cd941f7bf9
Pre-clear snapshot. Rewrote Current state / in-flight to the post-rebalance topology (cyberprev is the sec seat, gen-small stood up, char/coder retuned, all generative seats native 262K in-band, catalog+bench shipped) and added a Recent decisions index entry + detail file 2026-09-14-fv-seat-rebalance-gen-small.md. Also commits two previously-untracked durable artifacts (fv-to-ana-nat runbook + its memory detail) so memory does not lag the work. Archival: file is over the ~300-line soft cap (377) but 0 entries archived — the 4 age-eligible (>14d) entries all carry open deferred-work pointers (guarded), and all other entries are <14 days old. Reported per the archival discipline.
61 lines
4.3 KiB
Markdown
61 lines
4.3 KiB
Markdown
# 2026-09-14 — fv-ml1 seat rebalance: cyberprev→sec, gen-small stood up, all seats to native 262K
|
|
|
|
Second big fv-ml1 session on 2026-09-14 (after the all-night reorg + orca blocker in
|
|
[[2026-09-14-fv-seat-reorg-and-orca-blocker]]). Everything here is committed; regenerate
|
|
the live view with `scripts/seat-inventory.py` and the durable catalog is
|
|
`docs/pfi/llm-seat-catalog.md`.
|
|
|
|
## What changed (all committed)
|
|
|
|
1. **cyberprev quantized and promoted to the `sec` seat.** `hotdogs/Qwen3.8-27B-abliterated-cyber-preview`
|
|
(a cyber offensive-security **tool-calling SFT finetune** on an abliterated base — NOT
|
|
"an abliteration"; trained on `hotdogs/cyber-sft-agent-qwen38`, 8,400 rows / 22 tools).
|
|
⚠ The raw hotdogs export ships **broken**: 850/1199 body tensors carry a **tripled
|
|
`language_model.` prefix** (unsloth export bug, verified in the upstream HF index) —
|
|
the naive quant passed every structural check and served pure gibberish. Fix:
|
|
name-repair the source against a known-good reference (exact 1199-tensor match), then
|
|
house mixed-NVFP4 quant → `cyberprev-nvfp4-mixed-v2`. **A coherence smoke is the gate;
|
|
structural verification cannot substitute** (see [[reference_unsloth_tripled_prefix_export_bug]]).
|
|
Commits 1418edb/b94d173/196416f.
|
|
2. **mog-sec RETIRED; cyberprev is now `sec`/`sec-reasoning`.** Gateway repointed (commit
|
|
207798d). `sec` is a ROLE alias, so this is a promotion, not silent substitution.
|
|
⚠ mog-sec was the fleet's ONLY offense+**defense**/blue-team seat — the sec role is now
|
|
offense-only. mog-sec stack files retained for a one-command restore. Its "persona on
|
|
stock" description was ALSO wrong — M.O.G.-SEC is itself an offense+defense SFT finetune
|
|
(commit 3906c68). All three sec candidates are Qwen3.8-27B finetunes differing by focus.
|
|
3. **gen-small stood up (NEW seat).** `llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-Preserved-NVFP4-Experts-Only`
|
|
— already on disk at `/tank/aimodels/qwen36-35b-a3b-heretic-nvfp4` (no download). Heretic
|
|
v1.3.0 (MPOA), 88% fewer refusals @ 0.0015 KL, 19 MTP. GPU0 :8026, MTP k=3 **measured
|
|
69.6% accept / 3.09 len**. 3.8 was requested but no general Qwen3.8 A3B exists → 3.6 is
|
|
the fallback. Backs `summarizer` + `classifier`; new `classifier-large` → flash (accuracy
|
|
tier); `summarizer-large` stays on flash. Commit 05f3d3b.
|
|
4. **All sec/gen/char seats raised to native 262,144 with concurrency in the 1.2-2.5x band**
|
|
(operator target). coder: 8192@13.75x → 16384@4.70x (waste reclaimed; ~4.7x overshoots
|
|
2-3x because the 1.5B weight floor sits under the util knob's resolution — a KV-byte pin
|
|
would fix it, deferred). char-rp bumped 1.02→1.22x (util 0.40→0.52); GPU1 hit its ceiling
|
|
so char-rp-fast's util cap was trimmed (its KV is pinned, so it held 2.04x) — an in-GPU
|
|
rebalance rather than a utility-seat move.
|
|
5. **Durable docs:** `docs/pfi/llm-seat-catalog.md` (curated lineage/provenance/cards/speed)
|
|
+ `scripts/seat-bench.py` (reproducible warm-tok/s + depth-OOM bench). Commit dfa91a8.
|
|
|
|
## Measured — serial depth + warm tok/s (2026-09-14, warm/conc=1/greedy/n=3)
|
|
|
|
| seat | GPU | ctx | conc | warm tok/s | VRAM | depth clean to | OOM |
|
|
|---|---|---|---|---|---|---|---|
|
|
| cyberprev (sec) | 0 | 262144 | 1.37x | 136.6 | 47.1 GiB | 259,722 tok | none |
|
|
| gen-small | 0 | 262144 | 2.56x | 254.8 | 36.1 GiB | 254,526 tok | none |
|
|
| gen (flash-next) | 2 | 262144 | 1.31x | 170.7 | 95.3 GiB | 254,273 tok | none |
|
|
| char-rp | 1 | 262144 | 1.22x | 62.7 | 42.3 GiB | 254,858 tok | none |
|
|
| char-rp-fast | 1 | 262144 | 2.04x | 225.2 | 27.0 GiB | 254,823 tok | none |
|
|
| coder | 1 | 16384 | 4.70x | 337.3 | 5.6 GiB | 15,905 tok | none |
|
|
|
|
Every seat prefilled near max context with **zero OOM/CUBLAS/illegal-memory** (allocator-log
|
|
verified). Numbers are uncontended single-stream ceilings — re-derive with `scripts/seat-bench.py`.
|
|
|
|
## Open / deferred
|
|
|
|
- **mog-sec fate** — retired; restore for blue-team coverage vs leave retired. Operator's call.
|
|
- **coder ≤3x** — needs a `--kv-cache-memory` byte pin (compose change) vs the fragile util floor.
|
|
- **gen-small 2.56x** — a hair over the 2.5x band top; trimmable, left as useful headroom for the bulk seat.
|
|
- **mog-sec / sentinel-r3 stacks** retained on disk for restore; sentinel gateway aliases deleted (they 404).
|