memory: snapshot — fv-ml1 rebalance (cyberprev→sec, gen-small A3B, all seats native 262K)
Pre-clear snapshot. Rewrote Current state / in-flight to the post-rebalance topology (cyberprev is the sec seat, gen-small stood up, char/coder retuned, all generative seats native 262K in-band, catalog+bench shipped) and added a Recent decisions index entry + detail file 2026-09-14-fv-seat-rebalance-gen-small.md. Also commits two previously-untracked durable artifacts (fv-to-ana-nat runbook + its memory detail) so memory does not lag the work. Archival: file is over the ~300-line soft cap (377) but 0 entries archived — the 4 age-eligible (>14d) entries all carry open deferred-work pointers (guarded), and all other entries are <14 days old. Reported per the archival discipline.
This commit is contained in:
@@ -0,0 +1,15 @@
|
||||
# FV→ANA NAT repaired safely
|
||||
|
||||
Operator approved narrow fix, caution not to strand subnet. OPNsense MESH/opt6
|
||||
(tailscale0 100.64.0.8) lacked outbound NAT for forwarded LAN traffic. Temporary
|
||||
fv-ml1→hub /32 NAT proved diagnosis; persisted hybrid NAT rule source
|
||||
10.251.50.54/32 destination 10.250.0.0/16 translate interface address. Existing
|
||||
WAN NAT retained, filter rules byte-identical, no routes/mesh/host changes.
|
||||
Both stages guarded by independent rollback timers, disarmed after verification.
|
||||
|
||||
Verified hub HTTP200, ANA PostgreSQL TCP, Internet HTTPS200, reverse SSH, gateway
|
||||
management; Beszel18/18 up. Agent intentionally stops SSH45876 when WebSocket
|
||||
connects. BMC ping failed with no pre-change baseline; no BMC-health claim.
|
||||
Other FV sources and other remote subnets not covered by this narrow fix.
|
||||
Backup + rollback helper on gateway /root/fv-nat-repair-20260913. Full details:
|
||||
docs/runbooks/fv-to-ana-nat.md. No commit made.
|
||||
@@ -0,0 +1,60 @@
|
||||
# 2026-09-14 — fv-ml1 seat rebalance: cyberprev→sec, gen-small stood up, all seats to native 262K
|
||||
|
||||
Second big fv-ml1 session on 2026-09-14 (after the all-night reorg + orca blocker in
|
||||
[[2026-09-14-fv-seat-reorg-and-orca-blocker]]). Everything here is committed; regenerate
|
||||
the live view with `scripts/seat-inventory.py` and the durable catalog is
|
||||
`docs/pfi/llm-seat-catalog.md`.
|
||||
|
||||
## What changed (all committed)
|
||||
|
||||
1. **cyberprev quantized and promoted to the `sec` seat.** `hotdogs/Qwen3.8-27B-abliterated-cyber-preview`
|
||||
(a cyber offensive-security **tool-calling SFT finetune** on an abliterated base — NOT
|
||||
"an abliteration"; trained on `hotdogs/cyber-sft-agent-qwen38`, 8,400 rows / 22 tools).
|
||||
⚠ The raw hotdogs export ships **broken**: 850/1199 body tensors carry a **tripled
|
||||
`language_model.` prefix** (unsloth export bug, verified in the upstream HF index) —
|
||||
the naive quant passed every structural check and served pure gibberish. Fix:
|
||||
name-repair the source against a known-good reference (exact 1199-tensor match), then
|
||||
house mixed-NVFP4 quant → `cyberprev-nvfp4-mixed-v2`. **A coherence smoke is the gate;
|
||||
structural verification cannot substitute** (see [[reference_unsloth_tripled_prefix_export_bug]]).
|
||||
Commits 1418edb/b94d173/196416f.
|
||||
2. **mog-sec RETIRED; cyberprev is now `sec`/`sec-reasoning`.** Gateway repointed (commit
|
||||
207798d). `sec` is a ROLE alias, so this is a promotion, not silent substitution.
|
||||
⚠ mog-sec was the fleet's ONLY offense+**defense**/blue-team seat — the sec role is now
|
||||
offense-only. mog-sec stack files retained for a one-command restore. Its "persona on
|
||||
stock" description was ALSO wrong — M.O.G.-SEC is itself an offense+defense SFT finetune
|
||||
(commit 3906c68). All three sec candidates are Qwen3.8-27B finetunes differing by focus.
|
||||
3. **gen-small stood up (NEW seat).** `llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-Preserved-NVFP4-Experts-Only`
|
||||
— already on disk at `/tank/aimodels/qwen36-35b-a3b-heretic-nvfp4` (no download). Heretic
|
||||
v1.3.0 (MPOA), 88% fewer refusals @ 0.0015 KL, 19 MTP. GPU0 :8026, MTP k=3 **measured
|
||||
69.6% accept / 3.09 len**. 3.8 was requested but no general Qwen3.8 A3B exists → 3.6 is
|
||||
the fallback. Backs `summarizer` + `classifier`; new `classifier-large` → flash (accuracy
|
||||
tier); `summarizer-large` stays on flash. Commit 05f3d3b.
|
||||
4. **All sec/gen/char seats raised to native 262,144 with concurrency in the 1.2-2.5x band**
|
||||
(operator target). coder: 8192@13.75x → 16384@4.70x (waste reclaimed; ~4.7x overshoots
|
||||
2-3x because the 1.5B weight floor sits under the util knob's resolution — a KV-byte pin
|
||||
would fix it, deferred). char-rp bumped 1.02→1.22x (util 0.40→0.52); GPU1 hit its ceiling
|
||||
so char-rp-fast's util cap was trimmed (its KV is pinned, so it held 2.04x) — an in-GPU
|
||||
rebalance rather than a utility-seat move.
|
||||
5. **Durable docs:** `docs/pfi/llm-seat-catalog.md` (curated lineage/provenance/cards/speed)
|
||||
+ `scripts/seat-bench.py` (reproducible warm-tok/s + depth-OOM bench). Commit dfa91a8.
|
||||
|
||||
## Measured — serial depth + warm tok/s (2026-09-14, warm/conc=1/greedy/n=3)
|
||||
|
||||
| seat | GPU | ctx | conc | warm tok/s | VRAM | depth clean to | OOM |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| cyberprev (sec) | 0 | 262144 | 1.37x | 136.6 | 47.1 GiB | 259,722 tok | none |
|
||||
| gen-small | 0 | 262144 | 2.56x | 254.8 | 36.1 GiB | 254,526 tok | none |
|
||||
| gen (flash-next) | 2 | 262144 | 1.31x | 170.7 | 95.3 GiB | 254,273 tok | none |
|
||||
| char-rp | 1 | 262144 | 1.22x | 62.7 | 42.3 GiB | 254,858 tok | none |
|
||||
| char-rp-fast | 1 | 262144 | 2.04x | 225.2 | 27.0 GiB | 254,823 tok | none |
|
||||
| coder | 1 | 16384 | 4.70x | 337.3 | 5.6 GiB | 15,905 tok | none |
|
||||
|
||||
Every seat prefilled near max context with **zero OOM/CUBLAS/illegal-memory** (allocator-log
|
||||
verified). Numbers are uncontended single-stream ceilings — re-derive with `scripts/seat-bench.py`.
|
||||
|
||||
## Open / deferred
|
||||
|
||||
- **mog-sec fate** — retired; restore for blue-team coverage vs leave retired. Operator's call.
|
||||
- **coder ≤3x** — needs a `--kv-cache-memory` byte pin (compose change) vs the fragile util floor.
|
||||
- **gen-small 2.56x** — a hair over the 2.5x band top; trimmable, left as useful headroom for the bulk seat.
|
||||
- **mog-sec / sentinel-r3 stacks** retained on disk for restore; sentinel gateway aliases deleted (they 404).
|
||||
Reference in New Issue
Block a user