76 lines
4.1 KiB
Markdown
76 lines
4.1 KiB
Markdown
# Run 7 retired, Pfish-6 declared, and the detector bug that nearly buried it (2026-09-09)
|
||
|
||
Operator ruling, evening of 09-09: *"declare run 6 as Pfish-6 … we're gonna stay on 6 for now."*
|
||
|
||
## What happened, in order
|
||
|
||
1. **Run 7 trained clean** (542/542, `train_loss` 3.205) and brokkr's base window closed clean.
|
||
2. **The tuned window aborted** on the CSAM drift detector: `level=hit`, `counts=1/0/3`, two HARD
|
||
`child_term ^ act` flags. Fail-closed by design.
|
||
3. **I tore down both seats** — `erp-tune-v7` on gx10:8098 and, on my own judgement reversing an
|
||
operator instruction, the `trial` NVFP4 seat on ana-ml2:8021 (it was a quantized derivative of the
|
||
same tune, live on the shared-key gateway ~15:30–17:43 PT).
|
||
4. **The operator diagnosed it as a false positive** and he was right. `child_term` was a flat
|
||
alternation containing bare `minor|minors` — it matched the **ADJECTIVE**. Reproduced on the
|
||
untouched detector: `scan("He noticed a minor detail before intercourse.")` → `hit`,
|
||
`HARD child_term@13 ^ act@33`. A gate failure on a sentence about a detail.
|
||
5. **Fixed** (`cc42d76`): singular `minor` fires on NOMINAL use only — followed by punctuation, a
|
||
clause end, a relative pronoun, an auxiliary/copula or a conjunction; the adjective is followed by
|
||
the noun it modifies. Plural `minors` stays blanket. 6 regression cases each way, selftest 24/24.
|
||
Baby-family SOFT terms removed separately (`53a0192`) on the same ruling.
|
||
6. **Run 7 retired anyway**, on its own merits: primary FLAT (69 → 70.5, +2, flat at both the 12-word
|
||
threshold and the 20/60 cue-probe floor), both diversity families reduced past their floors,
|
||
long-context coherence 1.0 → 0.875 on its must-not-harm bar, unanswerable control valid at 1.0.
|
||
|
||
## Pfish-6 — the standing seat
|
||
|
||
`Pfish-6` = `erp-tune-v6-nvfp4a16`, the NVFP4A16 quant of the run-6 merge, served under that name.
|
||
|
||
ana-ml2 :8021 /tank/aimodels/erp-tune-v6-nvfp4a16 262,144 ctx
|
||
pfi-gx10 :8098 ~/erp-tune/serve/pfish6-nvfp4a16 262,144 ctx
|
||
gateway alias `trial` RETIRED, replaced by `Pfish-6`
|
||
|
||
## Speed work — what was and was not available
|
||
|
||
⚠ **vLLM's "your GPU does not have native support for FP4" on ana-ml2 is about the SCHEME, not the
|
||
card.** sm_120 HAS native FP4. NVFP4A16 is weight-only, activations stay 16-bit, so no FP4 GEMM is
|
||
ever issued and Marlin is correct. **Proven, not assumed:** forcing
|
||
`moe_backend=flashinfer_cutedsl` fails at engine init — *"does not support the deployment
|
||
configuration since kernel does not support current device cuda"*. I first misread the "out of
|
||
potential backends: [FLASHINFER_…]" log line as meaning faster kernels were sitting unused; that line
|
||
prints the static candidate list, not the viable ones.
|
||
|
||
**The real lever was concurrency.** `max-num-seqs` was 8, capping the seat far below its 534,649-token
|
||
KV pool. Raised to 32 (CUDA-graph capture sizes followed to 64). Measured on ana-ml2, 3 rounds each,
|
||
300-token generations:
|
||
|
||
| streams | aggregate tok/s |
|
||
|---|---|
|
||
| 1 | 219 |
|
||
| 4 | 676 |
|
||
| 8 | **1,269** — unreachable at the old cap |
|
||
| 16 | **2,170** — 3.2× the old ceiling |
|
||
|
||
Single-stream latency unchanged, so the raise costs nothing.
|
||
|
||
## ⚠ Head-to-head: ana-ml2 is 4.1× FASTER than the GX10, the reverse of the expectation
|
||
|
||
Same artifact, same 262,144 context, same Marlin kernel, 3 rounds each:
|
||
|
||
| | ana-ml2 (RTX PRO 6000 Blackwell) | pfi-gx10 (GB10) |
|
||
|---|---|---|
|
||
| n=1 | **219 tok/s** | 53.6 tok/s |
|
||
| n=4 aggregate | **676 tok/s** | 175 tok/s |
|
||
| KV pool | 534,649 tok (2.04× ctx) | 4,608,942 tok (17.6× ctx) |
|
||
|
||
Decode is bandwidth-bound; the discrete card's GDDR7 dwarfs the GX10's unified LPDDR5X. The GX10's
|
||
advantage is **capacity**, not speed — a 4.6M-token KV pool. Do not assume the training box is the
|
||
fast box for interactive serving.
|
||
|
||
## Preserved, nothing deleted
|
||
|
||
`run-07/adapter` 315 MB and `serve/merged-run07` 49 GiB on the GX10; `erp-tune-v7-nvfp4a16` 16 GiB and
|
||
`erp-tune-v7-bf16` 49 GiB on ana-ml2. Disposition is the operator's and he has not ruled on it.
|
||
|
||
See [[2026-09-09-ana-ml2-pool-actions-done]] for the same day's storage work.
|