Swap the MeroMero A4B onto the erp-seat seat as char-rp-fast, retire the Pfish-6 alias
Operator: "replace that a4b moe over pfish-6 -- remove the pfish-6 alias and create an alias for char-rp-fast." G4-MeroMero-26B-A4B-it-uncensored-heretic-NVFP4A16 is live on ana-ml2 :8021 under its own served name, behind the new gateway alias char-rp-fast. Pfish-6 is gone from the gateway and now returns an explicit 400 rather than a substitution; 0 of 17 LiteLLM keys scoped it, so nothing was orphaned. The compose project name stays erp-seat because asset-engine derives seat liveness from it. The first quant of that A4B served NaN and passed its healthcheck doing it. It was built with the dense v2-31B recipe, whose ignore list has no router regex, so all 30 MoE routers were quantized to 4 bits -- and a 4-bit router changes which experts run rather than degrading them. Quant rc=0, healthcheck green, correct KV pool, correct served name, and every completion returned finish_reason=length with the full token count and content: null. The model was emitting a full budget of tokens that decoded to the empty string. Raw /v1/completions was empty too, ruling out the chat template and the reasoning parser. The signal that named it was logprobs: vLLM refused to serialize the response, "Out of range float values are not JSON compliant: nan". The lesson is about the control rather than the router. That tree had already been structurally diffed and passed -- against a verified-good DENSE quant of the same Gemma-4 family. A dense model has no routers, so the single thing that was wrong was the single thing the control could not distinguish. Diffing instead against Pfish-6, a known-good quant of the same 26B-A4B MoE, gave it in one line: 222 ignore entries against 252, the 30 missing being layers.N.router.proj. A positive control is only worth what it can distinguish, and "same family" is not "same architecture class". Re-quantized with the MoE recipe, whose dry-run asserts 11,520 expert Linears and refuses a router in the quantize set before any GPU time. The live seat then passed prose with no channel-prefix leak, a solid-colour image read correctly, an auto tool call parsed, finite logprobs, and KV 534,649 tokens / 2.04x carried over from Pfish-6 unchanged. The broken tree is parked on ana-ml2 as ...-NVFP4A16.BROKEN-routers-quantized-20260910. Section 4.4's temp port was not reachable: 15.9 GiB of weights plus KV plus multimodal encoder-cache profiling does not fit in the ~19 GiB free beside GPU1's six other tenants -- 0.20 utilization refused admission, 0.185 OOM'd in encoder profiling. The substitute was reversibility and ordering: named .env backup, prove the seat on its real port while no alias points at it, move the alias last. That is why a NaN-serving seat never reached a consumer. The seat was down about 16 minutes across two attempts; no consumer saw a broken alias. Playbook gains the router-quant failure signature and the control-class rule in 3.15, and a logprobs check in 4.4. seat_verify.py carries that check as check 6. Quality is NOT established: no RP eval, no long-context check, no A/B against Pfish-6 or char-rp. Samplers are the author's card values, untuned here.
This commit is contained in:
+35
-7
@@ -1,6 +1,6 @@
|
||||
# Persistent memory — eshpfi-management
|
||||
|
||||
_Last updated: 2026-09-10 11:00 PT (R49 1-epoch pilot COMPLETE + all 3 arms cut, awaiting adjudication; **MeroMero BOTH quants landed** — v2 dense on attempt 5, serve test still owed; althing 3.6.2 on post office + both heralds; ~574 GB reclaimed)_
|
||||
_Last updated: 2026-09-10 11:35 PT (R49 1-epoch pilot COMPLETE + all 3 arms cut, awaiting adjudication; **MeroMero BOTH quants landed and the A4B is LIVE on :8021 as `char-rp-fast`, `Pfish-6` alias removed** — its first quant served NaN and looked healthy; althing 3.6.2 on post office + both heralds; ~574 GB reclaimed)_
|
||||
|
||||
> **Always check for `/tmp/infra-ops-handoff.md`** — if it exists and its
|
||||
> `Written:` stamp is under an hour old, read it (it carries the in-flight
|
||||
@@ -141,9 +141,20 @@ _As of 2026-09-10 10:25 PT._
|
||||
|
||||
### MeroMero seats
|
||||
|
||||
- ✅ `G4-MeroMero-26B-A4B-it-uncensored-heretic-NVFP4A16` — 16 G, W4A16, §4.3 post-steps DONE.
|
||||
⚠ It had the §3.14 **truncation cap baked into `tokenizer.json`** (`max_length: 8192`, because it
|
||||
was quantized *with* the corpus). Fixed; backup `tokenizer.json.bak-pre-truncfix`.
|
||||
- ✅ **LIVE on ana-ml2 `:8021` as gateway alias `char-rp-fast`** (operator, 2026-09-10: *"replace
|
||||
that a4b moe over pfish-6 — remove the pfish-6 alias and create an alias for char-rp-fast"*).
|
||||
`G4-MeroMero-26B-A4B-it-uncensored-heretic-NVFP4A16`, 16 G, served under its own true name on the
|
||||
**`erp-seat` stack** (name kept: asset-engine derives liveness from the compose project name).
|
||||
Verified end to end — prose, **vision (reads a solid-colour image)**, auto tool call, finite
|
||||
logprobs, and KV 534,649 tokens / 2.04x carried over from Pfish-6 unchanged.
|
||||
⚠⚠ **ITS FIRST QUANT SERVED NaN AND PASSED ITS HEALTHCHECK DOING IT.** Built with the DENSE
|
||||
recipe (no `re:.*router.*` in IGNORE) → all 30 MoE routers quantized to 4 bits → expert selection
|
||||
destroyed. Every request returned `finish_reason=length` with the FULL token count and
|
||||
`content: null`; the only tell was **NaN logprobs**. Re-quantized with
|
||||
`services/erp-seat-quant/quant_nvfp4a16_gemma4_moe.py` (its guard refuses exactly that). Broken
|
||||
tree parked at `...-NVFP4A16.BROKEN-routers-quantized-20260910` — **do not serve it**.
|
||||
⚠ Also had the §3.14 truncation cap baked in (`max_length: 8192`, quantized *with* the corpus);
|
||||
the MoE recipe's own post-step resets it.
|
||||
- ✅ `G4-MeroMero-v2-31B-heretic-NVFP4A16` — **19 G, landed on attempt 5.** Tensor table identical
|
||||
family-for-family to the 2026-08-21 canonical quant; **356 BF16 vision tensors preserved**;
|
||||
`input_activations=None` (genuinely A16). CPU load+generate coherent, 0 tensors on meta.
|
||||
@@ -152,11 +163,27 @@ _As of 2026-09-10 10:25 PT._
|
||||
the redundant `per_layer_config`, not to force global access.
|
||||
→ `persistent-memory.d/2026-09-10-meromero-quants-and-the-pinned-transformers-trap.md`,
|
||||
`services/meromero-quant/`, playbook **§3.17 (new)**
|
||||
- ⛔ **NEITHER quant has had its §4.4 serve test** — GPU1 has 19.9 GB free against 19.5 GB of v2
|
||||
weights, so it needs a live seat displaced. Operator's call; *"vllm servable"* unverified until then.
|
||||
- ⛔ **The v2 DENSE has still not had a serve test** — GPU1 has ~19 GB free against 19.5 GB of its
|
||||
weights, so it needs a live seat displaced. Operator's call; *"vllm servable"* unverified for the
|
||||
dense tree. (The A4B's serve test is DONE and green, above.)
|
||||
- ⚠ **A co-resident temp port could not be made to fit even for the 16 G A4B**, so §4.4's "temp port,
|
||||
never the live seat" was substituted with reversibility: named `.env` backup, prove the seat on its
|
||||
real port while no alias routes to it, move the alias last. Measured refusals: `gpu-memory-util 0.20`
|
||||
→ admission refused (18.26 GiB free vs 18.99 requested); `0.185` → past admission and past the KV
|
||||
reservation, then OOM in **multimodal encoder-cache profiling** (3 video items at max feature size),
|
||||
which is easy to forget when budgeting a vision model.
|
||||
- ⚠ **The A4B is 30 layers / kv 8 — Pfish-6's geometry**, so it fits 262k in the existing KV budget.
|
||||
The dense 31B is 60/16, ~4x KV per token, and does NOT fit 262k on GPU1 beside the other seats.
|
||||
- Pfish-6 remains the standing seat on ana-ml2 `:8021`. Nothing repointed, no `rp-fast` alias exists.
|
||||
- ⛔ **`Pfish-6` IS RETIRED from the gateway** (2026-09-10). Its seat now serves the A4B; a caller
|
||||
asking for `Pfish-6` gets an explicit 400 "Invalid model name", not a substitution. Audited first:
|
||||
**0 of 17 LiteLLM keys** scoped it, so nothing was orphaned. The artifact stays on disk at
|
||||
`/tank/aimodels/erp-tune-v6-nvfp4a16` and is the rollback target — `cp
|
||||
.env.pfish6.bak-20260910 .env && docker compose up -d` in `/opt/docker/compose/erp-seat` restores
|
||||
it in ~4 min.
|
||||
- The `char-rp` family is now **`char-rp` (stock Gemma-4 26B-A4B, :8016) / `char-rp-fast` (MeroMero
|
||||
A4B, :8021) / `char-rp-reasoning` (Dark-Scarlett-27B, :8019)**. All three verified working after
|
||||
the change. ⚠ The `char-rp` comment block in `stacks/litellm/conf/config.yaml` still describes its
|
||||
seat as MeroMero-v2; that has been stale since the 2026-08-24 swap to stock Gemma-4. Not fixed.
|
||||
|
||||
### Fleet
|
||||
|
||||
@@ -181,6 +208,7 @@ _As of 2026-09-10 10:25 PT._
|
||||
|
||||
- `[2026-09-10]` **R49 carrier SETTLED on dense `Qwen3-{0.6,1.7,4}B-Base`, overriding H02's own pin — the newest carrier was the SLOW one.** Dense 4.089 B trains 33% faster than hybrid 0.765 B; no fused SSM kernel installed. D1–D3 built, 1-epoch pilot beats the 3-epoch by 0.21 nats held-out. → `persistent-memory.d/2026-09-10-r49-babybronte-d1-d3-and-the-1-epoch-pilot.md`
|
||||
- `[2026-09-10]` **R49 adjudication routed to infra-ops entirely** (operator, relayed by brokkr: *"leave babybronte to infra — concentrate on r50 and the memory mechanism"*). brokkr handed over the Delta instrument and stepped off. ⚠ I now grade my own run; brokkr's decision rule is **ratified verbatim and frozen before any adapted text existed** and must not be amended after seeing numbers. Their controls: real Charlotte 1.65–2.17, **Anne at 2.374** — so the absolute band decides, never `nearest`.
|
||||
- `[2026-09-10]` **MeroMero A4B swapped onto the `erp-seat` seat as `char-rp-fast`; `Pfish-6` alias removed.** The A4B's FIRST quant used the dense recipe and 4-bit-quantized all 30 MoE routers — it passed its healthcheck and answered every request with the full token count decoding to the empty string, NaN logits the only tell. Re-quantized with the MoE recipe; live and verified (prose, vision, tool call, finite logprobs). Durable lesson: **a positive control must match the ARCHITECTURE CLASS** — the broken A4B was diffed against a good *dense* quant, which has no routers, so the clean result was meaningless. → playbook §3.15, §4.4
|
||||
- `[2026-09-10]` **MeroMero: BOTH quants landed in-house at W4A16 — A4B first try, v2 dense on attempt 5.** Published quants are all W4A4 (our measured long-context collapse) or nonexistent for v2. Operator: *"pull both ablits bf16, run our own quant."* The durable lesson is **§3.17**: `pip install llmcompressor` silently pins transformers down a version, so attempt 4's error was a moved toolchain, not the malformed upload it looked like — a known-good positive control is what told them apart. Serve test still owed. → `persistent-memory.d/2026-09-10-meromero-quants-and-the-pinned-transformers-trap.md`
|
||||
- `[2026-09-10]` **althing 3.6.2 deployed — post office + both heralds — and the fleet has TWO herald nodes, not seven.** Ask the post office's `nodes` table, not the box inventory. Cost a self-inflicted ~12 min bus outage. → `persistent-memory.d/2026-09-10-althing-362-rollout.md`
|
||||
- `[2026-09-10]` **A grep over a log that records your greps counts itself.** I reported forseti's drop defect as reproducing here with 3 drops in 21 s; the session had **zero**. Searching transcripts writes the search term into them. Filter by `"type":"system"` provenance, never content. Generalises to any instrument that can see itself. Auto-memory `feedback_grep_over_a_log_that_records_your_greps`.
|
||||
|
||||
Reference in New Issue
Block a user