# char-rp-fast — swapping the MeroMero A4B onto the erp-seat seat (2026-09-10)

Operator: *"replace that a4b moe over pfish-6 -- remove the pfish-6 alias and create an
alias for char-rp-fast."*

Result: `G4-MeroMero-26B-A4B-it-uncensored-heretic-NVFP4A16` is live on ana-ml2 `:8021`
behind gateway alias `char-rp-fast`. `Pfish-6` is gone from the gateway. It took two
attempts, because the first quant was broken in a way that looks exactly like a healthy seat.

## ⚠ The failure worth remembering: a 4-bit MoE router serves NaN and passes its healthcheck

The A4B built that morning used `services/meromero-quant/quant_a16_datafree.py` — the **dense**
v2-31B recipe. Its IGNORE list has no `re:.*router.*` entry, so all 30 MoE routers were quantized
to NVFP4. A 4-bit router does not degrade expert selection, it *changes which experts run*
(playbook §3.15).

What that looked like on the seat, in order of how convincing each signal was:

| signal | what it said |
|---|---|
| quant exit code | `rc=0`, 16 G, no warning |
| `docker` healthcheck | healthy in 210 s |
| engine log | KV pool 534,649 tokens, 2.04x — exactly right |
| `/v1/models` | correct served name, 262,144 context |
| every completion | `finish_reason: "length"`, **full** `completion_tokens` (120/120, 600/600) |
| `content` | `null`. Every time. |
| raw `/v1/completions` | `text: ''` — so it was not the chat template or the reasoning parser |
| **`logprobs: 1`** | **HTTP 400 `Out of range float values are not JSON compliant: nan`** |

The model was generating a full budget of tokens that decoded to the empty string, and the only
thing that named the fault was asking for logprobs. `seat_verify.py` now carries that as check 6.

**What actually found it** was not the CPU forward (started, then abandoned as too slow): it was
diffing `quantization_config.ignore` against **Pfish-6** — a known-good NVFP4A16 quant of the
*same architecture class*. 222 entries against 252, and the 30 missing were precisely
`model.language_model.layers.N.router.proj`.

⚠⚠ **The broken tree HAD been structurally diffed before it shipped — against a verified-good
DENSE 31B quant of the same Gemma-4 family, which came back clean.** A dense model has no
routers, so the one thing that was wrong was the one thing that control could not see. **A
positive control is only worth what it can distinguish; "same family" is not "same architecture
class."**

Fix: re-quantize with `quant_nvfp4a16_gemma4_moe.py`, whose `--dry-run` asserts
`layers × experts × 3 = 11,520` expert Linears and refuses if a router lands in the quantize set,
both before any GPU time. 90 seconds end to end. The broken tree is parked on ana-ml2 as
`...-NVFP4A16.BROKEN-routers-quantized-20260910`. **Do not serve it.**

## Why the seat went dark for ~16 minutes instead of not at all

Playbook §4.4 wants a temp port. It was not reachable, twice, and the numbers are worth keeping:

- `--gpu-memory-utilization 0.20` → **admission refused**: `Free memory on device cuda:0
  (18.26/94.97 GiB) on startup is less than desired GPU memory utilization (0.2, 18.99 GiB)`.
- `0.185` + `--kv-cache-memory 1.5 GB` + `--max-model-len 8192` + `--enforce-eager` → past
  admission, past the KV reservation, then `torch.OutOfMemoryError` during **multimodal
  encoder-cache profiling** (`profiled with 3 video items of the maximum feature size`). That
  profiling cost is easy to forget when budgeting a vision model.

15.9 GiB of weights plus a KV pool plus vision profiling does not fit in the ~19 GiB free beside
the other six GPU1 tenants. So the substitute was **reversibility and ordering**:

1. back the host `.env` up to a *named* file first (`.env.pfish6.bak-20260910`);
2. swap `.env`, `up -d`, and prove the seat on its real port **while no gateway alias points at
   it**;
3. move the gateway alias **last**.

That ordering is why the NaN-serving seat never reached a consumer — `char-rp-fast` did not exist
yet and `Pfish-6` still resolved to nothing else. The cost was ~16 minutes of that one seat being
down, twice, and nothing downstream saw a broken alias.

## The swap, as steps

```bash
# on ana-ml2, /opt/docker/compose/erp-seat
cp -n .env .env.pfish6.bak-20260910          # ROLLBACK LIVES HERE
# point ERP_MODEL / ERP_SERVED_NAME / ERP_CHAT_TEMPLATE at the new tree
sudo docker compose up -d                     # ~210 s to healthy

# verify BEFORE touching the gateway
python3 seat_verify.py http://127.0.0.1:8021/v1 <served-name>

# gateway (canonical: stacks/litellm/conf/config.yaml)
scripts/deploy-stack.sh ana-docker litellm --conf
ssh ana-docker 'cd /opt/docker/compose/litellm && sudo docker compose restart litellm'
```

**Rollback to Pfish-6** is `cp .env.pfish6.bak-20260910 .env && sudo docker compose up -d`,
~4 minutes. `/tank/aimodels/erp-tune-v6-nvfp4a16` is untouched.

## What was checked, and what was not

Verified on the live seat (`raw/char-rp-fast-seat-verification-2026-09-10.txt`): served name and
262,144 context; KV 534,649 tokens / 2.04x; clean prose with no `<|channel>thought` leak and no
reasoning field; **a solid-colour image read correctly**, so vision is tested rather than inferred
from a tensor count; an auto `tool_choice` call parsed with correct arguments; finite logprobs.
Through the gateway with the shared `all-agents-local` key: `char-rp-fast` answers, `Pfish-6`
returns an explicit `400 Invalid model name` rather than a substitution, and `char-rp` /
`char-rp-reasoning` are both unaffected.

Audited before removing the alias: **0 of 17 LiteLLM keys** named `Pfish-6` in their model
allowlist, so nothing was orphaned (1 of 17 is unrestricted and reaches whatever the gateway
serves). ⚠ The first attempt at that audit passed `size=200` and got a silent `422`, which the
script reported as "scanned 0 keys" — an empty result and a rejected query look identical if you
do not check.

**Not established:** anything about quality. No RP eval, no long-context check, no A/B against
Pfish-6 or `char-rp`. The samplers are the author's card values (Temp 0.8–1.0, MinP 0.05), not
tuned here. n=1 smoke output is not evidence about writing.

⚠ Pre-existing doc rot noticed and **not** fixed: the `char-rp` comment block in
`stacks/litellm/conf/config.yaml` still describes its `:8016` seat as MeroMero-v2. That has been
stale since the 2026-08-24 swap to stock Gemma-4.