fix(parakeet): move the seat from the empty GPU 3 to GPU 0

Placed on GPU 3 first because it was the empty card. That was the wrong read:
the seat is ~800 MiB, under 1% of a 96 GB card, so the question was never "where
does it fit" but "whose headroom is cheapest to spend".

vLLM sizes its KV cache as a fraction of TOTAL VRAM, not free VRAM. A resident
tenant on an otherwise-clean card therefore does not cost its own megabytes — it
costs the profiling margin of whatever full-size seat lands there later, and
flash-next needs 93 GiB of 96. A 96 GB card at 2 MiB can still take that; the
same card at 922 MiB is one where the next big seat needs its utilization
hand-trimmed, which this repo's flash-next history shows is both thin and silent
when it goes wrong.

Committed utilization per card is the number that governs, not free bytes:

    GPU 0   0.40 + 0.48                       = 0.88    ~13 GB spare  <- moved here
    GPU 1   0.52+0.24+0.10+0.055+0.03+0.03    = 0.975   ~4.3 GB
    GPU 2   0.96                                        ~1.8 GB
    GPU 3   -                                           kept empty as reserve

GPU 3 is back to 2 MiB / 97,247 MiB free and is now documented as a deliberate
reserve rather than a spare.

Post-move n=5 on the same clip: 0.68 / 0.54 / 0.54 / 0.52 / 0.53 s, median 0.54 s
against 0.50 s on GPU 3. The spreads overlap at this sample size and no difference
is claimed; the GPU 3 figure was taken on an idle card and is now noted as a best
case, since the seat shares GPU 0 with the hot serving path. Silence control and
the gateway round-trip both re-verified after the move.

Also records both Parakeet endpoints (FV v3 on Blackwell, IRV v2 on a 3090) and
the four confounds that make them not an A/B pair, sent to tts-dev for the bench.

persistent-memory.d/2026-09-15-parakeet-stt-fv-ml1.md

@@ -6,19 +6,45 @@ Operator asked for an STT service on fv-ml1's utility GPU plus a LiteLLM alias.
 
 `stacks/parakeet/` — Parakeet-TDT 0.6B **v3** int8 ONNX (25 European languages,
 464 MiB) under sherpa-onnx, behind ~90 lines of FastAPI we own. Container
-`parakeet`, port **8300**, **GPU 3** pinned by `device_ids`. Image
+`parakeet`, port **8300**, **GPU 0** pinned by `device_ids`. Image
 `local/parakeet:sherpa-onnx-v4` (5.09 GB).
 
 Not greenfield: the stack already existed, targeting irv-ml1. Retargeted rather
 than rewritten — the Ampere→Blackwell move was the only real question.
 
-## Why GPU 3
+## ⚠ Placement — got this wrong first, operator caught it
 
-GPU 0 = 84/96 GB, GPU 1 = 92.9/96, GPU 2 = 95.5/96 (the vLLM seats). **GPU 3 was
+Placed on the empty **GPU 3** initially, reading "the utility gpu" as "the spare
-at 2 MiB.** The dead on-host stub used `count: all`, which would have handed this
+card". Operator's correction: *"1gb total vram pressure — and you didn't load it on
-seat all four cards; replaced with an explicit `device_ids: ["3"]` per the fleet
+gpu 0?"* He is right, and the reason is sharper than "it fits anywhere".
-convention. Inside the container the pinned card presents as `cuda:0`, which is
+
-what ORT's CUDA EP takes by default.
+**vLLM sizes its KV cache as a fraction of TOTAL VRAM, not free VRAM.** So a
+resident tenant on an otherwise-clean card does not cost its own megabytes — it
+costs a future full-size seat's profiling margin. `flash-next` needs **93 GiB of
+96**. A 96 GB card at 2 MiB is a card that can still take that; the same card at
+922 MiB is a card where the next big seat's `--gpu-memory-utilization` has to be
+hand-trimmed, and the flash-next history in this repo shows exactly how thin and
+how silent that failure gets.
+
+The right question is not "where does 800 MiB fit" but "whose headroom is cheapest
+to spend":
+
+| GPU | committed util | spare |
+|---|---|---|
+| **0** | 0.40 + 0.48 = **0.88** | ~13 GB ← moved here |
+| 1 | **0.975** (six small seats) | ~4.3 GB |
+| 2 | **0.96** (flash-next) | ~1.8 GB |
+| 3 | — | **kept empty as reserve** |
+
+Moved the same night: one env var (`PARAKEET_GPU`) plus `compose up -d`. GPU 3 back
+to 2 MiB / 97,247 MiB free. Post-move n=5 on the same clip: 0.68 / 0.54 / 0.54 /
+0.52 / 0.53 s, median 0.54 s — **indistinguishable from the GPU 3 median of 0.50 s
+at this sample size**; the spreads overlap and no difference is claimed.
+
+The dead on-host stub used `count: all`, which would have handed this seat all four
+cards; replaced with an explicit `device_ids` pin per the fleet convention. Inside
+the container the pinned card presents as `cuda:0`, which is what ORT's CUDA EP
+takes by default.
 
 ## ⚠ The finding worth keeping: a 45-second first decode
 
@@ -30,7 +56,9 @@ at session creation. On sm_120:
 | first decode, cold container | **45.7 s** (n=1), reproduced at **45.1 s** on a second container |
 | warm, 8.52 s clip | **0.50 s** median (n=5: 0.65 / 0.53 / 0.48 / 0.47 / 0.50) |
 
-≈17× realtime warm, single-stream, one 8.52 s clip, int8, GPU 3 idle otherwise.
+≈17x realtime warm, single-stream, one 8.52 s clip, int8. ⚠ Measured on GPU 3 while
+it was idle; the seat now lives on GPU 0 beside the hot serving path, so treat that
+number as a best case.
 That is a smoke measurement with its harness stated, **not** a benchmark — no
 concurrency sweep, no length sweep, one clip.
 
@@ -48,8 +76,8 @@ returns *correct text*, just slowly. Our own log line `loading OfflineRecognizer
 The discriminator that actually settles it:
 
 ```
-nvidia-smi --query-compute-apps=pid,process_name,used_memory --format=csv -i 3
+nvidia-smi --query-compute-apps=pid,process_name,used_memory --format=csv -i 0
--> 1588301, /opt/venv/bin/python3, 922 MiB
+-> 1594431, /opt/venv/bin/python3, 794 MiB   (beside two VLLM::EngineCore entries)
 ```
 
 Timing is **not** a sufficient check either — the int8 model is fast enough on a
@@ -84,3 +112,23 @@ that is where the `ext-tts` family lives, and it needs no gateway restart.
   part of the 5-host normalisation).
 - `servers/fv-ml1/README.md` is still broadly stale — it claims 2 GPUs and a
   2026-07-22 stack list. Only the parakeet/GPU-3 rows were corrected.
+
+
+## Two Parakeets, and the bench Vuong asked for
+
+Both are live; endpoints sent to **tts-dev** 2026-09-15 for a head-to-head.
+
+| | FV (new) | IRV (existing, up 2 months) |
+|---|---|---|
+| endpoint | `http://10.251.50.54:8300/v1/audio/transcriptions` | `http://100.64.0.6:8765/...` or `http://10.6.110.50:8765/...` |
+| model | parakeet-tdt-0.6b-**v3** int8, 25 languages | parakeet-tdt-0.6b-**v2** int8, English only |
+| GPU | RTX PRO 6000 Blackwell **sm_120**, GPU 0, shares with 2 vLLM seats | RTX 3090 **sm_86**, shares with 4 processes, 4.0 GB free |
+| image | `local/parakeet:sherpa-onnx-v4` (has startup warmup) | `local/parakeet:sherpa-onnx-v2` (no warmup) |
+
+⚠ **`10.100.79.3:8765` is DEAD** — the retired wg0 lifeline, still the href on IRV's
+Homepage card. Same for `Speaches ASR` at `10.100.79.3:8204`.
+
+⚠ **These were never an A/B pair — four things differ at once** (model version,
+GPU architecture, card contention, image). A WER delta is a **v2-vs-v3** result, not
+an FV-vs-IRV one. Offered tts-dev a v2 container on FV as a second compose project so
+accuracy can be varied one factor at a time; not built unless they take it up.

persistent-memory.md

@@ -175,7 +175,7 @@ hardened for ha-dev (`d1769ed` ff); `kb` KB-search tool (`68fa80f`).
 
 ## Recent decisions
 
-- `[2026-09-15]` **Parakeet STT live on fv-ml1 GPU 3, behind LiteLLM `ext-stt` / `whisper-1`.** Retargeted the existing `stacks/parakeet/` (sherpa-onnx + our own FastAPI wrapper) from irv-ml1; v3 int8, 25 languages. ⚠ **ORT's CUDA EP compiles kernels lazily and the first decode on sm_120 took 45.7 s** — every later call ~0.5 s; a startup warmup in `app.py` now absorbs it, so the first real request is 0.65 s instead of a 45 s hang that no client would wait through. GPU use was **verified by a process on GPU 3 (922 MiB), not by the `provider=cuda` log line**, because ORT falls back to CPU silently and still returns correct text. Silence → `""` (null control), known sentence → near-exact (positive control). → `persistent-memory.d/2026-09-15-parakeet-stt-fv-ml1.md`
+- `[2026-09-15]` **Parakeet STT live on fv-ml1 GPU 0, behind LiteLLM `ext-stt` / `whisper-1`.** ⚠ **Placed on GPU 3 first, which was wrong — operator caught it.** A ~800 MiB seat should ride the card with the most uncommitted headroom (GPU 0, util 0.88, ~13 GB spare), not put the first fingerprint on the one pristine 96 GB card: vLLM sizes KV cache against TOTAL VRAM, so any tenant on an empty card eats a future full-size seat's profiling margin (flash-next needs 93 of 96 GiB). **GPU 3 is now a deliberate reserve at 2 MiB.** Retargeted the existing `stacks/parakeet/` (sherpa-onnx + our own FastAPI wrapper) from irv-ml1; v3 int8, 25 languages. ⚠ **ORT's CUDA EP compiles kernels lazily and the first decode on sm_120 took 45.7 s** — every later call ~0.5 s; a startup warmup in `app.py` now absorbs it, so the first real request is 0.65 s instead of a 45 s hang that no client would wait through. GPU use was **verified by a process on GPU 3 (922 MiB), not by the `provider=cuda` log line**, because ORT falls back to CPU silently and still returns correct text. Silence → `""` (null control), known sentence → near-exact (positive control). → `persistent-memory.d/2026-09-15-parakeet-stt-fv-ml1.md`
 
 - `[2026-09-15]` **`svos_miranda` Hermes plugin validated; found its load blocker.** Absolute intra-package imports (`from hermes_plugin.x`) could not resolve at the documented install name — fixed by svos-dev at `c964e64`. ⚠ **`hermes plugins validate` and `doctor` can NEVER pass this plugin**, by construction: validate's probe stub is config-blind AND returns `None` from `register_tool` (which the plugin's guard reads as a collision), and doctor runs under a temp `HERMES_HOME` with no config. ⚠ `doctor` exits **0** on ERROR (use `--ci`); `compat` reads a **nonexistent path as a pass**. Roster verified 8/7 by a probe supplying real settings. → `persistent-memory.d/2026-09-15-svos-miranda-plugin-validation.md`
 

servers/fv-ml1/README.md

@@ -110,14 +110,18 @@ seats, safe to leave: `mistral-medium-3.5`, `mistral-small-4(-heretic)`,
 `qwen36-27b-aeon`, `qwen-image-bench`, `vibevoice`, `comfyui`, `kokoro`,
 `vllm-qwen3`.
 
-**GPU 3 — utility card:**
+**Also on GPU 0 (non-vLLM):**
 
 | Container | Port | Serves | Notes |
 |-----------|------|--------|-------|
-| `parakeet` | 8300 | Parakeet-TDT 0.6B v3 int8 (25 languages) | ASR via sherpa-onnx, LiteLLM `ext-stt` / `whisper-1`. Relocated from irv-ml1 2026-09-15. `stacks/parakeet/`. |
+| `parakeet` | 8300 | Parakeet-TDT 0.6B v3 int8 (25 languages) | ASR via sherpa-onnx, LiteLLM `ext-stt` / `whisper-1`. Relocated from irv-ml1 2026-09-15. ~800 MiB. `stacks/parakeet/`. |
 
-⚠ The other three cards run 85-98% full, so GPU 3 is where a new small seat goes
+⚠ **GPU 3 is deliberately kept EMPTY (2 MiB).** It is the only card that can still
-until something bigger claims it.
+take a full-size seat — `flash-next` needs 93 GiB of 96 — and vLLM sizes its KV
+cache against *total* VRAM rather than free VRAM, so even a sub-1 GB tenant there
+eats into a future big seat's profiling margin. Small seats go on GPU 0, which has
+the most uncommitted headroom (its seats commit util 0.88; GPU 1 is at 0.975 and
+GPU 2 at 0.96).
 
 **Retired:**
 - `llama-swap` (former GGUF multiplexer on :9292) — replaced by dedicated

stacks/parakeet/.env.example

@@ -9,10 +9,14 @@
 # cleanly.
 PARAKEET_TAG=sherpa-onnx-v4
 
-# Which GPU to pin. fv-ml1 GPU 3 is the utility card — 0/1/2 carry the vLLM
+# Which GPU to pin. The container sees whichever card this names as cuda:0.
-# serving seats and sit at 85-98% VRAM, so this is the only one with room.
+#
-# The container sees whichever card this names as cuda:0 internally.
+# ⚠ GPU 0, deliberately, NOT the empty GPU 3. This seat is ~800 MiB and GPU 0 has
-PARAKEET_GPU=3
+# the most uncommitted headroom of the three working cards (its seats commit
+# util 0.88, ~13 GB spare; GPU 1 is at 0.975, GPU 2 at 0.96). Leaving GPU 3
+# untouched keeps a full 96 GB card available for a real seat — vLLM sizes KV
+# cache against TOTAL VRAM, so even a 1 GB tenant eats into a big seat's margin.
+PARAKEET_GPU=0
 
 # Host port for the FastAPI server (container listens on 8000). 8300 is
 # fv-ml1's established parakeet port; the 80xx range belongs to the vLLM seats.

stacks/parakeet/README.md

@@ -6,24 +6,35 @@ wrapper over [sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx)
 
 **Server:** fv-ml1 (Fountain Valley, `10.251.50.54`) — moved from irv-ml1 2026-09-15
 **Port:** 8300 (container 8000)
-**GPU:** **3**, pinned explicitly via `device_ids` — the utility card
+**GPU:** **0**, pinned explicitly via `device_ids` — shares the card with two vLLM seats
 **Image:** `local/parakeet:sherpa-onnx-v4` — built from `Dockerfile` +
 `app.py` + `entrypoint.sh` in this directory; **we own all the code**
 **Model:** `parakeet-tdt-0.6b-v3` int8, 25 European languages (~464 MiB)
 
-## Why GPU 3
+## Why GPU 0 and not the empty card
 
-fv-ml1 has four RTX PRO 6000 Blackwell Max-Q (96 GB each). Three carry the
+fv-ml1 has four RTX PRO 6000 Blackwell Max-Q (96 GB each). This seat holds
-vLLM serving seats and run 85–98 % full; GPU 3 is the utility card and was
+**~800 MiB — under 1 %** of one card, so the question is not "where does it
-empty (2 MiB) at placement time. A 0.6 B int8 ASR model is a rounding error
+fit" but "whose headroom can it spend most cheaply".
-next to those seats, but it still has to go somewhere that is not fighting
-them for VRAM.
 
-⚠ The pin is `deploy.resources.reservations.devices[].device_ids: ["3"]`,
+| GPU | committed `--gpu-memory-utilization` | spare |
-the fleet convention — **not** `count: all`, which is what the dead on-host
+|---|---|---|
-stub used and which would have handed this seat all four cards. Inside the
+| **0** | 0.40 + 0.48 = **0.88** | ~13 GB ← here |
-container the pinned card presents as `cuda:0`, which is what sherpa-onnx's
+| 1 | 0.52+0.24+0.10+0.055+0.03+0.03 = **0.975** | ~4.3 GB |
-CUDA execution provider takes by default.
+| 2 | **0.96** | ~1.8 GB |
+| 3 | — | *kept empty* |
+
+It was first placed on the empty GPU 3, which was wrong: **vLLM sizes its KV
+cache as a fraction of TOTAL VRAM, not free VRAM**, so any resident tenant on an
+otherwise-clean card eats directly into the profiling margin of whatever big seat
+lands there later — and `flash-next` needs 93 GiB of 96. A 96 GB card at 2 MiB is
+worth far more than 800 MiB of it. Moved to GPU 0 the same night.
+
+⚠ The pin is `deploy.resources.reservations.devices[].device_ids`, the fleet
+convention — **not** `count: all`, which is what the dead on-host stub used and
+which would have handed this seat all four cards. Inside the container the pinned
+card presents as `cuda:0`, which is what sherpa-onnx's CUDA execution provider
+takes by default.
 
 ## Why not the FastAPI community wrappers
 
@@ -70,13 +81,14 @@ The honest check is to watch the card while a transcription runs:
 ```bash
 # on fv-ml1 — terminal 1
 watch -n0.2 'nvidia-smi --query-compute-apps=pid,process_name,used_memory \
-    --format=csv -i 3'
+    --format=csv -i 0'
 
 # terminal 2 — send real audio, not silence
 curl -s -F file=@sample.wav http://127.0.0.1:8300/v1/audio/transcriptions
 ```
 
-A process must appear **on GPU 3** for the duration. If GPU 3 stays empty, the
+A process must appear **on GPU 0** for the duration — ~800 MiB alongside the two
+much larger `VLLM::EngineCore` entries. If it never appears, the
 CUDA EP did not initialise and you are on CPU regardless of what `.env` says.
 Confirm with the container's own startup log, which names the providers ORT
 actually registered:

stacks/parakeet/compose.yaml

@@ -6,16 +6,23 @@
 # prebuilt int8 quantized Parakeet-TDT from k2-fsa — and wrote our own ~50-line
 # wrapper we own end-to-end.
 #
-# HOST: fv-ml1, GPU 3 (relocated from irv-ml1 2026-09-15). GPU 3 is the utility
+# HOST: fv-ml1, GPU 0 (relocated from irv-ml1 2026-09-15).
-# card — the other three carry the vLLM serving seats and run 85-98% full, so a
-# seat placed anywhere else would fight them for VRAM.
 #
-# ⚠ GPU pin is `deploy.resources.reservations.devices[].device_ids`, the fleet
+# ⚠ GPU 0, NOT the empty GPU 3. This seat holds ~800 MiB — under 1% of a 96 GB
+# card — so it rides on the card with the most uncommitted headroom rather than
+# putting the first fingerprint on a pristine one. GPU 0's seats commit
+# util 0.40 + 0.48 = 0.88, leaving ~13 GB; GPU 1 is at 0.975 and GPU 2 at 0.96,
+# both too tight. Keeping GPU 3 at 2 MiB means it can still take a full-card seat
+# (flash-next needs 93 GiB) without a neighbour eating its profiling margin —
+# vLLM sizes KV cache as a fraction of TOTAL VRAM, not free VRAM, so a resident
+# tenant on an otherwise-empty card is worth more than its megabytes suggest.
+#
+# ⚠ The pin is `deploy.resources.reservations.devices[].device_ids`, the fleet
 # convention — NOT `runtime: nvidia` + NVIDIA_VISIBLE_DEVICES, and NOT
 # `count: all` (which is what the dead on-host stub did, and would have let this
-# tiny ASR seat see all four cards including the three that are full).
+# tiny ASR seat see all four cards). device_ids presents the pinned card as
-# device_ids ["3"] presents that card as cuda:0 INSIDE the container, which is
+# cuda:0 INSIDE the container, which is what sherpa-onnx's
-# what sherpa-onnx's CUDAExecutionProvider takes by default.
+# CUDAExecutionProvider takes by default.
 #
 # Model weights (~460 MB int8) download on first run via the entrypoint to
 # ${PARAKEET_MODELS_DIR}/ (persistent host bind mount). Subsequent starts skip
@@ -51,7 +58,7 @@ services:
         reservations:
           devices:
             - driver: nvidia
-              device_ids: ["${PARAKEET_GPU:-3}"]
+              device_ids: ["${PARAKEET_GPU:-0}"]
               capabilities: [gpu]
     networks:
       - tnet
@@ -67,7 +74,7 @@ services:
       - homepage.group=AI - Audio Tools
       - homepage.name=Parakeet ASR
       - homepage.icon=mdi-microphone
-      - homepage.description=Parakeet-TDT speech-to-text via sherpa-onnx (fv-ml1 GPU 3)
+      - homepage.description=Parakeet-TDT speech-to-text via sherpa-onnx (fv-ml1 GPU 0)
       - homepage.href=http://10.251.50.54:${PARAKEET_PORT}
 
 networks: