From ddd67df2e99d661b85a4818dc3a8e85ca2166a59 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Fri, 25 Sep 2026 23:58:08 -0700 Subject: [PATCH] revert(coder-seat): keep Qwen2.5-Coder on fv-ml1; nh3-ml1 copy removed (5x slower, same quality) --- persistent-memory.md | 1 + servers/nh3-ml1/README.md | 13 +++++++------ stacks/coder-seat/README.md | 11 +++++++---- 3 files changed, 15 insertions(+), 10 deletions(-) diff --git a/persistent-memory.md b/persistent-memory.md index a74457e..0f4f070 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -203,6 +203,7 @@ in `servers/pfi-gx10/README.md`. → `persistent-memory.d/2026-09-24-gx10-ac-res ## Recent decisions +- `[2026-09-26]` **Coder seat STAYS on fv-ml1 (Prime).** The nh3-ml1 copy gave the same quality (teacher-forced true-code logprob diff +0.008 ± 0.019) but ran ~5× slower (64-tok FIM ~1.0 s vs 0.2 s; 63 vs 338 tok/s), and freeing 6.3 GB on fv-ml1 GPU 1 (20 GB spare) bought little. Copy removed; recipe kept in `stacks/coder-seat/`. The RTX 2000E suits embed/rerank/classify, not latency-sensitive generation. Only coder was a candidate: parakeet stays (earlier ruling), voices is a generation seat. - `[2026-09-25]` **nh3-ml1 LIVE after the NH3 visit (SB off, IGFX restored): driver + CT 109 + TEI, parity vs esh-ml1 indistinguishable (cos min 0.999993 = own floor; overlap@10 1.000 vs MRL-256 control 0.684), same speed; monitoring/DNS wired. Autonomous: lxc-pve 6.0.0-2 upgrade on nh3-pve (Docker-in-LXC fix #7006). Gateway routing + AMT follow-up are Prime's.** → `persistent-memory.d/2026-09-25-nh3-ml1-live.md` - `[2026-09-25]` **nh3-ml1 = CT 109 @ 10.100.50.80, second TEI embed/rerank backend on nh3-pve's RTX 2000E; GPU playbooks made host-generic — BUILD DEFERRED: blocked on nh3-pve Secure Boot until the NH3 visit; SB handling and gateway routing are Prime's calls.** Tracked: `7ddd116`, Current state. → `persistent-memory.d/2026-09-25-nh3-ml1-standup.md` - `[2026-09-25]` **nh3-pve OOB: HOLD blind until the next NH3 visit (Prime via Miranda, 1333).** Target: NanoKVM → pfi-gx10, and the MS-01 on its own vPro. The IGFX BIOS fix rides the same visit, since AMT KVM captures only the iGPU. Tracked: `servers/nh3-pve/README.md` (`2b49be4`, `cdd7605`). diff --git a/servers/nh3-ml1/README.md b/servers/nh3-ml1/README.md index b8b971d..331cad0 100644 --- a/servers/nh3-ml1/README.md +++ b/servers/nh3-ml1/README.md @@ -33,12 +33,13 @@ Prime's call (recommendation: load-share; see below). VRAM ~2.7 GB for both, so ~13 GB is free. -**Also here since 2026-09-25: a copy of the code-completion seat.** -`stacks/coder-seat`, `vllm-coder` (Qwen2.5-Coder-1.5B, vLLM v0.24.0) on `:8020`, -about 5 GB. It is **not behind the gateway yet**: on this card it runs about 5× -slower than on fv-ml1 (a 64-token FIM completion takes ~1.0 s vs ~0.2 s), so -Prime is deciding. The GPU total with TEI is ~7.6 of 16 GB. Details: -`stacks/coder-seat/README.md`. +**Tried and removed (2026-09-25):** a copy of the code-completion seat +(Qwen2.5-Coder-1.5B). It gave the same quality but ran about 5× slower than on +fv-ml1 (a 64-token FIM completion took ~1.0 s vs ~0.2 s), so it stays on fv-ml1 +(Prime). The recipe and measurements are in `stacks/coder-seat/README.md`. +**Lesson for the next candidate:** this card has about a fifth of a Blackwell's +decode speed on small generative models, so it suits embedding, reranking and +classification, not latency-sensitive generation. ## Parity and speed vs esh-ml1 (2026-09-25 ~1540 PT) diff --git a/stacks/coder-seat/README.md b/stacks/coder-seat/README.md index 463c587..b06f5b1 100644 --- a/stacks/coder-seat/README.md +++ b/stacks/coder-seat/README.md @@ -17,11 +17,14 @@ Blackwells). It was chosen for Zed edit-predictions by a deep-research pass on | revision | `df3ce67c0e24480f20468b6ef2894622d69eb73b` (pinned; same as fv-ml1 and HF main on 2026-09-25) | | memory | 0.33 of 16,380 MiB ≈ 5.4 GB, the same absolute budget as on fv-ml1 | -## ⏸ Status 2026-09-25 2355: running, NOT behind the gateway (Prime deciding) +## ❌ NOT DEPLOYED. Decision (Prime, 2026-09-26 0000): the seat stays on fv-ml1 -The copy is up and parity-checked, but the gateway still points `coder-fast` at -fv-ml1. On the 50 W Ada the same model is **about 5× slower**, which Zed -edit-predictions would notice. +The nh3-ml1 copy was built, checked for parity, measured, and then removed (container, +image, model cache). It was the same quality but **about 5× slower** on the 50 W +Ada. That lag would reach Zed edit-predictions, and moving it would only have freed +6.3 GB on an fv-ml1 GPU that already had about 20 GB spare. The live seat is still +`stacks/vllm` on fv-ml1. This stack is kept as the tested recipe in case the +calculation changes (a faster card, or speculative decoding). **Speed, measured on each box** (3 interleaved reps each; fv-ml1 GPU 1 read 0% before and after):