revert(coder-seat): keep Qwen2.5-Coder on fv-ml1; nh3-ml1 copy removed (5x slower, same quality)
This commit is contained in:
@@ -203,6 +203,7 @@ in `servers/pfi-gx10/README.md`. → `persistent-memory.d/2026-09-24-gx10-ac-res
|
||||
|
||||
## Recent decisions
|
||||
|
||||
- `[2026-09-26]` **Coder seat STAYS on fv-ml1 (Prime).** The nh3-ml1 copy gave the same quality (teacher-forced true-code logprob diff +0.008 ± 0.019) but ran ~5× slower (64-tok FIM ~1.0 s vs 0.2 s; 63 vs 338 tok/s), and freeing 6.3 GB on fv-ml1 GPU 1 (20 GB spare) bought little. Copy removed; recipe kept in `stacks/coder-seat/`. The RTX 2000E suits embed/rerank/classify, not latency-sensitive generation. Only coder was a candidate: parakeet stays (earlier ruling), voices is a generation seat.
|
||||
- `[2026-09-25]` **nh3-ml1 LIVE after the NH3 visit (SB off, IGFX restored): driver + CT 109 + TEI, parity vs esh-ml1 indistinguishable (cos min 0.999993 = own floor; overlap@10 1.000 vs MRL-256 control 0.684), same speed; monitoring/DNS wired. Autonomous: lxc-pve 6.0.0-2 upgrade on nh3-pve (Docker-in-LXC fix #7006). Gateway routing + AMT follow-up are Prime's.** → `persistent-memory.d/2026-09-25-nh3-ml1-live.md`
|
||||
- `[2026-09-25]` **nh3-ml1 = CT 109 @ 10.100.50.80, second TEI embed/rerank backend on nh3-pve's RTX 2000E; GPU playbooks made host-generic — BUILD DEFERRED: blocked on nh3-pve Secure Boot until the NH3 visit; SB handling and gateway routing are Prime's calls.** Tracked: `7ddd116`, Current state. → `persistent-memory.d/2026-09-25-nh3-ml1-standup.md`
|
||||
- `[2026-09-25]` **nh3-pve OOB: HOLD blind until the next NH3 visit (Prime via Miranda, 1333).** Target: NanoKVM → pfi-gx10, and the MS-01 on its own vPro. The IGFX BIOS fix rides the same visit, since AMT KVM captures only the iGPU. Tracked: `servers/nh3-pve/README.md` (`2b49be4`, `cdd7605`).
|
||||
|
||||
@@ -33,12 +33,13 @@ Prime's call (recommendation: load-share; see below).
|
||||
|
||||
VRAM ~2.7 GB for both, so ~13 GB is free.
|
||||
|
||||
**Also here since 2026-09-25: a copy of the code-completion seat.**
|
||||
`stacks/coder-seat`, `vllm-coder` (Qwen2.5-Coder-1.5B, vLLM v0.24.0) on `:8020`,
|
||||
about 5 GB. It is **not behind the gateway yet**: on this card it runs about 5×
|
||||
slower than on fv-ml1 (a 64-token FIM completion takes ~1.0 s vs ~0.2 s), so
|
||||
Prime is deciding. The GPU total with TEI is ~7.6 of 16 GB. Details:
|
||||
`stacks/coder-seat/README.md`.
|
||||
**Tried and removed (2026-09-25):** a copy of the code-completion seat
|
||||
(Qwen2.5-Coder-1.5B). It gave the same quality but ran about 5× slower than on
|
||||
fv-ml1 (a 64-token FIM completion took ~1.0 s vs ~0.2 s), so it stays on fv-ml1
|
||||
(Prime). The recipe and measurements are in `stacks/coder-seat/README.md`.
|
||||
**Lesson for the next candidate:** this card has about a fifth of a Blackwell's
|
||||
decode speed on small generative models, so it suits embedding, reranking and
|
||||
classification, not latency-sensitive generation.
|
||||
|
||||
## Parity and speed vs esh-ml1 (2026-09-25 ~1540 PT)
|
||||
|
||||
|
||||
@@ -17,11 +17,14 @@ Blackwells). It was chosen for Zed edit-predictions by a deep-research pass on
|
||||
| revision | `df3ce67c0e24480f20468b6ef2894622d69eb73b` (pinned; same as fv-ml1 and HF main on 2026-09-25) |
|
||||
| memory | 0.33 of 16,380 MiB ≈ 5.4 GB, the same absolute budget as on fv-ml1 |
|
||||
|
||||
## ⏸ Status 2026-09-25 2355: running, NOT behind the gateway (Prime deciding)
|
||||
## ❌ NOT DEPLOYED. Decision (Prime, 2026-09-26 0000): the seat stays on fv-ml1
|
||||
|
||||
The copy is up and parity-checked, but the gateway still points `coder-fast` at
|
||||
fv-ml1. On the 50 W Ada the same model is **about 5× slower**, which Zed
|
||||
edit-predictions would notice.
|
||||
The nh3-ml1 copy was built, checked for parity, measured, and then removed (container,
|
||||
image, model cache). It was the same quality but **about 5× slower** on the 50 W
|
||||
Ada. That lag would reach Zed edit-predictions, and moving it would only have freed
|
||||
6.3 GB on an fv-ml1 GPU that already had about 20 GB spare. The live seat is still
|
||||
`stacks/vllm` on fv-ml1. This stack is kept as the tested recipe in case the
|
||||
calculation changes (a faster card, or speculative decoding).
|
||||
|
||||
**Speed, measured on each box** (3 interleaved reps each; fv-ml1 GPU 1 read 0%
|
||||
before and after):
|
||||
|
||||
Reference in New Issue
Block a user