revert(coder-seat): keep Qwen2.5-Coder on fv-ml1; nh3-ml1 copy removed (5x slower, same quality)

This commit is contained in:
vh
2026-09-25 23:58:08 -07:00
parent 7f066a4b79
commit ddd67df2e9
3 changed files with 15 additions and 10 deletions
+7 -6
View File
@@ -33,12 +33,13 @@ Prime's call (recommendation: load-share; see below).
VRAM ~2.7 GB for both, so ~13 GB is free.
**Also here since 2026-09-25: a copy of the code-completion seat.**
`stacks/coder-seat`, `vllm-coder` (Qwen2.5-Coder-1.5B, vLLM v0.24.0) on `:8020`,
about 5 GB. It is **not behind the gateway yet**: on this card it runs about 5×
slower than on fv-ml1 (a 64-token FIM completion takes ~1.0 s vs ~0.2 s), so
Prime is deciding. The GPU total with TEI is ~7.6 of 16 GB. Details:
`stacks/coder-seat/README.md`.
**Tried and removed (2026-09-25):** a copy of the code-completion seat
(Qwen2.5-Coder-1.5B). It gave the same quality but ran about 5× slower than on
fv-ml1 (a 64-token FIM completion took ~1.0 s vs ~0.2 s), so it stays on fv-ml1
(Prime). The recipe and measurements are in `stacks/coder-seat/README.md`.
**Lesson for the next candidate:** this card has about a fifth of a Blackwell's
decode speed on small generative models, so it suits embedding, reranking and
classification, not latency-sensitive generation.
## Parity and speed vs esh-ml1 (2026-09-25 ~1540 PT)