revert(coder-seat): keep Qwen2.5-Coder on fv-ml1; nh3-ml1 copy removed (5x slower, same quality)
This commit is contained in:
@@ -33,12 +33,13 @@ Prime's call (recommendation: load-share; see below).
|
||||
|
||||
VRAM ~2.7 GB for both, so ~13 GB is free.
|
||||
|
||||
**Also here since 2026-09-25: a copy of the code-completion seat.**
|
||||
`stacks/coder-seat`, `vllm-coder` (Qwen2.5-Coder-1.5B, vLLM v0.24.0) on `:8020`,
|
||||
about 5 GB. It is **not behind the gateway yet**: on this card it runs about 5×
|
||||
slower than on fv-ml1 (a 64-token FIM completion takes ~1.0 s vs ~0.2 s), so
|
||||
Prime is deciding. The GPU total with TEI is ~7.6 of 16 GB. Details:
|
||||
`stacks/coder-seat/README.md`.
|
||||
**Tried and removed (2026-09-25):** a copy of the code-completion seat
|
||||
(Qwen2.5-Coder-1.5B). It gave the same quality but ran about 5× slower than on
|
||||
fv-ml1 (a 64-token FIM completion took ~1.0 s vs ~0.2 s), so it stays on fv-ml1
|
||||
(Prime). The recipe and measurements are in `stacks/coder-seat/README.md`.
|
||||
**Lesson for the next candidate:** this card has about a fifth of a Blackwell's
|
||||
decode speed on small generative models, so it suits embedding, reranking and
|
||||
classification, not latency-sensitive generation.
|
||||
|
||||
## Parity and speed vs esh-ml1 (2026-09-25 ~1540 PT)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user