feat(coder-seat): Qwen2.5-Coder-1.5B copy on nh3-ml1 (not yet in the gateway)

Same model revision (df3ce67, blob sha matches fv-ml1), vLLM v0.24.0 and flags;
0.33 of the Ada (~5.4 GB, fv-ml1's absolute budget). Healthy, KV 79,808 tokens.

Parity (teacher-forced true code, 40 FIM prompts, 2,215 tokens, cache-salted):
each host is bit-exact with itself; cross-host |dlogprob| median 0.049, top-1
agreement 0.966 (Blackwell vs Ada + fp8 KV); against ground truth no quality
difference (mean logprob diff +0.008 +/- 0.019 SE). Speed on-box: ~5x slower
(64-tok FIM p50 ~1.0 s vs ~0.2 s; 63 vs 338 tok/s). Gateway coder-fast left on
fv-ml1 pending Prime's call.
This commit is contained in:
vh
2026-09-25 23:56:34 -07:00
parent c923727cf2
commit 7f066a4b79
4 changed files with 164 additions and 0 deletions
+23
View File
@@ -0,0 +1,23 @@
# coder-seat tunables (nh3-ml1). Copy to `.env` on the server.
# Same vLLM as fv-ml1's seat, so the move changes only the GPU.
VLLM_VERSION=v0.24.0
# Same port as on fv-ml1, so the gateway entry changes only its host.
CODER_PORT=8020
CODER_MODEL=Qwen/Qwen2.5-Coder-1.5B
CODER_REVISION=df3ce67c0e24480f20468b6ef2894622d69eb73b
CODER_SERVED_NAME=qwen2.5-coder-1.5b
CODER_MAX_MODEL_LEN=16384
CODER_MAX_NUM_SEQS=32
CODER_KV_CACHE_DTYPE=fp8
# Fraction of the Ada's TOTAL 16,380 MiB: 0.33 ≈ 5.4 GB, the same absolute
# budget fv-ml1 gave it (0.055 of 97,887 MiB). Weights ~3.1 GB bf16; fp8 KV is
# ~14 KB/token, so the rest holds far more than one 16k sequence. vLLM checks
# it against FREE memory at start; TEI holds ~2.6 GB of the card.
CODER_GPU_MEM_UTIL=0.33
# Matches fv-ml1: no key (internal network; the gateway fronts it).
API_KEY=