feat(reward-seat): move Skywork reward seat from fv-ml1 to esh-ml1; audit finds nothing superseding it
- Audit: Skywork-Reward-V2-Llama-3.1-8B is still #1 of 188 on AllenAI's RewardBench 2 per-sample results; no Skywork V3; the -40M sibling is vendor-marked experimental. Our AWQ W4A16 quant: 0.847 vs published bf16 0.860 on a 150-prompt sample (within +/-2.9 pt SE), 96.2% pairwise agreement. Double BOS from vLLM on pre-templated text costs a further ~2.7 pts; callers must send add_special_tokens=false. - No working consumer: 0 requests since 2026-09-13; Worldtree Domari points at a dead IP with a non-vLLM schema (reported to worldtree-dev). - Move: sha256-identical model copy; vLLM v0.24.0 on esh-ml1 :8003 at 0.55 util (KV 1.30x of a 16k request). Parity vs fv-ml1: 149/150 verdicts, 99.8% pairwise signs, raw |delta| median 0.049. - Gateway /scalar-judge passthrough -> 10.0.50.80:8003; fv-ml1 vllm-reward removed (~10.2 GB freed on GPU 1). stacks/vllm now holds only vllm-coder.
This commit is contained in:
@@ -32,9 +32,14 @@ vLLM here after a bake-off; see
|
||||
| `tei-embed` | `Qwen/Qwen3-Embedding-0.6B` | 8001 | `qwen3-embedding` (`hosted_vllm/`, `/v1`) |
|
||||
| `tei-rerank` | `BAAI/bge-reranker-v2-m3` | 8013 | `reranker`, `reranker-a3-bge-v2-m3` (`huggingface/`, no `/v1`) |
|
||||
|
||||
VRAM ~2.6 GB for both; the rest of the 16 GB is free for further utility seats.
|
||||
The vLLM image (`vllm/vllm-openai:v0.24.0`, ~30 GB) is still on disk: kept in
|
||||
case a vLLM-only seat (the reward classifier) moves here.
|
||||
VRAM ~2.6 GB for both.
|
||||
|
||||
**Also here since 2026-09-25: the reward seat** — `stacks/reward-seat`,
|
||||
`vllm-reward` (Skywork-Reward-V2-Llama-3.1-8B AWQ, vLLM v0.24.0) on `:8003`, behind
|
||||
the gateway passthrough `/scalar-judge`. It is the one vLLM seat on the box,
|
||||
because TEI cannot serve a Llama classifier. Audit and parity:
|
||||
[`stacks/reward-seat/README.md`](../../stacks/reward-seat/README.md). GPU total
|
||||
with all three: ~10.4 of 16.4 GB.
|
||||
|
||||
**Cut-over verified 2026-09-25** through the gateway against fv-ml1's vLLM seats
|
||||
just before they were retired: embed cosine median 0.999927 / min 0.999881
|
||||
|
||||
Reference in New Issue
Block a user