feat(reward-seat): move Skywork reward seat from fv-ml1 to esh-ml1; audit finds nothing superseding it
- Audit: Skywork-Reward-V2-Llama-3.1-8B is still #1 of 188 on AllenAI's RewardBench 2 per-sample results; no Skywork V3; the -40M sibling is vendor-marked experimental. Our AWQ W4A16 quant: 0.847 vs published bf16 0.860 on a 150-prompt sample (within +/-2.9 pt SE), 96.2% pairwise agreement. Double BOS from vLLM on pre-templated text costs a further ~2.7 pts; callers must send add_special_tokens=false. - No working consumer: 0 requests since 2026-09-13; Worldtree Domari points at a dead IP with a non-vLLM schema (reported to worldtree-dev). - Move: sha256-identical model copy; vLLM v0.24.0 on esh-ml1 :8003 at 0.55 util (KV 1.30x of a 16k request). Parity vs fv-ml1: 149/150 verdicts, 99.8% pairwise signs, raw |delta| median 0.049. - Gateway /scalar-judge passthrough -> 10.0.50.80:8003; fv-ml1 vllm-reward removed (~10.2 GB freed on GPU 1). stacks/vllm now holds only vllm-coder.
This commit is contained in:
@@ -14,7 +14,7 @@ VLLM_VERSION=v0.24.0
|
||||
# Host ports (container always listens on 8000 internally)
|
||||
# (EMBED_PORT 8001 / RERANK_PORT 8013 retired 2026-09-25 with their seats —
|
||||
# embedding + reranking moved to TEI on esh-ml1, stacks/embed-rerank.)
|
||||
REWARD_PORT=8003
|
||||
# REWARD_PORT=8003 (retired here 2026-09-25 — the reward seat moved to esh-ml1, stacks/reward-seat)
|
||||
|
||||
# GPU assignment — all services share this GPU
|
||||
# (ana-ml2 has 0 and 1; default 1 keeps 0 free for heavy LLM work in llama-swap)
|
||||
@@ -54,13 +54,13 @@ GPU_ID=1
|
||||
# each at 0.05 (mostly util-reservation waste); 0.03 (~3.6 GB) fits weights +
|
||||
# CUDA context with room, freeing ~4 GB back to granite. Recreate them ONE AT A
|
||||
# TIME — concurrent recreate races the memory-profiling assertion.
|
||||
REWARD_GPU_MEM_UTIL=0.10
|
||||
# REWARD_GPU_MEM_UTIL=0.10 (retired here 2026-09-25 — the reward seat moved to esh-ml1, stacks/reward-seat)
|
||||
|
||||
# Context length caps — lower these if VRAM is tight.
|
||||
# Qwen3-Embedding supports up to 32k; reranker up to 32k.
|
||||
# Skywork capped at 16k server-side as defense-in-depth; JudgeClient also
|
||||
# enforces the cap at dispatch time per spec.
|
||||
REWARD_MAX_MODEL_LEN=16384
|
||||
# REWARD_MAX_MODEL_LEN=16384 (retired here 2026-09-25 — the reward seat moved to esh-ml1, stacks/reward-seat)
|
||||
|
||||
# Optional API key — leave blank for no auth (fine on the internal network).
|
||||
# If set, all three services require `Authorization: Bearer <key>`.
|
||||
|
||||
Reference in New Issue
Block a user