feat(h2o): temporary h2o-lightning-4b eval seat on fv-ml1 GPU 3 + the seat front (admission, bearer, /decide/shared translator)
Prime's call via brokkr: h2o-lightning-4b replaces the intern-decision seat once
albok-dev re-validates Nemi's sensitivity judge on it. services/h2o-decision-front
builds on infra-hermes's admission proxy (30f36f2): 32,000-token refuse-not-trim,
the seat's bearer token, and intern-decision-serve's /decide/shared mapping onto
/v1/systemone so the judge runs unchanged. stacks/h2o-eval serves it on GPU 3
:8034 exactly as benched (vLLM 0.30.0 + unmodified vendor shim; weights hashed
against HF b43f759b). INC keeps serving production on GPU 1.
This commit is contained in:
Executable
+12
@@ -0,0 +1,12 @@
|
||||
#!/bin/bash
|
||||
# One container: the unmodified vendor shim on loopback, then the front. If either dies, the container exits and
|
||||
# Docker restarts it (restart: unless-stopped). The shim's own SHIM_* env (SHIM_NOUL_FLOOR etc.) passes through.
|
||||
set -eu
|
||||
python3 /model/h2o_lightning_shim.py --config /model/serve_config.json --vllm "$SHIM_VLLM" \
|
||||
--host 127.0.0.1 --port "$SHIM_PORT" &
|
||||
SHIM=$!
|
||||
python3 /front/front.py &
|
||||
FRONT=$!
|
||||
wait -n "$SHIM" "$FRONT" 2>/dev/null || true
|
||||
kill "$SHIM" "$FRONT" 2>/dev/null || true
|
||||
exit 1
|
||||
Reference in New Issue
Block a user