feat(intern-decision): decision seat swapped to h2o-lightning-4b on fv-ml1 GPU 1; eval seat down
The :8033 seat now serves h2oai/h2o-lightning-4b @ b43f759b on vLLM 0.30.0 with byte-pinned KV (4.48 GiB, 135,558 tokens) behind services/h2o-decision-front: 32k refuse-not-trim admission, the seat's bearer token, and /decide + /decide/shared translated onto /v1/systemone for albok-nemi's judge. URL, port, token and stack name are unchanged. INC down 13:08:18, h2o healthy 13:11:27. Post-checks on the live seat: /health at the pinned revision; JevBench public 205/231 (= bench); 422 over the cap; 16q at the cap warm 2.0-2.16 s; seat peak 16,164 MiB under 4 concurrent 16q x 32k calls against a 16,847 budget. compose.inc.yaml + rollback-to-inc.sh restore Intern-Decision-4B in one step. The temporary GPU 3 eval seat (stacks/h2o-eval) is down.
This commit is contained in:
Executable
+12
@@ -0,0 +1,12 @@
|
||||
#!/bin/bash
|
||||
# One container: the unmodified vendor shim on loopback, then the front. If either dies, the container exits and
|
||||
# Docker restarts it (restart: unless-stopped). The shim's own SHIM_* env (SHIM_NOUL_FLOOR etc.) passes through.
|
||||
set -eu
|
||||
python3 /model/h2o_lightning_shim.py --config /model/serve_config.json --vllm "$SHIM_VLLM" \
|
||||
--host 127.0.0.1 --port "$SHIM_PORT" &
|
||||
SHIM=$!
|
||||
python3 /front/front.py &
|
||||
FRONT=$!
|
||||
wait -n "$SHIM" "$FRONT" 2>/dev/null || true
|
||||
kill "$SHIM" "$FRONT" 2>/dev/null || true
|
||||
exit 1
|
||||
Reference in New Issue
Block a user