vh
2c3ef7a8af
Prime's call via brokkr: h2o-lightning-4b replaces the intern-decision seat once
albok-dev re-validates Nemi's sensitivity judge on it. services/h2o-decision-front
builds on infra-hermes's admission proxy (30f36f2): 32,000-token refuse-not-trim,
the seat's bearer token, and intern-decision-serve's /decide/shared mapping onto
/v1/systemone so the judge runs unchanged. stacks/h2o-eval serves it on GPU 3
:8034 exactly as benched (vLLM 0.30.0 + unmodified vendor shim; weights hashed
against HF b43f759b). INC keeps serving production on GPU 1.
1.0 KiB
1.0 KiB
h2o-eval (TEMPORARY) — fv-ml1 GPU 3, :8034
h2o-lightning-4b @ b43f759b behind services/h2o-decision-front (copied into conf/front/), for albok-dev's
re-validation of Nemi's sensitivity judge before the GPU 1 cutover (Prime 2026-10-12 via brokkr). Remove right after
the cutover: GPU 3 is for transient work only (Scriberr and MIA use it).
- Weights:
/tank/aimodels/h2o-lightning-4b-b43f759b(copied from the bench kit; model.safetensors sha2562d51561f221e1549d91bd1c38b3512a1435dd32950d454a9f040f1d4d11497b6, tokenizer.json5f9e4d49…, shim git oidfee7807e…, serve_config5e6fef1c…: all equal to Hugging Face at b43f759b). - vLLM exactly as benched:
--max-model-len 40960 --max-num-seqs 16 --gpu-memory-utilization 0.1506, vision limits as the vendor's serve.sh; healthy ~3 min afterup(graph capture), ~14.4 GB on GPU 3. .envon the host:FRONT_TOKEN= intern-decision's token (= vaultalbok/judge-token).- Remove:
cd /opt/docker/compose/h2o-eval && docker compose down(the weights stay for the GPU 1 seat).