vh
c08992f95f
The :8033 seat now serves h2oai/h2o-lightning-4b @ b43f759b on vLLM 0.30.0 with byte-pinned KV (4.48 GiB, 135,558 tokens) behind services/h2o-decision-front: 32k refuse-not-trim admission, the seat's bearer token, and /decide + /decide/shared translated onto /v1/systemone for albok-nemi's judge. URL, port, token and stack name are unchanged. INC down 13:08:18, h2o healthy 13:11:27. Post-checks on the live seat: /health at the pinned revision; JevBench public 205/231 (= bench); 422 over the cap; 16q at the cap warm 2.0-2.16 s; seat peak 16,164 MiB under 4 concurrent 16q x 32k calls against a 16,847 budget. compose.inc.yaml + rollback-to-inc.sh restore Intern-Decision-4B in one step. The temporary GPU 3 eval seat (stacks/h2o-eval) is down.
17 lines
1.3 KiB
Markdown
17 lines
1.3 KiB
Markdown
# h2o-eval (TEMPORARY) — fv-ml1 GPU 3, :8034
|
|
|
|
> **DOWN since 2026-10-12 1319 PT** (`docker compose down`), once the GPU 1 cutover's post-checks passed. The host keeps
|
|
> `/opt/docker/compose/h2o-eval` and the weights, so it can come back on GPU 3 for a transient test with `docker compose up -d`.
|
|
|
|
h2o-lightning-4b @ b43f759b behind `services/h2o-decision-front` (copied into `conf/front/`), for albok-dev's
|
|
re-validation of Nemi's sensitivity judge before the GPU 1 cutover (Prime 2026-10-12 via brokkr). **Remove right after
|
|
the cutover:** GPU 3 is for transient work only (Scriberr and MIA use it).
|
|
|
|
- Weights: `/tank/aimodels/h2o-lightning-4b-b43f759b` (copied from the bench kit; model.safetensors sha256
|
|
`2d51561f221e1549d91bd1c38b3512a1435dd32950d454a9f040f1d4d11497b6`, tokenizer.json `5f9e4d49…`, shim git oid
|
|
`fee7807e…`, serve_config `5e6fef1c…`: all equal to Hugging Face at b43f759b).
|
|
- vLLM exactly as benched: `--max-model-len 40960 --max-num-seqs 16 --gpu-memory-utilization 0.1506`, vision limits as
|
|
the vendor's serve.sh; healthy ~3 min after `up` (graph capture), ~14.4 GB on GPU 3.
|
|
- `.env` on the host: `FRONT_TOKEN` = intern-decision's token (= vault `albok/judge-token`).
|
|
- Remove: `cd /opt/docker/compose/h2o-eval && docker compose down` (the weights stay for the GPU 1 seat).
|