feat(h2o-eval): round 2 eval seat on GPU 3 with floor-on/floor-off fronts for the judge investigation
Prime (via albok-dev): understand why h2o failed and re-fit the judge. One vLLM on GPU 3 behind two fronts: :8034 with the shim's yes/no floor at 0.801 exactly as the cutover ran (same judge stamp), :8035 with SHIM_NOUL_FLOOR=0 and a distinct stamp (h2o-lightning-4b+noul-floor-0). Both refuse image URLs; the vLLM --allowed-media-domains backstop is now verified (400, no fetch).
This commit is contained in:
@@ -1,10 +1,17 @@
|
||||
# TEMPORARY h2o-lightning-4b decision seat on fv-ml1 GPU 3 (Prime 2026-10-12 via brokkr: "stand up the h2o copy on
|
||||
# gpu 3 if you need to test, it should replace the live seat as soon as possible and keep gpu3 only for transient
|
||||
# tasks"). It exists so albok-dev can re-validate Nemi's sensitivity judge on h2o before the GPU 1 cutover; REMOVE it
|
||||
# right after the cutover. INC (stacks/intern-decision) keeps serving production on GPU 1 meanwhile.
|
||||
# TRANSIENT h2o-lightning-4b eval seat on fv-ml1 GPU 3 (GPU 3 is for transient work only).
|
||||
# Round 1, 2026-10-12 1300-1319: albok-dev validated Nemi's judge on h2o before the GPU 1 cutover (Prime via brokkr).
|
||||
# Round 2, from 2026-10-12 ~1430 (Prime 1420, via albok-dev): "I'd like to understand why h2o failed, so stand it up
|
||||
# on gpu3 and refit it and let's get an understanding." albok-dev investigates the 154 texts h2o alone called adult
|
||||
# and re-fits the judge. Two fronts over ONE vLLM:
|
||||
# :8034 front the shim's yes/no floor ON (0.801), exactly as the live cutover ran; /health model stamp
|
||||
# h2o-lightning-4b@b43f759b..., identical to the cutover's
|
||||
# :8035 front-nofloor SHIM_NOUL_FLOOR=0 (floor OFF, brokkr's lever); stamp h2o-lightning-4b+noul-floor-0@b43f759b...
|
||||
# Each /health also reports backend.noul_floor {value, source}. NO load tests or other traffic while albok-dev's
|
||||
# runs are in (they name start/end on thread 01M4S026GWS354WZGXZ5E34F31). INC keeps :8033 on GPU 1, untouched.
|
||||
# Serving exactly as benched (docs/pfi/decision-sweep-followon-bench-2026-10-12.md, Amendment 1): stock vLLM 0.30.0
|
||||
# + the vendor's unmodified shim + the admission front (services/h2o-decision-front, copied into conf/front).
|
||||
# Weights: /tank/aimodels/h2o-lightning-4b-b43f759b (hashes match HF at b43f759b; see the stack README).
|
||||
# Inline images only (Prime 2026-10-12): LIGHTNING_IMAGE_URLS=off on both fronts + vLLM --allowed-media-domains.
|
||||
name: h2o-eval
|
||||
services:
|
||||
vllm:
|
||||
@@ -54,5 +61,30 @@ services:
|
||||
- /tank/aimodels/h2o-lightning-4b-b43f759b:/model:ro
|
||||
- /opt/docker/conf/h2o-eval/front:/front:ro
|
||||
command: ["bash", "/front/run.sh"]
|
||||
front-nofloor:
|
||||
image: python:3.12-slim
|
||||
container_name: h2o-eval-front-nofloor
|
||||
restart: unless-stopped
|
||||
network_mode: host
|
||||
depends_on: [vllm]
|
||||
env_file: .env
|
||||
environment:
|
||||
SHIM_VLLM: http://127.0.0.1:18199
|
||||
SHIM_PORT: "18192"
|
||||
SHIM_NOUL_FLOOR: "0" # the floor OFF (the shim: 0 turns it off)
|
||||
LIGHTNING_IMAGE_URLS: "off" # Prime 2026-10-12: inline images only, the seat never fetches an image URL
|
||||
FRONT_SHIM: http://127.0.0.1:18192
|
||||
FRONT_VLLM: http://127.0.0.1:18199
|
||||
FRONT_BIND: 0.0.0.0
|
||||
FRONT_PORT: "8035"
|
||||
FRONT_CAP: "32000"
|
||||
FRONT_SHIM_FILE: /model/h2o_lightning_shim.py
|
||||
FRONT_SERVE_CONFIG: /model/serve_config.json
|
||||
FRONT_MODEL_NAME: h2o-lightning-4b+noul-floor-0 # a distinct judge stamp
|
||||
FRONT_MODEL_REVISION: b43f759b6385d21182f87c3829931ec3d4dfa98a
|
||||
volumes:
|
||||
- /tank/aimodels/h2o-lightning-4b-b43f759b:/model:ro
|
||||
- /opt/docker/conf/h2o-eval/front:/front:ro
|
||||
command: ["bash", "/front/run.sh"]
|
||||
volumes:
|
||||
h2o-eval-vllm-cache:
|
||||
|
||||
Reference in New Issue
Block a user