Files

T

vh 2c3ef7a8af feat(h2o): temporary h2o-lightning-4b eval seat on fv-ml1 GPU 3 + the seat front (admission, bearer, /decide/shared translator)

Prime's call via brokkr: h2o-lightning-4b replaces the intern-decision seat once
albok-dev re-validates Nemi's sensitivity judge on it. services/h2o-decision-front
builds on infra-hermes's admission proxy (30f36f2): 32,000-token refuse-not-trim,
the seat's bearer token, and intern-decision-serve's /decide/shared mapping onto
/v1/systemone so the judge runs unchanged. stacks/h2o-eval serves it on GPU 3
:8034 exactly as benched (vLLM 0.30.0 + unmodified vendor shim; weights hashed
against HF b43f759b). INC keeps serving production on GPU 1.

2026-10-12 13:01:53 -07:00

1.6 KiB

Raw Blame History

h2o decision front

The front of the h2o-lightning-4b decision seat on fv-ml1 (seat swap ordered by Prime 2026-10-12, through brokkr). front.py sits in front of the vendor's unmodified h2o_lightning_shim.py, which sits in front of stock vLLM 0.30.0. It adds what the seat contract needs and the vendor stack lacks:

Admission (infra-hermes's tools/h2o-admission/admission_proxy.py @ 30f36f2, logic unchanged): count every decision request with h2o's own tokenizer over the shim's own renders, and refuse over 32,000 tokens with 422 + the count. The shim would otherwise trim silently.
The seat's bearer token on everything but GET /health, as intern-decision-serve has.
/decide and /decide/shared continuity for albok-nemi's sensitivity judge: intern-decision-serve's own mapping (one Jev choice question per decision, calls of at most 16, fields q/q1..qN), re-keyed into the semif result shape (results[i].native.probabilities[option id]). orderings other than none → 422.
/health with model: {name, source, revision}, which the judge stamps into its cache key.

run.sh runs the shim (loopback) and the front in one container (python:3.12-slim, host network). Stacks copy both files into conf/front/: stacks/h2o-eval (TEMPORARY, GPU 3, :8034) today.

Verified 2026-10-12 1300 on the eval seat: 401 without the token; /decide/shared in the judge's shape; a planted personal record → personal p_yes .911, a neutral note → .056 (n=1 each, smoke test only); 40,088 tokens → 422 with the count; /v1/systemone passthrough with _admission counts; reachable from nh3-dev.

1.6 KiB Raw Blame History

h2o decision front

1.6 KiB

Raw Blame History