vh
2c3ef7a8af
Prime's call via brokkr: h2o-lightning-4b replaces the intern-decision seat once
albok-dev re-validates Nemi's sensitivity judge on it. services/h2o-decision-front
builds on infra-hermes's admission proxy (30f36f2): 32,000-token refuse-not-trim,
the seat's bearer token, and intern-decision-serve's /decide/shared mapping onto
/v1/systemone so the judge runs unchanged. stacks/h2o-eval serves it on GPU 3
:8034 exactly as benched (vLLM 0.30.0 + unmodified vendor shim; weights hashed
against HF b43f759b). INC keeps serving production on GPU 1.
22 lines
1.6 KiB
Markdown
22 lines
1.6 KiB
Markdown
# h2o decision front
|
|
|
|
The front of the h2o-lightning-4b decision seat on fv-ml1 (seat swap ordered by Prime 2026-10-12, through brokkr).
|
|
`front.py` sits in front of the vendor's **unmodified** `h2o_lightning_shim.py`, which sits in front of stock vLLM
|
|
0.30.0. It adds what the seat contract needs and the vendor stack lacks:
|
|
|
|
- **Admission** (infra-hermes's `tools/h2o-admission/admission_proxy.py` @ 30f36f2, logic unchanged): count every
|
|
decision request with h2o's own tokenizer over the shim's own renders, and refuse over 32,000 tokens with 422 + the
|
|
count. The shim would otherwise trim silently.
|
|
- **The seat's bearer token** on everything but `GET /health`, as intern-decision-serve has.
|
|
- **`/decide` and `/decide/shared` continuity** for albok-nemi's sensitivity judge: intern-decision-serve's own
|
|
mapping (one Jev `choice` question per decision, calls of at most 16, fields `q`/`q1..qN`), re-keyed into the semif
|
|
result shape (`results[i].native.probabilities[option id]`). `orderings` other than `none` → 422.
|
|
- **`/health`** with `model: {name, source, revision}`, which the judge stamps into its cache key.
|
|
|
|
`run.sh` runs the shim (loopback) and the front in one container (`python:3.12-slim`, host network). Stacks copy both
|
|
files into `conf/front/`: `stacks/h2o-eval` (TEMPORARY, GPU 3, :8034) today.
|
|
|
|
Verified 2026-10-12 1300 on the eval seat: 401 without the token; `/decide/shared` in the judge's shape; a planted
|
|
personal record → `personal` p_yes .911, a neutral note → .056 (n=1 each, smoke test only); 40,088 tokens → 422 with
|
|
the count; `/v1/systemone` passthrough with `_admission` counts; reachable from nh3-dev.
|