feat(h2o-eval): round 2 eval seat on GPU 3 with floor-on/floor-off fronts for the judge investigation

Prime (via albok-dev): understand why h2o failed and re-fit the judge. One
vLLM on GPU 3 behind two fronts: :8034 with the shim's yes/no floor at 0.801
exactly as the cutover ran (same judge stamp), :8035 with SHIM_NOUL_FLOOR=0 and
a distinct stamp (h2o-lightning-4b+noul-floor-0). Both refuse image URLs; the
vLLM --allowed-media-domains backstop is now verified (400, no fetch).

persistent-memory.md

@@ -169,7 +169,7 @@ Every vLLM seat on a shared card (GPU 0, GPU 1) has byte-pinned KV (`--kv-cache-
 ### The decision seat `intern-decision` :8033 (Intern-Decision-4B; h2o staged)
 
 - **LIVE: Intern-Decision-4B** (`intern-decision-serve:0.1.3`), again since 2026-10-12 1403. The only production caller is albok-nemi's sensitivity judge (`/decide/shared`, threshold 0.5). Run `scripts/intern-decision-warmup` after an IMAGE change. → `persistent-memory.d/2026-09-30-semif-replaced-by-intern-decision.md`
-- **h2o-lightning-4b is STAGED** (`stacks/intern-decision/compose.h2o.yaml` + `switch-to-h2o.sh`, with a GPU 1 free-memory gate; weights `/tank/aimodels/h2o-lightning-4b-b43f759b`; front `services/h2o-decision-front`). Inline images only (`LIGHTNING_IMAGE_URLS=off`, verified; vLLM `--allowed-media-domains media.invalid` backstop NOT yet exercised). **It returns only after albok-dev re-fits the judge for it** (a 4.4 contract revision on an eval seat). ⚠ Its seat headroom under 4-way 32k load was 683 MiB.
+- **h2o-lightning-4b is STAGED** (`stacks/intern-decision/compose.h2o.yaml` + `switch-to-h2o.sh`, with a GPU 1 free-memory gate; weights `/tank/aimodels/h2o-lightning-4b-b43f759b`; front `services/h2o-decision-front`). Inline images only (`LIGHTNING_IMAGE_URLS=off` and the vLLM `--allowed-media-domains media.invalid` backstop, both verified). **It returns only after albok-dev re-fits the judge for it** (a 4.4 contract revision on an eval seat). **Round 2 eval seat UP since 1428 on GPU 3** (Prime 1420: understand why h2o failed, re-fit): `:8034` floor on (stamp = the cutover's), `:8035` floor off (`h2o-lightning-4b+noul-floor-0`), one vLLM; no load tests while albok-dev runs; down when they're done. Thread `01M4S026GWS354WZGXZ5E34F31`. ⚠ Its seat headroom under 4-way 32k load was 683 MiB.
 - **Open:** label ~50 real Wyrd/Cicada turns before trusting INC in production; its card makes no contamination claim.
 
 

stacks/h2o-eval/README.md

@@ -1,7 +1,23 @@
 # h2o-eval (TEMPORARY) — fv-ml1 GPU 3, :8034
 
-> **DOWN since 2026-10-12 1319 PT** (`docker compose down`), once the GPU 1 cutover's post-checks passed. The host keeps
+> **UP again since 2026-10-12 1428 PT, round 2** (Prime 1420, via albok-dev): "I'd like to understand why h2o failed, so
-> `/opt/docker/compose/h2o-eval` and the weights, so it can come back on GPU 3 for a transient test with `docker compose up -d`.
+> stand it up on gpu3 and refit it and let's get an understanding." albok-dev runs the investigation and the judge
+> re-fit (thread `01M4S026GWS354WZGXZ5E34F31`). **No load tests or other traffic while their runs are in.** Round 1
+> (1300–1319) was the pre-cutover validation. Take it down with `docker compose down` when albok-dev is done.
+>
+> | port | front | floor | judge stamp (`/health` model) |
+> |---|---|---|---|
+> | `:8034` | `h2o-eval-front` | ON, 0.801, as the live cutover ran | `h2o-lightning-4b@b43f759b…`, identical to the cutover's |
+> | `:8035` | `h2o-eval-front-nofloor` | OFF (`SHIM_NOUL_FLOOR=0`) | `h2o-lightning-4b+noul-floor-0@b43f759b…` |
+>
+> Both fronts sit over ONE vLLM (14,420 MiB, KV 135,070 tokens, util-sized because GPU 3 is otherwise empty). Each
+> `/health` also reports `backend.noul_floor {value, source}`.
+>
+> Checked at 1428:
+> - `/decide/shared` 200 on both, with the cutover's smoke values reproduced (adult 0.052, personal 0.876).
+> - An https image URL is refused on both: 422 "not accepted here".
+> - **The vLLM backstop answered for the first time.** An https image URL posted straight to vLLM got 400 "The URL must
+>   be from one of the allowed domains: ['media.invalid']", with no fetch.
 
 h2o-lightning-4b @ b43f759b behind `services/h2o-decision-front` (copied into `conf/front/`), for albok-dev's
 re-validation of Nemi's sensitivity judge before the GPU 1 cutover (Prime 2026-10-12 via brokkr). **Remove right after

stacks/h2o-eval/compose.yaml

@@ -1,10 +1,17 @@
-# TEMPORARY h2o-lightning-4b decision seat on fv-ml1 GPU 3 (Prime 2026-10-12 via brokkr: "stand up the h2o copy on
+# TRANSIENT h2o-lightning-4b eval seat on fv-ml1 GPU 3 (GPU 3 is for transient work only).
-# gpu 3 if you need to test, it should replace the live seat as soon as possible and keep gpu3 only for transient
+# Round 1, 2026-10-12 1300-1319: albok-dev validated Nemi's judge on h2o before the GPU 1 cutover (Prime via brokkr).
-# tasks"). It exists so albok-dev can re-validate Nemi's sensitivity judge on h2o before the GPU 1 cutover; REMOVE it
+# Round 2, from 2026-10-12 ~1430 (Prime 1420, via albok-dev): "I'd like to understand why h2o failed, so stand it up
-# right after the cutover. INC (stacks/intern-decision) keeps serving production on GPU 1 meanwhile.
+#   on gpu3 and refit it and let's get an understanding." albok-dev investigates the 154 texts h2o alone called adult
+#   and re-fits the judge. Two fronts over ONE vLLM:
+#     :8034 front          the shim's yes/no floor ON (0.801), exactly as the live cutover ran; /health model stamp
+#                          h2o-lightning-4b@b43f759b..., identical to the cutover's
+#     :8035 front-nofloor  SHIM_NOUL_FLOOR=0 (floor OFF, brokkr's lever); stamp h2o-lightning-4b+noul-floor-0@b43f759b...
+#   Each /health also reports backend.noul_floor {value, source}. NO load tests or other traffic while albok-dev's
+#   runs are in (they name start/end on thread 01M4S026GWS354WZGXZ5E34F31). INC keeps :8033 on GPU 1, untouched.
 # Serving exactly as benched (docs/pfi/decision-sweep-followon-bench-2026-10-12.md, Amendment 1): stock vLLM 0.30.0
 # + the vendor's unmodified shim + the admission front (services/h2o-decision-front, copied into conf/front).
 # Weights: /tank/aimodels/h2o-lightning-4b-b43f759b (hashes match HF at b43f759b; see the stack README).
+# Inline images only (Prime 2026-10-12): LIGHTNING_IMAGE_URLS=off on both fronts + vLLM --allowed-media-domains.
 name: h2o-eval
 services:
   vllm:
@@ -54,5 +61,30 @@ services:
       - /tank/aimodels/h2o-lightning-4b-b43f759b:/model:ro
       - /opt/docker/conf/h2o-eval/front:/front:ro
     command: ["bash", "/front/run.sh"]
+  front-nofloor:
+    image: python:3.12-slim
+    container_name: h2o-eval-front-nofloor
+    restart: unless-stopped
+    network_mode: host
+    depends_on: [vllm]
+    env_file: .env
+    environment:
+      SHIM_VLLM: http://127.0.0.1:18199
+      SHIM_PORT: "18192"
+      SHIM_NOUL_FLOOR: "0"           # the floor OFF (the shim: 0 turns it off)
+      LIGHTNING_IMAGE_URLS: "off"   # Prime 2026-10-12: inline images only, the seat never fetches an image URL
+      FRONT_SHIM: http://127.0.0.1:18192
+      FRONT_VLLM: http://127.0.0.1:18199
+      FRONT_BIND: 0.0.0.0
+      FRONT_PORT: "8035"
+      FRONT_CAP: "32000"
+      FRONT_SHIM_FILE: /model/h2o_lightning_shim.py
+      FRONT_SERVE_CONFIG: /model/serve_config.json
+      FRONT_MODEL_NAME: h2o-lightning-4b+noul-floor-0   # a distinct judge stamp
+      FRONT_MODEL_REVISION: b43f759b6385d21182f87c3829931ec3d4dfa98a
+    volumes:
+      - /tank/aimodels/h2o-lightning-4b-b43f759b:/model:ro
+      - /opt/docker/conf/h2o-eval/front:/front:ro
+    command: ["bash", "/front/run.sh"]
 volumes:
   h2o-eval-vllm-cache:

stacks/intern-decision/README.md

@@ -91,9 +91,9 @@ The shim switch was verified live at 1402, on the h2o seat with the front recrea
 - inline red still answered red at 0.984;
 - before the change, inline red/blue/green answered at 0.98+ and no-image came out near-uniform.
 
-The vLLM backstop has **not** been exercised: the seat was rolled back before vLLM was recreated. Verify it on the next
+The vLLM backstop was first exercised on the GPU 3 eval seat at 1428 (same image and flag): an https image URL
-switch, by posting an https image URL straight to vLLM's loopback port and expecting a refusal that names the allowed
+posted straight to vLLM got 400 "The URL must be from one of the allowed domains: ['media.invalid']", with no fetch.
-domains. INC needs neither setting, because its served checkpoint has no vision tower.
+Repeat that check on this seat's loopback port (`:18299`) on the next switch. INC needs neither setting, because its served checkpoint has no vision tower.
 
 ### Switching between the two