vh
30f36f2478
The vendor shim silently trims state at 32,000 tokens; the admission proxy is what turns the seat contract into a loud refusal, so it must front the shim in any deployment. Committing it with the Amendment 1 confirmation driver so executing the swap does not depend on a bench scratch dir surviving.
1.4 KiB
1.4 KiB
h2o admission proxy + gate probe set
Production-relevant because the seat swap to h2o-lightning-4b (ordered 2026-10-12) inherits a hard constraint: the shipped vendor shim SILENTLY trims state at 32,000 tokens. The seat contract is a REFUSING 32,000-token seat, and this proxy is what makes the refusal real. It must sit in front of the shim on any deployment.
admission_proxy.py— thin wrapper in front of the unmodified shippedh2o_lightning_shim.py. Imports the vendor's own renderers, counts each question's exact prompt with vLLM's/tokenize, and refuses over-cap with 422 + the named count (both currencies: per-question prompt max and the shim's shared-head+tails total). Env: ADMIT_CAP (default 32000), ADMIT_UP (shim URL), ADMIT_VLLM, ADMIT_PORT, ADMIT_SHIM (path to the vendor file). Measured gate overhead: the shape numbers pass with it in-line.gate_probe_set.py— the Amendment 1 confirmation driver (calibrate to the cap, depth-gate cells, cold/warm shapes, 4x concurrency, over-cap refusal). Written for the bench layout on fv-ml1 (/tank/bench); paths are hardcoded, adapt before re-running anywhere else.
Provenance: fv-ml1:/tank/bench/h2o_admission.py @ the confirmation run, results in docs/pfi/decision-sweep-followon-bench-2026-10-12.md (Amendment 1 addendum). Measured boot for capacity planning: ~154 s launch-to-ready, graph-capture bound (no warm-cache or eager-mode saving).