vh
30f36f2478
The vendor shim silently trims state at 32,000 tokens; the admission proxy is what turns the seat contract into a loud refusal, so it must front the shim in any deployment. Committing it with the Amendment 1 confirmation driver so executing the swap does not depend on a bench scratch dir surviving.
25 lines
1.4 KiB
Markdown
25 lines
1.4 KiB
Markdown
# h2o admission proxy + gate probe set
|
|
|
|
Production-relevant because the seat swap to h2o-lightning-4b (ordered
|
|
2026-10-12) inherits a hard constraint: the shipped vendor shim SILENTLY
|
|
trims state at 32,000 tokens. The seat contract is a REFUSING 32,000-token
|
|
seat, and this proxy is what makes the refusal real. It must sit in front of
|
|
the shim on any deployment.
|
|
|
|
- `admission_proxy.py` — thin wrapper in front of the unmodified shipped
|
|
`h2o_lightning_shim.py`. Imports the vendor's own renderers, counts each
|
|
question's exact prompt with vLLM's `/tokenize`, and refuses over-cap with
|
|
422 + the named count (both currencies: per-question prompt max and the
|
|
shim's shared-head+tails total). Env: ADMIT_CAP (default 32000), ADMIT_UP
|
|
(shim URL), ADMIT_VLLM, ADMIT_PORT, ADMIT_SHIM (path to the vendor file).
|
|
Measured gate overhead: the shape numbers pass with it in-line.
|
|
- `gate_probe_set.py` — the Amendment 1 confirmation driver (calibrate to the
|
|
cap, depth-gate cells, cold/warm shapes, 4x concurrency, over-cap refusal).
|
|
Written for the bench layout on fv-ml1 (`/tank/bench`); paths are hardcoded,
|
|
adapt before re-running anywhere else.
|
|
|
|
Provenance: fv-ml1:/tank/bench/h2o_admission.py @ the confirmation run,
|
|
results in docs/pfi/decision-sweep-followon-bench-2026-10-12.md (Amendment 1
|
|
addendum). Measured boot for capacity planning: ~154 s launch-to-ready,
|
|
graph-capture bound (no warm-cache or eager-mode saving).
|