Bisected context depth on the orcarouter checkpoint with non-repeating prompts
(unique random hex per probe, so prefix caching cannot short-circuit the
prefill). Six depths from 31,978 to 258,517 tokens, all served.
The load-bearing evidence is the engine allocator log: zero OOM, CUBLAS, or
illegal-memory entries across the run. That is the same detector that caught
the dealignai near-miss at 155K on the previous checkpoint, where it did fire.
The probe also ran under real concurrent operator load, making it a stricter
test than a solo run rather than a weaker one.
Positive control passed: a mis-sized first attempt produced a ~265K-token
prompt and got a clean 400 naming the limit instead of killing the engine, so
the probe could detect the failure mode it was looking for. Calibration for
re-runs: random hex words tokenize at 7.9 tokens/word here.
vLLM #54919 (long prefill starving decode for 3-7 minutes) did not reproduce:
258K prefilled in 28.9 s, roughly 8,900 tok/s, scaling near-linearly from 32K.
Records that the probe's memory-headroom half was BLIND and must not be reused.
It reported an identical 95,460 MiB used / 2,427 MiB free on every row across an
8x range of depths, which is the tell. Two causes: --kv-cache-memory pins the
pool and the engine logs "skipped memory profiling", so GPU usage is flat with
respect to depth; and the actual risk is a transient activation spike during
prefill, which before/after nvidia-smi bracketing structurally cannot observe.
Peak-activation headroom therefore remains unmeasured; the pass/fail result
rests on the allocator log alone.
Also qualifies the earlier 167.5 tok/s decode figure as a possibly-contended
lower bound, and records the operator's independent 140 tok/s average measured
in real use while this probe was loading the same card.
vh
4954cf6756