esh-pfi-infrastructure/docs/pfi/d3-vs-intern-decision-bench-2026-10-12.md

D3 vs Intern-Decision-4B — decision-model seat bench (2026-10-12)

Ordered by Prime; preregistration brokkr-smithy docs/surveys/vllm-sr-decision-landscape-2026-10-11/d3-bench-prereg.md (c93d7b0, amended b87a7ab), plus Amendment 1 (shape calibration, brokkr) and Amendment 2 (operator: memory budget = what fits on the seat's GPU, e3dbfac). Run by infra-hermes on fv-ml1 GPU 3. HALTED by operator call at 15:07Z — "fails on a number of levels… stash the findings and call it." This doc is the stash: everything below was measured; what was not run is stated plainly.

Harness: jevbench v1.2.16 (public 231 = easy/original/hard), the S1 battery (authored144 + perturbations108 + cicada W1/W2 + Wyrd, 1,748 paired scorable rows), and the 10-04 shape-runner lineage with a per-model calibrated 32k builder. All artifacts: fv-ml1:/tank/bench/arm-* and /tank/bench-jev-2026-10-04 (the latter untouched, lkraven-owned).

Headline

The incumbent (intern-decision-serve 0.1.3, pinned Intern-Decision-4B) is not displaced. Every d3 arm that clears the incumbent's accuracy needs either a GPU budget the seat cannot hold at the worst shape, or fails the latency contract, or both. The one arm inside the seat's own memory budget at the worst shape (d3-edge at bs=1) loses 8 battery points and is 28x slower than INC at 32k-class state. The quantization warrant closes: NVFP4 does not buy the 32k shape, because the 32k problem is O(q x state) activation and protocol accounting, not weight bytes.

1. bf16 ladder (Engine primary, vllm-sr dev CLI, GPU 3)

Revisions pinned per arm (edge e51ab425…, lite c91d55d6…, nano ccd7bc1a…, mini 40391eb6…, flash 06d33616…, d3-27 7f257ad7…). jevbench acc = public 231, median of 3 repeats (all repeats bit-exact per arm). Battery = labelled-evidence acc, single condition. Peak VRAM = steady-state serve MiB. p50 = harness per-call median ms.

arm	params	jevbench	battery single	peak MiB	p50 ms
INC (baseline)	4B	202	.910	8,822 (32k shape)	~1.9–2.0s at 32k
d3-edge	0.59B	158	.749	2,398	15
d3-lite	1.4B	168	.852	2,676	28
d3-nano	3.8B	175	.907	5,304	29
d3-mini	7.3B	188	.945	10,540	36
d3-flash	8.39B	190	.947	17,600	37
d3-27 (vendor path)	26.09B	206	.970	50,300	—

INC reproduction: 202/231 x3, bit-exact against the 10-04 record (out/intern-native). d3-27 is the only arm that beats INC on the public suite, and it is a path divergence: the Engine's placement estimator (4B/param + 2 GiB) refuses a 26B model on any single card we have; the vendor d3_server.py runs it at 50.3 GiB. Gate (jevbench >= INC - 3): PASS at 27B only.

Argmax cross-check (vendor vs Engine, same snapshot, 231 items): edge 230, lite 229, nano 229, mini 231, flash 231 — mismatches are borderline near-ties (max prob gap 2.7e-2). Engine fidelity: argmax identical on all non-borderline items; probabilities NOT exact (median gap 4.2e-4). The card's "matches exactly" claim is false as written; argmax-equivalence holds.

2. 32k shape cells (Amendment 1: per-model calibrated to ±2% of 32,768 realised tokens)

Builder length iterated per model in its own tokenizer; every arm converged to 32,743 realised tokens for the 1q state. 16q cells carry 16 x 32,743 + framing = 523,900 wire tokens because the d3 wire protocol counts state once PER QUESTION — O(q x state) accounting, measured, not predicted. INC's equivalent is a single forward over one state. cold = first request after ready; launch to ready wall time is reported separately below and is NOT folded into cold.

arm	crit21_short cold ms	1q32k cold ms	16q32k cold ms	16q32k peak MiB
INC	(2-call 16+5, see note)	1,970 (32,715 tok)	rejected: >32,768 cap at 33,324; maxfit 16q = 32,708 tok, 1,960 ms	8,822
d3-edge	1,242.6	895.4	20,790	28,404
d3-lite	89.1	1,139.5	18,668	28,938
d3-nano	103.3	1,442.1	23,568	33,486
d3-mini	157.1	3,565.8	deadline-bound (503 at 30.002s, all retries)	71,404
d3-flash	242.3	4,469.5	deadline-bound	80,396
d3-27 vendor	642.7	8,731.0	147,724 (366,172 tok, builder cap)	96,262

Notes. (a) INC crit21 on 10-04 is a client-side 16+5 split summed across two /v1/systemone calls (bench_shape.py --max-questions 16); /v1/systemone enforces 1..16 questions, so a 21-question crit is structurally out of contract. (b) The Engine's 30 s serving deadline is a configuration of the seat, not a bug; the wall sits between 3.8B (nano completes at 23.6s) and 7.3B (mini). (c) Activation peaks include 32k state at bs=8 (Engine BATCH_SIZE is hard-coded 8, family.py:65); every bs=8 arm exceeds even Amendment 2's budget at 16q32k. (d) Latency clause (16q32k <= 1.5x INC): INC's honest 32k-class cell is 1.96s, so the clause bar is ~2.9s. NO d3 arm meets it; closest is edge-b1 at 13.0s (6.6x).

3. bs=1 curve (vendor --batch-size 1, the only memory knob)

arm	16q32k peak MiB (bs=8 -> bs=1)	16q32k cold ms (bs=8 -> bs=1)
d3-edge	28,404 -> 4,778	20,790 -> 13,019
d3-mini	71,404 -> 18,514	deadline -> 53,770

The knob buys memory, never time: latency is protocol-intrinsic O(q x state). No setting puts a mini-class model inside the seat at the worst shape (mini-b1 still misses Amendment 2's budget by 1.6 GiB and fails latency at 18x INC).

4. Stage 4 — control quant trio and the battery sensitivity curve

d3-lite backbone, RTN (round-to-nearest, no calibration), vendor path, --verify none (the manifest hashes the main shard; modified packages can't self-verify). Paired deltas vs the path-matched bf16 vendor baseline, 1,748 rows, 95% bootstrap CI, Wyrd split included:

arm	weight rel-L2	jevbench	battery	paired delta [CI]	Wyrd delta [CI]	argmax flips
lite bf16 vendor (control)	0	168	.852	—	—	—
lite NVFP4 RTN	9.4%	164	.869	+0.0034 [-0.0086, +0.0154]	-0.019 [-0.055, +0.017]	117
lite INT3 RTN	19.9%	158	.739	-0.095 [-0.118, -0.072]	-0.131 [-0.176, -0.088]	476

The battery gate is monotone in weight error and functional: it cannot see 9.4% (NVFP4 RTN lands inside noise), it clearly sees 19.9% (INT3 collapses). This is the calibration warrant's premise confirmed: a quant that survives must move weights less than 9.4% AND keep paired deltas inside the band — i.e. the headroom to demonstrate is measured, and it is small.

5. Stage 5 — flash calibrated NVFP4 (the one arm completed before the halt)

ModelOpt NVFP4_DEFAULT_CFG, calibrated on the shared 512-row corpus (sha256 e7aa7b18…a31e, seeded jev-nvfp4-calib-2026-10-12, no scored item). Served in-process over a /v1/systemone shim (stage5_serve.py, prepare/run/respond mirroring the vendor contract, 1,405 quantizers): ModelOpt's fake-quant lives in the quantizer forward, so no package rewrite; accuracy is a true fake-quant read, memory is analytic + the byte table below.

jevbench 194 x3 (bf16 Engine control 190). Survive rules applied by the letter, paired vs arm-d3-flash, n=1,748:

rule	result	verdict
(a) battery pooled delta, CI lower bound >= -0.02	-0.0011 [-0.0086, +0.0057]	PASS
(b) public suite within ±3 of bf16 median	+4 (194 vs 190)	FAIL by the letter (upward; 10 flips: 6 up, 2 down, 2 both-wrong)
(c) Wyrd within -0.02	+0.0071 [-0.0119, +0.0262]	PASS
(d) argmax flips <= 4 of 231, both directions	10	FAIL by the letter

Byte table (bf16 snapshot index, total 15.63 GiB on disk): quantized linears (attn 3,980 + mlp 9,216 MiB) at 4.5 b/w = 0.28125x -> 3,711 MiB; kept wide (embed+head 1,940, vision 870, other 2) = 2,812 MiB. Projected export 6,524 MiB (~6.37 GiB) — single-digit, as expected. ModelOpt export_hf_checkpoint exists for a measured-export ground truth; not run before the halt. Latency of the fake-quant arm is dequant-matmul timing and is NOT reported as a speedup. Analytic 32k verdict unchanged: quantization does not touch the O(q x state) activation cost; the 16q32k cell fails identically at any weight width.

6. Amendment 2 — the seat GPU budget (measured read-only, GPU 1)

Prime: "our budget is now whatever fits in the current gpu where it's running."

• GPU 1 total: 97,887 MiB.
• Non-seat processes: vllm-meromero-r 43,994 MiB + vllm-gen-small 34,994 MiB = 78,988 MiB.
• Seat (intern-decision) resting: 8,812 MiB.
• Safety margin: 2,048 MiB (brokkr's suggested figure; no seat headroom policy exists).
• Budget: 16,851 MiB (~16.5 GiB).

Re-read of every fit clause against 16,851 MiB:

• Short shape: fits INC-class (8,822 measured) and every arm through mini (10,540); NVFP4-flash analytic ~6,524 fits; flash bf16 (17,600) misses by 749 MiB; 27B (50,300) does not.
• 16q32k: only edge-b1 (4,778) and INC-class fit. Every bs=8 arm is 27.6–96.3 GiB. The latency clause independently kills mini/flash/INC at that shape.
• Net: the wider budget upgrades mini from "memory-out" to "memory-in, latency-out" (53.8s at the only knob setting that fits), and leaves the conclusion unchanged.

7. What was NOT run because of the halt

• INC bench-copy arms (positive control inc-bf16-copy and fake-quant inc-fakequant, 249 language tensors RTN-NVFP4, median rel-L2 9.4%, vision kept bf16): the 10-04 intern_server.py shim and both packages were built and the arm was mid-boot when the halt landed. The INC fake-quant row is therefore absent — note the production engine cannot load NVFP4 at all (intern_decision_serve/engine.py pins the snapshot's inference.py by hash and validates dtype against {bfloat16, float16, float32} with an explicit raise; no compressed-tensors path exists in the image).
• NVFP4 arms for mini, 27B (fake-quant), nano, lite (calibrated), edge.
• ModelOpt measured export (ground-truth on-disk size).
• Paired battery deltas + Wyrd splits for the bf16 ladder arms (no quantized sibling to pair against; would need battery re-runs).

8. Teardown & disk

Engine stack stopped, all bench servers on GPU 3 stopped, :18080 iptables rule removed, guard read confirms GPU 3 at 2 MiB. Production GPU-0/1 seats untouched (one ~51 s exposure window during cross-checks earlier in the run, zero degradation observed — all RestartCount=0, no Xid).

Weights KEPT per instruction (Prime rules on deletion): /tank/bench/models/model-runtime 80 GB (all six d3 snapshots + INC copy staged), plus bench working set: inc-bf16-copy 8.5 GB, inc-fakequant 8.5 GB, quant-flash-nvfp4 16 GB, quant-lite-nvfp4 1.7 GB, quant-lite-int3 1.4 GB. /tank/bench total 180 GB; /tank at 62% with 3.3 TB free, so this is comfort, not pressure.