# `[2026-09-04/05]` vLLM RUNS on sm_121 — and the blocker was `ninja` not being on PATH, not the silicon

Long-standing open question closed: **vLLM 0.28.0 serves on the GB10 (aarch64, sm_121)** from a
stock wheel, no source build. Model loads, `torch.compile` completes (~29 s), CUDA graphs capture.

⚠ **The one trap, and it looks exactly like an sm_121 kernel problem:** FlashInfer JIT-builds its
sampling kernel at first use and needs **`ninja` on PATH**. It ships as a vLLM dependency but
lives in the venv `bin/`, so the failure surfaces as `FileNotFoundError: 'ninja'` from deep inside
a `profile_run` traceback. Same shape as the `python3-dev` trap that bit the training harness on
this box, and the third present-but-not-on-PATH false-absence on this hardware (after `nvcc`).

Launch with **both** `~/vllm-env/bin` and `/usr/local/cuda/bin` on PATH. Harmless and expected:
`Using default MoE config ... device_name=NVIDIA_GB10` — nobody has tuned MoE kernels for this chip.

⚠ **Open WebUI sends `tool_choice: "auto"` on every request**, which vLLM 400s unless launched with
`--enable-auto-tool-choice --tool-call-parser gemma4`. Taken from the working sibling seat
(`gemma4-charrp`), which is where the canonical flags live. **Deliberately did NOT copy
`--reasoning-parser gemma4`** — that seat pairs it with `--default-chat-template-kwargs
'{"enable_thinking": false}'`, and adding it alone moves output into `reasoning_content` with a
null `content`, which OWUI renders as an empty reply. A 400 traded for a blank box.

## Run 4 — the corpus arm

Run 4 adds an **airoboros-3.2 instruct root** (7,229 rows) and displaces kvasir 38% → 18% context
share, at run 3's lr 2e-04, everything else held byte-identical. Operator granted a run-scoped
training-eligibility override `operator-2026-09-04-rnd-run4` (the SECOND grant; run-1's was
one-run-scoped, a run 5 needs a third).

**Two peer artifacts were rejected before they cost GPU time, both by reading the harness rather
than accepting a "confirm this":**

1. `sample_kind: "instruct-dialogue"` — `core.py:521-544` dispatches on the **literal string** and
   raises on anything outside `rp-dialogue`/`prose-chunk`/`actual-play-passage`. Would have
   hard-stopped the encode on the first airoboros row. brokkr had labelled it *non-blocking*.
2. Recipe targets collapsed three dialogue roots into one descriptive label with `root_sha256:
   null` on three of four. Preflight resolves `roots_dir/<dataset_id>/clean-v1/CLEANROOT.json`
   literally and requires the sha.

⚠ **I did NOT fill the missing shas in myself**, though the values were known and it would have
taken two minutes. `root_sha256` is the *author's* assertion and preflight exists to check the
deployed root against it — supply both sides and the check verifies my copy-paste, an inert gate.
brokkr then went further and recomputed his shas **from shard bytes** rather than reading them
back out of the deployed CLEANROOT, which had the same defect one step removed.

**The kvasir prefix cut has three independent confirmations**: my build measured 1,613 samples /
3,347,622 ctx = **47.4%** of run-3 kvasir, which reproduces the recipe's 18/38 share ratio
(47.37%); the harness's own `[mix]` block then reported **ctx 0.1805** against the 18.0% target.

⚠ **`str.splitlines()` splits on U+2028**, which `json.dumps` leaves unescaped under
`ensure_ascii=False`. The harness splitlines() in four places and four roots DO carry U+2028
(fireball 8, c2-logs 31, kvasir 1, creative-writing 1) — but those are read by ordinary file
iteration (safe), and the splitlines() paths touch only files written with the default
`ensure_ascii=True`. Latent, one flag away from live, not acting. Found by brokkr the hard way.