Files
esh-pfi-infrastructure/services/gen-seat-mixed-quant/bench
vh e4576f0989 test(gen-seat): PPL on orcarouter — mean 7.07 / median 5.76, within noise of heresy
Measured against a spec-decode-free probe on GPU1 (fable downed to free the VRAM,
probe torn down after). eval_quality.py aborts PPL under --speculative-config, so
a nospec probe is the only way to read it.

  orcarouter  mean 7.0655  median 5.758
  heresy      mean 6.910   median 5.625   (+2.2% mean)
  our recipe's usual mixed-quant PPL: 7.059 -- orcarouter is identical to it

So orcarouter matches heresy on fidelity and wins on every other axis: MTP
acceptance 58.4% vs 47.2%, zero think-leak, vision 7/8. The gen-seat search that
ran through Cold-Fusion, heresy, and preetpatel is settled on orcarouter.
2026-08-21 02:06:52 -07:00
..