vh
11b688ff68
MuXodious/Qwen3.8-27B-absolute-heresy (Heretic v1.4.0 + SOMPOA, trial T377, pin c2374593) quantized through our mixed NVFP4+FP8 recipe and promoted after passing the full gate on the probe port. Gate vs incumbent -- MTP acceptance 47.2% (48.2%), decode 103.5 tok/s (96.4), prefill 6618/5403 at 6.7k/27k (6334/5085), TTFT 27k 5.00s (5.31s), perplexity 6.910 (7.059, 2.1% better), surface 6/6, abliteration compliance 4/4. On our battery-instruct arm -- the framing that actually elicits refusals -- 0/55 with zero EMPTY, so no catatonia at the hard edge. Speed deltas are image-confounded: the probe ran the seat's pinned nightly while the incumbent's stored numbers came from an earlier image. Read as not worse. Acceptance, perplexity, surface and refusal are apples-to-apples. All 7 LiteLLM aliases verified end-to-end. GPU0 at 91.3/97.9 GB with meromero healthy -- more headroom than the previous build. Incumbent weights untouched and .env.bak-heresy-20260817 in place for rollback. Candidate is a 2-day-old RC1 with ~348 downloads; watch real multi-turn use.
29 lines
644 B
JSON
29 lines
644 B
JSON
{
|
|
"tag": "heresy",
|
|
"model": "probe",
|
|
"tok_s_median": 103.46655635565449,
|
|
"tok_s_mean": 102.60692862947714,
|
|
"tok_s_min": 90.30587796547803,
|
|
"tok_s_max": 115.13389586042238,
|
|
"mtp_accept_median": 0.47231349640988196,
|
|
"rates": [
|
|
103.09345339858122,
|
|
115.13389586042238,
|
|
98.90613513574037,
|
|
90.30587796547803,
|
|
96.87736392506643,
|
|
103.83965931272775,
|
|
108.24742296674285,
|
|
104.45162047105809
|
|
],
|
|
"accs": [
|
|
0.46987951807228917,
|
|
0.5698198198198198,
|
|
0.4373795761078998,
|
|
0.3719298245614035,
|
|
0.4218455743879473,
|
|
0.47474747474747475,
|
|
0.5084388185654009,
|
|
0.4796747967479675
|
|
]
|
|
} |