vh
f85d102813
Gates run against the live seat while the operator tested in parallel. <think> leak (n=30, 4 prompt types + multi-turn) 0/30, 0 empty MTP acceptance 58.4% @ 117.11 tok/s median surface 6/6 abliteration survival 4/4 compliance deterministic quality gens coherent and correct PPL still blocked For scale on the leak gate, the abandoned h300 build scored 8/30 on this exact instrument, and its abliteration-survival samples had 2 of 4 open with "<think>Ok, let's figure this out:". Orcarouter has none. The headline is MTP acceptance. 58.4% against heresy's byte-identical base head at 47.2% is +11 points, and it sits level with our own in-band L35 at 59.1%. That is the additive in-band-vs-graft delta the entire Cold-Fusion experiment was built to measure and never cleanly delivered -- orcarouter handed it over for free because the author had already done the Robinson edit on the head. Surface 6/6 covers plain chat, vision, tool calling, the thinking split, a 36,042-token long-context retrieval, and streaming. PPL remains blocked on a spec-decode-free probe seat: it needs ~22 GB and GPU1 has ~16 GB free. Comparison target is heresy at 6.910.
29 lines
666 B
JSON
29 lines
666 B
JSON
{
|
|
"tag": "orcarouter",
|
|
"model": "qwen3.8-27b-uncensored",
|
|
"tok_s_median": 117.11450500766267,
|
|
"tok_s_mean": 116.03067609161164,
|
|
"tok_s_min": 106.97760569119777,
|
|
"tok_s_max": 121.32483831758886,
|
|
"mtp_accept_median": 0.5839516742770168,
|
|
"rates": [
|
|
116.28141812912571,
|
|
111.08269606466214,
|
|
117.94759188619963,
|
|
106.97760569119777,
|
|
118.02130932541095,
|
|
115.42894126871005,
|
|
121.32483831758886,
|
|
121.18100804999801
|
|
],
|
|
"accs": [
|
|
0.5776255707762558,
|
|
0.5424836601307189,
|
|
0.5902777777777778,
|
|
0.5052410901467506,
|
|
0.5949074074074074,
|
|
0.5736961451247166,
|
|
0.6190476190476191,
|
|
0.6214285714285714
|
|
]
|
|
} |