vh
e3ce713f7f
Live GEN_MODEL is now qwen38-27b-coldfusion-h300-nvfp4-mixed (ana-ml2 GPU0 :8015). Served-name left unchanged so all 7 LiteLLM aliases route without a gateway edit. Verification: KV pool 401,550 tok / 1.53x (baseline 403k / 1.54x) LiteLLM aliases 7/7 green vision 3/3 shapes, colour+form+position correct MTP acceptance 59.7% median @ 118.37 tok/s quality gens 4/4 correct abliteration 4/4 compliance PPL NOT measured (see below) The roadmap predicted ~47% acceptance for a pristine MTP graft versus L35's 59.1% in-band edit. Measured 59.7% on the same harness: there is no acceptance penalty, which removes the throughput argument for reimplementing MPOA. A single long-prose generation read 47.5% off the same counters -- below the 8-run minimum of 49.0% -- and would have "confirmed" the prediction by coincidence. Acceptance must be read from quickbench.py, never one sample. PPL is blocked on VRAM, not on the model: eval_quality.py aborts with "prompt_logprobs look uniform" under --speculative-config, and the probe-seat workaround needs ~22 GB while both cards sit at ~96% committed. Also normalizes the quant dir from root:0600 to llmuser:llmuser 0664 to match every other model dir, and records that config.json sha256 is byte-identical across the h300 and L35 quants and is therefore useless for confirming which weights are mounted (mtime and a head-hash are the discriminating views). Rollback is one line to .env.bak-pre-h300-20260820.
29 lines
670 B
JSON
29 lines
670 B
JSON
{
|
|
"tag": "coldfusion-h300",
|
|
"model": "qwen3.8-27b-uncensored",
|
|
"tok_s_median": 118.36546171598528,
|
|
"tok_s_mean": 118.03553233059495,
|
|
"tok_s_min": 105.10639324800316,
|
|
"tok_s_max": 125.65294847991646,
|
|
"mtp_accept_median": 0.5969955969955969,
|
|
"rates": [
|
|
113.31452411709044,
|
|
117.98858084908065,
|
|
118.74234258288992,
|
|
105.10639324800316,
|
|
125.65294847991646,
|
|
124.77477661037778,
|
|
123.08915699995879,
|
|
115.61553575744233
|
|
],
|
|
"accs": [
|
|
0.5555555555555556,
|
|
0.5925925925925926,
|
|
0.6013986013986014,
|
|
0.4897119341563786,
|
|
0.654320987654321,
|
|
0.6470588235294118,
|
|
0.6328502415458938,
|
|
0.5736961451247166
|
|
]
|
|
} |