vh
a6c1d3c454
A/B of the live STT seat (fv-ml1 GPU 0, sherpa-onnx int8 v3) against nvidia/parakeet-unified-en-0.6b, measured on GPU 3 with the seat's own image, k2-fsa's published unified int8 export, fp32/fp16 exports made with k2-fsa's recipe, v2 int8, and NeMo 3.0.0 (fp32, bf16 autocast, bf16 weights). - Seat int8 graph runs on one CPU thread (cpu/wall 1.00, GPU 2-9%). - unified-en under NeMo: -121/-234/-530 ms vs the seat at 1-3/3-8/8-20 s (paired, n=120/bin; floor <=6 ms; +50 ms positive control reads +52-54). - unified-en WER lower in every runtime: -0.7 pp clean, -1.5 pp other, -3.2 to -4.4 pp AMI (paired CIs exclude 0). - Seat defects found: hard 400 s input ceiling (HTTP 500), truncation after a quiet 1.5 s pause, and severe long-window dropouts (int8 v3 only). - B-bf16w needs +0.8 to +1.5 GB over the seat's 1,690 MiB on GPU 0. Raw requests, hypotheses, manifests and the full harness under services/parakeet-ab-2026-09-30/. No deploy; live seat untouched apart from 240 light test requests.
7 lines
1.5 KiB
JSON
7 lines
1.5 KiB
JSON
{"arm": "ab-b32", "desc": "B-fp32: unified-en, NeMo 3.0.0 torch", "port": 18306, "args": ["nemo", "fp32", "direct", "1"], "pid": 3297569, "t_started": 1790816206.0796213, "t_ready": 1790816217.889909, "cold_s": 11.8}
|
|
{"arm": "ab-b32c", "desc": "B-fp32, .nemo restored on CPU then moved to GPU", "port": 18314, "args": ["nemo", "fp32", "direct", "1", "1"], "pid": 3298989, "t_started": 1790816226.5622911, "t_ready": 1790816238.6615515, "cold_s": 12.1}
|
|
{"arm": "ab-b16w", "desc": "B-bf16w: encoder/decoder/joint weights in bf16, CPU load", "port": 18315, "args": ["nemo", "bf16w", "direct", "1", "1"], "pid": 3300531, "t_started": 1790816247.0396595, "t_ready": 1790816259.997797, "cold_s": 13.0}
|
|
{"arm": "ab-b16", "desc": "B-bf16: unified-en, NeMo 3.0.0 torch, autocast bf16", "port": 18307, "args": ["nemo", "bf16", "direct", "1"], "pid": 3302224, "t_started": 1790816272.6623154, "t_ready": 1790816284.4539192, "cold_s": 11.8}
|
|
{"arm": "ab-cf32", "desc": "C-fp32: unified-en fp32 ONNX (k2-fsa export recipe), seat runtime", "port": 18312, "args": ["sherpa", "/tank/spikes/parakeet-ab/models/unified-f32", "1", "onnx"], "pid": 3302918, "t_started": 1790816303.3597174, "t_ready": 1790816348.5139284, "cold_s": 45.2}
|
|
{"arm": "ab-af32", "desc": "A-fp32: the seat's v3 as fp32 ONNX (k2-fsa export recipe), seat runtime", "port": 18311, "args": ["sherpa", "/tank/spikes/parakeet-ab/models/v3-f32", "1", "onnx"], "pid": 3306068, "t_started": 1790816364.7978728, "t_ready": 1790816409.478592, "cold_s": 44.7}
|