docs(scriberr): Parakeet dropout investigation; proposed 0002 (gap retry + model path)
Prime's ask (via the coordinator): investigate the "Parakeet skips stretches of speech" finding, including other Parakeet weights. Investigation only; nothing deployed. Against ground truth (official SCOTUS transcript, Gutenberg #38916) the drops are real: production v3 loses 140 / 66 clean words per transcript on the two public files and ~50 on each private one (Whisper-referenced, Canary-confirmed; adjudicator 129/129 correct on the calibration). Cause: the v2/v3 0.6B weights collapse deep inside long full-attention windows; the encoder output is degraded, the audio alone transcribes fine, and 1.1B TDT/RNNT/CTC and CTC-0.6B never do it. Decoding (CUDA graphs, greedy variants, max_symbols, beam), slice length, local attention, loudness, resampling and a noise floor do not fix it. Controls: A-vs-A, silence positive control (>=15 words 36/36), null control, bootstrap floor. Proposed patch 0002 re-transcribes >=3 s stretches where the audio holds speech but no word came out (-80 to -90 % lost words on all four recordings, lower WER, no invented text, +10 MiB) and adds an explicit PARAKEET_MODEL_PATH with the loaded model recorded in JSON and ModelUsed. Reviewed at high effort, all findings fixed; built and tested as scriberr:local-blackwell-a353078-dropout2, not deployed. scriberr-rebuild: --patches takes DIR[:DIR...]; embeds and seam-checks both Parakeet scripts (seam-check --standard for the short-audio one).
This commit is contained in:
@@ -181,4 +181,7 @@ Peak GPU memory on a 35-minute file:
|
||||
peak is unchanged (5,496 MiB, n=3). See `patches/README.md` and
|
||||
`docs/pfi/scriberr-slicer-bench-2026-09-30.md`. That bench also found that
|
||||
Parakeet sometimes skips stretches of ≥10 words mid-slice, with or without the
|
||||
patch; that is still open.
|
||||
patch. **Investigated 2026-09-30** (`docs/pfi/parakeet-dropout-investigation-2026-09-30.md`):
|
||||
the losses are real (against ground truth) and belong to the v2/v3 weights over
|
||||
long windows; a proposed patch, `patches/proposed/0002`, re-transcribes speech that
|
||||
got no words and cuts them 80–90 %. It is not deployed; that is Prime's call.
|
||||
|
||||
Reference in New Issue
Block a user