docs(scriberr): Parakeet dropout investigation; proposed 0002 (gap retry + model path)

Prime's ask (via the coordinator): investigate the "Parakeet skips
stretches of speech" finding, including other Parakeet weights.
Investigation only; nothing deployed.

Against ground truth (official SCOTUS transcript, Gutenberg #38916) the
drops are real: production v3 loses 140 / 66 clean words per transcript on
the two public files and ~50 on each private one (Whisper-referenced,
Canary-confirmed; adjudicator 129/129 correct on the calibration). Cause:
the v2/v3 0.6B weights collapse deep inside long full-attention windows;
the encoder output is degraded, the audio alone transcribes fine, and
1.1B TDT/RNNT/CTC and CTC-0.6B never do it. Decoding (CUDA graphs, greedy
variants, max_symbols, beam), slice length, local attention, loudness,
resampling and a noise floor do not fix it. Controls: A-vs-A, silence
positive control (>=15 words 36/36), null control, bootstrap floor.

Proposed patch 0002 re-transcribes >=3 s stretches where the audio holds
speech but no word came out (-80 to -90 % lost words on all four
recordings, lower WER, no invented text, +10 MiB) and adds an explicit
PARAKEET_MODEL_PATH with the loaded model recorded in JSON and ModelUsed.
Reviewed at high effort, all findings fixed; built and tested as
scriberr:local-blackwell-a353078-dropout2, not deployed.

scriberr-rebuild: --patches takes DIR[:DIR...]; embeds and seam-checks
both Parakeet scripts (seam-check --standard for the short-audio one).
This commit is contained in:
vh
2026-09-30 15:48:02 -07:00
parent 1189adbf18
commit 38015a1977
7 changed files with 1162 additions and 17 deletions
+4 -1
View File
@@ -181,4 +181,7 @@ Peak GPU memory on a 35-minute file:
peak is unchanged (5,496 MiB, n=3). See `patches/README.md` and
`docs/pfi/scriberr-slicer-bench-2026-09-30.md`. That bench also found that
Parakeet sometimes skips stretches of ≥10 words mid-slice, with or without the
patch; that is still open.
patch. **Investigated 2026-09-30** (`docs/pfi/parakeet-dropout-investigation-2026-09-30.md`):
the losses are real (against ground truth) and belong to the v2/v3 weights over
long windows; a proposed patch, `patches/proposed/0002`, re-transcribes speech that
got no words and cuts them 80–90 %. It is not deployed; that is Prime's call.