esh-pfi-infrastructure/docs/pfi/parakeet-dropout-investigation-2026-09-30.md

# Parakeet dropout investigation (2026-09-30)

Prime's ask, via the coordinator, 2026-09-30: investigate the "Parakeet skips
stretches of speech" finding from the slicer bench
(`docs/pfi/scriberr-slicer-bench-2026-09-30.md`), and include a different
Parakeet weight. **Investigation only:** nothing here changed the live Scriberr
container or its `.env`; deploying anything is Prime's call.

**Privacy:** two of the four recordings are Prime's. Their audio, transcripts and
diarization stay in fv-ml1 `/tank/spikes/scriberr-slicer/private/` (mode 700).
This document carries metrics only.

## Verdict

- **The drops are real.** Against ground truth, today's Parakeet (v3, shipped
  slicer) loses whole stretches of clean speech: 140 words per transcript on a
  clean audiobook, 66 on a court argument, and about 50 on each of Prime's two
  recordings (Whisper-referenced, Canary-confirmed). The old "reference" was
  mostly right; on SCOTUS it drops speech too.
- **Root cause:** the Granary-era 0.6B weights (v3 and v2) sometimes stop
  producing words for tens of seconds deep inside a long full-attention window.
  The encoder output there is degraded (a fresh decoder recovers only 22 to 51 %),
  the same audio transcribed alone is fine, and older Parakeets with the **same
  TDT decoder** do not do it. It is chaotic in cut placement and not explained by
  loudness, SNR, speaking rate or language. Decoding settings, beam search,
  shorter slices, loudness normalisation and resampling do not fix it.
- **Fix that works:** re-transcribe any ≥ 3 s stretch where the audio holds
  speech but the model produced no words. That cuts the loss 80 to 90 % on all
  four recordings (to 19, 13, 7 and 5 words per transcript) with no invented
  text, at +10 MiB and no measurable time. It is patch 0002 (proposed, built and
  tested, **not deployed**), which also adds an explicit `PARAKEET_MODEL_PATH`.
- **Weights:** v2 loses nothing on Prime's recordings or SCOTUS but collapses on
  read speech; `parakeet-unified-en-0.6b` (newest) is the best-balanced
  punctuating Parakeet but needs NeMo 3.0.0 and a licence change. The 1.1B and
  CTC Parakeets never collapse but produce no punctuation.
- **Separate, smaller issue:** overlapping speakers (SCOTUS interjections) are
  lost by every single-stream model. That is not this bug.

## 1. Is the instrument right? Ground truth, not a model reference

The first finding measured losses against the whole-file local-attention
transcript. That reference could itself be inserting text, so the public
recordings were re-scored against real ground truth.

| file | ground truth | provenance |
|---|---|---|
| scotus (first 30 min of No. 22-451) | the official argument transcript, `supremecourt.gov/oral_arguments/argument_transcripts/2023/22-451_114p.pdf` (sha256 `4feb7786…78a2`) | cover pages, page and line numbers, running headers, time stamps, argument headings and speaker labels removed; cut where the audio ends by alignment (5,627 of 14,764 transcript words) |
| wilde (LibriVox section 1) | Project Gutenberg #38916, *The Trial of Oscar Wilde, from the Shorthand Reports* (the LibriVox page's own "online text" link), sha256 `2271f271…b3a9` | section 1 is the book's Preface, read by the narrator alone ("It is wrong for us…" to "…came too late."), plus LibriVox's standard spoken intro and outro |

**Normalisation:** Whisper's English normaliser (transformers 4.53.3, the version
in Scriberr's env, no spelling map), applied word by word to both sides so every
hypothesis token keeps its timestamp. Fillers are dropped, numbers become digits
and contractions are expanded.

**Alignment and the dropout detector:** difflib opcodes, then exact Levenshtein
inside each mismatch. A **dropout** is a stretch between solid matches (≥ 3
tokens) holding ≥ 10 reference words where the hypothesis emitted fewer than half
as many. An **insertion run** is the mirror image. Matching islands shorter than
3 tokens count as part of the stretch, so one spurious "the" cannot split a
skipped paragraph in two.

**Timed ground truth:** every ground-truth token gets the median start time of
the transcripts that matched it (14 transcripts); the 143 (scotus) and 61
(wilde) tokens that no transcript matched are interpolated. There are no time
reversals over 1 s.

### Result: the drops are real, measured against ground truth

The same outputs the first finding used (the shipped slicer and upstream's fixed
cutter, 3 placements each), now scored against ground truth:

| file | system | WER | dropouts | words dropped |
|---|---|---|---|---|
| wilde | whole-file local attention (the old stand-in reference) | 2.12 % | 0 | 0 |
| wilde | upstream fixed 120 s cutter | 2.57 % | 0 | 0 |
| wilde | shipped slicer, at 120 / 110 / 100 s | 2.15 / **13.02** / 2.40 % | 0 / 3 / 0 | 0 / **394** / 0 |
| scotus | whole-file local attention | 6.01 % | 6 | 146 |
| scotus | upstream fixed 120 s cutter | 4.54 % | 4 | 46 |
| scotus | shipped slicer, at 120 / 110 / 100 s | 5.42 / 5.89 / 5.30 % | 4 / 5 / 4 | 94 / 137 / 111 |

On Wilde, a clean single-narrator audiobook, the stand-in reference was right
and the chunked runs genuinely lost whole paragraphs; which paragraphs depends
only on where the cuts fall. On SCOTUS the stand-in reference **also** drops
speech (146 words), so the first finding's "reference insertions" there were
really reference deletions.

### Adjudicating Prime's recordings without ground truth

Two independent models give a second opinion on every disputed stretch (≥ 10
words one system has and another lacks): **Whisper large-v3** (openai, pinned
`06f233fe`, transformers sequential long-form decoding; its word times are
spread evenly within segments, so it gets a ±3 s window) and **Canary-1b-v2**
(the copy inside Scriberr's env, sha-matched to HF `d4557063`; cut in pauses,
no overlap, ±1 s window). "Speech is real" needs both to have ≥ 50 % of the
disputed words; "no speech" needs both under 20 %; anything else is ambiguous.

**The adjudicator is calibrated on the public files first**, where ground truth
gives the right answer. It decided 129 of 140 disputed stretches and **got all
129 right**; it abstained on 11, mostly crosstalk that the official transcript
renders differently. Whisper large-v3 on its own scores 1.96 % (wilde) and 3.93 %
(scotus) WER against ground truth with **zero** dropouts, so for scoring Prime's
files it serves as the reference, and a dropout counts only if Canary
independently has the words. On the public files that metric reproduces the
ground-truth clean-speech dropout counts within a few percent (1,060 vs 1,116;
287 vs 310; 240 vs 246; 5,450 vs 5,513; 515 vs 526 words).

### Controls and floor

- **A-vs-A.** Production v3 (full attention) is byte-identical across three
  separate processes at every placement tested. **Local attention is not:** at
  the same SCOTUS placement three runs dropped 68, 169 and 100 words (WER 4.73 to
  6.32 %). Its numbers below carry that run-to-run noise.
- **Positive control.** Copies of both public files with 30, 15, 5 and 4 s of
  audio replaced by digital silence (the ground truth still holds those words):
  the 30, 15 and 5 s stretches (15 to 73 words) were reported in all 36 runs; the
  4 s stretch (11 to 14 words) in 11 of 12.
- **Null control.** In those same runs, chunks that touch no silenced stretch get
  byte-identical input; with full attention their words were identical and their
  dropouts equal in all 6 runs, so the instrument manufactures nothing. (With
  local attention they differ; that is the non-determinism above.)
- **A "should-not-matter" perturbation** (−0.5 dB gain) left SCOTUS identical and
  Wilde within its placement spread (1,027 vs 1,116 words). Parakeet normalises
  each mel feature per chunk, so a constant gain is nearly a no-op by design.
- **Sensitivity floor:** a dropout of ≥ 15 words is caught every time (36/36);
  10 to 14 words, 11/12; shorter losses are not counted as dropouts at all (they
  still count in WER). Because the failure is chaotic in cut placement, every
  configuration is run at 8 placements (4 for the private files and the 1.1B
  diagnostics); totals over 8 placements that differ by less than about a
  quarter are not a difference.

## 2. What the real drops look like

Production v3 (the shipped slicer: 120 s chunks, 4 s overlap), 8 cut placements
per public file, each dropout compared with 20 random windows of the same length
from the same file (clean speech; SCOTUS crosstalk split out below):

| | Wilde (one narrator) | SCOTUS (argument) |
|---|---|---|
| dropouts / words | 17 / 1,116 | 29 / 730 |
| start, seconds into its chunk | median 47, **never before 16** | median 53, never before 13 |
| runs to the end of its chunk | 35 % | 10 % |
| level vs file speech level (drops / controls) | −1.0 / −1.35 dB | −0.1 / −2.0 dB |
| local SNR (drops / controls) | 42 / 41 dB | 29 / 27 dB |
| speaking rate (drops / controls) | 2.4 / 2.5 words/s | 4.0 / 3.2 words/s |
| pause just before (drops / controls) | 0.62 / 0.37 s | 0.02 / 0.02 s |
| overlapped speech, Sortformer (drops / controls) | 0 / 0 | **16 % / 0 %** of the window |
| non-English words within ±10 s | 0 | 0 |

Two different things are being counted:

- **A. Long-context collapse.** Clean speech lost in stretches of tens of seconds
  (up to 60 s), beginning at a sentence boundary well into a long chunk, often
  running to the chunk's end. It is **not explained by the audio**: level, SNR
  and rate match the controls, there is one speaker, and nothing is non-English.
  Which stretches go is chaotic: it moves with the cut placement, and on Wilde
  no word is lost by more than 75 % of placements.
- **B. Crosstalk.** On SCOTUS, a justice's interjection over counsel ("Well,
  wait a minute…") is lost at the same few spots in almost every run. A
  single-stream model transcribes one voice; the official transcript records
  both. That is a limit of single-stream ASR and of the transcript convention,
  not the failure this investigation is about, so it is **separated out**
  (diarized overlap ≥ 10 % of the stretch) in every comparison below.

Every comparison below counts **clean-speech dropped words per transcript**, mean
with a 95 % bootstrap interval over cut placements.

**Language ID is not the cause.** v3's only non-ASCII output is legitimate French
names in the text (Mallarmé, Comédie), never near a drop, and English-only v2
collapses too. Prime's recordings are English (function-word share 0.35 to 0.39,
the same as the public English files at 0.39 to 0.44; no non-ASCII letters).

## 3. Mechanism

- **The encoder output is degraded, not just the decoder.** On the three chunks
  that went fully quiet (0 words after the drop began), a fresh decoder state
  started on the same full-attention encoder output recovered only 22 to 51 % of
  the ground-truth words there. The same audio encoded on its own recovered 72 to
  102 %, and local attention over the full chunk 97 to 102 % (a fourth, partial
  drop: 98 %, 57 %, 98 %; a control chunk: all ≈ production). That is n = 4 hand-picked
  chunks, so it points the way; the unbiased tests below carry the weight.
- **It belongs to the weights, not to TDT decoding.** On the same slices,
  `parakeet-tdt-1.1b` (TDT), `parakeet-rnnt-1.1b`, `parakeet-ctc-1.1b`,
  `parakeet-ctc-0.6b` and both heads (TDT and CTC) of `parakeet-tdt_ctc-1.1b` all
  lose **0** words on Wilde. v3 and v2, the Granary-era 0.6B TDT models, collapse.
  The new `parakeet-unified-en-0.6b` (RNN-T) collapses once in 8.
- **It is a knife edge.** Deterministic for a given input (v3 at full attention is
  byte-identical across processes), but the float-level non-determinism of the
  local-attention kernel is enough to flip whether a stretch is transcribed
  (68 / 169 / 100 words at one placement).
- **Recording style matters, differently per weight.** Wilde and p2 are gated
  recordings (pauses near −64 and −74 dBFS); SCOTUS and p1 never go quiet (room
  tone at −40 and −36 dBFS). v2 collapses badly on Wilde but never on the other
  three; a −50 dBFS noise floor more than halves v3's and v2's Wilde losses but
  makes SCOTUS worse. Not a clean lever.

## 4. Hypotheses tested (unbiased: full files at 8 placements; clean-speech words per transcript)

| hypothesis | test | Wilde | SCOTUS | Prime p1 / p2 | verdict |
|---|---|---|---|---|---|
| — | **production v3** | 140 [48–240] | 66 [41–88] | 50 [40–63] / 51 [25–82] | baseline |
| (a) decoding | CUDA graphs on | identical output | identical | | no effect |
| (a) | greedy (per-frame) instead of greedy_batch | identical output | identical | | no effect |
| (a) | max_symbols 20 | identical at its 3 placements | identical | | no effect |
| (a) | TDT beam 4 (text only; NeMo 2.5.3 cannot give timestamps with TDT beam), all dropouts, vs greedy under the same slicing | 194 vs 136 | 488 vs 57 | | **worse**, 3–13× slower, unusable in Scriberr |
| (b) context | 60 s slices | 88 [41–138] | 61 [41–85] | 31 / 31 | within the floor |
| (b) | 30 s slices | 165 [95–236] | 77 [49–105] | 32 / 13 | worse on Wilde |
| (b) | local attention ±64 frames in 120 s slices | 34 [21–47] | 97 [72–121] | | trades one file for the other |
| (b) | local attention ±128 | 39 [24–51] | 128 [92–168] | 0 / 3 | trades; **non-deterministic** |
| (b) | local attention ±256 | 31 [15–47] | 80 [43–120] | 60 / 64 | no help on Prime's files |
| (c) preprocessing | −0.5 dB gain (null) | 128 | identical | 54 / 30 | no effect (the null) |
| (c) | loudness normalisation to −20 dBFS, peak-safe | **498** [382–613] | 66 | | **worse** on quiet audio |
| (c) | soxr resampling instead of ffmpeg (all dropouts; production on the same measure: 140 / 91) | 206 | 92 | | within the floor |
| (c) | −50 dBFS noise floor | 59 [30–87] | 87 [74–100] | | helps Wilde, hurts SCOTUS |
| (d) weights | see § 5 | | | | the lever |
| new | **re-transcribe speech gaps ≥ 3 s** (patch 0002) | **19 [0–38]** | **13 [3–25]** | **7 [0–22] / 5 [0–15]** | **fixes most of it** |

WER against ground truth moves the same way: production v3 4.83 % / 5.33 % median
(Wilde / SCOTUS) against 2.40 % / 4.47 % with the gap retry.

## 5. Candidates

Clean-speech **dropped words per transcript** (mean, 95 % bootstrap interval over
placements; public: against ground truth, 8 placements; Prime's: against Whisper,
confirmed by Canary, 4 placements) and **median WER** (public: against ground
truth; Prime's: against Whisper). **Memory**: per-process peak on the 35-min file
(p1), nvidia-smi every 0.2 s, only the run's own container PIDs, n = 3, zero
spread in every case. **Speed**: median wall time per job for the 35.3-min file
including model load, n = 3 (spread ±5 s). "CLI" rows ran the actual patched
Scriberr scripts under Scriberr's invocation; "lab" rows ran the lab harness
because Scriberr's env cannot run them as-is.

| candidate | Wilde words / WER | SCOTUS words / WER | p1 words / WER | p2 words / WER | peak MiB | job time | old GPU 1 budget (5,496) | GPU 3 (+Blender 270 MiB) | punctuation |
|---|---|---|---|---|---|---|---|---|---|
| **v3, production** (0001) | 140 [48–240] / 4.83 % | 66 [41–88] / 5.33 % | 50 / 2.96 % | 51 / 2.93 % | 5,496 (CLI) | 53 s | fits (0 spare) | fits | yes |
| **v3 + gap retry** (0001+0002) | **19** [0–38] / **2.40 %** | **13** [3–25] / **4.47 %** | **7** / 2.52 % | **5** / 2.38 % | 5,506 (CLI) | 48 s | **10 MiB over** | fits | yes |
| v2 | 689 [490–971] / 17.6 % | **0** / 4.55 % | **0** / 2.22 % | **0** / 2.09 % | 5,438 (CLI) | 49 s | fits | fits | yes |
| v2 + gap retry | 151 [116–192] / 5.84 % | **0** / 4.58 % | **0** / 2.25 % | **0** / 2.09 % | 5,438 (CLI) | 49 s | fits | fits | yes |
| parakeet-unified-en-0.6b (NeMo 3.0.0) | 31 [0–92] / **2.21 %** | 22 [14–32] / 4.92 % | **0** / **2.19 %** | **0** / **1.83 %** | 5,438 (lab) | 34 s | fits | fits | yes |
| parakeet-tdt-1.1b | **0** / 2.31 % | 3 / 6.78 % | **0** / 3.30 % | **0** / 2.65 % | 8,884 (lab) | 46 s | no | fits | **no** |
| parakeet-ctc-0.6b | **0** / 2.46 % | 3 / 6.41 % | – | – | 5,360 (lab) | 31 s | fits | fits | **no** |
| canary-1b-v2 (30 s pause cuts) | 11 [4–21] / 9.64 % ⚠ | 5 / 5.25 % | **0** / 3.88 % | **0** / 2.84 % | 10,504 (lab) | 112 s | no | fits | yes |
| *Whisper large-v3 (reference, 1 run)* | *0 / 1.96 %* | *0 / 3.93 %* | – | – | – | – | – | – | *yes* |

⚠ Canary's Wilde WER is its **hallucination loops**: 6 insertion runs across 8
placements, one of 417 words. It barely drops speech but invents it.

Other 1.1B diagnostics (no punctuation, so not candidates): `parakeet-rnnt-1.1b`
0 / 7, `parakeet-ctc-1.1b` 0 / 3, `parakeet-tdt_ctc-1.1b` TDT head 0 / 6 and CTC
head 0 / 0 (Wilde / SCOTUS words per transcript, 4 placements).

Provenance of every weight (all pulled revision-pinned, sha256 equal to the HF
LFS oid, in `/tank/aimodels/huggingface/hub/`):

| repo | revision | licence (read at the raw card) | file sha256 |
|---|---|---|---|
| nvidia/parakeet-tdt-0.6b-v3 (in Scriberr's env) | `541d1f99` | CC-BY-4.0 | `3cbdc858…` |
| nvidia/parakeet-tdt-0.6b-v2 | `ae9ad070` | CC-BY-4.0 | `d99e3995…` |
| nvidia/parakeet-unified-en-0.6b (2026-04-07, newest Parakeet) | `fe53cd88` | **NVIDIA Open Model License** | `ec23ed91…` |
| nvidia/parakeet-tdt-1.1b | `53276c64` | CC-BY-4.0 | `9c563d52…` |
| nvidia/parakeet-rnnt-1.1b | `2acc4c61` | CC-BY-4.0 | `535896f0…` |
| nvidia/parakeet-ctc-1.1b | `20e63a0f` | CC-BY-4.0 | `8e91253d…` |
| nvidia/parakeet-tdt_ctc-1.1b | `675e7868` | CC-BY-4.0 | `4e7ccfdd…` |
| nvidia/parakeet-ctc-0.6b | `ad09ba1c` | CC-BY-4.0 | `bc01f3f8…` |
| nvidia/canary-1b-v2 (in Scriberr's env) | `d4557063` | CC-BY-4.0 | `ae5ef1bf…` |
| openai/whisper-large-v3 (adjudicator only) | `06f233fe` | Apache-2.0 | `a8e94b85…` |

All repo ids were verified with an authenticated HF API call before any pull (no
phantoms). `parakeet-unified-en-0.6b` needs **NeMo 3.0.0**: its card says 2.7.3,
but released 2.7.3 lacks its encoder argument (`att_chunk_context_size`), and its
`.nemo` ships without a `validation_ds` config that `transcribe()` reads (a
two-line shim). It ran in a throwaway env, not Scriberr's (NeMo 2.5.3).

## 6. The fix that works on any weight: re-transcribe speech gaps

The mechanism says the lost audio is fine on its own; only its long-window
context breaks. So after stitching, the buffered script looks for stretches of
**≥ 3 s with no word where at least half the 25 ms frames sit within 12 dB of the
recording's typical speech level**, re-transcribes each on its own (pieces of at
most 60 s, 0.5 s of padding) and splices in the words that land inside it.
Output that triggers no retry is byte-identical to today's.

- **Effect:** v3's clean-speech losses fall 80 to 90 % on all four recordings
  (140 → 19, 66 → 13, 50 → 7, 51 → 5 words per transcript) and WER falls with
  them (Wilde 4.83 → 2.40 %, SCOTUS 5.33 → 4.47 %). It adds **no insertion runs**,
  so it is not making text up.
- **Trigger:** 3 s beat 6 s on Prime's files (p1 7 vs 27 words per transcript)
  and matched it on the public ones; 6 retries per 35-minute transcript were
  typical.
- **Cost:** peak 5,506 MiB vs 5,496 (the retry's shorter chunk shifts the
  allocator by 10 MiB); job time unchanged within the ±5 s run-to-run spread.
- **Residual:** stretches where the model still emits a few words (no clean gap),
  and retries that come back short. It does not rescue v2 on the audiobook
  (689 → 151).
- **Validity:** the production implementation (after code review) reproduces
  the measured lab version at every shared placement: identical dropped words
  on all 16 file-placement pairs tried and WER equal or up to 0.02 points lower
  (it now removes the odd retried word that repeated its neighbour). It is
  deterministic across processes, and the JSON seam checks pass for both
  scripts with the rebuilt segments.
- **Hardening from review:** a failing retry piece is skipped and a failing retry
  keeps the first pass (never loses a finished transcript); a piece that looks
  like a hallucination loop (mostly one repeated token, or > 7 words/s) is not
  spliced in; retried copies of the words at a gap's edge are dropped; audio goes
  through a per-run temp directory; frame levels use bounded memory (the first
  version needed ~1.7 GB of RAM for 35 min); frame indexing is correct at any
  sample rate. **Residual risk:** the gap detector is an energy test, so a loud
  non-speech stretch (a music bed) is retried; the loop guard is the only check
  on what comes back. None of the four recordings has music.

## 7. Recommendation

| claim | strength | basis | reversibility |
|---|---|---|---|
| The drops are real speech lost by Parakeet, not a reference artefact | **insist** | measured against ground truth on 2 files, 129/129 adjudications correct on the calibration | n/a |
| Ship patch 0002 (gap retry on by default, v3 weights) | **strongly recommend** | measured: 80–90 % fewer lost words on all 4 recordings, lower WER, no new insertions, n = 4–8 placements each | reversible (image rollback) |
| Keep v3 rather than switch to v2 | **lean** | measured: v2 is perfect on Prime's two files and SCOTUS but loses 151 words per transcript on read speech even with the retry, and is English-only; the right answer depends on what Prime transcribes | reversible (one env var) |
| Evaluate `parakeet-unified-en-0.6b` as the next weight, as its own project | **lean** | measured: the best-balanced punctuating Parakeet (0 / 0 on Prime's files, lowest WER there); but it needs NeMo 3.0.0 in Scriberr's env (Canary and Sortformer too), a packaging shim, and a licence change from CC-BY-4.0 to the NVIDIA Open Model License | costly to reverse (env rebuild) |
| Do not use local attention, beam search, shorter slices, loudness normalisation or a noise floor | **recommend against** | measured: each worse or mixed, local attention also non-deterministic | reversible |
| Do not switch to Canary-1b-v2 | **recommend against** | measured: hallucination loops (up to 417 invented words) and twice the memory | reversible |
| Do not use the 1.1B or CTC Parakeets | **recommend against** | read at source: no punctuation or casing, which Scriberr's segmentation needs | reversible |

## 8. The integration patch

`stacks/scriberr/patches/proposed/0002-parakeet-model-path-and-gap-retry.patch`
(on top of 0001, upstream `a353078`; sha256 `e3098bf2…7e62`). It is in
`proposed/`, so `scripts/scriberr-rebuild` does **not** apply it until it is moved
up a directory. What it does:

1. **Gap retry** (§ 6) in both scripts, `--retry-gaps SECS` (default 3, 0 off).
   Go never passes the flag, so the default is the behaviour; the JSON gains
   `retried_gaps`.
2. **`PARAKEET_MODEL_PATH`**: an absolute path (or one relative to the env) to
   the `.nemo` both scripts load; default unchanged. The JSON `model` field now
   reports the file actually loaded, and the Go adapter records it as
   `ModelUsed` instead of the hardcoded "parakeet-tdt-0.6b-v3". **No model is
   ever swapped in under v3's filename.**
3. Tests: 18 new unit tests (39 in total), and upstream's own standard and
   buffered tests pass in the built image. Reviewed at high effort; all ten
   findings fixed (§ 6).

**Validated as a build** (`scripts/scriberr-rebuild --suffix dropout2 --budget
5600 --patches stacks/scriberr/patches:stacks/scriberr/patches/proposed`): all
stages PASS, including the embed of both scripts, 39 unit tests, the JSON seam
for the long- and the short-audio script, and memory (5,506 MiB). The image
`scriberr:local-blackwell-a353078-dropout2` exists on fv-ml1 and is **not
deployed**.

**To ship it (Prime's call):** either deploy the already-built
`scriberr:local-blackwell-a353078-dropout2` per `patches/README.md` § Deploy, or
move the patch up into `stacks/scriberr/patches/` first (so it becomes part of
the carried set) and rebuild under a new suffix with `--budget 5600` (see the
note on the budget).

**To also switch weights (only if Prime chooses v2 or, later, another weight):**
in `stacks/scriberr/compose.yaml`, mount the shared model cache read-only and
name the pinned file. The revision is visible in the path and recorded in every
transcript's metadata:

```yaml
    volumes:
      - /tank/aimodels/huggingface:/models:ro
    environment:
      - PARAKEET_MODEL_PATH=/models/hub/models--nvidia--parakeet-tdt-0.6b-v2/snapshots/ae9ad07059c7c739ffaf932226a8fe64ae2620b0/parakeet-tdt-0.6b-v2.nemo
```

**How it survives upgrades:** `scriberr-rebuild` re-applies 0001 and 0002 to any
pinned upstream sha and stops on a conflict; the model choice lives in our
compose file, not in the image or the env directory, so an upgrade cannot
silently change it. If upstream ever grows its own model selection, 0002 is
dropped in favour of it.

**Budget note:** the rebuild script's default memory budget is still GPU 1's old
5,496 MiB. With Scriberr on GPU 3 that number no longer protects anything, but
the patched build peaks at 5,506, so either pass `--budget` or retire the old
default (a one-line change; left for Prime or the coordinator).

## 9. Harness, data and reproduction

All code is in fv-ml1 `/tank/spikes/scriberr-slicer/code/dropout/`: `lab.py`
(the production pipeline with every knob), `gtscore.py`, `boot.py`,
`adjudicate.py`, `adj_score.py`, `characterize.py`, `probe.py`, `fit.sh`,
`fitlab.sh`, plus the run scripts and configs. Metrics are in `…/metrics/`. The
public audio, ground truth and transcripts are in `…/public/` and `…/gt/`;
Prime's are in `…/private/` (mode 700). Throwaway envs: `…/envs/nemo300`
(NeMo 3.0.0, for the unified model).

Every run was a transient `--rm` container on GPU 3, with Scriberr's env and the
model cache mounted read-only. GPU 1 and the live Scriberr container were not
touched.