esh-pfi-infrastructure/persistent-memory.d/2026-09-30-scriberr-slicer-gap-retry-gpu3.md

Scriberr: CUDA OOM → slicer patch → Parakeet gap retry → GPU 3 (2026-09-30)

The OOM (0124): Scriberr's Parakeet path hit CUDA OOM on GPU 1 beside SemIf. Prime took SemIf offline at 0135, and the 35-min job then ran clean.

• First fix, live at 0900: PARAKEET_CHUNK_THRESHOLD_SECS=120 plus PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True.
• Peaks: 9,384 MiB at 300 s slices; 6,510 at 120 s; a ~5.6 GB fixed floor; 5,496 at 120 s with expandable_segments.
• Scriberr passes os.Environ() to its uv subprocess.

Slicer patch 0001 (Prime: "build the slicer"; docs/pfi/scriberr-slicer-bench-2026-09-30.md):

• A 4 s overlap inside the 120 s limit, handing over at a word both chunks transcribed.
• Damaged cuts fell from 52% to 22% against a 19% background (floor ±0.08; 4 files × 3 placements).
• Pause-aware cutting measured neutral and is opt-in.

Dropout investigation (Prime: include a different Parakeet weight; docs/pfi/parakeet-dropout-investigation-2026-09-30.md):

• The v3 losses are REAL against ground truth (SCOTUS official transcript, Gutenberg #38916): 140 / 66 / 50 / 51 words per transcript.
• Cause: the v2/v3 0.6B weights collapse deep in long windows. The 1.1B models do not, but they have no punctuation.
• No decoder, context, loudness or resampling fix worked.

Patch 0002 (gap retry plus PARAKEET_MODEL_PATH) re-transcribes any ≥3 s stretch of speech that produced no words, cutting the losses 80–90%.

• LIVE at 1602 as scriberr:local-blackwell-a353078-dropout2. Prime: "basic fix, no surgery for the new toolkit", so v3 stays.
• Rollback: .env.bak-20260930-pre-dropout2, or SCRIBERR_IMAGE set to slicer1.

Placement: moved to GPU 3 at 1322 (Prime), as an on-demand tenant like Blender. It steps aside to irv-ml1's A6000 when a full-size seat claims GPU 3; the A6000 is shared with bursty ComfyUI work.

Mechanics that survive upgrades:

• Scriberr REWRITES its Parakeet scripts from the go:embed copy on every environment prepare, so hand edits in the env dir are wiped.
• Patches live in stacks/scriberr/patches/ and are applied by scripts/scriberr-rebuild: pinned sha, git apply --check, a distinct tag, then the embed, unit, seam and memory stages. The budget is a 5,600 MiB regression guard.
• Upstream is quiet: 1 commit in 90 days, and the maintainer is restarting.
• The upstream PR for 0001 is prepared (patches/upstream-pr/) but NOT opened; that needs Prime.