esh-pfi-infrastructure/persistent-memory.d/2026-10-06-scriberr-diarization-fixed.md

[2026-10-06] Scriberr diarization fixed; model caches on /tank; HF_TOKEN a placeholder; GPU profiles

• Diarization FIXED 2026-10-06 ~1320 (Prime: "whisper doesn't work at all, need diarized transcripts"). Three faults: no HF_TOKEN (pyannote is gated). FINAL: the gated models were downloaded once, caches moved to /tank (HF_HOME/TORCH_HOME/NLTK_DATA under /app/whisperx-env), and HF_TOKEN is a placeholder (Prime 1330: "move the downloads and drop the token"). Cached gated models load with an invalid token; a NEW gated model needs one download with vault fv-ml1/hf-token. Saved profiles added: WhisperX large-v3 · GPU · speakers and Parakeet · GPU · speakers. Prime's 1 h transcript finished diarized at 1336 (3 speakers, 3m30s). ⚠ My 1330 recreate killed his 1326 run (an idle check that printed but did not gate); see auto-memory feedback_idle_check_must_gate_the_restart. torchcodec 0.16 in the pyannote env against torch 2.8 (AudioDecoder undefined; pinned to 0.7 by hand, durable as patches/0003, applied at the next rebuild); and the WhisperX adapter's CPU/small/float32 defaults. Both diarization paths verified on GPU 3. → stacks/scriberr/README.md § Diarization

• Faults, in the order found: (1) no HF token for the gated pyannote models (WhisperX died after a 12m45s CPU transcription; Parakeet+pyannote refused up front); (2) the pyannote env had torchcodec 0.16.0 (needs torch ≥2.11/CUDA 13) beside torch 2.8.0+cu128, so AudioDecoder was undefined; pinned to 0.7 by hand, made durable as stacks/scriberr/patches/0003 (applied at the next rebuild; git apply --check passes on a353078+0001+0002); (3) the WhisperX adapter defaults to cpu/small/float32.

• The token step went in two stages. First Prime's HF token was passed through compose (vault fv-ml1/hf-token, account lkraven, read role). Then Prime said "move the downloads and drop the token": HF_HOME/TORCH_HOME/NLTK_DATA now point under /app/whisperx-env (the /tank bind mount; ~6 GB had been in the container layer and was lost on every recreate), and HF_TOKEN is a compose placeholder. Measured: cached gated models load with an invalid token (huggingface_hub falls back to the cache on the 401), and public models still download (200). Scriberr only checks that the token is non-empty.

• Saved profiles (inserted as uid 10001, no restart; DB backup scriberr.db.bak-20261006-profiles): WhisperX large-v3 · GPU · speakers, Parakeet · GPU · speakers. Prime's 1 h upload then finished diarized: Parakeet 46 s (11,985 words), pyannote 2m38s, 3 speakers.

• ⚠ The 1330 recreate killed Prime's 1326 re-run: the GPU-busy check printed "1" but did not gate. Auto-memory feedback_idle_check_must_gate_the_restart.

• Commits: 5896a74, ec888c3, efd31b4. Full write-up: stacks/scriberr/README.md § Diarization.