fix(scriberr): model caches on /tank; HF_TOKEN is a placeholder (no real credential)
HF_HOME, TORCH_HOME and NLTK_DATA now point under /app/whisperx-env (the /tank bind mount). Before, ~6 GB of models sat in the container layer and every recreate threw them away. The cache was copied across (98/2/78 files, apparent sizes identical). HF_TOKEN is now a placeholder. Measured: with an invalid token, cached gated pyannote models still load (huggingface_hub falls back to the cache on the 401), and public models still download (HTTP 200). Scriberr only checks that the token is non-empty. Re-verified on GPU 3: pyannote community-1 and WhisperX+3.1 diarization both exit 0 from the /tank cache with no downloads.
This commit is contained in:
@@ -176,7 +176,7 @@ Every vLLM seat on a shared card (GPU 0, GPU 1) has byte-pinned KV (`--kv-cache-
|
||||
|
||||
### Scriberr (fv-ml1 GPU 3)
|
||||
|
||||
- **Diarization FIXED 2026-10-06 ~1320 (Prime: "whisper doesn't work at all, need diarized transcripts").** Three faults: no `HF_TOKEN` (pyannote is gated; now passed via compose, vault `fv-ml1/hf-token`); torchcodec 0.16 in the pyannote env against torch 2.8 (`AudioDecoder` undefined; pinned to 0.7 by hand, durable as `patches/0003`, applied at the next rebuild); and the WhisperX adapter's CPU/small/float32 defaults. Both diarization paths verified on GPU 3. → `stacks/scriberr/README.md` § Diarization
|
||||
- **Diarization FIXED 2026-10-06 ~1320 (Prime: "whisper doesn't work at all, need diarized transcripts").** Three faults: no `HF_TOKEN` (pyannote is gated). FINAL: the gated models were downloaded once, caches moved to /tank (`HF_HOME`/`TORCH_HOME`/`NLTK_DATA` under `/app/whisperx-env`), and `HF_TOKEN` is a placeholder (Prime 1330: "move the downloads and drop the token"). Cached gated models load with an invalid token; a NEW gated model needs one download with vault `fv-ml1/hf-token`. torchcodec 0.16 in the pyannote env against torch 2.8 (`AudioDecoder` undefined; pinned to 0.7 by hand, durable as `patches/0003`, applied at the next rebuild); and the WhisperX adapter's CPU/small/float32 defaults. Both diarization paths verified on GPU 3. → `stacks/scriberr/README.md` § Diarization
|
||||
|
||||
- **LIVE `scriberr:local-blackwell-a353078-dropout2`:** upstream a353078 plus patch 0001 (overlap slicer) and patch 0002 (gap retry, `PARAKEET_MODEL_PATH`), carried LOCALLY ONLY (Prime 2026-10-01: no upstream). v3 stays. `scripts/scriberr-rebuild` re-applies both. → `persistent-memory.d/2026-09-30-scriberr-slicer-gap-retry-gpu3.md`
|
||||
- **URL is https://scriberr.nh3.phasefinal.com since 2026-10-01** (fronted by the fleet TLS caddy on nh3-dev; Caddyfile backup `.bak-20261001-scriberr`). Prime's "can't initialize recorder" was the SecureContext rule: `getUserMedia` is refused on http://. The HTTPS name is in `ALLOWED_ORIGINS` (`playbooks/scriberr-https-origin.yaml`); http://10.251.50.54:8080 still works except for recording. ⚠ `/opt/docker/compose/scriberr` is lkraven-owned, so `deploy-stack.sh` as infra-ops FAILS there (Permission denied); upload single files with `elway --upload --sudo --owner lkraven:lkraven`.
|
||||
|
||||
@@ -138,11 +138,11 @@ rebuild — a local patch to carry, which is why it was not done.
|
||||
|
||||
Prime: "whisper doesn't work at all, and I need diarized transcripts." There were three separate faults, and all three looked like "transcription is broken":
|
||||
|
||||
1. **No Hugging Face token.** pyannote's diarization models are gated. Every diarized job died at the diarize step: WhisperX with `GatedRepoError 401` on `pyannote/speaker-diarization-3.1`, but only after a full CPU transcription (12m45s for a 1 h file), and Parakeet+pyannote with "HuggingFace token is required". **Fix:** `HF_TOKEN` is passed through compose. Its value is in the host `.env` only (vault `fv-ml1/hf-token`, account lkraven, read role; access to `speaker-diarization-3.1`, `segmentation-3.0` and `speaker-diarization-community-1` was checked).
|
||||
1. **No Hugging Face token.** pyannote's diarization models are gated. Every diarized job died at the diarize step: WhisperX with `GatedRepoError 401` on `pyannote/speaker-diarization-3.1`, but only after a full CPU transcription (12m45s for a 1 h file), and Parakeet+pyannote with "HuggingFace token is required". **Fix, final form (same day, Prime: "move the downloads and drop the token"):** the gated models were downloaded ONCE with a real token. All model caches now live on /tank (`HF_HOME`, `TORCH_HOME`, `NLTK_DATA` under `/app/whisperx-env`; before this they sat in the container layer and every recreate threw away ~6 GB). `HF_TOKEN` is a **placeholder** in compose, so no real credential lives in Scriberr. Measured: with an invalid token, cached gated models still load (huggingface_hub falls back to the cache on the 401), and public models still download (HTTP 200). The placeholder exists only because Scriberr refuses an EMPTY token. **A new gated model** needs one first download with the real token (vault `fv-ml1/hf-token`).
|
||||
2. **The separate pyannote env had a torchcodec that cannot load.** uv resolved torchcodec 0.16.0 beside torch 2.8.0+cu128. 0.16 needs torch ≥2.11 and CUDA 13 (`libnvrtc.so.13`), so pyannote.audio 4 has no decoder and fails with `name 'AudioDecoder' is not defined`. **Fix (live, by hand):** `torchcodec==0.7.*` (its table pairs 0.7 with torch 2.8), pinned in `/tank/scriberr/whisperx-env/pyannote/pyproject.toml`, then `uv lock && uv sync` as uid 10001; the old files are kept as `*.bak-20261006`. Scriberr only rewrites that pyproject if its import test fails at startup. **Durable fix:** `patches/0003-pyannote-env-pin-torchcodec.patch` pins torch/torchaudio 2.8 and torchcodec 0.7 in the embedded pyproject, so it lands at the next rebuild.
|
||||
3. **WhisperX defaults to CPU.** The adapter's defaults are `device cpu`, `model small`, `compute_type float32` (`whisperx_adapter.go`). A quick transcribe with no profile ran exactly that. **Use** device `cuda`, model `large-v3`, compute `float16`.
|
||||
|
||||
**Verified on a 4-min, multi-speaker clip of a real upload, GPU 3:** WhisperX large-v3/cuda/float16 + `pyannote/speaker-diarization-3.1` exited 0 in 65 s with 2 speakers. The pyannote env script with `speaker-diarization-community-1` (the adapter's only option, used by the Parakeet profile) exited 0 in 11 s with 3 speakers. The two pipelines disagree on the speaker count, so set min/max speakers when you know it. WhisperX's own env (pyannote.audio 4.0.4) loads 3.1 fine and feeds the pipeline in-memory audio, so it never needed torchcodec.
|
||||
**Re-verified with the placeholder token and the /tank cache (60 s clip): community-1 exited 0 in 8 s, WhisperX+3.1 exited 0 in 28 s, both 2 speakers, with no downloads and no 401s.** First verification, with the real token, on a 4-min multi-speaker clip of a real upload, GPU 3: WhisperX large-v3/cuda/float16 + `pyannote/speaker-diarization-3.1` exited 0 in 65 s with 2 speakers. The pyannote env script with `speaker-diarization-community-1` (the adapter's only option, used by the Parakeet profile) exited 0 in 11 s with 3 speakers. The two pipelines disagree on the speaker count, so set min/max speakers when you know it. WhisperX's own env (pyannote.audio 4.0.4) loads 3.1 fine and feeds the pipeline in-memory audio, so it never needed torchcodec.
|
||||
|
||||
## Optional: summarisation via the LiteLLM gateway
|
||||
|
||||
|
||||
@@ -72,15 +72,26 @@ services:
|
||||
# and takes out the Parakeet + Sortformer backends (WhisperX survives).
|
||||
# `copy` trades a little disk and time for it actually working.
|
||||
- UV_LINK_MODE=${SCRIBERR_UV_LINK_MODE:-copy}
|
||||
# Hugging Face token for the GATED pyannote models (speaker-diarization-3.1,
|
||||
# segmentation-3.0). Without it every diarized job dies at the diarize step:
|
||||
# WhisperX with a 401 GatedRepoError AFTER the whole transcription has run, and
|
||||
# Parakeet+pyannote with "HuggingFace token is required". So it looks as if
|
||||
# nothing works. Scriberr's config and both adapters read HF_TOKEN
|
||||
# (internal/config/config.go), and it reaches the uv subprocesses. Value: vault
|
||||
# `fv-ml1/hf-token` (account lkraven, read role; access to the gated repos was
|
||||
# checked 2026-10-06). It lives only in the host .env, never in git.
|
||||
- HF_TOKEN=${HF_TOKEN:-}
|
||||
# ── Model caches on /tank, not in the container layer (Prime, 2026-10-06) ──
|
||||
# Before this they lived under /home/appuser, which is the container's writable
|
||||
# layer, so EVERY recreate threw away ~6 GB (Whisper large-v3, pyannote, the
|
||||
# alignment model, NLTK) and the next job had to download it all again.
|
||||
# /app/whisperx-env is the persistent /tank/scriberr/whisperx-env bind mount.
|
||||
- HF_HOME=/app/whisperx-env/hf-cache
|
||||
- TORCH_HOME=/app/whisperx-env/torch-cache
|
||||
- NLTK_DATA=/app/whisperx-env/nltk_data
|
||||
# ⚠ HF_TOKEN is a PLACEHOLDER on purpose. No real credential lives here.
|
||||
# pyannote's diarization models are gated: downloading them needs a token whose
|
||||
# account accepted their terms. LOADING an already-downloaded copy does not.
|
||||
# Measured 2026-10-06: with an invalid token, huggingface_hub's check gets a 401
|
||||
# and falls back to the cached copy, both online and with HF_HUB_OFFLINE=1.
|
||||
# Public repos (any faster-whisper size) still download with an invalid token
|
||||
# (HTTP 200). The placeholder exists only because Scriberr refuses to start
|
||||
# pyannote diarization when HF_TOKEN is EMPTY (pyannote_adapter.go checks that
|
||||
# it is non-empty, never that it is valid).
|
||||
# To add a NEW gated model: run its first download once with a real token
|
||||
# (vault `fv-ml1/hf-token`) set for that one run, then put the placeholder back.
|
||||
- HF_TOKEN=${HF_TOKEN:-hf_placeholder_models_cached_on_tank}
|
||||
# ── Memory settings, measured while Scriberr shared GPU 1 (2026-09-30; it moved to
|
||||
# GPU 3 at 1322 the same day, so the 5.5 GB budget no longer binds, but the values stand). Prime: Scriberr shared GPU 1 with
|
||||
# intern-decision, which holds ~9.7 GB resting / 10.3 GB peak). ──────────
|
||||
|
||||
Reference in New Issue
Block a user