esh-pfi-infrastructure/stacks/scriberr

scriberr — self-hosted transcription + diarization (fv-ml1, GPU1)

Web UI for transcribing audio/video locally. WhisperX (Whisper + pyannote speaker diarization) with NVIDIA Parakeet/Canary also selectable; SQLite for state; optional summarisation and transcript chat against any OpenAI-compatible endpoint.

• Host: fv-ml1 (10.251.50.54) — GPU 3
• URL: https://scriberr.nh3.phasefinal.com (since 2026-10-01). The fleet TLS caddy on nh3-dev (~/.config/caddy-fleet/Caddyfile, wildcard *.nh3.phasefinal.com) reverse-proxies it to fv-ml1:8080. Use this name for recording: the browser recorder calls getUserMedia, which browsers refuse on http:// origins, and the page then sits at "Initializing recorder..." (console: "Failed to initialize recorder"). The name must also be in SCRIBERR_ALLOWED_ORIGINS, or Scriberr's CORS check rejects the app's own API calls (playbooks/scriberr-https-origin.yaml). The plain http://10.251.50.54:8080 still works for everything except recording.
• Upstream: https://github.com/rishikanthc/Scriberr

The image is built locally, and that is not incidental

fv-ml1's RTX PRO 6000 Blackwell cards are sm_120. Upstream's published images do not cover that:

image	built for	usable here
ghcr.io/rishikanthc/scriberr	CPU	yes, but no GPU
ghcr.io/rishikanthc/scriberr-cuda	sm_61 … sm_89 (Pascal→Ada)	no — no sm_120 kernels
ghcr.io/rishikanthc/scriberr-cuda-blackwell	sm_120	does not exist — documented in the upstream README but never published; GHCR returns no tags (checked 2026-08-23)

The sm_120 path upstream actually ships is Dockerfile.cuda.12.9 (CUDA 12.9.1 + cuDNN, PYTORCH_CUDA_VERSION=cu128), built from source. So we build it. Do not "simplify" the compose back to the published scriberr-cuda image — it will fail on these cards or quietly fall back to CPU.

Rebuilding

We carry local patches (patches/, currently the pause-aware Parakeet slicer), so a rebuild is one command from nh3-dev, pinned to an upstream sha:

scripts/scriberr-rebuild --sha <full upstream sha> --suffix slicer1

It makes a clean clone in /opt/docker/src/scriberr-<sha7>-<suffix> on fv-ml1, git apply --checks the patches (a conflict stops it), builds scriberr:local-blackwell-<sha7>-<suffix> beside the old images, and checks the embed, the unit tests, the Go↔Python JSON seam, and the GPU memory budget. Deploying it is a separate manual step: patches/README.md § Deploy.

The old checkout at /tank/scriberr/src/Scriberr (lkraven-owned, shallow) built the original scriberr:local-blackwell and is left as it was.

Deploy

# from this workstation
scripts/deploy-stack.sh fv-ml1 scriberr

Then on the host, the usual:

cd /opt/docker/compose/scriberr
docker compose config          # dry parse first
docker compose up -d scriberr  # target the service, not the whole stack

Storage — deliberately on /tank

/var/lib/docker on fv-ml1 sits on zroot at ~87% used. Whisper, pyannote and NeMo weights are multi-GB and land in the whisperx-env volume, so both mounts are bind-mounted onto /tank (4+ TB) instead of named volumes:

host path	container path	holds
/tank/scriberr/data	/app/data	SQLite DB, uploads, transcripts
/tank/scriberr/whisperx-env	/app/whisperx-env	Python env + model weights
/tank/scriberr/src/Scriberr	—	original build checkout (patched builds: /opt/docker/src/scriberr-<sha7>-<suffix>)

Both are owned by uid/gid 1000 to match PUID/PGID.

First run takes a while

On first start the container builds a Python environment and downloads several GB of model weights before the port answers — upstream says "several minutes". The healthcheck therefore has a 600 s start_period; the container will show starting, not unhealthy, during that window. Watch it with:

docker logs -f scriberr

Subsequent starts are fast because the env volume persists.

The PUID trap — read this before "fixing" the uid

This stack runs as uid/gid 10001, not the fleet-usual 1000, and the /tank/scriberr dirs are chowned to match. That is deliberate.

Dockerfile.cuda.12.9 creates appuser at uid 10001 — Ubuntu 24.04's base image already owns uid 1000 as ubuntu, so upstream moved their app user out of the way. It then chowns /app to 10001. But the entrypoint's PUID remapping only chowns /app/data and /app/whisperx-env — not /app itself. So running with PUID=1000 leaves the app unable to open its SQLite database and it crash-loops with:

Failed to connect to database: unable to open database file: out of memory (14)

That message is a red herring twice over: error 14 is SQLITE_CANTOPEN, not an OOM, and the machine has 566 GB of RAM. Diagnosis notes from 2026-08-23:

• SQLite itself writes fine to /tank as uid 1000 — the mount is not at fault.
• The app fails on a plain Docker named volume too — storage is not at fault.
• The published CPU image runs fine at PUID=1000, because in Dockerfile (the non-CUDA one) appuser is uid 1000. Only the CUDA 12.9 variant moved it.
• Same image at PUID=10001 starts clean. That is the whole difference.

If you ever want host files owned by 1000 instead, the fix is to patch Dockerfile.cuda.12.9 to userdel ubuntu and recreate appuser at 1000, then rebuild — a local patch to carry, which is why it was not done.

Gotchas

• SECURE_COOKIES must stay false while served over plain HTTP. At the production default of true the session cookie is marked Secure, the browser drops it, and login appears to succeed then bounces you straight back to the login page with nothing useful in the logs.
• ALLOWED_ORIGINS must list the real origin. Upstream defaults to localhost only; reaching the UI by host IP fails CORS until it is set.
• Never add NVIDIA_VISIBLE_DEVICES=all. Upstream's compose sets it, but here it would override the device_ids reservation and expose both cards — GPU0 belongs to the gen seat.
• This stack is a guest on GPU1, which it shares with the sec seat. If VRAM gets tight, this is the thing that should yield.

Diarization: what broke it and what fixed it (2026-10-06)

Prime: "whisper doesn't work at all, and I need diarized transcripts." There were three separate faults, and all three looked like "transcription is broken":

1. No Hugging Face token. pyannote's diarization models are gated. Every diarized job died at the diarize step: WhisperX with GatedRepoError 401 on pyannote/speaker-diarization-3.1, but only after a full CPU transcription (12m45s for a 1 h file), and Parakeet+pyannote with "HuggingFace token is required". Fix: HF_TOKEN is passed through compose. Its value is in the host .env only (vault fv-ml1/hf-token, account lkraven, read role; access to speaker-diarization-3.1, segmentation-3.0 and speaker-diarization-community-1 was checked).
2. The separate pyannote env had a torchcodec that cannot load. uv resolved torchcodec 0.16.0 beside torch 2.8.0+cu128. 0.16 needs torch ≥2.11 and CUDA 13 (libnvrtc.so.13), so pyannote.audio 4 has no decoder and fails with name 'AudioDecoder' is not defined. Fix (live, by hand): torchcodec==0.7.* (its table pairs 0.7 with torch 2.8), pinned in /tank/scriberr/whisperx-env/pyannote/pyproject.toml, then uv lock && uv sync as uid 10001; the old files are kept as *.bak-20261006. Scriberr only rewrites that pyproject if its import test fails at startup. Durable fix: patches/0003-pyannote-env-pin-torchcodec.patch pins torch/torchaudio 2.8 and torchcodec 0.7 in the embedded pyproject, so it lands at the next rebuild.
3. WhisperX defaults to CPU. The adapter's defaults are device cpu, model small, compute_type float32 (whisperx_adapter.go). A quick transcribe with no profile ran exactly that. Use device cuda, model large-v3, compute float16.

Verified on a 4-min, multi-speaker clip of a real upload, GPU 3: WhisperX large-v3/cuda/float16 + pyannote/speaker-diarization-3.1 exited 0 in 65 s with 2 speakers. The pyannote env script with speaker-diarization-community-1 (the adapter's only option, used by the Parakeet profile) exited 0 in 11 s with 3 speakers. The two pipelines disagree on the speaker count, so set min/max speakers when you know it. WhisperX's own env (pyannote.audio 4.0.4) loads 3.1 fine and feeds the pipeline in-memory audio, so it never needed torchcodec.

Optional: summarisation via the LiteLLM gateway

Scriberr speaks the OpenAI API, so point it at the fleet gateway instead of a paid vendor. In the UI under the AI provider settings:

• base URL: http://10.250.50.70:4000/v1
• model: summarizer (or gen / gen-reasoning)
• key: the shared all-agents gateway key

⚠ That key also reaches paid passthrough models (GLM, Kimi) on a shared tab. Keep the configured model on a free local seat.

Parakeet memory and slicing (measured 2026-09-30)

Moved to fv-ml1 GPU 3 at 1322 on 2026-09-30 (Prime). The GPU 1 budget below no longer binds. Scriberr is an on-demand tenant of GPU 3's reserve: it steps aside to irv-ml1's A6000 when a full-size seat claims GPU 3. A 20-min file was verified on GPU 3 at a 5,496 MiB peak. The settings below were not changed by the move.

Scriberr shares fv-ml1 GPU 1 with intern-decision (9.0 GiB cap, 9,876 MiB card peak). GPU 1's nvidia-smi Free is 15,442 MiB, so the budget left for Scriberr is about 5.5 GB (70 MiB spare at both peaks), so the compose file sets PARAKEET_CHUNK_THRESHOLD_SECS=120 and PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True. The numbers behind those settings are in the compose comments.

Peak GPU memory on a 35-minute file:

setting	peak
300 s slices	9,384 MiB
300 s slices + expandable_segments	6,962 MiB
120 s slices + expandable_segments	5,496 MiB (live setting)

• Whole file in one pass with local attention (--context-left/right 255, standard script): CUDA OOM at >16 GB. It asked for another 6.46 GiB at 9.8 GiB in use. It is not viable on this card budget.
• ⚠ Hand-editing the Parakeet scripts does not persist. Scriberr's PrepareEnvironment rewrites parakeet_transcribe.py and parakeet_transcribe_buffered.py into whisperx-env/parakeet/ from the copies embedded in the Go binary every time it prepares the environment. A change to the slicer has to go into the source checkout (internal/transcription/adapters/py/nvidia/parakeet_transcribe_buffered.py, embedded at build) and be rebuilt. Or it goes upstream (MIT).
• The two env knobs are read by upstream's Go code (parakeet_adapter.go), so they survive image upgrades for as long as upstream keeps them. Re-measure the peak after any upgrade.
• The slicer itself is patched (patches/0001-parakeet-pause-aware-slicer.patch, 2026-09-30). Adjacent 120 s slices now overlap by 4 s and are stitched at a word both transcribed, which cut the share of cuts with an error nearby from 52 % to 22 % against a 19 % background. The overlap sits inside the 120 s, so the peak is unchanged (5,496 MiB, n=3). See patches/README.md and docs/pfi/scriberr-slicer-bench-2026-09-30.md. That bench also found that Parakeet sometimes skips stretches of ≥10 words mid-slice, with or without the patch. Investigated 2026-09-30 (docs/pfi/parakeet-dropout-investigation-2026-09-30.md): the losses are real (against ground truth) and belong to the v2/v3 weights over long windows; a proposed patch, patches/proposed/0002, re-transcribes speech that got no words and cuts them 80–90 %. It is not deployed; that is Prime's call.