parakeet: rewrite on sherpa-onnx; own the wrapper end-to-end

The Shadowfita FastAPI wrapper hit two unfixed upstream bugs on the
first real /transcribe call — chunker return-shape mismatch (open
issue #16) and a `torchaudio.tensor` that doesn't exist (open #10).
Rather than babysit someone else's half-tested code, switched to
sherpa-onnx with the prebuilt int8 Parakeet-TDT tarball from k2-fsa,
and wrote our own ~60-line FastAPI wrapper.

Moving parts now owned in-tree:
  Dockerfile      CUDA 12.8 + cuDNN 9 runtime base, installs
                  sherpa-onnx==1.12.39+cuda12.cudnn9 + fastapi +
                  soundfile + libasound2 (sherpa-onnx links to ALSA
                  at load time even when we never touch a mic).
  app.py          OfflineRecognizer.from_transducer() once at startup;
                  /transcribe and /v1/audio/transcriptions both accept
                  multipart uploads and return {"text": ...}.
  entrypoint.sh   Idempotent model download to /models on first run
                  (~400 MB int8 tarball), then exec uvicorn.

Smoke test: 0.wav (bundled in the tarball, The House of the Seven
Gables excerpt) transcribes cleanly in ~1.2s on GPU.

PARAKEET_MODEL_URL in .env lets you swap to the v3 (25-language)
tarball without touching any other files. Wipe *.onnx + tokens.txt
from the models dir and the entrypoint re-downloads.
This commit is contained in:
vh
2026-04-24 00:18:45 -07:00
parent 82f95d7428
commit 01c5380059
6 changed files with 272 additions and 102 deletions
+20 -29
View File
@@ -5,45 +5,36 @@
# docker compose build
# docker compose up -d
# Pinned git SHA to build from. Bump + rebuild when you want upstream
# fixes. `main` latest as of 2026-04:
# https://github.com/Shadowfita/parakeet-tdt-0.6b-v2-fastapi
PARAKEET_SHA=31c5652b62d09653ad5ea8190c0ad0d35394174d
# Image tag. Bump when you change the Dockerfile / app.py so docker caches
# cleanly.
PARAKEET_TAG=sherpa-onnx-v2
# Host port for the FastAPI server (container listens on 8000)
PARAKEET_PORT=8765
# Bind address. 0.0.0.0 exposes on all interfaces including the WG
# tunnel IP (10.100.79.3). Use 127.0.0.1 to restrict to local-only.
# Bind address. 0.0.0.0 exposes on all interfaces including the WG tunnel IP
# (10.100.79.3). Use 127.0.0.1 to restrict to local-only.
PARAKEET_BIND=0.0.0.0
# Host path for the HuggingFace cache (parakeet-tdt-0.6b-v2 weights
# ~2.5 GB). Persistent across container recreates. Must exist before
# first `up` with ownership matching the container user (root inside
# this image — no UID juggling needed, but the host dir needs to be
# writable by the container).
# Host path for the ONNX model files — encoder/decoder/joiner/tokens.txt.
# Downloaded by the entrypoint on first run if absent. Must exist before
# first `up` (directory, not files).
PARAKEET_MODELS_DIR=/worktank/parakeet/models
# Inference precision. fp16 halves VRAM and is lossless for parakeet
# in practice; use fp32 only if fp16 shows degraded WER for your
# domain audio.
PARAKEET_MODEL_PRECISION=fp16
# Which sherpa-onnx release tarball to fetch on first boot. Default is the
# int8-quantized English-only v2 (~400 MB). Switch to the v3 tarball below
# to cover 25 European languages at a similar size:
# https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8.tar.bz2
PARAKEET_MODEL_URL=https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-nemo-parakeet-tdt-0.6b-v2-int8.tar.bz2
# Batch size for the transcribe queue. Larger = better throughput
# under load at the cost of per-request latency.
PARAKEET_BATCH_SIZE=4
# ONNX Runtime execution provider. `cuda` uses the GPU (requires nvidia
# runtime + matching CUDA/cuDNN in the image). `cpu` falls back to CPU —
# fine for low-volume dev use; ~4-8× slower on this host.
PARAKEET_PROVIDER=cuda
# Max single-clip duration (seconds). Longer inputs get rejected
# by the server with 400. Upstream default.
PARAKEET_MAX_AUDIO_DURATION=30
# Silero VAD threshold (0–1). Higher = stricter about what counts
# as speech (fewer false wake-ups on silence, more chance of clipping
# soft speech). 0.5 is upstream default.
PARAKEET_VAD_THRESHOLD=0.5
# End-to-end processing timeout per request (seconds).
PARAKEET_PROCESSING_TIMEOUT=60
# CPU threads per recognizer session. Irrelevant when provider=cuda;
# only matters for provider=cpu.
PARAKEET_NUM_THREADS=1
# Log level: DEBUG | INFO | WARNING | ERROR
PARAKEET_LOG_LEVEL=INFO