parakeet: rewrite on sherpa-onnx; own the wrapper end-to-end
The Shadowfita FastAPI wrapper hit two unfixed upstream bugs on the first real /transcribe call — chunker return-shape mismatch (open issue #16) and a `torchaudio.tensor` that doesn't exist (open #10). Rather than babysit someone else's half-tested code, switched to sherpa-onnx with the prebuilt int8 Parakeet-TDT tarball from k2-fsa, and wrote our own ~60-line FastAPI wrapper. Moving parts now owned in-tree: Dockerfile CUDA 12.8 + cuDNN 9 runtime base, installs sherpa-onnx==1.12.39+cuda12.cudnn9 + fastapi + soundfile + libasound2 (sherpa-onnx links to ALSA at load time even when we never touch a mic). app.py OfflineRecognizer.from_transducer() once at startup; /transcribe and /v1/audio/transcriptions both accept multipart uploads and return {"text": ...}. entrypoint.sh Idempotent model download to /models on first run (~400 MB int8 tarball), then exec uvicorn. Smoke test: 0.wav (bundled in the tarball, The House of the Seven Gables excerpt) transcribes cleanly in ~1.2s on GPU. PARAKEET_MODEL_URL in .env lets you swap to the v3 (25-language) tarball without touching any other files. Wipe *.onnx + tokens.txt from the models dir and the entrypoint re-downloads.
This commit is contained in:
@@ -5,45 +5,36 @@
|
||||
# docker compose build
|
||||
# docker compose up -d
|
||||
|
||||
# Pinned git SHA to build from. Bump + rebuild when you want upstream
|
||||
# fixes. `main` latest as of 2026-04:
|
||||
# https://github.com/Shadowfita/parakeet-tdt-0.6b-v2-fastapi
|
||||
PARAKEET_SHA=31c5652b62d09653ad5ea8190c0ad0d35394174d
|
||||
# Image tag. Bump when you change the Dockerfile / app.py so docker caches
|
||||
# cleanly.
|
||||
PARAKEET_TAG=sherpa-onnx-v2
|
||||
|
||||
# Host port for the FastAPI server (container listens on 8000)
|
||||
PARAKEET_PORT=8765
|
||||
|
||||
# Bind address. 0.0.0.0 exposes on all interfaces including the WG
|
||||
# tunnel IP (10.100.79.3). Use 127.0.0.1 to restrict to local-only.
|
||||
# Bind address. 0.0.0.0 exposes on all interfaces including the WG tunnel IP
|
||||
# (10.100.79.3). Use 127.0.0.1 to restrict to local-only.
|
||||
PARAKEET_BIND=0.0.0.0
|
||||
|
||||
# Host path for the HuggingFace cache (parakeet-tdt-0.6b-v2 weights
|
||||
# ~2.5 GB). Persistent across container recreates. Must exist before
|
||||
# first `up` with ownership matching the container user (root inside
|
||||
# this image — no UID juggling needed, but the host dir needs to be
|
||||
# writable by the container).
|
||||
# Host path for the ONNX model files — encoder/decoder/joiner/tokens.txt.
|
||||
# Downloaded by the entrypoint on first run if absent. Must exist before
|
||||
# first `up` (directory, not files).
|
||||
PARAKEET_MODELS_DIR=/worktank/parakeet/models
|
||||
|
||||
# Inference precision. fp16 halves VRAM and is lossless for parakeet
|
||||
# in practice; use fp32 only if fp16 shows degraded WER for your
|
||||
# domain audio.
|
||||
PARAKEET_MODEL_PRECISION=fp16
|
||||
# Which sherpa-onnx release tarball to fetch on first boot. Default is the
|
||||
# int8-quantized English-only v2 (~400 MB). Switch to the v3 tarball below
|
||||
# to cover 25 European languages at a similar size:
|
||||
# https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8.tar.bz2
|
||||
PARAKEET_MODEL_URL=https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-nemo-parakeet-tdt-0.6b-v2-int8.tar.bz2
|
||||
|
||||
# Batch size for the transcribe queue. Larger = better throughput
|
||||
# under load at the cost of per-request latency.
|
||||
PARAKEET_BATCH_SIZE=4
|
||||
# ONNX Runtime execution provider. `cuda` uses the GPU (requires nvidia
|
||||
# runtime + matching CUDA/cuDNN in the image). `cpu` falls back to CPU —
|
||||
# fine for low-volume dev use; ~4-8× slower on this host.
|
||||
PARAKEET_PROVIDER=cuda
|
||||
|
||||
# Max single-clip duration (seconds). Longer inputs get rejected
|
||||
# by the server with 400. Upstream default.
|
||||
PARAKEET_MAX_AUDIO_DURATION=30
|
||||
|
||||
# Silero VAD threshold (0–1). Higher = stricter about what counts
|
||||
# as speech (fewer false wake-ups on silence, more chance of clipping
|
||||
# soft speech). 0.5 is upstream default.
|
||||
PARAKEET_VAD_THRESHOLD=0.5
|
||||
|
||||
# End-to-end processing timeout per request (seconds).
|
||||
PARAKEET_PROCESSING_TIMEOUT=60
|
||||
# CPU threads per recognizer session. Irrelevant when provider=cuda;
|
||||
# only matters for provider=cpu.
|
||||
PARAKEET_NUM_THREADS=1
|
||||
|
||||
# Log level: DEBUG | INFO | WARNING | ERROR
|
||||
PARAKEET_LOG_LEVEL=INFO
|
||||
|
||||
Reference in New Issue
Block a user