vh
01c5380059
The Shadowfita FastAPI wrapper hit two unfixed upstream bugs on the first real /transcribe call — chunker return-shape mismatch (open issue #16) and a `torchaudio.tensor` that doesn't exist (open #10). Rather than babysit someone else's half-tested code, switched to sherpa-onnx with the prebuilt int8 Parakeet-TDT tarball from k2-fsa, and wrote our own ~60-line FastAPI wrapper. Moving parts now owned in-tree: Dockerfile CUDA 12.8 + cuDNN 9 runtime base, installs sherpa-onnx==1.12.39+cuda12.cudnn9 + fastapi + soundfile + libasound2 (sherpa-onnx links to ALSA at load time even when we never touch a mic). app.py OfflineRecognizer.from_transducer() once at startup; /transcribe and /v1/audio/transcriptions both accept multipart uploads and return {"text": ...}. entrypoint.sh Idempotent model download to /models on first run (~400 MB int8 tarball), then exec uvicorn. Smoke test: 0.wav (bundled in the tarball, The House of the Seven Gables excerpt) transcribes cleanly in ~1.2s on GPU. PARAKEET_MODEL_URL in .env lets you swap to the v3 (25-language) tarball without touching any other files. Wipe *.onnx + tokens.txt from the models dir and the entrypoint re-downloads.
41 lines
1.6 KiB
Bash
41 lines
1.6 KiB
Bash
# Parakeet ASR stack tunables. Copy to `.env` on irv-ml1 before deploying.
|
||
#
|
||
# cp .env.example .env
|
||
# # edit as needed
|
||
# docker compose build
|
||
# docker compose up -d
|
||
|
||
# Image tag. Bump when you change the Dockerfile / app.py so docker caches
|
||
# cleanly.
|
||
PARAKEET_TAG=sherpa-onnx-v2
|
||
|
||
# Host port for the FastAPI server (container listens on 8000)
|
||
PARAKEET_PORT=8765
|
||
|
||
# Bind address. 0.0.0.0 exposes on all interfaces including the WG tunnel IP
|
||
# (10.100.79.3). Use 127.0.0.1 to restrict to local-only.
|
||
PARAKEET_BIND=0.0.0.0
|
||
|
||
# Host path for the ONNX model files — encoder/decoder/joiner/tokens.txt.
|
||
# Downloaded by the entrypoint on first run if absent. Must exist before
|
||
# first `up` (directory, not files).
|
||
PARAKEET_MODELS_DIR=/worktank/parakeet/models
|
||
|
||
# Which sherpa-onnx release tarball to fetch on first boot. Default is the
|
||
# int8-quantized English-only v2 (~400 MB). Switch to the v3 tarball below
|
||
# to cover 25 European languages at a similar size:
|
||
# https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8.tar.bz2
|
||
PARAKEET_MODEL_URL=https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-nemo-parakeet-tdt-0.6b-v2-int8.tar.bz2
|
||
|
||
# ONNX Runtime execution provider. `cuda` uses the GPU (requires nvidia
|
||
# runtime + matching CUDA/cuDNN in the image). `cpu` falls back to CPU —
|
||
# fine for low-volume dev use; ~4-8× slower on this host.
|
||
PARAKEET_PROVIDER=cuda
|
||
|
||
# CPU threads per recognizer session. Irrelevant when provider=cuda;
|
||
# only matters for provider=cpu.
|
||
PARAKEET_NUM_THREADS=1
|
||
|
||
# Log level: DEBUG | INFO | WARNING | ERROR
|
||
PARAKEET_LOG_LEVEL=INFO
|