vh
01c5380059
The Shadowfita FastAPI wrapper hit two unfixed upstream bugs on the first real /transcribe call — chunker return-shape mismatch (open issue #16) and a `torchaudio.tensor` that doesn't exist (open #10). Rather than babysit someone else's half-tested code, switched to sherpa-onnx with the prebuilt int8 Parakeet-TDT tarball from k2-fsa, and wrote our own ~60-line FastAPI wrapper. Moving parts now owned in-tree: Dockerfile CUDA 12.8 + cuDNN 9 runtime base, installs sherpa-onnx==1.12.39+cuda12.cudnn9 + fastapi + soundfile + libasound2 (sherpa-onnx links to ALSA at load time even when we never touch a mic). app.py OfflineRecognizer.from_transducer() once at startup; /transcribe and /v1/audio/transcriptions both accept multipart uploads and return {"text": ...}. entrypoint.sh Idempotent model download to /models on first run (~400 MB int8 tarball), then exec uvicorn. Smoke test: 0.wav (bundled in the tarball, The House of the Seven Gables excerpt) transcribes cleanly in ~1.2s on GPU. PARAKEET_MODEL_URL in .env lets you swap to the v3 (25-language) tarball without touching any other files. Wipe *.onnx + tokens.txt from the models dir and the entrypoint re-downloads.
33 lines
1.2 KiB
Bash
33 lines
1.2 KiB
Bash
#!/bin/bash
|
|
# entrypoint: download int8 Parakeet weights on first run if the target dir is
|
|
# empty, then start the FastAPI app. Idempotent — subsequent starts skip the
|
|
# download when the files already exist.
|
|
|
|
set -euo pipefail
|
|
|
|
MODEL_DIR=${MODEL_DIR:-/models}
|
|
MODEL_URL=${MODEL_URL:-https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-nemo-parakeet-tdt-0.6b-v2-int8.tar.bz2}
|
|
|
|
required=(encoder.int8.onnx decoder.int8.onnx joiner.int8.onnx tokens.txt)
|
|
missing=0
|
|
for f in "${required[@]}"; do
|
|
[[ -f "$MODEL_DIR/$f" ]] || missing=1
|
|
done
|
|
|
|
if [[ "$missing" -eq 1 ]]; then
|
|
echo "[entrypoint] model files not present in $MODEL_DIR, downloading from $MODEL_URL"
|
|
mkdir -p "$MODEL_DIR"
|
|
tmp=$(mktemp -d)
|
|
wget -q --show-progress -O "$tmp/model.tar.bz2" "$MODEL_URL"
|
|
tar -xjf "$tmp/model.tar.bz2" -C "$tmp"
|
|
# Upstream tarballs extract to a single top-level dir; move its contents up.
|
|
extracted_root=$(find "$tmp" -mindepth 1 -maxdepth 1 -type d | head -n1)
|
|
shopt -s dotglob
|
|
mv "$extracted_root"/* "$MODEL_DIR"/
|
|
rm -rf "$tmp"
|
|
echo "[entrypoint] model extracted:"
|
|
ls -la "$MODEL_DIR"
|
|
fi
|
|
|
|
exec uvicorn app:app --host 0.0.0.0 --port 8000
|