# Parakeet ASR

NVIDIA Parakeet-TDT 0.6B (int8 ONNX) served by our own thin FastAPI
wrapper over [sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx)
(ONNX Runtime + CUDA).

**Server:** irv-ml1 (Irvine, WireGuard-only)
**Port:** 8765 (container 8000)
**GPU:** both exposed (`NVIDIA_VISIBLE_DEVICES=all`); sherpa-onnx uses
whichever CUDA ExecutionProvider picks
**Image:** `local/parakeet:sherpa-onnx-v1` — built from `Dockerfile` +
`app.py` + `entrypoint.sh` in this directory; **we own all the code**

## Why not the FastAPI community wrappers

Both `Shadowfita/parakeet-tdt-0.6b-v2-fastapi` and
`pnivek/Parakeet-ASR-FastAPI` look appealing on paper but have open,
unfixed bugs in the actual transcribe path (return-shape mismatches
after an unpinned `torchaudio` upgrade, `torchaudio.tensor` which
doesn't exist, etc.). We tried Shadowfita and hit #16+#10 on the
first real request. Rather than babysit someone else's half-tested
code, we moved to sherpa-onnx — ONNX Runtime is a stable base, k2-fsa
publishes prebuilt int8 Parakeet weights per release, and the
recognizer API is a three-line call.

## API endpoints

| Method + path | Purpose |
|---|---|
| `POST /transcribe` | Multipart file upload → `{"text": "..."}` |
| `POST /v1/audio/transcriptions` | Same body; OpenAI-compatible path |
| `GET /healthz` | Health probe (used by docker healthcheck) |

## Path layout

| Host path | Container path | Purpose | Restic? |
|---|---|---|---|
| `/worktank/parakeet/models/` | `/models` | ONNX encoder+decoder+joiner+tokens (~400 MB int8) | excluded (regenerable — re-downloads from the URL on first run if absent) |

## First-time deploy on irv-ml1

```bash
# 1. Push compose + Dockerfile + app + entrypoint
scripts/deploy-stack.sh irv-ml1 parakeet

# 2. Make sure the models dir exists (one-time, already done from the
#    earlier Shadowfita deploy; this is idempotent)
ssh -t irv-ml1 'sudo mkdir -p /worktank/parakeet/models && \
                sudo chown -R lkraven:lkraven /worktank/parakeet'

# 3. Build the image and bring up. First boot does a ~400 MB model
#    download via the entrypoint; allow 1–2 minutes before /healthz
#    flips healthy.
ssh irv-ml1 '
  cd /opt/docker/compose/parakeet && \
  cp -n .env.example .env && \
  docker compose config >/dev/null && \
  docker compose build && \
  docker compose up -d && \
  docker compose logs -f --tail=30
'
```

## Smoke test

```bash
# Over WG from the workstation
curl -F "file=@sample.wav" http://10.100.79.3:8765/transcribe
# → {"text": "hello world"}

# OpenAI-shape alias (for clients that only know /v1/audio/transcriptions)
curl -F "file=@sample.wav" http://10.100.79.3:8765/v1/audio/transcriptions
```

## Switching to the v3 (multilingual) model

The env var `PARAKEET_MODEL_URL` picks the release tarball. To swap
from the English-only v2 to the 25-language v3:

```bash
ssh irv-ml1 '
  cd /opt/docker/compose/parakeet && \
  sed -i "s|v2-int8|v3-int8|" .env && \
  # Wipe the v2 weights so the entrypoint re-downloads v3 on next up:
  rm -f /worktank/parakeet/models/*.onnx /worktank/parakeet/models/tokens.txt && \
  docker compose up -d && \
  docker compose logs -f --tail=30
'
```

## Upgrade sherpa-onnx or change the base image

Bump `PARAKEET_TAG` in `.env` to force a rebuild of the local image
after editing the `Dockerfile`, then:

```bash
scripts/deploy-stack.sh irv-ml1 parakeet
ssh irv-ml1 'cd /opt/docker/compose/parakeet && docker compose build && docker compose up -d'
```

Model files under `/worktank/parakeet/models/` are preserved across
image rebuilds.

## File layout

```
stacks/parakeet/
├── Dockerfile           # CUDA 12.8 + cuDNN 9 base, sherpa-onnx-cu12 wheel
├── app.py               # FastAPI — ~60 lines
├── entrypoint.sh        # downloads model on first run, then uvicorn
├── compose.yaml         # one service, bind-mounts the models dir
├── .env.example         # template; real .env lives on the server
└── README.md            # this file
```