parakeet + cosyvoice: add stacks + deploy to irv-ml1
Two new speech stacks on irv-ml1, both on the /worktank/<stack>/
pattern, no tnet (irv-ml1 is local-endpoints-only for now).
parakeet — ASR via Shadowfita/parakeet-tdt-0.6b-v2-fastapi:
- docker buildx git context pinned to SHA 31c5652; no source
vendored. Rebuild on SHA bump.
- GPU-capable FastAPI + Silero VAD + WS streaming.
- API: POST /transcribe, WS /ws/transcribe, GET /healthz. Not the
literal OpenAI `/v1/audio/transcriptions` path — note in README.
- HF cache at /worktank/parakeet/models/ (excluded from restic).
- Build ~158s first time; steady-state start ~40s.
cosyvoice — TTS via neosun/cosyvoice:v1.3.2 shipping
Fun-CosyVoice3-0.5B-2512 (CosyVoice 3, chosen over v2 for the
expanded 5,000-hour instruction-following data covering emotions,
speed, tones, dialects, accents, role-playing; ~150ms streaming
TTFB matches v2). API: /v1/audio/speech (OpenAI drop-in),
/v1/voices/create (cloning), /health.
- Host port 8190 (container 8188; host 8188 already taken by comfyui).
- /worktank/cosyvoice/{voices,input,output}/; voices include in
restic (precious — reproducing a clone needs the original ref
audio), input+output excluded (scratch).
- Model weights (~2-3 GB) live inside image layer; re-download on
tag bump, persist across `compose up -d`.
Both healthy on first deploy.
This commit is contained in:
@@ -0,0 +1,88 @@
|
||||
# Parakeet ASR
|
||||
|
||||
NVIDIA Parakeet-TDT 0.6B v2 speech-to-text served via a FastAPI
|
||||
wrapper with Silero VAD and WebSocket streaming.
|
||||
|
||||
**Server:** irv-ml1 (Irvine, WireGuard-only)
|
||||
**Port:** 8765 (configurable via `.env`)
|
||||
**GPUs:** both exposed (`NVIDIA_VISIBLE_DEVICES=all`); upstream
|
||||
respects `CUDA_VISIBLE_DEVICES` if later pinning is needed
|
||||
**Upstream:** [Shadowfita/parakeet-tdt-0.6b-v2-fastapi](https://github.com/Shadowfita/parakeet-tdt-0.6b-v2-fastapi)
|
||||
**Image:** built locally from a pinned git SHA via docker buildx's
|
||||
git URL context — no source vendored into this workspace
|
||||
|
||||
## API endpoints
|
||||
|
||||
| Method + path | Purpose |
|
||||
|---|---|
|
||||
| `POST /transcribe` | Batch transcription (multipart file upload) |
|
||||
| `WS /ws/transcribe` | Streaming with Silero VAD — partial + final segments |
|
||||
| `GET /healthz` | Health probe (used by docker healthcheck) |
|
||||
|
||||
Not the literal OpenAI `/v1/audio/transcriptions` path. If you have
|
||||
a downstream client that demands that URL shape, either point its
|
||||
base URL at `/transcribe`, or add a Traefik/nginx path alias in
|
||||
front.
|
||||
|
||||
## Path layout
|
||||
|
||||
| Host path | Container path | Purpose | Restic? |
|
||||
|---|---|---|---|
|
||||
| `/worktank/parakeet/models/` | `/models` (`HF_HOME`) | HF cache for parakeet-tdt-0.6b-v2 weights (~2.5 GB) | excluded (regenerable from HF) |
|
||||
|
||||
The container runs as root internally; host dir just needs to exist
|
||||
and be writable.
|
||||
|
||||
## First-time deploy on irv-ml1
|
||||
|
||||
```bash
|
||||
# 1. Push compose + env template
|
||||
scripts/deploy-stack.sh irv-ml1 parakeet
|
||||
|
||||
# 2. Create the models dir on the host. One-time sudo — /worktank
|
||||
# itself is root-owned.
|
||||
ssh -t irv-ml1 'sudo mkdir -p /worktank/parakeet/models && \
|
||||
sudo chown -R lkraven:lkraven /worktank/parakeet'
|
||||
|
||||
# 3. Build the image (first time only; ~5–10 min for torch + NeMo
|
||||
# wheels). Then `up`.
|
||||
ssh irv-ml1 '
|
||||
cd /opt/docker/compose/parakeet && \
|
||||
cp -n .env.example .env && \
|
||||
docker compose config >/dev/null && \
|
||||
docker compose build && \
|
||||
docker compose up -d && \
|
||||
docker compose logs -f --tail=30
|
||||
'
|
||||
```
|
||||
|
||||
First `/transcribe` request downloads parakeet-tdt-0.6b-v2 weights
|
||||
to `/worktank/parakeet/models/` (~2.5 GB).
|
||||
|
||||
## Smoke test
|
||||
|
||||
```bash
|
||||
# From the workstation over WG
|
||||
curl -F "file=@sample.wav" http://10.100.79.3:8765/transcribe
|
||||
```
|
||||
|
||||
## Rebuild against a newer upstream commit
|
||||
|
||||
```bash
|
||||
ssh irv-ml1 '
|
||||
cd /opt/docker/compose/parakeet && \
|
||||
sed -i "s/^PARAKEET_SHA=.*/PARAKEET_SHA=<new-sha>/" .env && \
|
||||
docker compose build && \
|
||||
docker compose up -d
|
||||
'
|
||||
```
|
||||
|
||||
Models dir is unaffected.
|
||||
|
||||
## Deploy updates
|
||||
|
||||
```bash
|
||||
# After editing compose.yaml or .env.example here
|
||||
scripts/deploy-stack.sh irv-ml1 parakeet
|
||||
ssh irv-ml1 'cd /opt/docker/compose/parakeet && docker compose up -d'
|
||||
```
|
||||
Reference in New Issue
Block a user