parakeet + cosyvoice: add stacks + deploy to irv-ml1
Two new speech stacks on irv-ml1, both on the /worktank/<stack>/
pattern, no tnet (irv-ml1 is local-endpoints-only for now).
parakeet — ASR via Shadowfita/parakeet-tdt-0.6b-v2-fastapi:
- docker buildx git context pinned to SHA 31c5652; no source
vendored. Rebuild on SHA bump.
- GPU-capable FastAPI + Silero VAD + WS streaming.
- API: POST /transcribe, WS /ws/transcribe, GET /healthz. Not the
literal OpenAI `/v1/audio/transcriptions` path — note in README.
- HF cache at /worktank/parakeet/models/ (excluded from restic).
- Build ~158s first time; steady-state start ~40s.
cosyvoice — TTS via neosun/cosyvoice:v1.3.2 shipping
Fun-CosyVoice3-0.5B-2512 (CosyVoice 3, chosen over v2 for the
expanded 5,000-hour instruction-following data covering emotions,
speed, tones, dialects, accents, role-playing; ~150ms streaming
TTFB matches v2). API: /v1/audio/speech (OpenAI drop-in),
/v1/voices/create (cloning), /health.
- Host port 8190 (container 8188; host 8188 already taken by comfyui).
- /worktank/cosyvoice/{voices,input,output}/; voices include in
restic (precious — reproducing a clone needs the original ref
audio), input+output excluded (scratch).
- Model weights (~2-3 GB) live inside image layer; re-download on
tag bump, persist across `compose up -d`.
Both healthy on first deploy.
This commit is contained in:
@@ -0,0 +1,67 @@
|
||||
# Parakeet ASR — NVIDIA Parakeet-TDT 0.6B v2 speech-to-text.
|
||||
#
|
||||
# Wraps Shadowfita/parakeet-tdt-0.6b-v2-fastapi (FastAPI + Silero VAD +
|
||||
# WebSocket streaming). Upstream provides no prebuilt image, so we
|
||||
# build from a pinned git commit via docker buildx's git URL context
|
||||
# — no source files vendored into this repo.
|
||||
#
|
||||
# Runs on irv-ml1 (dual GPU). Both GPUs exposed; upstream respects
|
||||
# CUDA_VISIBLE_DEVICES if you want to pin later.
|
||||
#
|
||||
# HF weights (~2.5 GB for parakeet-tdt-0.6b-v2) cache to
|
||||
# ${PARAKEET_MODELS_DIR} via HF_HOME=/models, persistent across
|
||||
# container recreates.
|
||||
#
|
||||
# API routes (per upstream README):
|
||||
# POST /transcribe — batch transcription
|
||||
# WS /ws/transcribe — streaming with Silero VAD
|
||||
# GET /healthz
|
||||
#
|
||||
# Not a literal OpenAI `/v1/audio/transcriptions` path; point clients
|
||||
# at /transcribe directly, or add a reverse-proxy alias if drop-in
|
||||
# compat is needed later.
|
||||
#
|
||||
# First `up` triggers a fresh docker build (python:3.10-slim +
|
||||
# torch + NeMo ≈ 5–10 min). Subsequent starts reuse the cached
|
||||
# image unless PARAKEET_SHA changes.
|
||||
#
|
||||
# All tunables live in .env — edit that, not this file.
|
||||
|
||||
services:
|
||||
parakeet:
|
||||
image: local/parakeet:${PARAKEET_SHA}
|
||||
build:
|
||||
context: https://github.com/Shadowfita/parakeet-tdt-0.6b-v2-fastapi.git#${PARAKEET_SHA}
|
||||
container_name: parakeet
|
||||
restart: unless-stopped
|
||||
runtime: nvidia
|
||||
ports:
|
||||
- "${PARAKEET_BIND:-0.0.0.0}:${PARAKEET_PORT}:8000"
|
||||
environment:
|
||||
- NVIDIA_VISIBLE_DEVICES=all
|
||||
- HF_HOME=/models
|
||||
- DEVICE=cuda
|
||||
- MODEL_PRECISION=${PARAKEET_MODEL_PRECISION:-fp16}
|
||||
- BATCH_SIZE=${PARAKEET_BATCH_SIZE:-4}
|
||||
- TARGET_SAMPLE_RATE=16000
|
||||
- MAX_AUDIO_DURATION=${PARAKEET_MAX_AUDIO_DURATION:-30}
|
||||
- VAD_THRESHOLD=${PARAKEET_VAD_THRESHOLD:-0.5}
|
||||
- PROCESSING_TIMEOUT=${PARAKEET_PROCESSING_TIMEOUT:-60}
|
||||
- LOG_LEVEL=${PARAKEET_LOG_LEVEL:-INFO}
|
||||
volumes:
|
||||
- ${PARAKEET_MODELS_DIR}:/models
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "curl -fsS http://localhost:8000/healthz >/dev/null || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
# First `up` may spend several minutes on torch/NeMo install
|
||||
# during the image build phase; after the image exists, startup
|
||||
# is ~30-60s (NeMo model load).
|
||||
start_period: 180s
|
||||
labels:
|
||||
- homepage.group=AI Systems
|
||||
- homepage.name=Parakeet ASR
|
||||
- homepage.icon=mdi-microphone
|
||||
- homepage.description=Parakeet-TDT speech-to-text (irv-ml1)
|
||||
- homepage.href=http://10.100.79.3:${PARAKEET_PORT}
|
||||
Reference in New Issue
Block a user