# Kyutai TTS

[kyutai/tts-1.6b-en_fr](https://huggingface.co/kyutai/tts-1.6b-en_fr)
— Kyutai's flagship streaming TTS (1.6B params, EN/FR bilingual,
trained on 2.5M hours), served via the
[NillPointer/Kyutai-TTS-Server](https://github.com/NillPointer/Kyutai-TTS-Server)
community OpenAI-compatible wrapper.

## Why this stack exists

Kyutai's claim is the **lowest streaming latency in this size class**:
220 ms in solo setup; up to 32 simultaneous streams under 350 ms on
a single L40-class GPU. Worth bench-comparing against:

| | claimed latency | use case |
|---|---|---|
| **Kyutai TTS** | **~220 ms** | streaming EN/FR, conversational dialogue heritage |
| Kokoro | ~26 ms TTFB measured | low-latency English, fixed voice library |
| Chatterbox Turbo | ~1.2 s TTFB measured | English w/ cloning + 9 paralinguistic tags |
| Fish Audio S2-Pro | ~150 ms claimed | richest paralinguistic English |
| Voxtral | ~70 ms model latency | multilingual EN/FR/DE/ES/IT/PT/NL/HI |

## Deployment notes

Kyutai's official deployment path is **Rust + websockets only** (no
HTTP, no OpenAI-compat). That doesn't fit the OpenAI-`/v1/audio/speech`
contract the rest of our TTS fleet uses. The
NillPointer/Kyutai-TTS-Server community wrapper bridges Kyutai's
native streaming to the OpenAI HTTP shape, which lets us slot it
into the same bench harness as the others.

**Tradeoff**: the wrapper adds Python overhead on the request path,
so measured latency on this stack will be *higher* than Kyutai's
220 ms claim (which is for the bare Rust server). If we measure
~400-500 ms TTFB end-to-end, the wrapper is the floor — Kyutai itself
is hitting its target.

## Architecture heritage

Kyutai's TTS shares the **Mimi** neural codec + **Moshi** dialogue
modeling framework. Both target full-duplex conversational AI (Moshi
is their flagship speech-text foundation model). The TTS-only model
is the "synthesis half" of the stack, distilled for low-latency
streaming.

Trained on **2.5M hours** — a different scaling regime from the
others (CosyVoice 5k, Fish 10M, Voxtral undisclosed).

## API

OpenAI-compat at `http://10.100.79.3:8198`:

```bash
# Single-shot synthesis.
curl -fsS -X POST http://10.100.79.3:8198/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{"model":"tts-1.6b-en_fr","input":"Hello there.","voice":"default","response_format":"wav"}' \
  > out.wav

# Streaming.
curl -fsS -X POST http://10.100.79.3:8198/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{"model":"tts-1.6b-en_fr","input":"long passage…","voice":"default","stream":true}' \
  | mpv --no-cache -

# Built-in voices.
curl http://10.100.79.3:8198/v1/audio/voices
```

## Deploy

```bash
scripts/elway irv-ml1 --playbook playbooks/deploy-kyutai-tts.yaml
```

First boot pulls the kyutai/tts-1.6b-en_fr checkpoint (~3-6 GB) into
the HF cache.

## Hardware footprint

- **VRAM**: ~4-6 GB practical. Pinned to GPU 0 (RTX 3090) by default
  — the A6000 is hosting the heavier Fish S2-Pro / Voxtral.
- **Disk**: ~6 GB for the checkpoint + HF cache.