# Sesame CSM

Sesame's **Conversational Speech Model** (the engine behind the
"Maya/Miles" demo) — a context-aware speech generator (Llama backbone +
Mimi audio codec) — served via
[phildougherty/sesame_csm_openai](https://github.com/phildougherty/sesame_csm_openai),
an OpenAI-compat wrapper.

**Server:** irv-ml1 (Irvine, WireGuard-only)
**Port:** 8201 (container listens on 8000)
**GPUs:** pins to device 0 (RTX 3090) by default; ~8 GB VRAM
**Image:** `local/csm:v1` — built locally from a pinned git SHA of the
wrapper via docker buildx's git URL context
**Upstream wrapper:** [phildougherty/sesame_csm_openai](https://github.com/phildougherty/sesame_csm_openai) (MIT)
**Upstream model:** [sesame/csm-1b](https://huggingface.co/sesame/csm-1b)
(**gated** — Sesame's own license) + `unsloth/Llama-3.2-1B` tokenizer + Mimi codec

## Why this stack exists — and what it is *not*

CSM is a **conversational** speech model, not a plain reader. It
conditions on prior conversation turns (text **and** audio) to choose
prosody and tone — it's designed to be the **speech layer of a voice
agent**, where a separate LLM produces the words and CSM voices them
with context-aware delivery.

- As **pure TTS** it works standalone (OpenAI-compat
  `POST /v1/audio/speech`, voices `alloy/echo/fable/onyx/nova/shimmer`
  plus cloned IDs).
- Its real edge — contextual prosody across turns — only pays off in
  **interactive / voice-agent** use, not monologue narration. For
  skaldsong's *reader*, the emotive single-voice engines (Fish S2-Pro,
  IndexTTS-2) and Dia (dialogue) remain the better fits; CSM is here for
  voice-agent experiments.

## ⚠️ Gated model — token required before first boot

`sesame/csm-1b` is gated. Before deploying:

1. Accept the license at <https://huggingface.co/sesame/csm-1b>.
2. Put a HF token (read scope) in `CSM_HF_TOKEN` in the host `.env`
   (never in `.env.example`).

Without it the first-boot model download 401s.

## Deploy

```bash
# from this workstation (irv-ml1 is WG-only — routes via ana-wg):
scripts/deploy-stack.sh irv-ml1 csm
# then on irv-ml1, first run builds from the pinned SHA:
#   docker compose up -d --build
```

First boot pulls csm-1b + the Llama-3.2-1B tokenizer + Mimi into
`CSM_CACHE_DIR`; the 600 s `start_period` covers it.

## Notes

- **Pin `CSM_SHA`** to a full 40-char commit before relying on this —
  `.env.example` ships `main`, which is not reproducible.
- The wrapper can also serve Dia-1.6B, but we run Dia from its own
  [`dia`](../dia/) stack — keep this one CSM-only to avoid overlap.
- Endpoints: `/v1/audio/speech` (OpenAI-compat), `/v1/audio/voices`,
  `/v1/audio/models`. No dedicated `/health` route — the healthcheck
  probes `/v1/audio/voices` (only answers once the model is loaded).