diff --git a/stacks/csm/.env.example b/stacks/csm/.env.example new file mode 100644 index 0000000..5c17db0 --- /dev/null +++ b/stacks/csm/.env.example @@ -0,0 +1,49 @@ +# Sesame CSM stack tunables. Copy to `.env` on irv-ml1 before deploying. + +# ── build pin ──────────────────────────────────────────────────────── +# SHA of phildougherty/sesame_csm_openai. Use the FULL 40-char SHA; +# `main` works but is NOT reproducible — pin before relying on it. +# https://github.com/phildougherty/sesame_csm_openai/commits/main +CSM_SHA=main + +# Local image tag — bump to force a fresh layer build. +CSM_TAG=v1 + +# ── network ────────────────────────────────────────────────────────── +# Host port. Container listens on 8000 internally. +# Reserved on irv-ml1: 8188 ComfyUI, 8190 CosyVoice, 8191 Qwen3-TTS, +# 8192 IndexTTS-2, 8193 Kokoro, 8194 VibeVoice, 8195 Fish-S2, +# 8196 Chatterbox, 8197 Voxtral, 8198 Kyutai, 8199 Zonos, 8200 Dia, +# 8765 Parakeet. 8201 picked here. +CSM_PORT=8201 + +# Bind address. 0.0.0.0 exposes on all interfaces (incl. the WG tunnel +# interface 10.100.79.3); 127.0.0.1 restricts to local-only. +CSM_BIND=0.0.0.0 + +# ── runtime / GPU ──────────────────────────────────────────────────── +# Devices visible inside the container. "0" pins to the RTX 3090 +# (24 GB) — CSM-1B (~8 GB) fits; use "1" for the A6000. +CSM_GPU_DEVICES=0 + +# Optional HF device_map for multi-GPU: auto | balanced | sequential. +# Leave empty for single-GPU (the common case here). +CSM_DEVICE_MAP= + +# ── secrets ────────────────────────────────────────────────────────── +# REQUIRED. sesame/csm-1b is a GATED model: accept its license at +# https://huggingface.co/sesame/csm-1b +# then paste a HF token (read scope) here. Without it, first-boot model +# download 401s. This file is .env.example (committed) — put the REAL +# token only in the .env on the host, which is gitignored. +CSM_HF_TOKEN= + +# ── persistent storage on the host ─────────────────────────────────── +# Voices dir — cloned/custom voice samples the wrapper serves by ID. +# Included in restic (clones need the original sample to recreate). +CSM_VOICES_DIR=/worktank/csm/voices + +# HuggingFace cache — csm-1b + Llama-3.2-1B tokenizer + Mimi land here +# on first start. Bind-mounted to survive recreate. Excluded from restic +# (regenerable from HF, given the token + license acceptance). +CSM_CACHE_DIR=/worktank/csm/cache diff --git a/stacks/csm/README.md b/stacks/csm/README.md new file mode 100644 index 0000000..ffbb689 --- /dev/null +++ b/stacks/csm/README.md @@ -0,0 +1,65 @@ +# Sesame CSM + +Sesame's **Conversational Speech Model** (the engine behind the +"Maya/Miles" demo) — a context-aware speech generator (Llama backbone + +Mimi audio codec) — served via +[phildougherty/sesame_csm_openai](https://github.com/phildougherty/sesame_csm_openai), +an OpenAI-compat wrapper. + +**Server:** irv-ml1 (Irvine, WireGuard-only) +**Port:** 8201 (container listens on 8000) +**GPUs:** pins to device 0 (RTX 3090) by default; ~8 GB VRAM +**Image:** `local/csm:v1` — built locally from a pinned git SHA of the +wrapper via docker buildx's git URL context +**Upstream wrapper:** [phildougherty/sesame_csm_openai](https://github.com/phildougherty/sesame_csm_openai) (MIT) +**Upstream model:** [sesame/csm-1b](https://huggingface.co/sesame/csm-1b) +(**gated** — Sesame's own license) + `unsloth/Llama-3.2-1B` tokenizer + Mimi codec + +## Why this stack exists — and what it is *not* + +CSM is a **conversational** speech model, not a plain reader. It +conditions on prior conversation turns (text **and** audio) to choose +prosody and tone — it's designed to be the **speech layer of a voice +agent**, where a separate LLM produces the words and CSM voices them +with context-aware delivery. + +- As **pure TTS** it works standalone (OpenAI-compat + `POST /v1/audio/speech`, voices `alloy/echo/fable/onyx/nova/shimmer` + plus cloned IDs). +- Its real edge — contextual prosody across turns — only pays off in + **interactive / voice-agent** use, not monologue narration. For + skaldsong's *reader*, the emotive single-voice engines (Fish S2-Pro, + IndexTTS-2) and Dia (dialogue) remain the better fits; CSM is here for + voice-agent experiments. + +## ⚠️ Gated model — token required before first boot + +`sesame/csm-1b` is gated. Before deploying: + +1. Accept the license at . +2. Put a HF token (read scope) in `CSM_HF_TOKEN` in the host `.env` + (never in `.env.example`). + +Without it the first-boot model download 401s. + +## Deploy + +```bash +# from this workstation (irv-ml1 is WG-only — routes via ana-wg): +scripts/deploy-stack.sh irv-ml1 csm +# then on irv-ml1, first run builds from the pinned SHA: +# docker compose up -d --build +``` + +First boot pulls csm-1b + the Llama-3.2-1B tokenizer + Mimi into +`CSM_CACHE_DIR`; the 600 s `start_period` covers it. + +## Notes + +- **Pin `CSM_SHA`** to a full 40-char commit before relying on this — + `.env.example` ships `main`, which is not reproducible. +- The wrapper can also serve Dia-1.6B, but we run Dia from its own + [`dia`](../dia/) stack — keep this one CSM-only to avoid overlap. +- Endpoints: `/v1/audio/speech` (OpenAI-compat), `/v1/audio/voices`, + `/v1/audio/models`. No dedicated `/health` route — the healthcheck + probes `/v1/audio/voices` (only answers once the model is loaded). diff --git a/stacks/csm/compose.yaml b/stacks/csm/compose.yaml new file mode 100644 index 0000000..d65ec5b --- /dev/null +++ b/stacks/csm/compose.yaml @@ -0,0 +1,62 @@ +# Sesame CSM-1B (Conversational Speech Model) served via +# phildougherty/sesame_csm_openai — an OpenAI-compat wrapper around +# Sesame's context-aware speech model (Llama backbone + Mimi codec). +# +# Why this stack exists alongside the other TTS: +# * CSM is a CONVERSATIONAL speech layer, not a plain reader — it +# conditions on prior turns (text + audio) to pick prosody, built +# for voice-AGENT turn-taking. As pure TTS it works standalone; +# its differentiator only pays off in interactive/agent use. +# * OpenAI-compat /v1/audio/speech with 6 standard voices +# (alloy/echo/fable/onyx/nova/shimmer) + cloned voice IDs. +# * ~8 GB VRAM; fits the 3090 (device 0) or A6000 (device 1). +# +# NOTE: sesame/csm-1b is a GATED model — you must accept its license at +# https://huggingface.co/sesame/csm-1b and supply CSM_HF_TOKEN in .env +# before first boot, or the model download 401s. +# +# Image built locally from the upstream wrapper via buildx git-context. +# All tunables live in .env — edit that, not this file. + +services: + csm: + image: local/csm:${CSM_TAG} + build: + context: https://github.com/phildougherty/sesame_csm_openai.git#${CSM_SHA} + dockerfile: Dockerfile + container_name: csm + restart: unless-stopped + runtime: nvidia + ports: + - "${CSM_BIND:-0.0.0.0}:${CSM_PORT}:8000" + environment: + - NVIDIA_VISIBLE_DEVICES=${CSM_GPU_DEVICES:-0} + - NVIDIA_DRIVER_CAPABILITIES=compute,utility + # Required: gated-model access token (accept the csm-1b license on + # HF first). Set the real value in .env, NEVER here. + - HF_TOKEN=${CSM_HF_TOKEN} + - HF_HUB_ENABLE_HF_TRANSFER=1 + - HF_HOME=/app/hf_cache + # Optional multi-GPU split: auto | balanced | sequential. + - CSM_DEVICE_MAP=${CSM_DEVICE_MAP:-} + volumes: + - ${CSM_VOICES_DIR}:/app/voices + - ${CSM_CACHE_DIR}:/app/hf_cache + healthcheck: + # The wrapper exposes no /health route; GET /v1/audio/voices + # returns the voice list only once the model is loaded, so it + # doubles as liveness + ready. python urllib (no curl in image), + # bound to 127.0.0.1 (uvicorn is IPv4-only). + test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; urllib.request.urlopen('http://127.0.0.1:8000/v1/audio/voices', timeout=5); sys.exit(0)\""] + interval: 30s + timeout: 10s + retries: 3 + # First boot pulls csm-1b + the Llama-3.2-1B tokenizer + Mimi — + # gated download, generous deadline. + start_period: 600s + labels: + - homepage.group=AI Systems + - homepage.name=Sesame CSM + - homepage.icon=mdi-account-voice-outline + - homepage.description=Conversational speech model — context-aware voice-agent TTS (irv-ml1) + - homepage.href=http://10.100.79.3:${CSM_PORT}