catalog: remove Sesame CSM stack — open csm-1b verified not deploy-worthy
Tear down the parked CSM stack (status: down, never successfully built). Bring-up attempts failed at the image build: upstream phildougherty/sesame_csm_openai pins no huggingface_hub version, which now resolves to 1.17.0 where the `huggingface-cli` the Dockerfile relies on has been removed (replaced by `hf`). Building would require vendoring + patching the upstream Dockerfile. Deep-research verdict (primary + community sourced) confirmed it isn't worth that: the acclaimed Maya/Miles demo runs a fine-tuned, larger CSM variant Sesame never open-sourced; the open csm-1b is the un-fine-tuned 1B base (only the smallest of 1B/3B/8B shipped, no newer checkpoint as of mid-2026). Ships no usable voices, can't generate text, English-only, can't stream real-time out of the box; absent from current TTS leaderboards and dominated by Kokoro/Dia2/Fish-S2/IndexTTS for narration. Removes: stacks/csm/, playbooks/deploy-csm.yaml, the csm catalog service + reproducibility_audit entries. Host state (compose dir, /worktank/csm) torn down on irv-ml1; no container/image existed.
This commit is contained in:
@@ -1566,97 +1566,6 @@ services:
|
||||
the single-model-per-endpoint rationale + both-start-together caveat).
|
||||
Image local/dia:v2. Verified end-to-end 2026-05-31.
|
||||
|
||||
- id: csm
|
||||
name: Sesame CSM (conversational)
|
||||
description: >
|
||||
Sesame's Conversational Speech Model (CSM-1B) — context-aware speech
|
||||
(Llama backbone + Mimi codec). Usable as plain TTS, but its edge is
|
||||
cross-turn prosody for voice agents, not narration.
|
||||
category: tts
|
||||
version: 1
|
||||
status: down
|
||||
host: irv-ml1
|
||||
lifecycle:
|
||||
stack: csm
|
||||
vram_gb: 8
|
||||
gpu_device_id: 0
|
||||
endpoint: http://10.100.79.3:8201/v1/audio/speech
|
||||
method: POST
|
||||
content_type: application/json
|
||||
model:
|
||||
id: sesame/csm-1b
|
||||
revision: null
|
||||
image: local/csm:v1
|
||||
fields:
|
||||
- name: model
|
||||
type: select
|
||||
options: [csm-1b]
|
||||
default: csm-1b
|
||||
- name: input
|
||||
type: textarea
|
||||
label: Text
|
||||
required: true
|
||||
max_length: 5000
|
||||
default: "Hello from Sesame." # canonical: sesame/csm-1b model card
|
||||
- name: voice
|
||||
type: select
|
||||
label: Voice
|
||||
default: alloy
|
||||
description: >
|
||||
alloy | echo | fable | onyx | nova | shimmer, or a cloned voice ID.
|
||||
- name: response_format
|
||||
type: select
|
||||
options: [mp3, opus, aac, flac, wav]
|
||||
default: mp3
|
||||
- name: speed
|
||||
type: slider
|
||||
min: 0.5
|
||||
max: 2.0
|
||||
step: 0.05
|
||||
default: 1.0
|
||||
- name: temperature
|
||||
type: slider
|
||||
min: 0.0
|
||||
max: 1.0
|
||||
step: 0.05
|
||||
default: 0.8
|
||||
description: Sampling temperature — higher = more variation.
|
||||
- name: topk
|
||||
type: number
|
||||
required: false
|
||||
description: Top-k sampling cutoff (1–100); no canonical default — wrapper's internal default applies if omitted.
|
||||
- name: max_audio_length_ms
|
||||
type: number
|
||||
required: false
|
||||
default: 90000
|
||||
description: Max generated audio length in milliseconds.
|
||||
response:
|
||||
type: audio
|
||||
mime_from_field: response_format
|
||||
reproducibility:
|
||||
seedable: false
|
||||
deterministic: false
|
||||
notes: >
|
||||
⚠️ No seed param and temperature-sampled → output varies run-to-run.
|
||||
Per this contract that is normally a fix-before-adding bug; catalogued
|
||||
by operator direction. asset-engine "regenerate/fork" will NOT
|
||||
reproduce a prior CSM take. Clean fix = upstream seed support.
|
||||
estimated_latency:
|
||||
cold_start_s: 12
|
||||
warm_per_unit: "~realtime on the 3090"
|
||||
license: "Sesame CSM license (gated, non-OSI); MIT (wrapper)"
|
||||
license_warning: |
|
||||
sesame/csm-1b is GATED under Sesame's own (non-OSI) license — accept
|
||||
terms on HF and review before any commercial/redistribution use. Host
|
||||
needs CSM_HF_TOKEN set before first start.
|
||||
notes: |
|
||||
Conversational speech model — context-aware prosody for voice agents,
|
||||
not a narration reader. Fields read from the phildougherty wrapper
|
||||
README/API (2026-05-31); confirm against live OpenAPI at deploy.
|
||||
Deployed to irv-ml1 but PARKED; gated-model token required before
|
||||
first start. Because it is non-deterministic, when started it belongs
|
||||
at status: experimental (not ready). Pin CSM_SHA before build.
|
||||
|
||||
- id: zonos
|
||||
name: Zonos-v0.1 TTS
|
||||
description: >
|
||||
@@ -1881,11 +1790,6 @@ reproducibility_audit:
|
||||
model_deterministic: false
|
||||
image_tag_mutable: false
|
||||
notes: "Dia2 1B streaming; same determinism story as dia2-2b. local/dia:v2."
|
||||
- service: csm
|
||||
seedable: false
|
||||
model_deterministic: false
|
||||
image_tag_mutable: false
|
||||
notes: "⚠️ No seed + temperature-sampled → non-reproducible. Catalogued experimental by operator direction; upstream seed support is the fix. Gated Sesame license."
|
||||
- service: zonos
|
||||
seedable: true
|
||||
model_deterministic: true
|
||||
|
||||
@@ -1,49 +0,0 @@
|
||||
# Sesame CSM stack tunables. Copy to `.env` on irv-ml1 before deploying.
|
||||
|
||||
# ── build pin ────────────────────────────────────────────────────────
|
||||
# SHA of phildougherty/sesame_csm_openai. Use the FULL 40-char SHA;
|
||||
# `main` works but is NOT reproducible — pin before relying on it.
|
||||
# https://github.com/phildougherty/sesame_csm_openai/commits/main
|
||||
CSM_SHA=main
|
||||
|
||||
# Local image tag — bump to force a fresh layer build.
|
||||
CSM_TAG=v1
|
||||
|
||||
# ── network ──────────────────────────────────────────────────────────
|
||||
# Host port. Container listens on 8000 internally.
|
||||
# Reserved on irv-ml1: 8188 ComfyUI, 8190 CosyVoice, 8191 Qwen3-TTS,
|
||||
# 8192 IndexTTS-2, 8193 Kokoro, 8194 VibeVoice, 8195 Fish-S2,
|
||||
# 8196 Chatterbox, 8197 Voxtral, 8198 Kyutai, 8199 Zonos, 8200 Dia,
|
||||
# 8765 Parakeet. 8201 picked here.
|
||||
CSM_PORT=8201
|
||||
|
||||
# Bind address. 0.0.0.0 exposes on all interfaces (incl. the WG tunnel
|
||||
# interface 10.100.79.3); 127.0.0.1 restricts to local-only.
|
||||
CSM_BIND=0.0.0.0
|
||||
|
||||
# ── runtime / GPU ────────────────────────────────────────────────────
|
||||
# Devices visible inside the container. "0" pins to the RTX 3090
|
||||
# (24 GB) — CSM-1B (~8 GB) fits; use "1" for the A6000.
|
||||
CSM_GPU_DEVICES=0
|
||||
|
||||
# Optional HF device_map for multi-GPU: auto | balanced | sequential.
|
||||
# Leave empty for single-GPU (the common case here).
|
||||
CSM_DEVICE_MAP=
|
||||
|
||||
# ── secrets ──────────────────────────────────────────────────────────
|
||||
# REQUIRED. sesame/csm-1b is a GATED model: accept its license at
|
||||
# https://huggingface.co/sesame/csm-1b
|
||||
# then paste a HF token (read scope) here. Without it, first-boot model
|
||||
# download 401s. This file is .env.example (committed) — put the REAL
|
||||
# token only in the .env on the host, which is gitignored.
|
||||
CSM_HF_TOKEN=
|
||||
|
||||
# ── persistent storage on the host ───────────────────────────────────
|
||||
# Voices dir — cloned/custom voice samples the wrapper serves by ID.
|
||||
# Included in restic (clones need the original sample to recreate).
|
||||
CSM_VOICES_DIR=/worktank/csm/voices
|
||||
|
||||
# HuggingFace cache — csm-1b + Llama-3.2-1B tokenizer + Mimi land here
|
||||
# on first start. Bind-mounted to survive recreate. Excluded from restic
|
||||
# (regenerable from HF, given the token + license acceptance).
|
||||
CSM_CACHE_DIR=/worktank/csm/cache
|
||||
@@ -1,65 +0,0 @@
|
||||
# Sesame CSM
|
||||
|
||||
Sesame's **Conversational Speech Model** (the engine behind the
|
||||
"Maya/Miles" demo) — a context-aware speech generator (Llama backbone +
|
||||
Mimi audio codec) — served via
|
||||
[phildougherty/sesame_csm_openai](https://github.com/phildougherty/sesame_csm_openai),
|
||||
an OpenAI-compat wrapper.
|
||||
|
||||
**Server:** irv-ml1 (Irvine, WireGuard-only)
|
||||
**Port:** 8201 (container listens on 8000)
|
||||
**GPUs:** pins to device 0 (RTX 3090) by default; ~8 GB VRAM
|
||||
**Image:** `local/csm:v1` — built locally from a pinned git SHA of the
|
||||
wrapper via docker buildx's git URL context
|
||||
**Upstream wrapper:** [phildougherty/sesame_csm_openai](https://github.com/phildougherty/sesame_csm_openai) (MIT)
|
||||
**Upstream model:** [sesame/csm-1b](https://huggingface.co/sesame/csm-1b)
|
||||
(**gated** — Sesame's own license) + `unsloth/Llama-3.2-1B` tokenizer + Mimi codec
|
||||
|
||||
## Why this stack exists — and what it is *not*
|
||||
|
||||
CSM is a **conversational** speech model, not a plain reader. It
|
||||
conditions on prior conversation turns (text **and** audio) to choose
|
||||
prosody and tone — it's designed to be the **speech layer of a voice
|
||||
agent**, where a separate LLM produces the words and CSM voices them
|
||||
with context-aware delivery.
|
||||
|
||||
- As **pure TTS** it works standalone (OpenAI-compat
|
||||
`POST /v1/audio/speech`, voices `alloy/echo/fable/onyx/nova/shimmer`
|
||||
plus cloned IDs).
|
||||
- Its real edge — contextual prosody across turns — only pays off in
|
||||
**interactive / voice-agent** use, not monologue narration. For
|
||||
skaldsong's *reader*, the emotive single-voice engines (Fish S2-Pro,
|
||||
IndexTTS-2) and Dia (dialogue) remain the better fits; CSM is here for
|
||||
voice-agent experiments.
|
||||
|
||||
## ⚠️ Gated model — token required before first boot
|
||||
|
||||
`sesame/csm-1b` is gated. Before deploying:
|
||||
|
||||
1. Accept the license at <https://huggingface.co/sesame/csm-1b>.
|
||||
2. Put a HF token (read scope) in `CSM_HF_TOKEN` in the host `.env`
|
||||
(never in `.env.example`).
|
||||
|
||||
Without it the first-boot model download 401s.
|
||||
|
||||
## Deploy
|
||||
|
||||
```bash
|
||||
# from this workstation (irv-ml1 is WG-only — routes via ana-wg):
|
||||
scripts/deploy-stack.sh irv-ml1 csm
|
||||
# then on irv-ml1, first run builds from the pinned SHA:
|
||||
# docker compose up -d --build
|
||||
```
|
||||
|
||||
First boot pulls csm-1b + the Llama-3.2-1B tokenizer + Mimi into
|
||||
`CSM_CACHE_DIR`; the 600 s `start_period` covers it.
|
||||
|
||||
## Notes
|
||||
|
||||
- **Pin `CSM_SHA`** to a full 40-char commit before relying on this —
|
||||
`.env.example` ships `main`, which is not reproducible.
|
||||
- The wrapper can also serve Dia-1.6B, but we run Dia from its own
|
||||
[`dia`](../dia/) stack — keep this one CSM-only to avoid overlap.
|
||||
- Endpoints: `/v1/audio/speech` (OpenAI-compat), `/v1/audio/voices`,
|
||||
`/v1/audio/models`. No dedicated `/health` route — the healthcheck
|
||||
probes `/v1/audio/voices` (only answers once the model is loaded).
|
||||
@@ -1,62 +0,0 @@
|
||||
# Sesame CSM-1B (Conversational Speech Model) served via
|
||||
# phildougherty/sesame_csm_openai — an OpenAI-compat wrapper around
|
||||
# Sesame's context-aware speech model (Llama backbone + Mimi codec).
|
||||
#
|
||||
# Why this stack exists alongside the other TTS:
|
||||
# * CSM is a CONVERSATIONAL speech layer, not a plain reader — it
|
||||
# conditions on prior turns (text + audio) to pick prosody, built
|
||||
# for voice-AGENT turn-taking. As pure TTS it works standalone;
|
||||
# its differentiator only pays off in interactive/agent use.
|
||||
# * OpenAI-compat /v1/audio/speech with 6 standard voices
|
||||
# (alloy/echo/fable/onyx/nova/shimmer) + cloned voice IDs.
|
||||
# * ~8 GB VRAM; fits the 3090 (device 0) or A6000 (device 1).
|
||||
#
|
||||
# NOTE: sesame/csm-1b is a GATED model — you must accept its license at
|
||||
# https://huggingface.co/sesame/csm-1b and supply CSM_HF_TOKEN in .env
|
||||
# before first boot, or the model download 401s.
|
||||
#
|
||||
# Image built locally from the upstream wrapper via buildx git-context.
|
||||
# All tunables live in .env — edit that, not this file.
|
||||
|
||||
services:
|
||||
csm:
|
||||
image: local/csm:${CSM_TAG}
|
||||
build:
|
||||
context: https://github.com/phildougherty/sesame_csm_openai.git#${CSM_SHA}
|
||||
dockerfile: Dockerfile
|
||||
container_name: csm
|
||||
restart: unless-stopped
|
||||
runtime: nvidia
|
||||
ports:
|
||||
- "${CSM_BIND:-0.0.0.0}:${CSM_PORT}:8000"
|
||||
environment:
|
||||
- NVIDIA_VISIBLE_DEVICES=${CSM_GPU_DEVICES:-0}
|
||||
- NVIDIA_DRIVER_CAPABILITIES=compute,utility
|
||||
# Required: gated-model access token (accept the csm-1b license on
|
||||
# HF first). Set the real value in .env, NEVER here.
|
||||
- HF_TOKEN=${CSM_HF_TOKEN}
|
||||
- HF_HUB_ENABLE_HF_TRANSFER=1
|
||||
- HF_HOME=/app/hf_cache
|
||||
# Optional multi-GPU split: auto | balanced | sequential.
|
||||
- CSM_DEVICE_MAP=${CSM_DEVICE_MAP:-}
|
||||
volumes:
|
||||
- ${CSM_VOICES_DIR}:/app/voices
|
||||
- ${CSM_CACHE_DIR}:/app/hf_cache
|
||||
healthcheck:
|
||||
# The wrapper exposes no /health route; GET /v1/audio/voices
|
||||
# returns the voice list only once the model is loaded, so it
|
||||
# doubles as liveness + ready. python urllib (no curl in image),
|
||||
# bound to 127.0.0.1 (uvicorn is IPv4-only).
|
||||
test: ["CMD-SHELL", "python3 -c \"import urllib.request,sys; urllib.request.urlopen('http://127.0.0.1:8000/v1/audio/voices', timeout=5); sys.exit(0)\""]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
# First boot pulls csm-1b + the Llama-3.2-1B tokenizer + Mimi —
|
||||
# gated download, generous deadline.
|
||||
start_period: 600s
|
||||
labels:
|
||||
- homepage.group=AI Systems
|
||||
- homepage.name=Sesame CSM
|
||||
- homepage.icon=mdi-account-voice-outline
|
||||
- homepage.description=Conversational speech model — context-aware voice-agent TTS (irv-ml1)
|
||||
- homepage.href=http://10.100.79.3:${CSM_PORT}
|
||||
Reference in New Issue
Block a user