vh
c7e21879ae
Records the deployment plus the finding that outlives it: no_speech_prob alone does not carry a hallucination threshold on this stack. Silence and room tone both hallucinated 'Thank you.' at no_speech_prob under 0.11, where a conventional >0.6 gate passes them through; avg_logprob separates ~6x better. Any future consumer asking for a Whisper hallucination gate should be pointed at the composite, not the single field. Also banks the digest-pin rationale (the VAD flag is an upstream-declared unstable variable name, so a floating tag can move gate semantics silently), the two deployment gotchas, the docker-vs-native GPU index inversion on this host, and Eyra's future diarization VRAM claim on the same A6000. Adds speaches to the irv-ml1 stack table alongside parakeet, with the reason both exist.
124 lines
5.3 KiB
Markdown
124 lines
5.3 KiB
Markdown
# irv-ml1
|
||
|
||
Secondary AI/ML inference host at the Irvine site. Formerly known as
|
||
`ana-ml1` when colocated at Anaheim; moved to Irvine and slated for
|
||
hostname rename to `irv-ml1` (OS-side rename pending — see below).
|
||
|
||
## Network
|
||
|
||
- **Reachable IP:** `10.100.79.3` (WireGuard tunnel endpoint)
|
||
- **No direct LAN access** — this host is reachable **only** via
|
||
WireGuard. Tunnel terminates at the NH3 site (10.100.0.0/16 WG
|
||
subnet). If WG is down, `scripts/refresh-server-info.sh irv-ml1`
|
||
will fail with "No route to host" — that's a WG issue, not a host
|
||
issue.
|
||
- **SSH:** `ssh irv-ml1` (config alias → `lkraven@10.100.79.3`,
|
||
key auth).
|
||
|
||
## Pending hostname rename
|
||
|
||
OS hostname still reports `ana-ml1` (both in `hostnamectl` and in
|
||
`system-details.txt`). To finish the rename:
|
||
|
||
```bash
|
||
ssh -t irv-ml1 'sudo hostnamectl set-hostname irv-ml1; \
|
||
sudo sed -i "s/ana-ml1/irv-ml1/g" /etc/hosts; \
|
||
cat /etc/hosts; hostname'
|
||
```
|
||
|
||
Then refresh the inventory snapshot so it reflects the new identity.
|
||
Not blocking anything — services don't care about the kernel's idea
|
||
of hostname.
|
||
|
||
## Hardware
|
||
|
||
- **Chassis:** (TBD — captured on next physical inspection)
|
||
- **CPU:** AMD Ryzen Threadripper 3970X (32 cores / 64 threads)
|
||
- **RAM:** 251.6 GB
|
||
- **GPUs:** 2× (unlike ana-ml2's matched pair):
|
||
- GPU 0: **NVIDIA GeForce RTX 3090** (24 GB VRAM)
|
||
- GPU 1: **NVIDIA RTX A6000** (48 GB VRAM)
|
||
- Total VRAM: 72 GB across both
|
||
- **OS:** Debian 12 (bookworm), kernel 6.1.0-37
|
||
- **Storage:**
|
||
- `/` on `/dev/nvme0n1p2` — 1.8 TB (78% used, ~393 GB free)
|
||
- `/worktank` — 462 GB (1% used — dedicated to Docker stacks' user state, e.g. ComfyUI models + workflows)
|
||
- `/storetank` — 1.8 TB (64% used, ~660 GB free)
|
||
|
||
## What it runs
|
||
|
||
### Native toolchain (`/opt`, owned by `llmuser`)
|
||
|
||
Predates the PFI docker convention; still the primary runtime for the
|
||
generative-AI stack:
|
||
|
||
- ComfyUI, SillyTavern, SDNext, fluxgym (image gen / SD)
|
||
- alltalk, alltalkv2, bark, kokoro, Orpheus-FastAPI, stablediffusion (TTS + voice)
|
||
- llama.cpp, llama-swap, koboldcpp, aphrodite (LLM inference)
|
||
- ollama (port 11434 listening on all interfaces) — native binary, not the container
|
||
- ai-toolkit, chat-ui, h2ogpt, o-textgen, lollms, bitsandbytes (misc ML frameworks)
|
||
- sillytavern-extras, simple-proxy-for-tavern (lkraven-owned)
|
||
|
||
### Docker stacks (`/opt/docker/compose/`, owned by `lkraven`)
|
||
|
||
Docker 29.4.1 with `nvidia` and `runc` runtimes. `lkraven` is in the
|
||
`docker` group. `traefik-net` external network exists for stacks that
|
||
need it.
|
||
|
||
| Stack | Port | GPU | Role |
|
||
|-------|------|-----|------|
|
||
| dockge | 5001 | — | Per-host Compose UI |
|
||
| beszel-agent-irv | 45876 | — | Metrics agent → Beszel hub on ana-docker (token mode through WG) |
|
||
| dozzle-agent-irv | 7007 | — | Log agent → Dozzle hub on ana-docker |
|
||
| comfyui | 8188 | all | ComfyUI (node-based SD/Flux) — runs independently of `/opt/ComfyUI` native install |
|
||
| kokoro | 8193 | all | Kokoro 82M TTS — lowest-latency English TTS, OpenAI-compatible /v1/audio/speech |
|
||
| chatterbox | 8196 | 0 (3090) | Chatterbox voice-cloning TTS |
|
||
| index-tts | 8192 | all | IndexTTS multilingual TTS |
|
||
| qwen3-tts | 8191 | all | Qwen3-TTS — 7B multilingual TTS |
|
||
| cosyvoice | 8190 | all | CosyVoice multilingual TTS |
|
||
| fish-s2 | 8195 | 1 (A6000) | Fish-Speech S2 TTS |
|
||
| kyutai-tts | 8198 | 0 (3090) | Kyutai TTS |
|
||
| vibevoice | 8194 | 1 (A6000) | Microsoft VibeVoice TTS |
|
||
| voxtral | 8197 | 1 (A6000) | Mistral Voxtral ASR |
|
||
| parakeet | 8765 | all | NVIDIA Parakeet ASR (transcription) — bare `{"text": …}`, no `no_speech_prob` |
|
||
| speaches | 8204 | 1 (A6000) | OpenAI-compatible faster-whisper ASR — `verbose_json` w/ per-segment `no_speech_prob`; serves Eyra. VAD pinned OFF, image digest-pinned |
|
||
| stable-audio-open | 8211 | 1 (A6000) | Stable Audio Open 1.0 — diffusion SFX/ambience generator |
|
||
| ace-step | 8210 | 1 (A6000) | ACE-Step 1.5 — Apache-2.0 hybrid diffusion+LLM music generation |
|
||
|
||
GPU pinning comes from the compose `NVIDIA_VISIBLE_DEVICES` default;
|
||
`all` means the model picks at runtime (typically GPU 0). The
|
||
asset-engine orchestrator (`docs/asset-engine/services.yaml` → per-service
|
||
`lifecycle:` block) gates start operations on per-device VRAM headroom.
|
||
|
||
Exposed Docker socket on `*:2375` (for the homepage integration hub on
|
||
esh-docker-vm, which auto-discovers containers on this host).
|
||
|
||
## Storage watch
|
||
|
||
Nothing acute. `/storetank` dropped from 92% → 64% after a prune pass
|
||
on the native-toolchain side; keep an eye on it since model weights
|
||
and training outputs accumulate steadily (misbehavior starts around
|
||
~95% on either ext4 or ZFS).
|
||
|
||
## Backup coverage
|
||
|
||
Restic via `resticprofile` + systemd timer (01:00 daily) → `rest-server-nh3`
|
||
(local to the WG endpoint site; lower latency than crossing back to
|
||
ana-side). Profile tracked at `configs/restic/irv-ml1/profiles.yaml`.
|
||
Excludes HuggingFace caches and bulk model files on `/storetank`
|
||
(regenerable from HF Hub).
|
||
|
||
When the ComfyUI stack ships, `/worktank/comfyui/basedir/{user,custom_nodes,input}`
|
||
should be added to the source set (workflows + hand-installed nodes);
|
||
`/worktank/comfyui/basedir/{models,output}` stay excluded (bulk /
|
||
regenerable).
|
||
|
||
## Refresh state
|
||
|
||
```bash
|
||
scripts/refresh-server-info.sh irv-ml1
|
||
```
|
||
|
||
**Caveat:** requires the WG tunnel to be up. If the refresh shows
|
||
"No route to host", bring WG up before retrying.
|