# irv-ml1 dead-address sweep — EXECUTED 2026-09-15 (operator-directed)

Closes `2026-09-15-irv-ml1-dead-wg0-address.md`. `10.100.79.3` (the wg0 lifeline
retired at the 2026-09-06 headscale cutover) is on **no interface anywhere**; a
request to it gets *no route*, not a refusal.

**Result: 0 of 112 Homepage cards point at it. Was 9.**

## Which address, and why it differs per host — this is the load-bearing part

⚠ **There is no single right answer.** The correct target depends on **who resolves
it**, and getting this wrong swaps a dead address for an unresolvable one — which is
what tts-dev nearly did on voice-studio.

| resolver | can it resolve `*.nh3.internal`? | use |
|---|---|---|
| operator's browser (`homepage.href`) | yes | **name** |
| Homepage container @ esh-docker-vm (`siteMonitor`) | **yes** (measured) | **name** |
| open-webui container @ esh-docker-vm | **yes** (measured) | **name** |
| containers on **irv-ml1** | **NO** | IP |
| containers on **ana-docker** | **NO** | IP |

⚠⚠ **litellm on ana-docker resolves the name ONLY because of its own `extra_hosts`
entry.** Testing from litellm and generalising would have been wrong: `asset-engine`
on the same host, with no `extra_hosts`, gets `Name or service not known`. **Test from
the container you are about to change, not a neighbour.**

Positive control used before committing to the name: the Homepage container actually
*fetched* ytvc's `/healthz` through `irv-ml1.nh3.internal` and got `{"ok":true,…}` —
the pattern already worked in production.

## What was changed

**irv-ml1 — 24 files swept**, 14 comment-only hits deliberately left as
port-allocation history.

- **7 running** containers recreated so labels took: `arbo`, `breeze-tts`, `comfyui`,
  `dockge`, `kokoro`, `tts-gateway`, `waterland-studio` (+ its `siteMonitor`).
  `dockge`'s href is built from `DOCKGE_HOST_IP` in `.env`, so one var fixed it.
- **7 dormant** containers in `Created`/`Exited` state carried stale labels because a
  compose edit alone does not touch an existing container object. Fixed with
  **`docker compose create --force-recreate`** — rebuilds the container with new
  labels **without starting it**, which is exactly right for a deliberately dormant
  stack: `chatterbox-fast`, `dots-tts`, `speaches`, `cosyvoice`, `fish-s2`,
  `index-tts`, `omnivoice`. `omnivoice-ref` (a different stack, Up 4 weeks) untouched.
- 17 more dormant compose files swept; inert until someone starts them.

## ⚠⚠ FOUR MORE LIVE BREAKAGES on OTHER hosts — the sweep's real find

The renumber left a trail off irv-ml1 that nobody grepped for. All silently dead nine
days, none alerting:

| host | service | was | now |
|---|---|---|---|
| esh-docker-vm | **open-webui** read-aloud TTS | `AUDIO_TTS_OPENAI_API_BASE_URL=http://10.100.79.3:8198/v1` | `irv-ml1.nh3.internal` |
| ana-docker | **asset-engine** (Up, healthy) | `INFERENCE_HOST=10.100.79.3` | `10.6.110.50` |
| ana-docker | skaldsong TTS (Exited 2 mo) | `SKALDSONG_TTS_BASE_URL=…8193` | `10.6.110.50` |
| ana-docker | skaldsong vibevoice | `…8194` | `10.6.110.50` |

Both running services recreated and **verified reaching their targets after the
change**, not just carrying the new string. Repo `.env.example` templates updated to
match per-host (a template hands the next person a dead address otherwise).

## ⚠ My own error, worth not repeating

I recreated **`breeze-tts` for a cosmetic label change** and took the fleet TTS path
down for its ~90 s CUDA-graph warm-up. `ext-tts` returned **500** until it finished.
Caught it only because I had taken a **baseline before touching it** (200 / 35,180
bytes) — without that, a 500 would have looked like a pre-existing fault.

⭐ **A label-only edit still costs a full model reload on a GPU container.** Batch
those into a window or accept a stale label; do not bounce a warming model for a link.

## Still open

- `speaches`'s label says `:8204`, which is **breeze-tts's live port**. Latent conflict
  if anyone starts it. Not fixed — it is a port-register question, not an address one.
- ⭐ The durable fix for ana-docker/irv-ml1 containers is **`extra_hosts` + the name**
  (what `litellm` and `talk` already do), so the IP lives in one place a single `.env`
  line moves. Left as an improvement rather than done, because `asset-engine` is
  CI-deployed and its compose is not mine to edit from here.