docs(memory): record the Parakeet bench result and a 96-place stale address on irv-ml1
tts-dev benched both endpoints against a Whisper baseline. FV wins at both clip lengths (155/391 ms vs IRV 354/1010 vs whisper-large-v3 457/690) — IRV is slower than the incumbent at 6.24 s, so the duplicate seat is now retirable on evidence rather than on tidiness. Their length sweep fits ~58 ms fixed + 56 ms per audio-second, asymptote ~17.8x realtime, independently reproducing our 17x on a different clip and a different harness. The gateway hop measured below their harness resolution, so ext-stt is the right consumer path. Two caveats recorded against our own numbers: their between-run variance is ±20% because GPU 0 carries the live chat path, and our 0.50 s median was taken on an idle GPU 3 — marked as a best case, not a comparable. Also records tts-dev's retraction, which is the durable lesson: published RTFx is batched throughput on datacenter hardware, not single-stream latency, and the two differ by ~200x. Their plan had projected 60-120 ms from it. Separately, chasing the one stale Homepage href they flagged turned up 96 occurrences of the retired wg0 lifeline 10.100.79.3 under /opt/docker on irv-ml1. Most are cosmetic, but voice-studio is genuinely broken: it is configured to reach studio-gate at that address, both are running, they sit on separate docker networks, and the address is on no interface on the host. Failing since the 2026-09-06 cutover with nothing alerting. ext-tts verified unaffected. Not fixed here — eight containers to recreate, three load-bearing, and the voice-studio repair touches app.py rather than config. Surfaced with evidence. The pattern is the third of its shape: a retired address needs a repo-wide grep by ADDRESS rather than by hostname, and container labels live in no file the sweep reads until the container is recreated.
This commit is contained in:
@@ -0,0 +1,58 @@
|
||||
# irv-ml1 still points at the retired wg0 lifeline `10.100.79.3` (2026-09-15)
|
||||
|
||||
Found while chasing a single stale Homepage href that tts-dev flagged after the
|
||||
Parakeet bench. It is not one card.
|
||||
|
||||
## Scope
|
||||
|
||||
`10.100.79.3` — the wg0 tunnel lifeline retired at the **2026-09-06 headscale
|
||||
cutover** — appears **96 times** under `/opt/docker` on irv-ml1. The address is on
|
||||
**no interface on that host**: it is `10.6.110.50` (Irvine LAN) and `100.64.0.6`
|
||||
(mesh). A request to it gets no route (`curl` → `000`), not a refusal.
|
||||
|
||||
32 homepage.href labels
|
||||
64 other (mostly README / .env.example / .bak — but not all)
|
||||
|
||||
**Eight RUNNING containers carry a dead `homepage.href`:** `breeze-tts`,
|
||||
`tts-gateway`, `arbo`, `dockge`, `waterland-studio`, `comfyui`, `parakeet`,
|
||||
`kokoro`.
|
||||
|
||||
## ⚠ One LIVE breakage, not just dead links
|
||||
|
||||
- **`voice-studio` cannot reach `studio-gate`.** Its running container carries
|
||||
`STUDIO_GATE_URL=http://10.100.79.3:8217`. `studio-gate` is up (4 weeks) and
|
||||
answers on 8217 at `127.0.0.1`, `10.6.110.50` and `100.64.0.6`. The two are on
|
||||
**separate docker networks** (`voice-studio_default` / `studio-gate_default`), so
|
||||
voice-studio must reach it by a host address — and it is using a dead one. Its
|
||||
gate calls have been failing since 2026-09-06 and nothing alerted.
|
||||
`voice-studio/app.py` also hardcodes the same dead address at `:8208` and `:8212`.
|
||||
One-line unblock: `STUDIO_GATE_URL` → `http://10.6.110.50:8217`.
|
||||
- **`waterland-studio`'s `homepage.siteMonitor`** points at
|
||||
`http://10.100.79.3:8410/api/health`, so Homepage reports it down while it runs fine.
|
||||
|
||||
## ✅ What is NOT affected — checked explicitly
|
||||
|
||||
**`tts-gateway` / `ext-tts` is fine.** Its live `.env` uses
|
||||
`irv-ml1.nh3.internal:8204`; only its `.bak` files and `.env.example` carry the dead
|
||||
IP. The fleet TTS path is unaffected — verified by actually generating audio
|
||||
through `ext-tts` during the Parakeet work.
|
||||
|
||||
## Why it was not fixed on the spot
|
||||
|
||||
Eight containers to recreate, three load-bearing (`arbo`, `tts-gateway`, `comfyui`),
|
||||
on a host outside the night's scope, and the voice-studio repair touches `app.py`
|
||||
rather than config — somebody else's code. Broken nine days already; it wants a
|
||||
scheduled pass, not a 02:00 improvisation. Surfaced to the operator with this
|
||||
evidence.
|
||||
|
||||
## The pattern this belongs to
|
||||
|
||||
Third instance of the same shape. The 2026-09-13 ana-ml2→fv-ml1 renumber left 16
|
||||
live Homepage entries on a dead IP; the sweep allowlist was built from files that
|
||||
mention the HOST, and an `href` mentions only an IP, so every label-only stack fell
|
||||
outside it **by construction**. Same failure here, different cutover.
|
||||
|
||||
⭐ **A retired address needs a repo-wide grep by ADDRESS, not by hostname, and it
|
||||
needs to cover running container labels — which live in no file the sweep reads
|
||||
unless the container is recreated.** Labels apply at creation, so a fixed compose
|
||||
with a stale container still serves the stale label.
|
||||
Reference in New Issue
Block a user