From a3184f66b9d26b8265b37f41054bd4d9b39dce22 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Thu, 8 Oct 2026 13:32:03 -0700 Subject: [PATCH] docs(nh3-svos): front door on mesh + LAN, internal DNS answers the LAN address; fv-ml1 off-mesh by design --- persistent-memory.md | 2 ++ servers/nh3-svos/README.md | 19 ++++++++++--------- 2 files changed, 12 insertions(+), 9 deletions(-) diff --git a/persistent-memory.md b/persistent-memory.md index 7794053..2777fce 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -187,6 +187,8 @@ Every vLLM seat on a shared card (GPU 0, GPU 1) has byte-pinned KV (`--kv-cache- ## Recent decisions +- `[2026-10-08]` **SVOS served on the LAN too (Prime):** Caddy binds `10.100.50.43:443` as well as `100.64.0.9:443`; the internal AdGuards answer `svos`/`svos-terminal.phasefinal.com` → `10.100.50.43` (MagicDNS still answers 100.64.0.9). Verified from LAN hosts at all three sites. → `servers/nh3-svos/README.md` +- `[2026-10-08]` **fv-ml1 is OFF the mesh again, by design**: its tailscaled is disabled, and the break-glass watchdog joins only on fleet loss (09-15 design, archived). The 0030 reboot closed a door that had been left open, so mesh-side tests need another peer, not fv-ml1. The watchdog timer is active. - `[2026-10-08]` **heid's runaway sweep killed on Prime's order: two `heidnotes/sweep.py` processes had written ~76 GB into a deleted-but-open file; nh3-dev root 86% → 66%** — the sweep read its own output. heid fixed it at the source. → `persistent-memory.d/2026-10-08-heid-runaway-sweep-killed.md` - `[2026-10-08]` **SVOS post office LIVE on nh3-svos** — Prime declared `svos`; its `ns:notifications` write grant was approved on his word at 0731 (card `card:4d3a8b26c17f472d828b06fd495192db`); a test message was ingested and delivered; `reach` ok, unread 0. Reported to svos-dev. - `[2026-10-07]` **No z.ai vision alias on the gateway (Prime).** glm-5.x is text-only at z.ai; glm-4.6v and glm-5.3-flash read images on our plan but stay off the gateway. diff --git a/servers/nh3-svos/README.md b/servers/nh3-svos/README.md index da647e6..2d14d20 100644 --- a/servers/nh3-svos/README.md +++ b/servers/nh3-svos/README.md @@ -67,16 +67,17 @@ scripts/elway infra-ops@10.100.50.43 --playbook playbooks/nh3-svos-host.yaml ## Front door: mesh-only `https://svos.phasefinal.com`, HTTP Basic auth (Prime, 2026-10-05) - **Mesh:** tailscaled joined headscale as `nh3-svos`, **100.64.0.9**, `--accept-dns=false --accept-routes=false`. -- **Name:** `svos.phasefinal.com` exists ONLY as a headscale MagicDNS extra record (`/var/lib/headscale/extra-records.json` - on nh3-headscale, hot-reloaded) → 100.64.0.9. There is no public record. **Since 2026-10-08 (Prime: "internal name - servers don't resolve svos.phasefinal.com") the three internal AdGuards (ana `10.250.50.70:8053`, esh `10.0.50.45`, - nh3 `10.100.50.40`) also carry hand-made rewrites** `svos.phasefinal.com` and `svos-terminal.phasefinal.com` → - `100.64.0.9`, outside `dns-sync.py`'s `.internal` authority, like `*.nh3.phasefinal.com`. The name now resolves - everywhere, but only a **mesh client** can connect: LAN-only hosts at all three sites get `000` (no route to - 100.64.0.0/10), measured 10-08. +- **Name:** `svos.phasefinal.com` (and `svos-terminal.phasefinal.com`) has no public record. Mesh clients get + **100.64.0.9** from the headscale MagicDNS extra records (`/var/lib/headscale/extra-records.json` on nh3-headscale, + hot-reloaded). **Since 2026-10-08 (Prime: "serve it on the LAN too and point the dns at the internal LAN address")** + the three internal AdGuards (ana `10.250.50.70:8053`, esh `10.0.50.45`, nh3 `10.100.50.40`) carry hand-made + rewrites for both names → **`10.100.50.43`**, outside `dns-sync.py`'s `.internal` authority (like + `*.nh3.phasefinal.com`). Verified 10-08 from LAN hosts at all three sites (nh3-dev, nh3-docker, esh-docker-vm, + ana-docker): `/` 401, `/sw.js` 200, terminal `/` 200, terminal `/health` 403. nh3-svos itself still resolves both + names to 100.64.0.9 via `/etc/hosts`, so the kernel's signed calls to the terminal keep arriving from 100.64.0.9. - **TLS:** Caddy 2.11.4 (the fleet `caddy-cf` build with the Cloudflare DNS module) at `/usr/local/bin/caddy`, unit - `caddy-svos.service`, config `/etc/caddy/Caddyfile` (canonical `configs/svos-front/Caddyfile`). It listens **only on - 100.64.0.9:443**, with no `:80`. The LE cert for `svos.phasefinal.com` comes via DNS-01, token `nh3-svos-caddy-dns01` + `caddy-svos.service`, config `/etc/caddy/Caddyfile` (canonical `configs/svos-front/Caddyfile`). It listens on + **100.64.0.9:443 and, since 2026-10-08, 10.100.50.43:443** (both site blocks), with no `:80` and nothing on the internet. The LE cert for `svos.phasefinal.com` comes via DNS-01, token `nh3-svos-caddy-dns01` (phasefinal.com only), in `/etc/caddy/env` (root:caddy 0640), vaulted `nh3-svos/cloudflare-caddy-dns01-token`. - **Identity: HTTP Basic auth, one account `Vuong`** (Prime: "Basic auth is fine, saved in browser is fine. Access is via mesh because we don't host externally." SVOS addresses him as Prime; the login is `Vuong`). The argon2id hash is in