From ad21302474deb77ddc5eaac6150ea3b471f9f3b0 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Fri, 21 Aug 2026 22:01:08 -0700 Subject: [PATCH] memory: ESH internal IPv6 live on two LANs; Cityside v4 static proven unprovisioned Operator-directed IPv6 enablement on ESH-UDMPM: esh-userland (VLAN 10, SSID PVC) and esh-server flipped from ipv6_interface_type=none to pd against the delegated 2607:73c0:402:1d00::/56. Verified esh-docker-vm egresses over v6 as its own address with no NAT, against v4 which still egresses as the shared CGNAT pool. No disruption through either write; originals backed up. NH3 stays v6-off deliberately -- its single /64 is reserved for network meshing, so nh3-dev having no v6 egress is correct rather than a fault. The v4 static is NOT provisioned, and this is now proven rather than inferred. A full gateway reboot -- verified by observed down-transition and an uptime reset from 140,846s to 183s -- forced a fresh DHCP DISCOVER and Cityside returned the identical address. Recorded alongside it: a DHCP renew could never have surfaced a new address in the first place, since renewal preserves the lease by design. Next step is a carrier ticket, not more testing. The v6 firewall posture was audited and is correct -- default-deny inbound for both IP versions, only four hand-written policies, none WAN-facing. My earlier warning that v6 removes CGNAT's accidental shielding was overstated and is retracted in the record. The audit's limitation is stated: it read rules, it never probed from outside, because no v6 vantage exists. Flat-zone lateral-movement finding parked as id 44 rather than actioned -- operator's call, moot until the mesh and segmentation design settle. --- persistent-memory.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/persistent-memory.md b/persistent-memory.md index 456c5ce..20dd7d6 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -125,6 +125,8 @@ _As of 2026-08-21 (late) β€” **the big AI-seat overhaul session; three seats set - **🟒 SPEACHES ASR β€” LIVE on irv-ml1 A6000 :8204 (2026-08-21, operator-approved).** OpenAI-compatible faster-whisper for **Eyra** (meeting recorder, `eyra-dev`). `Systran/faster-whisper-large-v3` + `Systran/faster-distil-whisper-large-v3`, fp16, both resident (`STT_MODEL_TTL=-1`), ~5.9 GB VRAM / 20 GB still free. **Deliberately co-exists with `parakeet` (:8765)** β€” parakeet is a TDT/transducer returning bare `{"text":…}` with **no `no_speech_prob` concept**, so it structurally cannot serve this consumer. **β˜… THE MEASURED FINDING IS WORTH MORE THAN THE DEPLOY: `no_speech_prob` ALONE IS A WEAK HALLUCINATION GATE.** Silence and pink room tone both produced the classic Whisper `"Thank you."` hallucination while `no_speech_prob` stayed **under 0.11** β€” a conventional `>0.6` threshold passes both through. `avg_logprob` separates ~6Γ— better (βˆ’0.114 speech vs βˆ’0.650/βˆ’0.724 non-speech); `compression_ratio` 1.141 vs 0.556. **Consumers must gate on a composite.** (Synthetic inputs β€” shape of the separation, not calibration constants.) **VAD pinned OFF** (`_UNSTABLE_VAD_FILTER=False`) at consumer request β€” they VAD-gate upstream; consequence is the service will transcribe silence and does not defend itself. **β˜… IMAGE PINNED BY DIGEST, not `:latest-cuda`** β€” the VAD flag is an upstream-declared *unstable* var name, so a floating bump could rename it, restore VAD, and move gate semantics with **no error and no log line**; bumping = deliberate + re-run the checks. ⚠ Two gotchas: `PRELOAD_MODELS` **does not download** (only loads already-cached β€” use `POST /v1/models/{id}`), and the bind-mounted cache needs a `hub/` subdir or **every** `/v1/models` 500s `CacheNotFound` **while `/health` still returns 200**. ⚠ docker `device_ids: ["1"]` = A6000 (native CUDA inverts this β€” sees A6000 as `cuda:0`). ⏳ **NOT DONE by design: no LiteLLM gateway alias** β€” agreed sequencing was direct-first; adding it means editing the shared gateway + re-running fidelity. ⏳ Eyra's **diarization workload** (GPU pyannote, gated weights, own HF token) is a **future VRAM claim on this same A6000** β€” arrives at their diarize milestone. Canonical `stacks/speaches/`, commit `aa5863c`. +- **🟒 ESH INTERNAL IPv6 β€” LIVE on two LANs 2026-08-21 (operator-directed), and the CGNAT static is a CARRIER problem.** Enabled `ipv6_interface_type: none β†’ pd` + `ipv6_pd_interface: wan` on **`esh-userland`** (VLAN 10, SSID `PVC`) and **`esh-server`** (10.0.50.0/24) on ESH-UDMPM. Cityside delegates **`2607:73c0:402:1d00::/56`**; per-VLAN /64s userland `:1d00`/`:1d01`, esh-server `:1d02`. **Verified: `esh-docker-vm` egresses over v6 as its OWN address (`2607:73c0:402:1d02:be24:11ff:fe13:3a8`) β€” no NAT** β€” vs v4 which egresses as the shared CGNAT pool `154.50.58.1`. Zero disruption through both writes (all subsystems ok, 54 wireless clients, Open WebUI 200, DNS resolving). Originals backed up. **⚠ NH3 stays v6-OFF deliberately β€” its single `/64` is RESERVED FOR NETWORK MESHING** (operator). **β˜… THE v4 STATIC IS NOT PROVISIONED β€” proven, stop testing it.** A full gateway **reboot** (verified by observed down-transition + uptime reset 140,846sβ†’183s) forced a fresh DHCP **DISCOVER** and Cityside returned the *identical* `100.104.3.250`. Also tried: physical cable pull (missed β€” WAN is **`eth8` = Port 9**, 2.5GE copper; ports 1–7 have **never carried a byte**), `renew-dhcp` (**not a valid devmgr verb**), `force-provision` (no WAN effect). **A DHCP *renew* could never have worked anyway β€” renewal PRESERVES the lease by design; only release/rebind or link-down forces a new DISCOVER.** Next step is a **Cityside ticket**: is the static applied or merely billed, and will it arrive as a DHCP reservation (a bounce picks it up) or a manual assignment (needs IP/mask/gw β†’ `wan_type=static`)? **v6 firewall AUDITED and CORRECT** β€” default-deny WANβ†’internal for BOTH versions, only 4 hand-written policies (all IoT-scoped, none WAN-facing); my earlier "v6 strips CGNAT's accidental shielding" warning was **overstated and is retracted**. ⚠ audit was config-based, **never probed from outside** (no v6 vantage exists β€” NH3 has none by design). Flat-zone lateral-movement finding **parked, id 44** `esh-split-cameras-mgmt-out-of-the-flat-internal`. Cleanup: the dead `Cable Internet` UCI modem (last seen 08-18, replaced by Cityside; failover is the 5G on WAN2) was **deleted from the controller**, which cleared the lingering `wlan`/`lan` **error** states to ok. `vpn: error` persists and predates all of this. β†’ auto-memory `reference_fleet_vpn_topology_ipv6` + - **🟒 FLEET `.internal` DNS β€” LIVE 2026-08-19.** `..internal`, sites `ana`/`esh`/`nh3`. `dns/internal.yaml` is the source of truth; `scripts/dns-sync.py` reconciles the three AdGuard resolvers (diff β†’ prompt β†’ apply, idempotent). 42 names resolving from all three sites. **Colo got its first resolver ever** (`stacks/adguard-ana/`, API on **8053** not 8080, no blocklists by design) β€” before this, ana-docker resolved straight against `1.1.1.1`. Auth = a dedicated `infra-ops` AdGuard user, password vaulted `nh3-dev/adguard-infra-ops-password`. **⏳ TWO OPEN, both operator's to schedule:** (a) colo hosts still point at `1.1.1.1` so they do not yet *use* the new resolver β€” repointing a site's DNS is a separate change; (b) the static-v6 convention (each server at its site's `/64` with low bits echoing the v4 octet, `esh-docker-vm` β†’ `…::45`) is **proposed, not ruled on**. The `v6:` column is empty and correct β€” no fleet host has a global v6 address yet. β†’ `persistent-memory.d/2026-08-19-fleet-internal-dns.md` - **🟒 WATERLAND STUDIO β€” containerised + live, irv-ml1:8410.** Displaced a bare `nohup` (PID 1283383). Warm 256Β²+anim render **6.1s**, beating the 7.4s bare-metal figure. `claude-bot` now has **read** on `vh/waterland` (operator-granted); token on irv-ml1 root-owned 0600, repo-scoped helper, nothing in `.git/config`. **βœ… All three upstream findings fixed and DEPLOYED at `b72425b`** (one `update.sh` run, 2026-08-19): PR #5 declared the CUDA headers and took `uv` out of the render path; PR #6 (operator-green-lit) added the startup rehydrate that fixes the unbounded job-store growth. Verified after: `/api/jobs` **1 β†’ 16** against 16 dirs on disk β€” API and volume agree for the first time β€” nothing wrongly reclaimed (16 < `RETAIN=40`), healthy on `backend: cupy`, warm render intact. The image **keeps** its `[ctk]` install (+0.3s, now a no-op audit) and `UV_NO_SYNC`/`UV_OFFLINE` as defence-in-depth. Thread closed by both sides. β†’ `persistent-memory.d/2026-08-19-waterland-studio-containerised.md`