diff --git a/persistent-memory.md b/persistent-memory.md index b785448..15de074 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -109,6 +109,9 @@ no longer deployed sidecars here. See Recent decisions.) ## Current state / in-flight +- **🟢 IPv6 PLAN SETTLED 2026-08-19 — endpoints, not internal numbering.** IPv6 is for **tunnel endpoints** (Site Magic + WireGuard), **not** for numbering LANs: an endpoint needs one routable address, not a prefix. **ESH has a whole `/56`** (`2607:73c0:1004:1::220`, `pd_size=56`) — the long-standing "ESH: none" note was wrong. **NH3** has its WAN GUA `2600:1700:b25:c110::48` plus one unused `/64`. **ANA** expects a `/48` or `/56` within days. **Nothing was ever down** — ESH is outbound and working; its v4 CGNAT (`100.104.3.250`, Cogent AS174 egress) is the ordinary CGNAT-v4-plus-generous-v6 pattern, not an outage, and the previous "CGNAT broke the mesh" framing sent a three-arm research effort chasing a non-problem. NH3 internal v6 was enabled on the `/64`, **verified working end-to-end**, then **deliberately reverted** — one `/64` lights one VLAN. AT&T prefix findings (the /60 is real but undelegatable; the live trick is multiple `IA_PD` in one solicit, not VRRP; the UDM can express neither) are filed as **reference only**. → auto-memory `reference_fleet_vpn_topology_ipv6` + + - **🔴→🟢 ESH OUTAGE 2026-08-19 — esh-pve hard-froze 03:34, ~4.5h, whole house lost DNS.** Presented as "wifi/routing issues"; internet was healthy throughout. Cause: `esh-userland` (VLAN 10, the `PVC` SSID) handed out **one** resolver, `10.0.50.45` (AdGuard on `esh-docker-vm`) — cross-VLAN, no secondary — and its hypervisor died. **Recovered by hand power-cycle; all VMs, cluster quorum and DNS restored.** Two fixes landed: gateway `10.0.10.1` added as secondary DNS on esh-userland (operator-approved, first confirmed WRITE on the ESH UDM key), and **`softdog` → `iTCO_wdt` hardware watchdog owned by systemd** (`playbooks/esh-pve-hardware-watchdog.yaml`, idempotent, verified armed) so a repeat self-recovers in 60s. **VM 102 pinned off** (`onboot: 0`) as the passthrough suspect. **⏳ OPEN:** (a) the watchdog is armed but **has not been proven to fire** — needs a deliberate wedge to confirm; (b) AMT/vPro still unusable until an onboard **RJ45** is cabled (the MS-01 is SFP+-only on the network and AMT cannot ride it); (c) kernel `6.8.12-16` rollback held in reserve if it freezes again. → `persistent-memory.d/2026-08-19-esh-pve-freeze-dns-spof.md` _As of 2026-08-19 (late) — a long infra session, all of it landed and committed: **fleet `*.internal` DNS is live** (git-sourced, 42 names, three resolvers incl. a NEW colo one), **waterland studio containerised** on irv-ml1, **Homepage cleaned up and themed** with Australis Skyfall + an Arbo-generated background, and **four unmanaged stacks adopted** into `stacks/` (two of which turned out to be quietly broken — SearXNG's healthcheck, SeaFile down 3 months). ⚠ 17 commits unpushed. ~~⚠ 2 unread althing messages from waterland-dev left deliberately undrained~~ — **drained 2026-08-19 post-`/clear`; thread closed, see below.**_