docs: refute the CGNAT-breaks-Site-Magic premise with a live test
The fleet IPv6 work was justified primarily by the expectation that ESH fiber landing behind CGNAT would break Site Magic on IPv4, making v6 the escape hatch. The fiber cutover provided a free natural experiment and the premise does not hold. Cox was unplugged, ESH failed over to the 5G WAN (already configured failover-only, so this needed no intervention), and the resulting WAN address is 192.168.200.111 -- RFC1918, double-NAT, no inbound path at all, which is strictly worse than the CGNAT that was feared. Site Magic stayed up throughout: all four ESH hosts reachable, ssh and command exec working, 20MB pulled over the tunnel, latency 15ms -> ~46ms as expected for cable to 5G. The mechanism is visible on the device: magic_site_to_site_vpn holds only `enabled` plus a WireGuard keypair, with peer orchestration in the UniFi cloud and no WAN binding of any kind. NH3's edge is publicly reachable, so the NAT'd side dials out and never needs reachability. Consequence: no addressing outcome on the new fiber -- public, CGNAT or double-NAT -- threatens the inter-site tunnel. IPv6 stays worth doing on its own merits but stops being urgent, and stops gating anything. Also worth recording that Site Magic cannot be pinned to a WAN. It rides whichever uplink is active, so the only lever is failover priority -- which moves all site traffic, not just the tunnel. The existing failover-only config on WAN2 already handles a primary-WAN outage correctly and needed no change.
This commit is contained in:
@@ -127,7 +127,7 @@ _As of 2026-08-18 — **gen seat SWAPPED to `absolute-heresy` and operator-confi
|
||||
|
||||
- **EVIDENCE HOLD (partial):** WT #394 FILE half STILL STANDS — do NOT delete on-disk gen dirs (`fiction/rex390-dcc`, `rex392-dcc`, `b59c147c5ce0`); rex393-fiction-* + r42-gate-* KEEP.
|
||||
|
||||
- **🟡 FLEET IPv6 — mapped, nothing enabled; WAITING ON ADDRESSES.** Driver is **ESH fiber installing 2026-08-18 landing the house behind CGNAT**, which breaks **Site Magic** (NH3↔ESH) on IPv4 → v6 is the escape hatch and the likely first consumer. State: **NH3 WAN live** (`2600:1700:b25:c110::48`, AT&T delegates exactly **one /64**), **colo none** (FortiGate has zero v6), **ESH none** (both WANs `wan_type_v6=disabled`). NH3 LANs all reverted to `ipv6_interface_type=none` per operator. Work when addresses land: v6 on `ana-wg` eth0 + a **v6 port-forward for UDP 31337** on the FortiGate (its WG socket is already dual-stack — no WG reconfig), flip the UDM WG server off `v4`-pinned binding, and **AAAA records** so the dynamic prefixes at all three sites don't break endpoints. Full detail + access recipes → `persistent-memory.d/2026-08-17-fleet-ipv6-mesh.md`.
|
||||
- **🟢 FLEET IPv6 — DRIVER REFUTED 2026-08-18; downgraded from urgent to nice-to-have.** The premise was that ESH fiber landing behind CGNAT would **break Site Magic (NH3↔ESH) on IPv4**, making v6 the escape hatch. **That is empirically false.** During the fiber cutover Cox was unplugged and ESH failed over to the 5G WAN, which handed out `192.168.200.111` — **RFC1918, double-NAT, zero inbound reachability, strictly worse than CGNAT** — and Site Magic stayed up carrying real traffic (all four ESH hosts pingable, ssh + exec fine, 15ms → ~46ms). Mechanism: Site Magic is **WireGuard** (`magic_site_to_site_vpn` on-device holds only `enabled` + a keypair), and **NH3's edge is publicly reachable at `70.230.226.88`**, so the NAT'd side just dials out; it never needs an inbound path. **So no NAT/CGNAT outcome on the fiber threatens the tunnel.** v6 remains worth doing on its own merits, but it is not gating anything. State: **NH3 WAN live** (`2600:1700:b25:c110::48`, AT&T delegates exactly **one /64**), **colo none** (FortiGate has zero v6), **ESH none** (both WANs `wan_type_v6=disabled`). NH3 LANs all reverted to `ipv6_interface_type=none` per operator. Work when addresses land: v6 on `ana-wg` eth0 + a **v6 port-forward for UDP 31337** on the FortiGate (its WG socket is already dual-stack — no WG reconfig), flip the UDM WG server off `v4`-pinned binding, and **AAAA records** so the dynamic prefixes at all three sites don't break endpoints. Full detail + access recipes → `persistent-memory.d/2026-08-17-fleet-ipv6-mesh.md`.
|
||||
|
||||
- **🟢 WT #401 (fd-leak deadlock) CLOSED 2026-08-17 — one ping still owed.** worldtree-dev closed it on our demo verify. Layers: **(a) their `e41b139`** pins `ulimits: nofile 65536/65536` in the worldtree compose anchor — **demo VERIFIED** (api + matrix recreated 22:55:34Z, `ulimit -Sn`=65536); **personal/pinned are covered-not-verified**, they inherit at their next promotion/recreate. **(b) our host floor is STAGED, NOT ACTIVE** — `/etc/docker/daemon.json` on corviduo-dev carries `default-ulimits nofile 65536/65536` but **`default-ulimits` is NOT SIGHUP-reloadable** (measured on 29.4.3: post-reload the daemon's own "Reloaded configuration" log omits it and a fresh container still reports 1024). Activation needs a full dockerd restart = bounces all 13 containers; **worldtree-dev explicitly does NOT want one**, and `live-restore:true`-then-restart is PARKED as a separate host-side improvement for the operator to rule on, never folded into #401. Playbook `playbooks/corviduo-dev-docker-default-ulimits.yaml` (verify step 3 fails BY DESIGN until a restart). Hourly fd tripwire on corviduo-dev stays armed. **⏳ OWED: ping worldtree-dev in thread `01M08QQ655XD6VKEV7MA9GX0NS` once worldtree-personal recreates and 65536 is confirmed there.** Commit `7f3f265`.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user