diff --git a/servers/nh3-dev/README.md b/servers/nh3-dev/README.md index 7dae013..ed04ce7 100644 --- a/servers/nh3-dev/README.md +++ b/servers/nh3-dev/README.md @@ -64,6 +64,24 @@ local Bash already executes here — no SSH-to-self needed for non-privileged wo (it existed only to satisfy the v1 CLI, which is now gone). Normalising the parent directory is **unresolved — operator's call**; `/opt/docker` itself is a separate three-way split (`755` root, `777` root on two hosts, `755` lkraven). +- **Reachable from the mesh at its LAN address — via a masquerade exception on + nh3-scale, not via anything on this box.** nh3-dev runs Tailscale, so it + carries the anti-spoof rule `-A ts-input -s 100.64.0.0/10 ! -i tailscale0 -j + DROP`. The fleet's subnet routers run `NoSNAT: true` with RFC1918 exempted + from masquerade (deliberate source preservation, a departure from Tailscale's + `--snat-subnet-routes=true` default), so a mesh client's packet arrived on + `ens18` still sourced `100.64.x` and was dropped there — silently. Every NH3 + host that does *not* run Tailscale was unaffected, which is what made it look + like a DNS or routing fault. Fixed 2026-09-14 by one rule in + `/usr/local/sbin/mesh-exit-masq.sh` on **nh3-scale (CT 107, nh3-pve)**: + `-d 10.100.10.50/32 -j MASQUERADE`, placed above the RFC1918 RETURNs. Copy + kept at `servers/nh3-pve/mesh-exit-masq.sh`. + ⚠ **Do not instead advertise `10.100.10.50/32` from nh3-dev.** Tried the same + day: it black-holed nh3-dev from ESH, Anaheim, FV and Irvine while leaving its + own LAN and the internet up. `ip rule` here puts `lookup 52` at priority 5270, + ahead of `main` at 32766, and becoming a subnet router let table 52 capture + cross-site traffic this node has no accepted route for (`RouteAll: false`). + A one-host check against its own LAN passes cleanly — test all four sites. - **ttyd fleet driver-seat** — web/iPad seat into the zellij `Claude` session (ttyd behind Caddy; OSC52 clipboard shim). User systemd services under `~/.config`. - **mead-hall** — Bifrost tool-provider sidecar (`:5173`), CI-deployed from diff --git a/servers/nh3-pve/mesh-exit-masq.sh b/servers/nh3-pve/mesh-exit-masq.sh new file mode 100644 index 0000000..2600ec5 --- /dev/null +++ b/servers/nh3-pve/mesh-exit-masq.sh @@ -0,0 +1,32 @@ +#!/bin/sh +# Masquerade mesh clients' INTERNET-bound (exit-node) traffic only; preserve site-to-site source. +iptables -t nat -F MESH-EXIT 2>/dev/null || iptables -t nat -N MESH-EXIT + +# ── Mesh-member hosts on this routed subnet — MUST come before the RFC1918 RETURNs ── +# A host that runs Tailscale itself installs an anti-spoof rule: +# -A ts-input -s 100.64.0.0/10 ! -i tailscale0 -j DROP +# With source preservation, a mesh client's packet reaches that host's ETHERNET +# interface still carrying its 100.64.x source, and is dropped there — silently, +# before anything can answer. Hosts that do NOT run Tailscale are unaffected, +# which is why every other NH3 address worked and only this one did not. +# Masquerading just these destinations makes them behave like every other host +# while leaving source preservation absolute for the rest of the subnet. +# (2026-09-14. Tailscale's own default is --snat-subnet-routes=true, i.e. SNAT +# everything; this file is the deliberate departure from that, so the exception +# belongs here rather than as a reason to abandon the design.) +# +# ⚠ Do NOT "fix" this instead by advertising the host's /32 from the host +# itself. That was tried on nh3-dev 2026-09-14 and black-holed it from ESH, +# Anaheim, FV and Irvine — `ip rule` there puts `lookup 52` at priority 5270, +# ahead of main at 32766, and becoming a subnet router let table 52 capture +# cross-site traffic the node had no accepted route for. Its own LAN and the +# internet kept working, so a narrow check looks clean. Fix it at the router. +iptables -t nat -A MESH-EXIT -d 10.100.10.50/32 -j MASQUERADE # nh3-dev + +iptables -t nat -A MESH-EXIT -d 10.0.0.0/8 -j RETURN +iptables -t nat -A MESH-EXIT -d 172.16.0.0/12 -j RETURN +iptables -t nat -A MESH-EXIT -d 192.168.0.0/16 -j RETURN +iptables -t nat -A MESH-EXIT -d 100.64.0.0/10 -j RETURN +iptables -t nat -A MESH-EXIT -j MASQUERADE +iptables -t nat -C POSTROUTING -s 100.64.0.0/10 -o eth0 -j MESH-EXIT 2>/dev/null \ + || iptables -t nat -A POSTROUTING -s 100.64.0.0/10 -o eth0 -j MESH-EXIT