feat(mesh): retire nh3-dev from the mesh and revert the masquerade it required

nh3-dev sits on the NH3 LAN and reaches every site through its own default
gateway; RouteAll was already false, so it never used the tunnel for routing.
Membership bought a 100.64.0.4 address nothing referenced -- grep across the
repo and ~/development found only docs and memory hits.

It also cost something concrete. A host running Tailscale installs
-A ts-input -s 100.64.0.0/10 ! -i tailscale0 -j DROP, and because the fleet's
subnet routers preserve source rather than masquerading RFC1918, a mesh client's
packet reached nh3-dev's ens18 still sourced 100.64.x and was dropped silently.
That is why nh3-dev.nh3.internal failed from the mesh while every NH3 host that
does not run Tailscale worked, and it needed a -d 10.100.10.50/32 -j MASQUERADE
exception on nh3-scale to paper over. Retiring the membership removed the
anti-spoof rule, so the exception went with it -- mesh-exit-masq.sh is back to
the two rules it had before yesterday.

Verified after: nh3-dev reachable at 10.100.10.50 from ESH, Anaheim, FV, Irvine
and NH3, and reaching all four sites plus the internet itself. fv-ml1
unaffected.

The mesh is now six nodes and every one has a job: three site routers,
vb-gateway, irv-ml1 (Irvine's own router, no separate scale node), and the
operator's MacBook Air. Nothing is enrolled just in case.
This commit is contained in:
vh
2026-09-15 01:05:23 -07:00
parent 959a743256
commit a65cdf65d9
3 changed files with 17 additions and 44 deletions
+12 -18
View File
@@ -64,24 +64,18 @@ local Bash already executes here — no SSH-to-self needed for non-privileged wo
(it existed only to satisfy the v1 CLI, which is now gone). Normalising the
parent directory is **unresolved — operator's call**; `/opt/docker` itself is a separate three-way split (`755` root, `777`
root on two hosts, `755` lkraven).
- **Reachable from the mesh at its LAN address — via a masquerade exception on
nh3-scale, not via anything on this box.** nh3-dev runs Tailscale, so it
carries the anti-spoof rule `-A ts-input -s 100.64.0.0/10 ! -i tailscale0 -j
DROP`. The fleet's subnet routers run `NoSNAT: true` with RFC1918 exempted
from masquerade (deliberate source preservation, a departure from Tailscale's
`--snat-subnet-routes=true` default), so a mesh client's packet arrived on
`ens18` still sourced `100.64.x` and was dropped there — silently. Every NH3
host that does *not* run Tailscale was unaffected, which is what made it look
like a DNS or routing fault. Fixed 2026-09-14 by one rule in
`/usr/local/sbin/mesh-exit-masq.sh` on **nh3-scale (CT 107, nh3-pve)**:
`-d 10.100.10.50/32 -j MASQUERADE`, placed above the RFC1918 RETURNs. Copy
kept at `servers/nh3-pve/mesh-exit-masq.sh`.
⚠ **Do not instead advertise `10.100.10.50/32` from nh3-dev.** Tried the same
day: it black-holed nh3-dev from ESH, Anaheim, FV and Irvine while leaving its
own LAN and the internet up. `ip rule` here puts `lookup 52` at priority 5270,
ahead of `main` at 32766, and becoming a subnet router let table 52 capture
cross-site traffic this node has no accepted route for (`RouteAll: false`).
A one-host check against its own LAN passes cleanly — test all four sites.
- **NOT a mesh node** (retired 2026-09-15). nh3-dev sits on the NH3 LAN and reaches
every site through its own default gateway — `RouteAll: false` meant it never used
the tunnel for routing anyway, so membership bought only a `100.64.0.4` address
nothing referenced. ⭐ It also *cost* something: a host running Tailscale installs
`-A ts-input -s 100.64.0.0/10 ! -i tailscale0 -j DROP`, and because the fleet's
subnet routers preserve source rather than masquerading RFC1918, a mesh client's
packet reached `ens18` still sourced `100.64.x` and was silently dropped — which is
why `nh3-dev.nh3.internal` failed from the mesh while every non-Tailscale NH3 host
worked. That needed a `-d 10.100.10.50/32 -j MASQUERADE` exception on nh3-scale;
retiring the membership removed the anti-spoof rule and the exception with it.
Verified after: reachable at `10.100.10.50` from ESH, Anaheim, FV, Irvine and NH3,
and reaching all four sites plus the internet itself.
- **ttyd fleet driver-seat** — web/iPad seat into the zellij `Claude` session (ttyd
behind Caddy; OSC52 clipboard shim). User systemd services under `~/.config`.
- **mead-hall** — Bifrost tool-provider sidecar (`:5173`), CI-deployed from