fix(fv): extend the Tailscale SNAT rule to NH3, ESH and Irvine

FV could not reach any site but Anaheim. The cause was a single outbound-NAT
rule on the FV gateway, added 2026-09-13 and scoped to Anaheim only --
docs/runbooks/fv-to-ana-nat.md says so in as many words: "Other remote sites
remain outside this fix's scope." Three mirrors added, same interface and
source, only the destination differing: 10.100.0.0/16, 10.0.0.0/16 and
10.6.110.0/24.

After: fv-ml1 reaches NH3, ESH, Anaheim, Irvine, the mesh and the internet.
Regression sweep clean across nh3-dev, nh3-docker and esh-docker-vm.

The runbook now records what the failure looks like, because it presents as a
routing or Tailscale fault and is neither. fv-ml1 reached mesh addresses
perfectly and LAN addresses not at all; the FV firewall log showed the outbound
passing with src=10.251.50.54 and no reply returning; temporary counting rules
proved nh3-scale received 5 packets and sent 4 replies; both peers' AllowedIPs
were correct. The discriminator that settles it is that every other site pair
works -- nh3-docker to esh/ana/FV and esh-docker-vm to FV all succeed -- so a
general subnet-to-subnet limitation is ruled out and only outbound SNAT is
left.

Also reverts the remote-site MASQUERADE rules added to nh3-scale earlier on the
asymmetric-return theory. They fired but were not the fix, so they are removed
rather than left to accumulate as NAT that achieves nothing.

Applied via source_nat/add_rule + apply with a pre-change config backup taken
first. Source scope is still fv-ml1's /32, so a second FV host will hit this
again -- flagged in the runbook.

docs/runbooks/fv-to-ana-nat.md

@@ -17,7 +17,52 @@ One manual IPv4 rule:
 
 No routes, access/filter rules, WAN settings, subnet advertisements, host
 Tailscale settings, or other FV source addresses were changed. Other remote
-sites remain outside this fix's scope. Translation means Anaheim services may
+sites remain outside this fix's scope.
+
+## ⭐ EXTENDED 2026-09-15 — NH3, ESH and Irvine now have the same rule
+
+The Anaheim-only scope above was the whole reason FV could not reach anywhere
+else, and it cost an evening to rediscover. Three rules added, each an exact
+mirror of the ANA one — same interface, same source, only the destination
+differs:
+
+| destination | description |
+|---|---|
+| `10.100.0.0/16` | `fv-ml1 to NH3 via Tailscale SNAT` |
+| `10.0.0.0/16`   | `fv-ml1 to ESH via Tailscale SNAT` |
+| `10.6.110.0/24` | `fv-ml1 to IRV via Tailscale SNAT` |
+
+Applied through `POST /api/firewall/source_nat/add_rule` + `.../apply`, with a
+pre-change `core/backup/download/this` taken first. After: fv-ml1 reaches all
+four sites, the mesh and the internet; nh3-dev, nh3-docker and esh-docker-vm
+regression-swept clean.
+
+⚠ **The source is still `10.251.50.54/32` — fv-ml1 only.** A second host at FV
+will have exactly this problem and no symptom that points at NAT. Broaden to
+`10.251.50.0/24`, or add a rule per host, when FV grows.
+
+### What this looks like when it breaks, so the next person skips the evening
+
+The failure presents as a routing or Tailscale problem and is neither:
+
+- fv-ml1 reaches mesh addresses (`100.64.0.x`) perfectly and LAN addresses not
+  at all — which reads like a subnet-router fault.
+- The FV firewall log shows the outbound **passing** on tailscale0 with
+  `src=10.251.50.54` and no reply ever returning, so nothing looks blocked.
+- The far-side router genuinely receives and replies — verified with temporary
+  counting rules on nh3-scale: 5 packets in, 4 replies out.
+- Both peers' Tailscale `AllowedIPs` are correct, so cryptokey routing is fine.
+- **Every other site pair works**, which rules out a general subnet-to-subnet
+  limitation: `nh3-docker → esh/ana/FV` and `esh-docker-vm → FV` all succeed.
+
+That combination means outbound SNAT, not routing. Check
+`/api/firewall/source_nat/search_rule` for a rule covering the destination
+before investigating anything else.
+
+⚠ Also retired on the way: a set of remote-site MASQUERADE rules added to
+nh3-scale's `mesh-exit-masq.sh` on the asymmetric-return theory. They fired
+(counters incremented) but were not the fix, and were reverted rather than left
+to accumulate. Translation means Anaheim services may
 see the gateway identity rather than fv-ml1's original LAN address.
 
 ## Evidence and verification

servers/nh3-pve/mesh-exit-masq.sh

@@ -31,23 +31,3 @@ iptables -t nat -A MESH-EXIT -j MASQUERADE
 iptables -t nat -C POSTROUTING -s 100.64.0.0/10 -o eth0 -j MESH-EXIT 2>/dev/null \
   || iptables -t nat -A POSTROUTING -s 100.64.0.0/10 -o eth0 -j MESH-EXIT
 
-# ── Remote-SITE sources, not just mesh clients ────────────────────────────────
-# The jump above matches only 100.64.0.0/10, so traffic from another site's LAN
-# arriving over the mesh never enters MESH-EXIT and keeps its original source.
-# An NH3 host then replies via its own LAN router instead of back through this
-# node, the path is asymmetric, and the reply is lost. Measured 2026-09-15:
-# fv-ml1 reached 100.64.0.1 and 100.64.0.4 fine while 10.100.50.40 failed
-# outright, and the FV firewall log showed the outbound passing with
-# src=10.251.50.54 and nothing ever coming back.
-#
-# Masquerading remote-site sources onto this node's LAN address makes the reply
-# return here, where the conntrack state lives. It costs source visibility for
-# cross-site traffic on the NH3 LAN — the same trade as the nh3-dev rule above,
-# and the alternative is no connectivity at all.
-#
-# ⚠ Sites are listed explicitly rather than using 10.0.0.0/8: a blanket rule
-# would also masquerade NH3-local traffic that has no business being rewritten.
-for site_net in 10.251.0.0/16 10.0.0.0/16 10.250.0.0/16 10.6.110.0/24; do
-  iptables -t nat -C POSTROUTING -s "$site_net" -o eth0 -j MASQUERADE 2>/dev/null \
-    || iptables -t nat -A POSTROUTING -s "$site_net" -o eth0 -j MASQUERADE
-done