ops(ana-gw): port 80 on the WAN address is the FortiOS ACME listener

The previous note attributed the open port to an ISP transparent
proxy. That was wrong. It terminates on the FortiGate: system acme is
bound to wan1, and FortiOS opens port 80 there to answer HTTP-01
challenges regardless of what allowaccess permits, which is why the
port stays open with the interface set to ping only. Every
non-challenge request returns a fixed 403 whose body reads ACME Access
Only. No DNAT is involved; of the fourteen virtual IPs only two land
on that address, neither on port 80.

The wrong conclusion came from a sniffer filtered on dst host, which
matches inbound packets alone and so excluded the replies being looked
for. Filtered bidirectionally the box is plainly seen emitting SYN-ACK
on port 80. The note records the rule.

Two consequences follow. The earlier warning that certificate renewal
would fail without http in allowaccess is retracted, since FortiOS
opens the challenge port itself. And the listener is not an
administrative surface, though its value is now marginal with WAN
administration closed, so the note records how to remove it and leaves
that decision open.

Also captured: four virtual IPs are all-port static NAT and map every
port of their external address.
This commit is contained in:
vh
2026-08-23 15:53:55 -07:00
parent 7bc9754e40
commit f39b66d2e1
2 changed files with 49 additions and 1 deletions
+1 -1
View File
@@ -110,7 +110,7 @@ no longer deployed sidecars here. See Recent decisions.)
_As of 2026-08-23 — a long multi-party ops session. The Anaheim tunnel cipher is now settled (closed: the remedy does not exist). The one thing still open with the operator is how long WAN admin on ana-gw stays open._
- **🔒 ana-gw WAN admin is CLOSED again (2026-08-23, operator-directed) and the FortiGate is scheduled for replacement.** The cutover contingency window is over: `wan1 allowaccess` is back to **`ping` only** (https + ssh removed) and `infra-ops` trusthost is back to **10.0.0.0/8 only** — verified from two sites that 443 and 22 are closed, and that management still works over the tunnel at 10.250.0.1. **There is no longer any out-of-band path to ana-gw**; if both tunnels drop, it is console-only. Re-open = `set allowaccess ping https` on wan1 plus widening the infra-ops trusthost (both one-liners, recorded in auto-memory). ⚠ Port 80 on 38.120.12.42 answers a bare `403` from **outside** — that is an ISP transparent HTTP proxy on the CLIENT side (NH3 and ESH are both behind CGNAT), **not** the FortiGate: inbound :80 SYNs from scanner ranges reach the box and get **no SYN-ACK**, and our own :80 packets never appear in a capture at the box at all.
- **🔒 ana-gw WAN admin is CLOSED again (2026-08-23, operator-directed) and the FortiGate is scheduled for replacement.** The cutover contingency window is over: `wan1 allowaccess` is back to **`ping` only** (https + ssh removed) and `infra-ops` trusthost is back to **10.0.0.0/8 only** — verified from two sites that 443 and 22 are closed, and that management still works over the tunnel at 10.250.0.1. **There is no longer any out-of-band path to ana-gw**; if both tunnels drop, it is console-only. Re-open = `set allowaccess ping https` on wan1 plus widening the infra-ops trusthost (both one-liners, recorded in auto-memory). ⚠ **Port 80 on 38.120.12.42 is still open and it IS the FortiGate** — the FortiOS **ACME HTTP-01 challenge listener**, opened by `config system acme / set interface "wan1"`, which **bypasses `allowaccess` by design**. Body is literally `ACME Access Only` (403, 101 bytes) for every path; confirmed by sniffer showing `wan1 out 38.120.12.42.80 -> ...: syn ack`. Not a DNAT (only two VIPs exist on .42: Kokoro `:8880`, wg `:31337`) and not an admin surface — it serves only challenge tokens. Closing it = `config system acme` unset interface, which also ends cert renewal.
- **✅ CLOSED 2026-08-23: the Anaheim tunnel "problem" is mostly a measurement artefact, and the AES-GCM cutover is impossible.** Operator authorised the cutover; it was attempted NH3-side-first and **cannot be done — UniFi's manual site-to-site IPsec implements no AES-GCM** (8 spellings rejected `api.err.InvalidPayload` against a passing `aes256` control; accepted enum is `aes128/aes192/aes256/3des` only). This blocks the ESH tunnel too, since both far ends are UDMs. The framing was also wrong twice over: **NH3's uplink is 1 Gbps** (not Anaheim's 2 Gbps — that is the real ceiling), and the tunnel does **692 Mbit/s at 8 streams** (the original stopped at 4 and reported ~550). Against WireGuard on the same UDM and uplink, the gap collapses from 2.3× at one stream to **15% at eight** — so re-architecting onto WireGuard is not worth it. Real constraint = **per-stream ~245 Mbit/s**, both endpoints idle. **Standing mitigation: parallelise bulk transfers (2.8× for free); for single-stream NFS use `nconnect=N`** — the `/mnt/smithy` mount on ana-ml2 at 24.7 MB/s is exactly this case and is the obvious test. FortiGate phase2 `pfi-ana-nh3` was left widened to `aes256-sha1 aes256gcm` (inert while the peer offers only CBC); UDM verified byte-identical to its pre-change snapshot. → `persistent-memory.d/2026-08-23-anaheim-ipsec-tunnel-ceiling.md`