ops(ana-gw): close out the Anaheim tunnel cipher question
The AES-GCM cutover was authorised and attempted, NH3 side first. It cannot be completed: UniFi's manual site-to-site IPsec implements no AEAD cipher. Eight GCM spellings were rejected with api.err.InvalidPayload while an otherwise identical body carrying aes256 returned rc:ok, which isolates the enum rather than the request as the cause. The accepted set is aes128/aes192/aes256/3des. Both Anaheim tunnels terminate on UniFi gateways, so this blocks the ESH tunnel on the same grounds. Measuring while testing also retires the premise. NH3's uplink is a 1 Gbps link, so Anaheim's 2 Gbps circuit was never the relevant ceiling, and the tunnel sustains 692 Mbit/s across 8 streams rather than the ~550 previously recorded from a 4-stream run. Compared with WireGuard over the same gateway and uplink, the deficit narrows from 2.3x at one stream to 15% at eight, so moving this link onto a different transport is not worth doing. The constraint is per-stream, around 245 Mbit/s, with both endpoints idle under load. Parallelising bulk transfers remains the mitigation and is worth 2.8x at no cost; NFS nconnect is the equivalent lever for single-stream mounts such as /mnt/smithy on ana-ml2. FortiGate phase2 for pfi-ana-nh3 keeps the widened proposal list, which is inert while the peer offers only CBC and avoids a further renegotiation. The UDM network object was diffed field-by-field against its pre-change snapshot and is unchanged.
This commit is contained in:
@@ -108,11 +108,11 @@ no longer deployed sidecars here. See Recent decisions.)
|
||||
(no NOPASSWD)** — stage model pulls to `/home`, not root-owned `/worktank`.
|
||||
## Current state / in-flight
|
||||
|
||||
_As of 2026-08-23 — a long multi-party ops session; open with the operator are the Anaheim tunnel cipher and how long WAN admin on ana-gw stays open._
|
||||
_As of 2026-08-23 — a long multi-party ops session. The Anaheim tunnel cipher is now settled (closed: the remedy does not exist). The one thing still open with the operator is how long WAN admin on ana-gw stays open._
|
||||
|
||||
- **🔓 ana-gw WAN admin is OPEN again as of 2026-08-23 — deliberately, and it should be re-closed after the FortiGate cutover.** Operator asked for it as the cutover contingency ("so I don't have to drive down there"). `https://ana-fw.phasefinal.com/` (wan1's own IP 38.120.12.42) serves the admin GUI on a browser-trusted LE cert; **https only**, and only the `infra-ops` account is WAN-reachable (`admin` remains 10.0.0.0/8-locked). Verified end-to-end with a real login from two sites. This partially reverses the 2026-08-12 hardening on a box still running the **EOL, actively-exploited FortiOS 7.2 branch** — so the exposure is CVE-shaped, and time-boxing it is the mitigation. Re-close = `config system interface / edit wan1 / set allowaccess ping / next / end`. Config backups pre+post at `/var/tmp/ana-gw-config-20260823T20*.conf` on nh3-dev. **Two follow-ons:** (a) ACME renewal for that cert needs **port 80** on wan1 — currently absent, next attempt ≈2026-09-27, expiry 2026-10-27; (b) ~5 SYN/s from `179.51.184.0/21` now draws SYN-ACKs, CPU impact nil, unmitigated by choice. Detail + the FortiOS trusthost gotchas: auto-memory `reference_fortigate_ana_gw_access`.
|
||||
|
||||
- **⏳ THE OPEN ITEM: Anaheim's IPsec tunnel delivers ~25% of a verified 2 Gbps circuit.** Circuit measured at **2,153 Mbit/s**; the tunnel ceilings ~550 Mbit/s aggregate, ~230 single-stream. FortiGate CPU **idle**, IPsec **NPU-offloaded**, interface error-free — so it is not crypto exhaustion. Both tunnels negotiate **`aes256-sha1`**; **AES-GCM is the proposed change** and the operator has signalled he will authorize it. Not executed: production edge, needs a matching change at NH3 + ESH, each tunnel drops during renegotiation. → `persistent-memory.d/2026-08-23-anaheim-ipsec-tunnel-ceiling.md`
|
||||
- **✅ CLOSED 2026-08-23: the Anaheim tunnel "problem" is mostly a measurement artefact, and the AES-GCM cutover is impossible.** Operator authorised the cutover; it was attempted NH3-side-first and **cannot be done — UniFi's manual site-to-site IPsec implements no AES-GCM** (8 spellings rejected `api.err.InvalidPayload` against a passing `aes256` control; accepted enum is `aes128/aes192/aes256/3des` only). This blocks the ESH tunnel too, since both far ends are UDMs. The framing was also wrong twice over: **NH3's uplink is 1 Gbps** (not Anaheim's 2 Gbps — that is the real ceiling), and the tunnel does **692 Mbit/s at 8 streams** (the original stopped at 4 and reported ~550). Against WireGuard on the same UDM and uplink, the gap collapses from 2.3× at one stream to **15% at eight** — so re-architecting onto WireGuard is not worth it. Real constraint = **per-stream ~245 Mbit/s**, both endpoints idle. **Standing mitigation: parallelise bulk transfers (2.8× for free); for single-stream NFS use `nconnect=N`** — the `/mnt/smithy` mount on ana-ml2 at 24.7 MB/s is exactly this case and is the obvious test. FortiGate phase2 `pfi-ana-nh3` was left widened to `aes256-sha1 aes256gcm` (inert while the peer offers only CBC); UDM verified byte-identical to its pre-change snapshot. → `persistent-memory.d/2026-08-23-anaheim-ipsec-tunnel-ceiling.md`
|
||||
|
||||
- **🟢 SEAT MAP (unchanged this session except selene).** **`gen`** = `orcarouter/Qwen3.8-27B-Uncensored` NVFP4-mixed, GPU0 :8015, **now 7 aliases** (see the collision note). **`char-rp`** = MeroMero-v2 dual-mode, GPU0 :8016, pinned `v0.26.0`. **`sec`/`sec-reasoning`** = M.O.G.-SEC on DFlash2, GPU1 :8019. **`selene` RETIRED** — 17.2 GiB reclaimed on GPU1 (free now ~19.4 GiB).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user