memory: snapshot — Anaheim tunnel at 25% of circuit; selene retired; hrafn CI fixed

Session captured for a context reset. Six new detail entries.

THE OPEN ITEM: Anaheim's IPsec tunnel delivers ~550 Mbit/s aggregate against a
circuit measured at 2,153 Mbit/s. Not WireGuard (it is IPsec on ana-gw), not
CPU (idle), not crypto exhaustion (NPU-offloaded), not the fibre. Both tunnels
negotiate aes256-sha1; AES-GCM proposed. Operator signalled authorization;
execution pending, untracked by operator choice.

Also recorded: selene retired after losing a head-to-head on its own job with
chat-judge moved to gen and the model name left to 404; the 7-alias collision
on the gen seat that makes cross-alias corroboration an echo; hrafn adopted and
its CI found to have been reporting green while deploying nothing for its whole
life; all three Worldtree instances de-armed from a 69-day-stale :latest and
the Matrix homeserver re-plumbed to personal; every secret-bearing .env on
ana-docker tightened to 0600; the pfi org closing the repo-creation half of the
credential-migration directive; booth kept-board deletion and link pruning.

Two entries under Tried and abandoned: the CI checkout assertion that broke the
pipeline twice and was removed, and my proposal to alias a retired model name
at a different model, which the operator correctly overruled.

Index 271 -> 282 lines, under the 300 cap, so no archival fired.

persistent-memory.d/2026-08-23-anaheim-ipsec-tunnel-ceiling.md

@@ -0,0 +1,71 @@
+# [2026-08-23] Anaheim's IPsec tunnel delivers ~25% of a verified 2 Gbps circuit
+
+The operator noticed site-to-site transfers were slow for a datacenter fiber
+handoff and asked whether WireGuard was the limit. It is not WireGuard, and the
+circuit is fine.
+
+## Measured
+
+```
+ana-docker -> internet, 8 parallel     2,153 Mbit/s   <- the 2 Gbps handoff, delivering
+ANA <-> NH3 through the tunnel, 4 par.   460 Mbit/s
+FortiGate's own recorded peak            554 Mbit/s
+ANA <-> NH3, single stream               227 Mbit/s
+ANA <-> ESH, single stream               249-265 Mbit/s
+ESH <-> NH3 (never touches ana-gw)       545-557 Mbit/s  on a SINGLE stream
+```
+
+Method: stdlib TCP probe (no ssh, no crypto, no compression) between site
+endpoints; raw circuit measured with 8 parallel HTTPS fetches from Hetzner
+Ashburn. Host NICs are virtio with no reported cap, so no host-side ceiling.
+
+## What it is not
+
+- **Not WireGuard.** Both Anaheim tunnels are IPsec on ana-gw
+  (`pfi-ana-nh3` -> 70.230.226.88, `ana-eshudm-dyn` -> the ESH UDM). WireGuard
+  on ana-wg is remote-access only and is not in this path. Traceroute confirms:
+  both slow paths have hop 1 = `10.250.50.1` (the FortiGate); the fast
+  ESH<->NH3 path rides a `192.168.x` Site Magic overlay and never touches it.
+- **Not CPU or crypto exhaustion.** FortiGate CPU was **100% idle across all
+  8 cores** during the tests, and both live tunnels report `npu_flag=03` with
+  `dec_npuid=1 enc_npuid=1` — encrypt *and* decrypt are hardware-offloaded.
+- **Not a 250 Mbit/s cap.** That was the first number and it is misleading —
+  single-stream TCP. Four parallel streams doubled it. Quote the aggregate.
+- **Not the interface.** wan1: `rxe=0 txe=0 rxd=0 txd=0`, no collisions.
+
+## Most likely cause
+
+Both tunnels negotiate **`aes256-sha1`** in phase 1 *and* phase 2 (dhgrp 14,
+IKEv2). AES-CBC + SHA1 is a two-pass operation; FortiGate NPUs are markedly
+faster on **AES-GCM**, which combines encryption and authentication in one
+pass. The datasheet IPsec headline for an 80F assumes GCM with large packets,
+not CBC+SHA1 at the 1438-byte tunnel MTU this link negotiates. The ~4x
+shortfall is consistent with that.
+
+## Not executed
+
+Changing the proposal is a **production-edge change requiring a matching
+change at the far end** (NH3 UDM and the ESH UDM), and each tunnel drops while
+it renegotiates. Left for the operator. See the index entry for authorization
+state.
+
+## Immediate mitigation, no config change
+
+Per-flow is the weak axis: a single stream over Site Magic gets 557 Mbit/s, a
+single stream through IPsec gets 227. **Anything moving bulk data across the
+Anaheim link should parallelise** — that alone roughly doubles throughput
+today.
+
+## Practical consequence already observed
+
+`/mnt/smithy` mounted on ana-ml2 reads at 24.7 MB/s sequential vs 98.3 MB/s
+from nh3-dev (same file, same mount) — that gap *is* this tunnel, not NFS and
+not the NAS. See [[2026-08-23-smithy-mount-ana-ml2]].
+
+## Access note
+
+ana-gw is a FortiGate-80F, FortiOS 7.2.10, at 10.250.0.1. `sshpass` is absent
+on nh3-dev; connect with paramiko via `uv run --with paramiko`. Password is
+vaulted at `fortigate/ana-gw-infra-ops-password`. **`diagnose vpn tunnel list`
+prints live ESP session keys** — never paste its output into althing, a
+booth, or a commit.