diff --git a/persistent-memory.d/2026-08-17-fleet-ipv6-mesh.md b/persistent-memory.d/2026-08-17-fleet-ipv6-mesh.md index 188c5ef..9bbe06d 100644 --- a/persistent-memory.d/2026-08-17-fleet-ipv6-mesh.md +++ b/persistent-memory.d/2026-08-17-fleet-ipv6-mesh.md @@ -165,3 +165,21 @@ looks like BGW rationing rather than a provisioning-profile limit. Live state at correction time: `wan_dhcpv6_pd_size: 64`, `wan1 v6 2600:1700:b25:c110::48`, all 5 NH3 LANs still `ipv6_interface_type: none`. + +### ⛔ CLOSED 2026-08-24 — operator ruling, do not re-raise + +The seven unclaimed `/64`s stay unclaimed. Two facts close it: + +- **The BGW has no IP-passthrough mode.** Operator confirmed, and we hold admin + on it — so the cheap path (let the UDM take the `/60` directly and carve it + natively, as it already does at ESH) does not exist here. +- **The only remaining route is a multi-DUID DHCPv6 client on a VM**, which + requires re-cabling to reach the BGW's DHCPv6 server, split-stack routing + (UDM for v4, VM for v6), and — the actual cost — **rebuilding the whole IPv6 + firewall policy in nftables on that VM**, because routing v6 around the UDM + bypasses its zone firewall entirely and would leave every LAN host globally + reachable. + +Operator's call: not worth it. **NH3 LANs stay `ipv6_interface_type: none`.** +Do not re-propose on the strength of "there are seven free prefixes" — the +prefixes are real, the firewall rebuild is why nobody wants them. diff --git a/persistent-memory.md b/persistent-memory.md index 17f4886..28a7a84 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -166,7 +166,7 @@ no longer deployed sidecars here. See Recent decisions.) - `[2026-08-17]` **Gen seat swapped to `absolute-heresy` — and the three bugs the swap exposed are worth more than the swap.** Candidate `MuXodious/Qwen3.8-27B-absolute-heresy` (Heretic v1.4.0 + SOMPOA, T377) beat the incumbent on refusals AND KL simultaneously, which is the unusual part — those normally trade off. Validated on the probe port per operator ruling, promoted, all 7 aliases green. **Durable lessons banked:** (1) **A CPU-only MTP head hash can replace the ~56 GB bf16 acceptance gate.** The `Qwen3_5ForConditionalGeneration` wrapper never loads the MTP head, so PEFT merges / Heretic runs / llm-compressor passes all leave `mtp.*` pristine — hashing it against a head we have already measured (the incumbent's, 47.7%) answers the question for free. Predicted 47.7%, measured 47.2%. Saved downing meromero. Tool: `services/gen-seat-mixed-quant/compare_mtp_head.py` (hash bf16 via **uint8 reinterpret** — numpy has no bfloat16). (2) **`post_quant.py` assumed a standalone `model-mtp.safetensors`**; a full checkpoint keeps `mtp.*` in a NUMBERED shard, so the copy silently no-op'd while the index was still rewritten to point at a file that never existed — 15 unresolvable tensors behind a correct-looking tensor count. Its own FAILED-CHECKS assertion caught it; **that is why the check exists rather than an assumption**. Fixed to extract. (3) **A probe that does not mirror the live seat manufactures failures.** `serve_probe.sh` hardcoded `:latest` (seat is a pinned nightly for #51113), had no tool-call/reasoning parsers, and its `--speculative-config` JSON died twice on quoting — **bash BRACE-EXPANDS `{"a":1,"b":2}` on the comma** unless single-quoted at the REMOTE shell. Adding the seat's flags took the surface test from 5/6 to **6/6**; the "tool calling broken" result was pure probe config. Commits `7997f11`,`254c588`,`2c36028`,`b0c2d3d`,`993421b`. -- `[2026-08-17]` **Fleet IPv6 mapped + the real VPN topology verified; the driver is CGNAT at ESH, not the WireGuard mesh.** New ESH fiber (installing 2026-08-18) lands the house behind **CGNAT**, which breaks **Site Magic** (NH3↔ESH `sdwan-mesh-tunnel`) on IPv4 — so IPv6 becomes load-bearing as the escape hatch, and that is its most likely first consumer. Topology as VERIFIED (a prior turn assumed wrong and was corrected): UniFi↔UniFi = **Site Magic**; colo↔UniFi = **IPsec IKEv2** (`pfi-ana-nh3` 158M/165M pkt = the workhorse, `ana-to-eshudm`); **WireGuard is an RA convention only, host-based on `ana-wg`** UDP 31337 behind a FortiGate VIP — the FortiGate never terminates WG (FortiOS 7.2 has none; 7.4 added it) so "upgrade the edge for WireGuard" is a **non-problem, do not re-derive**. IPv6 today: **NH3 WAN live** `2600:1700:b25:c110::48`, **colo none**, **ESH none**. **AT&T delegates exactly ONE /64** (`2600:1700:b25:c11f::/64`) — proven by forcing prefix-ID auto→`0` and watching the subnet NOT move, because the `c110`/`c11f` pattern otherwise reads convincingly as a /60. A mesh needs a routable **WAN** address, **not** PD. `ana-wg`'s WG socket is **already dual-stack** (`[::]:31337`) → v6 RA needs an address + a v6 port-forward, no WG reconfig. ⚠ UDM legacy `rest/firewallrule` returns **0 rules** (zone-based firewall) — use `v2/…/firewall-policies`; inbound v6 is default-deny and held. All three endpoints will be **dynamic** → extend the existing hostname pattern (`ana-fw`/`nh3.phasefinal.com`) to **AAAA**. Enabled PD on `nh3-iot` to measure, **reverted on operator instruction** (all 5 LANs back to `none`, verified). Also fixed: **`ana-wg` WireGuard key material was world-readable** (`wg0.conf` + `keys/*_priv` + `*_psk` + client `configs/*.conf` at 644) → now 600, dirs 700, service untouched. Detail → `persistent-memory.d/2026-08-17-fleet-ipv6-mesh.md`. +- `[2026-08-17]` **Fleet IPv6 mapped + the real VPN topology verified; the driver is CGNAT at ESH, not the WireGuard mesh.** New ESH fiber (installing 2026-08-18) lands the house behind **CGNAT**, which breaks **Site Magic** (NH3↔ESH `sdwan-mesh-tunnel`) on IPv4 — so IPv6 becomes load-bearing as the escape hatch, and that is its most likely first consumer. Topology as VERIFIED (a prior turn assumed wrong and was corrected): UniFi↔UniFi = **Site Magic**; colo↔UniFi = **IPsec IKEv2** (`pfi-ana-nh3` 158M/165M pkt = the workhorse, `ana-to-eshudm`); **WireGuard is an RA convention only, host-based on `ana-wg`** UDP 31337 behind a FortiGate VIP — the FortiGate never terminates WG (FortiOS 7.2 has none; 7.4 added it) so "upgrade the edge for WireGuard" is a **non-problem, do not re-derive**. IPv6 today: **NH3 WAN live** `2600:1700:b25:c110::48`, **colo none**, **ESH none**. **AT&T delegates one /64 PER REQUEST** (`2600:1700:b25:c11f::/64`) — and the BGW holds the whole `/60`, rationing `c118`–`c11f` one at a time while keeping `c110`–`c117`. So eight /64s exist; UniFi just solicits once. ⛔ **CLOSED 2026-08-24 — operator ruling, do not re-raise:** the BGW has **no IP-passthrough** (operator confirmed, and we have admin on it), so the only route to the other seven is a multi-DUID DHCPv6 client on a VM — which means split-stack routing and rebuilding the entire v6 firewall policy off the UDM. Juice not worth the squeeze. NH3 LANs stay v6-off. A mesh needs a routable **WAN** address, **not** PD. `ana-wg`'s WG socket is **already dual-stack** (`[::]:31337`) → v6 RA needs an address + a v6 port-forward, no WG reconfig. ⚠ UDM legacy `rest/firewallrule` returns **0 rules** (zone-based firewall) — use `v2/…/firewall-policies`; inbound v6 is default-deny and held. All three endpoints will be **dynamic** → extend the existing hostname pattern (`ana-fw`/`nh3.phasefinal.com`) to **AAAA**. Enabled PD on `nh3-iot` to measure, **reverted on operator instruction** (all 5 LANs back to `none`, verified). Also fixed: **`ana-wg` WireGuard key material was world-readable** (`wg0.conf` + `keys/*_priv` + `*_psk` + client `configs/*.conf` at 644) → now 600, dirs 700, service untouched. Detail → `persistent-memory.d/2026-08-17-fleet-ipv6-mesh.md`. - `[2026-08-17]` **Gen-seat multi-day degeneration RESOLVED — two compounding real causes, not one; the meta-lesson is "a mitigation that HELPS but doesn't FIX means a second cause, not a wrong one."** vLLM `qwen3_5_mtp`×GDN bug (#51113, real, fixed by nightly) + AEON full-W4A4 being lowest-fidelity (W4A4