From 1f504ce05a952b6d3c848abe222620fba6f252ee Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Sat, 5 Sep 2026 21:49:47 -0700 Subject: [PATCH] docs(pfi): refresh pfi-pve guest inventory from live state; headscale mesh plan --- docs/pfi/headscale-mesh-plan.md | 114 ++++++++++++ docs/pfi/proxmox-vms.md | 320 +++++++------------------------- 2 files changed, 182 insertions(+), 252 deletions(-) create mode 100644 docs/pfi/headscale-mesh-plan.md diff --git a/docs/pfi/headscale-mesh-plan.md b/docs/pfi/headscale-mesh-plan.md new file mode 100644 index 0000000..c872376 --- /dev/null +++ b/docs/pfi/headscale-mesh-plan.md @@ -0,0 +1,114 @@ +# Headscale overlay mesh — plan (2026-09-06) + +**Status:** planning. Operator direction 2026-09-06: replace Site Magic (NH3↔ESH) and the +FortiGate IPsec tunnels (colo↔NH3, colo↔ESH) with a self-hosted Tailscale-protocol overlay +(Headscale), one ultra-light LXC per PVE host, keeping the old tunnels as an emergency +backup. This document is the plan that a `.contract.md` will be cut from; nothing here is +provisioned yet. + +## Why Headscale (decided 2026-09-06) + +Tailscale clients are the best in class on the devices actually used for remote access +(iPad, Mac, Linux); the future OPNsense colo edge can be a Tailscale node natively +(`os-tailscale`); Headscale is one binary + one SQLite file to self-host, versus NetBird's +six-service stack with a mandatory IdP. NetBird's built-in UI and HA routing groups are +real but not decisive for three sites. UDP-blocked networks (in-flight Wi-Fi) work because +the client falls back to a relay over TCP 443; plain WireGuard (ana-wg) cannot. + +## Site facts that drive placement + +| site | edge | v4 | v6 | change risk | +|---|---|---|---|---| +| ANA colo | FortiGate 80F → **OPNsense on R420 within the month** | public static | pending | HIGH — edge in flux; incident history 2026 (breaker, PSU1, WAN admin closed) | +| NH3 | UDM SE, stable | dynamic-but-stable (DDNS) | single /64, **reserved for meshing** | low | +| ESH | UDM Pro Max | CGNAT now, **static soon**; 2G symmetric soon | /56 | low, improving | + +**Control plane at NH3 now** (only stable edge; static v6 + DDNS v4 → one DNS name). +**Reassess ESH** once its static v4 lands — it becomes the relay site regardless (2G +symmetric). **Never the colo:** the mesh's brain must not live in the building it exists +to reach in an emergency. Migration later = copy one SQLite file + move the DNS name; +clients follow the URL and never re-enrol. + +## Topology + +``` + headscale.phasefinal.com (A via DDNS, AAAA static) + │ HTTPS 443 (+ DERP later) + ┌─────────────────────────────┼───────────────────────────────┐ + │ NH3 nh3-pve │ │ + │ CT nh3-headscale ──control plane (1 core / 512M / 8G) │ + │ CT nh3-mesh-rtr ──subnet router, advertises 10.100.0.0/16│ + ├──────────────────────────────────────────────────────────────┤ + │ ESH esh-pve │ + │ CT esh-mesh-rtr ──subnet router, advertises 10.0.0.0/16 │ (relay/DERP here later) + ├──────────────────────────────────────────────────────────────┤ + │ ANA pfi-pve │ + │ CT ana-mesh-rtr ──subnet router, advertises 10.250.0.0/16│ (OPNsense takes this over later) + └──────────────────────────────────────────────────────────────┘ + Per-device clients: nh3-dev, laptops/iPad, corviduo-dev, ana-ml2 … (MagicDNS names) +``` + +Four unprivileged Debian 13 LXCs, each 1 vCPU / 512 MB / 8 GB, `onboot=1`, backed up by +the existing `pbs-ana` job. Routers need `/dev/net/tun` passed in and `ip_forward` (both +routine for unprivileged CTs; net sysctls are namespaced). + +Proposed ids/names (DHCP with reservation like every existing CT; recorded in +`dns/internal.yaml` as `..internal`): + +| PVE | next id | CT | note | +|---|---|---|---| +| nh3-pve (`root@10.100.250.60`, PVE 8.4.1) | 106 | `nh3-headscale` | control plane | +| nh3-pve | 107 | `nh3-mesh-rtr` | subnet router | +| esh-pve (`root@10.0.250.35`, PVE 8.4.20) | 108 | `esh-mesh-rtr` | subnet router | +| pfi-pve (`root@10.250.250.31`, PVE 8.3.5) | 114 | `ana-mesh-rtr` | subnet router; ana-wg (113) stays as independent WG fallback | + +`debian-13-standard_13.6-1_amd64.tar.zst` is in `pveam available` on all three (not yet +downloaded on any). + +## Access & credentials — all in hand (verified 2026-09-06) + +| need | have | +|---|---| +| provision LXCs on all three PVEs | `ssh root@` works on pfi-pve, nh3-pve, esh-pve (`infra-ops@` is refused on all three PVE hosts) | +| public DNS name + DDNS | Cloudflare all-zones DNS-edit token, vault `nh3-dev/.config/cloudflare/infra-ops-dns-token` | +| NH3 UDM port-forward 443 → nh3-headscale; static routes on both UDMs | UDM API keys, vault `unifi/pfi-udmse-api-key`, `unifi/esh-udmpm-api-key` (classic `/rest/*` read+write) | +| colo static route toward ana-mesh-rtr | FortiGate infra-ops SSH pw vaulted; reachable at 10.250.0.1 via the tunnel (`execute backup config` first). Moot once OPNsense lands | +| TLS for headscale | Let's Encrypt via Cloudflare DNS-01 (same token) — no inbound 80 needed | +| secrets (pre-auth keys, API key, DB) | vault under `nh3-headscale/…` | + +Nothing outstanding on credentials. Inputs still needed from the operator: confirm names/ids +above; 443 direct on the UDM vs behind the existing Caddy on nh3-dev; ACL posture (flat +"everything can reach everything" first, tighten later, is the recommendation). + +## Phases + +0. **Pre-flight (no changes):** confirm NH3 v6 /64 address for the AAAA; confirm DHCP + reservation ranges on the three sites; `pveam download` Debian 13 on all three. +1. **Control plane:** CT `nh3-headscale`; headscale in a container or the .deb (prefer .deb — + fewer layers in a 512M CT); Caddy/own TLS via DNS-01; UDM forward 443; DDNS updater + (UDM → Cloudflare, or a `ddclient`/cron in the CT); DERP = Tailscale public map + initially; headplane UI optional. Backup: `/var/lib/headscale/db.sqlite` is the only + state — PBS covers the CT; add a nightly `sqlite3 .backup` to `backupStore` too. +2. **First nodes:** nh3-dev + the operator's laptop/iPad enrol → prove MagicDNS, prove + TCP-443 relay path from a UDP-blocked network. +3. **Subnet routers:** three router CTs, `--advertise-routes` per site, approve routes in + headscale, SNAT off (`--snat-subnet-routes=false`) so source IPs survive, static routes + on each site gateway pointing the other two /16s at the local router. Test + site-to-site from clientless hosts. +4. **Cut over:** move day-to-day traffic onto the mesh; **disable (not delete)** Site Magic + and the two IPsec tunnels. Running all three at once makes route precedence murky on + the UDM and produces asymmetric paths. ana-wg stays as the out-of-band WG fallback. +5. **Later:** relay (DERP) at ESH when the 2G circuit is in; OPNsense as the colo router + node; HA router pairs (Tailscale HA subnet routers) per site; tighten ACLs. + +## Risks / open questions + +- Control-plane outage does not drop existing tunnels but stalls new logins and key + rotation — hence the SQLite backup and the "not at the colo" rule. +- Tailscale public DERPs carry relayed (encrypted) traffic through third-party infra until + a self-hosted DERP exists. Acceptable at first; fix in phase 5. +- CGNAT at ESH: outbound-only is fine; nothing at ESH needs to be dialled inbound. +- Headscale lags Tailscale feature-wise (tailnet lock, some ACL syntax). Not relevant to + phases 0–4. +- VM 106 on pfi-pve was once named "PFI-Tailscale" (pre-2026 inventory). No config + survives; nothing to reuse. diff --git a/docs/pfi/proxmox-vms.md b/docs/pfi/proxmox-vms.md index 25c24fc..cf8df6a 100644 --- a/docs/pfi/proxmox-vms.md +++ b/docs/pfi/proxmox-vms.md @@ -1,262 +1,78 @@ -# PFI-ANA Proxmox VM Inventory +# PFI-ANA Proxmox guest inventory (pfi-pve) -**Hypervisor**: Proxmox VE at `10.250.250.31:8006` -**Storage Pool**: `ospool` (CEPH/zfs — all VM disks reside here) -**Network Bridge**: `vmbr0` with VLAN tag `50` on all VMs -**QEMU Version**: 7.2.0 (primary), VM 106 on 8.1.5 +_Refreshed 2026-09-06 04:47Z from live `qm list`, `pct list`, `/etc/pve/*/N.conf` and +guest-agent IPs. Earlier versions of this file described guests that no longer exist +(VM 100 as the TrueNAS box, VM 106 as "PFI-Tailscale", VM 108 "PFI-ANA--DEV"); this +version is the live state._ -## Virtual Machines +**Hypervisor:** `pfi-pve` — Dell PowerEdge R750xs, Xeon Silver 4310 (48 threads), 188 GB, +Proxmox VE 8.3.5, `https://10.250.250.31:8006`, `ssh root@10.250.250.31` +(`infra-ops` is refused here). iDRAC `https://10.250.250.30`. -### VM 100 — PFI-ANA-TRUENAS +**Storage:** -| Property | Value | -|---|---| -| **VMID** | 100 | -| **OS Type** | Linux (l26) | -| **CPU** | 2 sockets × 2 cores = 4 vCPU (host passthrough) | -| **Memory** | 8,196 MB | -| **Disk** | `scsi0`: 80G on ospool | -| **CDROM** | `ide2`: TrueNAS-SCALE-22.12.1.iso _(original install media; current OS is Debian 12)_ | -| **Network** | `net0`: virtio, MAC `9A:90:79:7A:86:87`, vmbr0, VLAN 50 | -| **Boot** | scsi0 → ide2 → net0 | -| **Startup** | Order 2, delay 120s | -| **Onboot** | No | - -**Purpose**: NAS/NFS/SMB storage for the colo environment. - -**Current OS**: Debian 12 (bookworm), reprovisioned from the original TrueNAS SCALE install — SSH banner `OpenSSH_9.2p1 Debian-2+deb12u9`, no web UI on 80/443, no TrueNAS middleware. Shares are hand-managed via `/etc/exports` and `/etc/samba/smb.conf`. Mount paths kept the `/mnt//…` shape of the original ZFS layout for host compatibility. - ---- - -### VM 101 — PFI-ANA-DC - -| Property | Value | -|---|---| -| **VMID** | 101 | -| **OS Type** | Windows 11 | -| **BIOS** | OVMF (UEFI) with TPM 2.0 | -| **Machine** | pc-q35-7.2 | -| **CPU** | 2 sockets × 6 cores = 12 vCPU (host passthrough) | -| **Memory** | 24,576 MB | -| **Disk** | `scsi0`: 240G on ospool | -| **EFI Disk** | `efidisk0`: 1M on ospool | -| **TPM** | `tpmstate0`: 4M, v2.0 on ospool | -| **CDROM** | `scsi1`: virtio-win-0.1.229.iso (VirtIO drivers) | -| **Network** | `net0`: e1000, MAC `CE:C8:D7:FE:32:40`, vmbr0, VLAN 50 | -| **Boot** | scsi0 → net0 → ide0 → scsi1 | -| **Startup** | Order 5, delay 120s | -| **Onboot** | Yes | - -**Purpose**: Windows Domain Controller for the Anaheim environment. UEFI with TPM 2.0 suggests Active Directory / Group Policy services. - ---- - -### VM 102 — PFI-ANA-Docker - -| Property | Value | -|---|---| -| **VMID** | 102 | -| **OS Type** | Linux (l26) | -| **CPU** | 2 sockets × 4 cores = 8 vCPU (x86-64-v2-AES) | -| **Memory** | 16,384 MB | -| **Disk** | `scsi0`: 250G on ospool | -| **Network** | `net0`: virtio, MAC `BA:AF:E7:E9:79:23`, vmbr0, VLAN 50 | -| **Boot** | scsi0 → net0 → scsi1 | -| **Startup** | Order 4 | -| **Onboot** | Yes | - -**Purpose**: Primary Docker host for the colo. Runs Dockge for compose management and Traefik for reverse proxy. All Docker services documented in [docker-stack.md](docker-stack.md) run here. - ---- - -### VM 103 — PFI-SlaveBot - -| Property | Value | -|---|---| -| **VMID** | 103 | -| **OS Type** | Windows 10 | -| **Machine** | pc-i440fx-7.2 | -| **CPU** | 2 sockets × 4 cores = 8 vCPU (host passthrough) | -| **Memory** | 8,192 MB | -| **Disk** | `ide0`: 256G on ospool | -| **Network** | `net0`: e1000, MAC `CE:F0:49:C9:03:70`, vmbr0, VLAN 50 | -| **Boot** | ide0 → net0 → scsi0 | -| **Startup** | Not configured | -| **Onboot** | Yes | - -**Purpose**: Windows 10 workstation/bot. Likely a task automation or RDP-accessible machine. - ---- - -### VM 104 — PFI-Mongo - -| Property | Value | -|---|---| -| **VMID** | 104 | -| **OS Type** | Linux (l26) | -| **CPU** | 2 sockets × 4 cores = 8 vCPU (host passthrough) | -| **Memory** | 8,196 MB | -| **Disk** | `scsi0`: 256G on ospool | -| **Network** | `net0`: virtio, MAC `32:57:90:B2:66:61`, vmbr0, VLAN 50 | -| **Boot** | scsi0 → ide2 → net0 | -| **Startup** | Order 3, delay 60s | -| **Onboot** | Yes | - -**Purpose**: MongoDB server. Config file contains a connection string reference: `mongodb://10.250.50.81:27017/`. - ---- - -### VM 105 — PFI-Postgres - -| Property | Value | -|---|---| -| **VMID** | 105 | -| **OS Type** | Linux (l26) | -| **CPU** | 4 sockets × 4 cores = 16 vCPU | -| **Memory** | 8,196 MB | -| **Disk** | `scsi0`: 80G on ospool | -| **CDROM** | `ide2`: debian-11.6.0-amd64-netinst.iso | -| **Network** | `net0`: virtio, MAC `C2:1F:CC:71:66:D0`, vmbr0, VLAN 50 | -| **Serial** | `serial0`: socket (IPMI/serial console) | -| **Boot** | scsi0 → ide2 → net0 | -| **Startup** | Order 3, delay 60s | -| **Onboot** | Yes | - -**Purpose**: PostgreSQL database server running Debian 11. - ---- - -### VM 106 — PFI-Tailscale - -| Property | Value | -|---|---| -| **VMID** | 106 | -| **OS Type** | Linux (l26) | -| **CPU** | 2 sockets × 4 cores = 8 vCPU (x86-64-v2-AES) | -| **Memory** | 2,048 MB | -| **Disk** | `scsi0`: 256G on ospool | -| **CDROM** | `ide2`: debian-12.2.0-amd64-netinst.iso | -| **Network** | `net0`: virtio, MAC `BC:24:11:D7:E9:52`, vmbr0, VLAN 50 | -| **Boot** | scsi0 → ide2 → net0 | -| **Startup** | Not configured | -| **Onboot** | Yes | - -**Purpose**: Tailscale VPN node for mesh connectivity. Provides the VPN tunnel endpoints that link the three PFI sites together. Running Debian 12 (newer than most other VMs). Lightweight at 2G RAM. - ---- - -### VM 107 — PFI-Pteradactyl - -| Property | Value | -|---|---| -| **VMID** | 107 | -| **OS Type** | Linux (l26) | -| **CPU** | 2 sockets × 4 cores = 8 vCPU (host passthrough) | -| **Memory** | 8,192 MB | -| **Disk** | `scsi0`: 256G on ospool | -| **CDROM** | `ide2`: debian-11.6.0-amd64-netinst.iso | -| **Network** | `net0`: virtio, MAC `CA:44:37:8A:BF:E0`, vmbr0, VLAN 50 | -| **Boot** | scsi0 → ide2 → net0 | -| **Startup** | Not configured | -| **Onboot** | Yes | - -**Purpose**: Pterodactyl game server panel. Manages game server instances. - ---- - -### VM 108 — PFI-ANA--DEV - -| Property | Value | -|---|---| -| **VMID** | 108 | -| **OS Type** | Linux (l26) | -| **CPU** | 2 sockets × 4 cores = 8 vCPU | -| **Memory** | 8,192 MB | -| **Disk** | `scsi0`: 120G on ospool | -| **CDROM** | `ide2`: debian-11.6.0-amd64-netinst.iso | -| **Network** | `net0`: virtio, MAC `F2:EF:82:2C:AF:90`, vmbr0, VLAN 50 | -| **Boot** | scsi0 → ide2 → net0 | -| **Startup** | Order 10, delay 60s | -| **Onboot** | No | - -**Purpose**: Development environment. Not set to auto-boot, starts after core infrastructure (order 10). - ---- - -### VM 110 — PFI-ANA-Webhost - -| Property | Value | -|---|---| -| **VMID** | 110 | -| **OS Type** | Linux (l26) | -| **CPU** | 4 sockets × 4 cores = 16 vCPU | -| **Memory** | 4,096 MB (balloon: 1024 MB minimum) | -| **Disk** | `scsi0`: 250G on ospool | -| **CDROM** | `ide2`: debian-11.6.0-amd64-netinst.iso | -| **Network** | `net0`: virtio, MAC `E6:F9:3A:C9:61:2A`, vmbr0, VLAN 50 | -| **Boot** | scsi0 → ide2 → net0 | -| **Startup** | Order 30, up 120s, down 120s | -| **Onboot** | Yes | - -**Purpose**: Web hosting server. Highest startup order (30) — starts last. Also has the longest graceful shutdown timeout (120s). Memory ballooning enabled for dynamic allocation. - ---- - -### VM 111 — pfi-tacticalrmm - -| Property | Value | -|---|---| -| **VMID** | 111 | -| **OS Type** | Linux (l26) | -| **CPU** | 4 sockets × 4 cores = 16 vCPU | -| **Memory** | 8,192 MB | -| **Disk** | `scsi0`: 256G on ospool | -| **Network** | `net0`: virtio, MAC `BA:FA:65:F6:46:25`, vmbr0, VLAN 50 | -| **Boot** | scsi0 → ide2 → net0 | -| **Startup** | Order 20 | -| **Onboot** | Yes | - -**Purpose**: Tactical RMM (Remote Monitoring and Management) server. Provides IT management, remote access, and monitoring capabilities. - ---- - -## Startup Order Summary - -VMs are brought up in the following order on host boot: - -| Order | VMID | Name | Delay | +| id | type | backs | notes | |---|---|---|---| -| 2 | 100 | PFI-ANA-TRUENAS | 120s | -| 3 | 104 | PFI-Mongo | 60s | -| 3 | 105 | PFI-Postgres | 60s | -| 4 | 102 | PFI-ANA-Docker | — | -| 5 | 101 | PFI-ANA-DC | 120s | -| 10 | 108 | PFI-ANA--DEV | 60s | -| 20 | 111 | pfi-tacticalrmm | — | -| 30 | 110 | PFI-ANA-Webhost | 120s | +| `ospool` | zfspool, 2×mirror (bays 0–3), 10.9T raw | **every VM boot disk and every CT rootfs** except pbs-ana | the pool that boots the site; also holds `ospool/naspool-evac` (parked copy from the 2026-09-06 rebuild, to be destroyed) | +| `NASPool` | zfspool, **raidz2 ×6** (bays 4–8, 11), 21.7T usable | backup / bulk tier: `backupStore` (PBS-ANA datastore, restic repos), `pve-VMStorage`, `db`, `docker`, `webdav` | rebuilt 2026-09-06 (`docs/runbooks/pfi-pve-naspool-rebuild.md`); bays 9/10 dead (backplane), drives in them = cold spares | +| `naspool-vmstorage` | dir on `/NASPool/pve-VMStorage` | ISOs, templates, legacy vzdump (newest-per-guest only) | was `pve-truenas` until 2026-09-06 | +| `local-lvm` | lvmthin | pbs-ana's boot disk | | +| `local` | dir | ISOs | | +| `pbs-ana` | pbs at 10.250.50.90 | vzdump jobs (03:00 all guests except 100,109; 22:00 CT 109) | datastore lives on `NASPool/backupStore` via ana-nas NFS | -VMs without a startup order (103, 106, 107) will start based on their `onboot` setting but without a specific sequencing delay. +**Network:** all guests on `vmbr0` VLAN tag 50 (`10.250.50.0/24`), firewall enabled on +every NIC. Linux = virtio, Windows = e1000. CTs take DHCP (reserved). -## Resource Summary +## Virtual machines -| VMID | Name | vCPU | RAM (MB) | Disk | OS | -|---|---|---|---|---|---| -| 100 | PFI-ANA-TRUENAS | 4 | 8,196 | 80G | Debian 12 (reprovisioned from TrueNAS SCALE) | -| 101 | PFI-ANA-DC | 12 | 24,576 | 240G | Windows 11 | -| 102 | PFI-ANA-Docker | 8 | 16,384 | 250G | Linux | -| 103 | PFI-SlaveBot | 8 | 8,192 | 256G | Windows 10 | -| 104 | PFI-Mongo | 8 | 8,196 | 256G | Linux | -| 105 | PFI-Postgres | 16 | 8,196 | 80G | Debian 11 | -| 106 | PFI-Tailscale | 8 | 2,048 | 256G | Debian 12 | -| 107 | PFI-Pteradactyl | 8 | 8,192 | 256G | Debian 11 | -| 108 | PFI-ANA--DEV | 8 | 8,192 | 120G | Debian 11 | -| 110 | PFI-ANA-Webhost | 16 | 4,096 | 250G | Debian 11 | -| 111 | pfi-tacticalrmm | 16 | 8,192 | 256G | Linux | -| | **Totals** | **112** | **105,348** | **2,300G** | | +| VMID | name | IP | vCPU (sockets×cores) | RAM | disk | OS | onboot / startup | role | +|---|---|---|---|---|---|---|---|---| +| 100 | pbs-ana | 10.250.50.90 | 4 (1×4, x86-64-v2-AES) | 8 G | 32G local-lvm | Debian 12 | yes / — | Proxmox Backup Server, fleet primary. `servers/pbs-ana/` | +| 101 | PFI-ANA-DC | 10.250.50.60 | 12 (2×6, host) | 24 G | 240G ospool + EFI + TPM2 | Windows 11, q35, OVMF | yes / order 5, up 120s | Active Directory domain controller | +| 102 | PFI-ANA-Docker | 10.250.50.70 | 8 (2×4, x86-64-v2-AES) | 16 G | 250G ospool | Debian | yes / order 4 | `ana-docker`: LiteLLM gateway, gitea, Langfuse, AdGuard, Beszel/Dozzle hubs, rest-server-ana, backrest. `servers/ana-docker/` | +| 103 | PFI-SlaveBot | 10.250.50.151 | 8 (2×4, host) | 8 G | 256G ospool (ide0) | Windows 10, i440fx | yes / — | purpose undocumented; **not in `servers/`** | +| 104 | PFI-Mongo | 10.250.50.81 | 8 (2×4, host) | 8 G | 256G ospool | Linux | yes / order 3, up 60s | MongoDB; **not in `servers/`** | +| 105 | PFI-Postgres | 10.250.50.80 | 16 (4×4) | 8 G | 80G ospool | Debian 11 | yes / order 3, up 60s | shared Postgres (vaultwarden/gitea/paperless); PGDATA local since 2026-04-23. `servers/pfi-postgres/` | +| 106 | corviduo-dev | 10.250.50.152 | 8 (2×4, x86-64-v2-AES) | 8 G | 80G ospool | Debian 13 | yes / order 10 | Worldtree-team dev VM, PFI-hosted. `servers/corviduo-dev/` | +| 107 | PFI-Pteradactyl | 10.250.50.55 | 8 (2×4, host) | 8 G | 256G ospool | Debian 11 | yes / — | Pterodactyl game panel. `servers/pfi-pteradactyl/` | +| 110 | PFI-ANA-Webhost | 10.250.50.52 | 16 (4×4) | 4 G (balloon 1 G) | 250G ospool | Debian 11 | yes / order 30, up/down 120s | web workload. `servers/pfi-ana-webhost/` | +| 111 | pfi-tacticalrmm | 10.250.50.57 | 16 (4×4) | 8 G | 256G ospool | Linux | yes / order 20 | TacticalRMM. `servers/pfi-tacticalrmm/` | -## Network Notes +All VMs have the QEMU guest agent enabled. -- All VMs are on **VLAN 50** via `vmbr0`. -- All VMs have **firewall enabled** on the network interface. -- Linux VMs use `virtio` network adapters; Windows VMs use `e1000`. -- The MongoDB connection string embedded in VM 104's config references IP `10.250.50.81`, suggesting VLAN 50 maps to the `10.250.50.0/24` subnet within the `10.250.0.0/16` range. +## Containers + +| CTID | hostname | IP | cores | RAM | rootfs | privileged | onboot / startup | role | +|---|---|---|---|---|---|---|---|---| +| 109 | ana-nas | 10.250.50.50 | 4 | 2 G | 80G ospool | **yes** (apparmor unconfined, all devices) | yes / order 1, up 45s | NFS/SMB server straight off the ZFS pools — bind-mounts `/ospool/backup` and every `NASPool/*` dataset under `/mnt/…`. Data-serving SPOF. `servers/ana-nas/` | +| 112 | ana-filebot | 10.250.50.53 | 4 | 2 G | 80G ospool | **yes** (same binds as 109) | yes / order 10 | file-task automation. `servers/ana-filebot/` | +| 113 | ana-wg | 10.250.50.252 (+ wg0 10.30.10.1) | 4 | 2 G | 8G ospool | no (nesting) | yes / order 2 | WireGuard remote-access endpoint, UDP 31337 via FortiGate VIP. `servers/ana-wg/` | + +## Startup order (host boot) + +| order | guest | delay | +|---|---|---| +| 1 | CT 109 ana-nas | up 45s | +| 2 | CT 113 ana-wg | — | +| 3 | VM 104 PFI-Mongo, VM 105 PFI-Postgres | up 60s | +| 4 | VM 102 PFI-ANA-Docker | — | +| 5 | VM 101 PFI-ANA-DC | up 120s | +| 10 | VM 106 corviduo-dev, CT 112 ana-filebot | — | +| 20 | VM 111 pfi-tacticalrmm | — | +| 30 | VM 110 PFI-ANA-Webhost | up/down 120s | +| unordered | VM 100 pbs-ana, VM 103, VM 107 | onboot=1, no sequencing | + +Reverse this for a graceful host shutdown (done 2026-09-05 for the PERC swap). + +## Resource summary + +| | vCPU | RAM | disk | +|---|---|---|---| +| 10 VMs | 108 | 100 G | 2,180 G | +| 3 CTs | 12 | 6 G | 168 G | +| **total** | **120** on 48 threads (2.5× overcommit) | **106 G** of 188 G | on ospool 547 G actually allocated (thin) | + +## Not on this hypervisor + +`ana-ml2` (10.250.50.54) is bare-metal Supermicro; `sfsrv-ana` (10.250.250.115) is the +SureFire client's own Dell R630; `pfi-gx10` is at NH3.