fleet: register 9 hosts surfaced by gap-analysis audit

Six PFI VMs/LXCs previously known only via proxmox_inspect.sh — covered by vzdump but not in servers/, so operational context (roles, backup posture, ssh target) was missing: pfi-ana-webhost (VMID 110) — web workload ana-filebot (LXC 112) — file-task automation pfi-pteradactyl (VMID 107) — Pterodactyl game panel pfi-tacticalrmm (VMID 111) — TacticalRMM remote-management pfi-postgres (VMID 105) — shared Postgres (vaultwarden/gitea/ paperless backends) ana-wg (LXC 113) — WireGuard VPN gateway Plus three SureFire tenant hosts at the Anaheim colo: sfsrv-ana — tenant Proxmox hypervisor (10.250.250.115:8006) sf-ana-container — container workload on that Proxmox sf-r630 — physical R630 (iDRAC 10.250.250.110 for PFI-side hardware mgmt; OS is tenant-scoped) Each server dir has README + ssh-target where applicable. SureFire entries explicitly document tenancy scope: PFI provides hosting, SureFire owns the OS; management actions need tenant coordination. SureFire hosts have no ssh-target by default. Homepage Infra - ANA gains two new cards: - SFsrv-ANA (https://10.250.250.115:8006, si-proxmox icon) - SF-R630-iDRAC (https://10.250.250.110, si-dell icon) PFI-ANA-ML2 BMC gained an href since it has a usable web UI. CLAUDE.md fleet table extended with all 9 new rows. Placement-rules section notes the SureFire tenant boundary. Memory: new project_surefire_tenant.md so future sessions know sf-* hosts are tenant-scoped by default.
2026-04-21 14:29:43 -07:00
parent b33439499b
commit b842212b06
17 changed files with 405 additions and 1 deletions
@@ -55,7 +55,16 @@ Observed and standardized across servers:
 |------|-----|------|------|---------|
 | ana-ml2 | 10.250.50.54 | Anaheim (`10.250.0.0/16`) | GPU / AI inference (bare metal) | `servers/ana-ml2/README.md` |
 | ana-docker | 10.250.50.70 | Anaheim | General-purpose Docker host (non-GPU VM on pfi-pve) | `servers/ana-docker/README.md` |
+| pfi-ana-webhost | 10.250.50.52 | Anaheim | VM on pfi-pve (VMID 110) — web workload | `servers/pfi-ana-webhost/README.md` |
+| ana-filebot | 10.250.50.53 | Anaheim | LXC on pfi-pve (CT 112) — file-task automation | `servers/ana-filebot/README.md` |
+| pfi-pteradactyl | 10.250.50.55 | Anaheim | VM on pfi-pve (VMID 107) — Pterodactyl game panel | `servers/pfi-pteradactyl/README.md` |
+| pfi-tacticalrmm | 10.250.50.57 | Anaheim | VM on pfi-pve (VMID 111) — TacticalRMM | `servers/pfi-tacticalrmm/README.md` |
+| pfi-postgres | 10.250.50.80 | Anaheim | VM on pfi-pve (VMID 105) — shared Postgres (vaultwarden/gitea/paperless) | `servers/pfi-postgres/README.md` |
+| ana-wg | 10.250.50.252 | Anaheim | LXC on pfi-pve (CT 113) — WireGuard | `servers/ana-wg/README.md` |
 | pfi-pve | 10.250.250.31 | Anaheim | Proxmox VE hypervisor | `servers/pfi-pve/README.md` |
+| sfsrv-ana | 10.250.250.115 | Anaheim | **Tenant (SureFire)** — Proxmox VE hypervisor | `servers/sfsrv-ana/README.md` |
+| sf-ana-container | 10.250.150.100 | Anaheim | **Tenant (SureFire)** — container workload on sfsrv-ana | `servers/sf-ana-container/README.md` |
+| sf-r630 | iDRAC 10.250.250.110 | Anaheim | **Tenant (SureFire)** — physical Dell R630, iDRAC-managed from PFI side | `servers/sf-r630/README.md` |
 | nh3-docker | 10.100.50.40 | NH3 (`10.100.0.0/16`) | General-purpose Docker host (non-GPU VM on nh3-pve) | `servers/nh3-docker/README.md` |
 | nh3-pve | 10.100.250.60 | NH3 | Proxmox VE hypervisor | `servers/nh3-pve/README.md` |
 | esh-docker-vm | 10.0.50.45 | ESH home lab (`esteban.net`, `10.0.50.0/24`) | Home-lab Docker host (VM on esh-pve) | `servers/esh-docker-vm/README.md` |
@@ -69,6 +78,7 @@ Observed and standardized across servers:
 - NH-site non-GPU services → `nh3-docker`.
 - ESH home-lab workloads (`esteban.net`) → `esh-docker-vm` (general) or `vm-esh-nas` (needs direct NFS mounts from 10.0.50.50). Not part of the PFI colo topology, but shares monitoring/backup tooling.
 - Cross-site services (e.g. Beszel hub, Dozzle hub) live on `ana-docker` and pull from agents on the other hosts.
+- **SureFire (SF) tenant hosts** (`sf-*`, `sfsrv-ana`) are at the Anaheim colo but PFI doesn't control the OS. Inventory only — coordinate any management with the tenant.
 - **Hypervisors** (`pfi-pve`, `nh3-pve`, `esh-pve`, `esh-pve-nas`) are tracked for inventory / capacity planning. Don't deploy Docker stacks directly on them; new workloads land as VMs. `server_inspect.sh` captures host-level detail only — VM/LXC/ZFS enumeration needs Proxmox-native tooling (`qm list`, `pvesh get …`, `zpool list`).

 ## How to refresh a server's state
@@ -85,9 +85,20 @@
      icon: mdi-brain
      description: GPU host (bare-metal)
  - PFI-ANA-ML2 BMC:
-      ping: 10.250.250.50
+      href: https://10.250.250.50
      icon: mdi-brain
+      siteMonitor: https://10.250.250.50
      description: BMC (ana-ml2)
+  - SFsrv-ANA:
+      href: https://10.250.250.115:8006
+      icon: si-proxmox
+      siteMonitor: https://10.250.250.115:8006
+      description: Proxmox (SureFire tenant hypervisor at PFI colo)
+  - SF-R630-iDRAC:
+      href: https://10.250.250.110/
+      icon: si-dell
+      siteMonitor: https://10.250.250.110/
+      description: Dell R630 iDRAC (SureFire tenant hardware)

 - Infra - NH3:
  # NH3-Firewall (Fortigate 101F at 10.100.250.1) retired 2026-04-21 —
@@ -0,0 +1,39 @@
+# ana-filebot
+
+File-automation LXC at the Anaheim colo.
+
+## Network
+
+- **LAN IP:** 10.250.50.53
+- **SSH:** `root@ana-filebot` (LXC — typically root via key)
+
+## Infrastructure
+
+- **Hypervisor:** `pfi-pve` (**LXC 112**)
+- **Type:** LXC container
+- **Site:** Anaheim (PFI colo)
+
+## Role
+
+File-task automation — hostname referenced earlier as `ANA-FileBot` on
+the homepage. Handles routine file-processing workflows.
+
+## Backup coverage
+
+- **VM-image:** ✅ vzdump on pfi-pve (daily)
+- **File-level restic:** ❌ not yet configured
+
+Container-level backup from pfi-pve covers the rootfs. File-level
+restic would add fast per-file restore — worth adding if scripts /
+configs here are actively edited.
+
+## Refresh state
+
+```bash
+scripts/refresh-server-info.sh ana-filebot
+```
+
+## Discovered via
+
+FortiGate DHCP (MAC `bc:24:11:c0:35:77`) and pre-existing homepage
+reference as `ANA-FileBot`.
@@ -0,0 +1 @@
+root@10.250.50.53
@@ -0,0 +1,37 @@
+# ana-wg
+
+WireGuard VPN gateway at the Anaheim colo.
+
+## Network
+
+- **LAN IP:** 10.250.50.252
+- **SSH:** `root@ana-wg` (LXC — root via key)
+
+## Infrastructure
+
+- **Hypervisor:** `pfi-pve` (**LXC 113**)
+- **Type:** LXC container
+- **Site:** Anaheim (PFI colo)
+
+## Role
+
+WireGuard server providing VPN access into the PFI Anaheim network.
+Config lives at `/etc/wireguard/` inside the LXC.
+
+## Backup coverage
+
+- **VM-image:** ✅ vzdump on pfi-pve (daily) — captures the LXC rootfs
+  which contains the wireguard config + peer keys. Sufficient for
+  disaster recovery.
+- **File-level restic:** ❌ not needed — no DB, no high-churn state,
+  config is small and recoverable from vzdump.
+
+## Refresh state
+
+```bash
+scripts/refresh-server-info.sh ana-wg
+```
+
+## Discovered via
+
+FortiGate DHCP (MAC `bc:24:11:d7:e4:b7`).
@@ -0,0 +1 @@
+root@10.250.50.252
@@ -0,0 +1,40 @@
+# pfi-ana-webhost
+
+General-purpose webhost at the Anaheim colo.
+
+## Network
+
+- **LAN IP:** 10.250.50.52
+- **SSH:** `lkraven@pfi-ana-webhost` (key auth — verify on first run)
+
+## Infrastructure
+
+- **Hypervisor:** `pfi-pve` (VMID **110**)
+- **Type:** Linux VM
+- **Site:** Anaheim (PFI colo)
+
+## Role
+
+Web-facing workload host. DNS name includes `pfi-ana-webhost.phasefinal.com`.
+Specific services running here aren't catalogued in this workspace yet —
+refresh the snapshot and inspect.
+
+## Backup coverage
+
+- **VM-image:** ✅ vzdump on pfi-pve (daily, per the 2026-04-20 audit)
+- **File-level restic:** ❌ not yet configured
+
+If this host starts carrying any state worth per-file restore (app
+configs, TLS certs, DBs), add a `configs/restic/pfi-ana-webhost/`
+profile following the ana-ml2 template.
+
+## Refresh state
+
+```bash
+scripts/refresh-server-info.sh pfi-ana-webhost
+```
+
+## Discovered via
+
+`scripts/discover-fortigate.sh 10.250.250.1` on 2026-04-21 —
+FortiGate DHCP lease (MAC `e6:f9:3a:c9:61:2a`).
@@ -0,0 +1 @@
+lkraven@10.250.50.52
@@ -0,0 +1,57 @@
+# pfi-postgres
+
+Shared Postgres cluster at the Anaheim colo.
+
+## Network
+
+- **LAN IP:** 10.250.50.80
+- **Port:** 5432
+- **SSH:** `lkraven@pfi-postgres`
+
+## Infrastructure
+
+- **Hypervisor:** `pfi-pve` (VMID **105**)
+- **Type:** Linux VM
+- **Site:** Anaheim (PFI colo)
+
+## Role
+
+Shared Postgres server backing multiple ana-docker stacks:
+
+| Consumer | DB | User |
+|---|---|---|
+| vaultwarden | `vaultwarden` | `vaultwarden` |
+| gitea | `gitea` | `gitea` |
+| paperless-ngx (on esh-docker-vm) | `paperless-ng` | `paperless-ng` |
+| (Likely others — audit as more stacks surface) |
+
+## Backup coverage
+
+- **VM-image:** ✅ vzdump on pfi-pve (daily)
+- **App-consistent DB dumps:** ✅ each dependent stack's host runs a
+  pre-backup hook that `pg_dump`s its database from this server into
+  its own restic stage:
+  - ana-docker: vaultwarden + gitea (via `configs/restic/ana-docker/pre-backup.sh`)
+  - esh-docker-vm: paperless-ngx (via `configs/restic/esh-docker-vm/pre-backup.sh`)
+- **File-level restic on this host:** ❌ not configured. Optional — the
+  VM-image + per-consumer pg_dumps together cover most recovery paths.
+
+## Known weak password (rotate)
+
+The inline passwords currently in use for these databases are
+trivially weak:
+- `gitea` / `gitea`
+- `paperless-ng` / `paperless-ng`
+
+Rotation is on the post-backup-pipeline cleanup list. Exposed in the
+2026-04-21 transcripts.
+
+## Refresh state
+
+```bash
+scripts/refresh-server-info.sh pfi-postgres
+```
+
+## Discovered via
+
+FortiGate DHCP (MAC `c2:1f:cc:71:66:d0`).
@@ -0,0 +1 @@
+lkraven@10.250.50.80
@@ -0,0 +1,40 @@
+# pfi-pteradactyl
+
+Pterodactyl game-server panel at the Anaheim colo.
+
+## Network
+
+- **LAN IP:** 10.250.50.55
+- **Web UI:** http://10.250.50.55/ (listed on homepage under **Games**)
+- **SSH:** `lkraven@pfi-pteradactyl`
+
+## Infrastructure
+
+- **Hypervisor:** `pfi-pve` (VMID **107**)
+- **Type:** Linux VM
+- **Site:** Anaheim (PFI colo)
+
+## Role
+
+Pterodactyl Panel — web UI and daemons for running game servers.
+
+## Backup coverage
+
+- **VM-image:** ✅ vzdump on pfi-pve (daily)
+- **File-level restic:** ❌ not yet configured
+
+Game-server state (save files, world data, config) lives inside this
+VM. Volume-level restic covering /var/lib/pterodactyl or equivalent
+would be a useful layer on top of vzdump — worth a dedicated
+`configs/restic/pfi-pteradactyl/` profile if game state turnover is high.
+
+## Refresh state
+
+```bash
+scripts/refresh-server-info.sh pfi-pteradactyl
+```
+
+## Discovered via
+
+FortiGate DHCP (MAC `ca:44:37:8a:bf:e0`) and pre-existing homepage
+entry under **Games**.
@@ -0,0 +1 @@
+lkraven@10.250.50.55
@@ -0,0 +1,40 @@
+# pfi-tacticalrmm
+
+Tactical RMM (remote monitoring + management) server at the Anaheim colo.
+
+## Network
+
+- **LAN IP:** 10.250.50.57
+- **SSH:** `lkraven@pfi-tacticalrmm`
+
+## Infrastructure
+
+- **Hypervisor:** `pfi-pve` (VMID **111**)
+- **Type:** Linux VM
+- **Site:** Anaheim (PFI colo)
+
+## Role
+
+[TacticalRMM](https://tacticalrmm.com/) — open-source RMM platform.
+Monitors and manages endpoints, pushes patches, runs scripts, etc.
+
+## Backup coverage
+
+- **VM-image:** ✅ vzdump on pfi-pve (daily)
+- **File-level restic:** ❌ not yet configured
+
+TacticalRMM state (Postgres DB with inventory + automation history,
+MeshCentral config, agent registrations) is critical if used in
+production. Worth setting up app-consistent DB dumps + file-level
+restic if it's the primary management plane.
+
+## Refresh state
+
+```bash
+scripts/refresh-server-info.sh pfi-tacticalrmm
+```
+
+## Discovered via
+
+`scripts/discover-fortigate.sh 10.250.250.1` on 2026-04-21 —
+FortiGate DHCP lease (MAC `ba:fa:65:f6:46:25`).
@@ -0,0 +1 @@
+lkraven@10.250.50.57
@@ -0,0 +1,37 @@
+# sf-ana-container
+
+**SureFire tenant workload** at the Anaheim colo — a container
+host/VM running on the SureFire Proxmox (`sfsrv-ana`, 10.250.250.115).
+
+## Tenancy
+
+- **Owner:** SureFire (tenant)
+- **PFI role:** hosting provider
+- **Management scope:** coordinate with SureFire before any action
+
+## Network
+
+- **LAN IP:** 10.250.150.100
+- **SSH:** not wired — tenant equipment.
+
+## Infrastructure
+
+- **Hypervisor:** `sfsrv-ana` (SureFire Proxmox, 10.250.250.115)
+- **Site:** Anaheim (PFI colo)
+- **Subnet:** `10.250.150.0/24` — dedicated SureFire container subnet
+
+## Role
+
+Container/workload host for SureFire. Specific services aren't
+documented here (not PFI equipment).
+
+## Backup coverage
+
+- **Not currently backed up by the PFI fleet.** Needs coverage plan —
+  see `servers/sfsrv-ana/README.md` for the three approaches under
+  discussion.
+
+## Discovered via
+
+`scripts/discover-fortigate.sh 10.250.250.1` on 2026-04-21 (MAC
+`6a:09:9a:8a:08:a5`).
@@ -0,0 +1,36 @@
+# sf-r630
+
+**SureFire tenant physical server** at the Anaheim colo — Dell
+PowerEdge R630. Owner operates the OS; PFI provides rack + network.
+
+## Tenancy
+
+- **Owner:** SureFire (tenant)
+- **PFI role:** hosting provider
+
+## Network
+
+- **iDRAC BMC IP:** 10.250.250.110 (on the management subnet)
+- **OS-side LAN IP:** unknown — not surfaced via our DHCP discovery.
+  May be static, on a non-DHCP interface, or assigned to a different
+  subnet served by SureFire's own gear.
+- **iDRAC Web UI:** https://10.250.250.110/
+
+## Infrastructure
+
+- **Type:** Physical Dell PowerEdge R630
+- **Site:** Anaheim (PFI colo)
+- **Management:** iDRAC only from the PFI side (power, console, hardware
+  health). OS-level access is tenant-scoped.
+
+## Backup coverage
+
+- Not applicable from the PFI side — tenant equipment. If hosting
+  terms require PFI to provide backup, coordinate with SureFire on
+  in-VM or OS-agent approach.
+
+## Discovered via
+
+`scripts/discover-fortigate.sh 10.250.250.1` on 2026-04-21 — DHCP
+lease on the management interface (MAC `74:e6:e2:fe:2c:7c`, VCI
+`iDRAC`).
@@ -0,0 +1,51 @@
+# sfsrv-ana
+
+**SureFire tenant Proxmox host** at the Anaheim colo. Third-party
+equipment / workload — tracked here for inventory, backup coverage,
+and network awareness.
+
+## Tenancy
+
+- **Owner:** SureFire (tenant)
+- **PFI role:** hosting provider — provides rack, power, network
+- **Management scope:** coordinate with SureFire before any action
+
+## Network
+
+- **LAN IP:** 10.250.250.115
+- **Web UI:** https://10.250.250.115:8006 (Proxmox VE)
+- **SSH:** not currently wired into this workspace (tenant equipment).
+  If PFI ever gains admin access, add `ssh-target` here.
+
+## Infrastructure
+
+- **Type:** Proxmox VE hypervisor (bare metal)
+- **Site:** Anaheim (PFI colo)
+
+Hosts SureFire's own VMs, including:
+- `sf-ana-container` (10.250.150.100) on the container subnet
+
+## Backup coverage
+
+- **Not currently backed up by the PFI fleet.**
+- User flagged this as **needing coverage** — open plan item. Options:
+  1. Coordinate tenant-side backup of SureFire VMs using their own
+     target.
+  2. If PFI is responsible for backups of tenant workloads under the
+     hosting agreement, deploy restic clients to SureFire VMs writing
+     into a segregated repo on `rest-server-ana` (e.g. a dedicated
+     htpasswd user + encryption key scoped to SureFire).
+  3. File-level Proxmox vzdump into shared NAS storage, same pattern
+     as pfi-pve.
+- Decision pending.
+
+## Refresh state
+
+Not wired into `refresh-proxmox-info.sh` — would require SSH access as
+root (same flow as pfi-pve / nh3-pve / etc.). Add an `ssh-target` when
+ready.
+
+## Discovered via
+
+`scripts/discover-fortigate.sh 10.250.250.1` on 2026-04-21 (MAC
+`44:a8:42:33:d9:c3`).