Files
esh-pfi-infrastructure/servers/pfi-pve/README.md
T

62 lines
3.7 KiB
Markdown

# pfi-pve
Proxmox VE hypervisor for the Anaheim colo (`pve.phasefinal.com`). Hosts every VM in the PFI-ANA cluster.
## Network
- **LAN IP:** 10.250.250.31 (in-band, OS-side)
- **iDRAC (OOB):** 10.250.250.30 — Dell iDRAC web UI at
<https://10.250.250.30/> (homepage card: *PFI-r750xs-iDRAC*)
- **FQDN:** `pve.phasefinal.com`
- **Web UI:** `https://10.250.250.31:8006`
- **SSH:** `root@pfi-pve` (key auth; `ssh-target` file uses the hostname)
## Hardware
- **Chassis:** Dell PowerEdge R750xs (2U rackmount)
- **CPU:** Intel Xeon Silver 4310 @ 2.10 GHz
- **RAM:** 188 GB
- **PVE / kernel:** 9.2.21 / `7.0.14-20-pve` since 2026-10-04 (upgraded from 8.3.5; `docs/runbooks/pfi-pve-pve9-upgrade-plan.md`)
- **Boot:** UEFI → `\EFI\proxmox\grubx64.efi` on the BOSS card; `grub-efi-amd64` (was wrongly `grub-pc` until 2026-10-04). NIC names pinned by MAC (`/etc/systemd/network/10-pin-*.link`). Mgmt IP is DHCP on `vmbr0.250` (FortiGate lease)
- **iDRAC9:** Enterprise licence (virtual console), fw 6.10.30.00; creds `secret get pfi-pve/idrac-infra-ops` (user `infra-ops`)
- ⚠ **PSU1 has no input power** (repeatedly since 2026-04-23, latest 2026-09-11); running on PSU2 alone
- ⚠ **Rebooting this host takes all of Anaheim off the fleet for ~4 min** (CT 114 ana-scale is the only NH3/ESH↔ANA path) **and the LiteLLM gateway off for ~20 min** (boot storm). Out-of-band way in: FortiGate WAN SSH `infra-ops@38.120.12.42` (see the upgrade plan § Break-glass)
- **Storage:** local `pve-root` at 65G (49% used) + `pve` LVM pool for VM disks
## What it runs
The VMs/CTs in [`docs/pfi/proxmox-vms.md`](../../docs/pfi/proxmox-vms.md)
live here. Full enumeration as of 2026-05-14 (via `qm list` on the
hypervisor):
- **VM 100** — `pbs-ana` (Proxmox Backup Server; 10.250.50.90) — `servers/pbs-ana/`
- **VM 101** — `PFI-ANA-DC` — Active Directory domain controller; **not in use: shut down, `onboot 0`, excluded from vzdump since 2026-10-04 (Prime)**
- **VM 102** — `ana-docker` (PFI-ANA-Docker) — `servers/ana-docker/`
- **VM 103** — `PFI-SlaveBot` — **not yet in `servers/` inventory** (purpose TBD)
- **VM 104** — `PFI-Mongo` — MongoDB host; **not yet in `servers/` inventory**
- **VM 105** — `pfi-postgres` (PFI-Postgres; PGDATA on local VM disk as of 2026-04-23 migration off ana-nas NFS) — `servers/pfi-postgres/`
- **VM 106** — `corviduo-dev` (Worldtree-team dev VM, PFI-hosted) — `servers/corviduo-dev/`
- **VM 107** — `pfi-pteradactyl` (game panel) — `servers/pfi-pteradactyl/`
- **VM 110** — `pfi-ana-webhost` — `servers/pfi-ana-webhost/`
- **VM 111** — `pfi-tacticalrmm` — `servers/pfi-tacticalrmm/`
- **CT 109** — `ana-nas` (Debian LXC serving NFS/SMB; bind-mounts
from ospool; data-serving SPOF — see `memory/incident_ana_nas_spof.md`) — `servers/ana-nas/`
- **CT 112** — `ana-filebot` (file-task automation) — `servers/ana-filebot/`
- **CT 113** — `ana-wg` (WireGuard VPN) — `servers/ana-wg/`
`fv-ml1` (`10.251.50.54`) is a separate bare-metal Supermicro box,
not on this hypervisor. `sfsrv-ana` (`10.250.250.115`) runs on a
separate Dell R630, also not on this hypervisor.
## Refresh state
```bash
scripts/refresh-server-info.sh pfi-pve
```
The snapshot captures host/hardware/filesystem/network/ports but **not** VM-level detail (`server_inspect.sh` targets Docker hosts). Running `qm list`, `pvesh get /cluster/resources`, and `zpool list -v` manually is the current way to audit VMs and storage. A Proxmox-flavored inspect script is a future improvement.
## Placement rule
This is a hypervisor, not a target for Docker stacks. Add new workloads by standing up VMs here (or on ana-docker / fv-ml1, whichever fits) rather than deploying containers directly on the PVE host.