Files
esh-pfi-infrastructure/servers/pfi-pve
vh 91bda3c480 fv-ml1: complete the cutover — rename, renumber, DNS, and the LiteLLM repoint
The box is physically at Fountain Valley, renamed, renumbered onto 10.251/16,
and serving inference again. This lands the repo half of that.

Host: hostname ana-ml2 -> fv-ml1, pinned to 10.251.50.54 by a dnsmasq
reservation so the address the runbook, DNS and LiteLLM all assume is the
address it actually has. Its headscale node is renamed too.

The sweep ran from scripts/fv-ml1-rename-sweep.sh, whose allowlist is the
reason this diff touches current-state files and not the record. Dated
persistent-memory entries, archival-memory and incident notes still say
ana-ml2 in 31 and 62 places respectively, because that is what the box was
when those things happened. Rewriting them would make the history lie.

LiteLLM was the load-bearing piece and needed more than the api_base sed the
runbook describes. Twenty api_base entries repointed, but a grep-and-verify
pass also caught a LIVE pass_through_endpoints target for the scalar-judge
reward route still on the old address -- an api_base-only substitution would
have left it dead. Four prose references describing current state were
repointed as well; one historical note recording where a hand-test was run
is deliberately left pointing at 10.250.50.54.

Two facts in the server tables were wrong and are corrected here. The site is
Fountain Valley, not Anaheim. And the box has FOUR RTX PRO 6000 Blackwell
Max-Q, not two -- verified by nvidia-smi -L and independently by PCI
enumeration of four GB202GL devices. That is 391 GB of VRAM rather than 196,
which changes what fits on it.

DNS: fv-ml1, fv-ml1-bmc and fv-gw added under the fv site via the piggyback
approach, scriberr re-homed, and the ana-ml2 records removed. Applied to all
three resolvers. The BMC record carries a warning that its 802.1q VLAN tag
must stay disabled -- it shipped tagging VLAN 250 into an untagged port,
which made it invisible to every network-side diagnostic and is the reason
it appeared dead through several cable changes.

Verified end to end: summarizer and sec both answer through the Anaheim
gateway across the mesh to FV seats on different ports.
2026-09-12 22:00:50 -07:00
..

pfi-pve

Proxmox VE hypervisor for the Anaheim colo (pve.phasefinal.com). Hosts every VM in the PFI-ANA cluster.

Network

  • LAN IP: 10.250.250.31 (in-band, OS-side)
  • iDRAC (OOB): 10.250.250.30 — Dell iDRAC web UI at https://10.250.250.30/ (homepage card: PFI-r750xs-iDRAC)
  • FQDN: pve.phasefinal.com
  • Web UI: https://10.250.250.31:8006
  • SSH: root@pfi-pve (key auth; ssh-target file uses the hostname)

Hardware

  • Chassis: Dell PowerEdge R750xs (2U rackmount)
  • CPU: Intel Xeon Silver 4310 @ 2.10 GHz
  • RAM: 188 GB
  • Kernel: 6.8.12-8-pve (Proxmox 8.x)
  • Storage: local pve-root at 65G (49% used) + pve LVM pool for VM disks

What it runs

The VMs/CTs in docs/pfi/proxmox-vms.md live here. Full enumeration as of 2026-05-14 (via qm list on the hypervisor):

  • VM 100 — pbs-ana (Proxmox Backup Server; 10.250.50.90) — servers/pbs-ana/
  • VM 101 — PFI-ANA-DC — Active Directory domain controller; not yet in servers/ inventory
  • VM 102 — ana-docker (PFI-ANA-Docker) — servers/ana-docker/
  • VM 103 — PFI-SlaveBot — not yet in servers/ inventory (purpose TBD)
  • VM 104 — PFI-Mongo — MongoDB host; not yet in servers/ inventory
  • VM 105 — pfi-postgres (PFI-Postgres; PGDATA on local VM disk as of 2026-04-23 migration off ana-nas NFS) — servers/pfi-postgres/
  • VM 106 — corviduo-dev (Worldtree-team dev VM, PFI-hosted) — servers/corviduo-dev/
  • VM 107 — pfi-pteradactyl (game panel) — servers/pfi-pteradactyl/
  • VM 110 — pfi-ana-webhost — servers/pfi-ana-webhost/
  • VM 111 — pfi-tacticalrmm — servers/pfi-tacticalrmm/
  • CT 109 — ana-nas (Debian LXC serving NFS/SMB; bind-mounts from ospool; data-serving SPOF — see memory/incident_ana_nas_spof.md) — servers/ana-nas/
  • CT 112 — ana-filebot (file-task automation) — servers/ana-filebot/
  • CT 113 — ana-wg (WireGuard VPN) — servers/ana-wg/

fv-ml1 (10.251.50.54) is a separate bare-metal Supermicro box, not on this hypervisor. sfsrv-ana (10.250.250.115) runs on a separate Dell R630, also not on this hypervisor.

Refresh state

scripts/refresh-server-info.sh pfi-pve

The snapshot captures host/hardware/filesystem/network/ports but not VM-level detail (server_inspect.sh targets Docker hosts). Running qm list, pvesh get /cluster/resources, and zpool list -v manually is the current way to audit VMs and storage. A Proxmox-flavored inspect script is a future improvement.

Placement rule

This is a hypervisor, not a target for Docker stacks. Add new workloads by standing up VMs here (or on ana-docker / fv-ml1, whichever fits) rather than deploying containers directly on the PVE host.