diff --git a/ATLAS/Infrastructure/Infrastructure-PFI-Project-Index.md b/ATLAS/Infrastructure/Infrastructure-PFI-Project-Index.md index 7fdd07b..a51a387 100644 --- a/ATLAS/Infrastructure/Infrastructure-PFI-Project-Index.md +++ b/ATLAS/Infrastructure/Infrastructure-PFI-Project-Index.md @@ -16,9 +16,15 @@ keywords: - gpu - network - llama-swap -- 'yes' +- bare-metal +- 10.250.50.54 +- 10.250.50.70 +- ana-ml2 +- PFI-ANA_DOCKER +- inference-server +- VM-102 links: [] -modified: '2026-04-11T01:28:16+00:00' +modified: '2026-04-11T01:33:45+00:00' summary: Comprehensive architecture reference for the Infrastructure-PFI project covering three Tailscale-linked sites, 11 Proxmox VMs at PFI-ANA colo, 6 Docker services with GPU workloads, and llama-swap model management across 20 models. @@ -29,6 +35,7 @@ tags: - proxmox - colocation - reference +- hardware-audit title: Infrastructure-PFI — Full Project Index & Architecture Reference --- @@ -46,10 +53,23 @@ Infrastructure-PFI is the configuration and documentation project for the PFI ne |---|---|---|---|---| | **ESH-Home** | 10.0.0.0/16 | Proxmox VE | VM at 10.0.50.45 | Ubiquiti | | **PFI-NH3** | 10.100.0.0/16 | Proxmox VE | Proxmox VM | Ubiquiti | -| **PFI-ANA (Colo)** | 10.250.0.0/16 | Proxmox VE at 10.250.250.31:8006 | VM 102 (10.250.50.54, ana-ml2) | FortiGate | +| **PFI-ANA (Colo)** | 10.250.0.0/16 | Proxmox VE at 10.250.250.31:8006 | Bare metal at 10.250.50.54 · VM 102 (10.250.50.70, ana-ml2) | FortiGate | All sites are linked via Tailscale VPN (VM 106 at the colo). +### PFI-ANA Colo — Host Reference + +| Host | IP | Type | Role | +|---|---|---|---| +| **Bare Metal** | 10.250.50.54 | Physical server | Main inferencing server — direct GPU access, runs inference workloads | +| **VM 102** (PFI-ANA_DOCKER) | 10.250.50.70 | Proxmox VM | Docker host — llama-swap, ComfyUI, VibeVoice, Parakeet, ChromaDB, Dockge | +| **VM 100** (PFI-ANA-TRUENAS) | — | Proxmox VM | NAS/iSCSI storage | +| **VM 101** (PFI-ANA-DC) | — | Proxmox VM | Windows Domain Controller | +| **VM 104** (PFI-Mongo) | 10.250.50.81 | Proxmox VM | MongoDB server | +| **VM 106** (PFI-Tailscale) | — | Proxmox VM | VPN gateway (Tailscale) | + +The Proxmox hypervisor management interface is at `10.250.250.31:8006`. + --- ## File & Directory Map @@ -99,7 +119,7 @@ Infrastructure-PFI/ │ ├── backup-chromadb.sh # Hot backup with 30-day retention │ ├── health-check-chromadb.sh # Health monitoring with alerting │ ├── quickstart-chromadb.sh # One-command ChromaDB setup wrapper -│ └── inference-hardware-profile.sh # ⚠️ NEW — Hardware profiler for AI/ML inference planning (VM 102) +│ └── inference-hardware-profile.sh # Hardware profiler for AI/ML inference (bare metal 10.250.50.54) └── output/ # Generated output and logs (empty) ``` @@ -187,7 +207,8 @@ The llama-swap gateway manages **20 GGUF models** across multiple categories: - Linux VMs: `virtio` - Windows VMs: `e1000` - **Docker Network**: Services on external network `traefik-net` -- **Docker Host IP**: `10.250.50.54` (ana-ml2) +- **Docker Host IP**: `10.250.50.70` (VM 102 / ana-ml2 / PFI-ANA_DOCKER) +- **Bare Metal Inference Server IP**: `10.250.50.54` (physical server, direct GPU access) --- @@ -205,11 +226,13 @@ The llama-swap gateway manages **20 GGUF models** across multiple categories: | Script | Purpose | |---|---| | `nemotron-query.sh` | Bash client for querying Nemotron via llama-swap API | -| `inference-hardware-profile.sh` | **NEW** — Comprehensive hardware profiler for AI/ML inference planning | +| `inference-hardware-profile.sh` | Comprehensive hardware profiler for AI/ML inference planning | ### inference-hardware-profile.sh -**Target Host:** VM 102 (ana-ml2) at `10.250.50.54` — PFI-ANA Colo +**Target Hosts:** +- **Bare metal inferencing server** at `10.250.50.54` — primary target (direct GPU access) +- **VM 102 (PFI-ANA_DOCKER / ana-ml2)** at `10.250.50.70` — secondary target (GPU passthrough) A comprehensive system hardware profiler designed for AI/ML inference capacity planning. Captures the full hardware picture needed to evaluate and optimize GPU inference workloads. @@ -232,7 +255,7 @@ A comprehensive system hardware profiler designed for AI/ML inference capacity p | `--quick` | One-line summary | | `--gpu-only` | Full report without Docker sections | -> **⚠️ Action needed:** This script should be run on VM 102 to capture the **physical GPU model, VRAM capacity, and host hardware specs**, which are currently undocumented in this project. The output (especially `--json`) should be saved to close the hardware documentation gap. +> **⚠️ Action needed:** Run this script on the **bare metal server** (10.250.50.54) to capture the **physical GPU model, VRAM capacity, and host hardware specs**, which are currently undocumented. The output (especially `--json`) should be saved to close Known Issue #6. Optionally also run on VM 102 (10.250.50.70) for passthrough comparison. --- @@ -243,7 +266,7 @@ A comprehensive system hardware profiler designed for AI/ML inference capacity p 3. **Granite 4 Small** has `ttl: 0` (persistent caching) but is not assigned to the utility group 4. **Parakeet STT** references `.env` configuration not tracked in version control 5. **Qwen3-Coder-Next** download may be incomplete — `.incomplete` files present on disk -6. **Physical GPU model and VRAM undocumented** — the physical GPU passed through to VM 102 (its model, total VRAM, PCIe slot, etc.) is not recorded anywhere in project documentation. Run `inference-hardware-profile.sh --json` on VM 102 and save the output to close this gap. +6. **Physical GPU model and VRAM undocumented** — the physical GPU(s) in the bare metal inferencing server at 10.250.50.54 (and those passed through to VM 102 at 10.250.50.70) are not recorded anywhere in project documentation. Run `inference-hardware-profile.sh --json` on the bare metal server and save the output to close this gap. ---