From 5c48c059da1728c5915108cd9d5c4bd9bf5e480b Mon Sep 17 00:00:00 2001 From: Atlas Date: Fri, 10 Apr 2026 23:15:40 -0700 Subject: [PATCH] =?UTF-8?q?Update:=20ATLAS/Infrastructure/Infrastructure-P?= =?UTF-8?q?FI-Project-Index.md=20=E2=80=94=20Add=20hardware=20profile=20do?= =?UTF-8?q?cument=20reference=20to=20file=20tree,=20enrich=20Host=20Refere?= =?UTF-8?q?nce=20table=20with=20captured=20specs=20(dual=20EPYC=209254,=20?= =?UTF-8?q?566=20GB=20RAM,=202=C3=97=20RTX=206000=20Ada),=20update=20Known?= =?UTF-8?q?=20Issue=20#6=20as=20resolved,=20and=20update=20action=20note?= =?UTF-8?q?=20since=20hardware=20profile=20has=20been=20captured.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../Infrastructure-PFI-Project-Index.md | 30 +++++++++++-------- 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/ATLAS/Infrastructure/Infrastructure-PFI-Project-Index.md b/ATLAS/Infrastructure/Infrastructure-PFI-Project-Index.md index fb96661..9f61fcd 100644 --- a/ATLAS/Infrastructure/Infrastructure-PFI-Project-Index.md +++ b/ATLAS/Infrastructure/Infrastructure-PFI-Project-Index.md @@ -23,8 +23,11 @@ keywords: - PFI-ANA_DOCKER - inference-server - VM-102 +- RTX-6000-Ada +- EPYC-9254 +- hardware-profile links: [] -modified: '2026-04-11T01:49:39+00:00' +modified: '2026-04-11T06:15:40+00:00' summary: Comprehensive architecture reference for the Infrastructure-PFI project covering three Tailscale-linked sites, 11 Proxmox VMs at PFI-ANA colo, 6 Docker services with GPU workloads, and llama-swap model management across 20 models. @@ -59,17 +62,19 @@ All sites are linked via Tailscale VPN (VM 106 at the colo). ### PFI-ANA Colo — Host Reference -| Host | IP | Type | Role | -|---|---|---|---| -| **ana-ml2** (Bare Metal) | 10.250.50.54 | Physical server | Main inferencing server — direct GPU access, runs inference workloads | -| **VM 102** (PFI-ANA_DOCKER) | 10.250.50.70 | Proxmox VM | Docker host — llama-swap, ComfyUI, VibeVoice, Parakeet, ChromaDB, Dockge | -| **VM 100** (PFI-ANA-TRUENAS) | — | Proxmox VM | NAS/iSCSI storage | -| **VM 101** (PFI-ANA-DC) | — | Proxmox VM | Windows Domain Controller | -| **VM 104** (PFI-Mongo) | 10.250.50.81 | Proxmox VM | MongoDB server | -| **VM 106** (PFI-Tailscale) | — | Proxmox VM | VPN gateway (Tailscale) | +| Host | IP | Type | Hardware | Role | +|---|---|---|---|---| +| **ana-ml2** (Bare Metal) | 10.250.50.54 | Physical server | 2× EPYC 9254 (48c/96t), 566 GB RAM, 2× RTX 6000 Ada (~90 GB VRAM) | Main inferencing server — direct GPU access | +| **VM 102** (PFI-ANA_DOCKER) | 10.250.50.70 | Proxmox VM | 8 vCPU, 16 GB RAM, 250G disk | Docker host — llama-swap, ComfyUI, VibeVoice, Parakeet, ChromaDB, Dockge | +| **VM 100** (PFI-ANA-TRUENAS) | — | Proxmox VM | 4 vCPU, 8 GB RAM, 80G disk | NAS/iSCSI storage | +| **VM 101** (PFI-ANA-DC) | — | Proxmox VM | 12 vCPU, 24 GB RAM, 240G disk | Windows Domain Controller | +| **VM 104** (PFI-Mongo) | 10.250.50.81 | Proxmox VM | 8 vCPU, 8 GB RAM | MongoDB server | +| **VM 106** (PFI-Tailscale) | — | Proxmox VM | 8 vCPU, 2 GB RAM | VPN gateway (Tailscale) | The Proxmox hypervisor management interface is at `10.250.250.31:8006`. +> **Hardware Profile:** See [Hardware Profile: ana-ml2](ATLAS/Infrastructure/Infrastructure-PFI-Hardware-Profile-ana-ml2.md) for the full bare metal hardware specification (CPU, GPU, memory, software stack, inference capacity estimates). + --- ## File & Directory Map @@ -112,7 +117,8 @@ Infrastructure-PFI/ │ ├── proxmox-vms.md # 11 VMs, startup order, resource summary, network notes │ ├── chromadb-setup.md # ChromaDB deployment guide, auth, monitoring │ ├── model-list # GGUF inventory with ACTIVE/PENDING/REMOVED status -│ └── recommended-model-settings.md # Canonical inference settings for 5 model families +│ ├── recommended-model-settings.md # Canonical inference settings for 5 model families +│ └── hardware-profile-ana-ml2.md # ✅ Bare metal hardware spec (EPYC, RTX 6000 Ada, VRAM, etc.) ├── scripts/ │ ├── nemotron-query.sh # Bash client for Nemotron via llama-swap API │ ├── setup-chromadb.sh # Deploy ChromaDB on VM 102 @@ -255,7 +261,7 @@ A comprehensive system hardware profiler designed for AI/ML inference capacity p | `--quick` | One-line summary | | `--gpu-only` | Full report without Docker sections | -> **⚠️ Action needed:** Run this script on **ana-ml2** (bare metal server at 10.250.50.54) to capture the **physical GPU model, VRAM capacity, and host hardware specs**, which are currently undocumented. The output (especially `--json`) should be saved to close Known Issue #6. Optionally also run on VM 102 (10.250.50.70) for passthrough comparison. +> **✅ Completed:** Hardware profile captured for ana-ml2 on 2026-04-10. See [Hardware Profile: ana-ml2](ATLAS/Infrastructure/Infrastructure-PFI-Hardware-Profile-ana-ml2.md). Optionally run on VM 102 (10.250.50.70) for passthrough comparison. --- @@ -266,7 +272,7 @@ A comprehensive system hardware profiler designed for AI/ML inference capacity p 3. **Granite 4 Small** has `ttl: 0` (persistent caching) but is not assigned to the utility group 4. **Parakeet STT** references `.env` configuration not tracked in version control 5. **Qwen3-Coder-Next** download may be incomplete — `.incomplete` files present on disk -6. **Physical GPU model and VRAM undocumented** — the physical GPU(s) in the bare metal inferencing server ana-ml2 at 10.250.50.54 (and those passed through to VM 102 at 10.250.50.70) are not recorded anywhere in project documentation. Run `inference-hardware-profile.sh --json` on ana-ml2 and save the output to close this gap. +6. ~~**Physical GPU model and VRAM undocumented**~~ — **✅ RESOLVED (2026-04-10):** Full hardware profile captured via `inference-hardware-profile.sh`. ana-ml2 has **2× NVIDIA RTX 6000 Ada** (~90 GB combined VRAM) on dual EPYC 9254 with 566 GB RAM. See [Hardware Profile: ana-ml2](ATLAS/Infrastructure/Infrastructure-PFI-Hardware-Profile-ana-ml2.md). Remaining action: run profile on VM 102 (10.250.50.70) to document passthrough allocation. ---