vh/VuongKB
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update: ATLAS/Infrastructure/Infrastructure-PFI-Project-Index.md — Add hardware profile document reference to file tree, enrich Host Reference table with captured specs (dual EPYC 9254, 566 GB RAM, 2× RTX 6000 Ada), update Known Issue #6 as resolved, and update action note since hardware profile has been captured.

Browse Source
This commit is contained in:
Atlas
2026-04-10 23:15:40 -07:00
parent 11dfa9d225
commit 5c48c059da
1 changed files with 18 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ATLAS/Infrastructure/Infrastructure-PFI-Project-Index.md
+18 -12
View File
@@ -23,8 +23,11 @@ keywords:
- PFI-ANA_DOCKER - PFI-ANA_DOCKER
- inference-server - inference-server
- VM-102 - VM-102
- RTX-6000-Ada
- EPYC-9254
- hardware-profile
links: [] links: []
modified: '2026-04-11T01:49:39+00:00' modified: '2026-04-11T06:15:40+00:00'
summary: Comprehensive architecture reference for the Infrastructure-PFI project covering summary: Comprehensive architecture reference for the Infrastructure-PFI project covering
three Tailscale-linked sites, 11 Proxmox VMs at PFI-ANA colo, 6 Docker services three Tailscale-linked sites, 11 Proxmox VMs at PFI-ANA colo, 6 Docker services
with GPU workloads, and llama-swap model management across 20 models. with GPU workloads, and llama-swap model management across 20 models.
@@ -59,17 +62,19 @@ All sites are linked via Tailscale VPN (VM 106 at the colo).
### PFI-ANA Colo — Host Reference ### PFI-ANA Colo — Host Reference
| Host | IP | Type | Role | | Host | IP | Type | Hardware | Role |
|---|---|---|---| |---|---|---|---|---|
| **ana-ml2** (Bare Metal) | 10.250.50.54 | Physical server | Main inferencing server — direct GPU access, runs inference workloads | | **ana-ml2** (Bare Metal) | 10.250.50.54 | Physical server | 2× EPYC 9254 (48c/96t), 566 GB RAM, 2× RTX 6000 Ada (~90 GB VRAM) | Main inferencing server — direct GPU access |
| **VM 102** (PFI-ANA_DOCKER) | 10.250.50.70 | Proxmox VM | Docker host — llama-swap, ComfyUI, VibeVoice, Parakeet, ChromaDB, Dockge | | **VM 102** (PFI-ANA_DOCKER) | 10.250.50.70 | Proxmox VM | 8 vCPU, 16 GB RAM, 250G disk | Docker host — llama-swap, ComfyUI, VibeVoice, Parakeet, ChromaDB, Dockge |
| **VM 100** (PFI-ANA-TRUENAS) | — | Proxmox VM | NAS/iSCSI storage | | **VM 100** (PFI-ANA-TRUENAS) | — | Proxmox VM | 4 vCPU, 8 GB RAM, 80G disk | NAS/iSCSI storage |
| **VM 101** (PFI-ANA-DC) | — | Proxmox VM | Windows Domain Controller | | **VM 101** (PFI-ANA-DC) | — | Proxmox VM | 12 vCPU, 24 GB RAM, 240G disk | Windows Domain Controller |
| **VM 104** (PFI-Mongo) | 10.250.50.81 | Proxmox VM | MongoDB server | | **VM 104** (PFI-Mongo) | 10.250.50.81 | Proxmox VM | 8 vCPU, 8 GB RAM | MongoDB server |
| **VM 106** (PFI-Tailscale) | — | Proxmox VM | VPN gateway (Tailscale) | | **VM 106** (PFI-Tailscale) | — | Proxmox VM | 8 vCPU, 2 GB RAM | VPN gateway (Tailscale) |
The Proxmox hypervisor management interface is at `10.250.250.31:8006`. The Proxmox hypervisor management interface is at `10.250.250.31:8006`.
> **Hardware Profile:** See [Hardware Profile: ana-ml2](ATLAS/Infrastructure/Infrastructure-PFI-Hardware-Profile-ana-ml2.md) for the full bare metal hardware specification (CPU, GPU, memory, software stack, inference capacity estimates).
--- ---
## File & Directory Map ## File & Directory Map
@@ -112,7 +117,8 @@ Infrastructure-PFI/
│ ├── proxmox-vms.md # 11 VMs, startup order, resource summary, network notes │ ├── proxmox-vms.md # 11 VMs, startup order, resource summary, network notes
│ ├── chromadb-setup.md # ChromaDB deployment guide, auth, monitoring │ ├── chromadb-setup.md # ChromaDB deployment guide, auth, monitoring
│ ├── model-list # GGUF inventory with ACTIVE/PENDING/REMOVED status │ ├── model-list # GGUF inventory with ACTIVE/PENDING/REMOVED status
│ └── recommended-model-settings.md # Canonical inference settings for 5 model families │ ├── recommended-model-settings.md # Canonical inference settings for 5 model families
│ └── hardware-profile-ana-ml2.md # ✅ Bare metal hardware spec (EPYC, RTX 6000 Ada, VRAM, etc.)
├── scripts/ ├── scripts/
│ ├── nemotron-query.sh # Bash client for Nemotron via llama-swap API │ ├── nemotron-query.sh # Bash client for Nemotron via llama-swap API
│ ├── setup-chromadb.sh # Deploy ChromaDB on VM 102 │ ├── setup-chromadb.sh # Deploy ChromaDB on VM 102
@@ -255,7 +261,7 @@ A comprehensive system hardware profiler designed for AI/ML inference capacity p
| `--quick` | One-line summary | | `--quick` | One-line summary |
| `--gpu-only` | Full report without Docker sections | | `--gpu-only` | Full report without Docker sections |
> **⚠️ Action needed:** Run this script on **ana-ml2** (bare metal server at 10.250.50.54) to capture the **physical GPU model, VRAM capacity, and host hardware specs**, which are currently undocumented. The output (especially `--json`) should be saved to close Known Issue #6. Optionally also run on VM 102 (10.250.50.70) for passthrough comparison. > **✅ Completed:** Hardware profile captured for ana-ml2 on 2026-04-10. See [Hardware Profile: ana-ml2](ATLAS/Infrastructure/Infrastructure-PFI-Hardware-Profile-ana-ml2.md). Optionally run on VM 102 (10.250.50.70) for passthrough comparison.
--- ---
@@ -266,7 +272,7 @@ A comprehensive system hardware profiler designed for AI/ML inference capacity p
3. **Granite 4 Small** has `ttl: 0` (persistent caching) but is not assigned to the utility group 3. **Granite 4 Small** has `ttl: 0` (persistent caching) but is not assigned to the utility group
4. **Parakeet STT** references `.env` configuration not tracked in version control 4. **Parakeet STT** references `.env` configuration not tracked in version control
5. **Qwen3-Coder-Next** download may be incomplete — `.incomplete` files present on disk 5. **Qwen3-Coder-Next** download may be incomplete — `.incomplete` files present on disk
6. **Physical GPU model and VRAM undocumented** — the physical GPU(s) in the bare metal inferencing server ana-ml2 at 10.250.50.54 (and those passed through to VM 102 at 10.250.50.70) are not recorded anywhere in project documentation. Run `inference-hardware-profile.sh --json` on ana-ml2 and save the output to close this gap. 6. ~~**Physical GPU model and VRAM undocumented**~~ — **✅ RESOLVED (2026-04-10):** Full hardware profile captured via `inference-hardware-profile.sh`. ana-ml2 has **2× NVIDIA RTX 6000 Ada** (~90 GB combined VRAM) on dual EPYC 9254 with 566 GB RAM. See [Hardware Profile: ana-ml2](ATLAS/Infrastructure/Infrastructure-PFI-Hardware-Profile-ana-ml2.md). Remaining action: run profile on VM 102 (10.250.50.70) to document passthrough allocation.
--- ---