From 11dfa9d2253c77306bede17ac298896cfe20e84f Mon Sep 17 00:00:00 2001
From: Atlas <atlas@aipa.local>
Date: Fri, 10 Apr 2026 23:15:15 -0700
Subject: [PATCH] Create:
 ATLAS/Infrastructure/Infrastructure-PFI-Hardware-Profile-ana-ml2.md

---
 ...astructure-PFI-Hardware-Profile-ana-ml2.md | 136 ++++++++++++++++++
 1 file changed, 136 insertions(+)
 create mode 100644 ATLAS/Infrastructure/Infrastructure-PFI-Hardware-Profile-ana-ml2.md

diff --git a/ATLAS/Infrastructure/Infrastructure-PFI-Hardware-Profile-ana-ml2.md b/ATLAS/Infrastructure/Infrastructure-PFI-Hardware-Profile-ana-ml2.md
new file mode 100644
index 0000000..81b4ffd
--- /dev/null
+++ b/ATLAS/Infrastructure/Infrastructure-PFI-Hardware-Profile-ana-ml2.md
@@ -0,0 +1,136 @@
+---
+created: '2026-04-11T06:15:14+00:00'
+id: 65e03d42-d7cb-4404-9b55-38b72d37d51f
+keywords:
+- ana-ml2
+- 10.250.50.54
+- bare metal
+- RTX 6000 Ada
+- EPYC 9254
+- PFI-ANA
+- inference server
+- GPU
+- CUDA 12.8
+- hardware profile
+links: []
+modified: '2026-04-11T06:15:14+00:00'
+summary: Definitive hardware profile for ana-ml2, the bare metal inference server
+  at PFI-ANA colo (10.250.50.54). Dual AMD EPYC 9254, 566 GB RAM, 2× NVIDIA RTX 6000
+  Ada (~90 GB VRAM). Profiled 2026-04-10.
+tags:
+- infrastructure
+- pfi
+- pfi-ana
+- hardware
+- gpu
+- bare-metal
+- inference
+- ana-ml2
+title: 'Hardware Profile: ana-ml2 (Bare Metal Inference Server)'
+---
+
+# Hardware Profile: ana-ml2 (Bare Metal Inference Server)
+
+**Host:** ana-ml2
+**IP:** 10.250.50.54
+**Type:** Physical server (bare metal)
+**Site:** PFI-ANA (Colo)
+**Profile Date:** 2026-04-10
+**Profile Script:** `Infrastructure-PFI/scripts/inference-hardware-profile.sh`
+
+## System
+
+| Spec | Value |
+|---|---|
+| **OS** | Debian GNU/Linux 13 (trixie) |
+| **Kernel** | 6.12.41+deb13-amd64 |
+| **Architecture** | x86_64 |
+| **Uptime** | ~31 weeks |
+
+## CPU
+
+| Spec | Value |
+|---|---|
+| **Model** | AMD EPYC 9254 (2 sockets) |
+| **Physical Cores** | 48 (24 per socket) |
+| **Logical Threads** | 96 |
+| **Base Clock** | 1500 MHz |
+| **Max Boost** | 4153 MHz |
+| **L3 Cache** | 256 MiB (8 instances) |
+| **NUMA Nodes** | 2 |
+| **Scaling Governor** | `schedutil` — ⚠️ recommend switching to `performance` for consistent inference latency |
+
+## Memory
+
+| Spec | Value |
+|---|---|
+| **Total RAM** | 566.5 GB |
+| **Available** | 234.3 GB |
+| **Used** | 332.2 GB (58.6%) |
+| **Swap** | None |
+| **Type** | DDR5 (likely DDR5-5600 ECC — dmidecode parsing needs refinement for exact spec) |
+
+## GPU
+
+**2× NVIDIA RTX 6000 Ada Generation** (Compute Capability 8.9, Ada Lovelace)
+
+### GPU 0
+
+| Spec | Value |
+|---|---|
+| **VRAM Total** | 46,068 MiB (~45 GB, ~48 GB physical minus reserved) |
+| **VRAM Used** | 3,834 MiB (~3.8 GB) |
+| **VRAM Free** | 40,639 MiB (~39.7 GB) |
+| **Power Limit** | 300W / 300W max |
+| **Power Draw** | Idle (~50W) |
+| **Temperature** | 24°C |
+| **SM Clock Max** | 3105 MHz |
+| **Fan** | N/A (passive/blower) |
+
+### GPU 1
+
+| Spec | Value |
+|---|---|
+| **VRAM Total** | 46,068 MiB (~45 GB) |
+| **VRAM Used** | 4,951 MiB (~4.8 GB) |
+| **VRAM Free** | 39,522 MiB (~38.6 GB) |
+| **Power Limit** | 300W / 300W max |
+| **Temperature** | 25°C |
+| **SM Clock Max** | 3105 MHz |
+
+### Combined GPU Resources
+
+| Metric | Value |
+|---|---|
+| **Total VRAM** | ~90 GB (92,136 MiB) |
+| **Combined Free VRAM** | ~80 GB |
+| **Total Power Budget** | 600W |
+
+## Inferencing Capacity Estimates
+
+Based on ~80 GB combined free VRAM:
+
+| Quantization | Max Model Size | Examples |
+|---|---|---|
+| **Q4_K_M** | ~107B parameters | DeepSeek-V2, Mixtral 8x22B |
+| **Q8_0** | ~62B parameters | Llama 3.1 70B (tight fit) |
+| **FP16** | ~41B parameters | Llama 3.1 70B across 2 GPUs |
+
+## Software Stack
+
+| Component | Status |
+|---|---|
+| **NVIDIA Driver** | 580.65.06 |
+| **CUDA Version** | 12.8 |
+| **Docker** | Present |
+| **Python** | Present |
+
+## Known Issues
+
+1. **CPU governor** — Currently `schedutil`; switching to `performance` (`cpupower frequency-set -g performance`) recommended for inference workloads to reduce latency jitter.
+2. **Memory type** — dmidecode parsing in the profile script returns imprecise data ("2x Correction @ 5600 MT/s"). The RAM is DDR5-5600 ECC but the script needs a parsing fix to capture the exact DIMM type and count accurately.
+3. **No GPU fan data** — RTX 6000 Ada uses blower-style cooling; fan speed field returns N/A on this model.
+
+## Relationship to VM-102
+
+VM-102 (PFI-ANA_DOCKER, 10.250.50.70) is a Proxmox virtual machine running on this same bare metal server. It has GPU passthrough configured and hosts the Docker-based inference stack (llama-swap, ComfyUI, ChromaDB, etc.). Hardware resources are shared between the hypervisor and VM-102's passthrough allocations.
\ No newline at end of file