--- created: '2026-04-10T17:35:04+00:00' id: 850d5cfb-1b10-42cb-a3ad-fe4066c8cb17 keywords: - docker - vm - full - project - chromadb - models - model - reference - compose.yaml - vcpu - gb - gpu - network - llama-swap - bare-metal - 10.250.50.54 - 10.250.50.70 - ana-ml2 - PFI-ANA_DOCKER - inference-server - VM-102 links: [] modified: '2026-04-11T01:33:45+00:00' summary: Comprehensive architecture reference for the Infrastructure-PFI project covering three Tailscale-linked sites, 11 Proxmox VMs at PFI-ANA colo, 6 Docker services with GPU workloads, and llama-swap model management across 20 models. tags: - infrastructure - pfi - docker - proxmox - colocation - reference - hardware-audit title: Infrastructure-PFI — Full Project Index & Architecture Reference --- # Infrastructure-PFI — Full Project Index & Architecture Reference ## Overview Infrastructure-PFI is the configuration and documentation project for the PFI network environment. It covers three physical sites interconnected via Tailscale VPN, hosting a comprehensive infrastructure of Proxmox VMs, Docker services, and AI/ML workloads. --- ## Sites | Site | Network | Hypervisor | Docker Host | Network Gear | |---|---|---|---|---| | **ESH-Home** | 10.0.0.0/16 | Proxmox VE | VM at 10.0.50.45 | Ubiquiti | | **PFI-NH3** | 10.100.0.0/16 | Proxmox VE | Proxmox VM | Ubiquiti | | **PFI-ANA (Colo)** | 10.250.0.0/16 | Proxmox VE at 10.250.250.31:8006 | Bare metal at 10.250.50.54 · VM 102 (10.250.50.70, ana-ml2) | FortiGate | All sites are linked via Tailscale VPN (VM 106 at the colo). ### PFI-ANA Colo — Host Reference | Host | IP | Type | Role | |---|---|---|---| | **Bare Metal** | 10.250.50.54 | Physical server | Main inferencing server — direct GPU access, runs inference workloads | | **VM 102** (PFI-ANA_DOCKER) | 10.250.50.70 | Proxmox VM | Docker host — llama-swap, ComfyUI, VibeVoice, Parakeet, ChromaDB, Dockge | | **VM 100** (PFI-ANA-TRUENAS) | — | Proxmox VM | NAS/iSCSI storage | | **VM 101** (PFI-ANA-DC) | — | Proxmox VM | Windows Domain Controller | | **VM 104** (PFI-Mongo) | 10.250.50.81 | Proxmox VM | MongoDB server | | **VM 106** (PFI-Tailscale) | — | Proxmox VM | VPN gateway (Tailscale) | The Proxmox hypervisor management interface is at `10.250.250.31:8006`. --- ## File & Directory Map ``` Infrastructure-PFI/ ├── README.md # Project overview, site summary, structure ├── configs/pfi-ana/ │ ├── proxmox/ # Raw Proxmox VM configs (11 VMs) │ │ ├── vm-100.conf → PFI-ANA-TRUENAS (TrueNAS SCALE, 4 vCPU, 8GB, 80G disk) │ │ ├── vm-101.conf → PFI-ANA-DC (Windows 11 DC, 12 vCPU, 24GB, 240G, UEFI+TPM) │ │ ├── vm-102.conf → PFI-ANA-Docker (Linux, 8 vCPU, 16GB, 250G — the Docker host) │ │ ├── vm-103.conf → PFI-SlaveBot (Windows 10, 8 vCPU, 8GB, 256G) │ │ ├── vm-104.conf → PFI-Mongo (Linux, 8 vCPU, 8GB, 256G, MongoDB) │ │ ├── vm-105.conf → PFI-Postgres (Debian 11, 16 vCPU, 8GB, 80G, PostgreSQL) │ │ ├── vm-106.conf → PFI-Tailscale (Debian 12, 8 vCPU, 2GB, 256G — VPN gateway) │ │ ├── vm-107.conf → PFI-Pterodactyl (Debian 11, 8 vCPU, 8GB, 256G — game servers) │ │ ├── vm-108.conf → PFI-ANA--DEV (Debian 11, 8 vCPU, 8GB, 120G — dev env) │ │ ├── vm-110.conf → PFI-ANA-Webhost (Debian 11, 16 vCPU, 4GB, 250G — web hosting) │ │ └── vm-111.conf → pfi-tacticalrmm (Linux, 16 vCPU, 8GB, 256G — RMM) │ └── docker/ │ ├── compose/ # Live Docker Compose stacks (managed by Dockge) │ │ ├── llama-swap/compose.yaml → LLM gateway (port 9292, CUDA) │ │ ├── dockge/compose.yaml → Stack manager (port 5001) │ │ ├── comfyui/compose.yaml → Image gen (port 8188, GPU) │ │ ├── chromadb/compose.yaml → Vector DB (port 8000, token auth) │ │ ├── vibevoice/compose.yaml → TTS (port 8745, GPU) │ │ └── parakeet/compose.yaml → STT (port 8300, GPU, local build) │ ├── conf/ │ │ └── llama-swap/config.yaml # 20 model definitions, 3 groups │ └── compose-examples/ # Reference compose files (not live) │ ├── llama-swap/docker-compose.yml │ └── chromadb/ # ChromaDB with Traefik labels + demo app │ ├── docker-compose.yml │ ├── docker-compose.demo.yml │ ├── Dockerfile.demo │ └── scripts/demo.py ├── docs/pfi-ana/ │ ├── docker-stack.md # Docker conventions, 6 services, port map, ASCII diagram │ ├── proxmox-vms.md # 11 VMs, startup order, resource summary, network notes │ ├── chromadb-setup.md # ChromaDB deployment guide, auth, monitoring │ ├── model-list # GGUF inventory with ACTIVE/PENDING/REMOVED status │ └── recommended-model-settings.md # Canonical inference settings for 5 model families ├── scripts/ │ ├── nemotron-query.sh # Bash client for Nemotron via llama-swap API │ ├── setup-chromadb.sh # Deploy ChromaDB on VM 102 │ ├── backup-chromadb.sh # Hot backup with 30-day retention │ ├── health-check-chromadb.sh # Health monitoring with alerting │ ├── quickstart-chromadb.sh # One-command ChromaDB setup wrapper │ └── inference-hardware-profile.sh # Hardware profiler for AI/ML inference (bare metal 10.250.50.54) └── output/ # Generated output and logs (empty) ``` --- ## Proxmox VM Startup Order Ordered startup is configured to ensure dependencies are met before dependent services start. | Order | VMID | Name | Delay | Onboot | |---|---|---|---|---| | 2 | 100 | TRUENAS | 120s | No | | 3 | 104 | Mongo | 60s | Yes | | 3 | 105 | Postgres | 60s | Yes | | 4 | 102 | Docker | — | Yes | | 5 | 101 | DC (Windows) | 120s | Yes | | 10 | 108 | DEV | 60s | No | | 20 | 111 | Tactical RMM | — | Yes | | 30 | 110 | Webhost | 120s | Yes | **Note:** VMs 103 (SlaveBot), 106 (Tailscale), and 107 (Pterodactyl) start via `onboot` but without ordered delays. --- ## Docker Services (VM 102: ana-ml2) Six services run on the Docker host, with most leveraging GPU acceleration for AI workloads. | Service | Port | GPU | Image | |---|---|---|---| | Dockge | 5001 | No | `louislam/dockge:latest` | | llama-swap | 9292→8080 | Yes (nvidia) | `ghcr.io/mostlygeek/llama-swap:cuda` | | ComfyUI | 8188 | Yes | `mmartial/comfyui-nvidia-docker` | | ChromaDB | 8000 | No | `chromadb/chroma:latest` | | VibeVoice | 8745 | Yes | `eworkerinc/vibevoice:latest` | | Parakeet STT | 8300→8000 | Yes | `parakeet-stt` (local build) | **Network Note:** All Docker services run on the `traefik-net` external network. --- ## llama-swap Model Inventory The llama-swap gateway manages **20 GGUF models** across multiple categories: ### By Category - **Qwen 3.5**: 4 models - **Nemotron**: 2 models - **Gemma 4**: 2 models - **GLM**: 2 models - **Skyfall**: 2 models - **Coder**: 1 model - **Large/Special**: 1 model - **Granite**: 2 models - **Embeddings**: 2 models - **Reranking**: 1 model - **Pending**: 1 (Qwen3-Coder-Next download) ### Named Groups - **high-reasoning**: 4 models (primary reasoning workloads) - **heavy-moe**: 4 models (Mixture-of-Expert architectures) - **utility**: 3 persistent models (common utility tasks) --- ## Storage Layout on VM 102 | Host Path | Purpose | |---|---| | `/opt/docker/compose//` | Per-service Docker Compose files | | `/opt/docker/conf//` | Per-service configuration files | | `/tank/` | Large persistent data (AI models, images, voice data) | | `/tank/aimodels/llm/` | GGUF model files (mounted as `/models` in llama-swap) | | `/tank/aimodels/img/comfy/` | Image generation models for ComfyUI | | `/tank/chromadb/` | ChromaDB persistent vector data | | `/tank/vibevoice/` | VibeVoice Hugging Face cache, voices, state | | `/opt/docker/backups/chromadb/` | ChromaDB backup archives | --- ## Network Configuration - **VLAN**: All VMs on VLAN 50 via `vmbr0` - **NIC Type**: - Linux VMs: `virtio` - Windows VMs: `e1000` - **Docker Network**: Services on external network `traefik-net` - **Docker Host IP**: `10.250.50.70` (VM 102 / ana-ml2 / PFI-ANA_DOCKER) - **Bare Metal Inference Server IP**: `10.250.50.54` (physical server, direct GPU access) --- ## Scripts Reference ### ChromaDB Scripts (VM 102) | Script | Purpose | |---|---| | `setup-chromadb.sh` | Deploy ChromaDB container on VM 102 | | `backup-chromadb.sh` | Hot backup with 30-day retention | | `health-check-chromadb.sh` | Health monitoring with alerting | | `quickstart-chromadb.sh` | One-command ChromaDB setup wrapper | ### Utility Scripts | Script | Purpose | |---|---| | `nemotron-query.sh` | Bash client for querying Nemotron via llama-swap API | | `inference-hardware-profile.sh` | Comprehensive hardware profiler for AI/ML inference planning | ### inference-hardware-profile.sh **Target Hosts:** - **Bare metal inferencing server** at `10.250.50.54` — primary target (direct GPU access) - **VM 102 (PFI-ANA_DOCKER / ana-ml2)** at `10.250.50.70` — secondary target (GPU passthrough) A comprehensive system hardware profiler designed for AI/ML inference capacity planning. Captures the full hardware picture needed to evaluate and optimize GPU inference workloads. **What it gathers:** - System info (OS, kernel, uptime) - CPU details (model, cores, threads, clock speed, scaling governor) - Memory (total, available, used, swap) - Disk storage (all mounts) - GPU resources (model, VRAM, utilization, power, temperature, PCIe, clock speeds, throttling, ECC, active processes, MIG mode) - Docker container stats (CPU/memory per container) - Inference software stack (Python, PyTorch, CUDA, cuDNN, llama.cpp, Ollama, vLLM) - Estimated model capacity based on free VRAM (Q4, Q8, FP16) - Inference readiness score with actionable recommendations **Output modes:** | Flag | Output | |---|---| | *(default)* | Full colored terminal report | | `--json` | Machine-readable JSON | | `--quick` | One-line summary | | `--gpu-only` | Full report without Docker sections | > **⚠️ Action needed:** Run this script on the **bare metal server** (10.250.50.54) to capture the **physical GPU model, VRAM capacity, and host hardware specs**, which are currently undocumented. The output (especially `--json`) should be saved to close Known Issue #6. Optionally also run on VM 102 (10.250.50.70) for passthrough comparison. --- ## Known Issues & Observations 1. **Nemotron Super** uses `--n-gpu-layers 99` instead of 999 — likely configured to reserve VRAM for other tasks 2. **Kimi K2** only offloads 2 GPU layers — very CPU-heavy and slow inference 3. **Granite 4 Small** has `ttl: 0` (persistent caching) but is not assigned to the utility group 4. **Parakeet STT** references `.env` configuration not tracked in version control 5. **Qwen3-Coder-Next** download may be incomplete — `.incomplete` files present on disk 6. **Physical GPU model and VRAM undocumented** — the physical GPU(s) in the bare metal inferencing server at 10.250.50.54 (and those passed through to VM 102 at 10.250.50.70) are not recorded anywhere in project documentation. Run `inference-hardware-profile.sh --json` on the bare metal server and save the output to close this gap. --- *This document was generated following a full infrastructure audit. It serves as the primary reference for understanding, maintaining, and extending the PFI network environment.*