vh/VuongKB
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
01ed6d2dd3fe32b01793107b0ad7b5104801b38a
VuongKB/ATLAS/Infrastructure/Infrastructure-PFI-Project-Index.md
T

8.3 KiB
Raw Blame History

created, id, keywords, links, modified, summary, tags, title
created id keywords links modified summary tags title
2026-04-10T17:35:04+00:00 850d5cfb-1b10-42cb-a3ad-fe4066c8cb17
PFI
Tailscale VPN
Proxmox VE
Docker Compose
llama-swap
ChromaDB
ComfyUI
VibeVoice
Parakeet STT
VM startup order
GPU acceleration
vector database
2026-04-10T17:35:04+00:00 Comprehensive architecture reference for the Infrastructure-PFI project covering three Tailscale-linked sites, 11 Proxmox VMs at PFI-ANA colo, 6 Docker services with GPU workloads, and llama-swap model management across 20 models.
infrastructure
pfi
docker
proxmox
colocation
reference
Infrastructure-PFI — Full Project Index & Architecture Reference

Infrastructure-PFI — Full Project Index & Architecture Reference

Overview

Infrastructure-PFI is the configuration and documentation project for the PFI network environment. It covers three physical sites interconnected via Tailscale VPN, hosting a comprehensive infrastructure of Proxmox VMs, Docker services, and AI/ML workloads.

Sites

Site Network Hypervisor Docker Host Network Gear
ESH-Home 10.0.0.0/16 Proxmox VE VM at 10.0.50.45 Ubiquiti
PFI-NH3 10.100.0.0/16 Proxmox VE Proxmox VM Ubiquiti
PFI-ANA (Colo) 10.250.0.0/16 Proxmox VE at 10.250.250.31:8006 VM 102 (10.250.50.54, ana-ml2) FortiGate

All sites are linked via Tailscale VPN (VM 106 at the colo).

File & Directory Map

Infrastructure-PFI/
├── README.md                              # Project overview, site summary, structure
├── configs/pfi-ana/
│   ├── proxmox/                           # Raw Proxmox VM configs (11 VMs)
│   │   ├── vm-100.conf  → PFI-ANA-TRUENAS (TrueNAS SCALE, 4 vCPU, 8GB, 80G disk)
│   │   ├── vm-101.conf  → PFI-ANA-DC (Windows 11 DC, 12 vCPU, 24GB, 240G, UEFI+TPM)
│   │   ├── vm-102.conf  → PFI-ANA-Docker (Linux, 8 vCPU, 16GB, 250G — the Docker host)
│   │   ├── vm-103.conf  → PFI-SlaveBot (Windows 10, 8 vCPU, 8GB, 256G)
│   │   ├── vm-104.conf  → PFI-Mongo (Linux, 8 vCPU, 8GB, 256G, MongoDB)
│   │   ├── vm-105.conf  → PFI-Postgres (Debian 11, 16 vCPU, 8GB, 80G, PostgreSQL)
│   │   ├── vm-106.conf  → PFI-Tailscale (Debian 12, 8 vCPU, 2GB, 256G — VPN gateway)
│   │   ├── vm-107.conf  → PFI-Pterodactyl (Debian 11, 8 vCPU, 8GB, 256G — game servers)
│   │   ├── vm-108.conf  → PFI-ANA--DEV (Debian 11, 8 vCPU, 8GB, 120G — dev env)
│   │   ├── vm-110.conf  → PFI-ANA-Webhost (Debian 11, 16 vCPU, 4GB, 250G — web hosting)
│   │   └── vm-111.conf  → pfi-tacticalrmm (Linux, 16 vCPU, 8GB, 256G — RMM)
│   └── docker/
│       ├── compose/                       # Live Docker Compose stacks (managed by Dockge)
│       │   ├── llama-swap/compose.yaml    → LLM gateway (port 9292, CUDA)
│       │   ├── dockge/compose.yaml        → Stack manager (port 5001)
│       │   ├── comfyui/compose.yaml       → Image gen (port 8188, GPU)
│       │   ├── chromadb/compose.yaml      → Vector DB (port 8000, token auth)
│       │   ├── vibevoice/compose.yaml     → TTS (port 8745, GPU)
│       │   └── parakeet/compose.yaml      → STT (port 8300, GPU, local build)
│       ├── conf/
│       │   └── llama-swap/config.yaml     # 20 model definitions, 3 groups
│       └── compose-examples/              # Reference compose files (not live)
│           ├── llama-swap/docker-compose.yml
│           └── chromadb/                  # ChromaDB with Traefik labels + demo app
│               ├── docker-compose.yml
│               ├── docker-compose.demo.yml
│               ├── Dockerfile.demo
│               └── scripts/demo.py
├── docs/pfi-ana/
│   ├── docker-stack.md                    # Docker conventions, 6 services, port map, ASCII diagram
│   ├── proxmox-vms.md                     # 11 VMs, startup order, resource summary, network notes
│   ├── chromadb-setup.md                  # ChromaDB deployment guide, auth, monitoring
│   ├── model-list                         # GGUF inventory with ACTIVE/PENDING/REMOVED status
│   └── recommended-model-settings.md      # Canonical inference settings for 5 model families
├── scripts/
│   ├── nemotron-query.sh                  # Bash client for Nemotron via llama-swap API
│   ├── setup-chromadb.sh                  # Deploy ChromaDB on VM 102
│   ├── backup-chromadb.sh                 # Hot backup with 30-day retention
│   ├── health-check-chromadb.sh           # Health monitoring with alerting
│   └── quickstart-chromadb.sh             # One-command ChromaDB setup wrapper
└── output/                                # Generated output and logs (empty)

Proxmox VM Startup Order

Ordered startup is configured to ensure dependencies are met before dependent services start.

Order VMID Name Delay Onboot
2 100 TRUENAS 120s No
3 104 Mongo 60s Yes
3 105 Postgres 60s Yes
4 102 Docker — Yes
5 101 DC (Windows) 120s Yes
10 108 DEV 60s No
20 111 Tactical RMM — Yes
30 110 Webhost 120s Yes

Note: VMs 103 (SlaveBot), 106 (Tailscale), and 107 (Pterodactyl) start via onboot but without ordered delays.

Docker Services (VM 102: ana-ml2)

Six services run on the Docker host, with most leveraging GPU acceleration for AI workloads.

Service Port GPU Image
Dockge 5001 No louislam/dockge:latest
llama-swap 9292→8080 Yes (nvidia) ghcr.io/mostlygeek/llama-swap:cuda
ComfyUI 8188 Yes mmartial/comfyui-nvidia-docker
ChromaDB 8000 No chromadb/chroma:latest
VibeVoice 8745 Yes eworkerinc/vibevoice:latest
Parakeet STT 8300→8000 Yes parakeet-stt (local build)

Network Note: All Docker services run on the traefik-net external network.

llama-swap Model Inventory

The llama-swap gateway manages 20 GGUF models across multiple categories:

By Category

  • Qwen 3.5: 4 models
  • Nemotron: 2 models
  • Gemma 4: 2 models
  • GLM: 2 models
  • Skyfall: 2 models
  • Coder: 1 model
  • Large/Special: 1 model
  • Granite: 2 models
  • Embeddings: 2 models
  • Reranking: 1 model
  • Pending: 1 (Qwen3-Coder-Next download)

Named Groups

  • high-reasoning: 4 models (primary reasoning workloads)
  • heavy-moe: 4 models (Mixture-of-Expert architectures)
  • utility: 3 persistent models (common utility tasks)

Storage Layout on VM 102

Host Path Purpose
/opt/docker/compose/<service>/ Per-service Docker Compose files
/opt/docker/conf/<service>/ Per-service configuration files
/tank/ Large persistent data (AI models, images, voice data)
/tank/aimodels/llm/ GGUF model files (mounted as /models in llama-swap)
/tank/aimodels/img/comfy/ Image generation models for ComfyUI
/tank/chromadb/ ChromaDB persistent vector data
/tank/vibevoice/ VibeVoice Hugging Face cache, voices, state
/opt/docker/backups/chromadb/ ChromaDB backup archives

Network Configuration

  • VLAN: All VMs on VLAN 50 via vmbr0
  • NIC Type:
    • Linux VMs: virtio
    • Windows VMs: e1000
  • Docker Network: Services on external network traefik-net
  • Docker Host IP: 10.250.50.54 (ana-ml2)

Known Issues & Observations

  1. Nemotron Super uses --n-gpu-layers 99 instead of 999 — likely configured to reserve VRAM for other tasks
  2. Kimi K2 only offloads 2 GPU layers — very CPU-heavy and slow inference
  3. Granite 4 Small has ttl: 0 (persistent caching) but is not assigned to the utility group
  4. Parakeet STT references .env configuration not tracked in version control
  5. Qwen3-Coder-Next download may be incomplete — .incomplete files present on disk

This document was generated following a full infrastructure audit. It serves as the primary reference for understanding, maintaining, and extending the PFI network environment.

Reference in New Issue View Git Blame Copy Permalink