Files

T

vh 60367b307f servers: add new host dirs, refresh fleet snapshots, orientation doc

Bundles the inventory expansion since 2026-04-22:

- New host dirs (READMEs + ssh-target where dir name doesn't resolve):
    ana-nas, ana-wg, esh-vm-db, nh3-nas, pbs-ana, pbs-nh3.
- New PFI VM snapshots (registered + key-installed 2026-04-23):
    ana-filebot, pfi-ana-webhost, pfi-postgres, pfi-pteradactyl,
    pfi-tacticalrmm, sf-ana-container, sfsrv-ana (system + proxmox).
- servers/irv-ml1: ONBOARDING.md (the first-time setup notes from when
  the host was brought into the fleet) + ssh-target (10.100.79.3 over
  the WG tunnel — name doesn't DNS-resolve from this workstation).
- servers/{ana-ml2,pfi-pve,sf-r630}/README.md: updates to capture BMC
  IPs, the iDRAC vs OS hostname distinction (sf-r630 hardware =
  sfsrv-ana OS), and the ana-ml2 Supermicro BMC (10.250.250.50,
  distinct from the Dell R750xs iDRAC).
- configs/homepage/docker.yaml: irv-ml1-docker provider added so
  homepage auto-discovers irv-ml1's stacks over the WG tunnel.
- docs/orientation.md: narrative fleet overview written for fresh
  Claude sessions — sites, backup architecture, governing principles,
  gotchas, where-to-look guide. Pointed at from CLAUDE.md.

2026-04-24 21:56:46 -07:00

2.6 KiB

Raw Blame History

ana-ml2

Primary AI inference host for PFI.

Network

LAN IP: 10.250.50.54 (in-band, OS-side)
BMC (OOB): 10.250.250.50 — Supermicro IPMI web UI at https://10.250.250.50 (homepage card: PFI-ANA-ML2 BMC)
SSH: standard port 22 on 10.250.50.54

Hardware

Chassis: Supermicro mid-range inferencing server (bare metal, NOT Dell / not the same box as sf-r630 / sfsrv-ana)
CPU: AMD EPYC 9254 24-core (96 threads)
RAM: 566 GB
GPUs: 2x NVIDIA RTX 6000 Ada Generation (46 GB VRAM each, GPU 0 and GPU 1)
Storage: ZFS zroot (434 GB root) + tank pool (8.6 TB at /tank)
OS: Debian 13 (trixie), kernel 6.12.x
Docker: 29.3.1, runtimes: runc (default), nvidia, io.containerd.runc.v2

Key paths

Path	Purpose
`/opt/docker/compose/<stack>/`	Compose files
`/opt/docker/conf/<stack>/`	Config bind mounts
`/tank/aimodels/huggingface/`	HF cache (267 GB, pre-downloaded models)
`/tank/aimodels/llm/`	Legacy GGUF models (790 GB, referenced by llama-swap as `/models/`)
`/var/lib/docker/`	Docker data (on zroot)

Running stacks

Stack	Port	Notes
llama-swap	9292	GGUF model server via llama.cpp
vllm-embed (Qwen3)	8001	OpenAI-compatible embeddings; part of the `vllm-qwen3` stack (GPU 1)
vllm-rerank (Qwen3)	8002	OpenAI-compatible reranker; part of the `vllm-qwen3` stack (GPU 1)
dockge	5001	Docker stack management UI
dozzle-agent	7007	Log agent; reports to the Dozzle hub on ana-docker
beszel-agent	45876	Metrics agent; reports to the Beszel hub on ana-docker

Retired since last README update:

infinity — replaced by vllm-qwen3 after the upstream image stopped shipping a transformers build that knew Qwen3.
LibreChat (+ rag_api, vectordb, mongodb, meilisearch) — removed from this host.
searxng — now hosted on ana-docker for the whole fleet.
Residual networks (librechat_default, kokoro-tts-gpu_default) from prior experiments are still present; safe to docker network rm at leisure.

Refresh state

scripts/refresh-server-info.sh ana-ml2

Latest snapshot: system-details.txt (regenerate as needed).

GPU allocation policy

By default, no container is pinned. For predictable performance when multiple GPU workloads run concurrently:

GPU 0: heavy LLM (llama-swap big models).
GPU 1: light services (both vllm-qwen3 services share this GPU via --gpu-memory-utilization).

Use deploy.resources.reservations.devices[].device_ids: ["<id>"] in compose to pin.

2.6 KiB Raw Blame History