Files

T

vh e376d0aec9 Initial commit: PFI fleet inventory, stacks, tooling, and backup pipeline

Captures the full workspace state built up to this point:

  - CLAUDE.md + README.md describing conventions and the four-host fleet
    (ana-ml2, ana-docker, nh3-docker, esh-docker-vm).
  - Per-host notes under servers/<host>/ with ssh-target fallback files
    and latest system-details snapshots (two in-compose credential leaks
    scrubbed; the upstream compose files still need to move those to .env).
  - scripts/: server_inspect.sh (read-only remote diagnostic),
    refresh-server-info.sh (dir-driven discovery + snapshot capture with
    validation warnings), add-host.sh, sync-stacks.sh (pull
    compose/conf trees), deploy-stack.sh (push with per-file diff + prompt).
  - stacks/: canonical compose for backrest, beszel, dozzle, llama-swap,
    rest-server-ana, rest-server-nh3, vllm-qwen3, plus the retired
    infinity reference. All use the .env-driven + traefik-net + homepage
    label pattern.
  - configs/restic/ana-docker/: first resticprofile config + pre-backup
    hook (Synapse pg_dump, Seafile mysqldump, Vaultwarden SQLite); templates
    for the other three hosts to come.
  - docs/pfi/: general infrastructure reference carried over.
  - .gitignore excludes .env, stacks-mirror/, and assorted secret/state
    filenames to prevent re-leaks on later commits.

2026-04-20 14:29:48 -07:00

README.md

Initial commit: PFI fleet inventory, stacks, tooling, and backup pipeline

2026-04-20 14:29:48 -07:00

ssh-target

Initial commit: PFI fleet inventory, stacks, tooling, and backup pipeline

2026-04-20 14:29:48 -07:00

system-details.txt

Initial commit: PFI fleet inventory, stacks, tooling, and backup pipeline

2026-04-20 14:29:48 -07:00

README.md

ana-ml2

Primary AI inference host for PFI.

Network

LAN IP: 10.250.50.54
SSH: standard port 22

Hardware

CPU: AMD EPYC 9254 24-core (96 threads)
RAM: 566 GB
GPUs: 2x NVIDIA RTX 6000 Ada Generation (46 GB VRAM each, GPU 0 and GPU 1)
Storage: ZFS zroot (434 GB root) + tank pool (8.6 TB at /tank)
OS: Debian 13 (trixie), kernel 6.12.x
Docker: 29.3.1, runtimes: runc (default), nvidia, io.containerd.runc.v2

Key paths

Path	Purpose
`/opt/docker/compose/<stack>/`	Compose files
`/opt/docker/conf/<stack>/`	Config bind mounts
`/tank/aimodels/huggingface/`	HF cache (267 GB, pre-downloaded models)
`/tank/aimodels/llm/`	Legacy GGUF models (790 GB, referenced by llama-swap as `/models/`)
`/var/lib/docker/`	Docker data (on zroot)

Running stacks

Stack	Port	Notes
llama-swap	9292	GGUF model server via llama.cpp
vllm-embed (Qwen3)	8001	OpenAI-compatible embeddings; part of the `vllm-qwen3` stack (GPU 1)
vllm-rerank (Qwen3)	8002	OpenAI-compatible reranker; part of the `vllm-qwen3` stack (GPU 1)
dockge	5001	Docker stack management UI
dozzle-agent	7007	Log agent; reports to the Dozzle hub on ana-docker
beszel-agent	45876	Metrics agent; reports to the Beszel hub on ana-docker

Retired since last README update:

infinity — replaced by vllm-qwen3 after the upstream image stopped shipping a transformers build that knew Qwen3.
LibreChat (+ rag_api, vectordb, mongodb, meilisearch) — removed from this host.
searxng — now hosted on ana-docker for the whole fleet.
Residual networks (librechat_default, kokoro-tts-gpu_default) from prior experiments are still present; safe to docker network rm at leisure.

Refresh state

scripts/refresh-server-info.sh ana-ml2

Latest snapshot: system-details.txt (regenerate as needed).

GPU allocation policy

By default, no container is pinned. For predictable performance when multiple GPU workloads run concurrently:

GPU 0: heavy LLM (llama-swap big models).
GPU 1: light services (both vllm-qwen3 services share this GPU via --gpu-memory-utilization).

Use deploy.resources.reservations.devices[].device_ids: ["<id>"] in compose to pin.