Initial commit: PFI fleet inventory, stacks, tooling, and backup pipeline
Captures the full workspace state built up to this point:
- CLAUDE.md + README.md describing conventions and the four-host fleet
(ana-ml2, ana-docker, nh3-docker, esh-docker-vm).
- Per-host notes under servers/<host>/ with ssh-target fallback files
and latest system-details snapshots (two in-compose credential leaks
scrubbed; the upstream compose files still need to move those to .env).
- scripts/: server_inspect.sh (read-only remote diagnostic),
refresh-server-info.sh (dir-driven discovery + snapshot capture with
validation warnings), add-host.sh, sync-stacks.sh (pull
compose/conf trees), deploy-stack.sh (push with per-file diff + prompt).
- stacks/: canonical compose for backrest, beszel, dozzle, llama-swap,
rest-server-ana, rest-server-nh3, vllm-qwen3, plus the retired
infinity reference. All use the .env-driven + traefik-net + homepage
label pattern.
- configs/restic/ana-docker/: first resticprofile config + pre-backup
hook (Synapse pg_dump, Seafile mysqldump, Vaultwarden SQLite); templates
for the other three hosts to come.
- docs/pfi/: general infrastructure reference carried over.
- .gitignore excludes .env, stacks-mirror/, and assorted secret/state
filenames to prevent re-leaks on later commits.
This commit is contained in:
@@ -0,0 +1,61 @@
|
||||
# ana-ml2
|
||||
|
||||
Primary AI inference host for PFI.
|
||||
|
||||
## Network
|
||||
|
||||
- **LAN IP:** 10.250.50.54
|
||||
- **SSH:** standard port 22
|
||||
|
||||
## Hardware
|
||||
|
||||
- **CPU:** AMD EPYC 9254 24-core (96 threads)
|
||||
- **RAM:** 566 GB
|
||||
- **GPUs:** 2x NVIDIA RTX 6000 Ada Generation (46 GB VRAM each, GPU 0 and GPU 1)
|
||||
- **Storage:** ZFS `zroot` (434 GB root) + `tank` pool (8.6 TB at `/tank`)
|
||||
- **OS:** Debian 13 (trixie), kernel 6.12.x
|
||||
- **Docker:** 29.3.1, runtimes: runc (default), nvidia, io.containerd.runc.v2
|
||||
|
||||
## Key paths
|
||||
|
||||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `/opt/docker/compose/<stack>/` | Compose files |
|
||||
| `/opt/docker/conf/<stack>/` | Config bind mounts |
|
||||
| `/tank/aimodels/huggingface/` | HF cache (267 GB, pre-downloaded models) |
|
||||
| `/tank/aimodels/llm/` | Legacy GGUF models (790 GB, referenced by llama-swap as `/models/`) |
|
||||
| `/var/lib/docker/` | Docker data (on zroot) |
|
||||
|
||||
## Running stacks
|
||||
|
||||
| Stack | Port | Notes |
|
||||
|-------|------|-------|
|
||||
| llama-swap | 9292 | GGUF model server via llama.cpp |
|
||||
| vllm-embed (Qwen3) | 8001 | OpenAI-compatible embeddings; part of the `vllm-qwen3` stack (GPU 1) |
|
||||
| vllm-rerank (Qwen3) | 8002 | OpenAI-compatible reranker; part of the `vllm-qwen3` stack (GPU 1) |
|
||||
| dockge | 5001 | Docker stack management UI |
|
||||
| dozzle-agent | 7007 | Log agent; reports to the Dozzle hub on ana-docker |
|
||||
| beszel-agent | 45876 | Metrics agent; reports to the Beszel hub on ana-docker |
|
||||
|
||||
**Retired since last README update:**
|
||||
- `infinity` — replaced by `vllm-qwen3` after the upstream image stopped shipping a `transformers` build that knew Qwen3.
|
||||
- `LibreChat (+ rag_api, vectordb, mongodb, meilisearch)` — removed from this host.
|
||||
- `searxng` — now hosted on ana-docker for the whole fleet.
|
||||
- Residual networks (`librechat_default`, `kokoro-tts-gpu_default`) from prior experiments are still present; safe to `docker network rm` at leisure.
|
||||
|
||||
## Refresh state
|
||||
|
||||
```bash
|
||||
scripts/refresh-server-info.sh ana-ml2
|
||||
```
|
||||
|
||||
Latest snapshot: `system-details.txt` (regenerate as needed).
|
||||
|
||||
## GPU allocation policy
|
||||
|
||||
By default, no container is pinned. For predictable performance when multiple GPU workloads run concurrently:
|
||||
|
||||
- **GPU 0:** heavy LLM (llama-swap big models).
|
||||
- **GPU 1:** light services (both vllm-qwen3 services share this GPU via `--gpu-memory-utilization`).
|
||||
|
||||
Use `deploy.resources.reservations.devices[].device_ids: ["<id>"]` in compose to pin.
|
||||
@@ -0,0 +1 @@
|
||||
10.250.50.54
|
||||
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user