Initial commit: PFI fleet inventory, stacks, tooling, and backup pipeline
Captures the full workspace state built up to this point:
- CLAUDE.md + README.md describing conventions and the four-host fleet
(ana-ml2, ana-docker, nh3-docker, esh-docker-vm).
- Per-host notes under servers/<host>/ with ssh-target fallback files
and latest system-details snapshots (two in-compose credential leaks
scrubbed; the upstream compose files still need to move those to .env).
- scripts/: server_inspect.sh (read-only remote diagnostic),
refresh-server-info.sh (dir-driven discovery + snapshot capture with
validation warnings), add-host.sh, sync-stacks.sh (pull
compose/conf trees), deploy-stack.sh (push with per-file diff + prompt).
- stacks/: canonical compose for backrest, beszel, dozzle, llama-swap,
rest-server-ana, rest-server-nh3, vllm-qwen3, plus the retired
infinity reference. All use the .env-driven + traefik-net + homepage
label pattern.
- configs/restic/ana-docker/: first resticprofile config + pre-backup
hook (Synapse pg_dump, Seafile mysqldump, Vaultwarden SQLite); templates
for the other three hosts to come.
- docs/pfi/: general infrastructure reference carried over.
- .gitignore excludes .env, stacks-mirror/, and assorted secret/state
filenames to prevent re-leaks on later commits.
This commit is contained in:
@@ -0,0 +1,52 @@
|
||||
# llama-swap
|
||||
|
||||
GGUF model server with on-demand model swapping. Served via llama.cpp's `llama-server` under the llama-swap proxy.
|
||||
|
||||
**Server:** ana-ml2
|
||||
**Port:** 9292 (configurable via `.env`)
|
||||
**GPU:** both (unpinned — `runtime: nvidia` grants access to all devices; per-model GPU selection happens inside `config.yaml`)
|
||||
|
||||
## Files
|
||||
|
||||
- **`compose.yaml`** — canonical compose. Deployed to `/opt/docker/compose/llama-swap/compose.yaml` on ana-ml2.
|
||||
- **`.env.example`** — template for the per-host `.env`. Copy to `.env` on the server and tweak.
|
||||
- **`config.yaml`** — model definitions and groups. Deployed to `/opt/docker/conf/llama-swap/config.yaml` on the server.
|
||||
|
||||
Homepage labels are in the compose file under the `AI Systems` group, matching the convention used by `vllm-qwen3` and `infinity`.
|
||||
|
||||
## Deploy a fresh install
|
||||
|
||||
```bash
|
||||
scripts/deploy-stack.sh ana-ml2 llama-swap
|
||||
|
||||
ssh ana-ml2 '
|
||||
cd /opt/docker/compose/llama-swap && \
|
||||
cp -n .env.example .env && \
|
||||
docker compose config && \
|
||||
docker compose up -d && \
|
||||
docker compose logs --tail=30
|
||||
'
|
||||
```
|
||||
|
||||
## Model reference conventions
|
||||
|
||||
- **Modern entries:** use `-hf <user>/<repo>[:<quant>]` — reads from the shared HF cache, nothing to pre-stage outside `hf download`
|
||||
- **Legacy entries:** use `--model /models/<dir>/<file>.gguf` — reads GGUFs from `/tank/aimodels/llm/` (pre-HF-cache era, gradually being migrated)
|
||||
|
||||
New models should prefer the `-hf` pattern.
|
||||
|
||||
## Deploy updates to config only
|
||||
|
||||
```bash
|
||||
# After editing config.yaml here:
|
||||
scp config.yaml ana-ml2:/opt/docker/conf/llama-swap/config.yaml
|
||||
ssh ana-ml2 'cd /opt/docker/compose/llama-swap && docker compose restart'
|
||||
```
|
||||
|
||||
## Deploy updates to compose only
|
||||
|
||||
```bash
|
||||
# After editing compose.yaml or .env.example here:
|
||||
scripts/deploy-stack.sh ana-ml2 llama-swap
|
||||
ssh ana-ml2 'cd /opt/docker/compose/llama-swap && docker compose up -d'
|
||||
```
|
||||
Reference in New Issue
Block a user