vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit: PFI fleet inventory, stacks, tooling, and backup pipeline

Browse Source 
Captures the full workspace state built up to this point:

  - CLAUDE.md + README.md describing conventions and the four-host fleet
    (ana-ml2, ana-docker, nh3-docker, esh-docker-vm).
  - Per-host notes under servers/<host>/ with ssh-target fallback files
    and latest system-details snapshots (two in-compose credential leaks
    scrubbed; the upstream compose files still need to move those to .env).
  - scripts/: server_inspect.sh (read-only remote diagnostic),
    refresh-server-info.sh (dir-driven discovery + snapshot capture with
    validation warnings), add-host.sh, sync-stacks.sh (pull
    compose/conf trees), deploy-stack.sh (push with per-file diff + prompt).
  - stacks/: canonical compose for backrest, beszel, dozzle, llama-swap,
    rest-server-ana, rest-server-nh3, vllm-qwen3, plus the retired
    infinity reference. All use the .env-driven + traefik-net + homepage
    label pattern.
  - configs/restic/ana-docker/: first resticprofile config + pre-backup
    hook (Synapse pg_dump, Seafile mysqldump, Vaultwarden SQLite); templates
    for the other three hosts to come.
  - docs/pfi/: general infrastructure reference carried over.
  - .gitignore excludes .env, stacks-mirror/, and assorted secret/state
    filenames to prevent re-leaks on later commits.
This commit is contained in:
Vuong Hoang
2026-04-20 14:23:18 -07:00
commit e376d0aec9
55 changed files with 9101 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+28
View File
@@ -0,0 +1,28 @@
# Secrets — real .env files must never land here, only .env.example templates.
.env
.env.local
.env.*.local
**/.env
!**/.env.example
# Live mirror of server /opt/docker/{compose,conf}/ trees pulled by
# sync-stacks.sh. Contains upstream compose files that can carry embedded
# plaintext credentials (e.g. legacy seafile/paperless configs), so we
# don't track them in git. Audited, hand-curated copies live under
# stacks/<name>/ and are the source of truth.
stacks-mirror/
# Staged htpasswd / secrets files that might get written to /tmp during
# helper scripts.
htpasswd-new
*.netrc
# Editor / OS cruft
.DS_Store
*.swp
*.swo
*~
# Logs
*.log
*.log.*
CLAUDE.md
+150
View File
@@ -0,0 +1,150 @@
# CLAUDE.md
This workspace is for managing PFI infrastructure — servers, Docker stacks, and related configs. Spawn a dedicated Claude Code session here when working on infra so it doesn't clutter AIPA-MCP development context.
## Purpose
- Inventory of servers and their state
- Canonical copies of Docker Compose stacks deployed on those servers
- Scripts for inspecting and managing the infrastructure
- Conventions so all stacks look the same
This is a **reference workspace** — the authoritative copies of compose files and configs live **on the servers** under `/opt/docker/compose/<stack>/` and `/opt/docker/conf/<stack>/`. This workspace mirrors them for version control, editing, and planning.
## Conventions (enforce for every new stack)
Observed and standardized across servers:
- **Compose location on server:** `/opt/docker/compose/<stack>/compose.yaml`
- **Config mounts on server:** `/opt/docker/conf/<stack>/...`
- **Networks:** external `traefik-net`, aliased as `tnet` in compose
```yaml
networks:
tnet:
name: traefik-net
external: true
```
- **GPU reservation:** prefer `deploy.resources.reservations.devices` with explicit `device_ids` for pinning
```yaml
deploy:
resources:
reservations:
devices:
- driver: nvidia
device_ids: ["1"]
capabilities: [gpu]
```
- **Tunables:** `.env` in the same directory as `compose.yaml` — keep the compose file constant, edit the `.env`
- **Named volumes** for service state (pattern: `<stack>_<name>`)
- **Bind mounts** only for: model files (`/tank/aimodels/...`), config files (`/opt/docker/conf/...`), docker socket where required
- **Restart policy:** `restart: unless-stopped` for daemons
- **Homepage labels** on user-facing services:
```yaml
labels:
- homepage.group=AI Systems
- homepage.name=<ServiceName>
- homepage.icon=mdi-<icon>
- homepage.description=<short>
- homepage.href=http://<host-ip>:<port>
```
- **Healthchecks** on services that expose HTTP
## Servers
| Name | IP | Site | Role | Details |
|------|-----|------|------|---------|
| ana-ml2 | 10.250.50.54 | Anaheim (`10.250.0.0/16`) | GPU / AI inference | `servers/ana-ml2/README.md` |
| ana-docker | 10.250.50.70 | Anaheim (`10.250.0.0/16`) | General-purpose Docker host (non-GPU) | `servers/ana-docker/README.md` |
| nh3-docker | 10.100.50.40 | NH3 (`10.100.0.0/16`) | General-purpose Docker host (non-GPU) | `servers/nh3-docker/README.md` |
| esh-docker-vm | 10.0.50.45 | ESH home lab (`esteban.net`, `10.0.50.0/24`) | Home-lab Docker host (non-PFI scope) | `servers/esh-docker-vm/README.md` |
**Placement rules:**
- GPU-required stacks → `ana-ml2`.
- Anaheim non-GPU services → `ana-docker`.
- NH-site non-GPU services → `nh3-docker`.
- ESH home-lab workloads (`esteban.net`) → `esh-docker-vm`. Not part of the PFI colo topology, but shares monitoring/backup tooling.
- Cross-site services (e.g. Beszel hub, Dozzle hub) live on `ana-docker` and pull from agents on the other hosts.
## How to refresh a server's state
```bash
# Show help (no args)
scripts/refresh-server-info.sh
# Refresh every host discovered under servers/*/
scripts/refresh-server-info.sh all
# Refresh a specific host (must match a servers/<name>/ dir; ssh_config
# entry or servers/<name>/ssh-target handles how to reach it)
scripts/refresh-server-info.sh ana-docker
```
Fleet-wide runs require the literal `all` keyword — no-args prints help so you can't accidentally hit every host by forgetting a name.
The script pipes `server_inspect.sh` over SSH via stdin (no scp, no remote cleanup) and writes each `servers/<host>/system-details.txt` atomically — a failed run never clobbers the previous snapshot. The inspect script itself is read-only.
Each server dir can hold an `ssh-target` file (one line, `<ip>` or `<user>@<ip>`) as a fallback for when the dir name doesn't resolve via DNS or `~/.ssh/config`. The script prefers whatever ssh would resolve normally and only consults the file when that fails.
To register a new server:
```bash
scripts/add-host.sh <name> <ip-or-user@ip>
scripts/refresh-server-info.sh <name> # pull the first snapshot
```
To audit discovery without touching the network (checks permissions, unresolvable names with no fallback, missing README / system-details, malformed `ssh-target`):
```bash
scripts/refresh-server-info.sh --validate-only all
scripts/refresh-server-info.sh --validate-only <host>
```
## Stack mirror (pull / push)
Compose and config trees are mirrored into `stacks-mirror/<host>/<stack>/` so they can be diffed and version-controlled. Pull is fleet-wide and safe; push is one stack at a time with a diff + prompt.
```bash
# Pull compose + conf from every host into stacks-mirror/
scripts/sync-stacks.sh
scripts/sync-stacks.sh --dry-run # see what would change
scripts/sync-stacks.sh ana-docker # one host
# Push a local stack back to the server (diffs each file, prompts y/N)
scripts/deploy-stack.sh <host> <stack>
scripts/deploy-stack.sh <host> <stack> --compose # skip conf
scripts/deploy-stack.sh <host> <stack> --conf # skip compose
```
**Opt-out per stack:** create `stacks-mirror/<host>/<stack>/.no-sync` (skip both sides) or `stacks-mirror/<host>/<stack>/conf/.no-sync` (skip conf only).
**Always excluded in both directions** (secrets / runtime state): `.env`, `.env.*`, `acme.json`, `client_secrets.json`, `*.pem`, `*.key`, `*.crt`, `*.pfx`, `*.sqlite`, `*.sqlite3`, `*.db`, `*.log`, `*.log.*`, `*.pid`, `hub/`, `logs/`.
Requires `rsync` installed on this workstation and every host you sync against (`apt install rsync`).
## Layout
```
eshpfi-management/
├── CLAUDE.md # this file
├── README.md # human-facing overview
├── scripts/
│ └── server_inspect.sh # gather server state for compose planning
├── servers/
│ └── <name>/
│ ├── README.md
│ └── system-details.txt # latest server_inspect output
├── stacks/
│ └── <stack>/
│ ├── compose.yaml # deployed to /opt/docker/compose/<stack>/
│ ├── .env.example # template; real .env lives on server
│ └── README.md # what this stack does, how to deploy
└── docs/
└── pfi/ # general PFI infrastructure reference
```
## Working rules
- **Copies, not symlinks.** Files here reflect what's on the server at the time of the last sync. When you edit here, the server doesn't change until you deploy.
- **Never commit secrets.** Use `.env.example` templates; real `.env` files (with tokens, passwords) live on the server and are gitignored if/when this becomes a git repo.
- **Surgical edits.** When fixing one stack, don't touch unrelated ones. Follow AIPA-MCP's CLAUDE.md rules about scope discipline.
- **Sanity-check before deploying.** Run `docker compose config` (dry parse) before `docker compose up -d` on the server.
README.md
+117
View File
@@ -0,0 +1,117 @@
# eshpfi-management
Infrastructure management workspace for the PFI fleet (plus the ESH home-lab host). Tracks server state, canonical Docker Compose stacks, per-host configs, and the tooling that moves them around.
See **[CLAUDE.md](CLAUDE.md)** for the full set of conventions and the rules Claude Code sessions follow when working here.
## The fleet
| Host | IP | Site | Role |
|---|---|---|---|
| ana-ml2 | `10.250.50.54` | Anaheim (`10.250.0.0/16`) | GPU / AI inference |
| ana-docker | `10.250.50.70` | Anaheim | General-purpose Docker + cross-site hubs |
| nh3-docker | `10.100.50.40` | NH3 (`10.100.0.0/16`) | General-purpose Docker (NH site) |
| esh-docker-vm | `10.0.50.45` | ESH home lab (`esteban.net`) | Home-lab Docker (non-PFI scope) |
Per-host snapshots of the running system live under `servers/<host>/system-details.txt`, refreshed via `scripts/refresh-server-info.sh`.
## Layout
```
.
├── CLAUDE.md # conventions; loaded by Claude Code sessions
├── README.md # this file
├── scripts/ # workstation tooling
│ ├── server_inspect.sh # read-only diagnostic, runs on remote via stdin
│ ├── refresh-server-info.sh # pull fresh system-details.txt for one/all hosts
│ ├── add-host.sh # register a new server (writes servers/<name>/ssh-target)
│ ├── sync-stacks.sh # pull /opt/docker/{compose,conf}/ → stacks-mirror/
│ └── deploy-stack.sh # push stacks-mirror/<host>/<stack>/ with diff + prompt
├── servers/ # per-host notes + latest snapshot + ssh-target fallback
│ └── <host>/
│ ├── README.md
│ ├── system-details.txt # regenerate on demand
│ └── ssh-target # <ip> or <user>@<ip>, used when DNS fails
├── stacks/ # canonical compose files (source of truth)
│ └── <stack>/
│ ├── compose.yaml
│ ├── .env.example
│ └── README.md
├── stacks-mirror/ # gitignored — live mirror from sync-stacks.sh
├── configs/ # host-level config files that aren't docker-compose
│ └── restic/<host>/ # resticprofile configs + pre-backup hooks
└── docs/ # general reference (network, models, proxmox, etc.)
└── pfi/
```
## Current stacks
**GPU (ana-ml2):**
- `llama-swap` — GGUF model swapper via llama.cpp (port 9292)
- `vllm-qwen3` — embeddings (8001) + reranker (8002) via vLLM
**Anaheim non-GPU (ana-docker):**
- `traefik`, `crowdsec`, `gitea`, `vaultwarden`, `synapse`, `seafile`, `searxng`, `openwebui`, `sillytavern`, `mailrise`, `rustdesk`, `dockge`, `it-tools`
- Fleet services: `beszel` (metrics hub, port 8090), `dozzle-hub` (log viewer, 8088), `backrest` (restic UI, 9898)
- Backup target: `rest-server-ana` on port 8000
**NH3 (nh3-docker):**
- `adguard`, `dockge`, plus Beszel/Dozzle agents
**NH3 (Synology `10.100.50.50`):**
- `rest-server-nh3` — restic backup target (port 8000)
**ESH home lab (esh-docker-vm):**
- `adguard`, `homeassistant` (macvlan), `esphome`, `mosquitto`, `paperless-ngx`, `pgadmin`, `calibre`, `calibre-web`, `drawio`, `traefik`, `homepage`, `uptime-kuma`, plus Beszel/Dozzle agents
## Common tasks
**Refresh one host's snapshot:**
```bash
scripts/refresh-server-info.sh ana-docker
```
**Refresh all hosts:**
```bash
scripts/refresh-server-info.sh all
```
**Add a new host:**
```bash
scripts/add-host.sh <name> <ip-or-user@ip>
scripts/refresh-server-info.sh <name>
```
**Validate discovery (without hitting the network):**
```bash
scripts/refresh-server-info.sh --validate-only all
```
**Push a stack to a host (with diff + confirm):**
```bash
scripts/deploy-stack.sh <host> <stack>
```
**Pull every server's compose/conf trees into stacks-mirror/ (not committed — see `.gitignore`):**
```bash
scripts/sync-stacks.sh all
```
## Backup pipeline
Backups are driven by per-host `resticprofile` configs under `configs/restic/<host>/`, scheduled via systemd timers on each host:
- **Writes**: each host backs up to its site-local rest-server (`rest-server-ana` or `rest-server-nh3`), over HTTP basic-auth.
- **Authentication**: shared `.htpasswd` file on both rest-servers, one entry per host; credentials stored in `/etc/restic/restic.env` on each client host.
- **Encryption**: per-host client-side passphrase in `/etc/restic/password` (unique per repo; losing it = losing that host's backups).
- **Visibility**: Backrest (`http://10.250.50.70:9898`) shows every repo for browsing/restore.
- **Schedule**: backup at 01:00 daily, `forget` at 03:00 daily, weekly `check --read-data-subset 10%` on Sundays.
- **Prune**: manual ceremony (rest-server runs with `--append-only`, which blocks destructive prune ops).
- **Off-site**: cross-site rsync between the two rest-server data dirs is planned (not yet implemented).
## Authoritative vs. mirror
- **Authoritative:** files on each server under `/opt/docker/compose/<stack>/` and `/opt/docker/conf/<stack>/`.
- **This workspace:** source-of-truth copies under `stacks/<name>/` (hand-curated), and a gitignored mirror under `stacks-mirror/` pulled by `sync-stacks.sh`.
Edit in `stacks/`, push with `deploy-stack.sh`. Never commit `stacks-mirror/` — it can contain embedded plaintext secrets from upstream compose files that haven't been audited yet.
configs/restic/README.md
+118
View File
@@ -0,0 +1,118 @@
# configs/restic
Per-host restic backup configs, deployed into `/etc/restic/` on each server and driven by `resticprofile` + `systemd` timers.
## Layout
```
configs/restic/
├── README.md # this file
└── <host>/
├── profiles.yaml # committed, zero secrets
├── pre-backup.sh # committed, zero secrets
└── README.md # per-host notes (paths, containers, quirks)
```
On each server, deployed to `/etc/restic/`:
```
/etc/restic/
├── profiles.yaml # scp'd from configs/restic/<host>/profiles.yaml
├── pre-backup.sh # scp'd, 0755, root:root
├── password # 0400 root:root — client-side encryption passphrase
└── restic.env # 0600 root:root — RESTIC_REPOSITORY=rest:http://user:pw@host:port/path/
/var/lib/restic/
├── stage/ # temp staging for DB dumps; owned by root, 0700
└── last-success # unix timestamp of the last successful run
```
## Why this shape
- **No secrets in committed config.** `profiles.yaml` references `RESTIC_PASSWORD_FILE=/etc/restic/password` and loads `RESTIC_REPOSITORY` from `restic.env`. Both files live only on the host, 0400/0600 root-owned.
- **Creds go in the URL, not netrc.** restic's rest backend doesn't consult `~/.netrc` — HTTP basic-auth has to be embedded in the repository URL. Keeping that URL in an env-file (not the committed YAML) means the secret stays on the host.
- **Pre-hook runs DB dumps into a staging dir**, then `restic backup` includes that dir alongside the regular paths. One snapshot = one point-in-time.
- **`restic forget` is scheduled; `restic prune` is not.** Rest-server's `--append-only` blocks prune from the client side by design. Prune is a manual ceremony (flip the flag, run prune, flip back).
## Install restic + resticprofile on each host
```bash
# Current-enough restic. Debian 12 ships 0.14 (too old for some flags);
# Debian 13 ships 0.18. If you're on 12, grab the .deb from the upstream
# release page instead.
sudo apt install -y restic # or install 0.18+ from github.com/restic/restic/releases
# resticprofile is not in Debian. Download the .deb from its release page.
v=$(curl -sI https://github.com/creativeprojects/resticprofile/releases/latest \
| awk -F'/' '/^location:/{sub(/\r/,""); print $NF}')
v=${v#v}
url="https://github.com/creativeprojects/resticprofile/releases/download/v${v}/resticprofile_${v}_linux_amd64.deb"
curl -sL -o /tmp/resticprofile.deb "$url"
sudo dpkg -i /tmp/resticprofile.deb
rm /tmp/resticprofile.deb
resticprofile version
```
## Per-host deploy flow (done once per host)
```bash
HOST=ana-docker # or ana-ml2, nh3-docker, esh-docker-vm
# 1. Create the target dir (one-time)
ssh -t "$HOST" 'sudo install -d -o root -g root -m 0755 /etc/restic'
ssh -t "$HOST" 'sudo install -d -o root -g root -m 0700 /var/lib/restic/stage'
# 2. Seed the two secret files on the host (never in this repo):
# - the client-side encryption passphrase (the one used at `restic init`)
# - an env-file with the full RESTIC_REPOSITORY URL including HTTP creds
ssh -t "$HOST" 'sudo install -o root -g root -m 0400 /dev/null /etc/restic/password'
ssh -t "$HOST" 'sudo install -o root -g root -m 0600 /dev/null /etc/restic/restic.env'
# Seed content (replace <…> with real values from your vault):
ssh -t "$HOST" "echo '<encryption-passphrase>' | sudo tee /etc/restic/password >/dev/null"
ssh -t "$HOST" "echo 'RESTIC_REPOSITORY=rest:http://<user>:<http-pw>@<rest-server>:8000/<user>/' | sudo tee /etc/restic/restic.env >/dev/null"
# 3. Push the committed config + pre-hook
scp "configs/restic/$HOST/profiles.yaml" "$HOST:/tmp/profiles.yaml"
scp "configs/restic/$HOST/pre-backup.sh" "$HOST:/tmp/pre-backup.sh"
ssh -t "$HOST" "
sudo install -o root -g root -m 0644 /tmp/profiles.yaml /etc/restic/profiles.yaml
sudo install -o root -g root -m 0755 /tmp/pre-backup.sh /etc/restic/pre-backup.sh
rm -f /tmp/profiles.yaml /tmp/pre-backup.sh
"
# 4. Test the profile before scheduling
ssh -t "$HOST" 'sudo resticprofile --config /etc/restic/profiles.yaml --name default backup --dry-run'
# 5. When dry-run looks clean, wire up systemd timers
ssh -t "$HOST" 'sudo resticprofile --config /etc/restic/profiles.yaml schedule'
# → installs restic-backup@<profile>.{service,timer} units for backup/forget/check
# 6. Verify
ssh -t "$HOST" 'sudo systemctl list-timers | grep restic'
```
## Prune ceremony (quarterly or as needed)
Per-repo, when enough forgotten-but-still-on-disk snapshots accumulate:
```bash
# On the rest-server host (ana-docker or the Synology):
# 1. Stop the append-only rest-server, start one without the flag
# (simplest: edit the stack's .env, remove --append-only from OPTIONS,
# docker compose up -d)
# 2. From the client host, run prune
ssh -t <host> 'sudo resticprofile --config /etc/restic/profiles.yaml --name default prune'
# 3. Put --append-only back on the rest-server and docker compose up -d.
```
Alternatively stand up a second rest-server stack on port 8001 without `--append-only` and point prune runs at that endpoint; keep the append-only one for daily writes.
## Monitoring
Each successful run writes `/var/lib/restic/last-success` with the current unix timestamp (this is the `post-backup` hook in `profiles.yaml`). Beszel can be configured to alert when that file's mtime exceeds ~36 hours.
```bash
ssh <host> 'stat -c "%y %n" /var/lib/restic/last-success 2>/dev/null || echo no successful run yet'
```
configs/restic/ana-docker/pre-backup.sh
Executable
+75
View File
@@ -0,0 +1,75 @@
#!/bin/bash
# pre-backup.sh — ana-docker.
# Runs as root from resticprofile's `run-before`, before `restic backup`.
#
# Produces DB dumps in /var/lib/restic/stage/ so the nightly restic
# snapshot captures consistent point-in-time data for services whose
# raw volume files are not safe to back up live.
#
# Containers handled here:
# - synapse-db (Postgres 16)
# - seafile-mysql (MariaDB 10.6)
# - vaultwarden (SQLite w/ WAL; online .backup via sqlite3 if available)
#
# Gitea's DB is external (hosted elsewhere in the LAN) so we only back
# up its data volume; whoever owns the gitea Postgres backs it up
# separately.
#
# Idempotent: clears and recreates its staging files each run.
set -euo pipefail
STAGE=/var/lib/restic/stage
install -d -o root -g root -m 0700 "$STAGE"
log() { printf '%s pre-backup(ana-docker): %s\n' "$(date -Is)" "$*"; }
# Purge previous stage so stale dumps don't pile up and end up in the snapshot.
find "$STAGE" -mindepth 1 -maxdepth 1 -exec rm -rf {} +
# ---------- synapse-db (Postgres) ---------------------------------------------
if docker inspect synapse-db >/dev/null 2>&1; then
log "dumping synapse postgres"
# -Fc custom format, internally compressed + restore-to-subset friendly
docker exec synapse-db \
pg_dump -U synapse -d synapse -Fc --clean --if-exists \
> "$STAGE/synapse.pg_dump"
else
log "skip synapse: container not present"
fi
# ---------- seafile-mysql (MariaDB) -------------------------------------------
if docker inspect seafile-mysql >/dev/null 2>&1; then
log "dumping seafile mariadb"
# The root password lives in the container's own env (MYSQL_ROOT_PASSWORD);
# expand it inside the container so it never lands in the host's process list.
docker exec seafile-mysql sh -c \
'mysqldump -uroot -p"$MYSQL_ROOT_PASSWORD" --all-databases --single-transaction --quick 2>/dev/null' \
| gzip -c > "$STAGE/seafile.sql.gz"
else
log "skip seafile: container not present"
fi
# ---------- vaultwarden (SQLite + WAL) ----------------------------------------
# Vaultwarden uses SQLite in WAL mode. A live copy of db.sqlite3 + -wal is
# usually recoverable, but sqlite3's own .backup pragma is the correct way
# to get a consistent snapshot. If the vaultwarden image has sqlite3
# available, use it; otherwise rely on restic backing up the raw volume.
if docker inspect vaultwarden >/dev/null 2>&1; then
if docker exec vaultwarden sh -c 'command -v sqlite3 >/dev/null 2>&1'; then
log "dumping vaultwarden sqlite via .backup"
docker exec vaultwarden sqlite3 /data/db.sqlite3 \
".backup /tmp/vaultwarden.sqlite3"
docker cp vaultwarden:/tmp/vaultwarden.sqlite3 "$STAGE/vaultwarden.sqlite3"
docker exec vaultwarden rm -f /tmp/vaultwarden.sqlite3
else
log "skip vaultwarden .backup: sqlite3 not in container (raw volume still included via restic)"
fi
else
log "skip vaultwarden: container not present"
fi
# ---------- summary -----------------------------------------------------------
size=$(du -sh "$STAGE" 2>/dev/null | awk '{print $1}')
count=$(find "$STAGE" -type f | wc -l)
log "stage ready: $count files, $size total"
configs/restic/ana-docker/profiles.yaml
+76
View File
@@ -0,0 +1,76 @@
# resticprofile config for ana-docker.
#
# Writes to the Anaheim-side rest-server at 10.250.50.70 as user
# `ana-docker`. The full REST URL (including HTTP basic-auth creds)
# lives in /etc/restic/restic.env — loaded via env-file so this YAML
# carries zero secrets and is safe to version-control.
#
# The client-side encryption passphrase lives in /etc/restic/password.
version: "1"
global:
priority: low
ionice: true
ionice-class: 2
ionice-level: 7
min-memory: 100
default:
env-file: /etc/restic/restic.env # provides RESTIC_REPOSITORY=rest:http://user:pw@…
env:
RESTIC_PASSWORD_FILE: /etc/restic/password
initialize: false # repo was created by `restic init`
lock: /var/lock/restic-ana-docker.lock
backup:
verbose: 1
run-before:
- /etc/restic/pre-backup.sh
run-after:
- date +%s > /var/lib/restic/last-success
source:
- /opt/docker
- /var/lib/docker/volumes
- /var/lib/restic/stage
exclude:
# Docker internals we never want in a backup
- /var/lib/docker/volumes/backingFsBlockDev
- /var/lib/docker/volumes/metadata.db
# Raw DB files — we dump them via pre-backup.sh into /var/lib/restic/stage
- /var/lib/docker/volumes/synapse-db-data
- /var/lib/docker/volumes/synapse_synapse-db-data
- /var/lib/docker/volumes/seafile_db
# Ephemeral / regenerable junk
- /opt/docker/compose/*/logs
- /opt/docker/conf/traefik-ana/acme.json # secret material; excluded everywhere
- /opt/docker/conf/crowdsec/hub # upstream-managed, regenerable
- "**/*.log"
- "**/*.log.*"
- "**/*.pid"
tag:
- host:ana-docker
- site:ana
- fleet:pfi
schedule: "*-*-* 01:00:00"
schedule-permission: system
schedule-log: /var/log/restic-backup.log
forget:
keep-daily: 7
keep-weekly: 4
keep-monthly: 12
keep-yearly: 3
# NOTE: no `prune: true` — rest-server runs with --append-only, which
# blocks the destructive half of prune. See README.md "Prune ceremony".
tag:
- host:ana-docker
schedule: "*-*-* 03:00:00"
schedule-permission: system
schedule-log: /var/log/restic-forget.log
check:
read-data-subset: 10%
schedule: "Sun *-*-* 05:00:00"
schedule-permission: system
schedule-log: /var/log/restic-check.log
docs/pfi/chromadb-setup.md
+146
View File
@@ -0,0 +1,146 @@
# ChromaDB Setup Documentation
**Project**: Infrastructure-PFI
**Target Server**: PFI-ANA-Docker (VM 102)
**IP Address**: 10.250.50.x (VLAN 50)
**Status**: Ready for deployment
## Overview
ChromaDB is an embedded vector database optimized for AI/ML applications. This deployment provides:
- Persistent vector storage on `/tank/chromadb/`
- REST API on port 8000 (internal + Traefik-routed)
- Token-based authentication
- Automated backup and health monitoring
## Architecture
```
┌──────────────────────────────────────────────────────────┐
│ PFI-ANA-Docker (VM 102) │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌───────────┐ │
│ │ Traefik │───▶│ ChromaDB │ │ Dockge │ │
│ │ Reverse │ │ (Port 8000) │ │ Manager │ │
│ │ Proxy │ │ │ │ │ │
│ └──────────────┘ └──────┬───────┘ └───────────┘ │
│ │ │
│ ┌────────▼────────┐ │
│ │ /tank/chromadb │ │
│ │ (bind mount) │ │
│ └─────────────────┘ │
└──────────────────────────────────────────────────────────┘
```
## File Locations (on VM 102)
| Host Path | Purpose |
|---|---|
| `/opt/docker/conf/chromadb/` | Compose file, auth token, config |
| `/opt/docker/conf/chromadb/docker-compose.yml` | Main compose file |
| `/opt/docker/conf/chromadb/auth_token` | Token for API authentication |
| `/tank/chromadb/` | Persistent vector data (bind mount) |
| `/opt/docker/backups/chromadb/` | Backup archives |
## Authentication
This deployment uses **ChromaDB's native token auth**:
- A random 64-hex-char token is generated during setup (`openssl rand -hex 32`)
- The token is stored at `/opt/docker/conf/chromadb/auth_token` (mode 600)
- Clients must supply the token via `Settings`:
```python
import chromadb
from chromadb.config import Settings
client = chromadb.HttpClient(
host="10.250.50.x", # or chromadb.pfi.local via Traefik
port=8000,
settings=Settings(
chroma_client_auth_provider="chromadb.auth.token.TokenAuthClientProvider",
chroma_client_auth_credentials="YOUR_TOKEN_HERE",
),
)
print(client.heartbeat())
```
## Deployment Steps
### 1. Copy compose file to VM 102
```bash
scp configs/pfi-ana/docker/compose-examples/chromadb/docker-compose.yml \
root@10.250.50.x:/opt/docker/conf/chromadb/docker-compose.yml
```
### 2. Run the setup script (on VM 102)
```bash
# Copy scripts to VM 102
scp scripts/setup-chromadb.sh root@10.250.50.x:/opt/docker/conf/chromadb/
ssh root@10.250.50.x
# Run setup
cd /opt/docker/conf/chromadb
chmod +x setup-chromadb.sh
./setup-chromadb.sh
```
### 3. Verify
```bash
curl http://localhost:8000/api/v1/health
```
### 4. (Optional) Run the demo
```bash
# Copy demo files
scp -r configs/pfi-ana/docker/compose-examples/chromadb/ root@10.250.50.x:/tmp/chromadb-demo/
# On VM 102, edit CHROMA_TOKEN in docker-compose.demo.yml
cd /tmp/chromadb-demo
# Set CHROMA_TOKEN in docker-compose.demo.yml to match auth_token
docker compose -f docker-compose.demo.yml up
```
## Monitoring & Maintenance
| Task | Command |
|---|---|
| Health check | `./scripts/health-check-chromadb.sh` |
| View logs | `docker logs -f chromadb` |
| Backup | `./scripts/backup-chromadb.sh` |
| Restart | `docker compose restart` |
| Stop | `docker compose down` |
### Cron (daily backup at 2 AM)
```cron
0 2 * * * /opt/docker/conf/chromadb/backup-chromadb.sh >> /var/log/chromadb-backup.log 2>&1
```
## Networking
| Aspect | Value |
|---|---|
| Docker network | `traefik-net` (aliased as `tnet`) |
| Internal port | 8000 |
| Traefik host rule | `chromadb.pfi.local` |
| Traefik entrypoint | `websecure` (HTTPS) |
| TLS | Enabled via Traefik |
## Project Files
| File | Purpose |
|---|---|
| `configs/pfi-ana/docker/compose-examples/chromadb/docker-compose.yml` | Production compose (for Dockge) |
| `configs/pfi-ana/docker/compose-examples/chromadb/docker-compose.demo.yml` | Demo client |
| `configs/pfi-ana/docker/compose-examples/chromadb/Dockerfile.demo` | Demo image |
| `configs/pfi-ana/docker/compose-examples/chromadb/scripts/demo.py` | Demo test script |
| `scripts/setup-chromadb.sh` | Deployment script (run on VM 102) |
| `scripts/backup-chromadb.sh` | Backup script (run on VM 102) |
| `scripts/health-check-chromadb.sh` | Health monitoring (run on VM 102) |
| `scripts/quickstart-chromadb.sh` | Convenience wrapper for setup |
docs/pfi/docker-stack.md
+333
View File
@@ -0,0 +1,333 @@
# PFI-ANA Docker Stack
## Overview
PFI-ANA (the Colo) runs Docker services managed through **Dockge**, a compose-aware Docker management UI. All services that require inbound HTTP/HTTPS routing join a shared external Docker network called `traefik-net`, allowing **Traefik** to act as a reverse proxy and handle TLS termination and routing.
## Conventions
### Network
| Network | Docker Name | Purpose |
|---|---|---|
| Traefik network | `traefik-net` | Shared external network. Services join as `tnet` so Traefik can discover them. |
Every compose file that needs to be reachable through Traefik **must** include:
```yaml
networks:
tnet:
name: traefik-net
external: true
```
And the service must list `tnet` under its `networks` key.
### Storage Paths (Host)
| Host Path | Purpose |
|---|---|
| `/opt/docker/compose/<service>/` | Per-service compose files (managed by Dockge) |
| `/opt/docker/conf/<service>/` | Per-service configuration files (bind-mounted into containers) |
| `/tank/` | Large / persistent data storage (e.g., AI models, generated images, voice data) |
### GPU Support
Services requiring GPU access use the **NVIDIA Container Toolkit**:
```yaml
runtime: nvidia
```
or the more explicit device reservation:
```yaml
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
```
### Homepage Dashboard Labels
Several services include Docker labels for **Homepage** (a dashboard UI). The convention is:
```yaml
labels:
- homepage.group=<Group Name>
- homepage.name=<Display Name>
- homepage.icon=<icon identifier>
- homepage.description=<brief description>
- homepage.href=http://<host>:<port>
```
All services reference the VM 102 host IP `10.250.50.70` (PFI-ANA_DOCKER).
### Compose File Location
Dockge expects compose files under `/opt/docker/compose/` on the Docker host. The live compose files are version-controlled in this project under `configs/pfi-ana/docker/compose/`. Configuration files that containers bind-mount live under `configs/pfi-ana/docker/conf/`.
---
## Live Services Index
| Service | Host Port | Container Port | GPU | Homepage Group | Status | Compose File | Config File |
|---|---|---|---|---|---|---|---|
| **Dockge** | 5001 | 5001 | No | PFI-ANA | ✅ Live | `compose/dockge/compose.yaml` | — |
| **llama-swap** | 9292 | 8080 | Yes (CUDA) | — | ✅ Live | `compose/llama-swap/compose.yaml` | `conf/llama-swap/config.yaml` |
| **ComfyUI** | 8188 | 8188 | Yes (all caps) | AI Systems | ✅ Live | `compose/comfyui/compose.yaml` | — |
| **VibeVoice** | 8745 | 8745 | Yes (gpu) | AI Systems | ✅ Live | `compose/vibevoice/compose.yaml` | — |
| **Parakeet STT** | 8300 | 8000 | Yes (gpu) | AI Systems | ✅ Live | `compose/parakeet/compose.yaml` | — |
| **ChromaDB** | 8000 | 8000 | No | AI Systems | ✅ Live | `compose/chromadb/compose.yaml` | `/opt/docker/conf/chromadb/auth_token` |
> All paths relative to `configs/pfi-ana/docker/`.
---
## Service Details
### 1. Dockge — Docker Compose Management UI
- **Image**: `louislam/dockge:latest`
- **Port**: 5001 → 5001
- **Restart policy**: `unless-stopped`
- **Homepage group**: PFI-ANA
- **Compose file**: `compose/dockge/compose.yaml`
**Volumes**:
| Host / Volume | Container | Purpose |
|---|---|---|
| `/var/run/docker.sock` | `/var/run/docker.sock` | Docker socket for managing containers |
| `dockge_data` (named volume) | `/app/data` | Dockge application data |
| `/opt/docker/compose` | `/opt/docker/compose` | Compose stack directory |
**Environment**:
- `DOCKGE_STACKS_DIR=/opt/docker/compose` — tells Dockge where to find/manage compose stacks
**Notes**: Dockge is the management interface for all other compose stacks on this host. It has full Docker daemon access via the socket mount.
---
### 2. llama-swap — Multi-Model LLM Gateway
- **Image**: `ghcr.io/mostlygeek/llama-swap:cuda`
- **Port**: 9292 → 8080
- **Runtime**: `nvidia` (CUDA)
- **Compose file**: `compose/llama-swap/compose.yaml`
- **Config file**: `conf/llama-swap/config.yaml`
- **Interactive**: `stdin_open: true`, `tty: true` (required by llama-swap)
**Volumes**:
| Host Path | Container Path | Purpose |
|---|---|---|
| `/opt/docker/conf/llama-swap/config.yaml` | `/app/config.yaml` | llama-swap configuration (models, groups, params) |
| `/tank/aimodels/llm` | `/models` | LLM model files (GGUF format) |
**Configured Models** (from `config.yaml`):
| Model ID | Display Name | Quantization | Context Size | TTL (s) | Notes |
|---|---|---|---|---|---|
| `qwen3-4b` | Qwen3-4B-Instruct-2507-Q6_K | Q6_K | default | 0 (persistent) | Small general-purpose model |
| `glm4.5-air` | GLM-4.5-Air Q4_K_M | Q4_K_M | 40,000 | 600 | Flash attention enabled |
| `skyfall-r1-31b-q6k` | Skyfall 31B v4 | Q6_K_L | 40,000 | 600 | Flash attention, full GPU offload |
| `GLM-Steam-106B-QK4M-A12B` | GLM-Steam 106B A12B | Q4_K_M | 40,000 | 600 | 2-shard model, MoE with 12B active |
| `kimik2-q2kxl` | Kimi K2 Instruct | UD-Q2_K_XL | default | 600 | 8-shard model, only 2 GPU layers (CPU-heavy) |
| `qwen3-coder-30b-iq4-nl` | Qwen3 Coder 30B A3B | IQ4_NL | 40,000 | 0 (persistent) | MoE 3B active, coding-optimized |
| `unsloth-granite-4-small` | Granite 4.0 Small | Q4_K_M | 120,000 | 0 (persistent) | IBM Granite, deterministic (temp=0) |
| `qwen3.5-35-a3b` | Qwen 3.5 35B A3B | UD-Q4_K_XL | 32,768 | 0 (persistent) | MoE, thinking mode, temp=1.0 |
| `qwen3.5-35-a3b-code` | Qwen 3.5 35B A3B Code | UD-Q4_K_XL | 32,768 | 0 (persistent) | Same model, code-tuned params (temp=0.6) |
| `gemma4-26b-a4b` | Gemma 4 26B A4B | UD-Q4_K_XL | 32,768 | 600 | MoE 4B active, thinking enabled, supports images |
| `gemma4-31b-dense` | Gemma 4 31B Dense | UD-Q4_K_XL | 32,768 | 600 | Full dense model, thinking enabled, supports images |
| `embeddinggemma-300M` | Embedding Gemma 300M | Q8_0 | 2,048 | 0 (persistent) | Embedding model, cls pooling |
| `qwen3-embedding-0.6B` | Qwen3 Embedding 0.6B | Q8_0 | 32,768 | 0 (persistent) | Embedding model, mean pooling |
| `jina-reranker-v3-0.6B` | Jina Reranker v3 | Q8_0 | 32,768 | 0 (persistent) | Reranking model |
| `bge-reranker-v2-m3-0.6B` | BGE Reranker v2 m3 | Q8_0 | 32,768 | 0 (persistent) | Reranking model |
**Model Groups**:
| Group | Swap | Exclusive | Persistent | Members |
|---|---|---|---|---|
| `high-reasoning` | false | false | — | qwen3.5-35-a3b, qwen3.5-35-a3b-code, gemma4-31b-dense |
| `utility` | false | false | ✅ | embeddinggemma-300M, bge-reranker-v2-m3-0.6B |
**Global Settings**:
- `healthCheckTimeout`: 1200 seconds (20 minutes) — long timeout for large models
- `logLevel`: info
- `metricsMaxInMemory`: 1000
---
### 3. ComfyUI — Image Generation UI
- **Image**: `mmartial/comfyui-nvidia-docker:ubuntu24_cuda13.0-latest`
- **Port**: 8188 → 8188
- **Runtime**: `nvidia` with full device reservation (gpu, compute, utility capabilities)
- **Restart policy**: `unless-stopped`
- **Homepage group**: AI Systems
- **Compose file**: `compose/comfyui/compose.yaml`
**Volumes**:
| Host Path | Container Path | Purpose |
|---|---|---|
| `/tank/comfy/run` | `/comfy/mnt` | ComfyUI workspace / output directory |
| `/tank/aimodels/img/comfy` | `/basedir` | Image models and ComfyUI base directory |
**Environment**:
| Variable | Value | Purpose |
|---|---|---|
| `WANTED_UID` | 1001 | Run as user ID 1001 |
| `WANTED_GID` | 1002 | Run as group ID 1002 |
| `BASE_DIRECTORY` | /basedir | ComfyUI base directory path |
| `SECURITY_LEVEL` | weak | Relaxed security (private network) |
| `NVIDIA_VISIBLE_DEVICES` | all | Expose all GPUs |
| `NVIDIA_DRIVER_CAPABILITIES` | all | Enable all GPU capabilities |
**Notes**: Runs with user-mapped permissions (UID 1001 / GID 1002). The `basedir` points to the image model storage on `/tank`.
---
### 4. VibeVoice — Voice/Audio AI Service
- **Image**: `eworkerinc/vibevoice:latest`
- **Container name**: `vibevoice`
- **Port**: 8745 → 8745
- **GPU**: Yes (all devices, gpu capability)
- **Restart policy**: `unless-stopped`
- **Homepage group**: AI Systems
- **Compose file**: `compose/vibevoice/compose.yaml`
**Volumes**:
| Host Path | Container Path | Purpose |
|---|---|---|
| `/tank/vibevoice/hf` | `/root/.cache/huggingface` | HuggingFace model cache |
| `/tank/vibevoice/voices` | `/app/voices` | Voice data / presets |
| `/tank/vibevoice/state` | `/var/lib/eworker` | Application state persistence |
**Environment**:
| Variable | Value | Purpose |
|---|---|---|
| `ENABLE_1_5B` | true | Enable 1.5B parameter voice model |
| `ENABLE_LARGE` | true | Enable large voice model |
| `AUTH_REQUIRED` | true | Require authentication |
| `CORS_ENABLED` | true | Enable CORS headers |
| `ALLOWED_ORIGINS` | * | Allow all origins (development/private network) |
---
### 5. Parakeet STT — Speech-to-Text Service
- **Image**: `parakeet-stt` (locally built)
- **Port**: 8300 → 8000
- **GPU**: Yes (all devices, gpu capability)
- **Restart policy**: `unless-stopped`
- **Homepage group**: AI Systems
- **Compose file**: `compose/parakeet/compose.yaml`
- **Env file**: `.env` (not tracked in project — likely contains API keys or model config)
**Volumes**:
| Volume | Container Path | Purpose |
|---|---|---|
| `parakeet_cache` (named volume) | `/root/.cache` | Model download cache |
**Notes**: Uses a locally-built image (no registry prefix). The `.env` file is referenced but not stored in the project — it likely contains environment-specific configuration on the Docker host.
---
### 6. ChromaDB — Vector Database
- **Image**: `chromadb/chroma:latest`
- **Container name**: `chromadb`
- **Port**: 8000 → 8000
- **GPU**: No
- **Restart policy**: `unless-stopped`
- **Homepage group**: AI Systems
- **Compose file**: `compose/chromadb/compose.yaml`
**Volumes**:
| Host Path | Container Path | Mode | Purpose |
|---|---|---|---|
| `/opt/docker/conf/chromadb` | `/conf` | read-only | Config directory (contains auth_token) |
| `/tank/chromadb` | `/data` | read-write | Persistent vector data |
**Environment**:
| Variable | Value | Purpose |
|---|---|---|
| `CHROMA_SERVER_AUTHN_CREDENTIALS_FILE` | `/conf/auth_token` | Path to auth token file inside container |
| `CHROMA_SERVER_AUTHN_PROVIDER` | `chromadb.server.auth.token.TokenAuthenticationServerProvider` | Enable token-based authentication |
| `IS_PERSISTENT` | TRUE | Enable persistent storage |
| `PERSIST_DIRECTORY` | `/data` | Where vector data is stored inside container |
| `ANONYMIZED_TELEMETRY` | FALSE | Disable telemetry |
**Health Check**:
| Setting | Value |
|---|---|
| Test | `curl -f http://localhost:8000/api/v1/health` |
| Interval | 30s |
| Timeout | 10s |
| Retries | 3 |
| Start period | 40s |
**Authentication**: Token-based. The auth token is stored at `/opt/docker/conf/chromadb/auth_token` on the host (mode 600), generated with `openssl rand -hex 32`. Clients must supply this token to access the API.
**Notes**: CPU-only service (no GPU). The compose-examples directory contains a reference compose file with Traefik labels and a demo app — see [chromadb-setup.md](chromadb-setup.md) for full deployment instructions and the demo.
---
## Infrastructure Summary
```
┌──────────────────────────────────────────────────────────────┐
│ PFI-ANA (10.250.50.70) │
│ Docker Host (PFI-ANA_DOCKER) │
│ │
│ ┌──────────┐ Manages all compose stacks │
│ │ Dockge │◄─── /opt/docker/compose/* │
│ │ :5001 │ /var/run/docker.sock │
│ └──────────┘ │
│ │
│ ┌──────────────────┐ GPU via passthrough │
│ │ llama-swap :9292 (CUDA, multi-model gateway) │ │
│ │ ├── 15 models (chat, code, embedding, reranker) │ │
│ │ ├── 2 groups (high-reasoning, utility) │ │
│ │ └── Models from /tank/aimodels/llm │ │
│ └──────────────────┘ GPU via passthrough │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ GPU via │
│ │ ComfyUI │ │ VibeVoice │ │ Parakeet │ passthrough │
│ │ :8188 │ │ :8745 │ │ :8300 │ │
│ │ (GPU, img) │ │ (GPU, voice)│ │ (GPU, STT) │ │
│ └──────────┘ └──────────┘ └──────────┘ │
│ │
│ ┌──────────┐ │
│ │ ChromaDB │ Token auth, persistent vectors │
│ │ :8000 │ /tank/chromadb (data) │
│ │ (CPU only) │ /opt/docker/conf/chromadb (config) │
│ └──────────┘ │
│ │
│ ── All services on traefik-net (external) ── │
│ ── /tank/* = persistent large data storage ── │
│ ── /opt/docker/* = config + compose files ── │
└──────────────────────────────────────────────────────────────┘
```
## Port Allocation
| Port | Service | Protocol |
|---|---|---|
| 5001 | Dockge | HTTP |
| 8000 | ChromaDB | HTTP |
| 8188 | ComfyUI | HTTP |
| 8300 | Parakeet STT | HTTP (→ container 8000) |
| 8745 | VibeVoice | HTTP |
| 9292 | llama-swap | HTTP (→ container 8080) |
## Example Configurations
Reference/example compose files (not live) are stored in `configs/pfi-ana/docker/compose-examples/`:
- `compose-examples/llama-swap/docker-compose.yml` — earlier llama-swap reference
- `compose-examples/chromadb/` — ChromaDB with Traefik labels, auth, health checks, and demo app
Full ChromaDB setup instructions: [chromadb-setup.md](chromadb-setup.md)
docs/pfi/model-list.md
+81
View File
@@ -0,0 +1,81 @@
# PFI-ANA Model Inventory
# Generated from: find /models -maxdepth 3 -name "*.gguf" | sort
# Synchronized with llama-swap config.yaml on 2025-07-18
#
# Status Key:
# [ACTIVE] = configured in llama-swap
# [PENDING] = on disk, not yet in config (or still downloading)
# [REMOVED] = was in old config, not on disk
#
# ============================================================================
# KB Optimization Pass (2025-07-18):
# - Qwen 3.5: added presence_penalty, thinking mode enable_thinking
# - Qwen3-Coder-Next: temp 0.3→1.0, top-p 0.85→0.95, top-k 20→40, min-p 0→0.01
# - Gemma 4: added repeat_penalty 1.0, thinking mode enable_thinking
# - Nemotron: already applied in prior pass (--special, min-p 0.01, seed 3407)
# ============================================================================
[ACTIVE] ./ggml-org_Qwen3-Reranker-0.6B-Q8_0-GGUF/qwen3-reranker-0.6b-q8_0.gguf
[ACTIVE] ./unsloth_Qwen3.5-122B-A10B-GGUF/UD-Q4_K_XL/Qwen3.5-122B-A10B-UD-Q4_K_XL-00001-of-00003.gguf
[ACTIVE] ./unsloth_Qwen3.5-122B-A10B-GGUF/UD-Q4_K_XL/Qwen3.5-122B-A10B-UD-Q4_K_XL-00002-of-00003.gguf
[ACTIVE] ./unsloth_Qwen3.5-122B-A10B-GGUF/UD-Q4_K_XL/Qwen3.5-122B-A10B-UD-Q4_K_XL-00003-of-00003.gguf
[ACTIVE] ./ibm-granite_granite-4.0-micro-GGUF/granite-4.0-micro-Q4_K_M.gguf
[ACTIVE] ./unsloth_GLM-4.7-Flash-GGUF/GLM-4.7-Flash-UD-Q4_K_XL.gguf
[ACTIVE] ./jinaai_jina-reranker-v3-GGUF/jina-reranker-v3-Q8_0.gguf
[ACTIVE] ./unsloth_NVIDIA-Nemotron-3-Super-120B-A12B-GGUF/UD-Q4_K_XL/NVIDIA-Nemotron-3-Super-120B-A12B-UD-Q4_K_XL-00001-of-00003.gguf
[ACTIVE] ./unsloth_NVIDIA-Nemotron-3-Super-120B-A12B-GGUF/UD-Q4_K_XL/NVIDIA-Nemotron-3-Super-120B-A12B-UD-Q4_K_XL-00002-of-00003.gguf
[ACTIVE] ./unsloth_NVIDIA-Nemotron-3-Super-120B-A12B-GGUF/UD-Q4_K_XL/NVIDIA-Nemotron-3-Super-120B-A12B-UD-Q4_K_XL-00003-of-00003.gguf
[ACTIVE] ./bartowski_TheDrummer_Skyfall-31B-v4-GGUF/TheDrummer_Skyfall-31B-v4-Q6_K_L.gguf
[ACTIVE] ./RP/BeaverAI_Skyfall-R1-31B-v4a-GGUF/Skyfall-R1-31B-v4a-Q6_K.gguf
[ACTIVE] ./RP/bartowski_TheDrummer_GLM-Steam-106B-A12B-v1-GGUF/TheDrummer_GLM-Steam-106B-A12B-v1-Q4_K_M-00001-of-00002.gguf
[ACTIVE] ./RP/bartowski_TheDrummer_GLM-Steam-106B-A12B-v1-GGUF/TheDrummer_GLM-Steam-106B-A12B-v1-Q4_K_M-00002-of-00002.gguf
[ACTIVE] ./unsloth_Qwen3.5-9B-GGUF/Qwen3.5-9B-UD-Q4_K_XL.gguf
[ACTIVE] ./unsloth_granite-4.0-h-small-GGUF/granite-4.0-h-small-Q4_K_M.gguf
[ACTIVE] ./unsloth_Kimi-K2-Instruct-0905-GGUF/UD-Q2_K_XL/Kimi-K2-Instruct-0905-UD-Q2_K_XL-00001-of-00008.gguf
[ACTIVE] ./unsloth_Kimi-K2-Instruct-0905-GGUF/UD-Q2_K_XL/Kimi-K2-Instruct-0905-UD-Q2_K_XL-00002-of-00008.gguf
[ACTIVE] ./unsloth_Kimi-K2-Instruct-0905-GGUF/UD-Q2_K_XL/Kimi-K2-Instruct-0905-UD-Q2_K_XL-00003-of-00008.gguf
[ACTIVE] ./unsloth_Kimi-K2-Instruct-0905-GGUF/UD-Q2_K_XL/Kimi-K2-Instruct-0905-UD-Q2_K_XL-00004-of-00008.gguf
[ACTIVE] ./unsloth_Kimi-K2-Instruct-0905-GGUF/UD-Q2_K_XL/Kimi-K2-Instruct-0905-UD-Q2_K_XL-00005-of-00008.gguf
[ACTIVE] ./unsloth_Kimi-K2-Instruct-0905-GGUF/UD-Q2_K_XL/Kimi-K2-Instruct-0905-UD-Q2_K_XL-00006-of-00008.gguf
[ACTIVE] ./unsloth_Kimi-K2-Instruct-0905-GGUF/UD-Q2_K_XL/Kimi-K2-Instruct-0905-UD-Q2_K_XL-00007-of-00008.gguf
[ACTIVE] ./unsloth_Kimi-K2-Instruct-0905-GGUF/UD-Q2_K_XL/Kimi-K2-Instruct-0905-UD-Q2_K_XL-00008-of-00008.gguf
[PENDING] ./unsloth_Qwen3-Coder-Next-GGUF/Qwen3-Coder-Next-UD-Q4_K_XL.gguf
NOTE: Download may be incomplete (has .incomplete files). Config entry ready.
[ACTIVE] ./unsloth_Nemotron-3-Nano-30B-A3B-GGUF/Nemotron-3-Nano-30B-A3B-UD-Q4_K_XL.gguf
[ACTIVE] ./unsloth_gemma-4-26B-A4B-it-GGUF/gemma-4-26B-A4B-it-UD-Q4_K_XL.gguf
[ACTIVE] ./unsloth_Qwen3.5-35B-A3B-GGUF/Qwen3.5-35B-A3B-UD-Q4_K_XL.gguf
[ACTIVE] ./Qwen_Qwen3-Embedding-0.6B-GGUF/Qwen3-Embedding-0.6B-Q8_0.gguf
[ACTIVE] ./unsloth_gemma-4-31B-it-GGUF/gemma-4-31B-it-UD-Q4_K_XL.gguf
[ACTIVE] ./ggml-org_embeddinggemma-300M-GGUF/embeddinggemma-300M-Q8_0.gguf
# ============================================================================
# MODELS REMOVED FROM CONFIG (not on disk or superseded):
# ============================================================================
# [REMOVED] qwen3-4b — Qwen3-4B-Instruct-2507-Q6_K (not on disk)
# [REMOVED] glm4.5-air — GLM-4.5-Air-Q4_K_M (not on disk, superseded by glm4.7-flash)
# [REMOVED] qwen3-coder-30b — Qwen3-Coder-30B-A3B (not on disk, superseded by qwen3-coder-next)
# [REMOVED] bge-reranker-v2-m3 — Replaced by qwen3-reranker-0.6B
# [FIXED] glm-steam-106b — Path fixed: was missing RP/ prefix
# ============================================================================
# STILL DOWNLOADING (not yet configured):
# ============================================================================
# [PENDING] Hermes-4-14B — Multiple quant downloads in progress (.incomplete files)
# ============================================================================
docs/pfi/proxmox-vms.md
+260
View File
@@ -0,0 +1,260 @@
# PFI-ANA Proxmox VM Inventory
**Hypervisor**: Proxmox VE at `10.250.250.31:8006`
**Storage Pool**: `ospool` (CEPH/zfs — all VM disks reside here)
**Network Bridge**: `vmbr0` with VLAN tag `50` on all VMs
**QEMU Version**: 7.2.0 (primary), VM 106 on 8.1.5
## Virtual Machines
### VM 100 — PFI-ANA-TRUENAS
| Property | Value |
|---|---|
| **VMID** | 100 |
| **OS Type** | Linux (l26) |
| **CPU** | 2 sockets × 2 cores = 4 vCPU (host passthrough) |
| **Memory** | 8,196 MB |
| **Disk** | `scsi0`: 80G on ospool |
| **CDROM** | `ide2`: TrueNAS-SCALE-22.12.1.iso |
| **Network** | `net0`: virtio, MAC `9A:90:79:7A:86:87`, vmbr0, VLAN 50 |
| **Boot** | scsi0 → ide2 → net0 |
| **Startup** | Order 2, delay 120s |
| **Onboot** | No |
**Purpose**: TrueNAS SCALE storage appliance. Provides NAS/NFS/iSCSI to the colo environment.
---
### VM 101 — PFI-ANA-DC
| Property | Value |
|---|---|
| **VMID** | 101 |
| **OS Type** | Windows 11 |
| **BIOS** | OVMF (UEFI) with TPM 2.0 |
| **Machine** | pc-q35-7.2 |
| **CPU** | 2 sockets × 6 cores = 12 vCPU (host passthrough) |
| **Memory** | 24,576 MB |
| **Disk** | `scsi0`: 240G on ospool |
| **EFI Disk** | `efidisk0`: 1M on ospool |
| **TPM** | `tpmstate0`: 4M, v2.0 on ospool |
| **CDROM** | `scsi1`: virtio-win-0.1.229.iso (VirtIO drivers) |
| **Network** | `net0`: e1000, MAC `CE:C8:D7:FE:32:40`, vmbr0, VLAN 50 |
| **Boot** | scsi0 → net0 → ide0 → scsi1 |
| **Startup** | Order 5, delay 120s |
| **Onboot** | Yes |
**Purpose**: Windows Domain Controller for the Anaheim environment. UEFI with TPM 2.0 suggests Active Directory / Group Policy services.
---
### VM 102 — PFI-ANA-Docker
| Property | Value |
|---|---|
| **VMID** | 102 |
| **OS Type** | Linux (l26) |
| **CPU** | 2 sockets × 4 cores = 8 vCPU (x86-64-v2-AES) |
| **Memory** | 16,384 MB |
| **Disk** | `scsi0`: 250G on ospool |
| **Network** | `net0`: virtio, MAC `BA:AF:E7:E9:79:23`, vmbr0, VLAN 50 |
| **Boot** | scsi0 → net0 → scsi1 |
| **Startup** | Order 4 |
| **Onboot** | Yes |
**Purpose**: Primary Docker host for the colo. Runs Dockge for compose management and Traefik for reverse proxy. All Docker services documented in [docker-stack.md](docker-stack.md) run here.
---
### VM 103 — PFI-SlaveBot
| Property | Value |
|---|---|
| **VMID** | 103 |
| **OS Type** | Windows 10 |
| **Machine** | pc-i440fx-7.2 |
| **CPU** | 2 sockets × 4 cores = 8 vCPU (host passthrough) |
| **Memory** | 8,192 MB |
| **Disk** | `ide0`: 256G on ospool |
| **Network** | `net0`: e1000, MAC `CE:F0:49:C9:03:70`, vmbr0, VLAN 50 |
| **Boot** | ide0 → net0 → scsi0 |
| **Startup** | Not configured |
| **Onboot** | Yes |
**Purpose**: Windows 10 workstation/bot. Likely a task automation or RDP-accessible machine.
---
### VM 104 — PFI-Mongo
| Property | Value |
|---|---|
| **VMID** | 104 |
| **OS Type** | Linux (l26) |
| **CPU** | 2 sockets × 4 cores = 8 vCPU (host passthrough) |
| **Memory** | 8,196 MB |
| **Disk** | `scsi0`: 256G on ospool |
| **Network** | `net0`: virtio, MAC `32:57:90:B2:66:61`, vmbr0, VLAN 50 |
| **Boot** | scsi0 → ide2 → net0 |
| **Startup** | Order 3, delay 60s |
| **Onboot** | Yes |
**Purpose**: MongoDB server. Config file contains a connection string reference: `mongodb://10.250.50.81:27017/`.
---
### VM 105 — PFI-Postgres
| Property | Value |
|---|---|
| **VMID** | 105 |
| **OS Type** | Linux (l26) |
| **CPU** | 4 sockets × 4 cores = 16 vCPU |
| **Memory** | 8,196 MB |
| **Disk** | `scsi0`: 80G on ospool |
| **CDROM** | `ide2`: debian-11.6.0-amd64-netinst.iso |
| **Network** | `net0`: virtio, MAC `C2:1F:CC:71:66:D0`, vmbr0, VLAN 50 |
| **Serial** | `serial0`: socket (IPMI/serial console) |
| **Boot** | scsi0 → ide2 → net0 |
| **Startup** | Order 3, delay 60s |
| **Onboot** | Yes |
**Purpose**: PostgreSQL database server running Debian 11.
---
### VM 106 — PFI-Tailscale
| Property | Value |
|---|---|
| **VMID** | 106 |
| **OS Type** | Linux (l26) |
| **CPU** | 2 sockets × 4 cores = 8 vCPU (x86-64-v2-AES) |
| **Memory** | 2,048 MB |
| **Disk** | `scsi0`: 256G on ospool |
| **CDROM** | `ide2`: debian-12.2.0-amd64-netinst.iso |
| **Network** | `net0`: virtio, MAC `BC:24:11:D7:E9:52`, vmbr0, VLAN 50 |
| **Boot** | scsi0 → ide2 → net0 |
| **Startup** | Not configured |
| **Onboot** | Yes |
**Purpose**: Tailscale VPN node for mesh connectivity. Provides the VPN tunnel endpoints that link the three PFI sites together. Running Debian 12 (newer than most other VMs). Lightweight at 2G RAM.
---
### VM 107 — PFI-Pteradactyl
| Property | Value |
|---|---|
| **VMID** | 107 |
| **OS Type** | Linux (l26) |
| **CPU** | 2 sockets × 4 cores = 8 vCPU (host passthrough) |
| **Memory** | 8,192 MB |
| **Disk** | `scsi0`: 256G on ospool |
| **CDROM** | `ide2`: debian-11.6.0-amd64-netinst.iso |
| **Network** | `net0`: virtio, MAC `CA:44:37:8A:BF:E0`, vmbr0, VLAN 50 |
| **Boot** | scsi0 → ide2 → net0 |
| **Startup** | Not configured |
| **Onboot** | Yes |
**Purpose**: Pterodactyl game server panel. Manages game server instances.
---
### VM 108 — PFI-ANA--DEV
| Property | Value |
|---|---|
| **VMID** | 108 |
| **OS Type** | Linux (l26) |
| **CPU** | 2 sockets × 4 cores = 8 vCPU |
| **Memory** | 8,192 MB |
| **Disk** | `scsi0`: 120G on ospool |
| **CDROM** | `ide2`: debian-11.6.0-amd64-netinst.iso |
| **Network** | `net0`: virtio, MAC `F2:EF:82:2C:AF:90`, vmbr0, VLAN 50 |
| **Boot** | scsi0 → ide2 → net0 |
| **Startup** | Order 10, delay 60s |
| **Onboot** | No |
**Purpose**: Development environment. Not set to auto-boot, starts after core infrastructure (order 10).
---
### VM 110 — PFI-ANA-Webhost
| Property | Value |
|---|---|
| **VMID** | 110 |
| **OS Type** | Linux (l26) |
| **CPU** | 4 sockets × 4 cores = 16 vCPU |
| **Memory** | 4,096 MB (balloon: 1024 MB minimum) |
| **Disk** | `scsi0`: 250G on ospool |
| **CDROM** | `ide2`: debian-11.6.0-amd64-netinst.iso |
| **Network** | `net0`: virtio, MAC `E6:F9:3A:C9:61:2A`, vmbr0, VLAN 50 |
| **Boot** | scsi0 → ide2 → net0 |
| **Startup** | Order 30, up 120s, down 120s |
| **Onboot** | Yes |
**Purpose**: Web hosting server. Highest startup order (30) — starts last. Also has the longest graceful shutdown timeout (120s). Memory ballooning enabled for dynamic allocation.
---
### VM 111 — pfi-tacticalrmm
| Property | Value |
|---|---|
| **VMID** | 111 |
| **OS Type** | Linux (l26) |
| **CPU** | 4 sockets × 4 cores = 16 vCPU |
| **Memory** | 8,192 MB |
| **Disk** | `scsi0`: 256G on ospool |
| **Network** | `net0`: virtio, MAC `BA:FA:65:F6:46:25`, vmbr0, VLAN 50 |
| **Boot** | scsi0 → ide2 → net0 |
| **Startup** | Order 20 |
| **Onboot** | Yes |
**Purpose**: Tactical RMM (Remote Monitoring and Management) server. Provides IT management, remote access, and monitoring capabilities.
---
## Startup Order Summary
VMs are brought up in the following order on host boot:
| Order | VMID | Name | Delay |
|---|---|---|---|
| 2 | 100 | PFI-ANA-TRUENAS | 120s |
| 3 | 104 | PFI-Mongo | 60s |
| 3 | 105 | PFI-Postgres | 60s |
| 4 | 102 | PFI-ANA-Docker | — |
| 5 | 101 | PFI-ANA-DC | 120s |
| 10 | 108 | PFI-ANA--DEV | 60s |
| 20 | 111 | pfi-tacticalrmm | — |
| 30 | 110 | PFI-ANA-Webhost | 120s |
VMs without a startup order (103, 106, 107) will start based on their `onboot` setting but without a specific sequencing delay.
## Resource Summary
| VMID | Name | vCPU | RAM (MB) | Disk | OS |
|---|---|---|---|---|---|
| 100 | PFI-ANA-TRUENAS | 4 | 8,196 | 80G | TrueNAS SCALE |
| 101 | PFI-ANA-DC | 12 | 24,576 | 240G | Windows 11 |
| 102 | PFI-ANA-Docker | 8 | 16,384 | 250G | Linux |
| 103 | PFI-SlaveBot | 8 | 8,192 | 256G | Windows 10 |
| 104 | PFI-Mongo | 8 | 8,196 | 256G | Linux |
| 105 | PFI-Postgres | 16 | 8,196 | 80G | Debian 11 |
| 106 | PFI-Tailscale | 8 | 2,048 | 256G | Debian 12 |
| 107 | PFI-Pteradactyl | 8 | 8,192 | 256G | Debian 11 |
| 108 | PFI-ANA--DEV | 8 | 8,192 | 120G | Debian 11 |
| 110 | PFI-ANA-Webhost | 16 | 4,096 | 250G | Debian 11 |
| 111 | pfi-tacticalrmm | 16 | 8,192 | 256G | Linux |
| | **Totals** | **112** | **105,348** | **2,300G** | |
## Network Notes
- All VMs are on **VLAN 50** via `vmbr0`.
- All VMs have **firewall enabled** on the network interface.
- Linux VMs use `virtio` network adapters; Windows VMs use `e1000`.
- The MongoDB connection string embedded in VM 104's config references IP `10.250.50.81`, suggesting VLAN 50 maps to the `10.250.50.0/24` subnet within the `10.250.0.0/16` range.
docs/pfi/recommended-model-settings.md
+554
View File
@@ -0,0 +1,554 @@
# Recommended Model Inference Settings — Reference Document
> **Source:** AIPA Knowledge Base — compiled from 5 KB reference documents.
> **Last Updated:** 2025-07-14
> **Purpose:** Canonical reference for llama-server / llama.cpp inference parameters across all model families with KB-documented settings.
---
## Table of Contents
1. [NVIDIA Nemotron 3 Super (120B-A12B)](#1-nvidia-nemotron-3-super-120b-a12b)
2. [NVIDIA Nemotron 3 Nano (4B / 30B-A3B)](#2-nvidia-nemotron-3-nano-4b--30b-a3b)
3. [Qwen 3.5 Family (0.8B – 397B-A17B)](#3-qwen-35-family-08b--397b-a17b)
4. [Qwen3-Coder-Next (80B MoE)](#4-qwen3-coder-next-80b-moe)
5. [Google Gemma 4 Family (E2B – 31B)](#5-google-gemma-4-family-e2b--31b)
6. [Quick Reference Cards](#6-quick-reference-cards)
7. [Critical Warnings by Model](#7-critical-warnings-by-model)
8. [Models Without KB Settings](#8-models-without-kb-settings)
---
## 1. NVIDIA Nemotron 3 Super (120B-A12B)
### Model Overview
| Property | Value |
|---|---|
| Architecture | MoE — 120B total, **12B active parameters** |
| Max Context | **1,048,576** (1M tokens) |
| Recommended Starting Context | **16K or 32K** — increase gradually |
| Reasoning Tokens | `<think)>` (ID 12), `</think)>` (ID 13) |
| Positional Embeddings | **NoPE** — YaRN NOT needed |
| Best For | Multi-agent AI, high-efficiency reasoning, coding, math |
| Performance Tier | ~GPT-5.2 / Claude Opus 4.5 level |
### Inference Parameters
| Parameter | General Chat / Instruction | Tool Calling |
|---|---|---|
| `temperature` | **1.0** | **0.6** |
| `top_p` | **1.0** | **0.95** |
| `min_p` | **0.01** | **0.01** |
### Additional Settings
| Setting | Value | Notes |
|---|---|---|
| `--seed` | **3407** | Reproducibility |
| `--prio` | **2** or **3** | Priority scheduling |
| `--special` | Required | To see reasoning tokens |
| `--verbose-prompt` | Required | To see prepended `<think)>` tokens |
| `max_new_tokens` | 32,768 – 262,144 | Up to 1M |
### Quantization & Memory
| Precision | Memory Required |
|---|---|
| UD-Q2_K_XL (2-bit) | ~32–36 GB |
| **UD-Q4_K_XL (4-bit)** | **~64–72 GB** |
| 8-bit | ~128 GB |
| BF16 | ~240 GB |
### Example Command
```bash
./llama.cpp/llama-server \
--model Nemotron-3-Super-UD-Q4_K_XL.gguf \
--ctx-size 16384 \
--temp 1.0 --top-p 1.0 --min-p 0.01 \
--seed 3407 --special \
--flash-attn on \
--port 8001
```
---
## 2. NVIDIA Nemotron 3 Nano (4B / 30B-A3B)
### Model Variants
| Variant | Architecture | Context | Active Params | Best Fit |
|---|---|---|---|---|
| **Nano-4B** | Dense | 128K | 4B | Lightweight coding, math, agentic tasks |
| **Nano-30B-A3B** | MoE | 128K | 3B active | Best performance/size on 24GB devices |
### Inference Parameters (Both Variants)
| Parameter | General Chat / Instruction | Tool Calling |
|---|---|---|
| `temperature` | **1.0** | **0.6** |
| `top_p` | **1.0** | **0.95** |
| `min_p` | **0.01** | **0.01** |
### Additional Settings
| Setting | Value | Notes |
|---|---|---|
| `--seed` | **3407** | Reproducibility |
| `--special` | Not required | Standard chat template for Nano variants |
### Hardware & Quantization
#### Nano-4B
| Precision | Memory |
|---|---|
| Q8_0 (8-bit, recommended) | ~3 GB |
| 4-bit | ~5 GB |
#### Nano-30B-A3B
| Precision | Memory |
|---|---|
| **UD-Q4_K_XL (4-bit, recommended)** | **~24 GB** |
| 8-bit | ~36 GB |
### Example Commands
```bash
# Nano-4B (8-bit)
./llama.cpp/llama-server \
-hf unsloth/Nemotron-3-Nano-4B-GGUF:Q8_0 \
--ctx-size 16384 \
--temp 1.0 --top-p 1.0 --min-p 0.01 \
--seed 3407 --flash-attn on --port 8001
# Nano-30B-A3B (4-bit)
./llama.cpp/llama-server \
-hf unsloth/Nemotron-3-Nano-30B-A3B-GGUF:UD-Q4_K_XL \
--ctx-size 16384 \
--temp 1.0 --top-p 1.0 --min-p 0.01 \
--seed 3407 --flash-attn on --port 8001
```
---
## 3. Qwen 3.5 Family (0.8B – 397B-A17B)
### Model Variants
| Variant | Architecture | Context | Languages | Best Fit |
|---|---|---|---|---|
| **0.8B** | Dense | 256K | 201 | Smallest edge inference |
| **2B** | Dense | 256K | 201 | Small device inference |
| **4B** | Dense | 256K | 201 | Lightweight local use |
| **9B** | Dense | 256K | 201 | Capable small model |
| **27B** | Dense | 256K | 201 | Slightly more accurate than 35B-A3B; fits 18GB |
| **35B-A3B** | MoE (3B active) | 256K | 201 | Best speed/quality tradeoff; fits 22GB |
| **122B-A10B** | MoE (10B active) | 256K | 201 | High quality; needs ~70GB (4-bit) |
| **397B-A17B** | MoE (17B active) | 256K (extendable to 1M via YaRN) | 201 | Top-tier performance |
### 27B vs 35B-A3B Decision
- **27B** — Choose for slightly more accurate results when you can't fit a larger model.
- **35B-A3B** — Choose for much faster inference. MoE with only 3B active parameters.
### Hardware Requirements
| Variant | 3-bit | 4-bit | 6-bit | 8-bit | BF16 |
|---|---|---|---|---|---|
| **0.8B / 2B** | 3 GB | 3.5 GB | 5 GB | 7.5 GB | 9 GB |
| **4B** | 4.5 GB | 5.5 GB | 7 GB | 10 GB | 14 GB |
| **9B** | 5.5 GB | 6.5 GB | 9 GB | 13 GB | 19 GB |
| **27B** | 14 GB | 17 GB | 24 GB | 30 GB | 54 GB |
| **35B-A3B** | 17 GB | 22 GB | 30 GB | 38 GB | 70 GB |
| **122B-A10B** | 60 GB | 70 GB | 106 GB | 132 GB | 245 GB |
| **397B-A17B** | 180 GB | 214 GB | 340 GB | 512 GB | 810 GB |
### Inference Parameters
#### Thinking Mode
| Parameter | General Tasks | Precise Coding (e.g. WebDev) |
|---|---|---|
| `temperature` | **1.0** | **0.6** |
| `top_p` | **0.95** | **0.95** |
| `top_k` | **20** | **20** |
| `min_p` | **0.0** | **0.0** |
| `presence_penalty` | **1.5** | **0.0** |
| `repetition_penalty` | **1.0** (disabled) | **1.0** (disabled) |
#### Non-Thinking (Instruct) Mode
| Parameter | General Tasks | Reasoning Tasks |
|---|---|---|
| `temperature` | **0.7** | **1.0** |
| `top_p` | **0.8** | **0.95** |
| `top_k` | **20** | **20** |
| `min_p` | **0.0** | **0.0** |
| `presence_penalty` | **1.5** | **1.5** |
| `repetition_penalty` | **1.0** (disabled) | **1.0** (disabled) |
### Thinking Mode Control
Enable thinking:
```bash
--chat-template-kwargs '{"enable_thinking":true}'
```
Disable thinking:
```bash
--chat-template-kwargs '{"enable_thinking":false}'
```
#### Default Thinking Behavior by Variant
| Variant | Thinking Default |
|---|---|
| **0.8B, 2B, 4B, 9B** (Small) | **Disabled** — must explicitly enable |
| **27B, 35B-A3B, 122B-A10B, 397B-A17B** | **Enabled** — must explicitly disable if unwanted |
### Context Settings
| Setting | Value |
|---|---|
| Max context window | **262,144** (256K) |
| Context extension | Up to **1M** via YaRN |
| Recommended starting context | **16,384** (16K) for responsiveness |
| Adequate output length | **32,768** tokens |
### Quantization Notes
- All GGUFs use **Unsloth Dynamic 2.0** quantization — important layers upcasted to 8 or 16-bit even in 4-bit.
- Recommended starting point: **Dynamic 4-bit** (`UD-Q4_K_XL`).
- Minimum recommended: **Dynamic 2-bit** (`UD-Q2_K_XL`).
### Example Commands
```bash
# 35B-A3B — Thinking Mode (General)
./llama.cpp/llama-server \
-hf unsloth/Qwen3.5-35B-A3B-GGUF:UD-Q4_K_XL \
--ctx-size 16384 \
--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.00 \
--chat-template-kwargs '{"enable_thinking":true}' \
--flash-attn on --port 8001
# 9B — Thinking Enabled (small models default to disabled)
./llama.cpp/llama-server \
-hf unsloth/Qwen3.5-9B-GGUF:UD-Q4_K_XL \
--ctx-size 16384 \
--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.00 \
--chat-template-kwargs '{"enable_thinking":true}' \
--flash-attn on --port 8001
```
---
## 4. Qwen3-Coder-Next (80B MoE)
### Model Overview
| Property | Value |
|---|---|
| Architecture | MoE — 80B total, **3B active parameters** |
| Max Context | **262,144** (256K) |
| Recommended Context | **32,768** for less memory use |
| Thinking Mode | **Non-reasoning only** — no `<think)>` blocks |
| Best For | Fast agentic coding, long-horizon reasoning, complex tool use |
| Performance Tier | Comparable to models with 10–20× more active parameters |
### Inference Parameters
| Parameter | Value | Notes |
|---|---|---|
| `temperature` | **1.0** | |
| `top_p` | **0.95** | |
| `top_k` | **40** | Note: higher than Qwen3.5 general |
| `min_p` | **0.01** | llama.cpp default is 0.05 — override to 0.01 |
| `repetition_penalty` | **1.0** (disabled) | Only increase if you see looping |
### Quick Reference One-Liner
```
temperature=1.0, top_p=0.95, top_k=40, min_p=0.01, repetition_penalty=1.0
```
### Hardware Requirements
| Precision | Memory Required |
|---|---|
| 3-bit (UD-IQ3_XXS) | ~34 GB |
| **4-bit (UD-Q4_K_XL)** | **~46 GB** |
| 8-bit | ~85 GB |
| BF16 | ~160 GB |
### Key Differences from Qwen3.5 General Models
- **No thinking mode** — this is a non-reasoning model; ultra-quick code responses
- **Higher `top_k`** (40 vs. 20) — broader sampling for creative code generation
- `enable_thinking` flag is not applicable
### Example Commands
```bash
# llama-server deployment
./llama.cpp/llama-server \
--model Qwen3-Coder-Next-UD-Q4_K_XL.gguf \
--alias "unsloth/Qwen3-Coder-Next" \
--seed 3407 \
--temp 1.0 --top-p 0.95 --min-p 0.01 --top-k 40 \
--ctx-size 32768 \
--flash-attn on --port 8001
```
### vLLM FP8 Dynamic (GPU Premium)
```bash
CUDA_VISIBLE_DEVICES='0,1,2,3' vllm serve unsloth/Qwen3-Coder-Next-FP8-Dynamic \
--served-model-name unsloth/Qwen3-Coder-Next \
--tensor-parallel-size 4 \
--tool-call-parser qwen3_coder \
--enable-auto-tool-choice \
--dtype bfloat16 --seed 3407 \
--max-model-len 200000 \
--gpu-memory-utilization 0.93 \
--port 8001
```
---
## 5. Google Gemma 4 Family (E2B – 31B)
### Model Variants
| Variant | Architecture | Context | Modalities | Best Fit |
|---|---|---|---|---|
| **E2B** | Dense + PLE | 128K | Text, Image, Audio | Phone / edge, ASR, speech translation |
| **E4B** | Dense + PLE | 128K | Text, Image, Audio | Laptops, fast local multimodal |
| **26B-A4B** | MoE (4B active) | 256K | Text, Image | Best speed/quality tradeoff |
| **31B** | Dense | 256K | Text, Image | Strongest performance |
### Inference Parameters (All Variants)
These are Google's default Gemma 4 parameters:
| Parameter | Value |
|---|---|
| `temperature` | **1.0** |
| `top_p` | **0.95** |
| `top_k` | **64** |
| `repetition_penalty` | **1.0** (disabled — only increase if looping) |
| End-of-sentence token | `<turn|>` |
### Context Length
- **E2B / E4B**: max **128K**
- **26B-A4B / 31B**: max **256K**
- **Practical tip**: Start with **32K** for responsiveness, then increase as needed.
### Hardware Requirements
| Variant | 4-bit | 8-bit | BF16/FP16 |
|---|---|---|---|
| **E2B** | 4 GB | 5–8 GB | 10 GB |
| **E4B** | 5.5–6 GB | 9–12 GB | 16 GB |
| **26B-A4B** | 16–18 GB | 28–30 GB | 52 GB |
| **31B** | 17–20 GB | 34–38 GB | 62 GB |
### Quantization Recommendations
- **E2B / E4B** (small): prefer **Q8_0** (8-bit) for quality.
- **26B-A4B / 31B** (large): prefer **UD-Q4_K_XL** (Dynamic 4-bit) as starting point.
### 26B-A4B vs 31B Decision
- **26B-A4B** — Choose when RAM is limited. MoE with 4B active params = faster. Slight quality tradeoff.
- **31B** — Choose when you have ≥20 GB (4-bit) and want maximum quality. Slower inference.
### Thinking Mode
**Enable thinking** — add to system prompt:
```
<|think|>
You are a careful coding assistant. Explain your answer clearly.
```
Model outputs:
```
<|channel>thought
[internal reasoning]
<channel|>
[final answer]
```
**Disable thinking** via llama-server:
```bash
--chat-template-kwargs '{"enable_thinking":false}'
```
> **Multi-turn rule:** Only keep the final visible answer in chat history. Do NOT feed prior thought blocks back.
### Multimodal Settings
- **Images/Audio** go **before text** in prompts.
- **Video**: pass frames first, then instruction.
- **Audio** is only on E2B and E4B. Max audio: 30s. Max video: 60s (1 fps).
#### Visual Token Budgets
| Budget | Use Case |
|---|---|
| 70 / 140 | Classification, captioning, fast video |
| 280 / 560 | General multimodal chat, charts, UI |
| 1120 | OCR, document parsing, handwriting |
### Example Commands
```bash
# 26B-A4B (Dynamic 4-bit)
./llama.cpp/llama-server \
-hf unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q4_K_XL \
--temp 1.0 --top-p 0.95 --top-k 64 \
--ctx-size 32768 --flash-attn on --port 8001
# 31B (Dynamic 4-bit) with thinking
./llama.cpp/llama-server \
-hf unsloth/gemma-4-31B-it-GGUF:UD-Q4_K_XL \
--temp 1.0 --top-p 0.95 --top-k 64 \
--ctx-size 32768 \
--chat-template-kwargs '{"enable_thinking":true}' \
--flash-attn on --port 8001
```
---
## 6. Quick Reference Cards
### Temperature by Model & Use Case
| Model Family | General / Chat | Coding / Precise | Tool Calling | Non-Thinking |
|---|---|---|---|---|
| **Nemotron 3 Super** | 1.0 | — | 0.6 | — |
| **Nemotron 3 Nano** | 1.0 | — | 0.6 | — |
| **Qwen 3.5** (thinking) | 1.0 | 0.6 | — | — |
| **Qwen 3.5** (non-thinking) | — | — | — | 0.7 (general) / 1.0 (reasoning) |
| **Qwen3-Coder-Next** | 1.0 | 1.0 | — | — |
| **Gemma 4** | 1.0 | 1.0 | — | — |
### Top-P by Model & Use Case
| Model Family | General / Chat | Coding / Precise | Tool Calling | Non-Thinking |
|---|---|---|---|---|
| **Nemotron 3 Super** | 1.0 | — | 0.95 | — |
| **Nemotron 3 Nano** | 1.0 | — | 0.95 | — |
| **Qwen 3.5** (thinking) | 0.95 | 0.95 | — | — |
| **Qwen 3.5** (non-thinking) | — | — | — | 0.8 (general) / 0.95 (reasoning) |
| **Qwen3-Coder-Next** | 0.95 | 0.95 | — | — |
| **Gemma 4** | 0.95 | 0.95 | — | — |
### Top-K by Model
| Model Family | Top-K |
|---|---|
| **Nemotron 3** | Default (not specified) |
| **Qwen 3.5** | **20** |
| **Qwen3-Coder-Next** | **40** |
| **Gemma 4** | **64** |
### Min-P by Model
| Model Family | Min-P |
|---|---|
| **Nemotron 3 (all)** | **0.01** |
| **Qwen 3.5** | **0.0** |
| **Qwen3-Coder-Next** | **0.01** |
| **Gemma 4** | Default (not specified) |
### Presence Penalty by Model
| Model Family | General | Coding | Reasoning |
|---|---|---|---|
| **Nemotron 3** | Default | Default | Default |
| **Qwen 3.5** (thinking) | **1.5** | **0.0** | **1.5** |
| **Qwen 3.5** (non-thinking) | **1.5** | — | **1.5** |
| **Qwen3-Coder-Next** | Default | Default | Default |
| **Gemma 4** | Default | Default | Default |
### Repetition Penalty by Model
| Model Family | Value | Notes |
|---|---|---|
| **Nemotron 3** | Default | — |
| **Qwen 3.5** | **1.0** (disabled) | — |
| **Qwen3-Coder-Next** | **1.0** (disabled) | Only increase if looping |
| **Gemma 4** | **1.0** (disabled) | Only increase if looping |
### Seed Values
| Model Family | Recommended Seed |
|---|---|
| **Nemotron 3 (all)** | **3407** |
| **Qwen 3.5** | 3407 (optional) |
| **Qwen3-Coder-Next** | **3407** |
| **Gemma 4** | Default |
---
## 7. Critical Warnings by Model
### Nemotron 3 Super
- ⚠️ Do NOT attempt 1M context on first run — increase gradually from 16K/32K.
- ⚠️ Setting context to 1M may trigger **CUDA OOM and crash**.
- ⚠️ Router-layer fine-tuning **disabled by default** in Unsloth for MoE models.
### Qwen 3.5
- ⚠️ **No Qwen3.5 GGUF works in Ollama** due to separate mmproj vision files. Use llama.cpp-compatible backends.
- ⚠️ `presence_penalty` above 0.0 may cause **slight performance decrease**.
- ⚠️ If getting gibberish, check context length or add `--cache-type-k bf16 --cache-type-v bf16`.
- ⚠️ Redownload older GGUFs — all updated with improved quantization and tool-calling template fixes.
### Qwen3-Coder-Next
- ⚠️ **Update llama.cpp** — a previous bug in `vectorized key_gdiff` caused looping/output issues.
- ⚠️ **No Ollama support** — use llama.cpp-compatible backends.
- ⚠️ If context length too low, may see `exceeds the available context size` errors.
- ⚠️ Tool-calling improved after llama.cpp parsing fixes (Feb 19 update) — use recent version.
### Gemma 4
- ⚠️ **Do NOT use CUDA 13.2 runtime** for any GGUF — causes poor outputs.
- ⚠️ Use `llama-server` (not `llama-cli`) for thinking control — more reliable.
- ⚠️ Multi-turn: **only keep the final visible answer** in chat history. Do NOT feed prior thought blocks back.
---
## 8. Models Without KB Settings
The following model families are deployed in the Infrastructure-PFI environment but **do not have KB-documented inference parameters**. Settings for these models use general best practices or vendor defaults:
| Model | Notes |
|---|---|
| **DeepSeek R1 0528** | No KB doc — use general MoE defaults |
| **Mistral Small 3.1** | No KB doc — use vendor defaults |
| **GLM-4.7-Flash** | No KB doc — use vendor defaults |
| **GLM Steam 106B-A12B** | No KB doc — use general MoE defaults |
| **Granite 4.0 Micro** | No KB doc — use IBM defaults |
| **Kimi K2** | No KB doc — use general MoE defaults |
| **Skyfall R1 31B v4a** | No KB doc — use general defaults |
| **Hermes 4 14B** | No KB doc (download pending) — use vendor defaults |
---
## KB Source Documents
| Document | Path in KB |
|---|---|
| Nemotron 3 Super Running Parameters | `reference/nemotron-3-super-running-parameters.md` |
| Nemotron 3 Nano Running Parameters | `reference/nemotron-3-nano-running-parameters.md` |
| Qwen 3.5 Running Parameters | `reference/qwen3.5-running-parameters.md` |
| Qwen3-Coder-Next Running Parameters | `reference/qwen3-coder-next-running-parameters.md` |
| Gemma 4 Running Parameters | `reference/gemma-4-running-parameters.md` |
---
*Document generated by Linus (Systems Architect) from AIPA Knowledge Base content curated by Atlas.*
docs/pfi/vm-102-matrix-appservice.md
+261
View File
@@ -0,0 +1,261 @@
---
id: vm-102-matrix-appservice
title: "VM 102 — Matrix Appservice Configuration"
summary: "AIPA Matrix Application Service bridge setup for VM 102. Covers appservice registration, agent virtual users, room routing, env vars, operational notes, and troubleshooting."
tags: ["infrastructure", "pfi", "pfi-ana", "matrix", "docker", "vm-102", "appservice", "bridge", "aipa", "deployment", "integration"]
keywords: ["matrix.pfi.local", "appservice", "as_token", "hs_token", "aipa-bridge", "matrix_bridge.py", "8009", "send_notification", "virtual-users", "room-routing", "element"]
links:
- "[VM 102 — Matrix Synapse Deployment](vm-102-matrix-synapse.md)"
- "[Matrix Bridge — Application Service Integration](../../../KB/projects/aipa/matrix-bridge.md)"
- "[Docker Stack Conventions](docker-stack.md)"
created: 2026-04-11T00:00:00+00:00
modified: 2026-04-11T00:00:00+00:00
path: docs/pfi-ana/vm-102-matrix-appservice.md
---
# VM 102 — Matrix Appservice Configuration
## Overview
The AIPA Matrix bridge registers with Synapse as a Matrix Application Service. This means:
- Synapse **pushes** all relevant events to the bridge (no polling)
- The bridge manages **virtual users** for each agent — no real accounts needed
- The bridge authenticates to Synapse with a shared `as_token`
- Synapse authenticates its pushes to the bridge with a shared `hs_token`
> Prerequisite: [VM 102 — Matrix Synapse Deployment](vm-102-matrix-synapse.md) must be complete.
---
## Application Service Model
```
Synapse ──── push events ────► AIPA Bridge (port 8009)
│
routes to agent
│
posts response ◄──── Synapse ◄──── Element ◄──── User
```
### Agent Virtual Users
| Agent | Matrix ID | Display Name |
|----------|----------------------------|--------------|
| Atlas | `@atlas:matrix.pfi.local` | Atlas |
| Linus | `@linus:matrix.pfi.local` | Linus |
| Hermione | `@hermione:matrix.pfi.local` | Hermione |
These are **virtual** — managed entirely by the bridge. Do not register them as real Synapse accounts.
---
## Step 1 — Generate Tokens
Two random tokens are needed:
```bash
python3 -c "import secrets; print(secrets.token_hex(32))" # as_token (bridge → Synapse)
python3 -c "import secrets; print(secrets.token_hex(32))" # hs_token (Synapse → bridge)
```
---
## Step 2 — Create Appservice Registration File
File: `/opt/docker/data/synapse/aipa_appservice.yaml`
```yaml
id: aipa-bridge
url: http://<BRIDGE_HOST_IP>:8009 # IP/hostname the bridge is reachable from Synapse container
as_token: "<AS_TOKEN_FROM_STEP_1>"
hs_token: "<HS_TOKEN_FROM_STEP_1>"
sender_localpart: aipa-bot # @aipa-bot:matrix.pfi.local (unused fallback sender)
namespaces:
users:
- exclusive: true
regex: "@(atlas|linus|hermione):.*"
rooms: []
aliases: []
rate_limited: false
```
- **`exclusive: true`** — only the bridge can act as those users; no one can register `@atlas` as a real account.
- **`url`** — must be reachable from inside the Synapse container. If the bridge runs on the VM host, use the host IP or Docker gateway IP (e.g., `172.17.0.1`).
### Restart Synapse to Load
```bash
cd /opt/docker/compose/synapse && docker compose restart synapse
```
---
## Step 3 — Configure AIPA Environment
### env.sh additions
```bash
# ── Matrix bridge ──────────────────────────────────────────────────────
export MATRIX_HOMESERVER_URL="http://localhost:8008"
export MATRIX_SERVER_NAME="matrix.pfi.local"
export MATRIX_AS_TOKEN="<as_token from Step 1>"
export MATRIX_HS_TOKEN="<hs_token from Step 1>"
# Optional: room to send agent notifications when no channel is specified
# export MATRIX_DEFAULT_ROOM="!roomid:matrix.pfi.local"
```
### providers.yaml matrix section
```yaml
matrix:
homeserver_url: "${MATRIX_HOMESERVER_URL}"
server_name: "${MATRIX_SERVER_NAME}"
as_token: "${MATRIX_AS_TOKEN}"
hs_token: "${MATRIX_HS_TOKEN}"
bridge_host: "0.0.0.0"
bridge_port: 8009
default_notification_room: "${MATRIX_DEFAULT_ROOM}"
agents:
atlas:
display_name: "Atlas"
avatar_url: ""
linus:
display_name: "Linus"
avatar_url: ""
hermione:
display_name: "Hermione"
avatar_url: ""
```
---
## Step 4 — Start the Bridge
```bash
source .venv/bin/activate && source env.sh
python -m core.matrix_bridge
```
Options:
```
python -m core.matrix_bridge --host 0.0.0.0 --port 8009
python -m core.matrix_bridge --no-profile-setup # skip display name / avatar init
```
On first start, the bridge:
1. Registers display names for each agent user
2. Starts the appservice HTTP server on port 8009
3. Listens for Matrix transactions from Synapse
4. Accepts invitations to rooms
5. Begins routing messages
---
## Step 5 — First Use
1. Open Element at `http://10.250.50.70:8080`
2. Log in with your admin account (server: `matrix.pfi.local`)
3. Open a DM with `@atlas:matrix.pfi.local`
4. Send a message — the bridge routes it to Atlas and posts the response
To start a room with a specific agent, invite them:
- New room → Invite `@linus:matrix.pfi.local` → Linus joins automatically
- All messages in that room go to Linus
---
## Room Routing
1. User invites an agent user to a room (or opens a DM)
2. Bridge receives the `m.room.member` invite event, agent auto-joins
3. Room is permanently mapped to that agent in `sessions/matrix_rooms.json`
4. All subsequent messages in that room are routed to the mapped agent
5. If multiple agent users are in the same room, the first invite wins for session purposes;
subsequent agents each get their own session (multi-agent collaboration rooms)
---
## Operational Notes
### Room Session Persistence
The bridge persists the room→agent→session mapping to `sessions/matrix_rooms.json`
under `AIPA_ROOT`. If the bridge restarts, existing rooms continue their sessions.
To reset a room's conversation history:
- Ask the agent `/reset`, or
- Delete the entry from `matrix_rooms.json` and restart the bridge.
### Formatting
The bridge converts markdown in agent responses to Matrix HTML
(`format: org.matrix.custom.html`). Code blocks, bold, and inline code are
rendered correctly in Element.
### Typing Indicators
The bridge sends a typing indicator (`m.typing`) while the agent is processing,
so users see the animated dots while waiting.
### Proactive Notifications
Agents can push messages to Matrix rooms via `send_notification`:
```
send_notification(
content="KB rebuild complete — 1,247 chunks indexed.",
channel="!roomid:matrix.pfi.local"
)
```
- If `channel` is set, it must be a Matrix room ID (`!roomid:server`)
- If `channel` is omitted, the notification goes to `MATRIX_DEFAULT_ROOM` if configured
### Agent Users Always Show as Offline
Expected behavior — virtual users don't have presence. This is normal for appservice users.
---
## Troubleshooting
| Symptom | Likely Cause | Fix |
|---------|-------------|-----|
| Bridge starts but Synapse doesn't push events | Appservice URL wrong in registration YAML | Verify `url:` is reachable from the Synapse container; check `docker inspect synapse` network |
| Agent joins room but doesn't respond | `hs_token` mismatch | Verify token in `aipa_appservice.yaml` matches `MATRIX_HS_TOKEN` env var |
| 401 errors from Synapse | `as_token` mismatch | Verify token in `aipa_appservice.yaml` matches `MATRIX_AS_TOKEN` env var |
| Agent user shows as offline always | Expected — virtual users don't have presence | Normal for appservice users; no fix needed |
| Messages loop (agent replies to itself) | Bridge not filtering its own messages | Check `SENDER_LOCALPART` filter in bridge; bridge ignores all managed agent users as senders |
| Element can't connect to homeserver | Wrong `base_url` in element-config.json | Must be the IP/hostname Element's browser can reach, not the Docker container name |
---
## Dependencies
| Dependency | Version | Purpose |
|------------|---------|---------|
| `markdown` | any | Markdown→HTML rendering for Matrix responses |
| `fastapi` | any | Appservice HTTP server |
| `httpx` | any | Client-Server API calls to Synapse |
Install: `pip install markdown` (or `pip install -r requirements.txt`)
---
## Security Notes
- The `as_token` and `hs_token` are secrets equivalent to admin credentials. Store
them in `env.sh` (gitignored) and never commit them.
- The bridge runs with `MATRIX_HS_TOKEN` to authenticate Synapse's push requests. Any
request without this token in the `Authorization` header is rejected (403).
- Restrict port 8009 to internal access only (firewall or bind to `127.0.0.1` if bridge and Synapse are on the same host).
---
## Sources
- Source deployment guide: `projects/matrix/matrix-deployment.md` (2026-04-11)
- AIPA bridge code: `core/matrix_bridge.py`
- AIPA bridge KB entry: `KB/projects/aipa/matrix-bridge.md`
- Matrix Synapse appservice docs: https://element-hq.github.io/synapse/latest/application_services.html
docs/pfi/vm-102-matrix-synapse.md
+340
View File
@@ -0,0 +1,340 @@
---
id: vm-102-matrix-synapse
title: "VM 102 — Matrix Synapse Deployment"
summary: "Docker deployment of Synapse homeserver with PostgreSQL and Element Web on VM 102 (PFI-ANA-Docker). Covers compose file, configuration, storage paths, and admin setup."
tags: ["infrastructure", "pfi", "pfi-ana", "matrix", "docker", "vm-102", "synapse", "self-hosted", "deployment"]
keywords: ["matrix.pfi.local", "synapse", "element-web", "10.250.50.70", "VM-102", "PFI-ANA-Docker", "postgres:16", "appservice", "AIPA"]
links:
- "[Matrix Protocol Reference](../Infrastructure-PFI-Project-Index.md)"
- "[VM-102 Proxmox Config](../../../configs/pfi-ana/proxmox/vm-102.conf)"
- "[Docker Stack Conventions](docker-stack.md)"
created: 2026-04-11T00:00:00+00:00
modified: 2026-04-11T00:00:00+00:00
path: docs/pfi-ana/vm-102-matrix-synapse.md
---
# VM 102 — Matrix Synapse Deployment
## Overview
Matrix (Synapse) is deployed on VM 102 (PFI-ANA-Docker, `10.250.50.70`) as the primary
human-to-agent communication channel for AIPA. The stack consists of:
- **Synapse** — Matrix homeserver (event routing, auth, persistence)
- **PostgreSQL 16** — Synapse database backend
- **Element Web** — Web client for users
Federation is disabled (internal-only deployment). Registration is disabled (admin-created accounts only).
---
## Architecture
```
User (Element client)
│ m.room.message events
▼
┌──────────────────────┐
│ Synapse │ Matrix homeserver — event routing, auth, persistence
│ (homeserver) │ Port 8008 (Client-Server API)
└──────────┬───────────┘
│ Appservice push PUT /_matrix/app/v1/transactions/{txnId}
▼
┌──────────────────────────────────────────┐
│ AIPA Matrix Bridge (core/matrix_bridge.py) │
│ Port 8009 │
└──────────────────────────────────────────┘
```
> Full appservice bridge details: [VM 102 — Matrix Appservice Configuration](vm-102-matrix-appservice.md)
---
## Components
| Component | Image | Port | Purpose |
|-----------------|--------------------------------|-------|-----------------------------------|
| `synapse` | `matrixdotorg/synapse:latest` | 8008 | Matrix homeserver (Client-Server API) |
| `synapse-db` | `postgres:16` | — | Synapse database |
| `element-web` | `vectorim/element-web:latest` | 8080 | Web client for users |
---
## Storage Paths
| Host Path | Purpose |
|------------------------------------|-----------------------------------------|
| `/opt/docker/compose/synapse/` | Compose file (managed by Dockge) |
| `/opt/docker/conf/synapse/` | Configuration files |
| `/opt/docker/data/synapse/` | Synapse data (`homeserver.yaml`, signing keys, appservice reg) |
| `/opt/docker/conf/synapse/element-config.json` | Element Web client config |
> Follows the VM 102 convention: compose in `/opt/docker/compose/<service>/`, config in `/opt/docker/conf/<service>/`.
### Volumes
| Volume / Path | Container Path | Purpose |
|--------------------------------|-------------------------------|-----------------------------|
| `/opt/docker/data/synapse` | `/data` | Synapse config, signing keys, media store |
| `synapse-db-data` (named vol) | `/var/lib/postgresql/data` | PostgreSQL persistent data |
---
## Step 1 — Create Directory Structure
```bash
mkdir -p /opt/docker/data/synapse
mkdir -p /opt/docker/conf/synapse
mkdir -p /opt/docker/compose/synapse
```
---
## Step 2 — Generate Synapse Config
```bash
docker run --rm \
-v /opt/docker/data/synapse:/data \
-e SYNAPSE_SERVER_NAME=matrix.pfi.local \
-e SYNAPSE_REPORT_STATS=no \
matrixdotorg/synapse:latest generate
```
This writes `/opt/docker/data/synapse/homeserver.yaml` and
`/opt/docker/data/synapse/matrix.pfi.local.signing.key`.
**Do not modify `server_name` after generation — it is permanent.**
---
## Step 3 — Edit homeserver.yaml
Open `/opt/docker/data/synapse/homeserver.yaml` and apply:
```yaml
# Use PostgreSQL instead of SQLite (required for production)
database:
name: psycopg2
args:
user: synapse
password: synapse_db_password # match POSTGRES_PASSWORD in compose
database: synapse
host: synapse-db
cp_min: 5
cp_max: 10
# Disable open registration — accounts are created by admin only
enable_registration: false
# Disable federation (internal deployment only)
federation_domain_whitelist: []
# Allow the application service to be registered (add AFTER generating the AS file)
app_service_config_files:
- /data/aipa_appservice.yaml
```
---
## Step 4 — Docker Compose
File: `/opt/docker/compose/synapse/docker-compose.yml`
```yaml
---
# =============================================================================
# Synapse Matrix Homeserver — AIPA internal deployment on VM 102
# =============================================================================
#
# Conventions:
# - Config: /opt/docker/conf/synapse/
# - Data: /opt/docker/data/synapse/ (bind mount) + synapse-db-data (named vol)
# - Compose: /opt/docker/compose/synapse/
# - Network: synapse-net (dedicated, not on traefik-net)
#
# Notes:
# - Federation disabled (internal only)
# - Port 8448 (federation) commented out
# - Resource limits set for VM 102 (8 vCPU, 16 GB RAM)
services:
synapse-db:
image: postgres:16
container_name: synapse-db
restart: unless-stopped
environment:
POSTGRES_USER: synapse
POSTGRES_PASSWORD: synapse_db_password
POSTGRES_DB: synapse
POSTGRES_INITDB_ARGS: "--encoding=UTF-8 --lc-collate=C --lc-ctype=C"
volumes:
- synapse-db-data:/var/lib/postgresql/data
networks:
- synapse-net
healthcheck:
test: ["CMD-SHELL", "pg_isready -U synapse"]
interval: 10s
timeout: 5s
retries: 5
synapse:
image: matrixdotorg/synapse:latest
container_name: synapse
restart: unless-stopped
depends_on:
synapse-db:
condition: service_healthy
ports:
- "8008:8008" # Client-Server API (HTTP)
# - "8448:8448" # Server-Server API (federation) — disabled for internal use
volumes:
- /opt/docker/data/synapse:/data
networks:
- synapse-net
deploy:
resources:
limits:
memory: 1G
cpus: "2.0"
reservations:
memory: 256M
healthcheck:
test: ["CMD-SHELL", "curl -fsS http://localhost:8008/health || exit 1"]
interval: 30s
timeout: 10s
retries: 3
start_period: 30s
element-web:
image: vectorim/element-web:latest
container_name: element-web
restart: unless-stopped
volumes:
- /opt/docker/conf/synapse/element-config.json:/app/config.json:ro
ports:
- "8080:80"
networks:
- synapse-net
volumes:
synapse-db-data:
networks:
synapse-net:
name: synapse-net
```
### Start
```bash
cd /opt/docker/compose/synapse
docker compose up -d
```
### Verify
```bash
docker compose ps
curl http://localhost:8008/health
```
---
## Step 5 — Element Web Configuration
File: `/opt/docker/conf/synapse/element-config.json`
```json
{
"default_server_config": {
"m.homeserver": {
"base_url": "http://10.250.50.70:8008",
"server_name": "matrix.pfi.local"
}
},
"brand": "AIPA",
"default_theme": "dark",
"disable_guests": true,
"disable_login_language_selector": true
}
```
---
## Step 6 — Create Admin User
```bash
docker exec -it synapse register_new_matrix_user \
-u admin \
-p 'yourpassword' \
-a \
http://localhost:8008
```
> The `-a` flag makes the user an admin. Omit for regular users.
**Agent users** (`@atlas`, `@linus`, `@hermione`) are **virtual** — managed by the
appservice. Do **not** register them as real accounts.
---
## Port Allocation
| Port | Service | Purpose | Protocol |
|------|---------------|----------------------------|----------|
| 8008 | Synapse | Client-Server API | HTTP |
| 8080 | Element Web | Web client | HTTP |
| 8009 | AIPA Bridge | Appservice endpoint | HTTP |
> See [docker-stack.md](docker-stack.md) for the full VM 102 port allocation table.
---
## Network
This deployment uses a **dedicated `synapse-net` network** (not `traefik-net`), because:
- Synapse is accessed directly by IP (no public domain routing needed)
- The AIPA bridge connects to Synapse at `http://localhost:8008` from the host
- Element Web connects at the VM IP:8008 from the browser
If TLS/reverse proxy is added later, join `traefik-net` and add Traefik labels.
---
## Security Notes
- Synapse is exposed on port 8008 (HTTP). For any externally accessible deployment,
put it behind a TLS-terminating reverse proxy (Traefik/nginx) and restrict 8009
to internal access only.
- Registration is disabled (`enable_registration: false`) — accounts created by admin only.
- Federation is disabled (`federation_domain_whitelist: []`) — internal use only.
- The `as_token` and `hs_token` in the appservice registration are secrets equivalent to
admin credentials. Store in `env.sh` (gitignored), never commit.
---
## Troubleshooting
| Symptom | Likely Cause | Fix |
|---------|-------------|-----|
| Bridge starts but Synapse doesn't push events | Appservice URL wrong in registration YAML | Verify `url:` is reachable from the Synapse container; check `docker inspect synapse` network |
| 401 errors from Synapse | `as_token` mismatch | Verify token in `aipa_appservice.yaml` matches `MATRIX_AS_TOKEN` env var |
| Element can't connect to homeserver | Wrong `base_url` in element-config.json | Must be the IP/hostname Element's browser can reach, not the Docker container name |
| Synapse won't start | Database connection failure | Verify `synapse-db` is healthy first; check password matches in `homeserver.yaml` and compose |
---
## Next Steps
After Synapse is running and healthy, configure the AIPA appservice bridge:
→ [VM 102 — Matrix Appservice Configuration](vm-102-matrix-appservice.md)
---
## Sources
- Source deployment guide: `projects/matrix/matrix-deployment.md` (2026-04-11)
- VM 102 Proxmox config: `configs/pfi-ana/proxmox/vm-102.conf`
- Docker Stack conventions: `docs/pfi-ana/docker-stack.md`
- Matrix Protocol Reference: `infrastructure/matrix-docker-deployment.md` (KB)
scripts/add-host.sh
Executable
+69
View File
@@ -0,0 +1,69 @@
#!/usr/bin/env bash
# add-host.sh — register a new server so refresh-server-info.sh picks it up.
#
# Creates servers/<name>/ and writes servers/<name>/ssh-target with the
# given IP (or user@ip). The next `scripts/refresh-server-info.sh <name>`
# run will discover the host and pull its first system-details.txt.
#
# Usage:
# scripts/add-host.sh <name> <ip-or-user@ip>
# scripts/add-host.sh <name> <ip-or-user@ip> --force # overwrite existing
#
# Example:
# scripts/add-host.sh la-docker 10.60.50.12
# scripts/add-host.sh edge-01 admin@203.0.113.9
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
SERVERS_DIR="$REPO_ROOT/servers"
FORCE=0
POSITIONAL=()
for arg in "$@"; do
case "$arg" in
-f|--force) FORCE=1 ;;
-h|--help) sed -n '2,14p' "$0"; exit 0 ;;
-*) echo "error: unknown flag $arg" >&2; exit 2 ;;
*) POSITIONAL+=("$arg") ;;
esac
done
if [ "${#POSITIONAL[@]}" -ne 2 ]; then
echo "usage: $(basename "$0") <name> <ip-or-user@ip> [--force]" >&2
exit 2
fi
name="${POSITIONAL[0]}"
target="${POSITIONAL[1]}"
if [[ ! "$name" =~ ^[a-zA-Z0-9][a-zA-Z0-9._-]*$ ]]; then
echo "error: invalid host name '$name' (expected [a-zA-Z0-9._-])" >&2
exit 2
fi
# Trivial sanity check on the target — not exhaustive, just catches typos.
if [[ -z "$target" ]] || [[ "$target" =~ [[:space:]] ]]; then
echo "error: invalid ssh target '$target'" >&2
exit 2
fi
host_dir="$SERVERS_DIR/$name"
ssh_target_file="$host_dir/ssh-target"
if [ -f "$ssh_target_file" ] && [ "$FORCE" -ne 1 ]; then
existing=$(awk 'NF{print $1; exit}' "$ssh_target_file")
if [ "$existing" = "$target" ]; then
echo "host '$name' already registered with target '$target' — nothing to do"
exit 0
fi
echo "error: $ssh_target_file already exists (currently '$existing'); pass --force to overwrite" >&2
exit 1
fi
mkdir -p "$host_dir"
printf '%s\n' "$target" > "$ssh_target_file"
printf 'registered: %s → %s\n' "$name" "$target"
printf 'next: scripts/refresh-server-info.sh %s\n' "$name"
scripts/deploy-stack.sh
Executable
+257
View File
@@ -0,0 +1,257 @@
#!/usr/bin/env bash
# deploy-stack.sh — push a local stacks-mirror dir to a server, with
# per-file diff and confirmation prompt.
#
# Layout assumed:
# stacks-mirror/<host>/<stack>/<file> → <host>:/opt/docker/compose/<stack>/<file>
# stacks-mirror/<host>/<stack>/conf/<file> → <host>:/opt/docker/conf/<stack>/<file>
#
# Secrets / runtime state are never pushed (same exclude list as
# sync-stacks.sh): .env*, acme.json, *.key/crt/pem/pfx, *.sqlite*, *.db,
# *.log*, *.pid, hub/, logs/, client_secrets.json.
#
# The script:
# 1. Runs rsync --dry-run to find which files would change.
# 2. Prints a unified diff for each changed/added file (deletions noted).
# 3. Prompts [y/N]; applies the rsync only on 'y'.
#
# Usage:
# scripts/deploy-stack.sh <host> <stack>
# scripts/deploy-stack.sh <host> <stack> --yes # skip prompt (use sparingly)
# scripts/deploy-stack.sh <host> <stack> --compose # push only compose side
# scripts/deploy-stack.sh <host> <stack> --conf # push only conf side
set -euo pipefail
if ! command -v rsync >/dev/null 2>&1; then
echo "error: rsync is not installed on this workstation" >&2
echo " install it (e.g. 'sudo apt install rsync') and ensure the target host has it too" >&2
exit 2
fi
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
SERVERS_DIR="$REPO_ROOT/servers"
MIRROR_DIR="$REPO_ROOT/stacks-mirror"
EXCLUDES=(
# Include .env.example / *.env.example templates before the broader
# .env* exclude — rsync processes these in order, first match wins.
--include='.env.example'
--include='*.env.example'
--exclude=.env
--exclude='.env.*'
--exclude=acme.json
--exclude=client_secrets.json
--exclude='*.pem'
--exclude='*.key'
--exclude='*.crt'
--exclude='*.pfx'
--exclude='*.sqlite'
--exclude='*.sqlite3'
--exclude='*.db'
--exclude='*.log'
--exclude='*.log.*'
--exclude='*.pid'
--exclude='hub/'
--exclude='logs/'
)
HOST=
STACK=
ASSUME_YES=0
DO_COMPOSE=1
DO_CONF=1
for a in "$@"; do
case "$a" in
--yes|-y) ASSUME_YES=1 ;;
--compose) DO_CONF=0 ;;
--conf) DO_COMPOSE=0 ;;
-h|--help) sed -n '2,22p' "$0"; exit 0 ;;
-*) echo "error: unknown flag $a" >&2; exit 2 ;;
*)
if [ -z "$HOST" ]; then HOST="$a"
elif [ -z "$STACK" ]; then STACK="$a"
else echo "error: unexpected positional '$a'" >&2; exit 2
fi
;;
esac
done
[ -n "$HOST" ] || { echo "usage: $(basename "$0") <host> <stack>" >&2; exit 2; }
[ -n "$STACK" ] || { echo "usage: $(basename "$0") <host> <stack>" >&2; exit 2; }
resolve_target() {
local host="$1"
local effective
effective=$(ssh -G "$host" 2>/dev/null | awk '/^hostname /{print $2; exit}')
if [ -n "$effective" ] && getent hosts "$effective" >/dev/null 2>&1; then
echo "$host"; return
fi
local fb="$SERVERS_DIR/$host/ssh-target"
if [ -f "$fb" ]; then awk 'NF{print $1; exit}' "$fb"; return; fi
echo "$host"
}
TARGET=$(resolve_target "$HOST")
STACK_DIR="$MIRROR_DIR/$HOST/$STACK"
[ -d "$STACK_DIR" ] || { echo "error: $STACK_DIR not found (pull with sync-stacks.sh first)" >&2; exit 2; }
# Collect the two src/dest pairs we need to consider.
PAIRS=() # each entry: "<kind>|<src>|<dest>"
if [ "$DO_COMPOSE" -eq 1 ]; then
PAIRS+=("compose|$STACK_DIR/|$TARGET:/opt/docker/compose/$STACK/")
fi
if [ "$DO_CONF" -eq 1 ] && [ -d "$STACK_DIR/conf" ]; then
PAIRS+=("conf|$STACK_DIR/conf/|$TARGET:/opt/docker/conf/$STACK/")
fi
[ "${#PAIRS[@]}" -gt 0 ] || { echo "nothing to deploy"; exit 0; }
# --------- Dry-run summary: which files would change, per kind. --------
declare -A CHANGED_FILES_BY_KIND=() # kind → newline-separated list
declare -A DELETED_FILES_BY_KIND=()
declare -A RAW_RSYNC_OUT_BY_KIND=() # kind → raw rsync itemize output
any_change=0
for entry in "${PAIRS[@]}"; do
IFS='|' read -r kind src dest <<<"$entry"
extra=()
[ "$kind" = compose ] && extra+=(--exclude='conf/')
# Pre-create the remote dir. Without this, rsync against a nonexistent
# destination can fail in ways the dry-run doesn't surface cleanly.
remote_path="/opt/docker/$kind/$STACK/"
if ! ssh -o BatchMode=yes -o ConnectTimeout=10 "$TARGET" \
"mkdir -p '$remote_path'" 2>/dev/null; then
echo "error: could not create $remote_path on $TARGET (check perms / ssh)" >&2
exit 2
fi
tmp_out=$(mktemp) tmp_err=$(mktemp)
rc=0
rsync -az --delete --dry-run \
--out-format='%i %n' \
"${EXCLUDES[@]}" "${extra[@]}" \
"$src" "$dest" >"$tmp_out" 2>"$tmp_err" || rc=$?
if [ "$rc" -ne 0 ]; then
echo "error: rsync dry-run failed (exit $rc) for $src → $dest" >&2
sed 's/^/ /' "$tmp_err" >&2
rm -f "$tmp_out" "$tmp_err"
exit 2
fi
mapfile -t lines < "$tmp_out"
RAW_RSYNC_OUT_BY_KIND[$kind]=$(cat "$tmp_out")
rm -f "$tmp_out" "$tmp_err"
changed=""
deleted=""
for ln in "${lines[@]}"; do
# Itemized codes (rsync uses '<' for push, '>' for pull):
# <f+++++++++ newfile (new file, push)
# >f+++++++++ newfile (new file, pull)
# <f..t...... file.yaml (content update, push)
# *deleting oldfile (deletion, either direction)
# .d..t...... ./ (metadata on a dir — skip)
# cd+++++++++ somedir/ (new dir — skip, we diff files only)
[ -z "$ln" ] && continue
code=$(awk '{print $1}' <<<"$ln")
name=$(awk '{ $1=""; sub(/^ /,""); print }' <<<"$ln")
[ -z "$name" ] && continue
[ "${name: -1}" = "/" ] && continue # directory entry
case "$code" in
'*deleting') deleted+="$name"$'\n' ;;
'<f'*|'>f'*) changed+="$name"$'\n' ;;
*) : ;; # dir entries, metadata-only, unknown
esac
done
CHANGED_FILES_BY_KIND[$kind]="$changed"
DELETED_FILES_BY_KIND[$kind]="$deleted"
if [ -n "$changed$deleted" ]; then any_change=1; fi
done
if [ "$any_change" -eq 0 ]; then
echo "up to date: $HOST/$STACK is already in sync with server."
# Diagnostic: if the remote dir actually looks empty, we may have been
# fooled by an rsync quirk — dump what rsync saw so the user can tell.
for entry in "${PAIRS[@]}"; do
IFS='|' read -r kind _ _ <<<"$entry"
raw=${RAW_RSYNC_OUT_BY_KIND[$kind]:-}
remote_path="/opt/docker/$kind/$STACK/"
remote_count=$(ssh -o BatchMode=yes "$TARGET" \
"find '$remote_path' -mindepth 1 -maxdepth 1 2>/dev/null | wc -l" \
2>/dev/null || echo "?")
printf ' %s: remote has %s entries, rsync itemize output:\n' "$kind" "$remote_count"
if [ -z "$raw" ]; then
printf ' (empty — rsync reported no work)\n'
else
sed 's/^/ /' <<<"$raw"
fi
done
exit 0
fi
# --------- Print diffs. -----------------------------------------------
divider() { printf '\n%s\n' "------------------------------------------------------------"; }
for entry in "${PAIRS[@]}"; do
IFS='|' read -r kind src dest <<<"$entry"
remote_base="/opt/docker/$kind/$STACK"
changed=${CHANGED_FILES_BY_KIND[$kind]:-}
deleted=${DELETED_FILES_BY_KIND[$kind]:-}
[ -z "$changed$deleted" ] && continue
printf '\n=== %s → %s ===\n' "$src" "$dest"
while IFS= read -r rel; do
[ -z "$rel" ] && continue
local_file="$src$rel"
remote_file="$remote_base/$rel"
divider
if ssh -o BatchMode=yes "$TARGET" "[ -f '$remote_file' ]" 2>/dev/null; then
printf 'MODIFY %s\n' "$rel"
diff -u --label "a/$rel (remote)" --label "b/$rel (local)" \
<(ssh -o BatchMode=yes "$TARGET" "cat '$remote_file'" 2>/dev/null) \
"$local_file" || true
else
printf 'ADD %s\n' "$rel"
diff -u --label /dev/null --label "b/$rel (local)" \
/dev/null "$local_file" || true
fi
done <<<"$changed"
while IFS= read -r rel; do
[ -z "$rel" ] && continue
remote_file="$remote_base/$rel"
divider
printf 'DELETE %s\n' "$rel"
diff -u --label "a/$rel (remote)" --label /dev/null \
<(ssh -o BatchMode=yes "$TARGET" "cat '$remote_file'" 2>/dev/null) \
/dev/null || true
done <<<"$deleted"
done
divider
if [ "$ASSUME_YES" -ne 1 ]; then
read -r -p "Apply these changes to $TARGET? [y/N] " ans
case "$ans" in
y|Y|yes|YES) ;;
*) echo "aborted."; exit 1 ;;
esac
fi
# --------- Apply. -----------------------------------------------------
for entry in "${PAIRS[@]}"; do
IFS='|' read -r kind src dest <<<"$entry"
extra=()
[ "$kind" = compose ] && extra+=(--exclude='conf/')
printf 'pushing %s → %s\n' "$src" "$dest"
rsync -az --delete \
"${EXCLUDES[@]}" "${extra[@]}" \
"$src" "$dest"
done
echo "done."
scripts/refresh-server-info.sh
Executable
+260
View File
@@ -0,0 +1,260 @@
#!/usr/bin/env bash
# refresh-server-info.sh — pull a fresh system-details.txt from servers.
#
# Hosts are discovered by listing `servers/*/` directory names. Each name
# is used as the SSH target, so put matching entries in ~/.ssh/config to
# customize user / port / identity.
#
# Fallback: if the dir name doesn't resolve (and ssh_config doesn't rewrite
# it), the script looks for `servers/<host>/ssh-target` (one line, containing
# an IP or `user@ip`) and uses that instead. This keeps the tool working
# from a fresh clone without requiring DNS or ssh_config setup.
#
# For each host:
# 1. Run scripts/server_inspect.sh on the remote via `ssh <target> 'bash -s'`.
# 2. Write output atomically to `servers/<host>/system-details.txt`.
# A failed SSH/run never clobbers the previous good snapshot.
#
# Exit status is non-zero if any host failed.
#
# Usage:
# scripts/refresh-server-info.sh show this help
# scripts/refresh-server-info.sh all refresh every host
# scripts/refresh-server-info.sh ana-docker refresh one host
# scripts/refresh-server-info.sh ana-docker ana-ml2 refresh several
# scripts/refresh-server-info.sh --dry-run all preview, no ssh
# scripts/refresh-server-info.sh --validate-only all checks only
# scripts/refresh-server-info.sh --validate-only <h> checks one host
#
# Discovery is validated before any ssh is attempted. Each host prints
# an indented "! <reason>" line per warning (unreadable files, empty or
# missing ssh-target, unresolvable name with no fallback, missing README
# or system-details, etc). Warnings never block the refresh — they're
# informational — but are surfaced so drift is visible.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
INSPECT="$SCRIPT_DIR/server_inspect.sh"
SERVERS_DIR="$REPO_ROOT/servers"
if [ ! -f "$INSPECT" ]; then
echo "error: $INSPECT not found" >&2
exit 2
fi
DRY_RUN=0
VALIDATE_ONLY=0
REQUESTED=()
for arg in "$@"; do
case "$arg" in
--dry-run) DRY_RUN=1 ;;
--validate-only|--validate) VALIDATE_ONLY=1 ;;
-h|--help)
sed -n '2,32p' "$0"
exit 0
;;
-*) echo "error: unknown flag $arg" >&2; exit 2 ;;
*) REQUESTED+=("$arg") ;;
esac
done
discover_hosts() {
find "$SERVERS_DIR" -mindepth 1 -maxdepth 1 -type d -printf '%f\n' | sort
}
# No positional args → show help. Fleet-wide operations must be explicit.
if [ "${#REQUESTED[@]}" -eq 0 ]; then
sed -n '2,32p' "$0"
exit 0
fi
# The literal keyword "all" expands to every discovered host. Using it as
# a sentinel (rather than making "no args" mean "all") makes fleet-wide
# runs deliberate — useful when you might otherwise hit 20 hosts by accident.
if [ "${#REQUESTED[@]}" -eq 1 ] && [ "${REQUESTED[0]}" = "all" ]; then
mapfile -t HOSTS < <(discover_hosts)
else
# Check for 'all' mixed with other names — almost certainly a mistake.
for r in "${REQUESTED[@]}"; do
if [ "$r" = "all" ]; then
echo "error: 'all' must be the only argument when used" >&2
exit 2
fi
done
HOSTS=("${REQUESTED[@]}")
fi
if [ "${#HOSTS[@]}" -eq 0 ]; then
echo "error: no hosts found under $SERVERS_DIR" >&2
exit 2
fi
resolve_target() {
# Prints the SSH target to use for a given dir name.
# Preference order:
# 1. The dir name — if ssh's effective hostname (after ssh_config) resolves.
# 2. Contents of servers/<host>/ssh-target (first whitespace token).
# 3. The dir name as-is — let ssh fail with its own error.
local host="$1"
local effective
effective=$(ssh -G "$host" 2>/dev/null | awk '/^hostname /{print $2; exit}')
if [ -n "$effective" ] && getent hosts "$effective" >/dev/null 2>&1; then
echo "$host"
return
fi
local fallback="$SERVERS_DIR/$host/ssh-target"
if [ -f "$fallback" ]; then
awk 'NF{print $1; exit}' "$fallback"
return
fi
echo "$host"
}
validate_host() {
# Prints one warning per line. Empty output = clean.
local host="$1"
local dir="$SERVERS_DIR/$host"
if [ ! -d "$dir" ]; then
printf '%s\n' "dir missing: $dir"
return
fi
if [ ! -r "$dir" ] || [ ! -x "$dir" ]; then
printf '%s\n' "dir not readable/searchable (check permissions)"
return
fi
local stf="$dir/ssh-target"
if [ -e "$stf" ]; then
if [ ! -f "$stf" ]; then
printf '%s\n' "ssh-target is not a regular file"
elif [ ! -r "$stf" ]; then
printf '%s\n' "ssh-target not readable"
elif [ ! -s "$stf" ]; then
printf '%s\n' "ssh-target is empty"
else
local t
t=$(awk 'NF{print $1; exit}' "$stf" 2>/dev/null || true)
if [ -z "$t" ]; then
printf '%s\n' "ssh-target has no non-blank content"
elif [[ "$t" =~ [[:space:]] ]]; then
printf '%s\n' "ssh-target first token contains whitespace ('$t')"
fi
fi
fi
local effective=""
effective=$(ssh -G "$host" 2>/dev/null | awk '/^hostname /{print $2; exit}')
local resolves=0
if [ -n "$effective" ] && getent hosts "$effective" >/dev/null 2>&1; then
resolves=1
fi
if [ "$resolves" -eq 0 ] && [ ! -s "$stf" ]; then
printf '%s\n' "name '$host' does not resolve and no ssh-target fallback present"
fi
[ -f "$dir/README.md" ] || printf '%s\n' "README.md missing"
if [ ! -f "$dir/system-details.txt" ]; then
printf '%s\n' "system-details.txt missing (never refreshed)"
else
validate_snapshot "$dir/system-details.txt"
fi
}
validate_snapshot() {
# Read the captured system-details.txt and yield a warning for each
# known problem marker that indicates the remote inspect ran but didn't
# have the rights / environment it needed.
local file="$1"
if grep -q 'docker daemon not reachable by current user' "$file"; then
printf '%s\n' "remote user cannot reach docker daemon (add to 'docker' group or fix socket perms)"
fi
if grep -qE '^Server:[[:space:]]*$' "$file" \
&& ! grep -q 'docker daemon not reachable by current user' "$file"; then
printf '%s\n' "docker 'Server:' line is blank (daemon down or inaccessible)"
fi
if grep -q '^nvidia-smi present, but failed' "$file"; then
printf '%s\n' "nvidia-smi failed on remote (driver broken or no permission)"
fi
if ! grep -q '^===== DONE =====' "$file"; then
printf '%s\n' "snapshot appears truncated (missing trailing '===== DONE =====' marker)"
fi
}
print_warnings() {
# $1 = indent, rest = warning lines
local indent="$1"; shift
local w
for w in "$@"; do
printf '%s! %s\n' "$indent" "$w"
done
}
pad=0
for h in "${HOSTS[@]}"; do (( ${#h} > pad )) && pad=${#h}; done
if [ "$VALIDATE_ONLY" -eq 1 ]; then
printf 'Validating %d host(s):\n' "${#HOSTS[@]}"
total_warn=0
for host in "${HOSTS[@]}"; do
mapfile -t warnings < <(validate_host "$host")
if [ "${#warnings[@]}" -eq 0 ]; then
printf ' %-*s ok\n' "$pad" "$host"
else
printf ' %-*s %d warning(s)\n' "$pad" "$host" "${#warnings[@]}"
print_warnings " " "${warnings[@]}"
total_warn=$((total_warn + ${#warnings[@]}))
fi
done
if [ "$total_warn" -gt 0 ]; then exit 1; fi
exit 0
fi
printf 'Refreshing %d host(s):\n' "${#HOSTS[@]}"
failed=()
for host in "${HOSTS[@]}"; do
out="$SERVERS_DIR/$host/system-details.txt"
tmp="$out.new"
mapfile -t warnings < <(validate_host "$host")
target=$(resolve_target "$host")
if [ "$target" = "$host" ]; then
label="$host"
else
label="$host → $target"
fi
printf ' %-*s ' "$pad" "$label"
if [ "$DRY_RUN" -eq 1 ]; then
printf 'would run: ssh %s bash -s < %s > %s\n' "$target" "$INSPECT" "$out"
[ "${#warnings[@]}" -gt 0 ] && print_warnings " " "${warnings[@]}"
continue
fi
mkdir -p "$SERVERS_DIR/$host"
if ssh -o BatchMode=yes -o ConnectTimeout=10 "$target" 'bash -s' < "$INSPECT" > "$tmp" 2> "$tmp.err"; then
mv "$tmp" "$out"
rm -f "$tmp.err"
bytes=$(wc -c < "$out")
printf 'ok (%s bytes)\n' "$bytes"
else
rc=$?
rm -f "$tmp"
err=$(head -n 1 "$tmp.err" 2>/dev/null || true)
rm -f "$tmp.err"
printf 'FAIL (rc=%d) %s\n' "$rc" "$err"
failed+=("$host")
fi
[ "${#warnings[@]}" -gt 0 ] && print_warnings " " "${warnings[@]}"
done
if [ "${#failed[@]}" -gt 0 ]; then
printf '\n%d host(s) failed: %s\n' "${#failed[@]}" "${failed[*]}" >&2
exit 1
fi
scripts/server_inspect.sh
Executable
+166
View File
@@ -0,0 +1,166 @@
#!/usr/bin/env bash
# server_inspect.sh — collect server details for writing Docker Compose files.
#
# Conventions this script assumes and reports on:
# - Compose files: /opt/docker/compose/<stack>/{docker-compose.yml,compose.yaml}
# - Config mounts: /opt/docker/conf/<stack>/...
# - Named volumes preferred over bind mounts for persistent state.
#
# Safe: read-only. No modifications are made.
# Usage:
# bash server_inspect.sh # print to stdout
# bash server_inspect.sh /tmp/report.txt # also save to file
set -u
OUT="${1:-}"
if [ -n "$OUT" ]; then exec > >(tee "$OUT") 2>&1; fi
hr() { printf '\n===== %s =====\n\n' "$*"; }
sub() { printf '\n----- %s -----\n' "$*"; }
have() { command -v "$1" >/dev/null 2>&1; }
# --- HOST --------------------------------------------------------------------
hr "HOST"
echo "Hostname: $(hostname -f 2>/dev/null || hostname)"
echo "Date: $(date -Iseconds)"
echo "Uptime: $(uptime -p 2>/dev/null || uptime)"
if [ -f /etc/os-release ]; then
. /etc/os-release
echo "OS: ${PRETTY_NAME:-unknown}"
fi
echo "Kernel: $(uname -r)"
echo "Arch: $(uname -m)"
# --- HARDWARE ----------------------------------------------------------------
hr "HARDWARE"
if [ -f /proc/cpuinfo ]; then
echo "CPU cores: $(grep -c ^processor /proc/cpuinfo)"
echo "CPU model: $(awk -F: '/model name/ {print $2; exit}' /proc/cpuinfo | sed 's/^[[:space:]]*//')"
fi
if [ -f /proc/meminfo ]; then
awk '/^MemTotal:|^MemAvailable:/ {printf "%-11s %.1f GB\n", $1, $2/1024/1024}' /proc/meminfo
fi
# --- GPUS --------------------------------------------------------------------
hr "GPUS"
if have nvidia-smi; then
nvidia-smi --query-gpu=index,name,memory.total,memory.free,driver_version --format=csv
else
echo "nvidia-smi not present (no NVIDIA GPUs or driver not installed)"
fi
# --- FILESYSTEM --------------------------------------------------------------
hr "FILESYSTEMS (df)"
df -h -x tmpfs -x devtmpfs -x overlay 2>/dev/null
hr "PERSISTENT MOUNTS (/etc/fstab, non-comment)"
if [ -r /etc/fstab ]; then
grep -vE '^\s*(#|$)' /etc/fstab
fi
hr "TARGETED DATA PATHS"
for path in /tank /models /opt /opt/docker /opt/docker/compose /opt/docker/conf \
/var/lib/docker /data /srv; do
if [ -d "$path" ]; then
size=$(du -sh "$path" 2>/dev/null | awk '{print $1}')
echo "$path (total: ${size:-?})"
ls -la --time-style=long-iso "$path" 2>/dev/null | sed 's/^/ /' | head -30
echo
fi
done
# --- DOCKER ------------------------------------------------------------------
hr "DOCKER"
if ! have docker; then
echo "docker not installed"
else
docker version --format 'Server: {{.Server.Version}} Client: {{.Client.Version}}' 2>/dev/null \
|| echo "docker daemon not reachable by current user"
sub "docker info"
docker info --format 'Containers: {{.Containers}} (running {{.ContainersRunning}}, paused {{.ContainersPaused}}, stopped {{.ContainersStopped}})
Images: {{.Images}}
Runtimes: {{.Runtimes}}
Default runtime: {{.DefaultRuntime}}
Storage driver: {{.Driver}}
Root dir: {{.DockerRootDir}}
Server version: {{.ServerVersion}}' 2>/dev/null
sub "running containers"
docker ps --format 'table {{.Names}}\t{{.Image}}\t{{.Status}}\t{{.Ports}}' 2>/dev/null
sub "all containers"
docker ps -a --format 'table {{.Names}}\t{{.Image}}\t{{.Status}}' 2>/dev/null
sub "networks"
docker network ls --format 'table {{.Name}}\t{{.Driver}}\t{{.Scope}}' 2>/dev/null
sub "networks (external, non-default — worth knowing for compose external: true)"
docker network ls --filter driver=bridge --format '{{.Name}}' 2>/dev/null \
| grep -vE '^(bridge|host|none)$' || true
sub "named volumes"
docker volume ls --format 'table {{.Name}}\t{{.Driver}}' 2>/dev/null
sub "compose projects currently running"
docker ps --format '{{.Label "com.docker.compose.project"}}' 2>/dev/null \
| sort -u | grep -v '^$' || echo "(none)"
fi
# --- COMPOSE FILES -----------------------------------------------------------
hr "COMPOSE FILES (/opt/docker/compose/)"
if [ -d /opt/docker/compose ]; then
find /opt/docker/compose -maxdepth 3 -type f \
\( -name 'docker-compose.y*ml' -o -name 'compose.y*ml' \) 2>/dev/null \
| sort | while read -r f; do
printf '\n>>> %s\n' "$f"
cat "$f"
done
else
echo "/opt/docker/compose not present"
fi
# --- CONFIG LAYOUT -----------------------------------------------------------
hr "CONFIG LAYOUT (/opt/docker/conf/ — top 200 entries)"
if [ -d /opt/docker/conf ]; then
find /opt/docker/conf -maxdepth 4 2>/dev/null | sort | head -200
else
echo "/opt/docker/conf not present"
fi
# --- PORTS -------------------------------------------------------------------
hr "LISTENING PORTS"
if have ss; then
ss -tlnH 2>/dev/null | awk '{print $4}' | sort -u
elif have netstat; then
netstat -tln 2>/dev/null | awk 'NR>2 {print $4}' | sort -u
else
echo "ss and netstat both unavailable"
fi
# --- HF / MODEL CACHES -------------------------------------------------------
hr "MODEL / HUGGINGFACE CACHES"
for path in /tank/aimodels/huggingface /tank/aimodels/llm ~/.cache/huggingface \
/data/huggingface /opt/huggingface; do
if [ -d "$path" ]; then
size=$(du -sh "$path" 2>/dev/null | awk '{print $1}')
echo "$path (${size:-?})"
if [ -d "$path/hub" ]; then
echo " hub entries:"
ls "$path/hub" 2>/dev/null | sed 's/^/ /' | head -30
fi
echo
fi
done
# --- SYSTEMD SERVICES (docker-adjacent) --------------------------------------
hr "DOCKER-ADJACENT SYSTEMD SERVICES"
if have systemctl; then
systemctl list-units --type=service --state=running --no-pager --no-legend 2>/dev/null \
| awk '{print $1, $4}' \
| grep -Ei 'docker|container|traefik|nvidia' || echo "(none matching)"
fi
hr "DONE"
echo "Paste the above back into the chat, or pass a path as argv[1] to save."
scripts/sync-stacks.sh
Executable
+190
View File
@@ -0,0 +1,190 @@
#!/usr/bin/env bash
# sync-stacks.sh — pull /opt/docker/{compose,conf}/<stack>/ from every
# server into version-controlled `stacks-mirror/<host>/<stack>/`.
#
# Layout (flat per stack):
# stacks-mirror/<host>/<stack>/ <- mirrors /opt/docker/compose/<stack>/
# stacks-mirror/<host>/<stack>/conf/ <- mirrors /opt/docker/conf/<stack>/
#
# Opt-out (per-stack, per-kind):
# stacks-mirror/<host>/<stack>/.no-sync → skip stack entirely
# stacks-mirror/<host>/<stack>/conf/.no-sync → skip conf only
# The marker file is preserved; only the rsync is suppressed. Create the
# marker manually for any stack you don't want mirrored.
#
# Secrets and runtime state are always excluded regardless of opt-out:
# .env, .env.*, acme.json, client_secrets.json,
# *.pem, *.key, *.crt, *.pfx,
# *.sqlite, *.sqlite3, *.db, *.log, *.log.*, *.pid,
# hub/, logs/
#
# Usage:
# scripts/sync-stacks.sh # pull from every discovered host
# scripts/sync-stacks.sh ana-docker nh3-docker
# scripts/sync-stacks.sh --dry-run # show what would change, no writes
set -euo pipefail
if ! command -v rsync >/dev/null 2>&1; then
echo "error: rsync is not installed on this workstation" >&2
echo " install it (e.g. 'sudo apt install rsync') and ensure every remote host has it too" >&2
exit 2
fi
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
SERVERS_DIR="$REPO_ROOT/servers"
MIRROR_DIR="$REPO_ROOT/stacks-mirror"
EXCLUDES=(
# Include .env.example / *.env.example templates before the broader
# .env* exclude — rsync processes these in order, first match wins.
--include='.env.example'
--include='*.env.example'
--exclude=.env
--exclude='.env.*'
--exclude=acme.json
--exclude=client_secrets.json
--exclude='*.pem'
--exclude='*.key'
--exclude='*.crt'
--exclude='*.pfx'
--exclude='*.sqlite'
--exclude='*.sqlite3'
--exclude='*.db'
--exclude='*.log'
--exclude='*.log.*'
--exclude='*.pid'
--exclude='hub/'
--exclude='logs/'
)
DRY_RUN=0
REQUESTED=()
for arg in "$@"; do
case "$arg" in
--dry-run) DRY_RUN=1 ;;
-h|--help) sed -n '2,24p' "$0"; exit 0 ;;
-*) echo "error: unknown flag $arg" >&2; exit 2 ;;
*) REQUESTED+=("$arg") ;;
esac
done
resolve_target() {
local host="$1"
local effective
effective=$(ssh -G "$host" 2>/dev/null | awk '/^hostname /{print $2; exit}')
if [ -n "$effective" ] && getent hosts "$effective" >/dev/null 2>&1; then
echo "$host"; return
fi
local fb="$SERVERS_DIR/$host/ssh-target"
if [ -f "$fb" ]; then awk 'NF{print $1; exit}' "$fb"; return; fi
echo "$host"
}
list_remote_subdirs() {
# $1 = ssh target, $2 = remote parent path
ssh -o BatchMode=yes -o ConnectTimeout=10 "$1" \
"find '$2' -maxdepth 1 -mindepth 1 -type d -printf '%f\n' 2>/dev/null | sort" \
2>/dev/null || true
}
sync_one() {
# $1 = host, $2 = ssh target, $3 = stack,
# $4 = 'compose'|'conf' (kind),
# $5 = local dest dir
local host="$1" target="$2" stack="$3" kind="$4" dest="$5"
local remote_src="/opt/docker/$kind/$stack/"
local skip="$dest/.no-sync"
local label
if [ "$kind" = conf ]; then label='conf '; else label='compose'; fi
mkdir -p "$dest"
if [ -f "$skip" ]; then
printf ' %s skip (.no-sync)\n' "$label"
return 0
fi
local extra=()
# Don't recurse into conf/ from the compose side — it's its own mirror target.
[ "$kind" = compose ] && extra+=(--exclude='conf/')
[ "$DRY_RUN" -eq 1 ] && extra+=(--dry-run)
local err rc=0
err=$(
rsync -az --delete --info=stats0,flist0 \
"${EXCLUDES[@]}" "${extra[@]}" \
"$target:$remote_src" "$dest/" 2>&1
) || rc=$?
if [ $rc -ne 0 ]; then
printf ' %s FAIL (rc=%d) %s\n' "$label" "$rc" "$(echo "$err" | head -n 1)"
return 1
fi
if [ "$DRY_RUN" -eq 1 ]; then
printf ' %s dry-run ok\n' "$label"
else
printf ' %s ok\n' "$label"
fi
return 0
}
if [ "${#REQUESTED[@]}" -eq 0 ]; then
mapfile -t HOSTS < <(find "$SERVERS_DIR" -mindepth 1 -maxdepth 1 -type d -printf '%f\n' | sort)
else
HOSTS=("${REQUESTED[@]}")
fi
[ "${#HOSTS[@]}" -eq 0 ] && { echo "error: no hosts found" >&2; exit 2; }
[ "$DRY_RUN" -eq 1 ] && echo "(dry-run)"
mkdir -p "$MIRROR_DIR"
total_fail=0
for host in "${HOSTS[@]}"; do
target=$(resolve_target "$host")
printf '%s (%s):\n' "$host" "$target"
mapfile -t compose_stacks < <(list_remote_subdirs "$target" /opt/docker/compose)
mapfile -t conf_stacks < <(list_remote_subdirs "$target" /opt/docker/conf)
if [ "${#compose_stacks[@]}" -eq 0 ] && [ "${#conf_stacks[@]}" -eq 0 ]; then
printf ' (no stacks discovered — check ssh + remote /opt/docker layout)\n'
continue
fi
# Union of stack names.
mapfile -t all_stacks < <(printf '%s\n' "${compose_stacks[@]}" "${conf_stacks[@]}" | sort -u | grep .)
# Warn about local stacks that no longer exist on the remote.
if [ -d "$MIRROR_DIR/$host" ]; then
for local_stack in "$MIRROR_DIR/$host"/*/; do
[ -d "$local_stack" ] || continue
name=$(basename "$local_stack")
if ! printf '%s\n' "${all_stacks[@]}" | grep -qxF "$name"; then
printf ' ! %s exists locally but not on remote (stale — remove manually if intentional)\n' "$name"
fi
done
fi
for stack in "${all_stacks[@]}"; do
printf ' %s\n' "$stack"
stack_root="$MIRROR_DIR/$host/$stack"
if [ -f "$stack_root/.no-sync" ]; then
printf ' skip (.no-sync at stack root)\n'
continue
fi
if printf '%s\n' "${compose_stacks[@]}" | grep -qxF "$stack"; then
sync_one "$host" "$target" "$stack" compose "$stack_root" || total_fail=$((total_fail+1))
fi
if printf '%s\n' "${conf_stacks[@]}" | grep -qxF "$stack"; then
sync_one "$host" "$target" "$stack" conf "$stack_root/conf" || total_fail=$((total_fail+1))
fi
done
done
if [ "$total_fail" -gt 0 ]; then
printf '\n%d sync operation(s) failed\n' "$total_fail" >&2
exit 1
fi
servers/ana-docker/README.md
+67
View File
@@ -0,0 +1,67 @@
# ana-docker
General-purpose Docker host for the Anaheim colo. Runs everything at `10.250.0.0/16` that doesn't need a GPU — pair it with ana-ml2, which handles the GPU workloads.
## Network
- **LAN IP:** 10.250.50.70
- **FQDN:** `ana-docker.phasefinal.com`
- **SSH:** standard port 22
- **Traefik entrypoints:** 80/443 terminate here; cert resolver `anaprod` (Let's Encrypt)
## Hardware
- **CPU:** 8 vCPU (QEMU virtual — this is a VM)
- **RAM:** 15.6 GB
- **GPUs:** none
- **Storage:** 245 GB root (ext4) + NFS mounts from `10.250.50.50` (TrueNAS)
- **OS:** Debian 12 (bookworm), kernel 6.1.x
- **Docker:** 20.10.24
## Key paths
| Path | Purpose |
|------|---------|
| `/opt/docker/compose/<stack>/` | Compose files (also a git repo) |
| `/opt/docker/conf/<stack>/` | Config bind mounts |
| `/opt/docker/data/` | Service state (legacy — most stacks now use named volumes) |
| `/mnt/compose` | NFS — `10.250.50.50:/mnt/docker` |
| `/mnt/backup` | NFS — `10.250.50.50:/mnt/backup` (restic target) |
| `/mnt/tnvms` | NFS — `10.250.50.50:/mnt/pve-VMStorage` |
## Running stacks
| Stack | Port(s) | Notes |
|-------|---------|-------|
| traefik | 80 / 443 / 8380 | Reverse proxy + TLS (cert resolver `anaprod`) |
| crowdsec (+ blocklist-mirror) | 41412 | IPS; bouncer runs as a Traefik plugin |
| gitea | 3000 / 222 | Git hosting — `gitea.phasefinal.com` |
| vaultwarden | 9080 | Password vault — `vaultwarden.phasefinal.com` |
| synapse (+ synapse-db, element-web) | internal | Matrix homeserver — `matrix.phasefinal.com`, client at `chat.phasefinal.com` |
| seafile (+ mariadb, memcached) | 9180 | File sync — `seafile.phasefinal.com` |
| searxng | 9996 | Private search metaengine |
| openwebui | 3100 | Chat UI frontend |
| sillytavern | 8100 | Chat UI |
| mailrise | 8025 | SMTP-to-notification gateway |
| rustdesk (hbbs + hbbr) | host-net 21115-21119 | Remote desktop relay — `rustdesk.phasefinal.com` |
| dockge | 5001 | Docker stack management UI |
| beszel | 8090 | Fleet metrics hub (+ local agent); agents on the other hosts report here |
| dozzle (hub as `dozzle-hub`) | 8088 | Fleet log viewer; agents on the other hosts report here |
| restic rest-server | 8000 | Anaheim-side restic endpoint (writes to TrueNAS NFS at `/mnt/backup/restic/repo/ana/`); paired with `rest-server-nh3` on the Synology for the NH3 side |
| backrest | 9898 | Fleet-wide restic snapshot viewer / restore UI — points at both rest-servers |
| it-tools | 8780 | Dev utilities |
| mattermost | — | Stopped; kept around for reference |
Portainer was retired from this host; stack management is now handled via Dockge + Beszel.
## Refresh state
```bash
scripts/refresh-server-info.sh ana-docker
```
Latest snapshot: `system-details.txt` (regenerate as needed).
## Placement rule
If a new stack needs a GPU it goes on **ana-ml2**, otherwise it lands here.
servers/ana-docker/ssh-target
+1
View File
@@ -0,0 +1 @@
10.250.50.70
servers/ana-docker/system-details.txt
+1317
View File
File diff suppressed because it is too large Load Diff
servers/ana-ml2/README.md
+61
View File
@@ -0,0 +1,61 @@
# ana-ml2
Primary AI inference host for PFI.
## Network
- **LAN IP:** 10.250.50.54
- **SSH:** standard port 22
## Hardware
- **CPU:** AMD EPYC 9254 24-core (96 threads)
- **RAM:** 566 GB
- **GPUs:** 2x NVIDIA RTX 6000 Ada Generation (46 GB VRAM each, GPU 0 and GPU 1)
- **Storage:** ZFS `zroot` (434 GB root) + `tank` pool (8.6 TB at `/tank`)
- **OS:** Debian 13 (trixie), kernel 6.12.x
- **Docker:** 29.3.1, runtimes: runc (default), nvidia, io.containerd.runc.v2
## Key paths
| Path | Purpose |
|------|---------|
| `/opt/docker/compose/<stack>/` | Compose files |
| `/opt/docker/conf/<stack>/` | Config bind mounts |
| `/tank/aimodels/huggingface/` | HF cache (267 GB, pre-downloaded models) |
| `/tank/aimodels/llm/` | Legacy GGUF models (790 GB, referenced by llama-swap as `/models/`) |
| `/var/lib/docker/` | Docker data (on zroot) |
## Running stacks
| Stack | Port | Notes |
|-------|------|-------|
| llama-swap | 9292 | GGUF model server via llama.cpp |
| vllm-embed (Qwen3) | 8001 | OpenAI-compatible embeddings; part of the `vllm-qwen3` stack (GPU 1) |
| vllm-rerank (Qwen3) | 8002 | OpenAI-compatible reranker; part of the `vllm-qwen3` stack (GPU 1) |
| dockge | 5001 | Docker stack management UI |
| dozzle-agent | 7007 | Log agent; reports to the Dozzle hub on ana-docker |
| beszel-agent | 45876 | Metrics agent; reports to the Beszel hub on ana-docker |
**Retired since last README update:**
- `infinity` — replaced by `vllm-qwen3` after the upstream image stopped shipping a `transformers` build that knew Qwen3.
- `LibreChat (+ rag_api, vectordb, mongodb, meilisearch)` — removed from this host.
- `searxng` — now hosted on ana-docker for the whole fleet.
- Residual networks (`librechat_default`, `kokoro-tts-gpu_default`) from prior experiments are still present; safe to `docker network rm` at leisure.
## Refresh state
```bash
scripts/refresh-server-info.sh ana-ml2
```
Latest snapshot: `system-details.txt` (regenerate as needed).
## GPU allocation policy
By default, no container is pinned. For predictable performance when multiple GPU workloads run concurrently:
- **GPU 0:** heavy LLM (llama-swap big models).
- **GPU 1:** light services (both vllm-qwen3 services share this GPU via `--gpu-memory-utilization`).
Use `deploy.resources.reservations.devices[].device_ids: ["<id>"]` in compose to pin.
servers/ana-ml2/ssh-target
+1
View File
@@ -0,0 +1 @@
10.250.50.54
servers/ana-ml2/system-details.txt
+739
View File
File diff suppressed because one or more lines are too long
servers/esh-docker-vm/README.md
+76
View File
@@ -0,0 +1,76 @@
# esh-docker-vm
General-purpose Docker host at the **ESH home-lab site** (`esteban.net` / `10.0.0.0/8` space). VM, no GPU. Separate scope from the PFI colo work but tracked here because it's part of the same fleet.
## Network
- **LAN IP:** 10.0.50.45
- **FQDN:** `esh-vm-docker.esteban.net`
- **Subnet:** `10.0.50.0/24` (inferred from macvlan config)
- **SSH:** standard port 22
- **Traefik entrypoints:** 80/443 terminate here; DNS-01 challenge via AWS Route53; protected by CrowdSec Traefik plugin
## Hardware
- **CPU:** 16 vCPU (QEMU virtual — VM)
- **RAM:** 15.6 GB
- **GPUs:** none
- **Storage:** 250 GB root (ext4) + NFS mounts from `10.0.50.50`
- **OS:** Debian 12 (bookworm), kernel 6.1.x
- **Docker:** running (daemon socket at `/var/run/docker.sock`; also listens on `:2375`)
## Key paths
| Path | Purpose |
|------|---------|
| `/opt/docker/compose/<stack>/` | Compose files (also a git repo) |
| `/opt/docker/conf/<stack>/` | Config bind mounts |
| `/opt/docker/docker-bu/` | Legacy backup staging (ad-hoc) |
| `/mnt/compose` | NFS — `10.0.50.50:/mnt/compose` |
| `/mnt/backup` | NFS — `10.0.50.50:/mnt/backup` (volume-backup sidecars write here) |
| `/mnt/documents` | NFS — `10.0.50.50:/mnt/documents` (Paperless-ngx source) |
| `/mnt/books` | NFS — `10.0.50.50:/mnt/books` (Calibre library) |
## Running stacks
| Stack | Port(s) | Notes |
|-------|---------|-------|
| traefik | 80 / 443 / 8380 | Reverse proxy + TLS (AWS Route53 DNS-01); CrowdSec bouncer plugin |
| cloudflare-ddns-esh | — | Keeps the `esteban.net` record in sync with the dynamic WAN IP |
| adguard | 53 / 853 / 8080 / 8443 / 3000 / 784 | DNS + DoT for the ESH site |
| dockge | 5001 | Docker stack management UI |
| homepage | 5100 | Dashboard (`eshhome` / `eshhome.esteban.net`) |
| uptime-kuma | 3001 | Service uptime monitor |
| homeassistant | macvlan `10.0.50.46:8123` | Home automation (direct LAN IP via macvlan on `ens18`) |
| esphome | host net / 6052 | ESPHome firmware dashboard |
| mosquitto | 1883 | MQTT broker |
| calibre | 8082 / 8181 / 8281 | Ebook server |
| calibre-web | 8083 | Ebook web UI |
| paperless-ngx (+ redis broker + volume-backup sidecar) | 8200 | Document archive; Postgres on `10.0.50.60:5432` |
| pgadmin (+ volume-backup sidecar) | 5050 | Postgres admin UI |
| drawio | 8087 / 8447 | Diagram editor |
| dozzle-agent | 7007 | Log agent; feeds ana-docker's Dozzle hub |
| beszel-esh-vm-docker (agent) | 45876 | Metrics agent; feeds ana-docker's Beszel hub |
| portainer (+ portainer_agent) | 9443 / 8000 / 9001 | Container management UI (ad-hoc, no compose dir under `/opt/docker/compose/`) |
## Refresh state
```bash
scripts/refresh-server-info.sh esh-docker-vm
```
Latest snapshot: `system-details.txt` (regenerate as needed).
## Cross-site monitoring
Like `nh3-docker`, this host runs **Dozzle** and **Beszel** agents that report back to the hubs on `ana-docker`, so container logs and metrics show up alongside PFI hosts in the shared dashboards.
## Notes
- **Macvlan for Home Assistant** — the HA container gets its own LAN IP (`10.0.50.46`) via a macvlan network on `ens18`, avoiding NAT so multicast/mDNS for HA discovery works cleanly.
- **External Postgres** — Paperless-ngx connects to a DB running elsewhere (`10.0.50.60:5432`), not a sidecar. Paperless creds in that compose file currently look like defaults; rotate before exposing.
- **Volume backups already in place** — `paperless-ngx` and `pgadmin` include `offen/docker-volume-backup:latest` sidecars that tar named volumes to `/mnt/backup/docker/esh-vm-docker/<stack>/`. When the fleet-wide restic plan lands, decide whether to subsume these or leave the per-stack sidecars alone.
## Placement rule
Home-lab workloads for the ESH site go here. Not part of the PFI colo topology.
servers/esh-docker-vm/ssh-target
+1
View File
@@ -0,0 +1 @@
10.0.50.45
servers/esh-docker-vm/system-details.txt
+961
View File
@@ -0,0 +1,961 @@
===== HOST =====
Hostname: esh-vm-docker.esteban.net
Date: 2026-04-19T22:15:59-07:00
Uptime: up 2 weeks, 6 days, 5 hours, 59 minutes
OS: Debian GNU/Linux 12 (bookworm)
Kernel: 6.1.0-41-amd64
Arch: x86_64
===== HARDWARE =====
CPU cores: 16
CPU model: QEMU Virtual CPU version 2.5+
MemTotal: 15.6 GB
MemAvailable: 11.4 GB
===== GPUS =====
nvidia-smi not present (no NVIDIA GPUs or driver not installed)
===== FILESYSTEMS (df) =====
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 250G 122G 117G 52% /
10.0.50.50:/mnt/backup 92T 26G 92T 1% /mnt/backup
10.0.50.50:/mnt/compose 1.5T 4.1G 1.5T 1% /mnt/compose
10.0.50.50:/mnt/documents 92T 0 92T 0% /mnt/documents
10.0.50.50:/mnt/books 92T 96G 92T 1% /mnt/books
===== PERSISTENT MOUNTS (/etc/fstab, non-comment) =====
UUID=a2e1bc05-3afa-4803-9ab2-dfee1d5c9e6c / ext4 errors=remount-ro 0 1
UUID=4f4fa84b-2b71-4ee5-9be4-2deb4c287d8e none swap sw 0 0
/dev/sr0 /media/cdrom0 udf,iso9660 user,noauto 0 0
10.0.50.50:/mnt/compose /mnt/compose nfs defaults 0 0
10.0.50.50:/mnt/books /mnt/books nfs defaults 0 0
10.0.50.50:/mnt/documents /mnt/documents nfs defaults 0 0
10.0.50.50:/mnt/backup /mnt/backup nfs defaults 0 0
===== TARGETED DATA PATHS =====
/opt (total: 62M)
total 16
drwxr-xr-x 4 root root 4096 2025-03-18 23:59 .
drwxr-xr-x 19 root root 4096 2025-12-29 14:50 ..
drwxr-xr-x 5 lkraven lkraven 4096 2025-03-18 23:58 docker
drwxrwxrwx 6 lkraven lkraven 4096 2024-06-13 22:00 docker-bu
/opt/docker (total: 53M)
total 32
drwxr-xr-x 5 lkraven lkraven 4096 2025-03-18 23:58 .
drwxr-xr-x 4 root root 4096 2025-03-18 23:59 ..
drwxr-xr-x 18 lkraven lkraven 4096 2026-04-19 01:08 compose
drwxr-xr-x 7 lkraven lkraven 4096 2025-11-29 00:26 conf
drwxr-xr-x 8 lkraven lkraven 4096 2025-03-18 23:59 .git
-rw-r--r-- 1 lkraven lkraven 34 2025-03-18 23:58 .gitignore
-rw-r--r-- 1 lkraven lkraven 1059 2025-03-18 23:58 LICENSE
-rw-r--r-- 1 lkraven lkraven 60 2025-03-18 23:58 README.md
/opt/docker/compose (total: 172K)
total 76
drwxr-xr-x 18 lkraven lkraven 4096 2026-04-19 01:08 .
drwxr-xr-x 5 lkraven lkraven 4096 2025-03-18 23:58 ..
drwxr-xr-x 2 lkraven lkraven 4096 2025-03-18 23:58 adguard
drwxr-xr-x 2 root root 4096 2026-04-19 01:07 beszel-esh-vm-docker
drwxr-xr-x 2 lkraven lkraven 4096 2025-03-18 23:58 calibre
drwxr-xr-x 2 lkraven lkraven 4096 2025-03-18 23:58 calibre-web
drwxr-xr-x 2 root root 4096 2025-10-03 22:01 cloudflare-ddns-esh
drwxr-xr-x 2 lkraven lkraven 4096 2025-03-18 23:58 dockge
drwxr-xr-x 2 root root 4096 2026-04-19 00:30 dozzle-agent
drwxr-xr-x 2 lkraven lkraven 4096 2025-03-18 23:58 drawio
drwxr-xr-x 2 lkraven lkraven 4096 2025-03-18 23:58 esphome
drwxr-xr-x 2 lkraven lkraven 4096 2025-08-31 17:39 homeassistant
drwxr-xr-x 2 lkraven lkraven 4096 2025-05-19 22:10 homepage
drwxr-xr-x 2 root root 4096 2025-11-29 00:40 mosquitto
drwxr-xr-x 2 lkraven lkraven 4096 2025-08-28 09:37 paperless-ngx
drwxr-xr-x 2 lkraven lkraven 4096 2025-08-31 17:41 pgadmin
-rw-r--r-- 1 lkraven lkraven 25 2025-03-18 23:58 README.md
drwxr-xr-x 2 lkraven lkraven 4096 2025-05-19 22:10 traefik
drwxr-xr-x 2 lkraven lkraven 4096 2025-03-18 23:58 uptimekuma
/opt/docker/conf (total: 50M)
total 28
drwxr-xr-x 7 lkraven lkraven 4096 2025-11-29 00:26 .
drwxr-xr-x 5 lkraven lkraven 4096 2025-03-18 23:58 ..
drwxr-xr-x 2 nas nas 4096 2026-03-20 21:00 calibre-web
drwxr-xr-x 2 lkraven lkraven 4096 2025-10-03 22:13 cloudflare-ddns
drwxr-xr-x 5 lkraven lkraven 4096 2026-03-30 22:30 homepage
drwxr-xr-x 2 1883 1883 4096 2025-11-29 00:50 mosquitto
drwxr-xr-x 3 lkraven lkraven 4096 2025-03-18 23:58 traefik-esh
/var/lib/docker (total: 4.0K)
/srv (total: 20K)
total 12
drwxr-xr-x 3 root root 4096 2024-06-13 21:52 .
drwxr-xr-x 19 root root 4096 2025-12-29 14:50 ..
drwxrwxrwx 3 root root 4096 2024-06-13 21:54 backups
===== DOCKER =====
Server: 20.10.24+dfsg1 Client: 20.10.24+dfsg1
----- docker info -----
Containers: 24 (running 21, paused 0, stopped 3)
Images: 84
Runtimes: map[io.containerd.runc.v2:{runc [] <nil>} io.containerd.runtime.v1.linux:{runc [] <nil>} runc:{runc [] <nil>}]
Default runtime: runc
Storage driver: overlay2
Root dir: /var/lib/docker
Server version: 20.10.24+dfsg1
----- running containers -----
NAMES IMAGE STATUS PORTS
beszel-agent henrygd/beszel-agent:latest Up 21 hours
dozzle-agent amir20/dozzle:latest Up 22 hours 0.0.0.0:7007->7007/tcp, 8080/tcp
traefik traefik:latest Up 3 days 0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp, 0.0.0.0:8380->8080/tcp, :::8380->8080/tcp
paperless-ngx-webserver-1 ghcr.io/paperless-ngx/paperless-ngx:latest Up 2 weeks (healthy) 0.0.0.0:8200->8000/tcp, :::8200->8000/tcp
paperless-ngx-broker-1 redis:7 Up 2 weeks 6379/tcp
paperless-ngx-backup-1 offen/docker-volume-backup:latest Up 2 weeks
homepage ghcr.io/gethomepage/homepage:latest Up 2 weeks (healthy) 0.0.0.0:5100->3000/tcp, :::5100->3000/tcp
homeassistant homeassistant/home-assistant:latest Up 30 hours
mosquitto eclipse-mosquitto:latest Up 2 weeks 0.0.0.0:1883->1883/tcp, :::1883->1883/tcp
adguardhome adguard/adguardhome Up 2 weeks 67-68/udp, 0.0.0.0:53->53/udp, :::53->53/udp, 443/udp, 0.0.0.0:53->53/tcp, :::53->53/tcp, 853/udp, 0.0.0.0:853->853/tcp, :::853->853/tcp, 3000/udp, 5443/tcp, 0.0.0.0:3000->3000/tcp, 0.0.0.0:784->784/udp, :::3000->3000/tcp, :::784->784/udp, 5443/udp, 6060/tcp, 0.0.0.0:8080->80/tcp, :::8080->80/tcp, 0.0.0.0:8443->443/tcp, :::8443->443/tcp
cloudflare-ddns timothyjmiller/cloudflare-ddns:latest Up 2 weeks
pgadmin4_container dpage/pgadmin4 Up 2 weeks 443/tcp, 0.0.0.0:5050->80/tcp, :::5050->80/tcp
pgadmin-backup-1 1127ad194f2f Up 2 weeks
esphome ghcr.io/esphome/esphome Up 2 weeks (healthy)
drawio jgraph/drawio Up 2 weeks (healthy) 0.0.0.0:8087->8080/tcp, :::8087->8080/tcp, 0.0.0.0:8447->8443/tcp, :::8447->8443/tcp
calibre-web lscr.io/linuxserver/calibre-web:latest Up 2 weeks 0.0.0.0:8083->8083/tcp, :::8083->8083/tcp
calibre lscr.io/linuxserver/calibre:latest Up 2 weeks 3000-3001/tcp, 0.0.0.0:8181->8181/tcp, :::8181->8181/tcp, 0.0.0.0:8082->8080/tcp, :::8082->8080/tcp, 0.0.0.0:8281->8081/tcp, :::8281->8081/tcp
45d2522a8cb6_uptime-kuma louislam/uptime-kuma:latest Up 2 weeks (healthy) 0.0.0.0:3001->3001/tcp, :::3001->3001/tcp
dockge-dockge-1 louislam/dockge:latest Up 2 weeks (healthy) 0.0.0.0:5001->5001/tcp, :::5001->5001/tcp
portainer_agent portainer/agent:2.19.4 Up 2 weeks 0.0.0.0:9001->9001/tcp, :::9001->9001/tcp
portainer portainer/portainer-ce:latest Up 2 weeks 0.0.0.0:8000->8000/tcp, :::8000->8000/tcp, 0.0.0.0:9443->9443/tcp, :::9443->9443/tcp, 9000/tcp
----- all containers -----
NAMES IMAGE STATUS
beszel-agent henrygd/beszel-agent:latest Up 21 hours
dozzle-agent amir20/dozzle:latest Up 22 hours
traefik traefik:latest Up 3 days
paperless-ngx-webserver-1 ghcr.io/paperless-ngx/paperless-ngx:latest Up 2 weeks (healthy)
paperless-ngx-broker-1 redis:7 Up 2 weeks
paperless-ngx-backup-1 offen/docker-volume-backup:latest Up 2 weeks
homepage ghcr.io/gethomepage/homepage:latest Up 2 weeks (healthy)
homeassistant homeassistant/home-assistant:latest Up 30 hours
mosquitto eclipse-mosquitto:latest Up 2 weeks
adguardhome adguard/adguardhome Up 2 weeks
friendly_maxwell 0745ced90756 Exited (13) 4 months ago
focused_pascal 0745ced90756 Exited (13) 4 months ago
pedantic_proskuriakova 0745ced90756 Exited (3) 4 months ago
cloudflare-ddns timothyjmiller/cloudflare-ddns:latest Up 2 weeks
pgadmin4_container dpage/pgadmin4 Up 2 weeks
pgadmin-backup-1 1127ad194f2f Up 2 weeks
esphome ghcr.io/esphome/esphome Up 2 weeks (healthy)
drawio jgraph/drawio Up 2 weeks (healthy)
calibre-web lscr.io/linuxserver/calibre-web:latest Up 2 weeks
calibre lscr.io/linuxserver/calibre:latest Up 2 weeks
45d2522a8cb6_uptime-kuma louislam/uptime-kuma:latest Up 2 weeks (healthy)
dockge-dockge-1 louislam/dockge:latest Up 2 weeks (healthy)
portainer_agent portainer/agent:2.19.4 Up 2 weeks
portainer portainer/portainer-ce:latest Up 2 weeks
----- networks -----
NAME DRIVER SCOPE
adguard_default bridge local
bridge bridge local
calibre-web_default bridge local
cloudflare-ddns-esh_default bridge local
homeassistant_macvlan_net macvlan local
homepage_default bridge local
host host local
none null local
pgadmin_default bridge local
traefik-net bridge local
----- networks (external, non-default — worth knowing for compose external: true) -----
adguard_default
calibre-web_default
cloudflare-ddns-esh_default
homepage_default
pgadmin_default
traefik-net
----- named volumes -----
VOLUME NAME DRIVER
6cdde6cf4b58151d434e8a3da23e77d9c04acdaa574a91ff7bbe60ea113a2adf local
6f7206c067f22a53bf7d0c737f74075f849c11fd535b884c4d3cbb73d275bc21 local
007fc81bd8008b22173f8e4a365b2ca43a37080902ab961a197dde0d8782c786 local
8b1ff384b6cb4859c64a21b51d32780027a3682dc1852a36617b59870ea15e32 local
adguard_adguard-confdir local
adguard_adguard-workdir local
ae127367f71c43b662b16c8232a135c59f2a873dcdf606dfa01967fad1ac915e local
beszel-esh-vm-docker_beszel_agent_data local
beszel-vm-esh-nas_beszel_agent_data local
c7a5978e1b811c4d5b15404e405c0ec32a541f8a921a054b0f725a3010a76684 local
dockge_dockge_data local
dozzle-agent_dozzle_agent_data local
homeassistant_homeassistant_data local
mosquitto_mosquitto_data local
mosquitto_mosquitto_log local
paperless-ngx_data local
paperless-ngx_media local
paperless-ngx_redisdata local
pgadmin_pgadmin-data local
portainer_data local
uptimekuma_uptime-kuma local
----- compose projects currently running -----
adguard
beszel-esh-vm-docker
calibre
calibre-web
cloudflare-ddns-esh
dockge
dozzle-agent
drawio
esphome
homeassistant
homepage
mosquitto
paperless-ngx
pgadmin
traefik
uptimekuma
===== COMPOSE FILES (/opt/docker/compose/) =====
>>> /opt/docker/compose/adguard/docker-compose.yml
#version: "3"
services:
adguardhome:
image: adguard/adguardhome
container_name: adguardhome
ports:
- 53:53/tcp
- 53:53/udp
- 784:784/udp
- 853:853/tcp
- 3000:3000/tcp
- 8080:80/tcp
- 8443:443/tcp
volumes:
- adguard-workdir:/opt/adguardhome/work
- adguard-confdir:/opt/adguardhome/conf
restart: unless-stopped
labels:
# This means the container will be stopped during backup to ensure
# backup integrity. You can omit this label if stopping during backup
# not required. Can be omitted if false.
- docker-volume-backup.stop-during-backup=false
- homepage.group=ESH
- homepage.name=AdGuard Home
- homepage.icon=si-adguard
- homepage.description=DNS
- homepage.href=http://10.0.50.45:8080
- homepage.widget.type=adguard
- homepage.widget.url=http://10.0.50.45:8080
- homepage.widget.username=lkraven
- homepage.widget.password=[REDACTED-upstream-compose-inlines-this]
networks:
- tnet
volumes:
adguard-workdir: null
adguard-confdir: null
networks:
tnet:
name: traefik-net
external: true
>>> /opt/docker/compose/beszel-esh-vm-docker/compose.yaml
# Beszel — lightweight server/container monitoring.
#
# Hub: single web UI with the SQLite store. Agents: per-host metric collectors
# that the hub pulls from over SSH.
#
# Multi-host layout via compose profiles:
# COMPOSE_PROFILES=hub → hub only (ana-docker)
# COMPOSE_PROFILES=hub,agent → hub + local agent on the same host
# COMPOSE_PROFILES=agent → agent only (ana-ml2)
#
# The agent uses network_mode: host so it sees real host CPU/mem/net/disk
# counters rather than container-scoped ones — that's why it can't share
# the tnet network with the hub.
#
# All tunables live in .env — edit that, not this file.
services:
beszel:
image: henrygd/beszel:${BESZEL_VERSION}
container_name: beszel
profiles: [hub]
restart: unless-stopped
ports:
- "${BESZEL_PORT}:8090"
volumes:
- beszel_data:/beszel_data
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:8090/api/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 15s
networks:
- tnet
labels:
- homepage.group=PFI-ANA
- homepage.name=Beszel
- homepage.icon=mdi-chart-line
- homepage.description=Server + container monitoring
- homepage.href=http://10.250.50.70:${BESZEL_PORT}
beszel-agent:
image: henrygd/beszel-agent:${BESZEL_VERSION}
container_name: beszel-agent
profiles: [agent]
restart: unless-stopped
network_mode: host
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- beszel_agent_data:/var/lib/beszel-agent
environment:
- PORT=${BESZEL_AGENT_PORT:-45876}
- KEY=${BESZEL_HUB_KEY}
- HUB_URL=${HUB_URL}
- TOKEN=${BESZEL_TOKEN}
- EXTRA_FILESYSTEMS=${BESZEL_EXTRA_FS:-}
volumes:
beszel_data:
beszel_agent_data:
networks:
tnet:
name: traefik-net
external: true
>>> /opt/docker/compose/calibre/compose.yaml
version: "2.1"
services:
calibre:
image: lscr.io/linuxserver/calibre:latest
container_name: calibre
security_opt:
- seccomp:unconfined #optional
environment:
- PUID=2000
- PGID=2000
- TZ=America/Los_Angeles
- PASSWORD= #optional
- CLI_ARGS= #optional
volumes:
- /mnt/books/calibre:/config
ports:
- 8082:8080
- 8181:8181
- 8281:8081
restart: unless-stopped
labels:
- homepage.group=Media
- homepage.name=Calibre
- homepage.icon=mdi-bookshelf
- homepage.description=EBook Server (esh)
- homepage.href=http://10.0.50.45:8082
networks:
- tnet
networks:
tnet:
name: traefik-net
external: true
>>> /opt/docker/compose/calibre-web/compose.yaml
services:
calibre-web:
image: lscr.io/linuxserver/calibre-web:latest
container_name: calibre-web
environment:
- PUID=2000
- PGID=2000
- TZ=Etc/UTC
- DOCKER_MODS=linuxserver/mods:universal-calibre #optional
- OAUTHLIB_RELAX_TOKEN_SCOPE=1 #optional
volumes:
- /opt/docker/conf/calibre-web:/config
- /mnt/books/calibre/calibre_library:/books
labels:
- homepage.group=Media
- homepage.name=Calibre-Web
- homepage.icon=si-calibreweb
- homepage.description=EBook Server (esh)
- homepage.href=http://10.0.50.45:8083
ports:
- 8083:8083
restart: unless-stopped
networks: {}
>>> /opt/docker/compose/cloudflare-ddns-esh/compose.yaml
services:
cloudflare-ddns:
image: timothyjmiller/cloudflare-ddns:latest
container_name: cloudflare-ddns
volumes:
- /opt/docker/conf/cloudflare-ddns/config.json:/config.json
restart: unless-stopped
networks: {}
>>> /opt/docker/compose/dockge/compose.yaml
services:
dockge:
image: louislam/dockge:latest
restart: unless-stopped
ports:
# Host Port : Container Port
- 5001:5001
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- dockge_data:/app/data
- /opt/docker/compose:/opt/docker/compose
labels:
- homepage.group=ESH
- homepage.name=Dockge
- homepage.icon=si-portainer
- homepage.description=Docker
- homepage.href=http://10.0.50.45:5001
environment:
# Tell Dockge where is your stacks directory
- DOCKGE_STACKS_DIR=/opt/docker/compose
networks:
- tnet
volumes:
dockge_data: null
networks:
tnet:
name: traefik-net
external: true
>>> /opt/docker/compose/dozzle-agent/compose.yaml
# Dozzle — container log viewer.
#
# Multi-host layout via compose profiles:
# COMPOSE_PROFILES=hub → runs the web UI (deploy on ana-docker)
# COMPOSE_PROFILES=agent → runs the remote agent (deploy on ana-ml2)
#
# Same compose.yaml on both servers; per-host `.env` picks the profile.
#
# All tunables live in .env — edit that, not this file.
services:
dozzle:
image: amir20/dozzle:${DOZZLE_VERSION}
container_name: dozzle
profiles:
- hub
restart: unless-stopped
ports:
- ${DOZZLE_PORT}:8080
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- dozzle_data:/data
environment:
- DOZZLE_HOSTNAME=${DOZZLE_HOSTNAME}
- DOZZLE_REMOTE_AGENT=${DOZZLE_REMOTE_AGENT:-}
- DOZZLE_AUTH_PROVIDER=${DOZZLE_AUTH_PROVIDER:-none}
- DOZZLE_USERNAME=${DOZZLE_USERNAME:-}
- DOZZLE_PASSWORD=${DOZZLE_PASSWORD:-}
healthcheck:
test:
- CMD
- /dozzle
- healthcheck
interval: 30s
timeout: 10s
retries: 3
start_period: 15s
networks:
- tnet
labels:
- homepage.group=PFI-ANA
- homepage.name=Dozzle
- homepage.icon=mdi-text-box-search
- homepage.description=Container logs (ana-docker + ana-ml2)
- homepage.href=http://10.250.50.70:${DOZZLE_PORT}
dozzle-agent:
image: amir20/dozzle:${DOZZLE_VERSION}
container_name: dozzle-agent
profiles:
- agent
restart: unless-stopped
command: agent
ports:
- ${DOZZLE_AGENT_BIND:-0.0.0.0}:${DOZZLE_AGENT_PORT}:7007
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- dozzle_agent_data:/data
environment:
- DOZZLE_HOSTNAME=${DOZZLE_HOSTNAME}
networks:
- tnet
volumes:
dozzle_data: null
dozzle_agent_data: null
networks:
tnet:
name: traefik-net
external: true
>>> /opt/docker/compose/drawio/compose.yaml
services:
drawio:
image: jgraph/drawio
container_name: drawio
restart: unless-stopped
ports:
- 8087:8080
- 8447:8443
healthcheck:
test:
- CMD-SHELL
- curl -f http://127.0.0.1:8080 || exit 1
interval: 1m30s
timeout: 10s
retries: 5
start_period: 10s
labels:
- homepage.group=Apps
- homepage.name=draw.io
- homepage.icon=mdi-pencil
- homepage.description=Draw.IO Graphing (esh)
- homepage.href=http://10.0.50.45:8087
networks:
- tnet
networks:
tnet:
name: traefik-net
external: true
>>> /opt/docker/compose/esphome/compose.yaml
version: '3'
services:
esphome:
container_name: esphome
image: ghcr.io/esphome/esphome
volumes:
- /path/to/esphome/config:/config
- /etc/localtime:/etc/localtime:ro
restart: always
privileged: true
network_mode: host
environment:
- USERNAME=test
- PASSWORD=ChangeMe
>>> /opt/docker/compose/homeassistant/compose.yaml
services:
homeassistant:
image: homeassistant/home-assistant:latest
container_name: homeassistant
environment:
- PUID=2000
- PGID=2000
- TZ=America/Los_Angeles
volumes:
- homeassistant_data:/config
restart: unless-stopped
env_file:
- .env
networks:
macvlan_net:
ipv4_address: 10.0.50.46
labels:
- homepage.group=Apps
- homepage.name=Home Assistant
- homepage.icon=si-homeassistant
- homepage.description=Home Automation (esh)
- homepage.href=http://10.0.50.46:8123
networks:
macvlan_net:
driver: macvlan
driver_opts:
parent: ens18
ipam:
config:
- subnet: 10.0.50.0/24
volumes:
homeassistant_data: null
>>> /opt/docker/compose/homepage/compose.yaml
services:
homepage:
image: ghcr.io/gethomepage/homepage:latest
container_name: homepage
environment:
PUID: 1000
PGID: 1000
ports:
- 5100:3000
volumes:
- /opt/docker/conf/homepage:/app/config # Make sure your local config directory exists
- /var/run/docker.sock:/var/run/docker.sock:ro # optional, for docker integrations
restart: unless-stopped
networks:
- tnet
labels:
- traefik.enable=true
- traefik.http.routers.homepage.rule=host(`eshhome`) || host(`10.0.50.45`)
|| host (`eshhome.esteban.net`)
- traefik.http.services.homepage.loadbalancer.server.port=3000
- traefik.http.routers.homepage.priority=1
env_file:
- .env
networks:
tnet:
name: traefik-net
external: true
>>> /opt/docker/compose/mosquitto/compose.yaml
services:
mosquitto:
container_name: mosquitto
image: eclipse-mosquitto:latest
restart: unless-stopped
ports:
- 1883:1883/tcp
volumes:
- /opt/docker/conf/mosquitto:/mosquitto/config
- mosquitto_data:/mosquitto/data
- mosquitto_log:/mosquitto/log
networks:
- tnet
labels:
- homepage.group=Apps
- homepage.name=Mosquitto
- homepage.icon=mdi-bug
- homepage.description=MQQT Broker port 1883
networks:
tnet:
name: traefik-net
external: true
volumes:
mosquitto_data: null
mosquitto_log: null
>>> /opt/docker/compose/paperless-ngx/compose.yaml
#version: "3.4"
services:
broker:
image: docker.io/library/redis:7
restart: unless-stopped
volumes:
- redisdata:/data
networks:
- tnet
webserver:
image: ghcr.io/paperless-ngx/paperless-ngx:latest
restart: unless-stopped
depends_on:
- broker
ports:
- 8200:8000
healthcheck:
test:
- CMD
- curl
- -fs
- -S
- --max-time
- "2"
- http://localhost:8000
interval: 30s
timeout: 10s
retries: 5
volumes:
- data:/usr/src/paperless/data
- media:/usr/src/paperless/media
- /mnt/documents/paperless/export:/usr/src/paperless/export
- /mnt/documents/paperless/consume:/usr/src/paperless/consume
# env_file: docker-compose.env
environment:
PAPERLESS_REDIS: redis://broker:6379
PAPERLESS_DBHOST: 10.0.50.60
PAPERLESS_DBPORT: 5432
PAPERLESS_DBNAME: paperless-ng
PAPERLESS_DBUSER: paperless-ng
PAPERLESS_DBPASS: paperless-ng
# PAPERLESS_MEDIA_ROOT=/data/media
PAPERLESS_CONSUMPTION_DIR: /usr/src/paperless/consume
PAPERLESS_EXPORT_DIR: /usr/src/paperless/export
PAPERLESS_CONSUMER_POLLING: 30
# PAPERLESS_DATA_DIR=/config
labels:
- homepage.group=Media
- homepage.name=PaperlessNGX
- homepage.icon=mdi-file-cabinet
- homepage.description=Document Store (esh)
- homepage.href=http://10.0.50.45:8200
- homepage.sitemonitor=http://10.0.50.45:8200
- homepage.widget.type=paperlessngx
- homepage.widget.url=http://10.0.50.45:8200
- homepage.widget.key=${API_KEY}
networks:
- tnet
env_file:
- .env
backup:
image: offen/docker-volume-backup:latest
restart: always
volumes:
# volumes to backup.
- data:/backup/paperless-data:ro
- media:/backup/paperless-media:ro
- redisdata:/backup/paperless-redisdata:ro
- /mnt/backup/docker/esh-vm-docker/paperless:/archive
# can omit below if not stopping service.
# - /var/run/docker.sock:/var/run/docker.sock:ro
env_file:
- .env
networks:
- tnet
networks:
tnet:
name: traefik-net
external: true
volumes:
data: null
media: null
redisdata: null
>>> /opt/docker/compose/pgadmin/compose.yaml
# version: "3.8"
services:
pgadmin:
image: dpage/pgadmin4
container_name: pgadmin4_container
restart: always
ports:
- 5050:80
environment:
PGADMIN_DEFAULT_EMAIL: ${DEFAULT_EMAIL}
PGADMIN_DEFAULT_PASSWORD: ${DEFAULT_PASSWORD}
volumes:
- pgadmin-data:/var/lib/pgadmin
labels:
- homepage.group=Apps
- homepage.name=PGAdmin
- homepage.icon=mdi-database
- homepage.description=pgsql Manager (esh)
- homepage.href=http://10.0.50.45:5050
networks:
- tnet
env_file:
- .env
backup:
image: offen/docker-volume-backup:latest
restart: always
volumes:
# volumes to backup.
- pgadmin-data:/backup/pgadmin-data:ro
- /mnt/backup/docker/esh-vm-docker/pgadmin:/archive
# can omit below if not stopping service.
# - /var/run/docker.sock:/var/run/docker.sock:ro
env_file:
- .env
networks:
tnet:
name: traefik-net
external: true
volumes:
pgadmin-data: null
>>> /opt/docker/compose/traefik/docker-compose.yml
#version: "3.3"
services:
traefik:
image: traefik:latest
container_name: traefik
command:
- --log.level=DEBUG
- --configFile=/etc/traefik/traefik.yml
- --api.insecure=true
- --providers.docker=true
- --providers.docker.exposedbydefault=false
- --entrypoints.web.address=:80
- --experimental.plugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin
- --experimental.plugins.bouncer.version=v1.6.0
environment:
- AWS_ACCESS_KEY_ID=${AWS_KEY}
- AWS_SECRET_ACCESS_KEY=${AWS_SECRET}
- AWS_REGION=${AWS_REGION}
- AWS_HOSTED_ZONE_ID=${AWS_ZONEID}
ports:
- 80:80
- 8380:8080
- 443:443
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- /opt/docker/conf/traefik-esh:/etc/traefik
labels:
- homepage.group=ESH
- homepage.name=traefik
- homepage.icon=si-traefikproxy
- homepage.sitemonitor=http://10.0.50.45:8380
- homepage.href=http://10.0.50.45:8380
- homepage.widget.type=traefik
- homepage.widget.url=http://10.0.50.45:8380
- traefik.http.middlewares.crowdsec.plugin.bouncer.enabled=true
- traefik.http.middlewares.crowdsec.plugin.bouncer.crowdseclapikey=${CROWDSEC_KEY}
- traefik.http.middlewares.crowdsec.plugin.bouncer.crowdseclapihost=crowdsec:8080
- traefik.http.middlewares.crowdsec.plugin.bouncer.crowdseclapischeme=http
- traefik.http.middlewares.crowdsec.plugin.bouncer.crowdsecMode=live
- traefik.http.middlewares.crowdsec.plugin.bouncer.defaultDecisionSeconds=60
networks:
- tnet
env_file:
- .env
networks:
tnet:
name: traefik-net
external: true
>>> /opt/docker/compose/uptimekuma/compose.yaml
services:
uptime-kuma:
restart: always
ports:
- 3001:3001
volumes:
- uptime-kuma:/app/data
container_name: uptime-kuma
image: louislam/uptime-kuma:latest
labels:
- homepage.group=Apps
- homepage.name=Uptime Kuma
- homepage.icon=mdi-arrow-up-bold-circle
- homepage.description=Service Monitoring (esh)
- homepage.href=http://10.0.50.45:3001
- homepage.widget.type=uptimekuma
- homepage.widget.url=http://10.0.50.45:3001
- homepage.widget.slug=nethealth
networks:
- tnet
volumes:
uptime-kuma: {}
networks:
tnet:
name: traefik-net
external: true
===== CONFIG LAYOUT (/opt/docker/conf/ — top 200 entries) =====
/opt/docker/conf
/opt/docker/conf/calibre-web
/opt/docker/conf/calibre-web/app.db
/opt/docker/conf/calibre-web/.CALIBRE_RELEASE
/opt/docker/conf/calibre-web/calibre-web.log
/opt/docker/conf/calibre-web/calibre-web.log.1
/opt/docker/conf/calibre-web/calibre-web.log.2
/opt/docker/conf/calibre-web/client_secrets.json
/opt/docker/conf/calibre-web/gdrive.db
/opt/docker/conf/calibre-web/.key
/opt/docker/conf/cloudflare-ddns
/opt/docker/conf/cloudflare-ddns/config.json
/opt/docker/conf/homepage
/opt/docker/conf/homepage/bookmarks.yaml
/opt/docker/conf/homepage/custom.css
/opt/docker/conf/homepage/custom.js
/opt/docker/conf/homepage/docker.yaml
/opt/docker/conf/homepage/imgs
/opt/docker/conf/homepage/imgs/bg1.jpg
/opt/docker/conf/homepage/imgs/Thumbs.db
/opt/docker/conf/homepage/kubernetes.yaml
/opt/docker/conf/homepage/logs
/opt/docker/conf/homepage/logs/homepage.log
/opt/docker/conf/homepage/nas-tls
/opt/docker/conf/homepage/nas-tls/ca.pem
/opt/docker/conf/homepage/nas-tls/cert.pem
/opt/docker/conf/homepage/nas-tls/key.pem
/opt/docker/conf/homepage/proxmox.yaml
/opt/docker/conf/homepage/services.yaml
/opt/docker/conf/homepage/settings.yaml
/opt/docker/conf/homepage/widgets.yaml
/opt/docker/conf/mosquitto
/opt/docker/conf/mosquitto/mosquitto.conf
/opt/docker/conf/mosquitto/mosquitto.log
/opt/docker/conf/mosquitto/mosquitto.passwd
/opt/docker/conf/traefik-esh
/opt/docker/conf/traefik-esh/certs
/opt/docker/conf/traefik-esh/certs/acme.json
/opt/docker/conf/traefik-esh/traefik.yml
===== LISTENING PORTS =====
0.0.0.0:111
0.0.0.0:1883
0.0.0.0:22
0.0.0.0:3000
0.0.0.0:3001
0.0.0.0:443
0.0.0.0:5001
0.0.0.0:5050
0.0.0.0:5100
0.0.0.0:53
0.0.0.0:6052
0.0.0.0:7007
0.0.0.0:80
0.0.0.0:8000
0.0.0.0:8080
0.0.0.0:8082
0.0.0.0:8083
0.0.0.0:8087
0.0.0.0:8181
0.0.0.0:8200
0.0.0.0:8281
0.0.0.0:8380
0.0.0.0:8443
0.0.0.0:8447
0.0.0.0:853
0.0.0.0:9001
0.0.0.0:9443
[::]:111
127.0.0.1:35469
[::]:1883
[::]:22
*:2375
[::]:3000
[::]:3001
[::]:443
[::]:5001
[::]:5050
[::]:5100
[::]:53
[::]:80
[::]:8000
[::]:8080
[::]:8082
[::]:8083
[::]:8087
[::]:8181
[::]:8200
[::]:8281
[::]:8380
[::]:8443
[::]:8447
[::]:853
[::]:9001
[::]:9443
===== MODEL / HUGGINGFACE CACHES =====
===== DOCKER-ADJACENT SYSTEMD SERVICES =====
containerd.service running
docker.service running
===== DONE =====
Paste the above back into the chat, or pass a path as argv[1] to save.
servers/nh3-docker/README.md
+52
View File
@@ -0,0 +1,52 @@
# nh3-docker
General-purpose Docker host for the New Hampshire (nh3) site. Small VM, no GPU. Separate LAN from the Anaheim colo.
## Network
- **LAN IP:** 10.100.50.40
- **LAN subnet:** 10.100.0.0/16 (NH site)
- **FQDN:** `nh3-docker.phasefinal.com`
- **SSH:** standard port 22
## Hardware
- **CPU:** 8 vCPU (QEMU virtual — VM)
- **RAM:** 7.8 GB
- **GPUs:** none
- **Storage:** 125 GB root (ext4) + NFS mounts from `10.100.50.50` (Synology)
- **OS:** Debian 12 (bookworm), kernel 6.1.x
- **Docker:** running (daemon socket at `/var/run/docker.sock`; also listens on `:2375` — be aware if firewalling)
## Key paths
| Path | Purpose |
|------|---------|
| `/opt/docker/compose/<stack>/` | Compose files (also a git repo) |
| `/opt/docker/conf/<stack>/` | Config bind mounts (empty as of last inspection) |
| `/mnt/compose` | NFS — `10.100.50.50:/volume1/compose` |
| `/mnt/backup` | NFS — `10.100.50.50:/volume1/Backup` |
## Running stacks
| Stack | Port(s) | Notes |
|-------|---------|-------|
| adguard | 53 / 853 / 8080 / 8443 / 3000 / 784 | AdGuard Home — primary DNS for the NH site |
| dockge | 5001 | Docker stack management UI |
| dozzle-agent | 7007 | Log agent; hub on ana-docker pulls from here |
| beszel-nh3-docker (agent) | 45876 | Metrics agent; hub on ana-docker |
| portainer (+ portainer_agent) | 9443 / 8000 / 9001 | Container management UI (ad-hoc, no compose dir under `/opt/docker/compose/`) |
No Traefik / TLS terminator on this host — services are accessed on raw LAN ports. If that changes, mirror the ana-docker pattern (`/opt/docker/conf/traefik-nh3/…` + `anaprod`-style cert resolver).
## Refresh state
```bash
scripts/refresh-server-info.sh nh3-docker
```
Latest snapshot: `system-details.txt` (regenerate as needed).
## Placement rule
No GPU → this is the default target for NH-site Docker workloads. GPU-only stacks still go to **ana-ml2**; non-GPU Anaheim-specific services stay on **ana-docker**.
servers/nh3-docker/ssh-target
+1
View File
@@ -0,0 +1 @@
10.100.50.40
servers/nh3-docker/system-details.txt
+383
View File
@@ -0,0 +1,383 @@
===== HOST =====
Hostname: nh3-docker.phasefinal.com
Date: 2026-04-19T22:16:00-07:00
Uptime: up 5 weeks, 4 days, 15 hours, 4 minutes
OS: Debian GNU/Linux 12 (bookworm)
Kernel: 6.1.0-21-amd64
Arch: x86_64
===== HARDWARE =====
CPU cores: 8
CPU model: QEMU Virtual CPU version 2.5+
MemTotal: 7.8 GB
MemAvailable: 6.5 GB
===== GPUS =====
nvidia-smi not present (no NVIDIA GPUs or driver not installed)
===== FILESYSTEMS (df) =====
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 125G 6.5G 112G 6% /
===== PERSISTENT MOUNTS (/etc/fstab, non-comment) =====
UUID=dc0e74f7-3973-4501-b04e-1d9a3888f739 / ext4 errors=remount-ro 0 1
UUID=cc35604a-fcb3-41e8-9a30-1b85e28eea99 none swap sw 0 0
/dev/sr0 /media/cdrom0 udf,iso9660 user,noauto 0 0
10.100.50.50:/volume1/compose /mnt/compose nfs defaults 0 0
10.100.50.50:/volume1/Backup /mnt/backup nfs defaults 0 0
===== TARGETED DATA PATHS =====
/opt (total: 1.6M)
total 12
drwxr-xr-x 3 root root 4096 2024-05-30 00:24 .
drwxr-xr-x 18 root root 4096 2024-05-14 14:44 ..
drwxrwxrwx 5 root root 4096 2024-05-30 19:12 docker
/opt/docker (total: 1.6M)
total 32
drwxrwxrwx 5 root root 4096 2024-05-30 19:12 .
drwxr-xr-x 3 root root 4096 2024-05-30 00:24 ..
drwxr-xr-x 6 lkraven lkraven 4096 2026-04-19 01:03 compose
drwxr-xr-x 2 lkraven lkraven 4096 2024-05-30 19:11 conf
drwxr-xr-x 8 lkraven lkraven 4096 2024-05-31 19:05 .git
-rw-r--r-- 1 lkraven lkraven 14 2024-05-30 19:12 .gitignore
-rw-r--r-- 1 lkraven lkraven 1059 2024-05-30 00:27 LICENSE
-rw-r--r-- 1 lkraven lkraven 60 2024-05-30 00:27 README.md
/opt/docker/compose (total: 56K)
total 28
drwxr-xr-x 6 lkraven lkraven 4096 2026-04-19 01:03 .
drwxrwxrwx 5 root root 4096 2024-05-30 19:12 ..
drwxr-xr-x 2 lkraven lkraven 4096 2024-05-30 19:14 adguard
drwxr-xr-x 2 root root 4096 2026-04-19 01:03 beszel-nh3-docker
drwxr-xr-x 2 lkraven lkraven 4096 2024-05-30 19:14 dockge
drwxr-xr-x 2 root root 4096 2026-04-19 00:44 dozzle-agent
-rw-r--r-- 1 lkraven lkraven 25 2024-05-30 00:27 README.md
/opt/docker/conf (total: 4.0K)
total 8
drwxr-xr-x 2 lkraven lkraven 4096 2024-05-30 19:11 .
drwxrwxrwx 5 root root 4096 2024-05-30 19:12 ..
/var/lib/docker (total: 4.0K)
/srv (total: 4.0K)
total 8
drwxr-xr-x 2 root root 4096 2024-05-14 14:43 .
drwxr-xr-x 18 root root 4096 2024-05-14 14:44 ..
===== DOCKER =====
Server: 20.10.24+dfsg1 Client: 20.10.24+dfsg1
----- docker info -----
Containers: 6 (running 6, paused 0, stopped 0)
Images: 13
Runtimes: map[io.containerd.runc.v2:{runc [] <nil>} io.containerd.runtime.v1.linux:{runc [] <nil>} runc:{runc [] <nil>}]
Default runtime: runc
Storage driver: overlay2
Root dir: /var/lib/docker
Server version: 20.10.24+dfsg1
----- running containers -----
NAMES IMAGE STATUS PORTS
beszel-agent henrygd/beszel-agent:latest Up 21 hours
dozzle-agent amir20/dozzle:latest Up 22 hours 0.0.0.0:7007->7007/tcp, 8080/tcp
dockge-dockge-1 louislam/dockge:latest Up 5 weeks (healthy) 0.0.0.0:5001->5001/tcp, :::5001->5001/tcp
adguardhome adguard/adguardhome Up 5 weeks 67-68/udp, 0.0.0.0:53->53/udp, :::53->53/udp, 443/udp, 0.0.0.0:53->53/tcp, :::53->53/tcp, 853/udp, 0.0.0.0:853->853/tcp, :::853->853/tcp, 3000/udp, 5443/tcp, 0.0.0.0:3000->3000/tcp, 0.0.0.0:784->784/udp, :::3000->3000/tcp, :::784->784/udp, 5443/udp, 6060/tcp, 0.0.0.0:8080->80/tcp, :::8080->80/tcp, 0.0.0.0:8443->443/tcp, :::8443->443/tcp
portainer_agent portainer/agent:2.19.4 Up 5 weeks 0.0.0.0:9001->9001/tcp, :::9001->9001/tcp
portainer portainer/portainer-ce:latest Up 5 weeks 0.0.0.0:8000->8000/tcp, :::8000->8000/tcp, 0.0.0.0:9443->9443/tcp, :::9443->9443/tcp, 9000/tcp
----- all containers -----
NAMES IMAGE STATUS
beszel-agent henrygd/beszel-agent:latest Up 21 hours
dozzle-agent amir20/dozzle:latest Up 22 hours
dockge-dockge-1 louislam/dockge:latest Up 5 weeks (healthy)
adguardhome adguard/adguardhome Up 5 weeks
portainer_agent portainer/agent:2.19.4 Up 5 weeks
portainer portainer/portainer-ce:latest Up 5 weeks
----- networks -----
NAME DRIVER SCOPE
bridge bridge local
host host local
none null local
traefik-net bridge local
----- networks (external, non-default — worth knowing for compose external: true) -----
traefik-net
----- named volumes -----
VOLUME NAME DRIVER
adguard_adguard-confdir local
adguard_adguard-workdir local
beszel-nh3-docker_beszel_agent_data local
dockge_dockge_data local
dozzle-agent_dozzle_agent_data local
portainer_data local
----- compose projects currently running -----
adguard
beszel-nh3-docker
dockge
dozzle-agent
===== COMPOSE FILES (/opt/docker/compose/) =====
>>> /opt/docker/compose/adguard/docker-compose.yml
#version: "3"
services:
adguardhome:
image: adguard/adguardhome
container_name: adguardhome
ports:
- 53:53/tcp
- 53:53/udp
- 784:784/udp
- 853:853/tcp
- 3000:3000/tcp
- 8080:80/tcp
- 8443:443/tcp
volumes:
- adguard-workdir:/opt/adguardhome/work
- adguard-confdir:/opt/adguardhome/conf
restart: unless-stopped
labels:
- homepage.group=PFI-NH3
- homepage.name=AdGuard Home
- homepage.icon=si-adguard
- homepage.description=DNS
- homepage.href=http://10.100.50.40:8080
- homepage.widget.type=adguard
- homepage.widget.url=http://10.100.50.40:8080
- homepage.widget.username=lkraven
- homepage.widget.password=${userpass}
networks:
- tnet
env_file:
- .env
volumes:
adguard-workdir: null
adguard-confdir: null
networks:
tnet:
name: traefik-net
external: true
>>> /opt/docker/compose/beszel-nh3-docker/compose.yaml
# Beszel — lightweight server/container monitoring.
#
# Hub: single web UI with the SQLite store. Agents: per-host metric collectors
# that the hub pulls from over SSH.
#
# Multi-host layout via compose profiles:
# COMPOSE_PROFILES=hub → hub only (ana-docker)
# COMPOSE_PROFILES=hub,agent → hub + local agent on the same host
# COMPOSE_PROFILES=agent → agent only (ana-ml2)
#
# The agent uses network_mode: host so it sees real host CPU/mem/net/disk
# counters rather than container-scoped ones — that's why it can't share
# the tnet network with the hub.
#
# All tunables live in .env — edit that, not this file.
services:
beszel:
image: henrygd/beszel:${BESZEL_VERSION}
container_name: beszel
profiles: [hub]
restart: unless-stopped
ports:
- "${BESZEL_PORT}:8090"
volumes:
- beszel_data:/beszel_data
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:8090/api/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 15s
networks:
- tnet
labels:
- homepage.group=PFI-ANA
- homepage.name=Beszel
- homepage.icon=mdi-chart-line
- homepage.description=Server + container monitoring
- homepage.href=http://10.250.50.70:${BESZEL_PORT}
beszel-agent:
image: henrygd/beszel-agent:${BESZEL_VERSION}
container_name: beszel-agent
profiles: [agent]
restart: unless-stopped
network_mode: host
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- beszel_agent_data:/var/lib/beszel-agent
environment:
- PORT=${BESZEL_AGENT_PORT:-45876}
- KEY=${BESZEL_HUB_KEY}
- HUB_URL=${HUB_URL}
- TOKEN=${BESZEL_TOKEN}
- EXTRA_FILESYSTEMS=${BESZEL_EXTRA_FS:-}
volumes:
beszel_data:
beszel_agent_data:
networks:
tnet:
name: traefik-net
external: true
>>> /opt/docker/compose/dockge/compose.yaml
services:
dockge:
image: louislam/dockge:latest
restart: unless-stopped
ports:
# Host Port : Container Port
- 5001:5001
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- dockge_data:/app/data
- /opt/docker/compose:/opt/docker/compose
environment:
# Tell Dockge where is your stacks directory
- DOCKGE_STACKS_DIR=/opt/docker/compose
labels:
- homepage.group=PFI-NH3
- homepage.name=Dockge
- homepage.icon=si-portainer
- homepage.description=Docker
- homepage.href=http://10.100.50.40:5001
networks:
- tnet
volumes:
dockge_data: null
networks:
tnet:
name: traefik-net
external: true
>>> /opt/docker/compose/dozzle-agent/compose.yaml
# Dozzle — container log viewer.
#
# Multi-host layout via compose profiles:
# COMPOSE_PROFILES=hub → runs the web UI (deploy on ana-docker)
# COMPOSE_PROFILES=agent → runs the remote agent (deploy on ana-ml2)
#
# Same compose.yaml on both servers; per-host `.env` picks the profile.
#
# All tunables live in .env — edit that, not this file.
services:
dozzle:
image: amir20/dozzle:${DOZZLE_VERSION}
container_name: dozzle
profiles:
- hub
restart: unless-stopped
ports:
- ${DOZZLE_PORT}:8080
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- dozzle_data:/data
environment:
- DOZZLE_HOSTNAME=${DOZZLE_HOSTNAME}
- DOZZLE_REMOTE_AGENT=${DOZZLE_REMOTE_AGENT:-}
- DOZZLE_AUTH_PROVIDER=${DOZZLE_AUTH_PROVIDER:-none}
- DOZZLE_USERNAME=${DOZZLE_USERNAME:-}
- DOZZLE_PASSWORD=${DOZZLE_PASSWORD:-}
healthcheck:
test:
- CMD
- /dozzle
- healthcheck
interval: 30s
timeout: 10s
retries: 3
start_period: 15s
networks:
- tnet
labels:
- homepage.group=PFI-ANA
- homepage.name=Dozzle
- homepage.icon=mdi-text-box-search
- homepage.description=Container logs (ana-docker + ana-ml2)
- homepage.href=http://10.250.50.70:${DOZZLE_PORT}
dozzle-agent:
image: amir20/dozzle:${DOZZLE_VERSION}
container_name: dozzle-agent
profiles:
- agent
restart: unless-stopped
command: agent
ports:
- ${DOZZLE_AGENT_BIND:-0.0.0.0}:${DOZZLE_AGENT_PORT}:7007
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- dozzle_agent_data:/data
environment:
- DOZZLE_HOSTNAME=${DOZZLE_HOSTNAME}
networks:
- tnet
volumes:
dozzle_data: null
dozzle_agent_data: null
networks:
tnet:
name: traefik-net
external: true
===== CONFIG LAYOUT (/opt/docker/conf/ — top 200 entries) =====
/opt/docker/conf
===== LISTENING PORTS =====
0.0.0.0:111
0.0.0.0:22
0.0.0.0:3000
0.0.0.0:5001
0.0.0.0:53
0.0.0.0:7007
0.0.0.0:8000
0.0.0.0:8080
0.0.0.0:8443
0.0.0.0:853
0.0.0.0:9001
0.0.0.0:9443
[::]:111
127.0.0.1:40185
[::]:22
*:2375
[::]:3000
[::]:5001
[::]:53
[::]:8000
[::]:8080
[::]:8443
[::]:853
[::]:9001
[::]:9443
===== MODEL / HUGGINGFACE CACHES =====
===== DOCKER-ADJACENT SYSTEMD SERVICES =====
containerd.service running
docker.service running
===== DONE =====
Paste the above back into the chat, or pass a path as argv[1] to save.
stacks/backrest/.env.example
+17
View File
@@ -0,0 +1,17 @@
# backrest stack tunables. Copy to `.env` on ana-docker before deploying.
#
# cp .env.example .env
# # edit if needed
# docker compose up -d
#
# Repos, S3 endpoints, restic passwords, and schedules are all configured
# inside the Backrest web UI after first boot — nothing belongs here.
# Image version — pin for reproducibility (`latest` for edge)
BACKREST_VERSION=latest
# Host port for the web UI (container listens on 9898 internally)
BACKREST_PORT=9898
# Timezone — affects scheduler display and log timestamps
TZ=America/Los_Angeles
stacks/backrest/README.md
+65
View File
@@ -0,0 +1,65 @@
# backrest
Web UI over restic repositories. Runs **once, on ana-docker**, and points at every per-host restic repo on both site-local S3 endpoints to give a single pane of glass for snapshot history, restores, and alerts.
**Server:** ana-docker
**Port:** `http://10.250.50.70:9898`
## Role in the fleet
- Actual backups are run by per-host `systemd` timers calling `restic` (see the backup design; not yet deployed). Each host writes to its site-local S3 bucket (TrueNAS on ana, Synology on nh3).
- **This container does not run backups by default** — it's a viewer/manager pointed at existing repos. (Backrest *can* be the scheduler instead of systemd timers; we're keeping the scheduler on the host for simplicity.)
- Because it only needs to talk to S3 endpoints (not host filesystems), no bind mounts of host paths are required.
## Deploy
```bash
# On ana-docker:
sudo mkdir -p /opt/docker/compose/backrest
sudo chown $USER /opt/docker/compose/backrest
cd /opt/docker/compose/backrest
# scp compose.yaml + .env.example from this workspace, then:
cp .env.example .env
# edit .env if you want a different port or TZ
docker compose config
docker compose up -d
docker compose logs -f
```
Open `http://10.250.50.70:9898` and set the admin credentials on first load.
## First-time configuration (in the UI)
For each per-host repo (to be added once the restic pipeline is running):
1. **Add Repository** → fill in:
- **ID:** e.g. `ana-docker`, `ana-ml2`, `nh3-docker`, `esh-docker-vm`
- **URI:** `s3:https://10.250.50.50:4521/pfi-backups/ana/ana-docker/` (or the nh3 endpoint, depending on host)
- **Password:** the restic passphrase for that host
- **Env:** `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` for the S3 endpoint
2. *(Do not create a plan unless you want Backrest to drive the schedule — leave schedules to the systemd timers on each host.)*
3. Verify: the repo should list snapshots pulled by the host's systemd timer within a few minutes.
## Backup scope, when wired up
- ana-docker, ana-ml2 → TrueNAS S3 at `10.250.50.50:4521`
- nh3-docker, esh-docker-vm → Synology S3 at `10.100.50.50:4521`
- Cross-site rclone sync makes each bucket also hold the other site's data, so you can restore *either* host from *either* side if one NAS is down.
## Scaling knobs
- **Upgrade:** `docker compose pull && docker compose up -d`.
- **Backup of Backrest itself:** its config/DB lives in the `backrest_config` and `backrest_data` volumes — include those in ana-docker's restic plan so recreating the UI doesn't mean re-entering every repo.
## Why not `offen/docker-volume-backup`?
Two instances on `esh-docker-vm` (paperless-ngx, pgadmin) currently use the `offen/docker-volume-backup` sidecar pattern. Those will be retired once restic is in place:
- No encryption — backups sit in the clear on NFS.
- No dedup — every run writes a full tarball; storage grows linearly.
- Per-stack config — every new service needs its own sidecar wiring.
- No cross-host index — restores require knowing which tarball lives where.
Restic + Backrest solves all four at the cost of one extra binary per host. Leave the sidecars running until the restic plan is verified, then remove them in a scheduled change.
stacks/backrest/compose.yaml
+58
View File
@@ -0,0 +1,58 @@
# Backrest — web UI over restic repositories.
#
# Role here: single central viewer for every host's restic repo on both
# site-local S3 endpoints (TrueNAS at ana, Synology at nh3). Per-host
# `restic` runs will still be driven by systemd timers on each host; this
# stack is how we see what ran, browse snapshots, and restore.
#
# Repos and S3 credentials are configured in the Backrest UI after first
# boot — nothing baked into this file. Data (its own SQLite + queue) lives
# in a named volume so the config survives container recreation.
#
# All tunables live in .env — edit that, not this file.
services:
backrest:
image: garethgeorge/backrest:${BACKREST_VERSION}
container_name: backrest
hostname: backrest
restart: unless-stopped
ports:
- "${BACKREST_PORT}:9898"
volumes:
- backrest_data:/data
- backrest_config:/config
- backrest_cache:/cache
- backrest_tmp:/tmp
environment:
- BACKREST_DATA=/data
- BACKREST_CONFIG=/config/config.json
- XDG_CACHE_HOME=/cache
- TMPDIR=/tmp
- TZ=${TZ:-America/Los_Angeles}
- BACKREST_PORT=0.0.0.0:9898
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:9898/"]
interval: 30s
timeout: 10s
retries: 3
start_period: 15s
networks:
- tnet
labels:
- homepage.group=PFI-ANA
- homepage.name=Backrest
- homepage.icon=mdi-backup-restore
- homepage.description=Restic snapshot viewer / restore UI
- homepage.href=http://10.250.50.70:${BACKREST_PORT}
volumes:
backrest_data:
backrest_config:
backrest_cache:
backrest_tmp:
networks:
tnet:
name: traefik-net
external: true
stacks/beszel/.env.example
+43
View File
@@ -0,0 +1,43 @@
# beszel stack tunables. Copy to `.env` on each server before deploying.
#
# cp .env.example .env
# # edit for this host
# docker compose up -d
#
# Same compose.yaml on both servers — COMPOSE_PROFILES picks the role.
# Image version — pin for reproducibility (`latest` for edge)
BESZEL_VERSION=latest
# ------------------------------------------------------------------------
# On ana-docker (hub + local agent):
# COMPOSE_PROFILES=hub,agent
# BESZEL_PORT=8090
# BESZEL_HUB_KEY=<copy from hub "Add system" dialog after first boot>
#
# On ana-ml2 (agent only):
# COMPOSE_PROFILES=agent
# BESZEL_HUB_KEY=<same key as above>
# ------------------------------------------------------------------------
COMPOSE_PROFILES=hub,agent
# ---- Hub-only -----------------------------------------------------------
# Host port for the web UI (container listens on 8090 internally).
BESZEL_PORT=8090
# ---- Agent-only ---------------------------------------------------------
# Host port the agent listens on. The hub SSHes into agents over this port.
BESZEL_AGENT_PORT=45876
# Hub's SSH public key — paste from the hub UI on first run.
# Grab it by clicking "Add System" → copy the key shown in the dialog.
BESZEL_HUB_KEY=
# Extra filesystems to track beyond the root mount, comma-separated.
# Examples:
# on ana-ml2: /tank
# on ana-docker: /mnt/backup,/mnt/compose
BESZEL_EXTRA_FS=
stacks/beszel/README.md
+88
View File
@@ -0,0 +1,88 @@
# beszel
Lightweight monitoring — CPU, memory, disk, network, and per-container stats for every Docker host, with alerts over email/webhook. Pairs with Dozzle (logs) on the same server.
**Deploys to:**
- **ana-docker** (hub + local agent) — UI at `http://10.250.50.70:8090`
- **ana-ml2** (agent only) — listens on `10.250.50.54:45876`
- **nh3-docker** (agent only, cross-site) — listens on `10.100.50.40:45876`
Same compose.yaml on each host. Per-host `.env` sets `COMPOSE_PROFILES` to bring up the right combination. Each agent host is added individually in the hub UI.
## How hub ↔ agent auth works
Beszel uses SSH-key-based auth: the hub generates its own keypair on first boot, and each agent must be seeded with the hub's **public key** via the `KEY` env var. Agents listen on a port (default 45876); the hub pulls metrics by connecting to them with that key.
Operator flow on first deploy:
1. Bring up the **hub** on ana-docker with `BESZEL_HUB_KEY=` blank and the agent profile disabled.
2. Open the UI, create the admin account, click **Add System** — Beszel shows the public key.
3. Copy the key into `BESZEL_HUB_KEY` in the `.env` on both hosts.
4. Re-deploy the hub with `COMPOSE_PROFILES=hub,agent` to add the local agent; deploy the agent on ana-ml2.
5. Back in the UI, **Add System** with `host=127.0.0.1 port=45876` (local) and `host=10.250.50.54 port=45876` (ana-ml2).
## Deploy — hub + local agent (ana-docker)
```bash
ssh ana-docker
sudo mkdir -p /opt/docker/compose/beszel
sudo chown $USER /opt/docker/compose/beszel
cd /opt/docker/compose/beszel
# scp compose.yaml + .env.example, then:
cp .env.example .env
# First pass — hub only, no key yet:
# COMPOSE_PROFILES=hub
# BESZEL_PORT=8090
docker compose up -d
# Open http://10.250.50.70:8090 → create admin → click "Add System" →
# copy the displayed public key into BESZEL_HUB_KEY.
# Second pass — add the local agent:
# COMPOSE_PROFILES=hub,agent
# BESZEL_EXTRA_FS=/mnt/backup,/mnt/compose
docker compose up -d
```
## Deploy — agent (ana-ml2)
```bash
ssh ana-ml2
sudo mkdir -p /opt/docker/compose/beszel
sudo chown $USER /opt/docker/compose/beszel
cd /opt/docker/compose/beszel
# scp the same compose.yaml + .env.example, then:
cp .env.example .env
# Edit to:
# COMPOSE_PROFILES=agent
# BESZEL_HUB_KEY=<same key as the hub>
# BESZEL_EXTRA_FS=/tank
docker compose up -d
docker compose logs -f
```
Then in the hub UI, **Add System** with `host=10.250.50.54`, `port=45876`.
## Verify
```bash
# Hub health
curl -s http://10.250.50.70:8090/api/health
# Agent reachable
ssh ana-docker 'nc -zv 10.250.50.54 45876'
# Local agent reachable from hub container
docker exec beszel nc -zv host.docker.internal 45876
```
## Sizing / impact
The agent is ~10 MB RAM and negligible CPU — runs fine alongside anything on ana-ml2 including GPU workloads. Host-mode networking means it has no port conflicts with other stacks as long as `BESZEL_AGENT_PORT` stays unique.
## Alerts
Configured inside the hub UI (Settings → Notifications). Supports email (SMTP), Gotify, ntfy, Discord, Slack, and generic webhooks. Alert rules attach to per-system or global thresholds (CPU, memory, disk, container down, etc.).
stacks/beszel/compose.yaml
+63
View File
@@ -0,0 +1,63 @@
# Beszel — lightweight server/container monitoring.
#
# Hub: single web UI with the SQLite store. Agents: per-host metric collectors
# that the hub pulls from over SSH.
#
# Multi-host layout via compose profiles:
# COMPOSE_PROFILES=hub → hub only (ana-docker)
# COMPOSE_PROFILES=hub,agent → hub + local agent on the same host
# COMPOSE_PROFILES=agent → agent only (ana-ml2)
#
# The agent uses network_mode: host so it sees real host CPU/mem/net/disk
# counters rather than container-scoped ones — that's why it can't share
# the tnet network with the hub.
#
# All tunables live in .env — edit that, not this file.
services:
beszel:
image: henrygd/beszel:${BESZEL_VERSION}
container_name: beszel
profiles: [hub]
restart: unless-stopped
ports:
- "${BESZEL_PORT}:8090"
volumes:
- beszel_data:/beszel_data
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:8090/api/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 15s
networks:
- tnet
labels:
- homepage.group=PFI-ANA
- homepage.name=Beszel
- homepage.icon=mdi-chart-line
- homepage.description=Server + container monitoring
- homepage.href=http://10.250.50.70:${BESZEL_PORT}
beszel-agent:
image: henrygd/beszel-agent:${BESZEL_VERSION}
container_name: beszel-agent
profiles: [agent]
restart: unless-stopped
network_mode: host
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- beszel_agent_data:/var/lib/beszel-agent
environment:
- PORT=${BESZEL_AGENT_PORT:-45876}
- KEY=${BESZEL_HUB_KEY}
- EXTRA_FILESYSTEMS=${BESZEL_EXTRA_FS:-}
volumes:
beszel_data:
beszel_agent_data:
networks:
tnet:
name: traefik-net
external: true
stacks/dozzle/.env.example
+50
View File
@@ -0,0 +1,50 @@
# dozzle stack tunables. Copy to `.env` on each server before deploying.
#
# cp .env.example .env
# # edit for this host
# docker compose up -d
#
# Same compose.yaml on both servers — COMPOSE_PROFILES picks the role.
# Image version — pin for reproducibility (`latest` for edge)
DOZZLE_VERSION=latest
# ------------------------------------------------------------------------
# On ana-docker (hub):
# COMPOSE_PROFILES=hub
# DOZZLE_HOSTNAME=ana-docker
# DOZZLE_REMOTE_AGENT=10.250.50.54:7007
#
# On ana-ml2 (agent):
# COMPOSE_PROFILES=agent
# DOZZLE_HOSTNAME=ana-ml2
# ------------------------------------------------------------------------
COMPOSE_PROFILES=hub
# Display name for this host in the UI (shown as a tab / section header).
DOZZLE_HOSTNAME=ana-docker
# ---- Hub-only -----------------------------------------------------------
# Host port for the web UI (container listens on 8080 internally).
DOZZLE_PORT=8088
# Comma-separated list of remote agents the hub should connect to.
# Leave blank if this host only views its own containers.
DOZZLE_REMOTE_AGENT=10.250.50.54:7007,10.100.50.40:7007
# Auth — `none` is fine behind the LAN / a reverse proxy with auth.
# Switch to `simple` and set USERNAME/PASSWORD to gate the UI itself.
DOZZLE_AUTH_PROVIDER=none
DOZZLE_USERNAME=
DOZZLE_PASSWORD=
# ---- Agent-only ---------------------------------------------------------
# Host port the agent listens on (container listens on 7007 internally).
DOZZLE_AGENT_PORT=7007
# Bind address — restrict to the LAN interface if you want belt-and-braces
# beyond what the firewall already enforces. Default 0.0.0.0 exposes on all.
DOZZLE_AGENT_BIND=0.0.0.0
stacks/dozzle/README.md
+74
View File
@@ -0,0 +1,74 @@
# dozzle
Container log viewer. One UI on **ana-docker** aggregates logs from every Docker host via remote agents.
**Deploys to:**
- **ana-docker** (hub) — UI at `http://10.250.50.70:8088`
- **ana-ml2** (agent) — listens on `10.250.50.54:7007`
- **nh3-docker** (agent, cross-site) — listens on `10.100.50.40:7007`
One compose.yaml lives on each host. The per-host `.env` sets `COMPOSE_PROFILES=hub` or `COMPOSE_PROFILES=agent` so `docker compose up -d` brings up the right service. On the hub, add every agent to `DOZZLE_REMOTE_AGENT` as a comma-separated list (e.g. `10.250.50.54:7007,10.100.50.40:7007`).
## Auth / TLS note
Dozzle agents and hub auto-generate mTLS certificates on first run. On the trusted LAN (10.250.0.0/16) the default config is fine. If you ever expose an agent beyond the LAN, generate and pin certificates explicitly per the Dozzle docs (`dozzle generate`). The web UI itself is unauthenticated by default — flip `DOZZLE_AUTH_PROVIDER=simple` and set `DOZZLE_USERNAME`/`DOZZLE_PASSWORD` in the hub `.env` if you want a login gate.
## Deploy — hub (ana-docker)
```bash
ssh ana-docker
sudo mkdir -p /opt/docker/compose/dozzle
sudo chown $USER /opt/docker/compose/dozzle
cd /opt/docker/compose/dozzle
# scp compose.yaml + .env.example from this workspace, then:
cp .env.example .env
# Ensure:
# COMPOSE_PROFILES=hub
# DOZZLE_HOSTNAME=ana-docker
# DOZZLE_REMOTE_AGENT=10.250.50.54:7007
# DOZZLE_PORT=8088
docker compose config
docker compose up -d
docker compose logs -f
```
## Deploy — agent (ana-ml2)
```bash
ssh ana-ml2
sudo mkdir -p /opt/docker/compose/dozzle
sudo chown $USER /opt/docker/compose/dozzle
cd /opt/docker/compose/dozzle
# scp the same compose.yaml + .env.example, then:
cp .env.example .env
# Edit to:
# COMPOSE_PROFILES=agent
# DOZZLE_HOSTNAME=ana-ml2
# DOZZLE_AGENT_PORT=7007
docker compose config
docker compose up -d
docker compose logs -f
```
## Verify
```bash
# Hub health (from anywhere on LAN)
curl -s http://10.250.50.70:8088/healthz
# Agent reachable from the hub's perspective
ssh ana-docker 'nc -zv 10.250.50.54 7007'
# Open http://10.250.50.70:8088 — you should see two tabs:
# "ana-docker" (local containers) and "ana-ml2" (via agent).
```
## Troubleshooting
- **Hub shows only local containers:** agent is unreachable. Check firewall rules on ana-ml2 (port 7007 must be open from 10.250.50.70) and that the agent is actually listening (`ss -tlnp | grep 7007`).
- **Agent keeps restarting:** verify the docker.sock bind mount is read-only and the socket exists.
- **Certificate mismatch after image upgrade:** delete the `dozzle_data` and `dozzle_agent_data` volumes on both hosts and redeploy to regenerate.
stacks/dozzle/compose.yaml
+66
View File
@@ -0,0 +1,66 @@
# Dozzle — container log viewer.
#
# Multi-host layout via compose profiles:
# COMPOSE_PROFILES=hub → runs the web UI (deploy on ana-docker)
# COMPOSE_PROFILES=agent → runs the remote agent (deploy on ana-ml2)
#
# Same compose.yaml on both servers; per-host `.env` picks the profile.
#
# All tunables live in .env — edit that, not this file.
services:
dozzle:
image: amir20/dozzle:${DOZZLE_VERSION}
container_name: dozzle
profiles: [hub]
restart: unless-stopped
ports:
- "${DOZZLE_PORT}:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- dozzle_data:/data
environment:
- DOZZLE_HOSTNAME=${DOZZLE_HOSTNAME}
- DOZZLE_REMOTE_AGENT=${DOZZLE_REMOTE_AGENT:-}
- DOZZLE_AUTH_PROVIDER=${DOZZLE_AUTH_PROVIDER:-none}
- DOZZLE_USERNAME=${DOZZLE_USERNAME:-}
- DOZZLE_PASSWORD=${DOZZLE_PASSWORD:-}
healthcheck:
test: ["CMD", "/dozzle", "healthcheck"]
interval: 30s
timeout: 10s
retries: 3
start_period: 15s
networks:
- tnet
labels:
- homepage.group=PFI-ANA
- homepage.name=Dozzle
- homepage.icon=mdi-text-box-search
- homepage.description=Container logs (ana-docker + ana-ml2)
- homepage.href=http://10.250.50.70:${DOZZLE_PORT}
dozzle-agent:
image: amir20/dozzle:${DOZZLE_VERSION}
container_name: dozzle-agent
profiles: [agent]
restart: unless-stopped
command: agent
ports:
- "${DOZZLE_AGENT_BIND:-0.0.0.0}:${DOZZLE_AGENT_PORT}:7007"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- dozzle_agent_data:/data
environment:
- DOZZLE_HOSTNAME=${DOZZLE_HOSTNAME}
networks:
- tnet
volumes:
dozzle_data:
dozzle_agent_data:
networks:
tnet:
name: traefik-net
external: true
stacks/infinity/.env.example
+30
View File
@@ -0,0 +1,30 @@
# Infinity stack tunables. Copy this to `.env` on the server before deploying.
#
# cp .env.example .env
# # edit .env with real values
# docker compose up -d
# Image version — pin for reproducibility (`latest` for edge)
INFINITY_VERSION=latest
# Port exposed on host
INFINITY_PORT=7997
# GPU assignment (ana-ml2 has 0 and 1; default 1 keeps 0 free for heavy LLM work)
GPU_ID=1
# Models — both served simultaneously; reference by the full repo name in requests
EMBED_MODEL=Qwen/Qwen3-Embedding-0.6B
RERANK_MODEL=Qwen/Qwen3-Reranker-0.6B
# Inference engine: torch (widest support) or optimum (ONNX, sometimes faster)
ENGINE=torch
# Batch size — 32 is a safe default; bump for throughput if VRAM allows
BATCH_SIZE=32
# Optional API key — leave blank for no auth (fine on the internal network)
API_KEY=
# HuggingFace token — only needed for gated models
HF_TOKEN=
stacks/infinity/README.md
+65
View File
@@ -0,0 +1,65 @@
# infinity
OpenAI-compatible embeddings + reranker server. One container serves both embedding and reranker models simultaneously.
**Server:** ana-ml2
**Port:** 7997 (infinity default)
**GPU:** pinned to GPU 1 by default (configurable via `.env`)
## What it replaces / supersedes
- `qwen3-embedding-0.6B` entry in llama-swap (llama.cpp GGUF → infinity transformer)
- `qwen3-reranker-0.6B` entry in llama-swap
Once infinity is verified stable, retire those two entries from `stacks/llama-swap/config.yaml`.
## Deploy
```bash
# On ana-ml2:
sudo mkdir -p /opt/docker/compose/infinity
sudo chown $USER /opt/docker/compose/infinity
cd /opt/docker/compose/infinity
# Copy compose.yaml + .env.example here (e.g. via scp from this workspace)
# Then:
cp .env.example .env
# edit .env — pick GPU, models, etc.
# Pre-download models into the shared HF cache (optional, speeds first boot)
HF_HOME=/tank/aimodels/huggingface hf download "$(grep ^EMBED_MODEL .env | cut -d= -f2)"
HF_HOME=/tank/aimodels/huggingface hf download "$(grep ^RERANK_MODEL .env | cut -d= -f2)"
# Dry-parse
docker compose config
# Launch
docker compose up -d
docker compose logs -f
```
## Verify
```bash
# Health
curl -s http://localhost:7997/health
# Embedding
curl -s http://localhost:7997/embeddings \
-H "Content-Type: application/json" \
-d '{"model":"Qwen/Qwen3-Embedding-0.6B","input":["hello world"]}' | jq .
# Reranker
curl -s http://localhost:7997/rerank \
-H "Content-Type: application/json" \
-d '{"model":"Qwen/Qwen3-Reranker-0.6B","query":"what is a cat","documents":["cats are mammals","dogs bark"]}' | jq .
# Listed models
curl -s http://localhost:7997/models | jq .
```
## Scaling knobs
- **`BATCH_SIZE`** in `.env` — bigger = higher throughput, more VRAM. 32 is safe; try 64 or 128 if you have headroom.
- **Model size** — Qwen3-Embedding/Reranker come in 0.6B / 4B / 8B. Pick based on quality-vs-latency tradeoff. On RTX 6000 Ada 46 GB, the 8B pair fits easily (~20 GB VRAM).
- **`ENGINE=optimum`** — uses ONNX runtime, sometimes faster. Requires the model to have ONNX weights available; fall back to `torch` if it errors on startup.
stacks/infinity/compose.yaml
+63
View File
@@ -0,0 +1,63 @@
# Infinity — OpenAI-compatible embeddings + reranker server.
#
# Serves embedding and reranker models simultaneously from one container
# on port 7997 (HTTP). Consumers: AIPA agents (search/retrieval), LibreChat
# RAG, anything that needs vector embeddings.
#
# All tunables live in .env — edit that, not this file.
#
# Pre-download models to avoid first-run delay:
# HF_HOME=/tank/aimodels/huggingface hf download Qwen/Qwen3-Embedding-0.6B
# HF_HOME=/tank/aimodels/huggingface hf download Qwen/Qwen3-Reranker-0.6B
services:
infinity:
image: michaelf34/infinity:${INFINITY_VERSION}
container_name: infinity
restart: unless-stopped
ports:
- "${INFINITY_PORT}:7997"
volumes:
- /tank/aimodels/huggingface:/hfcache
environment:
- HF_HOME=/hfcache
- HF_HUB_CACHE=/hfcache/hub
- HUGGING_FACE_HUB_TOKEN=${HF_TOKEN:-}
command: >
v2
--model-id ${EMBED_MODEL}
--model-id ${RERANK_MODEL}
--engine ${ENGINE}
--device cuda
--batch-size ${BATCH_SIZE}
--host 0.0.0.0
--port 7997
--api-key ${API_KEY:-}
deploy:
resources:
reservations:
devices:
- driver: nvidia
device_ids:
- "${GPU_ID}"
capabilities:
- gpu
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:7997/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 120s
networks:
- tnet
labels:
- homepage.group=AI Systems
- homepage.name=Infinity
- homepage.icon=mdi-vector-arrange-below
- homepage.description=Embeddings + Reranker API (ana-ml2)
- homepage.href=http://10.250.50.54:7997/docs
networks:
tnet:
name: traefik-net
external: true
stacks/llama-swap/.env.example
+18
View File
@@ -0,0 +1,18 @@
# llama-swap stack tunables. Copy to `.env` on ana-docker before deploying.
#
# cp .env.example .env
# # edit if needed
# docker compose up -d
# Image tag. `cuda` is the CUDA-enabled build; pin to a specific release
# (e.g. `cuda-v0.0.6`) for reproducibility once upstream tags stabilize.
LLAMA_SWAP_VERSION=cuda
# Host port for the OpenAI-compatible API (container listens on 8080)
LLAMA_SWAP_PORT=9292
# Host paths
# --- Legacy GGUF models referenced by config.yaml as `-m /models/<file>`
MODELS_DIR=/tank/aimodels/llm
# --- Shared HuggingFace cache used by `-hf` model entries
HF_CACHE_DIR=/tank/aimodels/huggingface
stacks/llama-swap/README.md
+52
View File
@@ -0,0 +1,52 @@
# llama-swap
GGUF model server with on-demand model swapping. Served via llama.cpp's `llama-server` under the llama-swap proxy.
**Server:** ana-ml2
**Port:** 9292 (configurable via `.env`)
**GPU:** both (unpinned — `runtime: nvidia` grants access to all devices; per-model GPU selection happens inside `config.yaml`)
## Files
- **`compose.yaml`** — canonical compose. Deployed to `/opt/docker/compose/llama-swap/compose.yaml` on ana-ml2.
- **`.env.example`** — template for the per-host `.env`. Copy to `.env` on the server and tweak.
- **`config.yaml`** — model definitions and groups. Deployed to `/opt/docker/conf/llama-swap/config.yaml` on the server.
Homepage labels are in the compose file under the `AI Systems` group, matching the convention used by `vllm-qwen3` and `infinity`.
## Deploy a fresh install
```bash
scripts/deploy-stack.sh ana-ml2 llama-swap
ssh ana-ml2 '
cd /opt/docker/compose/llama-swap && \
cp -n .env.example .env && \
docker compose config && \
docker compose up -d && \
docker compose logs --tail=30
'
```
## Model reference conventions
- **Modern entries:** use `-hf <user>/<repo>[:<quant>]` — reads from the shared HF cache, nothing to pre-stage outside `hf download`
- **Legacy entries:** use `--model /models/<dir>/<file>.gguf` — reads GGUFs from `/tank/aimodels/llm/` (pre-HF-cache era, gradually being migrated)
New models should prefer the `-hf` pattern.
## Deploy updates to config only
```bash
# After editing config.yaml here:
scp config.yaml ana-ml2:/opt/docker/conf/llama-swap/config.yaml
ssh ana-ml2 'cd /opt/docker/compose/llama-swap && docker compose restart'
```
## Deploy updates to compose only
```bash
# After editing compose.yaml or .env.example here:
scripts/deploy-stack.sh ana-ml2 llama-swap
ssh ana-ml2 'cd /opt/docker/compose/llama-swap && docker compose up -d'
```
stacks/llama-swap/compose.yaml
+50
View File
@@ -0,0 +1,50 @@
# llama-swap — GGUF model server with on-demand model swapping.
#
# Proxies OpenAI-compatible API requests to llama.cpp server instances
# and swaps which model is loaded into VRAM per request. Runs on
# ana-ml2 using both GPUs dynamically (no explicit device pinning —
# llama-swap picks per-model-definition).
#
# Model definitions live in /opt/docker/conf/llama-swap/config.yaml on
# the server. Canonical copy of that config is config.yaml in this
# workspace; deploy with scp + `docker compose restart` or the script
# at the bottom of README.md.
#
# All tunables live in .env — edit that, not this file.
services:
llama-swap:
image: ghcr.io/mostlygeek/llama-swap:${LLAMA_SWAP_VERSION}
container_name: llama-swap
restart: unless-stopped
stdin_open: true
tty: true
runtime: nvidia
ports:
- "${LLAMA_SWAP_PORT}:8080"
volumes:
- /opt/docker/conf/llama-swap/config.yaml:/app/config.yaml
- ${MODELS_DIR}:/models
- ${HF_CACHE_DIR}:/hfcache
environment:
- HF_HOME=/hfcache
- HF_HUB_CACHE=/hfcache/hub
healthcheck:
test: ["CMD-SHELL", "curl -fsS http://localhost:8080/ >/dev/null || exit 1"]
interval: 30s
timeout: 10s
retries: 3
start_period: 30s
networks:
- tnet
labels:
- homepage.group=AI Systems
- homepage.name=llama-swap
- homepage.icon=mdi-swap-horizontal
- homepage.description=GGUF model swapper (llama.cpp; ana-ml2)
- homepage.href=http://10.250.50.54:${LLAMA_SWAP_PORT}
networks:
tnet:
name: traefik-net
external: true
stacks/llama-swap/config.yaml
+463
View File
@@ -0,0 +1,463 @@
# ============================================================================
# llama-swap configuration for PFI-ANA
# Optimized and synchronized with /models disk inventory
# Last updated: 2026-04-10
#
# KB Sources:
# - reference/nemotron-3-super-running-parameters.md
# - reference/nemotron-3-nano-running-parameters.md
# - reference/qwen3.5-running-parameters.md
# - reference/qwen3-coder-next-running-parameters.md
# - reference/gemma-4-running-parameters.md
# - reference/qwen3-embedding-running-parameters.md
# - reference/qwen3-reranker-running-parameters.md
#
# Changelog:
# 2025-07-22: Removed jina-reranker-v3 (unused, out of rotation).
# 2026-04-10: Fixed Qwen3-Embedding pooling (mean→last; causal LM uses last-token
# pooling). Fixed ctx-size 4096→8192 for embedding+reranker. Fixed
# reranker: removed --embeddings flag (not an embedding model).
# 2026-04-17: Added Qwen3.6-35B-A3B Abliterated Heretic Q8_0 via -hf syntax.
# Requires HF_HOME=/hfcache in compose (see docker-compose.yml).
# New convention: use -hf repo[:quant] instead of --model /path.
# ============================================================================
# Default 1200 seconds (20 min) to wait for model to be available to load.
healthCheckTimeout: 1200
# logLevel: sets the logging value
# - optional, default: info
# - Valid log levels: debug, info, warn, error
logLevel: info
# metricsMaxInMemory: maximum number of metrics to keep in memory
# - optional, default: 1000
metricsMaxInMemory: 1000
# startPort: sets the starting port number for the automatic ${PORT} macro.
# - optional, default: 5800
# - the ${PORT} macro can be used in model.cmd and model.proxy settings
# - it is automatically incremented for every model that uses it
# startPort: 10001
models:
# ==========================================================================
# QWEN 3.5 MODELS (KB-recommended settings)
# - Thinking mode: temp 1.0, top-p 0.95, top-k 20, min-p 0.0, presence_penalty 1.5
# - Coding (precise): temp 0.6, top-p 0.95, top-k 20, min-p 0.0, presence_penalty 0.0
# - Non-thinking general: temp 0.7, top-p 0.8, top-k 20, min-p 0.0, presence_penalty 1.5
# - Context: 256K native (start 16K-32K for responsiveness)
# - Gibberish fix: add --cache-type-k bf16 --cache-type-v bf16
# - No Ollama support for Qwen3.5 GGUFs — use llama.cpp only
# ==========================================================================
"qwen3.5-35-a3b":
name: "Qwen 3.5 35B-A3B Thinking"
description: "MoE reasoning model. 3B active params, general-purpose thinking/chat."
ttl: 600
cmd: |
/app/llama-server
--context-shift
--model /models/unsloth_Qwen3.5-35B-A3B-GGUF/Qwen3.5-35B-A3B-UD-Q4_K_XL.gguf
--port ${PORT}
--n-gpu-layers 999
--ctx-size 32768
--flash-attn on
--temp 1.0
--top-p 0.95
--top-k 20
--min-p 0.00
--presence-penalty 1.5
--chat-template-kwargs '{"enable_thinking":true}'
"qwen3.5-122b-a10b":
name: "Qwen 3.5 122B-A10B UD-Q4_K_XL"
description: "Large MoE reasoning model. 10B active params, heavy reasoning tasks."
ttl: 600
cmd: |
/app/llama-server
--context-shift
--model /models/unsloth_Qwen3.5-122B-A10B-GGUF/UD-Q4_K_XL/Qwen3.5-122B-A10B-UD-Q4_K_XL-00001-of-00003.gguf
--port ${PORT}
--n-gpu-layers 999
--ctx-size 32768
--flash-attn on
--temp 1.0
--top-p 0.95
--top-k 20
--min-p 0.00
--presence-penalty 1.5
--chat-template-kwargs '{"enable_thinking":true}'
"qwen3.5-9b":
name: "Qwen 3.5 9B UD-Q4_K_XL"
description: "Dense 9B model. Lightweight general-purpose chat and reasoning."
ttl: 600
cmd: |
/app/llama-server
--context-shift
--model /models/unsloth_Qwen3.5-9B-GGUF/Qwen3.5-9B-UD-Q4_K_XL.gguf
--port ${PORT}
--n-gpu-layers 999
--ctx-size 32768
--flash-attn on
--temp 1.0
--top-p 0.95
--top-k 20
--min-p 0.00
--presence-penalty 1.5
--chat-template-kwargs '{"enable_thinking":true}'
# --------------------------------------------------------------------------
# Qwen 3.6 — uses -hf syntax, reads from HF_HOME=/hfcache (host pre-download)
# --------------------------------------------------------------------------
"qwen3.6-35-a3b-abliterated":
name: "Qwen 3.6 35B-A3B Abliterated Heretic Q8_0"
description: "Qwen3.6 MoE, 3B active. Abliterated/heretic variant of BF16 quantized to Q8_0. ~38GB."
ttl: 600
cmd: |
/app/llama-server
--context-shift
--jinja
-hf IIEleven11/Qwen3.6-35B-A3B-Abliterated-Heretic-BF16-Q8_0-GGUF
--port ${PORT}
--n-gpu-layers 999
--ctx-size 32768
--flash-attn on
--temp 1.0
--top-p 0.95
--top-k 20
--min-p 0.00
--presence-penalty 1.5
--repeat-penalty 1.0
--reasoning on
--reasoning-format deepseek
# ==========================================================================
# NEMOTRON MODELS (KB-recommended settings)
# - General Chat: temp 1.0, top-p 1.0, min_p 0.01
# - Tool Calling: temp 0.6, top-p 0.95, min_p 0.01
# - NoPE architecture: no YaRN needed
# - DEPRECATED --special flag for reasoning tokens
# - --special flag causes issues.
# - Start ctx 16K-32K, increase cautiously
# ==========================================================================
"nemotron-3-super-120b":
name: "NVIDIA Nemotron 3 Super 120B-A12B UD-Q4_K_XL"
description: "Flagship NVIDIA reasoning model. 12B active of 120B, MoE. 64-72GB VRAM at Q4."
ttl: 600
cmd: |
/app/llama-server
--context-shift
--model /models/unsloth_NVIDIA-Nemotron-3-Super-120B-A12B-GGUF/UD-Q4_K_XL/NVIDIA-Nemotron-3-Super-120B-A12B-UD-Q4_K_XL-00001-of-00003.gguf
--port ${PORT}
--n-gpu-layers 999
--ctx-size 16384
--flash-attn on
--temp 1.0
--top-p 1.0
--min-p 0.01
--seed 3407
"nemotron-3-nano-30b":
name: "NVIDIA Nemotron 3 Nano 30B-A3B UD-Q4_K_XL"
description: "Compact Nemotron. 3B active of 30B, MoE. ~24GB at Q4. Best performance/size on 24GB GPUs."
ttl: 600
cmd: |
/app/llama-server
--context-shift
--special
--model /models/unsloth_Nemotron-3-Nano-30B-A3B-GGUF/Nemotron-3-Nano-30B-A3B-UD-Q4_K_XL.gguf
--port ${PORT}
--n-gpu-layers 999
--ctx-size 32768
--flash-attn on
--temp 1.0
--top-p 1.0
--min-p 0.01
--seed 3407
# ==========================================================================
# GEMMA 4 MODELS (KB-recommended settings)
# - All variants: temp 1.0, top-p 0.95, top-k 64, repeat_penalty 1.0
# - Thinking: enable via --chat-template-kwargs '{"enable_thinking":true}'
# - Multi-turn: only keep final visible answer in history (not thought blocks)
# - Context: E2B/E4B=128K, 26B-A4B/31B=256K. Start at 32K.
# - ⚠️ Do NOT use CUDA 13.2 runtime — causes poor outputs
# - Use llama-server (not llama-cli) for thinking control
# ==========================================================================
"gemma4-26b-a4b":
name: "Gemma 4 26B-A4B"
description: "Google DeepMind Gemma 4 MoE, 4B active params, 256K context. Best speed/quality tradeoff."
ttl: 600
cmd: |
/app/llama-server
--context-shift
--model /models/unsloth_gemma-4-26B-A4B-it-GGUF/gemma-4-26B-A4B-it-UD-Q4_K_XL.gguf
--port ${PORT}
--n-gpu-layers 999
--ctx-size 32768
--flash-attn on
--temp 1.0
--top-p 0.95
--top-k 64
--repeat-penalty 1.0
--chat-template-kwargs '{"enable_thinking":true}'
"gemma4-31b-dense":
name: "Gemma 4 31B Dense"
description: "Google DeepMind Gemma 4 dense 31B. Maximum quality for complex reasoning, 256K context."
ttl: 600
cmd: |
/app/llama-server
--context-shift
--model /models/unsloth_gemma-4-31B-it-GGUF/gemma-4-31B-it-UD-Q4_K_XL.gguf
--port ${PORT}
--n-gpu-layers 999
--ctx-size 32768
--flash-attn on
--temp 1.0
--top-p 0.95
--top-k 64
--repeat-penalty 1.0
--chat-template-kwargs '{"enable_thinking":true}'
# ==========================================================================
# GLM MODELS
# ==========================================================================
"glm4.7-flash":
name: "GLM 4.7 Flash UD-Q4_K_XL"
description: "THUDM GLM 4.7 Flash. Fast inference, general-purpose chat."
ttl: 600
cmd: |
/app/llama-server
--context-shift
--model /models/unsloth_GLM-4.7-Flash-GGUF/GLM-4.7-Flash-UD-Q4_K_XL.gguf
--port ${PORT}
--n-gpu-layers 999
--ctx-size 40000
--flash-attn on
--temp 0.6
--top-p 0.95
"glm-steam-106b":
name: "GLM Steam 106B-A12B Q4_K_M"
description: "TheDrummer GLM Steam MoE. 12B active of 106B. Creative and RP-focused."
ttl: 600
cmd: |
/app/llama-server
--context-shift
--model /models/RP/bartowski_TheDrummer_GLM-Steam-106B-A12B-v1-GGUF/TheDrummer_GLM-Steam-106B-A12B-v1-Q4_K_M-00001-of-00002.gguf
--port ${PORT}
--n-gpu-layers 999
--ctx-size 40000
--flash-attn on
--temp 0.6
--top-p 0.95
# ==========================================================================
# SKYFALL MODELS
# ==========================================================================
"skyfall-r1-31b-q6k":
name: "Skyfall R1 31B v4 Q6_K_L"
description: "TheDrummer Skyfall R1 31B v4. General-purpose reasoning."
ttl: 600
cmd: |
/app/llama-server
--context-shift
--model /models/bartowski_TheDrummer_Skyfall-31B-v4-GGUF/TheDrummer_Skyfall-31B-v4-Q6_K_L.gguf
--port ${PORT}
--n-gpu-layers 999
--ctx-size 40000
--flash-attn on
"skyfall-r1-31b-v4a":
name: "Skyfall R1 31B v4a Q6_K (RP)"
description: "BeaverAI Skyfall R1 v4a variant. RP/creative-focused."
ttl: 600
cmd: |
/app/llama-server
--context-shift
--model /models/RP/BeaverAI_Skyfall-R1-31B-v4a-GGUF/Skyfall-R1-31B-v4a-Q6_K.gguf
--port ${PORT}
--n-gpu-layers 999
--ctx-size 40000
--flash-attn on
# ==========================================================================
# CODER MODELS
# ==========================================================================
"qwen3-coder-next":
name: "Qwen3 Coder Next UD-Q4_K_XL"
description: "Latest Qwen3 Coder. Non-reasoning model, optimized for code gen. KB: temp 1.0, top-k 40, min-p 0.01."
ttl: 600
cmd: |
/app/llama-server
--context-shift
--model /models/unsloth_Qwen3-Coder-Next-GGUF/Qwen3-Coder-Next-UD-Q4_K_XL.gguf
--port ${PORT}
--n-gpu-layers 999
--ctx-size 32768
--flash-attn on
--temp 1.0
--top-p 0.95
--top-k 40
--min-p 0.01
--repeat-penalty 1.0
# ==========================================================================
# LARGE / SPECIAL-PURPOSE MODELS
# ==========================================================================
"kimik2-q2kxl":
name: "Kimi K2 Instruct UD-Q2_K_XL"
description: "Moonshot Kimi K2. Huge MoE model (8-shard Q2). Limited GPU layers due to size."
ttl: 600
cmd: |
/app/llama-server
--context-shift
--model /models/unsloth_Kimi-K2-Instruct-0905-GGUF/UD-Q2_K_XL/Kimi-K2-Instruct-0905-UD-Q2_K_XL-00001-of-00008.gguf
--port ${PORT}
--n-gpu-layers 2
--temp 0.6
--top-p 0.95
# ==========================================================================
# GRANITE MODELS (IBM)
# ==========================================================================
"granite-4-small":
name: "Granite 4.0 Small Q4_K_M"
description: "IBM Granite 4.0 Small. Deterministic utility model for structured tasks."
ttl: 0
cmd: |
/app/llama-server
--context-shift
--model /models/unsloth_granite-4.0-h-small-GGUF/granite-4.0-h-small-Q4_K_M.gguf
--port ${PORT}
--n-gpu-layers 999
--ctx-size 120000
--flash-attn on
--top-p 1.0
--temp 0.0
--top-k 0
"granite-4-micro":
name: "Granite 4.0 Micro Q4_K_M"
description: "IBM Granite 4.0 Micro. Ultra-lightweight for fast structured responses."
ttl: 600
cmd: |
/app/llama-server
--context-shift
--model /models/ibm-granite_granite-4.0-micro-GGUF/granite-4.0-micro-Q4_K_M.gguf
--port ${PORT}
--n-gpu-layers 999
--ctx-size 32768
--flash-attn on
--temp 0.0
--top-p 1.0
# ==========================================================================
# EMBEDDING MODELS (persistent, always loaded)
# ==========================================================================
"embeddinggemma-300M":
name: "Embedding Gemma 300M"
description: "Google Embedding Gemma for vectorization."
ttl: 0
cmd: |
/app/llama-server
--embedding
--pooling cls
--model /models/ggml-org_embeddinggemma-300M-GGUF/embeddinggemma-300M-Q8_0.gguf
--port ${PORT}
--n-gpu-layers 0
--ctx-size 2048
--batch-size 1024
--no-mmap
--ubatch-size 1024
--cont-batching
--threads 24
"qwen3-embedding-0.6B":
name: "Qwen3 Embedding 0.6B"
description: "Qwen3 Embedding model for vectorization. 32K context, last-token pooling (decoder/causal LM)."
ttl: 0
cmd: |
/app/llama-server
--embeddings
--pooling last
--model /models/Qwen_Qwen3-Embedding-0.6B-GGUF/Qwen3-Embedding-0.6B-Q8_0.gguf
--port ${PORT}
--n-gpu-layers 0
--ctx-size 8192
--batch-size 8192
--ubatch-size 2048
--no-mmap
--cont-batching
--threads 24
# ==========================================================================
# RERANKING MODELS (persistent, always loaded)
# ==========================================================================
"qwen3-reranker-0.6B":
name: "Qwen3 Reranker 0.6B"
description: "Qwen3 Reranker for retrieval reranking. Causal LM scoring yes/no logits at last token. Replaces BGE v2."
ttl: 0
cmd: |
/app/llama-server
--reranking
--pooling rank
--model /models/ggml-org_Qwen3-Reranker-0.6B-Q8_0-GGUF/qwen3-reranker-0.6b-q8_0.gguf
--port ${PORT}
--n-gpu-layers 0
--ctx-size 8192
--batch-size 8192
--ubatch-size 2048
--cont-batching
--threads 24
# NOTE: jina-reranker-v3 removed 2025-07-22 — unused, out of rotation.
# Qwen3 Reranker handles all reranking duties.
# ============================================================================
# GROUPS
# - swap: false = models in group can coexist in memory
# - exclusive: false = group can share memory with other groups
# - persistent: true = models never unload (for utility/embedding)
# ============================================================================
groups:
"high-reasoning":
swap: false
exclusive: false
members:
- "qwen3.5-35-a3b"
- "gemma4-31b-dense"
- "nemotron-3-nano-30b"
"heavy-moe":
swap: false
exclusive: false
members:
- "qwen3.5-122b-a10b"
- "nemotron-3-super-120b"
- "kimik2-q2kxl"
- "glm-steam-106b"
"utility":
swap: false
exclusive: false
persistent: true
members:
- "embeddinggemma-300M"
- "qwen3-embedding-0.6B"
- "qwen3-reranker-0.6B"
stacks/rest-server-ana/.env.example
+41
View File
@@ -0,0 +1,41 @@
# rest-server-ana stack tunables. Copy to `.env` on ana-docker.
#
# cp .env.example .env
# # edit if needed
# docker compose up -d
#
# Matches stacks/rest-server-nh3/.env.example — keep them aligned so
# restic clients see the same URL shape against either endpoint.
REST_SERVER_VERSION=latest
# Host port the container listens on (container internal is 8000)
REST_PORT=8000
# Where restic pack files live. TrueNAS NFS share is already mounted at
# /mnt/backup on ana-docker; the "repo/ana" subdir is the historical
# location used by the prior (non-private-repos) rest-server.
#
# With --private-repos, layout becomes:
# ${DATA_DIR}/ana-docker/
# ${DATA_DIR}/ana-ml2/
# ${DATA_DIR}/nh3-docker/
# ${DATA_DIR}/esh-docker-vm/
# Plus the auth file at ${DATA_DIR}/.htpasswd.
DATA_DIR=/mnt/backup/restic/repo/ana
# UID/GID the container process runs as. Must match the owner of
# DATA_DIR so NFS root_squash doesn't bite. On ana-docker this is the
# `lkraven` user (1000:1000).
REST_UID=1000
REST_GID=1000
# Timezone — affects log lines and /metrics timestamps
TZ=America/Los_Angeles
# Extra rest-server flags. Examples:
# --prometheus-no-auth — make /metrics public (needed if Beszel or
# Prometheus scrapes without creds)
# --no-verify-upload — trust the client's hash; faster writes
# Leave blank unless you have a reason.
EXTRA_OPTIONS=
stacks/rest-server-ana/README.md
+143
View File
@@ -0,0 +1,143 @@
# rest-server-ana
Anaheim-site restic backup endpoint. Replaces the older `restic` stack on ana-docker with the same auth model as `rest-server-nh3` on the Synology, so every client host uses identical URL shapes against either endpoint.
**Server:** ana-docker (`10.250.50.70`)
**Port:** `http://10.250.50.70:8000`
**Data:** `/mnt/backup/restic/repo/ana/` (TrueNAS NFS mount on the host)
Paired with:
- **`rest-server-nh3`** on the Synology (`10.100.50.50:8000`, data on Btrfs).
- A cross-site rsync job (TBD, on ana-docker) that mirrors each site's data tree to the other so either NAS can fully restore either site's hosts.
## What changed from the old `restic` stack
| | old `restic` on ana-docker | this stack |
|---|---|---|
| `--private-repos` | no | **yes** |
| `--append-only` | no | **yes** |
| `--prometheus` | no | **yes** |
| healthcheck | no | yes |
| `.env`-driven | no | yes |
| restart policy | none | `unless-stopped` |
| image version | floating `latest` | `${REST_SERVER_VERSION}` |
| stack dir on server | `/opt/docker/compose/restic/` | `/opt/docker/compose/rest-server-ana/` |
Data path is unchanged (`/mnt/backup/restic/repo/ana/`) so nothing new needs to be allocated on TrueNAS.
## Pre-deploy: clean the data dir and create htpasswd
Since there's nothing in the existing path we want to keep, start fresh so the on-disk layout matches `--private-repos`:
```bash
ssh ana-docker '
# Stop the old stack so port 8000 and the data dir are free
cd /opt/docker/compose/restic
docker compose down
# Wipe the old non-private-repos layout
sudo rm -rf /mnt/backup/restic/repo/ana/*
sudo rm -rf /mnt/backup/restic/repo/ana/.htpasswd # if present
# Create the htpasswd file. Use the same passwords here as on the NH3
# Synology so each host has one credential that works at either endpoint.
sudo touch /mnt/backup/restic/repo/ana/.htpasswd
sudo chmod 600 /mnt/backup/restic/repo/ana/.htpasswd
'
# Generate htpasswd entries locally (one per host) and append. Using the
# `httpd:2.4-alpine` throwaway container so we do not depend on
# apache2-utils being installed on ana-docker.
for user in ana-docker ana-ml2 nh3-docker esh-docker-vm; do
read -rs -p "password for $user (must match the NH3 Synology): " pw; echo
docker run --rm httpd:2.4-alpine htpasswd -nbB "$user" "$pw" \
| ssh ana-docker 'sudo tee -a /mnt/backup/restic/repo/ana/.htpasswd >/dev/null'
done
```
If you run that locally and don't have Docker here, equivalent on the server:
```bash
ssh ana-docker "docker run --rm httpd:2.4-alpine htpasswd -nbB <user> '<pw>'" \
| ssh ana-docker 'sudo tee -a /mnt/backup/restic/repo/ana/.htpasswd >/dev/null'
```
## Deploy
Stage the new stack and push it:
```bash
# Stage the stack into the mirror (if not already done via sync-stacks.sh)
mkdir -p stacks-mirror/ana-docker/rest-server-ana
cp stacks/rest-server-ana/compose.yaml stacks/rest-server-ana/.env.example \
stacks-mirror/ana-docker/rest-server-ana/
scripts/deploy-stack.sh ana-docker rest-server-ana
```
Confirm at the prompt. Then on the server:
```bash
ssh ana-docker '
cd /opt/docker/compose/rest-server-ana
cp -n .env.example .env
docker compose config
docker compose up -d
docker compose logs --tail=30
'
```
## Retire the old stack
Once the new one is healthy and the first repo has initialized successfully from a client:
```bash
ssh ana-docker '
cd /opt/docker/compose/restic
docker compose down
# Optionally remove the old stack dir (keep it for a release or two
# in case you need to roll back):
# rm -rf /opt/docker/compose/restic
'
```
## Verify
```bash
# 401 from the root — service up, auth enforced
curl -sS -o /dev/null -w 'unauth status=%{http_code}\n' \
http://10.250.50.70:8000/
# 200 / 404 from a real user+password — auth valid, --private-repos path OK
curl -sS -o /dev/null -w 'auth status=%{http_code}\n' \
-u ana-docker:<password> http://10.250.50.70:8000/ana-docker/
# Init a repo from a client host (one-time per host)
ssh ana-docker '
export RESTIC_REPOSITORY="rest:http://ana-docker:<rest-pw>@10.250.50.70:8000/ana-docker/"
export RESTIC_PASSWORD="<client-side-encryption-passphrase>"
restic init
'
```
## Prune ceremony
Same as `rest-server-nh3` — prune is blocked by `--append-only`. Two options, pick one per endpoint:
- **Temporary flag flip:** edit compose, remove `--append-only` from `OPTIONS`, `docker compose up -d`, run `restic forget --prune` from origin hosts, put the flag back, `docker compose up -d`. Quarterly change.
- **Second endpoint on a different port:** stand up a sibling container (e.g. port `8001`) against the same data dir without `--append-only`, reachable only from a trusted host. Everyday backups still hit `:8000`.
If you go the second-endpoint route, copy this stack to `stacks/rest-server-ana-prune/` with `REST_PORT=8001` and `--append-only` removed from the compose.
## Off-site replication
Scheduled on ana-docker (to be written):
```bash
# Pull NH3's tree down to this side
rsync -avz --delete admin@10.100.50.50:/volume1/Backup/restic/ /mnt/backup/restic-mirror-nh3/
# Push our tree to NH3
rsync -avz --delete /mnt/backup/restic/repo/ana/ admin@10.100.50.50:/volume1/Backup/restic-mirror-ana/
```
Two unidirectional syncs, each running in the direction its data flows. Prune runs only at the origin so the mirror shrinks correctly.
stacks/rest-server-ana/compose.yaml
+59
View File
@@ -0,0 +1,59 @@
# rest-server (Anaheim) — restic backup target for the fleet.
#
# Deploys to ana-docker. Data dir is on the TrueNAS NFS mount
# (/mnt/backup/restic/repo/ana) so snapshots on the NAS side protect the
# backup blobs themselves.
#
# Mirrors stacks/rest-server-nh3/ in every meaningful way — same auth
# model, same on-disk layout, same operational semantics — so each client
# host uses an identical URL shape against either endpoint:
#
# rest:http://<user>:<pw>@10.100.50.50:8000/<user>/ (NH3 Synology)
# rest:http://<user>:<pw>@10.250.50.70:8000/<user>/ (this stack)
#
# Auth model:
# --private-repos : URL path must start with /<user>/ and the HTTP
# basic-auth user must match. Per-host repos are
# strictly isolated.
# --append-only : on-disk data can be added but not removed or
# rewritten; a compromised host can't wipe its own
# history. Prune requires disabling this (see README).
#
# Credentials come from /data/.htpasswd — see README for populating it.
#
# All tunables live in .env — edit that, not this file.
services:
rest-server:
image: restic/rest-server:${REST_SERVER_VERSION}
container_name: rest-server
restart: unless-stopped
# Run as the UID that owns the NFS-backed data dir, so file I/O
# is not subject to NFS root_squash. On ana-docker this is lkraven (1000).
user: "${REST_UID:-1000}:${REST_GID:-1000}"
ports:
- "${REST_PORT}:8000"
volumes:
- ${DATA_DIR}:/data
environment:
- OPTIONS=--private-repos --append-only --prometheus ${EXTRA_OPTIONS:-}
- TZ=${TZ:-America/Los_Angeles}
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://localhost:8000/metrics >/dev/null 2>&1 || [ $? -eq 6 ] && exit 0 || exit 1"]
interval: 30s
timeout: 10s
retries: 3
start_period: 15s
networks:
- tnet
labels:
- homepage.group=PFI-ANA
- homepage.name=Restic (rest-server)
- homepage.icon=mdi-cloud-upload
- homepage.description=Anaheim restic endpoint (data on TrueNAS NFS)
- homepage.href=http://10.250.50.70:${REST_PORT}
networks:
tnet:
name: traefik-net
external: true
stacks/rest-server-nh3/.env.example
+35
View File
@@ -0,0 +1,35 @@
# rest-server (NH3 Synology) tunables. Copy to `.env` on the Synology.
#
# cp .env.example .env
# # edit if needed
# docker compose up -d
#
# Synology-specific notes:
# - Put this stack under /volume1/docker/compose/rest-server/
# (Container Manager's default project root pattern on DSM 7.x).
# - Point DATA_DIR at a Btrfs share you own — /volume1/Backup/restic
# is the natural choice since /volume1/Backup is already the
# fleet-facing backup share (exported via NFS).
REST_SERVER_VERSION=latest
# Host port the container listens on (container internal port is 8000)
REST_PORT=8000
# Where restic pack files live on the Synology. Must be a writable
# Btrfs path. Will hold one subdir per user (--private-repos layout):
# ${DATA_DIR}/ana-docker/
# ${DATA_DIR}/ana-ml2/
# ${DATA_DIR}/nh3-docker/
# ${DATA_DIR}/esh-docker-vm/
# Plus the auth file at ${DATA_DIR}/.htpasswd.
DATA_DIR=/volume1/Backup/restic
# Timezone — affects log lines and the /metrics timestamps
TZ=America/Los_Angeles
# Any extra rest-server flags (rare). Some useful ones:
# --no-verify-upload — trust the client's hash; faster writes
# --max-size=<bytes> — cap per-repo size
# Leave blank unless you have a reason.
EXTRA_OPTIONS=
stacks/rest-server-nh3/README.md
+124
View File
@@ -0,0 +1,124 @@
# rest-server-nh3
NH3-site restic backup endpoint. Runs in Synology Container Manager on `10.100.50.50` and stores pack files on a Btrfs share so Synology snapshots protect against local corruption.
**Server:** Synology RS2418+ at `10.100.50.50`
**Port:** `http://10.100.50.50:8000` (configurable via `.env`)
**Data:** `/volume1/Backup/restic/` (configurable)
Paired with the existing `restic rest-server` on **ana-docker** (`http://10.250.50.70:8000`, data on TrueNAS NFS) as the Anaheim-side endpoint. Each fleet host backs up to the rest-server closest to it; an rsync job on ana-docker mirrors the two trees against each other for off-site redundancy.
## Auth model
`--private-repos` + `--append-only`, enforced via htpasswd:
- One HTTP basic-auth user **per host** (`ana-docker`, `ana-ml2`, `nh3-docker`, `esh-docker-vm`).
- Each user can only write under `/<username>/…` — a compromised host can't see or delete another host's data.
- Append-only means a compromised client can add to its own repo but can't rewrite or delete existing packs, so ransomware on a backed-up host doesn't destroy history.
- **Trade-off:** `restic forget --prune` can't run against an append-only endpoint. Prune ceremony described at the bottom of this file.
## Pre-deploy: create the data path and htpasswd
On the Synology (SSH in as an admin-capable user, or DSM *File Station*):
```bash
# 1. Create the restic data share on a Btrfs volume
ssh admin@10.100.50.50 'sudo mkdir -p /volume1/Backup/restic && \
sudo chown 1000:1000 /volume1/Backup/restic && \
sudo chmod 700 /volume1/Backup/restic'
# 2. Generate htpasswd entries. The Synology doesn't ship apache2-utils,
# so use a throwaway container:
ssh admin@10.100.50.50 'cd /volume1/Backup/restic && \
sudo touch .htpasswd && sudo chown 1000:1000 .htpasswd && sudo chmod 600 .htpasswd'
for user in ana-docker ana-ml2 nh3-docker esh-docker-vm; do
read -s -p "password for $user: " pw; echo
ssh admin@10.100.50.50 \
"docker run --rm httpd:2.4-alpine htpasswd -nbB $user '$pw'" \
| ssh admin@10.100.50.50 "sudo tee -a /volume1/Backup/restic/.htpasswd >/dev/null"
done
```
Record every password in your off-host password manager (1Password / Vaultwarden etc.) — you'll paste them into Backrest and into the systemd timer configs later.
## Deploy
In Synology **Container Manager**:
1. *Project* → **Create** → Name `rest-server`, Path `/volume1/docker/compose/rest-server/`.
2. Copy `compose.yaml` into the project path; copy `.env.example` → `.env` and edit if needed (default `REST_PORT=8000` and `DATA_DIR=/volume1/Backup/restic` should be fine).
3. Start the project.
CLI equivalent (if you have SSH + a shell account that can run Docker on the NAS):
```bash
ssh admin@10.100.50.50
sudo mkdir -p /volume1/docker/compose/rest-server
sudo chown $USER /volume1/docker/compose/rest-server
cd /volume1/docker/compose/rest-server
# scp the files from this workspace, then:
cp .env.example .env
docker compose config
docker compose up -d
docker compose logs -f
```
## Verify
From this workstation:
```bash
# Should return "200 OK" or redirect to /metrics; anything 5xx is a problem.
curl -u ana-docker:<password> -sv http://10.100.50.50:8000/ana-docker/ -o /dev/null
# Once restic is wired up, init the repo (one-time, per host):
RESTIC_REPOSITORY='rest:http://ana-docker:<password>@10.100.50.50:8000/ana-docker/' \
RESTIC_PASSWORD='<client-side-encryption-passphrase>' \
restic init
```
Restic URI shape for each host (paste into Backrest when adding the repo):
```
rest:http://<user>:<pass>@10.100.50.50:8000/<user>/
```
## Prune ceremony (because of --append-only)
Because `--append-only` blocks deletes, `restic forget --prune` will fail against the live endpoint. Two options, pick one and stick with it:
### Option A — temporary flag flip (simplest, requires a maintenance window)
1. On the Synology, edit the stack's `.env` and set `EXTRA_OPTIONS=--no-auth` — **only kidding, don't.** Set `EXTRA_OPTIONS= ` and comment out `--append-only` in the compose `OPTIONS=` line (or parameterize if you prefer).
2. `docker compose up -d` to restart with deletes allowed.
3. Run `restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 3 --prune` from the origin host.
4. Restore `--append-only` and `docker compose up -d`.
Treat this as a quarterly change, not a cron job. Schedule it so you're present if restic hits anything weird.
### Option B — second endpoint on a different port (automation-friendly)
Stand up a second `rest-server` container against the same `DATA_DIR` without `--append-only`, listening on e.g. `8001`, reachable only from within the Synology / over VPN. A scheduled prune job hits that endpoint; day-to-day backup traffic continues to hit `:8000` in append-only mode.
If you end up wanting this, copy this stack to `stacks/rest-server-nh3-prune/` with `REST_PORT=8001` and `--append-only` removed.
## Off-site replication
Scheduled on ana-docker:
```bash
# Example — not the final script, just illustrating the shape.
rsync -avz --delete \
admin@10.100.50.50:/volume1/Backup/restic/ \
/mnt/backup/restic-mirror-nh3/
```
`rsync` is safe because restic packs are immutable once written — nothing under `/volume1/Backup/restic/<user>/data/` gets rewritten, only added or (during prune) removed. A raw `rsync --delete` with prune running only on the origin side is enough; no filesystem-level locks required.
## What doesn't live here
- No `.env` in the committed copy — only `.env.example`.
- `.htpasswd` is never checked in, never synced via `sync-stacks.sh` (its `*.ht*` isn't in the global exclude but the data dir is outside the stack path).
- Client-side restic passwords (the encryption passphrase for each repo) are separate from the HTTP auth passwords and never stored on the Synology.
stacks/rest-server-nh3/compose.yaml
+40
View File
@@ -0,0 +1,40 @@
# rest-server (NH3 Synology) — restic backup target for the fleet.
#
# Deploys into Synology Container Manager on 10.100.50.50. Data lives on
# a Btrfs shared folder so it gets Synology snapshots + optional
# replication to a sibling share if you configure one later.
#
# Auth model:
# --private-repos : every URL path must start with /<username>/ and the
# HTTP basic-auth user must match. One user per host.
# Per-host repos are strictly isolated.
# --append-only : on-disk data can be ADDED but not REMOVED or REWRITTEN.
# A compromised host can't delete its own history.
# Prune requires disabling this (see README).
#
# Credentials come from /data/.htpasswd — see README for how to populate
# it. That file is mounted read-only into the container.
#
# All tunables live in .env — edit that, not this file.
services:
rest-server:
image: restic/rest-server:${REST_SERVER_VERSION}
container_name: rest-server
restart: unless-stopped
ports:
- "${REST_PORT}:8000"
volumes:
- ${DATA_DIR}:/data
environment:
- OPTIONS=--private-repos --append-only --prometheus ${EXTRA_OPTIONS:-}
- TZ=${TZ:-America/Los_Angeles}
# rest-server stores repos under /data and looks for /data/.htpasswd
# automatically — no extra bind mount needed as long as the htpasswd
# file is created inside DATA_DIR before startup.
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://localhost:8000/metrics >/dev/null || exit 1"]
interval: 30s
timeout: 10s
retries: 3
start_period: 15s
stacks/vllm-qwen3/.env.example
+40
View File
@@ -0,0 +1,40 @@
# vllm-qwen3 stack tunables. Copy this to `.env` on the server before deploying.
#
# cp .env.example .env
# # edit .env with real values
# docker compose up -d
# Image version — pin for reproducibility (`latest` for edge)
VLLM_VERSION=latest
# Host ports (container always listens on 8000 internally)
EMBED_PORT=8001
RERANK_PORT=8002
# GPU assignment — both services share this GPU
# (ana-ml2 has 0 and 1; default 1 keeps 0 free for heavy LLM work)
GPU_ID=1
# Models — reference by full repo name in API requests
EMBED_MODEL=Qwen/Qwen3-Embedding-0.6B
RERANK_MODEL=Qwen/Qwen3-Reranker-0.6B
# GPU memory split — fractions are of TOTAL GPU memory, not free memory.
# When two vLLM services share a GPU, each profiler needs its own slice to
# fit both the model and KV cache, so small values cause the second-to-start
# service to OOM on KV cache allocation. 0.40 + 0.40 leaves ~20% headroom
# and is comfortably above the minimum for two 0.6B Qwen3 models at 8k ctx.
EMBED_GPU_MEM_UTIL=0.40
RERANK_GPU_MEM_UTIL=0.40
# Context length caps — lower these if VRAM is tight.
# Qwen3-Embedding supports up to 32k; reranker up to 32k.
EMBED_MAX_MODEL_LEN=8192
RERANK_MAX_MODEL_LEN=8192
# Optional API key — leave blank for no auth (fine on the internal network).
# If set, both services require `Authorization: Bearer <key>`.
API_KEY=
# HuggingFace token — only needed for gated models
HF_TOKEN=
stacks/vllm-qwen3/README.md
+80
View File
@@ -0,0 +1,80 @@
# vllm-qwen3
Qwen3 embedding + reranker served via vLLM. Replaces the unmaintained Infinity stack.
**Server:** ana-ml2
**Ports:** `8001` (embed), `8002` (rerank) — both configurable via `.env`
**GPU:** both services share GPU 1 by default (configurable)
## Why two services
vLLM runs **one model per process**, so embedding and reranking each get their own container. Both pin to the same GPU and split VRAM via `--gpu-memory-utilization`. Both use `--runner pooling` so the OpenAI server exposes `/v1/embeddings` (for the embedder) and `/rerank`, `/score` (for the reranker, which also needs the `--hf-overrides` described below).
## Reranker caveat
Qwen/Qwen3-Reranker-0.6B is a causal-LM checkpoint. The `--hf-overrides` flag in `compose.yaml` re-maps it to `Qwen3ForSequenceClassification` so vLLM's `/rerank` and `/score` endpoints work and the model emits only `no`/`yes` class logits instead of the full 151k-token distribution.
If that override breaks after a vLLM upgrade, the pre-converted checkpoint `tomaarsen/Qwen3-Reranker-0.6B-seq-cls` is a drop-in replacement that needs no overrides — set `RERANK_MODEL=tomaarsen/Qwen3-Reranker-0.6B-seq-cls` in `.env` and remove the `--hf-overrides` line from the compose.
## Deploy
```bash
# On ana-ml2:
sudo mkdir -p /opt/docker/compose/vllm-qwen3
sudo chown $USER /opt/docker/compose/vllm-qwen3
cd /opt/docker/compose/vllm-qwen3
# Copy compose.yaml + .env.example here (e.g. via scp from this workspace)
cp .env.example .env
# edit .env — pick GPU, ports, memory split, etc.
# Pre-download models (optional, speeds first boot)
HF_HOME=/tank/aimodels/huggingface hf download "$(grep ^EMBED_MODEL .env | cut -d= -f2)"
HF_HOME=/tank/aimodels/huggingface hf download "$(grep ^RERANK_MODEL .env | cut -d= -f2)"
# Dry-parse
docker compose config
# Launch
docker compose up -d
docker compose logs -f
```
First boot compiles CUDA graphs and can take 2–3 minutes per service. The `start_period: 180s` healthcheck grace reflects that.
## Verify
```bash
# Health
curl -s http://localhost:8001/health
curl -s http://localhost:8002/health
# Embedding (OpenAI-compatible)
curl -s http://localhost:8001/v1/embeddings \
-H "Content-Type: application/json" \
-d '{"model":"Qwen/Qwen3-Embedding-0.6B","input":["hello world"]}' | jq .
# Reranker
curl -s http://localhost:8002/rerank \
-H "Content-Type: application/json" \
-d '{"model":"Qwen/Qwen3-Reranker-0.6B","query":"what is a cat","documents":["cats are mammals","dogs bark"]}' | jq .
# Listed models
curl -s http://localhost:8001/v1/models | jq .
curl -s http://localhost:8002/v1/models | jq .
```
## Scaling knobs
- **`EMBED_GPU_MEM_UTIL` / `RERANK_GPU_MEM_UTIL`** — fractions of **total** GPU VRAM each service reserves (not of free VRAM). Both services profile independently, so each slice must be large enough to fit that service's model + KV cache with no knowledge of the other. Setting them too low causes the second-to-start container to OOM on KV cache allocation with `Available KV cache memory: -X.XX GiB`. Default 0.40/0.40 (= 0.80 total) leaves ~20% GPU headroom and works cleanly for the 0.6B pair at 8k context; raise for 4B/8B variants or drop `max-model-len` if you need more room.
- **`EMBED_MAX_MODEL_LEN` / `RERANK_MAX_MODEL_LEN`** — lower to reduce KV-cache allocation if VRAM is tight. Qwen3 supports up to 32k natively.
- **Larger models** — Qwen3-Embedding/Reranker come in 0.6B / 4B / 8B. Swap `EMBED_MODEL` / `RERANK_MODEL` and bump the memory fractions accordingly.
- **Separate GPUs** — if contention hurts latency, split them: add a second `GPU_ID_RERANK` variable and point each service at its own device. (Requires a small compose edit; currently both share `${GPU_ID}`.)
## Migrating off Infinity
Once this stack is verified stable:
1. Stop the infinity stack (`docker compose down` under `/opt/docker/compose/infinity/`).
2. Update consumers (AIPA agents, LibreChat RAG) to point at `:8001` for embeddings and `:8002` for rerank.
3. Delete `stacks/infinity/` from this workspace.
stacks/vllm-qwen3/compose.yaml
+133
View File
@@ -0,0 +1,133 @@
# vLLM — Qwen3 Embedding + Reranker (one stack, two services).
#
# Replaces the unmaintained Infinity stack. vLLM runs one model per process,
# so this stack brings up two containers sharing a single GPU:
#
# vllm-embed — Qwen3-Embedding served as an OpenAI /v1/embeddings server
# vllm-rerank — Qwen3-Reranker served as a /rerank + /score server
#
# The reranker is a causal-LM checkpoint; --hf-overrides re-maps it to
# Qwen3ForSequenceClassification so vLLM's reranking endpoints work and the
# model only emits two class logits (no/yes) instead of the full 151k vocab.
#
# All tunables live in .env — edit that, not this file.
#
# Pre-download models to avoid first-run delay:
# HF_HOME=/tank/aimodels/huggingface hf download Qwen/Qwen3-Embedding-0.6B
# HF_HOME=/tank/aimodels/huggingface hf download Qwen/Qwen3-Reranker-0.6B
services:
vllm-embed:
image: vllm/vllm-openai:${VLLM_VERSION}
container_name: vllm-embed
restart: unless-stopped
ipc: host
ports:
- "${EMBED_PORT}:8000"
volumes:
- /tank/aimodels/huggingface:/hfcache
environment:
- HF_HOME=/hfcache
- HF_HUB_CACHE=/hfcache/hub
- HUGGING_FACE_HUB_TOKEN=${HF_TOKEN:-}
- VLLM_API_KEY=${API_KEY:-}
command:
- ${EMBED_MODEL}
- --served-model-name
- ${EMBED_MODEL}
- --runner
- pooling
- --host
- 0.0.0.0
- --port
- "8000"
- --gpu-memory-utilization
- ${EMBED_GPU_MEM_UTIL}
- --max-model-len
- ${EMBED_MAX_MODEL_LEN}
- --dtype
- auto
deploy:
resources:
reservations:
devices:
- driver: nvidia
device_ids:
- "${GPU_ID}"
capabilities:
- gpu
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 180s
networks:
- tnet
labels:
- homepage.group=AI Systems
- homepage.name=vLLM Embed (Qwen3)
- homepage.icon=mdi-vector-arrange-below
- homepage.description=Qwen3 Embedding via vLLM (ana-ml2)
- homepage.href=http://10.250.50.54:${EMBED_PORT}/docs
vllm-rerank:
image: vllm/vllm-openai:${VLLM_VERSION}
container_name: vllm-rerank
restart: unless-stopped
ipc: host
ports:
- "${RERANK_PORT}:8000"
volumes:
- /tank/aimodels/huggingface:/hfcache
environment:
- HF_HOME=/hfcache
- HF_HUB_CACHE=/hfcache/hub
- HUGGING_FACE_HUB_TOKEN=${HF_TOKEN:-}
- VLLM_API_KEY=${API_KEY:-}
command:
- ${RERANK_MODEL}
- --served-model-name
- ${RERANK_MODEL}
- --runner
- pooling
- --hf-overrides
- '{"architectures":["Qwen3ForSequenceClassification"],"classifier_from_token":["no","yes"],"is_original_qwen3_reranker":true}'
- --host
- 0.0.0.0
- --port
- "8000"
- --gpu-memory-utilization
- ${RERANK_GPU_MEM_UTIL}
- --max-model-len
- ${RERANK_MAX_MODEL_LEN}
- --dtype
- auto
deploy:
resources:
reservations:
devices:
- driver: nvidia
device_ids:
- "${GPU_ID}"
capabilities:
- gpu
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 180s
networks:
- tnet
labels:
- homepage.group=AI Systems
- homepage.name=vLLM Rerank (Qwen3)
- homepage.icon=mdi-sort-variant
- homepage.description=Qwen3 Reranker via vLLM (ana-ml2)
- homepage.href=http://10.250.50.54:${RERANK_PORT}/docs
networks:
tnet:
name: traefik-net
external: true