vh
e376d0aec9
Captures the full workspace state built up to this point:
- CLAUDE.md + README.md describing conventions and the four-host fleet
(ana-ml2, ana-docker, nh3-docker, esh-docker-vm).
- Per-host notes under servers/<host>/ with ssh-target fallback files
and latest system-details snapshots (two in-compose credential leaks
scrubbed; the upstream compose files still need to move those to .env).
- scripts/: server_inspect.sh (read-only remote diagnostic),
refresh-server-info.sh (dir-driven discovery + snapshot capture with
validation warnings), add-host.sh, sync-stacks.sh (pull
compose/conf trees), deploy-stack.sh (push with per-file diff + prompt).
- stacks/: canonical compose for backrest, beszel, dozzle, llama-swap,
rest-server-ana, rest-server-nh3, vllm-qwen3, plus the retired
infinity reference. All use the .env-driven + traefik-net + homepage
label pattern.
- configs/restic/ana-docker/: first resticprofile config + pre-backup
hook (Synapse pg_dump, Seafile mysqldump, Vaultwarden SQLite); templates
for the other three hosts to come.
- docs/pfi/: general infrastructure reference carried over.
- .gitignore excludes .env, stacks-mirror/, and assorted secret/state
filenames to prevent re-leaks on later commits.
31 lines
884 B
Bash
31 lines
884 B
Bash
# Infinity stack tunables. Copy this to `.env` on the server before deploying.
|
|
#
|
|
# cp .env.example .env
|
|
# # edit .env with real values
|
|
# docker compose up -d
|
|
|
|
# Image version — pin for reproducibility (`latest` for edge)
|
|
INFINITY_VERSION=latest
|
|
|
|
# Port exposed on host
|
|
INFINITY_PORT=7997
|
|
|
|
# GPU assignment (ana-ml2 has 0 and 1; default 1 keeps 0 free for heavy LLM work)
|
|
GPU_ID=1
|
|
|
|
# Models — both served simultaneously; reference by the full repo name in requests
|
|
EMBED_MODEL=Qwen/Qwen3-Embedding-0.6B
|
|
RERANK_MODEL=Qwen/Qwen3-Reranker-0.6B
|
|
|
|
# Inference engine: torch (widest support) or optimum (ONNX, sometimes faster)
|
|
ENGINE=torch
|
|
|
|
# Batch size — 32 is a safe default; bump for throughput if VRAM allows
|
|
BATCH_SIZE=32
|
|
|
|
# Optional API key — leave blank for no auth (fine on the internal network)
|
|
API_KEY=
|
|
|
|
# HuggingFace token — only needed for gated models
|
|
HF_TOKEN=
|