vh
6f7bb4885e
Decision recorded in CLAUDE.md ("Stack tree convention") and memory
(convention_stacks_vs_mirror.md):
stacks/<stack>/ canonical / intent. git-tracked.
deploy-stack.sh reads from here.
stacks-mirror/<host>/<stack>/ snapshot / reality. gitignored.
sync-stacks.sh writes here. Used
for drift inspection only — never
a deploy source.
Bug this fixes: deploy-stack.sh was reading from the mirror, so edits
to stacks/llama-swap/config.yaml never reached ana-ml2. Today's
two new model entries (qwen3.6-35-a3b-heretic + qwen3.6-27b) lived
in the canonical for hours but the deploy reported "in sync" because
the script only diffed mirror vs server.
Changes:
* deploy-stack.sh: source switched from MIRROR_DIR/$HOST/$STACK to
STACKS_DIR/$STACK. Header comment + error message updated.
* sync-stacks.sh: header explicitly identifies its role as drift
detection; documents the diff command for comparing canonical vs
mirror.
* stacks/llama-swap/{config.yaml → conf/config.yaml}: matches the
deploy mapping (conf/ in canonical → /opt/docker/conf/ on host).
* CLAUDE.md: "Stack mirror (pull / push)" section rewritten as
"Stack tree convention (canonical vs mirror)" with the role table
+ workflow rules + diff recipe. Layout diagram updated.
llama-swap
GGUF model server with on-demand model swapping. Served via llama.cpp's llama-server under the llama-swap proxy.
Server: ana-ml2
Port: 9292 (configurable via .env)
GPU: both (unpinned — runtime: nvidia grants access to all devices; per-model GPU selection happens inside config.yaml)
Files
compose.yaml— canonical compose. Deployed to/opt/docker/compose/llama-swap/compose.yamlon ana-ml2..env.example— template for the per-host.env. Copy to.envon the server and tweak.config.yaml— model definitions and groups. Deployed to/opt/docker/conf/llama-swap/config.yamlon the server.
Homepage labels are in the compose file under the AI Systems group, matching the convention used by vllm-qwen3 and infinity.
Deploy a fresh install
scripts/deploy-stack.sh ana-ml2 llama-swap
ssh ana-ml2 '
cd /opt/docker/compose/llama-swap && \
cp -n .env.example .env && \
docker compose config && \
docker compose up -d && \
docker compose logs --tail=30
'
Model reference conventions
- Modern entries: use
-hf <user>/<repo>[:<quant>]— reads from the shared HF cache, nothing to pre-stage outsidehf download - Legacy entries: use
--model /models/<dir>/<file>.gguf— reads GGUFs from/tank/aimodels/llm/(pre-HF-cache era, gradually being migrated)
New models should prefer the -hf pattern.
Deploy updates to config only
# After editing config.yaml here:
scp config.yaml ana-ml2:/opt/docker/conf/llama-swap/config.yaml
ssh ana-ml2 'cd /opt/docker/compose/llama-swap && docker compose restart'
Deploy updates to compose only
# After editing compose.yaml or .env.example here:
scripts/deploy-stack.sh ana-ml2 llama-swap
ssh ana-ml2 'cd /opt/docker/compose/llama-swap && docker compose up -d'