stacks/ is canonical; stacks-mirror/ is drift snapshot — stop confusing the two

Decision recorded in CLAUDE.md ("Stack tree convention") and memory
(convention_stacks_vs_mirror.md):

  stacks/<stack>/                    canonical / intent. git-tracked.
                                     deploy-stack.sh reads from here.
  stacks-mirror/<host>/<stack>/      snapshot / reality. gitignored.
                                     sync-stacks.sh writes here. Used
                                     for drift inspection only — never
                                     a deploy source.

Bug this fixes: deploy-stack.sh was reading from the mirror, so edits
to stacks/llama-swap/config.yaml never reached ana-ml2. Today's
two new model entries (qwen3.6-35-a3b-heretic + qwen3.6-27b) lived
in the canonical for hours but the deploy reported "in sync" because
the script only diffed mirror vs server.

Changes:
* deploy-stack.sh: source switched from MIRROR_DIR/$HOST/$STACK to
  STACKS_DIR/$STACK. Header comment + error message updated.
* sync-stacks.sh: header explicitly identifies its role as drift
  detection; documents the diff command for comparing canonical vs
  mirror.
* stacks/llama-swap/{config.yaml → conf/config.yaml}: matches the
  deploy mapping (conf/ in canonical → /opt/docker/conf/ on host).
* CLAUDE.md: "Stack mirror (pull / push)" section rewritten as
  "Stack tree convention (canonical vs mirror)" with the role table
  + workflow rules + diff recipe. Layout diagram updated.
This commit is contained in:
vh
2026-04-26 22:07:30 -07:00
parent b48667f33a
commit 6f7bb4885e
4 changed files with 48 additions and 20 deletions
+28 -12
View File
@@ -140,23 +140,36 @@ scripts/refresh-server-info.sh --validate-only all
scripts/refresh-server-info.sh --validate-only <host> scripts/refresh-server-info.sh --validate-only <host>
``` ```
## Stack mirror (pull / push) ## Stack tree convention (canonical vs mirror)
Compose and config trees are mirrored into `stacks-mirror/<host>/<stack>/` so they can be diffed and version-controlled. Pull is fleet-wide and safe; push is one stack at a time with a diff + prompt. Two trees, distinct roles. **They are NOT interchangeable.**
| tree | role | git | who writes | who reads |
|---|---|---|---|---|
| `stacks/<stack>/` | **canonical / intent** — source of truth for what we want deployed | tracked | you / Claude | `deploy-stack.sh` |
| `stacks-mirror/<host>/<stack>/` | **snapshot / reality** — what's currently on each host | gitignored | `sync-stacks.sh` | drift inspection |
**Why two:** keeps "intent" (committed, reviewable, deployed) cleanly separate from "reality on the server right now" (often drifts, useful to compare, not durable). Editing the mirror does NOT affect what gets deployed.
```bash ```bash
# Pull compose + conf from every host into stacks-mirror/ # Edit the canonical, then push it to the host:
scripts/sync-stacks.sh # stacks/<stack>/<file> → /opt/docker/compose/<stack>/<file>
scripts/sync-stacks.sh --dry-run # see what would change # stacks/<stack>/conf/<file> → /opt/docker/conf/<stack>/<file>
scripts/sync-stacks.sh ana-docker # one host $EDITOR stacks/<stack>/compose.yaml
scripts/deploy-stack.sh <host> <stack> # diffs vs live, prompts y/N
scripts/deploy-stack.sh <host> <stack> --compose # skip conf side
scripts/deploy-stack.sh <host> <stack> --conf # skip compose side
# Push a local stack back to the server (diffs each file, prompts y/N) # Pull current host state into the gitignored snapshot tree (drift check):
scripts/deploy-stack.sh <host> <stack> scripts/sync-stacks.sh # every host
scripts/deploy-stack.sh <host> <stack> --compose # skip conf scripts/sync-stacks.sh ana-docker # one host
scripts/deploy-stack.sh <host> <stack> --conf # skip compose scripts/sync-stacks.sh --dry-run # see what would change
# Compare canonical (intent) vs mirror (reality) for one stack:
diff -ru stacks/<stack>/ stacks-mirror/<host>/<stack>/
``` ```
**Opt-out per stack:** create `stacks-mirror/<host>/<stack>/.no-sync` (skip both sides) or `stacks-mirror/<host>/<stack>/conf/.no-sync` (skip conf only). **Opt-out per stack** (mirror only — sync-stacks.sh skip): create `stacks-mirror/<host>/<stack>/.no-sync` (skip both sides) or `stacks-mirror/<host>/<stack>/conf/.no-sync` (skip conf only).
**Always excluded in both directions** (secrets / runtime state): `.env`, `.env.*`, `acme.json`, `client_secrets.json`, `*.pem`, `*.key`, `*.crt`, `*.pfx`, `*.sqlite`, `*.sqlite3`, `*.db`, `*.log`, `*.log.*`, `*.pid`, `hub/`, `logs/`. **Always excluded in both directions** (secrets / runtime state): `.env`, `.env.*`, `acme.json`, `client_secrets.json`, `*.pem`, `*.key`, `*.crt`, `*.pfx`, `*.sqlite`, `*.sqlite3`, `*.db`, `*.log`, `*.log.*`, `*.pid`, `hub/`, `logs/`.
@@ -174,11 +187,14 @@ eshpfi-management/
│ └── <name>/ │ └── <name>/
│ ├── README.md │ ├── README.md
│ └── system-details.txt # latest server_inspect output │ └── system-details.txt # latest server_inspect output
├── stacks/ ├── stacks/ # canonical/intent — git-tracked source of truth
│ └── <stack>/ │ └── <stack>/
│ ├── compose.yaml # deployed to /opt/docker/compose/<stack>/ │ ├── compose.yaml # deployed to /opt/docker/compose/<stack>/
│ ├── conf/<file> # deployed to /opt/docker/conf/<stack>/<file>
│ ├── .env.example # template; real .env lives on server │ ├── .env.example # template; real .env lives on server
│ └── README.md # what this stack does, how to deploy │ └── README.md # what this stack does, how to deploy
├── stacks-mirror/ # gitignored snapshot of live host state (drift detection)
│ └── <host>/<stack>/ # populated by sync-stacks.sh, NOT a deploy source
└── docs/ └── docs/
└── pfi/ # general PFI infrastructure reference └── pfi/ # general PFI infrastructure reference
``` ```
+12 -7
View File
@@ -1,10 +1,15 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# deploy-stack.sh — push a local stacks-mirror dir to a server, with # deploy-stack.sh — push a canonical stack from `stacks/` to a server,
# per-file diff and confirmation prompt. # with per-file diff and confirmation prompt.
#
# Source of truth is `stacks/<stack>/` (the canonical, git-tracked tree).
# `stacks-mirror/` is a separate, gitignored snapshot of what's currently
# on each host (pulled by sync-stacks.sh) — it's used for drift detection,
# NOT as the deploy source. See CLAUDE.md "Stack tree convention".
# #
# Layout assumed: # Layout assumed:
# stacks-mirror/<host>/<stack>/<file> → <host>:/opt/docker/compose/<stack>/<file> # stacks/<stack>/<file> → <host>:/opt/docker/compose/<stack>/<file>
# stacks-mirror/<host>/<stack>/conf/<file> → <host>:/opt/docker/conf/<stack>/<file> # stacks/<stack>/conf/<file> → <host>:/opt/docker/conf/<stack>/<file>
# #
# Secrets / runtime state are never pushed (same exclude list as # Secrets / runtime state are never pushed (same exclude list as
# sync-stacks.sh): .env*, acme.json, *.key/crt/pem/pfx, *.sqlite*, *.db, # sync-stacks.sh): .env*, acme.json, *.key/crt/pem/pfx, *.sqlite*, *.db,
@@ -32,7 +37,7 @@ fi
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
SERVERS_DIR="$REPO_ROOT/servers" SERVERS_DIR="$REPO_ROOT/servers"
MIRROR_DIR="$REPO_ROOT/stacks-mirror" STACKS_DIR="$REPO_ROOT/stacks"
EXCLUDES=( EXCLUDES=(
# Include .env.example / *.env.example templates before the broader # Include .env.example / *.env.example templates before the broader
@@ -100,9 +105,9 @@ resolve_target() {
} }
TARGET=$(resolve_target "$HOST") TARGET=$(resolve_target "$HOST")
STACK_DIR="$MIRROR_DIR/$HOST/$STACK" STACK_DIR="$STACKS_DIR/$STACK"
[ -d "$STACK_DIR" ] || { echo "error: $STACK_DIR not found (pull with sync-stacks.sh first)" >&2; exit 2; } [ -d "$STACK_DIR" ] || { echo "error: $STACK_DIR not found — author the canonical stack first (see stacks/<other>/ for examples)" >&2; exit 2; }
# Collect the two src/dest pairs we need to consider. # Collect the two src/dest pairs we need to consider.
PAIRS=() # each entry: "<kind>|<src>|<dest>" PAIRS=() # each entry: "<kind>|<src>|<dest>"
+8 -1
View File
@@ -1,6 +1,13 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# sync-stacks.sh — pull /opt/docker/{compose,conf}/<stack>/ from every # sync-stacks.sh — pull /opt/docker/{compose,conf}/<stack>/ from every
# server into version-controlled `stacks-mirror/<host>/<stack>/`. # server into the gitignored `stacks-mirror/<host>/<stack>/` snapshot tree.
#
# Role: drift detection. The mirror reflects what's actually running on
# each host RIGHT NOW. The canonical source of truth lives at
# `stacks/<stack>/` (git-tracked), and `deploy-stack.sh` pushes from
# there. Use the mirror to compare what we have to what's deployed:
# diff -ru stacks/<stack>/ stacks-mirror/<host>/<stack>/
# See CLAUDE.md "Stack tree convention" for the full split.
# #
# Layout (flat per stack): # Layout (flat per stack):
# stacks-mirror/<host>/<stack>/ <- mirrors /opt/docker/compose/<stack>/ # stacks-mirror/<host>/<stack>/ <- mirrors /opt/docker/compose/<stack>/