diff --git a/servers/nh3-dev/README.md b/servers/nh3-dev/README.md index d2a059c..22ba73b 100644 --- a/servers/nh3-dev/README.md +++ b/servers/nh3-dev/README.md @@ -16,6 +16,37 @@ local Bash already executes here — no SSH-to-self needed for non-privileged wo (dante, ACL'd to the WG net). Residential egress for colo services gated on their datacenter IP (e.g. YouTube bot-gate). Runbook + setup committed; consumers point `*_PROXY` at it. +- **Docker runtime — `docker-ce` since 2026-09-14.** Was Debian's `docker.io` + 20.10.24 + the Python `docker-compose` 1.29.2 v1 CLI + `containerd` 1.6.20, + with **no `cli-plugins` directory at all** — so `docker compose` (space) was + not a command: it printed a help blurb and **exited 0**, which a deploy script + cannot distinguish from success. Migrated via + `playbooks/upgrade-docker-ce.yaml` to docker-ce 29.8.0 / compose plugin v5.5.1 + / containerd.io 2.3.5 / buildx v0.37.1. **The v1 `docker-compose` (hyphen) + binary is gone and no shim was installed** (operator ruling 2026-09-14) — fix + callers, don't paper over them. + - ⚠ `vastblue-u5-pg` (an empty `postgres:16` probe container, `restart: no`, + plain `docker run`, anonymous volume) **and its volume were removed** when + the old daemon stopped. The playbook is not the cause — it has no `rm`, + `prune` or `purge`, and the other five containers survived, two of them + long-exited. Almost certainly `--rm` / `AutoRemove=true`, unprovable after + the fact because the container record is gone. Measured beforehand as **zero + user tables in every database**, so no data was lost. Lesson: capture + `AutoRemove` and `RestartPolicy` together when snapshotting a container you + are about to bounce. + - ⚠ The playbook's restart loop runs as `infra-ops` and cannot read a + root-owned `0600` stack `.env` (`/opt/docker/compose/beszel/.env`), so it + reports that stack as FAILED even when `restart: unless-stopped` brings it + back fine. Playbook-side fix pending. +- **`/opt/docker/compose` ownership — nh3-dev is the fleet outlier.** + `root:root` here; `lkraven:lkraven` on irv-ml1, nh3-docker, ana-docker and + esh-docker-vm. So "can a project session deploy its own stack" is false only + on the box where sessions actually run. `/opt/docker/compose/talk` was created + `lkraven`-owned 2026-09-14 so tts-dev can migrate `talk` out of `~/talk` + (a convention violation that hides it from anything walking + `/opt/docker/compose/*/`). Normalising the parent is **unresolved — operator's + call**; `/opt/docker` itself is a separate three-way split (`755` root, `777` + root on two hosts, `755` lkraven). - **ttyd fleet driver-seat** — web/iPad seat into the zellij `Claude` session (ttyd behind Caddy; OSC52 clipboard shim). User systemd services under `~/.config`. - **mead-hall** — Bifrost tool-provider sidecar (`:5173`), CI-deployed from