feat(nh3-dev): migrate to docker-ce 29.8 + compose plugin; drop compose v1

Operator cleared the swap and ruled out a docker-compose v1 shim. Ran
playbooks/upgrade-docker-ce.yaml: docker.io 20.10.24 -> docker-ce 29.8.0,
docker-compose 1.29.2 -> compose plugin v5.5.1, containerd 1.6.20 ->
containerd.io 2.3.5, buildx v0.37.1 added. 12 changed, 0 failed, verify
4/4. talk and beszel-agent back healthy on their restart policies.

The pre-state was worse than 'old': there was no cli-plugins directory, so
'docker compose' was not a command and exited 0 on a help blurb — a silent
no-op that reads as a successful deploy.

Records two things the run surfaced. vastblue-u5-pg and its anonymous
volume were removed when the old daemon stopped; the playbook has no rm,
prune or purge and five other containers survived, so the cause is almost
certainly --rm, unprovable now that the record is gone. It was measured
beforehand as zero user tables in every database, so nothing was lost. And
the playbook's restart loop runs as infra-ops and cannot read a root-owned
0600 stack .env, so it false-FAILs that stack.

Also notes that nh3-dev is the only host where /opt/docker/compose is
root-owned; the other four are lkraven. Created /opt/docker/compose/talk
as lkraven so tts-dev can move talk out of ~/talk. Normalising the parent
is left to the operator.
This commit is contained in:
vh
2026-09-14 12:46:12 -07:00
parent 25a7d05f51
commit 92a4114b90
+31
View File
@@ -16,6 +16,37 @@ local Bash already executes here — no SSH-to-self needed for non-privileged wo
(dante, ACL'd to the WG net). Residential egress for colo services gated on their
datacenter IP (e.g. YouTube bot-gate). Runbook + setup committed; consumers point
`*_PROXY` at it.
- **Docker runtime — `docker-ce` since 2026-09-14.** Was Debian's `docker.io`
20.10.24 + the Python `docker-compose` 1.29.2 v1 CLI + `containerd` 1.6.20,
with **no `cli-plugins` directory at all** — so `docker compose` (space) was
not a command: it printed a help blurb and **exited 0**, which a deploy script
cannot distinguish from success. Migrated via
`playbooks/upgrade-docker-ce.yaml` to docker-ce 29.8.0 / compose plugin v5.5.1
/ containerd.io 2.3.5 / buildx v0.37.1. **The v1 `docker-compose` (hyphen)
binary is gone and no shim was installed** (operator ruling 2026-09-14) — fix
callers, don't paper over them.
- ⚠ `vastblue-u5-pg` (an empty `postgres:16` probe container, `restart: no`,
plain `docker run`, anonymous volume) **and its volume were removed** when
the old daemon stopped. The playbook is not the cause — it has no `rm`,
`prune` or `purge`, and the other five containers survived, two of them
long-exited. Almost certainly `--rm` / `AutoRemove=true`, unprovable after
the fact because the container record is gone. Measured beforehand as **zero
user tables in every database**, so no data was lost. Lesson: capture
`AutoRemove` and `RestartPolicy` together when snapshotting a container you
are about to bounce.
- ⚠ The playbook's restart loop runs as `infra-ops` and cannot read a
root-owned `0600` stack `.env` (`/opt/docker/compose/beszel/.env`), so it
reports that stack as FAILED even when `restart: unless-stopped` brings it
back fine. Playbook-side fix pending.
- **`/opt/docker/compose` ownership — nh3-dev is the fleet outlier.**
`root:root` here; `lkraven:lkraven` on irv-ml1, nh3-docker, ana-docker and
esh-docker-vm. So "can a project session deploy its own stack" is false only
on the box where sessions actually run. `/opt/docker/compose/talk` was created
`lkraven`-owned 2026-09-14 so tts-dev can migrate `talk` out of `~/talk`
(a convention violation that hides it from anything walking
`/opt/docker/compose/*/`). Normalising the parent is **unresolved — operator's
call**; `/opt/docker` itself is a separate three-way split (`755` root, `777`
root on two hosts, `755` lkraven).
- **ttyd fleet driver-seat** — web/iPad seat into the zellij `Claude` session (ttyd
behind Caddy; OSC52 clipboard shim). User systemd services under `~/.config`.
- **mead-hall** — Bifrost tool-provider sidecar (`:5173`), CI-deployed from