feat(nh3-dev): migrate to docker-ce 29.8 + compose plugin; drop compose v1

Operator cleared the swap and ruled out a docker-compose v1 shim. Ran
playbooks/upgrade-docker-ce.yaml: docker.io 20.10.24 -> docker-ce 29.8.0,
docker-compose 1.29.2 -> compose plugin v5.5.1, containerd 1.6.20 ->
containerd.io 2.3.5, buildx v0.37.1 added. 12 changed, 0 failed, verify
4/4. talk and beszel-agent back healthy on their restart policies.

The pre-state was worse than 'old': there was no cli-plugins directory, so
'docker compose' was not a command and exited 0 on a help blurb — a silent
no-op that reads as a successful deploy.

Records two things the run surfaced. vastblue-u5-pg and its anonymous
volume were removed when the old daemon stopped; the playbook has no rm,
prune or purge and five other containers survived, so the cause is almost
certainly --rm, unprovable now that the record is gone. It was measured
beforehand as zero user tables in every database, so nothing was lost. And
the playbook's restart loop runs as infra-ops and cannot read a root-owned
0600 stack .env, so it false-FAILs that stack.

Also notes that nh3-dev is the only host where /opt/docker/compose is
root-owned; the other four are lkraven. Created /opt/docker/compose/talk
as lkraven so tts-dev can move talk out of ~/talk. Normalising the parent
is left to the operator.
This commit is contained in:
vh
2026-09-14 12:46:12 -07:00
parent 25a7d05f51
commit 92a4114b90
+31
View File
@@ -16,6 +16,37 @@ local Bash already executes here — no SSH-to-self needed for non-privileged wo
(dante, ACL'd to the WG net). Residential egress for colo services gated on their (dante, ACL'd to the WG net). Residential egress for colo services gated on their
datacenter IP (e.g. YouTube bot-gate). Runbook + setup committed; consumers point datacenter IP (e.g. YouTube bot-gate). Runbook + setup committed; consumers point
`*_PROXY` at it. `*_PROXY` at it.
- **Docker runtime — `docker-ce` since 2026-09-14.** Was Debian's `docker.io`
20.10.24 + the Python `docker-compose` 1.29.2 v1 CLI + `containerd` 1.6.20,
with **no `cli-plugins` directory at all** — so `docker compose` (space) was
not a command: it printed a help blurb and **exited 0**, which a deploy script
cannot distinguish from success. Migrated via
`playbooks/upgrade-docker-ce.yaml` to docker-ce 29.8.0 / compose plugin v5.5.1
/ containerd.io 2.3.5 / buildx v0.37.1. **The v1 `docker-compose` (hyphen)
binary is gone and no shim was installed** (operator ruling 2026-09-14) — fix
callers, don't paper over them.
- ⚠ `vastblue-u5-pg` (an empty `postgres:16` probe container, `restart: no`,
plain `docker run`, anonymous volume) **and its volume were removed** when
the old daemon stopped. The playbook is not the cause — it has no `rm`,
`prune` or `purge`, and the other five containers survived, two of them
long-exited. Almost certainly `--rm` / `AutoRemove=true`, unprovable after
the fact because the container record is gone. Measured beforehand as **zero
user tables in every database**, so no data was lost. Lesson: capture
`AutoRemove` and `RestartPolicy` together when snapshotting a container you
are about to bounce.
- ⚠ The playbook's restart loop runs as `infra-ops` and cannot read a
root-owned `0600` stack `.env` (`/opt/docker/compose/beszel/.env`), so it
reports that stack as FAILED even when `restart: unless-stopped` brings it
back fine. Playbook-side fix pending.
- **`/opt/docker/compose` ownership — nh3-dev is the fleet outlier.**
`root:root` here; `lkraven:lkraven` on irv-ml1, nh3-docker, ana-docker and
esh-docker-vm. So "can a project session deploy its own stack" is false only
on the box where sessions actually run. `/opt/docker/compose/talk` was created
`lkraven`-owned 2026-09-14 so tts-dev can migrate `talk` out of `~/talk`
(a convention violation that hides it from anything walking
`/opt/docker/compose/*/`). Normalising the parent is **unresolved — operator's
call**; `/opt/docker` itself is a separate three-way split (`755` root, `777`
root on two hosts, `755` lkraven).
- **ttyd fleet driver-seat** — web/iPad seat into the zellij `Claude` session (ttyd - **ttyd fleet driver-seat** — web/iPad seat into the zellij `Claude` session (ttyd
behind Caddy; OSC52 clipboard shim). User systemd services under `~/.config`. behind Caddy; OSC52 clipboard shim). User systemd services under `~/.config`.
- **mead-hall** — Bifrost tool-provider sidecar (`:5173`), CI-deployed from - **mead-hall** — Bifrost tool-provider sidecar (`:5173`), CI-deployed from