playbooks/upgrade-docker-ce: bake in the override.conf rewrite from nh3-docker debug

After nh3-docker's swap, two systemd unit gotchas surfaced that the
playbook now handles automatically:

* The docker.io-era /etc/systemd/system/docker.service.d/override.conf
  hardcoded ExecStart=/usr/sbin/dockerd; docker-ce installs at
  /usr/bin/dockerd → daemon failed status=203/EXEC.
* The shipped docker-ce unit's ExecStart=dockerd -H fd:// conflicts
  with daemon.json hosts: (defined for the 0.0.0.0:2375 homepage
  discovery binding) → "conflicting host options".

The "Rewrite docker.service drop-in" step now backs up any existing
override, probes daemon.json for a hosts: setting, and installs an
override that strips -H from ExecStart when needed. Also added an
explicit systemctl reset-failed step to clear the start-rate-limit
state that 3 failed install-time starts leave behind.

configs/homepage/docker.yaml: comment out irv-ml1-docker provider —
20s-per-poll ETIMEDOUTs from the stalled host were drowning homepage's
logs and apparently blocking ana-pfi-docker discovery (the Miniflux
card in the News group wouldn't render until removal). Re-enable when
irv-ml1 is back.

STATUS.md: new "Active migration" section tracking the docker-ce
rollout — nh3-docker done; esh-docker-vm + ana-docker queued.
This commit is contained in:
vh
2026-04-26 14:12:06 -07:00
parent a3ab1a7b6c
commit be9109bc95
3 changed files with 91 additions and 17 deletions
+33
View File
@@ -317,6 +317,39 @@ Once recovered, update this section with what fixed it and remove the
rule: refresh STATUS.md at the end of any session where a 🟧 or 🟥
item closes, or three+ smaller items land.
## Active migration — docker.io 20.10 → docker-ce 29.x
Bookworm's Debian-packaged `docker.io` is pinned at 20.10.24 with
client/daemon API 1.41. Newer compose clients (1.52+) refuse to talk
to it ("client version 1.52 is too new" during `docker compose
build`). The migration moves each host to Docker's official `docker-ce`
APT repo (29.x). Playbook: `playbooks/upgrade-docker-ce.yaml`.
Order (least → most blast radius):
1. ~~**nh3-docker**~~ — **done 2026-04-24.** docker-ce 29.4.1, two
stacks (portainer agent + portainer) restarted clean. Debug surfaced
two unit-file gotchas now baked into the playbook:
- `/etc/systemd/system/docker.service.d/override.conf` from the
docker.io era hardcoded `/usr/sbin/dockerd` (docker-ce installs
at `/usr/bin/dockerd`) → daemon failed status=203/EXEC.
- The shipped docker-ce unit's `ExecStart=dockerd -H fd:// …`
conflicts with `daemon.json` `hosts:` (defined for the 0.0.0.0:2375
homepage-discovery binding) → "conflicting host options".
Playbook now backs up any existing override, probes `daemon.json`
for `hosts:`, and installs an override that strips `-H` when needed.
2. **esh-docker-vm** — pending. Run from your TTY:
`scripts/elway esh-docker-vm --playbook playbooks/upgrade-docker-ce.yaml`
3. **ana-docker** — pending. Largest blast radius (homepage, miniflux,
news-digest, synapse, seafile, vaultwarden, gitea, openwebui, …).
Verify each card / service after.
Also during the nh3-docker debug: `configs/homepage/docker.yaml`
commented out the `irv-ml1-docker` provider — the 20s-per-poll
ETIMEDOUTs from the stalled host were drowning homepage's logs and
appeared to block ana-pfi-docker discovery (News group's Miniflux card
wouldn't render until removal). Re-enable when irv-ml1 is back.
## Session milestones — 2026-04-24 (the "tooling day" + housekeeping pm)
Morning / early afternoon — the original tooling day: