playbooks/upgrade-docker-ce: bake in the override.conf rewrite from nh3-docker debug

After nh3-docker's swap, two systemd unit gotchas surfaced that the
playbook now handles automatically:

* The docker.io-era /etc/systemd/system/docker.service.d/override.conf
  hardcoded ExecStart=/usr/sbin/dockerd; docker-ce installs at
  /usr/bin/dockerd → daemon failed status=203/EXEC.
* The shipped docker-ce unit's ExecStart=dockerd -H fd:// conflicts
  with daemon.json hosts: (defined for the 0.0.0.0:2375 homepage
  discovery binding) → "conflicting host options".

The "Rewrite docker.service drop-in" step now backs up any existing
override, probes daemon.json for a hosts: setting, and installs an
override that strips -H from ExecStart when needed. Also added an
explicit systemctl reset-failed step to clear the start-rate-limit
state that 3 failed install-time starts leave behind.

configs/homepage/docker.yaml: comment out irv-ml1-docker provider —
20s-per-poll ETIMEDOUTs from the stalled host were drowning homepage's
logs and apparently blocking ana-pfi-docker discovery (the Miniflux
card in the News group wouldn't render until removal). Re-enable when
irv-ml1 is back.

STATUS.md: new "Active migration" section tracking the docker-ce
rollout — nh3-docker done; esh-docker-vm + ana-docker queued.
This commit is contained in:
vh
2026-04-26 14:12:06 -07:00
parent a3ab1a7b6c
commit be9109bc95
3 changed files with 91 additions and 17 deletions
+48 -14
View File
@@ -126,29 +126,63 @@ steps:
shell: DEBIAN_FRONTEND=noninteractive apt-get install -y {{ pkgs_to_install }}
sudo: true
- name: Move aside any stale docker.service drop-in override
# The docker.io era often left an /etc/systemd/system/docker.service.d/
# override.conf adding `-H tcp://...` to ExecStart for remote docker
# discovery. After the swap, the override (a) hard-codes
# /usr/sbin/dockerd which no longer exists, and (b) duplicates the
# `hosts:` setting in /etc/docker/daemon.json — dockerd refuses to
# start when both are set ("conflicting host options"). daemon.json
# is the modern way; the override is redundant.
- name: Rewrite docker.service drop-in to match the new package layout
# Two related fixes for the systemd unit, learned the hard way on
# nh3-docker. The docker.io era left an
# /etc/systemd/system/docker.service.d/override.conf hardcoding
# ExecStart=/usr/sbin/dockerd (Debian package path) and usually
# adding `-H tcp://...` for remote discovery.
#
# After the swap to docker-ce two things are wrong:
# 1. /usr/sbin/dockerd doesn't exist — docker-ce installs at
# /usr/bin/dockerd. Daemon fails status=203/EXEC.
# 2. If daemon.json defines `hosts:` (typical here for the
# 0.0.0.0:2375 homepage-discovery binding), then ANY `-H`
# flag in ExecStart conflicts: dockerd refuses to start with
# "conflicting host options".
#
# The shipped docker-ce unit's ExecStart is `dockerd -H fd://
# --containerd=/run/containerd/containerd.sock` — that `-H fd://`
# ALSO conflicts with daemon.json hosts:. So we always need an
# override that strips `-H` entirely when daemon.json defines
# hosts.
#
# Procedure:
# * Back up any existing override (preserves it as .pre-upgrade)
# * Probe /etc/docker/daemon.json for a `hosts:` setting
# * If hosts: is defined → install an override that clears the
# base unit's ExecStart and replaces it WITHOUT -H
# * Otherwise no override is needed (base unit's `-H fd://` is fine)
shell: |
f=/etc/systemd/system/docker.service.d/override.conf
d=/etc/systemd/system/docker.service.d
f="$d/override.conf"
sudo install -d -m 0755 "$d"
if [ -f "$f" ]; then
echo " moving aside $f"
echo " backing up existing $f → ${f}.pre-upgrade"
sudo mv "$f" "${f}.pre-upgrade"
fi
if [ -f /etc/docker/daemon.json ] && grep -q '"hosts"' /etc/docker/daemon.json; then
echo " daemon.json has hosts: — installing override that strips -H from ExecStart"
sudo tee "$f" >/dev/null <<'OVR'
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd --containerd=/run/containerd/containerd.sock
OVR
else
echo " daemon.json has no hosts: — base unit's ExecStart is fine, no override needed"
fi
sudo: true
- name: systemctl daemon-reload (CRITICAL — package swap put dockerd at
/usr/bin/dockerd; without reload, systemd keeps the old unit's
ExecStart pointing at /usr/sbin/dockerd from docker.io and fails
with status=203/EXEC "No such file or directory")
- name: systemctl daemon-reload (so the new unit + override are loaded)
shell: systemctl daemon-reload
sudo: true
- name: Reset the docker.service failed state (3 failed starts during
the install often leaves it in start-rate-limited state — reset
so the next start isn't immediately denied)
shell: systemctl reset-failed docker.service
sudo: true
- name: Enable + start the new daemon (idempotent)
shell: systemctl enable --now docker
sudo: true