runbook(park 100): results — 15 patched, parks and deferrals; memory

2026-10-05 03:55:58 -07:00
parent 49c90f3083
commit 241b14bc66
2 changed files with 28 additions and 1 deletions
@@ -92,3 +92,29 @@ same window as its Docker storage move (its holds are listed at the bottom).

 When done, or at 0700 at the latest, send infra-ops ONE message with a table: host | pending before → after | holds |
 reboot-required | parked + reason. Keep the pre/post outputs; they are the evidence.
+
+## Result (2026-10-05 0355, infra-hermes + infra-ops)
+
+**Patched clean (15):** ana-filebot, nh3-extdev, nh3-albok (already current), esh-matter, nh3-ml1, esh-ml1, corviduo-dev,
+pbs-nh3, pbs-ana, the five hypervisors (already current), esh-docker-vm (dead Coral repo commented out), ana-docker,
+nh3-docker, nh3-dev, vm-esh-nas (engine 28 → 29, NFS mounts intact), esh-vm-db (current bookworm despite lists stale
+since 2024), plus fv-ml1 (infra-ops, in the Docker-move window). Pre/post checks matched everywhere. No reboots.
+User-facing impact: two Gitea 403 blips (0310–0314, 0342) while ana-docker's apt loaded the host. Cause: the CrowdSec
+bouncer failing closed on slow LAPI answers. Self-cleared.
+
+**Parked for Prime:**
+- pfi-pteradactyl: already broken before the run (apache2 failed, panel down). 164 pending.
+- pfi-tacticalrmm: certbot renewal failed 2026-10-04 1831 (web + MeshCentral fine). 164 pending.
+- pfi-gx10: wants a `dgx-release` 7.5 → 7.6 upgrade, a new bootloader package (`nvidia-spark-grub-kho`) and kernel
+  7.0.0-1019-nvidia. 212 pending.
+- ana-nas (NFS SPOF) and irv-ml1 (Prime: off-limits): out of scope.
+
+**Deferred to a daytime window:**
+- reboots to activate new kernels: ana-docker, nh3-dev, nh3-extdev, corviduo-dev, vm-esh-nas, esh-vm-db. None
+  builds NVIDIA with DKMS.
+- Docker engine bumps held on ana-docker, nh3-docker, esh-docker-vm, corviduo-dev, nh3-dev, esh-ml1, nh3-ml1, pfi-gx10.
+- fv-ml1: ZFS 2.3.9, nvidia-container-toolkit 1.20, and a kernel with an NVIDIA driver that builds on it.
+- CrowdSec bouncer `updateMaxFailure` (0 = block on the first failed LAPI pull): infra-ops leans toward ~3.
+
+**Lesson:** four hosts were first parked as "no NOPASSWD sudo" because the ssh aliases log in as lkraven. As
+infra-ops@<ip> all four had it (auto-memory `feedback_ssh_alias_defaults_to_lkraven`).