feat(esh-ml1): wire telemetry and monitoring; fix Dozzle's stale agent list

- Beszel: NVIDIA agent (stacks/beszel hosts/esh-ml1.yaml), hub system
  registered with Status/Disk/CPU/Memory/Temperature alerts; GPU samples
  verified (RTX 2000E Ada util, VRAM, power, temp).
- Uptime Kuma: health monitors for the TEI embed (:8001) and rerank (:8013)
  services — the sole backends behind the gateway, so the one exception to
  "seats are out of Kuma's lane". Reward seat excluded (no consumer).
- Homepage: dockerd on tcp/2375 bound to 10.0.50.80 (playbook step);
  esh-ml1-docker added to docker.yaml.
- Dozzle: agent v10.4.1 on esh-ml1. The hub's DOZZLE_REMOTE_AGENT still
  named ana-ml2's 10.250.50.54 and irv-ml1's retired 10.100.79.3; repointed
  to 10.251.50.54 / 10.6.110.50 (fv-ml1 and irv-ml1 logs were missing).
- Documented nh3-dev's Beszel agent failing on NH3 cold start (bind of an
  automounted NAS share); revived by hand, fix still open.
This commit is contained in:
vh
2026-09-25 09:20:36 -07:00
parent 52612cbe96
commit 65dc586497
9 changed files with 165 additions and 20 deletions
+10
View File
@@ -23,6 +23,16 @@ filesystem samples verified; fleet 13/14 up with known fv-ml1 outage.
| irv-ml1 | beszel-agent-irv | /worktank, /storetank, /mnt/smithy |
| vm-esh-nas | beszel-agent-esh-nas | /mnt/books, /mnt/share, /mnt/music, /mnt/media |
| nh3-dev | beszel | /mnt/backup, /mnt/smithy |
| esh-ml1 (added 2026-09-25) | beszel | none — NVIDIA image (`hosts/esh-ml1.yaml`) for the RTX 2000E Ada |
⚠ **nh3-dev's agent was DOWN from the 2026-09-24 NH3 power recovery until
2026-09-25.** Docker could not bind `/mnt/smithy` at boot ("no such device"): since
`1cbde50` the NAS shares are automounted, and nh3-nas was not up yet. Docker does not
retry a container that fails to *create*, so `unless-stopped` never brought it back.
Started by hand. **It will recur on the next NH3 cold start** until the agent stops
depending on NAS mounts at boot. The NAS capacity is already reported by
nh3-nas's own agent, so dropping the two extra filesystems here is the simplest fix.
Open item.
Use `infra-ops@<ip>` with passwordless sudo, except vm-esh-nas:
`lkraven@10.0.50.154` has Docker access. Irvine's hub address is