vh
65dc586497
- Beszel: NVIDIA agent (stacks/beszel hosts/esh-ml1.yaml), hub system registered with Status/Disk/CPU/Memory/Temperature alerts; GPU samples verified (RTX 2000E Ada util, VRAM, power, temp). - Uptime Kuma: health monitors for the TEI embed (:8001) and rerank (:8013) services — the sole backends behind the gateway, so the one exception to "seats are out of Kuma's lane". Reward seat excluded (no consumer). - Homepage: dockerd on tcp/2375 bound to 10.0.50.80 (playbook step); esh-ml1-docker added to docker.yaml. - Dozzle: agent v10.4.1 on esh-ml1. The hub's DOZZLE_REMOTE_AGENT still named ana-ml2's 10.250.50.54 and irv-ml1's retired 10.100.79.3; repointed to 10.251.50.54 / 10.6.110.50 (fv-ml1 and irv-ml1 logs were missing). - Documented nh3-dev's Beszel agent failing on NH3 cold start (bind of an automounted NAS share); revived by hand, fix still open.
86 lines
3.8 KiB
Markdown
86 lines
3.8 KiB
Markdown
# dozzle
|
|
|
|
Container log viewer. One UI on **ana-docker** aggregates logs from every Docker host via remote agents.
|
|
|
|
**Deploys to:**
|
|
- **ana-docker** (hub) — UI at `http://10.250.50.70:8088`; live compose dir `/opt/docker/compose/dozzle-hub`
|
|
- **fv-ml1** (agent) — listens on `10.251.50.54:7007`
|
|
- **esh-docker-vm** / **vm-esh-nas** (agents) — `10.0.50.45:7007`, `10.0.50.154:7007`
|
|
- **irv-ml1** (agent) — `10.6.110.50:7007` (mesh address)
|
|
- **esh-ml1** (agent, added 2026-09-25) — `10.0.50.80:7007`, pinned `v10.4.1` = the hub's version; compose dir `dozzle-agent`. The host needed an (empty) `traefik-net` network because the compose declares it external.
|
|
- **nh3-docker** (agent, cross-site) — `10.100.50.40:7007`. ⚠ **Stopped by hand ~2026-04 (Exited 0) and left that way**; the hub logs a refused connection for it. Revive it or drop it from the list deliberately.
|
|
|
|
⚠ **The hub's agent list silently rots when a host moves.** Until 2026-09-25 it
|
|
still named ana-ml2's `10.250.50.54` and irv-ml1's retired `10.100.79.3`, so fv-ml1
|
|
and irv-ml1 logs had been missing from Dozzle since their moves (2026-09-12 and
|
|
2026-09-06). Nothing reported it. After any re-IP, check the hub's
|
|
`DOZZLE_REMOTE_AGENT`, and check that `docker logs dozzle` shows `"clients":N` equal to
|
|
reachable agents + 1.
|
|
- **corviduo-dev** (agent) — listens on `10.250.50.152:7007`. Compose at `/home/vh/docker/compose/dozzle-agent/` (not `/opt/docker/compose/` — see `servers/corviduo-dev/README.md` for why)
|
|
|
|
One compose.yaml lives on each host. The per-host `.env` sets `COMPOSE_PROFILES=hub` or `COMPOSE_PROFILES=agent` so `docker compose up -d` brings up the right service. On the hub, add every agent to `DOZZLE_REMOTE_AGENT` as a comma-separated list (e.g. `10.251.50.54:7007,10.100.50.40:7007`).
|
|
|
|
## Auth / TLS note
|
|
|
|
Dozzle agents and hub auto-generate mTLS certificates on first run. On the trusted LAN (10.250.0.0/16) the default config is fine. If you ever expose an agent beyond the LAN, generate and pin certificates explicitly per the Dozzle docs (`dozzle generate`). The web UI itself is unauthenticated by default — flip `DOZZLE_AUTH_PROVIDER=simple` and set `DOZZLE_USERNAME`/`DOZZLE_PASSWORD` in the hub `.env` if you want a login gate.
|
|
|
|
## Deploy — hub (ana-docker)
|
|
|
|
```bash
|
|
ssh ana-docker
|
|
sudo mkdir -p /opt/docker/compose/dozzle
|
|
sudo chown $USER /opt/docker/compose/dozzle
|
|
cd /opt/docker/compose/dozzle
|
|
|
|
# scp compose.yaml + .env.example from this workspace, then:
|
|
cp .env.example .env
|
|
# Ensure:
|
|
# COMPOSE_PROFILES=hub
|
|
# DOZZLE_HOSTNAME=ana-docker
|
|
# DOZZLE_REMOTE_AGENT=10.251.50.54:7007
|
|
# DOZZLE_PORT=8088
|
|
|
|
docker compose config
|
|
docker compose up -d
|
|
docker compose logs -f
|
|
```
|
|
|
|
## Deploy — agent (fv-ml1)
|
|
|
|
```bash
|
|
ssh fv-ml1
|
|
sudo mkdir -p /opt/docker/compose/dozzle
|
|
sudo chown $USER /opt/docker/compose/dozzle
|
|
cd /opt/docker/compose/dozzle
|
|
|
|
# scp the same compose.yaml + .env.example, then:
|
|
cp .env.example .env
|
|
# Edit to:
|
|
# COMPOSE_PROFILES=agent
|
|
# DOZZLE_HOSTNAME=fv-ml1
|
|
# DOZZLE_AGENT_PORT=7007
|
|
|
|
docker compose config
|
|
docker compose up -d
|
|
docker compose logs -f
|
|
```
|
|
|
|
## Verify
|
|
|
|
```bash
|
|
# Hub health (from anywhere on LAN)
|
|
curl -s http://10.250.50.70:8088/healthz
|
|
|
|
# Agent reachable from the hub's perspective
|
|
ssh ana-docker 'nc -zv 10.251.50.54 7007'
|
|
|
|
# Open http://10.250.50.70:8088 — you should see two tabs:
|
|
# "ana-docker" (local containers) and "fv-ml1" (via agent).
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
- **Hub shows only local containers:** agent is unreachable. Check firewall rules on fv-ml1 (port 7007 must be open from 10.250.50.70) and that the agent is actually listening (`ss -tlnp | grep 7007`).
|
|
- **Agent keeps restarting:** verify the docker.sock bind mount is read-only and the socket exists.
|
|
- **Certificate mismatch after image upgrade:** delete the `dozzle_data` and `dozzle_agent_data` volumes on both hosts and redeploy to regenerate.
|