docs: add FleetTools — an agent-family-agnostic index of fleet capability
Every agent on nh3-dev — Claude, Codex, Grok, Aider — needs the same answers: what runs here, how do I call it, what will bite me. Until now that lived in ~/.claude/CLAUDE.md, which only Claude sessions load, and it was interleaved with operator preferences that other families have no use for. Two-tier by design, matching the persistent-memory split: FLEETTOOLS.md is a 135-line index an agent reads whole, and each entry links to a detail file it opens only when it actually needs that tool. Reading the index costs about a fifth of reading the tree. Detail paths are absolute so they resolve from any working directory, since a non-Claude agent will cat the path rather than follow a markdown link. ~/FLEETTOOLS.md symlinks to the index for discovery. Rule zero is that live inventories get queried, not transcribed: Homepage /api/services, asset-engine /api/v1/services, LiteLLM /v1/models, and every FastAPI seat's /openapi.json. A copied service table would be stale within a month and this repo already has a standing rule against second copies that drift. Contents verified against the running fleet rather than copied from existing docs: binaries resolved on PATH, seven endpoints probed live, the LiteLLM roster counted at 40 models where the old note said ~30. No credentials are included; the vault and its CLI are pointed at instead.
This commit is contained in:
@@ -0,0 +1,32 @@
|
||||
# Observability — where to look when something is wrong
|
||||
|
||||
| surface | URL | what it answers |
|
||||
|---|---|---|
|
||||
| **Homepage** | `http://10.0.50.45:5100` | the curated fleet service map; `/api/services` is machine-readable |
|
||||
| **Beszel** | `http://10.250.50.70:8090` | host + container up/down, resource history; alerts land in althing |
|
||||
| **Dozzle** | `http://10.250.50.70:8088` | container logs, ana-docker + ana-ml2 |
|
||||
| **Uptime Kuma** | `http://10.0.50.45:3001` | ESH-side service monitoring |
|
||||
| **Backrest** | `http://10.250.50.70:9898` | restic snapshot browse / restore |
|
||||
| **task-board** | `http://10.250.50.70:7878` | what an assistant session is working on right now |
|
||||
| Dockge (per host) | `<host>:5001` | compose UI on ana-docker, ana-ml2, esh-docker-vm, irv-ml1, nh3-docker |
|
||||
| PBS | `10.250.50.90:8007` (primary) · `10.100.50.90:8007` (DR) | VM/LXC image backups |
|
||||
|
||||
**Homepage's `/api/services` is the single best machine-readable inventory of the
|
||||
fleet** and it is maintained as a side effect of normal work. Query it rather than
|
||||
keeping a list.
|
||||
|
||||
## Diagnostic reflexes
|
||||
|
||||
- **All hosts at one site flapping together = a WAN/mesh event, not host failure.**
|
||||
Check uptimes first; they will show weeks. Then read the site's subnet-router
|
||||
tailscaled log.
|
||||
- **ana-nas is a SPOF** for postgres, rest-server-ana, PBS-ANA and cross-site restic.
|
||||
Check it first on any multi-failure.
|
||||
- **After a power loss, crowdsec fails closed**: it exits 255 and the Traefik bouncer
|
||||
403s ALL HTTP. Restart crowdsec THEN traefik. This is a recurring recovery step.
|
||||
- **A filter can silently narrow your window.** `docker logs --since HH:MM` returning 0
|
||||
while the line demonstrably exists, a `find -maxdepth` undercount, a character class
|
||||
that omits digits — re-run unfiltered before concluding absence. **Absence of a
|
||||
signal is not a safe reading of it.**
|
||||
- **Timestamp every state report.** An untimestamped claim is a claim about "now" and
|
||||
has manufactured phantom events in peer artifacts before.
|
||||
Reference in New Issue
Block a user