vh
53c3e8000e
Every agent on nh3-dev — Claude, Codex, Grok, Aider — needs the same answers: what runs here, how do I call it, what will bite me. Until now that lived in ~/.claude/CLAUDE.md, which only Claude sessions load, and it was interleaved with operator preferences that other families have no use for. Two-tier by design, matching the persistent-memory split: FLEETTOOLS.md is a 135-line index an agent reads whole, and each entry links to a detail file it opens only when it actually needs that tool. Reading the index costs about a fifth of reading the tree. Detail paths are absolute so they resolve from any working directory, since a non-Claude agent will cat the path rather than follow a markdown link. ~/FLEETTOOLS.md symlinks to the index for discovery. Rule zero is that live inventories get queried, not transcribed: Homepage /api/services, asset-engine /api/v1/services, LiteLLM /v1/models, and every FastAPI seat's /openapi.json. A copied service table would be stale within a month and this repo already has a standing rule against second copies that drift. Contents verified against the running fleet rather than copied from existing docs: binaries resolved on PATH, seven endpoints probed live, the LiteLLM roster counted at 40 models where the old note said ~30. No credentials are included; the vault and its CLI are pointed at instead.
53 lines
2.2 KiB
Markdown
53 lines
2.2 KiB
Markdown
# Fleet SSH — the infra-ops identity and host map
|
|
|
|
## Identity
|
|
|
|
`ssh infra-ops@<host>` — a dedicated NOPASSWD-sudo user on PFI-owned boxes, key
|
|
`~/.ssh/infra-ops_ed25519`. Covers **every** fleet host including all four Proxmox
|
|
hypervisors, corviduo-dev and esh-docker-vm.
|
|
|
|
**Excludes the SureFire tenant hosts only** (`sf-*`, `sfsrv-ana`).
|
|
|
|
⚠ **Test with `sudo -n -v` first** rather than assuming. Not every box has been
|
|
bootstrapped — `pbs-nh3` (10.100.50.90), for instance, has no infra-ops user and is
|
|
reachable only as `lkraven@`.
|
|
|
|
⚠ **Remote sudo needs `ssh -t`.** Read-only probes can use plain ssh.
|
|
|
|
## Sites and hosts
|
|
|
|
| site | subnet | gateway |
|
|
|---|---|---|
|
|
| Anaheim (colo) | `10.250.0.0/16` | FortiGate `10.250.0.1` |
|
|
| NH3 | `10.100.0.0/16` | UDM-SE `10.100.0.1` |
|
|
| ESH home lab | `10.0.50.0/24` (`esteban.net`) | UDM-PM `10.0.0.1` |
|
|
| Fountain Valley | `10.251.0.0/16` | |
|
|
| Irvine | via headscale mesh | |
|
|
|
|
Full host table with roles: `~/development/eshpfi-management/CLAUDE.md` § Servers.
|
|
Live per-host snapshots: `servers/<name>/system-details.txt`.
|
|
Internal DNS names are `<host>.<site>.internal`, source of truth `dns/internal.yaml`.
|
|
|
|
## Mesh
|
|
|
|
Headscale is primary since 2026-09-06 (IPsec tunnels dormant). Subnet routers are
|
|
`nh3-scale` / `ana-scale` / `esh-scale` at `100.64.0.1` / `.3` / `.2`.
|
|
|
|
⚠ **Disable a site-to-site tunnel BEFORE adding a mesh route for the same prefix** or
|
|
they fight.
|
|
⚠ **`accept-routes` needs a return path FIRST.** On a dev box it can black-hole its own
|
|
LAN (route table 52 beats main). Gateway routes or SNAT before, never on your own box.
|
|
|
|
## Traps
|
|
|
|
- **SureFire tenant hosts are client property** under the hosting agreement. Coordinate
|
|
before touching anything that affects data. This is a contractual posture, not a
|
|
security one.
|
|
- **Confirm a reboot by observing DOWN**, then return. Poll for disappearance and
|
|
cross-check the boot timestamp — "never rebooted" and "rebooted very fast" look
|
|
identical otherwise.
|
|
- **`chroot` rbind needs `--make-rslave`.** Without it, `umount -R` unmounts the LIVE
|
|
host's cgroup/devpts, which looks exactly like disk failure.
|
|
- **`pct snapshot` exits 0 when it refuses** (bind-mounted guest). Use `zfs snapshot`
|
|
and read it back.
|