# Fleet SSH — the infra-ops identity and host map

## Identity

`ssh infra-ops@<host>` — a dedicated NOPASSWD-sudo user on PFI-owned boxes, key
`~/.ssh/infra-ops_ed25519`. Covers **every** fleet host including all four Proxmox
hypervisors, corviduo-dev and esh-docker-vm.

**Excludes the SureFire tenant hosts only** (`sf-*`, `sfsrv-ana`).

⚠ **Test with `sudo -n -v` first** rather than assuming. Not every box has been
bootstrapped — `pbs-nh3` (10.100.50.90), for instance, has no infra-ops user and is
reachable only as `lkraven@`.

⚠ **Remote sudo needs `ssh -t`.** Read-only probes can use plain ssh.

## Sites and hosts

| site | subnet | gateway |
|---|---|---|
| Anaheim (colo) | `10.250.0.0/16` | FortiGate `10.250.0.1` |
| NH3 | `10.100.0.0/16` | UDM-SE `10.100.0.1` |
| ESH home lab | `10.0.50.0/24` (`esteban.net`) | UDM-PM `10.0.0.1` |
| Fountain Valley | `10.251.0.0/16` | |
| Irvine | via headscale mesh | |

Full host table with roles: `~/development/eshpfi-management/CLAUDE.md` § Servers.
Live per-host snapshots: `servers/<name>/system-details.txt`.
Internal DNS names are `<host>.<site>.internal`, source of truth `dns/internal.yaml`.

## Mesh

Headscale is primary since 2026-09-06 (IPsec tunnels dormant). Subnet routers are
`nh3-scale` / `ana-scale` / `esh-scale` at `100.64.0.1` / `.3` / `.2`.

⚠ **Disable a site-to-site tunnel BEFORE adding a mesh route for the same prefix** or
they fight.
⚠ **`accept-routes` needs a return path FIRST.** On a dev box it can black-hole its own
LAN (route table 52 beats main). Gateway routes or SNAT before, never on your own box.

## Traps

- **SureFire tenant hosts are client property** under the hosting agreement. Coordinate
  before touching anything that affects data. This is a contractual posture, not a
  security one.
- **Confirm a reboot by observing DOWN**, then return. Poll for disappearance and
  cross-check the boot timestamp — "never rebooted" and "rebooted very fast" look
  identical otherwise.
- **`chroot` rbind needs `--make-rslave`.** Without it, `umount -R` unmounts the LIVE
  host's cgroup/devpts, which looks exactly like disk failure.
- **`pct snapshot` exits 0 when it refuses** (bind-mounted guest). Use `zfs snapshot`
  and read it back.