First real run surfaced 31 gap rows, ~20 of which were noise. These
changes reduce the output to actionable signal.
1. discover-unifi: /ea/devices now filters out
- IPs outside the fleet LAN range (UDM's WAN IP appearing as a
"device", ISP uplink records with public IPs)
- UDM self-records (isConsole=true, or IP matches wans[].ipv4)
- UCI records (UniFi Cable Internet = ISP modem tracking)
LAN filter regex defaults to ^10\. (matches 10.0.0.0/8); override
via UNIFI_LAN_FILTER env var if you run other private ranges.
2. discover-gaps: new --ignore-unifi flag drops rows where the final
SOURCE column starts with "unifi:". Useful for "show me servery
things to manage, not the fleet's network hardware."
3. discover-gaps: known-IP set now pulls IPs from
servers/*/proxmox-details.txt AND servers/*/system-details.txt in
addition to README.md and ssh-target. Consequence: VMs tracked by
proxmox_inspect.sh are automatically counted as known without
needing a separate servers/<vmname>/ dir. Also strips meaningless
addresses (127.*, 0.0.0.0, 169.254.*) so they can't false-positive
a "known" match.
4. MAC normalization: both discover-fortigate and discover-unifi now
emit xx:xx:xx:xx:xx:xx lowercase. Previously FortiGate used colon
format, UniFi used no-separator uppercase — same MAC looked
different per source. Fortigate does tolower() in awk; UniFi uses
a shared jq `norm_mac` function.
Raw dumps of /ea/hosts and /ea/devices surfaced the actual JSON:
- /ea/hosts: LAN IP isn't at top-level ipAddress (that's WAN public);
it's buried in reportedState.ipAddrs[] mixed with WAN + link-local.
Have to pick the first RFC1918 entry that ISN'T also a WAN interface
IP (reportedState.wans[].ipv4). Name/mac/model all live under
reportedState.{hostname,mac,hardware.shortname}.
- /ea/devices: outer records are per-host wrappers; real AP/switch
records are in the nested `devices` array with top-level `ip`, `mac`,
`name`, `model` fields. Previous parser was reading the wrapper and
getting all `-`.
Reorder all TSV outputs so IP is column 1 — makes discover-gaps.sh
work uniformly against both FortiGate and UniFi sources. Sites TSV
dropped its IP slot since sites have no meaningful IP (metadata only).
Verified against the real payloads the user captured: ESH-UDMPM now
surfaces as 10.0.0.1 (LAN) instead of 192.168.200.111 (WAN2, RFC1918
but excluded via the wans cross-check). A sample device record
(E7-ESH-Media at 10.0.250.176) flattens correctly into a single TSV row.
Rewrite to use Ubiquiti's public cloud API at api.ui.com instead of
logging into individual controllers via session cookies. Benefits:
- One API key covers every UniFi OS device on the account (no
per-controller login logic, no cookie jar lifecycle).
- Read-only by design (auth keys are scoped).
- Works across sites transparently.
Three endpoints wired up: hosts (controllers / Cloud Keys), sites,
and devices (APs / switches). Each emits a distinct TSV shape so the
output can be concatenated and still parsed.
`all` mode runs all three and prints section markers on stderr so
the stdout stream stays clean TSV suitable for discover-gaps.sh.
Pagination handled via nextToken. Rate limit not enforced locally;
Ubiquiti documents generous defaults for read endpoints.
Note: Site Manager API (early access) doesn't appear to expose a
connected-client list directly. For endpoint discovery (IP + MAC of
connected clients like laptops, IoT, etc.) we'd still need to hit
each local controller's REST API — follow-up if the infrastructure-
level data isn't enough.
Requires: curl (present), jq (apt install jq).
Three scripts that surface hosts on the fleet's networks that aren't
already tracked under servers/*/. Goal: spot servers that need management
coverage (inventory, backup, monitoring) without wandering the LAN by
hand.
discover-fortigate.sh SSH to a FortiGate admin, run
`execute dhcp lease-list all`, emit TSV
(IP, MAC, hostname, vdom, source).
SSH was picked over the REST API for now
because it needs no API-token plumbing. The
parser is defensive about FortiOS output
format drift (multiple VDOM sections,
optional hostname). API variant can replace
it when the extra robustness is worth the
token setup.
discover-unifi.sh Cookie-auth REST call against a UniFi
Controller. Tries /api/auth/login (UniFi OS
/ UDM / Cloud Key Gen2+) first; falls back
to legacy /api/login for self-hosted
controllers. Output is the same TSV shape
as the FortiGate script so the two mix.
Needs UNIFI_USER / UNIFI_PASS env and jq.
discover-gaps.sh Consumes one or more TSVs from the sources
above. Builds the set of managed IPs from
servers/*/ssh-target (plus a grep of README
files for documented IPs) and prints any
discovered IPs not in that set.
Exit code is 1 if gaps found — suitable for
cron alerting.
Common pipeline:
scripts/discover-fortigate.sh ana-fw.phasefinal.com > /tmp/ana.tsv
scripts/discover-fortigate.sh nh3-gw.phasefinal.com > /tmp/nh3.tsv
UNIFI_USER=admin UNIFI_PASS=… scripts/discover-unifi.sh esh-uc.esteban.net > /tmp/esh.tsv
scripts/discover-gaps.sh /tmp/ana.tsv /tmp/nh3.tsv /tmp/esh.tsv
First-time use probably needs:
- SSH access configured to each FortiGate (admin login, key preferred)
- UniFi user with read access (the built-in API read-only role works)
- `jq` installed on the runner (for UniFi script)
vh