Files
esh-pfi-infrastructure/docs/pfi/opnsense-api-reference.md
T
vh 80d982d1d8 feat(backup): stage the FV firewall config in ana-docker's nightly restic run
The FV edge firewall was not backed up anywhere. Its config now lands in
/var/lib/restic/stage/fv-gateway-config.xml via ana-docker's pre-backup hook,
so the existing 01:00 restic snapshot captures it. ana-docker is one of the
three egress addresses the firewall's WAN allowlist permits, which is why the
pull lives there rather than with the FV hardware — a site that has lost power
cannot back itself up, and FV lost power two days ago.

Non-fatal by design: an unreachable firewall must not abort the nightly
database dumps. But a bad pull must not be promoted either. The summary loop
only rejects EMPTY staged files, and this endpoint answers an auth failure
with a perfectly non-empty HTML error page — which would have been backed up
as a firewall config that is the right size and restores nothing. The block
checks the body really contains <opnsense> and writes nothing otherwise.

Three tests cover it, including the HTML-error-page case. The first draft of
those tests was worthless: _fv returned a Path out of a TemporaryDirectory
context, so the tree was deleted before the assertions ran and every
exists()-is-False check passed regardless of what the script did. Only the
positive test failed, which is the sole reason the broken negatives were
caught. They now snapshot inside the tempdir's lifetime, and the docstring
says why.

Also records two OPNsense API lessons in docs/pfi/opnsense-api-reference.md:
endpoints are actions and must never be probed for existence by POSTing at
them — that is how /api/core/system/reboot took the FV site dark for 3.5
minutes while looking for an apply call this same file already documented —
and the apply step is service/reconfigure, which auth/user notably lacks, so
an API-only key edit persists in config.xml and does nothing until the OS user
sync runs at boot.

Credentials in /etc/restic/fv-gateway.env (root:600), template committed,
values vaulted as fv-gateway/opnsense-api-{key,secret}. Pre-change config
snapshot vaulted as fv-gateway/config-backup-20260914.
2026-09-15 00:12:50 -07:00

149 lines
6.3 KiB
Markdown

# OPNsense API — vendored reference + field-shape notes
**Source:** <https://docs.opnsense.org/development/api.html> (fetched 2026-09-13)
**Why vendored:** the upstream page documents the REST conventions but explicitly
*not* parameter shapes — "the auto-generated API documentation captures endpoints
and HTTP methods but not all parameter details." Those shapes are where the time
goes, so the hard-won ones are recorded in § Field shapes below.
## Authentication
HTTP Basic auth: **key = username, secret = password**.
```sh
curl -s -u "$KEY:$SECRET" "http://<gw>/api/<module>/<controller>/<command>"
```
Fleet creds are vaulted: `fv-gateway/opnsense-api-key`, `fv-gateway/opnsense-api-secret`
(see `secret get`). Authorization is per-user **Effective Privileges** — a key only
reaches endpoints its owner is authorized for.
## Conventions
- **Endpoint pattern:** `/api/<module>/<controller>/<command>/[<param>/...]`
- **GET** retrieves; **POST** creates, updates, or executes an action.
- Request bodies and responses are `application/json`.
Search endpoints take a paging body and return a rows envelope:
```json
{"current":1,"rowCount":7,"sort":{},"searchPhrase":""}
```
```json
{"total":10,"rowCount":7,"current":1,
"rows":[{"id":"configd","locked":1,"running":1,"description":"System Configuration Daemon","name":"configd"}]}
```
## Service control
`/api/<module>/service/{status,start,stop,restart,reconfigure}` — `reconfigure`
writes config **and** applies it, which is normally the one you want after a
`settings/set`. Observed status values include `disabled`, `stopped`, `running`
— note **`disabled` ≠ `stopped`**: a disabled service will not start until its
model's `enabled` field is set to `"1"`.
## ⚠ Endpoints are ACTIONS — never probe for existence by calling them
A 404 tells you an endpoint is absent; a 200 tells you it **ran**. There is no
safe "does this exist?" POST against a live firewall.
**2026-09-15:** looking for the call that applies a user change, a loop POSTed an
empty body at four guessed endpoints to see which returned 404. One of them was
`/api/core/system/reboot`. It returned 200 because it rebooted the FV edge
firewall, taking the whole site — including the BMC, which sits behind it —
dark for 3.5 minutes. The call it was actually looking for is documented
directly above, in § Service control, in this file.
- Read this reference and the upstream endpoint list first.
- If you must discover, use **GET** on a `get`/`search`/`status` command, never
POST on an unknown name.
- Take `/api/core/backup/download/this` **before** any write. That part went
right and is the only reason the change was reversible.
### The one useful thing that fell out of it
`POST /api/core/system/reboot` with `{}` is a **reliable remote reboot** for the
FV gateway — it came back cleanly on its own in ~3.5 min from an API-initiated
restart, which is a capability worth knowing deliberately rather than by
accident. `/api/core/service/restart/<id>` (e.g. `openssh`) restarts one service
without the site outage, and is almost always what you want instead.
## Applying a change — `service/reconfigure`, not a reboot
`settings/set` (and `auth/user/set`) write config.xml. They do **not** sync the
change to the running system. The apply step is the module's service endpoint:
```sh
POST /api/<module>/service/reconfigure {}
```
⚠ Some modules have no `reconfigure` and return `{"errorMessage":"Endpoint not
found"}` — `auth/user` is one. For those the OS-level sync happens on the UI's
own save path or at boot, so an API-only key edit sits in config.xml and does
nothing until then. Verified 2026-09-15: `authorizedkeys` + `shell` for
`infra-ops` persisted immediately but SSH kept refusing, and started working
after a reboot completed the user sync.
⚠ `POST` with **no body at all** returns `411 Length Required`. Send `{}`.
## Field shapes — the part upstream does not document
⚠ **A `settings/get` response is NOT a valid `settings/set` body.** They are
different shapes, and mixing them returns a bare
`{"errorMessage":"Unexpected error, check log for details"}` with no indication
of which field was wrong.
**Selection / option fields.** `get` returns every option with a `selected` flag;
`set` wants only the selected key.
```jsonc
// get
"useExitNode": {"": {"value": "None", "selected": 1}, "abc": {"value":"node-a","selected":0}}
// set
"useExitNode": ""
```
**Array / list fields are UUID-keyed objects, not lists.** This is the one that
costs an afternoon. `get` renders an empty array as `[]`, which misleads you into
POSTing a list.
```jsonc
// get, when empty -> looks like a plain list
"subnets": {"subnet4": []}
// set -> object keyed by a UUID you generate, values are the item's own fields
"subnets": {"subnet4": {"<uuid4>": {"subnet": "10.251.0.0/16", "description": "..."}}}
```
**Discovering an array item's inner fields — two reliable tricks:**
1. POST the wrong shape on purpose. Validation names the full path:
`{"result":"failed","validations":{"settings.subnets.subnet4.<uuid>.subnet":"A value is required."}}`
2. Many models expose a template getter: `/api/<module>/settings/get_<item>`
returns the blank item, e.g. `get_subnet` → `{"subnet4":{"subnet":"","description":""}}`
**Partial `set` is supported** — POST only the fields you are changing rather than
round-tripping the whole model. This sidesteps every other field's shape problem
and is the recommended approach for a surgical change.
**Validation error format:** `{"result":"failed","validations":{"<dotted.path>":"<message>"}}`.
A successful save is `{"result":"saved"}`.
## Working example — enabling a subnet router (2026-09-13, FV gateway)
```jsonc
POST /api/tailscale/settings/set
{"settings":{"enabled":"1",
"subnets":{"subnet4":{"<uuid4>":{"subnet":"10.251.0.0/16",
"description":"FV site subnet router"}}}}}
POST /api/tailscale/service/reconfigure {}
```
**Always read back.** `settings/get` after the write, and re-check anything the
change could disturb (here: `/api/firewall/source_nat/search_rule` total stayed
at 2). A write that reports success is not evidence the value landed.
## Related
- `docs/runbooks/fv-to-ana-nat.md` — the FV gateway's manual SNAT rule
- `docs/runbooks/fv-site-dark-20260913.md` — the outage this reference came out of