feat(secrets-broker): nh3-dev backfill complete (25/25) + attachment + resilient run

Scope corrected to per-dev-box (CC sessions on this box), not a fleet service; each
box duplicates the stack and backs up its own local secrets, hostname-namespaced.

CLI:
- backfill is local-only (scan this box's ~/development/*/{env.sh,.env} + ~/.config
  credentials; exclude bootstrap.env/examples/AIPA-Data archives).
- large files (>6000 B) route to a bw ATTACHMENT instead of the note field
  (Vaultwarden caps notes at ~10000 encrypted chars); get/verify read it back.
- backfill catches per-item failures and continues (bw errors raise BwError,
  main converts to a clean exit); idempotent upsert makes re-runs safe.

Backfilled all 25 nh3-dev secret files into the infra-ops org's Default collection
(folder = hostname), every one round-trip verified (2 large via attachment, 23 via
note). README added for duplicating the stack to new dev boxes. Contract scope +
data-model sections updated (bw, org/collection, per-box).
This commit is contained in:
vh
2026-08-11 16:35:06 -07:00
parent 41359eaff9
commit 850a1976d5
3 changed files with 234 additions and 113 deletions
@@ -21,18 +21,21 @@ agents, scripts, and the operator use. Solves two problems at once: **durability
(secrets currently backed up nowhere) and **lookup** (no canonical place to find a
credential).
This is a **central store**, not (yet) a deploy-time source of truth: host `.env`
files stay in place; the vault is the authoritative *registry + backup*. The
pull-at-deploy evolution is explicitly out of scope for v1 (see Out of scope).
**Scope (corrected 2026-08-11):** this serves the **CC sessions on THIS dev box
(nh3-dev)** — a per-box credential store + backup, NOT a fleet service. New dev
boxes get this same stack **copied and duplicated one at a time** (each box: its own
`bw` + a per-box `bootstrap.env`, backing up its own local secrets, hostname-
namespaced). **No daemon / no central broker** — the subprocess CLI is the final
shape. Local `.env`/`env.sh` files stay in place; the vault is the authoritative
*registry + backup*. Deploy-time source-of-truth (pull-at-deploy) stays out of scope.
## Architecture
- **Store:** a **dedicated Vaultwarden service account** (`secrets-broker@…`), its
personal vault *is* the registry. (An Organization + per-host Collections is the
textbook multi-user structure, but **rbw's write path into org collections is
weak** — see the rbw capability gate — so v1 uses the service account's own vault
with a folder/name taxonomy. Org migration is a future option if granular
human ACLs are ever needed.)
- **Store:** the operator-provisioned service account (`infra-ops@phasefinal.com`)
in the **`infra-ops` org** (id `d30c6b58…`), **Default collection** (id
`b829376a…`). Items land in that org collection so the operator's **primary
account** (shared into the org) sees them too; organised by **folder** (per box,
named for the hostname) + a `<host>/<path>` name convention.
- **Client:** **`bw`** (official Bitwarden CLI, installed to `~/.local` on nh3-dev).
Switched from `rbw` after rbw's `register` returned an undebuggable 400 against this
Vaultwarden despite valid creds (a direct `client_credentials` grant + both prelogin
@@ -40,10 +43,12 @@ pull-at-deploy evolution is explicitly out of scope for v1 (see Out of scope).
(`login --apikey` via `BW_CLIENTID`/`BW_CLIENTSECRET` env, `unlock --passwordenv
--raw` → `BW_SESSION`) and full write support — org collections + attachments
included, which reopens the org-vs-personal-vault choice rbw had foreclosed.
- **Broker host + identity:** runs as the session user (`lkraven`) on **nh3-dev**
(this machine — where the live CC sessions + rbw live); rbw 1.15.0 is also present
on nh3-extdev. Fleet ssh + the trust boundary for the bootstrap secret.
- **Code home:** `eshpfi-management/services/secrets-broker/`.
- **Runs as** the session user (`lkraven`) on **nh3-dev**, invoked directly by CC
sessions on this box. No ssh, no daemon; each dev box runs its own copy against its
own local secrets.
- **Code home:** `eshpfi-management/services/secrets-broker/` (the `secret` CLI is
copied to each dev box; the per-box `bootstrap.env` lives in
`~/.config/secrets-broker/`, `0600`, never committed).
## Data model / taxonomy (lookup-first)
@@ -51,13 +56,13 @@ pull-at-deploy evolution is explicitly out of scope for v1 (see Out of scope).
folders are unavailable: `<host>/<stack>/<file>` (e.g. `ana-docker/gitea/.env`),
or `<domain>/<name>` for non-host creds (`gitea/claude-bot-token`,
`wireguard/irv-ml1`, `certs/phasefinal-wildcard`).
- **Folders** (if rbw supports `add --folder` — gate) group by class: `hosts/`,
`tokens/`, `certs/`, `ssh-keys/`, `db/`, `wireguard/`, `misc/`.
- **Item type by shape:** a credential pair → Bitwarden *login* (username/password/
uri); an env file or freeform blob → *secure note* (whole file in the note body).
- **Binary secrets** (`*.pem`, `*.key`, `*.pfx`, `acme.json`, `client_secrets.json`)
→ **base64 into a note field** (rbw has no attachments). Cap ~64 KB/item; larger
keys are flagged, not silently truncated.
- **Folders** group by class (one per box, named for the hostname); the item name
carries the real path so lookup works regardless.
- **Item type:** everything is a *secure note* (type 2) — the file/secret content in
the note body (arbitrary `KEY=VALUE` env content isn't a clean login shape).
- **Binary secrets** (`*.pem`, `*.key`, `*.pfx`) → **base64 into a hidden field**
(`content_b64`); portable and rbw-agnostic. bw could use attachments, but base64
keeps a single fetch path. Cap ~64 KB/item.
- Metadata fields on every item: `source_host`, `source_path`, `synced_at`,
`sha256` (of the plaintext, for drift detection without decrypting to compare).