ops(svos): U13 phase B amendment — full kernel-origin path list, Miranda's env and origin on nh3-dev, enrollment secrets, app-high-seat declared

Per svoperatingsystem dc241f8 u13-svos-edit.md items 1, 4, 5 and 6 (Prime's
go stands): the kernel origin passes exactly the method-pinned list (apps and
Miranda's agent connector; 15 pairs reach the kernel, the rest are Caddy 404);
nh3-dev's apps env is rebuilt as dc241f8 with svos-miranda 0.0.3 and
svos-agent 0.0.3 and gains the svos-miranda origin (nh3-svos only); three
enrollment secrets minted into the vault; app-high-seat declared with one
reload. Also fixes nh3-dev's fleet Caddy sharing admin localhost:2019 with
ttyd-caddy via SO_REUSEPORT (reloads could land on either): now :2021.
This commit is contained in:
vh
2026-10-10 16:56:47 -07:00
parent feef8330ca
commit fa567fa150
5 changed files with 86 additions and 18 deletions
+16
View File
@@ -1,5 +1,9 @@
{ {
email lkraven@lkraven.com email lkraven@lkraven.com
# Own admin port (2026-10-10). ttyd-caddy (stock caddy, ~/.config/caddy) also binds localhost:2019 and both sockets
# use SO_REUSEPORT, so a `reload` on 2019 landed on EITHER instance: this config was rejected by the stock binary
# (no cloudflare module), and ttyd's valid config could have replaced this one. ExecReload reads this address.
admin localhost:2021
} }
# Fleet internal TLS — caddy is the WILDCARD CERT AUTHORITY for *.nh3.phasefinal.com # Fleet internal TLS — caddy is the WILDCARD CERT AUTHORITY for *.nh3.phasefinal.com
@@ -39,6 +43,18 @@
} }
} }
} }
# - svos-miranda: app-miranda (svos-miranda 0.0.3, 127.0.0.1:8408), beside the Hermes gateway it calls on loopback;
# added 2026-10-10 per svoperatingsystem dc241f8 u13-svos-edit.md item 4. Not started before the edit.
@svos_miranda host svos-miranda.nh3.phasefinal.com
handle @svos_miranda {
@not_nh3_svos not remote_ip 10.100.50.43
handle @not_nh3_svos {
respond 403
}
handle {
reverse_proxy 127.0.0.1:8408
}
}
@svos_highseat host svos-highseat.nh3.phasefinal.com @svos_highseat host svos-highseat.nh3.phasefinal.com
handle @svos_highseat { handle @svos_highseat {
@not_nh3_svos not remote_ip 10.100.50.43 @not_nh3_svos not remote_ip 10.100.50.43
+36 -5
View File
@@ -221,9 +221,12 @@ https://svos-terminal.phasefinal.com:443 {
# a request there is never a human and loopback trust never applies, so every call authenticates with its own bearer. # a request there is never a human and loopback trust never applies, so every call authenticates with its own bearer.
# NO Basic auth, and Authorization passes through untouched. # NO Basic auth, and Authorization passes through untouched.
# - Source allowlisted to nh3-dev (10.100.10.50); anything else gets 403 from Caddy. # - Source allowlisted to nh3-dev (10.100.10.50); anything else gets 403 from Caddy.
# - Paths: svos-dev's app surface (/auth/enroll, /auth/proofs, /registry/*, /verbs/*, /seats/enroll) plus # - Paths, method-pinned, exactly svoperatingsystem docs/operations/u13-svos-edit.md item 1 at dc241f8 (every caller on
# /auth/kernel-key, which every svos_connector client fetches to trust the kernel's signatures (client.kernel_key; it # nh3-dev, read at source): the apps (bridge, terminal, app-miranda, app-high-seat) GET /auth/kernel-key, POST
# is also the joint runbook's B2 check: refused by the KERNEL with 401 without a credential). Anything else 404. # /auth/enroll, POST /auth/proofs, POST /registry/apps/*/manifest, /verbs/*, POST /seats/enroll; Miranda's agent
# connector GET /auth/whoami, /registry/apps, /registry/apps/*, /principals/*/grants, /seats, /events,
# /artifacts/resolve (and /verbs/*). Anything else 404. B2's check: GET /auth/kernel-key without a credential is a
# 401 from the KERNEL, not from Caddy.
# - Client identity/forwarding headers are deleted as on the other sites; the same 1 MiB body cap; nothing streams. # - Client identity/forwarding headers are deleted as on the other sites; the same 1 MiB body cap; nothing streams.
https://svos-kernel.phasefinal.com:443 { https://svos-kernel.phasefinal.com:443 {
bind 10.100.50.43 bind 10.100.50.43
@@ -247,8 +250,36 @@ https://svos-kernel.phasefinal.com:443 {
handle @not_nh3_dev { handle @not_nh3_dev {
respond 403 respond 403
} }
@app_surface path /auth/enroll /auth/proofs /auth/kernel-key /registry/* /verbs/* /seats/enroll @app_get {
handle @app_surface { method GET
path /auth/kernel-key /auth/whoami /registry/apps /registry/apps/* /seats /events /artifacts/resolve
}
handle @app_get {
reverse_proxy 127.0.0.1:8410
}
@app_grants {
method GET
path_regexp ^/principals/[^/]+/grants$
}
handle @app_grants {
reverse_proxy 127.0.0.1:8410
}
@app_post {
method POST
path /auth/enroll /auth/proofs /seats/enroll
}
handle @app_post {
reverse_proxy 127.0.0.1:8410
}
@app_manifest {
method POST
path_regexp ^/registry/apps/[^/]+/manifest$
}
handle @app_manifest {
reverse_proxy 127.0.0.1:8410
}
@app_verbs path /verbs/*
handle @app_verbs {
reverse_proxy 127.0.0.1:8410 reverse_proxy 127.0.0.1:8410
} }
handle { handle {
+2 -2
View File
@@ -118,7 +118,7 @@ _As of 2026-10-09 ~1955 PT. The newest subsection is first; older subsections ca
### 2026-10-09: live now (as of 2026-10-09 ~1955 PT) ### 2026-10-09: live now (as of 2026-10-09 ~1955 PT)
- **Nothing is mid-action.** No claims held (`scripts/ops-log claims`). eshpfi is pushed through this snapshot's commit. ⚠ The working tree AND index are shared with infra-hermes and subagents: commit with `git commit -- <paths>` (auto-memory `feedback_shared_git_index_commit_pathspecs`). `graphify-out/GRAPH_REPORT.md` stays modified and uncommitted on purpose (auto-regenerated). - **Nothing is mid-action.** No claims held (`scripts/ops-log claims`). eshpfi is pushed through this snapshot's commit. ⚠ The working tree AND index are shared with infra-hermes and subagents: commit with `git commit -- <paths>` (auto-memory `feedback_shared_git_index_commit_pathspecs`). `graphify-out/GRAPH_REPORT.md` stays modified and uncommitted on purpose (auto-regenerated).
- **U13 phase B (infra) DONE 2026-10-10 1646–1650 on Prime's go:** nh3-svos on v0.0.9 (`afab123`, schema 12) with app-terminal + app-postoffice HELD on b588510 by drop-ins (remove at the edit; keep `releases/b588510` until then); kernel app origin `svos-kernel.phasefinal.com` (LAN, nh3-dev only); `svos-seathost` + `svos-highseat` origins on nh3-dev (nh3-svos only); bridge env `/opt/svos/current/apps-env` on nh3-dev. Next is the atomic edit (highseat-dev's runbook `svos/docs/u13-cutover-runbook.md`, Prime's go; E2 is S→I). → `servers/nh3-svos/README.md`, `servers/nh3-dev/README.md` - **U13 phase B (infra) DONE 2026-10-10 1646–1650 on Prime's go:** nh3-svos on v0.0.9 (`afab123`, schema 12) with app-terminal + app-postoffice HELD on b588510 by drop-ins (remove at the edit; keep `releases/b588510` until then); kernel app origin `svos-kernel.phasefinal.com` (LAN, nh3-dev only); `svos-seathost` + `svos-highseat` + `svos-miranda` origins on nh3-dev (nh3-svos only); apps env `/opt/svos/current/apps-env` (dc241f8: bridge, terminal, miranda 0.0.3, agent) on nh3-dev; three enrollment secrets vaulted, app-high-seat declared (B5). Next is the atomic edit (highseat-dev's runbook `svos/docs/u13-cutover-runbook.md`, Prime's go; E2 is S→I). → `servers/nh3-svos/README.md`, `servers/nh3-dev/README.md`
- **Owed by me (no input needed):** - **Owed by me (no input needed):**
- Destroy `zroot/ROOT/debian@pre-patch-20261008` on fv-ml1 ~10-15 if nothing regressed. - Destroy `zroot/ROOT/debian@pre-patch-20261008` on fv-ml1 ~10-15 if nothing regressed.
- nh3-svos housekeeping: releases `6c6abbe` and `a737a06` are no longer referenced by the front door and can be deleted (`ac42ace` holds the same static as the current `b588510`). - nh3-svos housekeeping: releases `6c6abbe` and `a737a06` are no longer referenced by the front door and can be deleted (`ac42ace` holds the same static as the current `b588510`).
@@ -186,7 +186,7 @@ Every vLLM seat on a shared card (GPU 0, GPU 1) has byte-pinned KV (`--kv-cache-
## Recent decisions ## Recent decisions
- `[2026-10-10]` **U13 phase B built (Prime 1639: "implement as asked")**: B1 kernel 0.0.9 + schema 12 (vzdump `vm/111/2026-10-10T23:45:41Z` + offline `pre-afab123` copy first), B2 `svos-kernel.phasefinal.com` → `--app-listen 127.0.0.1:8410` (allowlist nh3-dev; `/auth/kernel-key` ADDED to svos-dev's path list because every connector client fetches it), B3 seat-host origin + bridge env on nh3-dev, B4 app-high-seat origin. Kernel answers 401 `no principal` through the origin; other sources 403. - `[2026-10-10]` **U13 phase B built (Prime 1639: "implement as asked")**: B1 kernel 0.0.9 + schema 12 (vzdump `vm/111/2026-10-10T23:45:41Z` + offline `pre-afab123` copy first), B2 `svos-kernel.phasefinal.com` → `--app-listen 127.0.0.1:8410` (allowlist nh3-dev; `/auth/kernel-key` ADDED to svos-dev's path list because every connector client fetches it), B3 seat-host origin + bridge env on nh3-dev, B4 app-high-seat origin; then svos-dev's dc241f8 amendment: full method-pinned path list, Miranda's app + agent env and `svos-miranda` origin on nh3-dev, three secrets minted to the vault, app-high-seat declared. Found + fixed: nh3-dev's fleet Caddy shared admin :2019 with ttyd-caddy (SO_REUSEPORT), now :2021. Kernel answers 401 `no principal` through the origin; other sources 403.
- `[2026-10-10]` **esh-nas-pve `nvme/ROOT@pre-pve9` destroyed 0750 (607M)** after a week clean on PVE 9.2.21 / 7.0.14-20 (up 6 d, no err-level journal in 7 d, pools healthy, all guests up; only `openipmi.service` failed, the known item). Left: `nvme/ROOT/pve-1@pre-upgrade-20260818T141652Z` (now 1.43G, a pre-PVE-9 rollback point that is useless as one) and `nvme/subvol-105-disk-0@pre-vaapi-20260911`. - `[2026-10-10]` **esh-nas-pve `nvme/ROOT@pre-pve9` destroyed 0750 (607M)** after a week clean on PVE 9.2.21 / 7.0.14-20 (up 6 d, no err-level journal in 7 d, pools healthy, all guests up; only `openipmi.service` failed, the known item). Left: `nvme/ROOT/pve-1@pre-upgrade-20260818T141652Z` (now 1.43G, a pre-PVE-9 rollback point that is useless as one) and `nvme/subvol-105-disk-0@pre-vaapi-20260911`.
- `[2026-10-09]` **vh gets every albok read group (Prime 2244: "vh gets albok-everything")**: albok-read, albok-personal and albok-feedback (the restricted wing's default-deny ends for vh only), plus his four Macs' keys on vh (`playbooks/nh3-albok-vh-viewer.yaml`). No net new root path: those Macs already reach root there via lkraven@nh3-dev and the infra-ops key. Later restricted groups (1513+) are NOT included. - `[2026-10-09]` **vh gets every albok read group (Prime 2244: "vh gets albok-everything")**: albok-read, albok-personal and albok-feedback (the restricted wing's default-deny ends for vh only), plus his four Macs' keys on vh (`playbooks/nh3-albok-vh-viewer.yaml`). No net new root path: those Macs already reach root there via lkraven@nh3-dev and the infra-ops key. Later restricted groups (1513+) are NOT included.
- `[2026-10-09]` **nh3-dev: orphaned svos pytest killed 2242** (14.4 GB RSS, +2.3 GB/h, swap 100%, Beszel 90% alert; SIGTERM ignored, SIGKILL; used 25.4 → 11.3 GB). highseat-dev's retry-loop test with an instant fake sleep; they added pytest-timeout 300 s and a 50-attempt bound. - `[2026-10-09]` **nh3-dev: orphaned svos pytest killed 2242** (14.4 GB RSS, +2.3 GB/h, swap 100%, Beszel 90% alert; SIGTERM ignored, SIGKILL; used 25.4 → 11.3 GB). highseat-dev's retry-loop test with an instant fake sleep; they added pytest-timeout 300 s and a 50-attempt bound.
+15 -6
View File
@@ -185,12 +185,21 @@ The new SVOS kernel runs on nh3-svos; at the U13 edit three of its apps run HERE
and both 502 until their app listens: and both 502 until their app listens:
- `https://svos-seathost.nh3.phasefinal.com` → `127.0.0.1:8404`, app-terminal's listener (unbuffered). - `https://svos-seathost.nh3.phasefinal.com` → `127.0.0.1:8404`, app-terminal's listener (unbuffered).
- `https://svos-highseat.nh3.phasefinal.com` → `127.0.0.1:8773`, app-high-seat (the High Seat's signed-only listener). - `https://svos-highseat.nh3.phasefinal.com` → `127.0.0.1:8773`, app-high-seat (the High Seat's signed-only listener).
- **Apps env `/opt/svos/current/apps-env`** (→ `releases/afab123`, lkraven-owned): python 3.13.2 copied to - `https://svos-miranda.nh3.phasefinal.com` → `127.0.0.1:8408`, app-miranda (svos-miranda), which calls the Hermes
`/opt/svos/python/` (so it does not depend on `~/.local/share/uv`), the release's hashed `apps-requirements.txt`, then gateway's API server on `127.0.0.1:8765`. Miranda's agent connector (svos-agent) will listen on `127.0.0.1:8407`,
svos-connector 0.0.7, **svos-postoffice 0.0.2 with its `push` extra satisfied by althing-core 3.7.0** installed loopback only (the gateway plugin's `connector_url`), with no origin.
(non-editable) from a clean archive of althing `7a2a1c1`, and svos-terminal 0.0.3. ⚠ althing-core's own deps - ⚠ **The fleet Caddy's admin endpoint is `localhost:2021` since 2026-10-10.** It used to share `localhost:2019` with
(python-ulid 4.0.1, pyyaml 6.0.3) are resolved, not hash-pinned. No units yet: the bridge and terminal units, their ttyd-caddy (the stock binary, `~/.config/caddy`) through SO_REUSEPORT, so a `reload` landed on either process: one
config and their credentials are written at the edit. The kernel is reached at `https://svos-kernel.phasefinal.com` U13 reload was rejected by the stock caddy ("unknown module dns.providers.cloudflare"), and a ttyd reload could
have replaced this config. `ExecReload` reads the address from the Caddyfile, so reloads are deterministic now.
- **Apps env `/opt/svos/current/apps-env`** (→ `releases/dc241f8`, lkraven-owned; `releases/afab123` is an earlier,
unreferenced env): python 3.13.2 copied to `/opt/svos/python/` (so it does not depend on `~/.local/share/uv`), the
release's hashed `apps-requirements.txt`, bifrost 1.2.1, svos-connector 0.0.7, **svos-postoffice 0.0.2 with its
`push` extra satisfied by althing-core 3.7.0** installed (non-editable) from a clean archive of althing `7a2a1c1`,
svos-terminal 0.0.3, **svos-miranda 0.0.3** (built from dc241f8: Miranda's charter with the app-high-seat lines) and
svos-agent 0.0.3. ⚠ althing-core's own deps
(python-ulid 4.0.1, pyyaml 6.0.3) are resolved, not hash-pinned. No units yet: every app's unit, config and credentials
are written at the edit, and none of them starts before it (two live Mirandas never overlap). The kernel is reached at `https://svos-kernel.phasefinal.com`
(LAN, allowlisted to this box). No input firewall here, so a LAN listener for the bridge needs no rule. (LAN, allowlisted to this box). No input firewall here, so a LAN listener for the bridge needs no rule.
## Box-wide Playwright / Chromium (2026-06-04) ## Box-wide Playwright / Chromium (2026-06-04)
+17 -5
View File
@@ -42,11 +42,23 @@ configuration, enrollment and acceptance.
- **B2, the kernel's app origin:** drop-in `svos-kernel.service.d/20-app-listen.conf` (`configs/svos-kernel/`) adds - **B2, the kernel's app origin:** drop-in `svos-kernel.service.d/20-app-listen.conf` (`configs/svos-kernel/`) adds
`--app-listen 127.0.0.1:8410`; Caddy serves `https://svos-kernel.phasefinal.com` on **10.100.50.43 only** (own LE `--app-listen 127.0.0.1:8410`; Caddy serves `https://svos-kernel.phasefinal.com` on **10.100.50.43 only** (own LE
cert, DNS-01), no Basic auth, bearer passed, identity headers stripped, **source allowlisted to nh3-dev** (10.100.10.50), cert, DNS-01), no Basic auth, bearer passed, identity headers stripped, **source allowlisted to nh3-dev** (10.100.10.50),
paths `/auth/enroll`, `/auth/proofs`, `/auth/kernel-key`, `/registry/*`, `/verbs/*`, `/seats/enroll`, else 404. paths **method-pinned to svoperatingsystem dc241f8 `u13-svos-edit.md` item 1** (the apps' and Miranda's agent
`/auth/kernel-key` is ADDED to svos-dev's list: every `svos_connector` client fetches it to trust the kernel's connector's calls: GET `/auth/kernel-key` `/auth/whoami` `/registry/apps[/*]` `/principals/*/grants` `/seats` `/events`
signatures, and it is the runbook's B2 check. The three internal AdGuards answer the name with 10.100.50.43. `/artifacts/resolve`; POST `/auth/enroll` `/auth/proofs` `/seats/enroll` `/registry/apps/*/manifest`; `/verbs/*`), else
Tested: from nh3-dev `/auth/kernel-key` and `/registry/apps` → **401 `no principal` from the kernel**, `/health` and 404. The three internal AdGuards answer the name with 10.100.50.43. Tested from nh3-dev: all 15 allowed method+path
`/api/health` → 404; from nh3-svos and nh3-docker → 403. Caddyfile backup `/etc/caddy/Caddyfile.pre-kernel-origin`. pairs → **401 from the kernel** (JSON `detail`); `/health`, `/stream`, `/cards`, POST `/auth/kernel-key`, DELETE
`/registry/apps/x`, `/principals/a/b/grants` → Caddy's empty 404; from nh3-svos and nh3-docker → 403. Backups
`/etc/caddy/Caddyfile.pre-kernel-origin`, `.pre-kernel-paths`. ⚠ **Validate from a caddy-readable path**
(`/etc/caddy/Caddyfile.check`): the first deploy validated `/tmp/svos-stage-*/` (0700 infra-ops), got "permission
denied", and a `| tail` hid it. The config was valid (Caddy's own reload accepted it), and the validator was then
positive-controlled (a bogus directive → rc 1).
- **B5 + enrollment secrets (2026-10-10 1654):** `svos-kernel mint-secret` (run as infra-ops without sudo, so no
sudo-io copy) minted three tokens straight into the vault, read back by hash: `nh3-svos/app-high-seat-enrollment-secret`,
`nh3-svos/app-miranda-enrollment-secret`, `nh3-svos/miranda-persona-enrollment-secret`. Only app-high-seat is
declared so far (`[[apps]]` with `enrollment_sha256`, one reload, config_gen `22a05c20e17f93ce` → `1bd948c754e571da`,
backup `instance.toml.pre-high-seat`); the other two are declared in the edit.
- nh3-svos stays on afab123: svoperatingsystem `dc241f8` changes only svos-miranda (0.0.3, the charter), which runs on
nh3-dev, so a dc241f8 release here would change nothing that runs.
- The app-side origins on nh3-dev (`svos-seathost`, `svos-highseat`) and the bridge env are in `servers/nh3-dev/README.md`. - The app-side origins on nh3-dev (`svos-seathost`, `svos-highseat`) and the bridge env are in `servers/nh3-dev/README.md`.
## Release b588510 (2026-10-09 0540, kernel 0.0.8: the SQLite reader-pool fix) ## Release b588510 (2026-10-09 0540, kernel 0.0.8: the SQLite reader-pool fix)