From ec8436495cdedade6b030adb73c85a24d467fd18 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Mon, 19 Oct 2026 09:03:11 -0700 Subject: [PATCH] feat(svos): v0.0.26 UI 0.0.30 published on nh3-svos (the flows' cadence); no restart --- persistent-memory.md | 2 +- servers/nh3-svos/README.md | 6 ++++++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/persistent-memory.md b/persistent-memory.md index 9c38cdf..560bf05 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -124,7 +124,7 @@ _As of 2026-10-18 ~2125 PT. The newest subsection is first; older subsections ca - **SVOS v0.0.17 (15c2d04) LIVE since 10-18 2255** (Prime's "deploy now" 2235 via svos-dev, thread `01M595YP8R62Y665C38MHPH6YX`): kernel 0.0.17 + UI 0.0.21 on nh3-svos, config gen `29bc36bf074b8245` (the six runtimes' prompt/working/window), voice `utterance_seconds = 60`; nh3-dev `current → 15c2d04`, only svos-terminal 0.0.6 restarted. No card: terminal revision 6 changes only `version`, so it pinned silently. Verified: 13 `seat/activity_changed` events at 2255:13 (ops-infra working = the positive control, 12 waiting; dev-ldp stays unknown, its 14-day-old claude matches neither mark). Not verified by infra-ops: GET /seats (needs a principal) and the hero "Fleet · working · N" (Prime's eyes). Rollback: servers/nh3-svos/README.md § Release 15c2d04. - **SVOS task launches were broken since v0.0.11 by OUR front door, FIXED 10-19 0118:** the kernel-origin Caddy (`svos-kernel.phasefinal.com`, path allowlist from U13 B2) never admitted the seat host's U15 reports (POST /tasks/start|progress|checking|exit), so every launch died `start_unreported` ("seat launch failed"). Prime's first two tasks (0110/0111) failed that way. Paths added (`configs/svos-front/Caddyfile`, `playbooks/svos-front-caddy-task-reports.yaml`); probe 401 from the kernel, `/auth/nope` still 404. ⚠ **When SVOS adds a host->kernel endpoint, this allowlist must grow with it.** **VERIFIED 0116:** Prime's relaunch `task:f0e9cc59…` ran hermes/gen-large 28 s, exit 0, check exit 0, settled ok, and produced a 1-page PDF. The result read then 403'd for lack of `prime → app:app-terminal read` (artifact resolve dispatches through app-terminal's read verb); grant 267 was approved 0118:44 and also covers the /tasks scan step. - **[2026-10-19] NH3 WAN outage ~0351–0825 (hosts did NOT reboot; WAN IP unchanged 70.230.226.88).** After it, these were re-run and came back clean: headscale-ddns, nemi-vault-pull, claude-memory-sync, ha-backup. backup-freshness reported DEGRADED, which led to: ⚠ **nh3-svos 10-18 21:30 fs-freeze FAILED because the v0.0.16 deploy left `50-svos.pre-judge` (0755) IN `/etc/qemu/fsfreeze-hook.d/`**. qemu's hook runs EVERY executable there (only .bak/.orig/~/.dpkg-* are skipped; a dir passes `-x` too), so the copy ran as a second freeze and refused. Moved to `/root/fsfreeze-hook-retired/`, __pycache__ removed at 0838; infra-hermes' live freeze at 0838:38 ran clean (only 50-svos). **Never back a hook up inside a hook dir.** Tonight's 21:30 is the real check. -- **SVOS v0.0.25 (f40bf6d) UI 0.0.29 LIVE since 10-19 0855** (UI-only: U23 the tree in depth). Processes on 2f29d0d (kernel/knowledge/voice) + d14dc94 (judge/interface) code. Before: v0.0.24 (d14dc94) since 10-19 0841 (U22 card summaries: judge 0.0.2 explain, interface 0.0.8 summaries, UI 0.0.28; 4 cards approved by Prime 0839–0841; grants 294/295). → servers/nh3-svos/README.md § Release d14dc94. Before: v0.0.22 (2f29d0d) UI 0.0.26 since 10-19 0125 (task result files by path). Before: v0.0.21 (45d3a48) UI 0.0.25 since 10-19 0104 (UI-only: the /tasks draft survives a hidden tab). Processes still on 15c2d04 (kernel/knowledge/judge/voice) and 6d4cb1c (interface) code — keep those dirs. Before it: v0.0.20 (81cc5a2) UI 0.0.24 since 10-19 0055 (UI-only again: `stronger` look by default; the /tasks draft waits 125 s, after a 10 s browser abort discarded four good drafts). Nothing restarted. → servers/nh3-svos/README.md § Release 81cc5a2. Before it: (UI-only: U21 working look behind `?look=strong|stronger`, default unchanged; Prime picks, a later patch sets the default). Nothing restarted: the processes run 15c2d04 (kernel/knowledge/judge/voice) and 6d4cb1c (interface) code — keep both dirs. → servers/nh3-svos/README.md § Release 91e9f2d. +- **SVOS v0.0.26 (a83c223) UI 0.0.30 LIVE since 10-19 ~0905** (UI-only: the flows' cadence). Before: v0.0.25 (f40bf6d) UI 0.0.29 since 0855. Processes on 2f29d0d (kernel/knowledge/voice) + d14dc94 (judge/interface) code. Before: v0.0.24 (d14dc94) since 10-19 0841 (U22 card summaries: judge 0.0.2 explain, interface 0.0.8 summaries, UI 0.0.28; 4 cards approved by Prime 0839–0841; grants 294/295). → servers/nh3-svos/README.md § Release d14dc94. Before: v0.0.22 (2f29d0d) UI 0.0.26 since 10-19 0125 (task result files by path). Before: v0.0.21 (45d3a48) UI 0.0.25 since 10-19 0104 (UI-only: the /tasks draft survives a hidden tab). Processes still on 15c2d04 (kernel/knowledge/judge/voice) and 6d4cb1c (interface) code — keep those dirs. Before it: v0.0.20 (81cc5a2) UI 0.0.24 since 10-19 0055 (UI-only again: `stronger` look by default; the /tasks draft waits 125 s, after a 10 s browser abort discarded four good drafts). Nothing restarted. → servers/nh3-svos/README.md § Release 81cc5a2. Before it: (UI-only: U21 working look behind `?look=strong|stronger`, default unchanged; Prime picks, a later patch sets the default). Nothing restarted: the processes run 15c2d04 (kernel/knowledge/judge/voice) and 6d4cb1c (interface) code — keep both dirs. → servers/nh3-svos/README.md § Release 91e9f2d. - **SVOS v0.0.18 (6d4cb1c, U20 settings) LIVE since 10-18 2352** (thread `01M59D7NTZNN75SRP0M4ZQF91A`): infra-ops pushed svoperatingsystem main + tag on Prime's in-session "push it" (his own push never landed). Interface 0.0.7 + UI 0.0.22; only svos-interface restarted; card approved by Prime 2352:16 (via Matrix), interface down 30 s. ⚠ The kernel process still runs 15c2d04's code and reports 0.0.17 until its next restart (identical code). Config gen `d750257258f9fb68` since 10-19 0101 (`[[seats.repos]]` + repo-albok 0016, + repo-dread [path dreadnaught; the session handle is dread-dev, and the repo name must match the handle] and repo-svtc 0101, Prime via svos-dev; backups `instance.toml.pre-repo-albok`, `.pre-repo-dread-svtc`). Both `ns:interface-settings` grants are IN (issued by infra-ops as Prime on his word, approved by him 2356; grant ids 207/208). **Open, Prime's:** the /settings and panel checks plus his listening test. Details + rollback: servers/nh3-svos/README.md § Release 6d4cb1c. - **SVOS (before v0.0.17): v0.0.16 `b51aa5e` on nh3-svos (10-18 1634–2254)** with app-judge (drafting gen-large; six local task models; `[tasks].judge` on, gen `13241a40`); nh3-dev on `3726d73`. Card audio VERIFIED 10-18 (3/3, 4.5–5.5 s). voice.toml `utterance_seconds` was 30 until v0.0.17 (the old web client refused >30); 60 since. **Open:** one drafted scratch task from /tasks (Draft → launch → approve; expect routing, admitted, routed, launching, started) — verifies v0.0.14's gate, the hermes query-file route and U19; Prime must launch it (every spawn is an owner card). Details + rollback material: servers/nh3-svos/README.md. - **Worldtree:** all three on v1.0.0b195 `a416ac01c5a2` since 1112 10-16; Matrix bridge profile-gated (`COMPOSE_PROFILES=matrix` on demo + personal box .env; pinned none, renders api only). Retired-config strips DONE (demo `2240110`, personal `4d1aae2`). ⚠ Pinned's seeded host config dir `/opt/worldtree-pinned/config` is NOT tracked in vh/worldtree-instance-configs and `deploy-wt-config` refuses pinned: capture it. The daily memory gate keeps its harness scratch since 10-18 (`--keep-data`, `~/.local/state/wt-memory-gate-batch/scratch//`, 14-day user tmpfiles); 10-18 PASS. Keep defaults `library:` (thread `01M4WA58P2M001K1420NN72WFM`). diff --git a/servers/nh3-svos/README.md b/servers/nh3-svos/README.md index 035ecd6..7a9a938 100644 --- a/servers/nh3-svos/README.md +++ b/servers/nh3-svos/README.md @@ -23,6 +23,12 @@ configuration, enrollment and acceptance. | App-consistent hook | svos-dev's `/etc/qemu/fsfreeze-hook.d/50-svos` stops the SVOS services on freeze and restarts the ones that were running on thaw. ⚠ **Debian's qemu-ga does NOT run fsfreeze hooks unless configured** (no `-F` in the unit, no `qemu-ga.conf` by default). Here `/etc/qemu/qemu-ga.conf` points it at **`/usr/local/sbin/fsfreeze-dispatch`** (`configs/qemu-ga/fsfreeze-dispatch`), which unlike the stock `/etc/qemu/fsfreeze-hook` propagates failure: if a freeze hook fails, it thaws that hook and the completed ones and exits 1, so qemu-ga refuses the freeze. **Live drill 2026-10-05 1952–1953 (services running):** quiesced backup `vm/111/2026-10-06T02:52:11Z` → kernel+interface down ~1–2 s, back by themselves, health ok. Refused freeze (temporary `60-drill-refuse` after 50-svos, since removed) `vm/111/2026-10-06T02:53:03Z` → the dispatcher unwound and restarted the services, and then **⚠ vzdump did NOT abort: it logged `ERROR: unable to freeze guest fs`, backed up UNFROZEN (crash-consistent, app live), and reported `Backup job finished successfully`, rc 0.** Both images mapped from PBS and mounted read-only (`ro,noload`) on nh3-pve-2, never booted: `svos.db` integrity ok (each image still carries its ~0.9 MB WAL: keep `-wal` with the db when restoring), interface `pins.sqlite3` integrity ok, config/key/config-generation hashes equal live, table counts equal live except `events`. **Monitoring:** `scripts/check-backup-freshness.sh` now reads the LOG of every OK vzdump task and reports ERROR lines; a freeze/thaw error is an APPLICATION-CONSISTENCY FAILURE, verdict **DEGRADED, exit 4** (alerted by the daily wrapper to infra-hermes). nh3-pve-2 was not in the check's node lists at all before 2026-10-05; it is now. Drill B is acknowledged by UPID in the script | | Monitoring | Beszel native agent 0.18.7 (`playbooks/beszel-native.yaml`, `configs/beszel-agent/hosts/nh3-svos.env`), hub system `t6y646ifa3x7xxv`, alerts CPU>95%/15m, Disk>85%/5m, Memory>90%/10m, offline/2m. Up 2026-10-05 | +## Release a83c223 / v0.0.26, the flows' cadence — UI-only publish (2026-10-19; svos-dev thread `01M5AE9MKBP9180CXS2EW8Z8NW`) + +- Prime pushed the tag `v0.0.26` → `a83c2237c3be`, and the watcher published (`current → a83c223` + inventory). Nothing restarted. +- Kit `192b4a44…`: kernel 0.0.26 (a version bump only) and UI 0.0.30, handle `/assets/index-CgwBk4g7.js` + `index-CNudli_R.css`; f40bf6d is the previous handle. +- Rollback: `current → releases/f40bf6d`, then `svos-front-inventory`. + ## Release f40bf6d / v0.0.25, U23 the tree in depth — UI-only publish (2026-10-19 0855; svos-dev thread `01M5ADWS1PZ5NWRXAVJVWGYTAE`) - Prime pushed the tag `v0.0.25` → `f40bf6d78d06`. The watcher saw it at 0855:01 and published (`current → f40bf6d` + inventory). Nothing restarted.