docs(nh3-svos): refreshed to SVOS ac42ace — kernel 0.0.7 + Type=notify pair, thaw race fixed (drill 2 -> 0)

2026-10-08 15:17:56 -07:00
parent b04cee5579
commit d2f9f561c2
2 changed files with 21 additions and 1 deletions
@@ -23,6 +23,26 @@ configuration, enrollment and acceptance.
 | App-consistent hook | svos-dev's `/etc/qemu/fsfreeze-hook.d/50-svos` stops the SVOS services on freeze and restarts the ones that were running on thaw. ⚠ **Debian's qemu-ga does NOT run fsfreeze hooks unless configured** (no `-F` in the unit, no `qemu-ga.conf` by default). Here `/etc/qemu/qemu-ga.conf` points it at **`/usr/local/sbin/fsfreeze-dispatch`** (`configs/qemu-ga/fsfreeze-dispatch`), which unlike the stock `/etc/qemu/fsfreeze-hook` propagates failure: if a freeze hook fails, it thaws that hook and the completed ones and exits 1, so qemu-ga refuses the freeze. **Live drill 2026-10-05 1952–1953 (services running):** quiesced backup `vm/111/2026-10-06T02:52:11Z` → kernel+interface down ~1–2 s, back by themselves, health ok. Refused freeze (temporary `60-drill-refuse` after 50-svos, since removed) `vm/111/2026-10-06T02:53:03Z` → the dispatcher unwound and restarted the services, and then **⚠ vzdump did NOT abort: it logged `ERROR: unable to freeze guest fs`, backed up UNFROZEN (crash-consistent, app live), and reported `Backup job finished successfully`, rc 0.** Both images mapped from PBS and mounted read-only (`ro,noload`) on nh3-pve-2, never booted: `svos.db` integrity ok (each image still carries its ~0.9 MB WAL: keep `-wal` with the db when restoring), interface `pins.sqlite3` integrity ok, config/key/config-generation hashes equal live, table counts equal live except `events`. **Monitoring:** `scripts/check-backup-freshness.sh` now reads the LOG of every OK vzdump task and reports ERROR lines; a freeze/thaw error is an APPLICATION-CONSISTENCY FAILURE, verdict **DEGRADED, exit 4** (alerted by the daily wrapper to infra-hermes). nh3-pve-2 was not in the check's node lists at all before 2026-10-05; it is now. Drill B is acknowledged by UPID in the script |
 | Monitoring | Beszel native agent 0.18.7 (`playbooks/beszel-native.yaml`, `configs/beszel-agent/hosts/nh3-svos.env`), hub system `t6y646ifa3x7xxv`, alerts CPU>95%/15m, Disk>85%/5m, Memory>90%/10m, offline/2m. Up 2026-10-05 |

+## Release ac42ace (2026-10-08 1515, svos-dev thread `01M4ES8Q110HNH8J4WWCGE6S4K`)
+
+- `/opt/svos/current` → `releases/ac42ace`: kernel **0.0.7** (schema 11, no migration), connector base 0.0.6,
+  interface UI **0.0.7** (top bar with the Pacific clock; the tree as the desk's stage), svos-agent 0.0.2 (staged).
+  Every other app wheel is unchanged, so there was no re-enrollment and no config change. Built from a clean archive,
+  the same way as a737a06; third-party pins are unchanged. **Previous = `a737a06`** (the front door's previous-asset
+  handle points at it). `6c6abbe` is no longer referenced and is safe to delete (it cannot read schema 11).
+- **Kernel readiness pair installed together:** `svos-kernel.service` now has `Type=notify`, `NotifyAccess=main` and
+  `TimeoutStartSec=300` (copied from the release's `deployment/nh3-svos/svos-kernel.service`; the old unit is kept as
+  `.pre-notify`). `systemctl start svos-kernel` now returns only after `READY=1` (0.61 s, `/health` 200 at once). ⚠
+  Install a kernel ≥ 0.0.7 and this unit as a pair, never one alone: `Type=notify` on an older kernel times the start
+  out at 300 s.
+- **Freeze/thaw drill, with a control:** the pre-refresh vzdump on the OLD kernel (15:14) made terminal and knowledge
+  fail once at thaw (the known race). The post-refresh vzdump (`vm/111/2026-10-08T22:16:12Z`, 15:16) showed **0**
+  app startup failures; all six units came back, and `/health` read up 4, degraded 1 within ~2 s. The one-poll
+  "down" for the terminal right after the restart is its first health poll, not a fault.
+- Acceptance: `/health` ok, up 4, degraded 1. Chromium phone + desk over the LAN door: the top bar is present with
+  clock `1517`, the desk stage shows the tree (4 stage layers + hero), no overflow, 0 page errors; desk Connect voice
+  shows "Voice connected".
+
 ## Release a737a06 + apps (2026-10-07, svos-dev thread `01M4D1GNYBW7QVY7RF05R412V2`, Prime's 2240 decision)

 - **Release:** `/opt/svos/current` → `releases/a737a06` (previous `6c6abbe` kept: it cannot read schema 11, and the