vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(svos): v0.0.27 live on nh3-svos — interface 0.0.9 (owner-only state dir), UI 0.0.32; no card

Browse Source
This commit is contained in:
vh
2026-10-19 09:54:41 -07:00
parent b1b9a06011
commit 25029feaf4
4 changed files with 80 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
servers/nh3-svos/README.md
+8
View File
@@ -23,6 +23,14 @@ configuration, enrollment and acceptance.
| App-consistent hook | svos-dev's `/etc/qemu/fsfreeze-hook.d/50-svos` stops the SVOS services on freeze and restarts the ones that were running on thaw. ⚠ **Debian's qemu-ga does NOT run fsfreeze hooks unless configured** (no `-F` in the unit, no `qemu-ga.conf` by default). Here `/etc/qemu/qemu-ga.conf` points it at **`/usr/local/sbin/fsfreeze-dispatch`** (`configs/qemu-ga/fsfreeze-dispatch`), which unlike the stock `/etc/qemu/fsfreeze-hook` propagates failure: if a freeze hook fails, it thaws that hook and the completed ones and exits 1, so qemu-ga refuses the freeze. **Live drill 2026-10-05 1952–1953 (services running):** quiesced backup `vm/111/2026-10-06T02:52:11Z` → kernel+interface down ~1–2 s, back by themselves, health ok. Refused freeze (temporary `60-drill-refuse` after 50-svos, since removed) `vm/111/2026-10-06T02:53:03Z` → the dispatcher unwound and restarted the services, and then **⚠ vzdump did NOT abort: it logged `ERROR: unable to freeze guest fs`, backed up UNFROZEN (crash-consistent, app live), and reported `Backup job finished successfully`, rc 0.** Both images mapped from PBS and mounted read-only (`ro,noload`) on nh3-pve-2, never booted: `svos.db` integrity ok (each image still carries its ~0.9 MB WAL: keep `-wal` with the db when restoring), interface `pins.sqlite3` integrity ok, config/key/config-generation hashes equal live, table counts equal live except `events`. **Monitoring:** `scripts/check-backup-freshness.sh` now reads the LOG of every OK vzdump task and reports ERROR lines; a freeze/thaw error is an APPLICATION-CONSISTENCY FAILURE, verdict **DEGRADED, exit 4** (alerted by the daily wrapper to infra-hermes). nh3-pve-2 was not in the check's node lists at all before 2026-10-05; it is now. Drill B is acknowledged by UPID in the script |
| Monitoring | Beszel native agent 0.18.7 (`playbooks/beszel-native.yaml`, `configs/beszel-agent/hosts/nh3-svos.env`), hub system `t6y646ifa3x7xxv`, alerts CPU>95%/15m, Disk>85%/5m, Memory>90%/10m, offline/2m. Up 2026-10-05 |
## Release d5e15c3 / v0.0.27, interface 0.0.9 + UI 0.0.32 (2026-10-19 0954; Prime 1005 via svos-dev, thread `01M5AH9A7QGJ018HNTSGKNGAA7`)
- The tag was already on origin. Kit `6d3da96e…`: kernel 0.0.27 (a version bump; not restarted), app-interface 0.0.9 (the stores share `state.py`:
`state/apps/interface` 0700, pins' WAL sidecars 0600), UI 0.0.32 (`/assets/index-YF0e2xQ9.js`; U23 v0.2, compact inbox row, `?look=` retired).
- Steps: backup `pre-d5e15c3` (the four interface stores, integrity ok), `current → d5e15c3`, front door, restart ONLY svos-interface. **No card:**
the manifest is unchanged (revision 6 still pinned). The interface dir was already `700 svc-svos`. Health ok, 8 up, gen `d750257258f9fb68`.
- Rollback: `current → releases/a83c223`, `svos-front-inventory`, restart svos-interface.
## Release a83c223 / v0.0.26, the flows' cadence — UI-only publish (2026-10-19; 0903; svos-dev thread `01M5AE9MKBP9180CXS2EW8Z8NW`)
- Prime pushed the tag `v0.0.26` → `a83c2237c3be`, and the watcher published (`current → a83c223` + inventory). Nothing restarted.