feat(svos): v0.0.17 live on nh3-svos and nh3-dev — kernel 0.0.17 + UI 0.0.21, the six runtimes' activity marks (gen 29bc36bf), terminal 0.0.6, voice utterance_seconds 60; activity events verified
This commit is contained in:
@@ -23,6 +23,36 @@ configuration, enrollment and acceptance.
|
||||
| App-consistent hook | svos-dev's `/etc/qemu/fsfreeze-hook.d/50-svos` stops the SVOS services on freeze and restarts the ones that were running on thaw. ⚠ **Debian's qemu-ga does NOT run fsfreeze hooks unless configured** (no `-F` in the unit, no `qemu-ga.conf` by default). Here `/etc/qemu/qemu-ga.conf` points it at **`/usr/local/sbin/fsfreeze-dispatch`** (`configs/qemu-ga/fsfreeze-dispatch`), which unlike the stock `/etc/qemu/fsfreeze-hook` propagates failure: if a freeze hook fails, it thaws that hook and the completed ones and exits 1, so qemu-ga refuses the freeze. **Live drill 2026-10-05 1952–1953 (services running):** quiesced backup `vm/111/2026-10-06T02:52:11Z` → kernel+interface down ~1–2 s, back by themselves, health ok. Refused freeze (temporary `60-drill-refuse` after 50-svos, since removed) `vm/111/2026-10-06T02:53:03Z` → the dispatcher unwound and restarted the services, and then **⚠ vzdump did NOT abort: it logged `ERROR: unable to freeze guest fs`, backed up UNFROZEN (crash-consistent, app live), and reported `Backup job finished successfully`, rc 0.** Both images mapped from PBS and mounted read-only (`ro,noload`) on nh3-pve-2, never booted: `svos.db` integrity ok (each image still carries its ~0.9 MB WAL: keep `-wal` with the db when restoring), interface `pins.sqlite3` integrity ok, config/key/config-generation hashes equal live, table counts equal live except `events`. **Monitoring:** `scripts/check-backup-freshness.sh` now reads the LOG of every OK vzdump task and reports ERROR lines; a freeze/thaw error is an APPLICATION-CONSISTENCY FAILURE, verdict **DEGRADED, exit 4** (alerted by the daily wrapper to infra-hermes). nh3-pve-2 was not in the check's node lists at all before 2026-10-05; it is now. Drill B is acknowledged by UPID in the script |
|
||||
| Monitoring | Beszel native agent 0.18.7 (`playbooks/beszel-native.yaml`, `configs/beszel-agent/hosts/nh3-svos.env`), hub system `t6y646ifa3x7xxv`, alerts CPU>95%/15m, Disk>85%/5m, Memory>90%/10m, offline/2m. Up 2026-10-05 |
|
||||
|
||||
## Release 15c2d04 / v0.0.17, seat activity (2026-10-18 2254–2255; Prime's "deploy now" 2235 via svos-dev, thread `01M595YP8R62Y665C38MHPH6YX`)
|
||||
|
||||
- **Kit:** kernel 0.0.17, app-terminal 0.0.6 wheels and web build UI 0.0.21 from a clean archive of the tag. Everything else was
|
||||
carried byte-identical from b51aa5e (no source change; uv.lock changed only the kernel version). Kit sha256 `f6e4ff1e…`.
|
||||
`release.json` now says schema 14 (the carried value said 13; there is no migration in this release).
|
||||
Playbooks: `nh3-svos-15c2d04-v0.0.17-stage.yaml`, then `nh3-svos-15c2d04-v0.0.17.yaml`, then `nh3-dev-svos-15c2d04-v0.0.17.yaml`.
|
||||
- **nh3-svos:** offline backup `backup/pre-15c2d04` (svos.db, integrity ok, user_version 14), `instance.toml.pre-15c2d04`,
|
||||
`apps/voice.toml.pre-15c2d04`. The six `[[seats.runtimes]]` got `prompt`/`working`/`window` (+ the U17 comment lines) copied
|
||||
verbatim from `docs/operations/u13-edit-payload.md` @15c2d04 by a script generated from that file, inserted under each
|
||||
`program` line. Config gen **`29bc36bf074b8245`** (was `13241a403080788c`). Kernel READY in 0.79 s.
|
||||
voice.toml `utterance_seconds` 30 → 60 (UI 0.0.21's client admits 0.1..120 s). All 8 apps up by 2254:52.
|
||||
- ⚠ The first cutover run failed at the edit: svc-svos cannot read infra-ops's 0700 stage dir, so the script is now fed on
|
||||
stdin. The kernel was down from 2254:08 to 2254:22 in total, including the fix and the re-run; the voice app came back last, at about 2254:36.
|
||||
- **nh3-dev:** `/opt/svos/releases/15c2d04` = 3726d73's apps-env with svos-terminal 0.0.6 (uv pip freeze differs in that line only),
|
||||
`current → 15c2d04`, only svos-terminal restarted (2255:05). postoffice, miranda and agent still run 3726d73's identical code.
|
||||
`terminal.toml` unchanged (`activity_interval_seconds` defaults to 5).
|
||||
- **No card:** terminal manifest revision 6 changes only `version` (0.0.5 → 0.0.6), so the kernel pinned it at 2255:07 without a
|
||||
card. The expected held-manifest card never existed.
|
||||
- **Verified (events table):** 13 `seat/activity_changed` at 2255:13, the first pass after the terminal restart. ops-infra was
|
||||
`working` (this session, mid-deploy: the positive control); the 12 other live seats were `waiting`. dev-ldp is live with no
|
||||
activity event, so it stays `unknown`. Its claude has been running 14 days, and with both marks declared, a screen that
|
||||
matches neither is unknown by design. The likeliest cause is an older harness screen (reasoned, not checked). The new
|
||||
kernel talked to the old 0.0.5 host for 43 s without an error. The only `malformed reply` pair came at 2255:06, while the
|
||||
terminal was restarting. GET `/seats` was not read: it needs a principal, and infra-ops borrowed none.
|
||||
- **Rollback:**
|
||||
- nh3-svos: stop the apps and the kernel, restore `instance.toml.pre-15c2d04` (an old kernel refuses the runtime keys) and
|
||||
`apps/voice.toml.pre-15c2d04`, `current → releases/b51aa5e`, start, run `svos-front-inventory`. svos.db needs no restore
|
||||
(no migration); `backup/pre-15c2d04` is there if wanted.
|
||||
- nh3-dev: `current → releases/3726d73`, restart svos-terminal (revision 5 again; likely pinned without a card, as above).
|
||||
|
||||
## Release b51aa5e / v0.0.16, the judge app (2026-10-18 1633–1642; Prime: "go with your picks, cut over now"; svos-dev thread `01M57XAEZVQ39DKNYXRTMC29S5`)
|
||||
|
||||
- **Kit:** kernel 0.0.16, app-judge 0.0.1 and UI 0.0.20 from a clean archive of the tag. The rest was carried from 57d90d6. Kit sha256 `25f78195…`.
|
||||
|
||||
Reference in New Issue
Block a user