Commit Graph
189 Commits
Author SHA1 Message Date
vh 1feb385ca1 ops(svos): v0.0.26 (UI 0.0.30, the flows' cadence) staged on nh3-svos; publish gated on the tag 2026-10-19 09:01:20 -07:00
vh 86532d261c ops(svos): v0.0.25 (UI 0.0.29, U23 the tree in depth) staged on nh3-svos; publish gated on the tag 2026-10-19 08:54:22 -07:00
vh 4e3b3b60ea fix(nh3-svos): stray 50-svos.pre-judge in fsfreeze-hook.d broke the nightly freeze since 10-18 (moved out); NH3 WAN outage recovery; v0.0.16 playbook backs up outside the hook dir 2026-10-19 08:40:34 -07:00
vh ea2eba76a6 ops(svos): v0.0.24 (supersedes the unpushed v0.0.23; U22 after review) staged on nh3-svos; two-card cutover playbooks ready 2026-10-19 02:30:33 -07:00
vh 4b3f7769c0 ops(svos): v0.0.23 (U22: judge 0.0.2 explain, interface 0.0.8 summaries, UI 0.0.27) staged on nh3-svos; judge.toml explain_seconds; two-card cutover playbooks ready 2026-10-19 02:00:51 -07:00
vh 9e8f439597 ops(svos): v0.0.22 (UI 0.0.26) staged on nh3-svos; publish gated on the tag 2026-10-19 01:24:21 -07:00
vh d5e41d8cad fix(svos-front): kernel app origin admits the seat host's task reports (POST /tasks/start|progress|checking|exit); every task launch had failed start_unreported on a Caddy 404 2026-10-19 01:15:11 -07:00
vh 265faa4b15 ops(svos): v0.0.21 (UI 0.0.25, the /tasks draft survives a hidden tab) staged on nh3-svos; publish gated on the tag 2026-10-19 01:03:18 -07:00
vh f9ec331919 ops(svos): repo-dread (dreadnaught; handle dread-dev) and repo-svtc in [[seats.repos]] (gen d7502572) 2026-10-19 01:01:31 -07:00
vh 5be01a8744 fix(scriberr): patch 0004 — chat context window for gateway model names (LLM_DEFAULT_CONTEXT_WINDOW=131072); image ctxwin1 live, idle-gated deploy; svos v0.0.20 staged 2026-10-19 00:55:37 -07:00
vh 48b8f20919 ops(svos): v0.0.19 (UI 0.0.23, U21 working look) staged on nh3-svos; UI-only publish playbook ready, gated on the tag reaching origin 2026-10-19 00:32:54 -07:00
vh 78b676bb7f ops(svos): repo-albok in [[seats.repos]] so albok-dev is adopted as a seat (gen db56b0ed) 2026-10-19 00:17:29 -07:00
vh ec113b6efd ops(svos): v0.0.18 staged on nh3-svos (kernel 0.0.18, interface 0.0.7, UI 0.0.22); cutover playbook ready, gated on the tag reaching origin and Prime at the board 2026-10-18 23:25:46 -07:00
vh 4cc605ecbc feat(svos): v0.0.17 live on nh3-svos and nh3-dev — kernel 0.0.17 + UI 0.0.21, the six runtimes' activity marks (gen 29bc36bf), terminal 0.0.6, voice utterance_seconds 60; activity events verified 2026-10-18 22:57:27 -07:00
vh 9c9d6a777a feat(svos): [tasks].judge = app-judge after Prime's grants; card audio verified (3/3 in 4.5-5.5 s) 2026-10-18 16:42:06 -07:00
vh 4f5c01390d feat(svos): v0.0.16 live on nh3-svos — app-judge enrolled and up (gen-large drafting, six local task models); owner grants pending 2026-10-18 16:34:58 -07:00
vh e2d2f63693 feat(svos): v0.0.16 (b51aa5e) staged on nh3-svos — kernel 0.0.16, app-judge 0.0.1, UI 0.0.20; cutover held for Prime 2026-10-18 09:27:58 -07:00
vh 251c5a8687 feat(svos): v0.0.15 live on nh3-svos (short spoken summons, voice 0.0.2); utterance_seconds 60, Matrix request_timeout 30 2026-10-17 11:01:56 -07:00
vh e85ea97101 fix(svos nh3-svos): Matrix request_timeout_seconds 60 — stopgap for card audio dying at exactly 15 s (svos-dev) 2026-10-17 09:17:21 -07:00
vh 77b75aa172 feat(svos): v0.0.14 live on nh3-svos + nh3-dev (cards approved 0909); hermes task runtime on the query-file route (Prime) 2026-10-17 09:11:10 -07:00
vh 39458e013a fix(svos v0.0.14 playbook): terminal verify reads the ScanError line (svos-dev: scan:on is only on the signed /health) 2026-10-17 07:53:30 -07:00
vh b72783cb3c feat(svos): v0.0.14 (3726d73) release playbooks — staged on nh3-svos and built on nh3-dev; cutover in three phases held for Prime's card window 2026-10-17 07:50:44 -07:00
vh eb2a0a532c feat(nh3-dev): /srv/nemi/books book drop (fiction/ nonfiction/ rejected/) for albok unit 8 2026-10-16 11:09:13 -07:00
vh dd816662dc ops(svos): main 33a7077 (v0.0.13+2) on nh3-svos + nh3-dev — kernel 0.0.13, interface 0.0.5 (briefs), postoffice 0.0.3, UI 0.0.18
On svos-dev's ask, under Prime's 0955 go. The interface unit gets
SVOS_ARTIFACT_HOST=nh3-svos via a drop-in, and the briefs artifact mapping uses
the same host. Both changed manifests were held for the owner, and each app
read down (wrong-revision) until Prime approved its card: interface at
11:09:45, postoffice at 11:11:17. The postoffice was rolled only once he was
at the board, since a held bridge carries no Miranda mail. Health: schema 13,
config_gen 33f001f4e2dbc7dc, apps up 7.
2026-10-14 11:11:59 -07:00
vh a84f6b00df ops(svos): kernel 0.0.12 (d95078c) on nh3-svos; hermes runtime task-capable (Prime via svos-dev)
Kernel-only release from a clean archive of v0.0.12; everything else is
carried from 452a592. Offline backup first; READY in 0.74 s, schema 13, apps
up 7. The existing hermes runtime gained process = python3 and the one-off
task_argv (hermes -z, --accept-hooks, no --yolo), and its argv moved to the
launcher's chat form. check-config valid; config_gen 21489c1f0fff3c10.
2026-10-14 04:17:35 -07:00
vh 5e8307ed7a ops(svos): v0.0.11 (452a592) on nh3-svos + nh3-dev — kernel 0.0.11, schema 13, U15 [tasks] config; terminal 0.0.4
Built in-session from a clean archive of the tag (kernel, connector base and
terminal wheels; everything else carried from 7f95e54). Ran after the 21:30
vzdump of VM 111, with an offline backup of the schema 12 database first.
check-config accepted the new [tasks] table and the tasks artifact mapping
(host nh3-svos, the terminal's artifact_host). Kernel READY in 1.17 s,
schema 13, apps up 7. nh3-dev's current moved to 452a592; svos-terminal 0.0.4
registered manifest revision 4, which was pinned without a card.
2026-10-13 21:33:16 -07:00
vh 7ca2f76d21 feat(albok): personal/inbox wing on nh3-albok + /srv/nemi/inbox capture root on nh3-dev (unit 4.4 slice C, gate 2)
Prime ruled albok subsumes note intake (1530). The wing is like personal/notes
(albok-personal 1511, not service-restricted); health reads 7 wings ready.
Nemi's token was re-minted with read+write on the new wing and vaulted over
albok/token-nemi. /srv/nemi/inbox is lkraven:nemi-drop 2770 with a default ACL,
plus u:lkraven:rx on /srv/nemi, so Nemi reaches it from lkraven's user manager,
whose groups predate nemi-drop. Both paths are verified by the playbook.
2026-10-13 17:31:49 -07:00
vh eda7025ed1 ops(svos): A171 — miranda's app-postoffice write grant autonomous (Prime 2026-10-13 1400) 2026-10-13 13:59:30 -07:00
vh 3e634bb3d0 ops(infra-ops): per-host satellite keys; satellite-1 authorized on esh-nas
Satellite agents get their OWN key under the infra-ops identity instead of a
copy of the fleet key, so one can be revoked by deleting a line. The list is
playbooks/files/infra-ops-satellite-keys (target short hostname + public key).
bootstrap-infra-ops-user.yaml now writes the fleet key plus that host's
satellite keys, exactly, so a re-run keeps listed keys and removes unlisted
ones. satellite-1 (SHA256:KEFUjceF...) is authorized on esh-nas only (Prime,
2026-10-13); the playbook re-run there installed both keys, and the fleet key
still logs in with sudo.
2026-10-13 13:33:04 -07:00
vh 30a7220b2c ops(esh-nas): infra-ops identity bootstrapped in CT 103 via pct from esh-pve-nas
Prime's ruling 2026-10-13. esh-nas (10.0.50.50) had no infra-ops user, so key
login was refused. The same steps as bootstrap-infra-ops-user.yaml (uid/gid
850, fleet key, NOPASSWD sudoers with log_output and the BULK_DATA carve-out,
visudo-validated) ran inside the container from its host. Verified with a
direct ssh infra-ops@10.0.50.50 and sudo -n whoami = root.
2026-10-13 13:25:21 -07:00
vh cd3c4b8be9 feat(gitea-runner): opt-in fast runner on nh3-docker (label nh3-docker), no docker socket in jobs
Prime's CI fix A. ana-docker's runner sits on HDD mirrors with no SLOG; synced
8 KiB writes measured ~34 ms there vs ~2.7 ms on nh3-docker (same probe, one run,
n=3 each). Repos opt in with runs-on: nh3-docker; the runner carries no
pfi-fleet label. Its config is tighter because nh3-docker also runs NH3 DNS and
the post office: docker_host '-' (verified absent in the job, present on the
ana-docker baseline), capacity 1, jobs capped at 4 GiB / 6 CPUs, a network per
job. The deploy playbook takes runner_config; each runner's homepage name is
its runner name.
2026-10-12 17:27:46 -07:00
vh 70b7281182 ops(albok): vh joins every albok read group; macOS withdrawal-test run staged
Prime, 2026-10-09: "vh gets albok-everything". playbooks/nh3-albok-vh-viewer.yaml
joins vh to albok-read, albok-personal and albok-feedback on nh3-albok and
authorizes his four Macs' public keys for vh; verified at a fresh login that
every wing's .browse/ lists and reads.

The withdrawal harness gains a macOS driver (run-mac.sh: every mountable FUSE
backend, a read-only smoke of the real wings, results copied back), env
overrides for a non-container client, and summarize.py. Sanity-checked as vh
from the Linux container before staging.
2026-10-09 22:45:52 -07:00
vh c35f6510bf feat(fv-ml1): rebalance GPUs 0/1 by usage; contexts to published maximums; gen-small default output cap
Layout: GPU 0 = cyberprev (KV 16 -> 15 GiB, 1.29x), erp-seat (moved from
GPU 1, KV 8.49 -> 5.5 GiB, 1.32x), voices (ctx 8,192 -> 40,960, its native
maximum, KV 6 GiB, 1.07x), coder (moved from GPU 1, ctx 16,384 -> 32,768,
the card maximum, 2.35x), parakeet-nemo. GPU 1 = gen-small (moved from
GPU 0, 5.5 GiB, 1.76x), meromero-rp (22 GiB, 1.32x), intern-decision.

Sizing rule: KV >= 1.25x max context rounded up to 0.5 GiB, checked against
7 days of LiteLLM traffic and the engines' own KV-usage logs. Each card keeps
spare memory for its bursty tenant (parakeet's cap on GPU 0, intern-decision's
32k call on GPU 1). YaRN extensions are not enabled: they do not fit, and the
Qwen cards warn static YaRN degrades short prompts.

gen-small gets --override-generation-config max_new_tokens 32768. It applies
only to requests that omit max_tokens (vLLM get_max_tokens) and answers 9
runaway ~260k-token replies in 7 days; the longest normal reply was 6,357.

Applied with playbooks/fv-ml1-rebalance-20261005.yaml (boot-check guard and
health/KV verification per seat); all nine seats healthy, gateway smoke
tests pass.
2026-10-05 22:51:17 -07:00
vh 6851ecbd0b fix(fv-ml1): pin vLLM KV in bytes on meromero (22 GiB) and coder; gen-small KV 8 -> 5.5 GiB
meromero-rp sized its KV from --gpu-memory-utilization, and vLLM's profile
counts co-booting neighbours as its own non-torch memory. Three boots gave
20.37 / 26.74 / 29.04 GiB of KV, and the 2026-10-05 Docker-move restart cut
GPU 1 free memory to 3,981 MiB, below intern-decision's 32k-token budget.
Coder read negative KV on every co-boot. Both are now pinned, so util only
gates the boot check (meromero util 0.52 -> 0.47, just above its footprint).

GPU 0 plateaus at ~60 MiB free 1-2 days after a boot (Beszel), so
gen-small's KV drops to 5.5 GiB (461,215 tokens, 1.76x) and parakeet-nemo
stays on GPU 0 with room for its 3,840 MiB cap. GPU 3 stays clear.

Applied with playbooks/fv-ml1-kv-pin-20261005.yaml: GPU 1 free 3,981 ->
11,801 MiB, GPU 0 495 -> 5,039 MiB right after boot; all nine seats healthy.
2026-10-05 21:40:28 -07:00
vh e369b1c769 feat(nh3-svos): credential-free SVOS offline shell before Basic auth; contract static routes; backup drill results
svos-dev: the service worker fetches its hash-pinned INVENTORY with credentials omitted. Exactly those files plus /sw.js are now served before basic_auth (GET/HEAD, exact files). The list is generated from the release's sw.js by svos-front-inventory, which hash-checks each file and validates before reload. Gated static routes follow the web/README hosting contract: missing assets 404 instead of index.html, other document requests get index.html, everything else 404, unversioned files no-cache. acceptance-matrix.sh: 35/35 live; 16 FAIL against the previous config (positive control). git added to the nh3-svos playbook (kernel's remote check). Live backup drill: quiesced backup ~1-2 s outage; a refused freeze does NOT abort vzdump (unfrozen backup, job reported successful); both images restore-mounted read-only and verified.
2026-10-05 19:58:19 -07:00
vh d7e845eb64 feat(nh3-svos): VM 111 on nh3-pve-2 for the new SVOS (svos-dev, Prime); generic cloud-init VM playbook 2026-10-05 18:59:53 -07:00
vh d3f59aa2c5 feat(albok): move albok-service to a dedicated VM, nh3-albok (VM 110 on nh3-pve-2)
albok-service leaves nh3-docker before unit 4 puts the personal vault and
licensed books into restricted wings. On nh3-docker, portainer, portainer_agent,
dockge (socket rw) and dockerd on plaintext tcp/2375 were each root over
/srv/albok. The new host runs only albok-service: no socket containers and no
TCP Docker API, so its Homepage tile is a static services.yaml entry.

- nh3-pve-2: vmbr0 made VLAN-aware (applied by reboot, no guests running);
  pbs-ana storage in the new namespace nh3-pve-2, plus a 21:30 vzdump job
  (pbs-nh3's recursive sync mirrors it)
- VM 110 nh3-albok, 10.100.50.44 on VLAN 50, Debian 13 cloud-init, 4 vCPU
  x86-64-v3 / 12 GiB / 100 GB; built to fleet conventions (vh 1000,
  infra-ops 850, docker 851)
- compose: mem_limit 4g -> 8g, homepage labels removed; host port 8392 kept
- cutover by tar stream, verified by digest (contents plus owner, group, mode,
  mtime and type identical); albok.nh3.internal now points at nh3-albok
- old /srv/albok stays on nh3-docker (container removed, compose renamed)
  until acceptance and the operator's approval to delete it
- sudo-io archive hosts and ops-log STACK_HOSTS include nh3-albok
2026-10-04 23:33:24 -07:00
vh d9d1dd9325 feat(sudo-io): stop recording bulk data (sudoers BULK_DATA !log_output fleet-wide), drop >64 MiB streams from archives, prune existing bulk recordings 2026-10-04 22:55:49 -07:00
vh 73129a0564 ops(esh-pve): upgraded to PVE 9.2.21 / kernel 7.0.14-20 (one reboot, Prime on site); ESH cluster uniform on 9 2026-10-04 07:45:05 -07:00
vh 9a6b23581e ops(esh-pve): PVE 9 prep done (8.4.21, microcode, systemd-boot out, removable shim, NIC pins, vmbr0 bridge-ports fixed) 2026-10-04 07:32:13 -07:00
vh b67ab48833 ops(pfi-pve): upgraded to PVE 9.2.21 / kernel 7.0.14-20; reboot test + post-check recorded; pre-reboot checks corrected for PVE 9 2026-10-04 07:16:21 -07:00
vh 075ba0930e ops(pfi-pve): PVE 9 prep done (8.4.21, grub-efi-amd64 on the UEFI/LVM box, microcode, NIC pins); upgrade plan + playbooks 2026-10-04 06:11:13 -07:00
vh 453c927ace feat(qnap-lcd): Proxmox-correct front-panel LCD menu for esh-nas-pve; albok redeploy note 2026-10-03 16:14:43 -07:00
vh 241a668f3c ops(esh-nas-pve): PVE 9 upgrade playbook (snapshot, trixie sources, detached dist-upgrade) 2026-10-03 15:52:44 -07:00
vh f648e18b1a ops(esh-nas-pve): PVE 9 no-downtime prep (8.4.21, microcode, systemd-boot out, NIC pins) 2026-10-03 15:31:00 -07:00
vh d9e09fed14 ops(nh3-pve): PVE 9 post-upgrade fixes (7.0 headers + nvidia DKMS, microcode, enterprise repo off); old VM102 snapshot removed 2026-10-03 14:09:41 -07:00
vh 4b2a81bf39 feat(pve-nag-patch): remove the subscription popup on the PVE hosts, re-applied by an apt hook 2026-10-03 13:27:11 -07:00
vh 9b1cfa7344 feat(nh3-pve-2): drop stale 1 TB boot entry, wipe it into LVM-thin storage vmstore 2026-10-03 13:17:28 -07:00
vh 00a6fbd849 feat(cira-tunnel-watchdog): auto-drop stale AMT CIRA tunnels on MeshCentral's MPS 2026-10-03 12:34:35 -07:00
vh 8a4dbf2341 fix(albok-service): 0.1.2 on nh3-docker (fd leak); add restricted wing personal/agent-feedback; Nemi credentials 2026-10-03 02:54:18 -07:00