vh
f76429c457
memory: snapshot — 2026-10-04 2300 (PVE 9 + PBS 4 fleet-wide, webhost destroyed, phasefinal on Pages, sudo-io archiver; 7 settled entries archived)
2026-10-04 23:03:58 -07:00
vh
d9d1dd9325
feat(sudo-io): stop recording bulk data (sudoers BULK_DATA !log_output fleet-wide), drop >64 MiB streams from archives, prune existing bulk recordings
2026-10-04 22:55:49 -07:00
vh
008bd685aa
feat(sudo-io-archive): weekly fleet archive of sudo session recordings (pack, verify, then clear); fv-ml1 HF_HOME on tank and root-pool cleanup
2026-10-04 22:48:05 -07:00
vh
626e0c7f94
memory: dragonfireacoustics-site now on gitea; fv-ml1 unused vLLM images removed
2026-10-04 17:59:21 -07:00
vh
905862a58e
ops(phasefinal-web): dedicated infra-ops-pages token; apex->www as a Cloudflare Redirect Rule
2026-10-04 17:37:32 -07:00
vh
513f6ef38f
ops: phasefinal.com moved to Cloudflare Pages; pfi-ana-webhost destroyed and removed from inventory, DNS and the FortiGate
2026-10-04 17:28:29 -07:00
vh
55e4d01b49
ops(pfi-ana-webhost): retired (VM 110 stopped, FortiGate policy 18 disabled); dragonfireacoustics.com located on Cloudflare Pages
2026-10-04 16:57:49 -07:00
vh
0e3ab447af
ops: retire VM 101 (DC) and VM 104 (nh3-laser) from running + backups; infra-ops login on pbs-ana/pbs-nh3; nh3-pve-2 follow-ups verified done
2026-10-04 16:37:15 -07:00
vh
d0c1bc628a
ops(pbs): pbs-nh3 + pbs-ana upgraded to PBS 4.2.7 / Debian 13; guest-agent driver script, plan and log
2026-10-04 11:44:57 -07:00
vh
73129a0564
ops(esh-pve): upgraded to PVE 9.2.21 / kernel 7.0.14-20 (one reboot, Prime on site); ESH cluster uniform on 9
2026-10-04 07:45:05 -07:00
vh
b67ab48833
ops(pfi-pve): upgraded to PVE 9.2.21 / kernel 7.0.14-20; reboot test + post-check recorded; pre-reboot checks corrected for PVE 9
2026-10-04 07:16:21 -07:00
vh
a4b7626a13
memory: snapshot — 2026-10-04 ~0546 (albok 0.1.6; 8 settled 09-19 entries archived)
2026-10-04 05:46:09 -07:00
vh
53b3428d13
memory: snapshot — 2026-10-03 ~2045 (esh-nas-pve PVE 9, albok 0.1.5, QNAP LCD, open calls)
2026-10-03 20:40:50 -07:00
vh
d4c5ae8d00
memory: NH3 Mac mini asleep; fix tomorrow on site
2026-10-03 16:23:15 -07:00
vh
9ba8962885
ops(esh-nas-pve): upgraded to PVE 9.2.21; /boot-LV-full trap and fixes recorded
2026-10-03 16:06:33 -07:00
vh
48f28cfe93
ops(esh-nas-pve): PVE 8 reboot test done; two restart foot-guns recorded
2026-10-03 15:46:37 -07:00
vh
5f8a1311a5
memory: snapshot — 2026-10-03 ~1440 (nh3-pve clean on 9, RustDesk 1.1.16, esh-nas-pve PVE 9 assessed)
2026-10-03 15:23:19 -07:00
vh
d9e09fed14
ops(nh3-pve): PVE 9 post-upgrade fixes (7.0 headers + nvidia DKMS, microcode, enterprise repo off); old VM102 snapshot removed
2026-10-03 14:09:41 -07:00
vh
08c3271084
memory: nh3-pve on PVE 9; post-check findings (nh3-ml1 down: no 7.0 headers)
2026-10-03 14:06:01 -07:00
vh
5abd43e1c8
memory: snapshot — 2026-10-03 in-flight (nh3-pve 8→9 under way), two-tier split of 27 entries, 7 archived
2026-10-03 13:36:42 -07:00
vh
22224b5335
ops(nh3-pve-2): register host, network/AMT/disk layout and the re-IP lesson
2026-10-03 13:17:46 -07:00
vh
00a6fbd849
feat(cira-tunnel-watchdog): auto-drop stale AMT CIRA tunnels on MeshCentral's MPS
2026-10-03 12:34:35 -07:00
vh
73da3175e2
docs(albok-service): pushed tag verified to match the deployed image
2026-10-03 10:00:03 -07:00
vh
f8290f64bf
memory: worldtree gate duty reframed as post-deletion regression check (infra-ops 01M413W4)
2026-10-03 07:50:47 -07:00
vh
3a6d997c99
ops(esh-nas-pve): tank verified healthy after two scrubs; disk kept; PVE 9 blocker cleared
2026-10-03 06:28:48 -07:00
vh
70fa0fbd53
memory: Nemi timer installed
2026-10-03 05:58:12 -07:00
vh
8a4dbf2341
fix(albok-service): 0.1.2 on nh3-docker (fd leak); add restricted wing personal/agent-feedback; Nemi credentials
2026-10-03 02:54:18 -07:00
vh
1fb36b32d2
memory: ESH tank first scrub done, second scrub running
2026-10-03 01:18:24 -07:00
vh
3a91eff2e6
memory: ESH tank scrub repairing the Aug 20 replacement's unrepaired blocks
2026-10-02 23:33:07 -07:00
vh
55004090d1
ops(esh-pve-2): register host, AMT phoning home to MeshCentral; note MeshCentral first-CIRA crash race
2026-10-02 22:25:37 -07:00
vh
6315ce38d5
memory: ESH tank verified residual, cleared, full scrub running
2026-10-02 19:26:30 -07:00
vh
62254023f3
docs: esh-pve-cluster PVE 8->9 upgrade plan (not executed); ESH tank DEGRADED finding
2026-10-02 19:20:28 -07:00
vh
5439d5fdc1
albok-service: 0.1.1 (health fix) on nh3-docker
2026-10-02 18:04:57 -07:00
vh
d3958655c1
feat(albok-service): deploy the fleet knowledgebase service on nh3-docker
...
albok-service 0.1.0 (vh/albok 0b37431), image pfi/albok-service pinned by
digest, published on host port 8392 because 8390 is the post office.
Host prep playbook creates the fixed ids (albok 1500, albok-read 1510,
albok-personal 1511) and the local store/private roots; the container
gets a mounted /etc/group and group_add so the service can resolve and
chgrp its wing dirs. The config carries a LiteLLM key scoped to
qwen3-embedding and lives outside the deploy-synced conf dir. DNS name
albok.nh3.internal.
2026-10-02 17:56:34 -07:00
vh
9c233ca255
memory: nh3-pve-2 end-of-day state and on-site reinstall checklist
2026-10-02 17:40:50 -07:00
vh
34659ae9e7
ops(nh3-pve-2): AMT 21 configured (KVM, no-consent, listener) and phoning home to MeshCentral
2026-10-02 14:54:56 -07:00
vh
e73adfe7ae
dns+memory: nh3-pve-2-amt 10.100.250.63 (UDM port 5 on nh3-mgmt, reservation)
2026-10-02 14:43:16 -07:00
vh
59c9c0715a
memory: MS-03 roles — esh-dev inherits nh3-dev's sessions; nh3-pve-2 purpose TBD by design
2026-10-02 09:17:39 -07:00
vh
536230170e
memory: one MS-03 deploys as nh3-pve-2 (AMT on DHCP for phone-home; addressing plan)
2026-10-02 09:10:31 -07:00
vh
1a2d763de4
ops(nh3-pve): AMT phones home to MeshCentral (CIRA); AMT on DHCP; LAN management now dark by design
2026-10-02 09:06:24 -07:00
vh
61ba384d28
memory: demo fix-forward 7ab6ae40 verified; deploy-gate gap filed as Worldtree #423
2026-10-02 08:24:33 -07:00
vh
1219cfaca9
memory: demo rolled back from broken c2d87263 to fa8bc51c; b193 retired-key list for demo
2026-10-02 08:17:24 -07:00
vh
8b81579bca
ops(pfi-tacticalrmm): MeshCentral to hybrid mode; nh3-pve AMT added and connected
2026-10-02 08:14:19 -07:00
vh
cd30592e55
fix(wt-memory-gate-batch): pass --legacy-mode only when the deployed harness has it
...
b193 (U11b) retired the legacy plane and removed --legacy-mode from the
harness; passing it there fails the daily batch. The wrapper now greps
core/memory_acceptance at the deployed sha and omits the flag when it is
gone, so it is correct on both sides of the b192 -> b193 deploy (checked:
7a83f2f -> passes it, c2d87263 -> omits it).
2026-10-02 07:59:03 -07:00
vh
1cc8ad3292
memory: U11b step 5 done — live legacy memory deleted on demo and personal (H2 0), stamp sent
2026-10-02 07:54:50 -07:00
vh
6e15f610bc
memory: U11b gate streak 3 of 3; step 5 held for Prime's direct go
2026-10-02 07:50:12 -07:00
vh
902e16630f
feat(dev-backup): add daily and weekly retention (48 hourly + 30 daily + 12 weekly)
...
Prime's ruling 2026-10-02. retention.py picks the snapshots to delete:
the newest 48, plus the newest of each of the last 30 days and of each of
the last 12 ISO weeks, counting only days and weeks that have snapshots.
Names that are not exactly YYYY-MM-DD_HHMM are never selected, and the
NAS side refuses any path outside that pattern. The unit fails unless
the number kept equals the number expected. Live run: deleted 1, 0
errors, 48 kept as expected.
2026-10-02 07:42:45 -07:00
vh
49c8fdf383
memory: nh3-dev snapshot waits for a natural reboot (Prime); post-boot checks listed
2026-10-02 07:36:15 -07:00
vh
6ec76bf35c
ops(nh3-dev): grow root into the 378 GB disk online; swap moved to /swapfile
...
Prime grew VM 102's scsi0 from 250 to 378 GB after the root alert hit 85%
twice in 14 h. The swap partition sat right after sda1 and blocked growth,
so the playbook moves swap to a 4 GB /swapfile, deletes sda5/sda2, grows
sda1 in place (start sector unchanged) and ext4 online, and sets initramfs
RESUME=none so boots don't wait for the vanished swap. Root is 372 GB, 58%.
2026-10-02 07:33:36 -07:00
vh
abcf4c3166
fix(scriberr): serve over HTTPS via the fleet TLS caddy so the browser recorder works
...
The in-browser recorder calls getUserMedia, which browsers refuse on http://
origins, so it sat at "Initializing recorder...". scriberr.nh3.phasefinal.com
is now fronted by the fleet TLS caddy on nh3-dev (wildcard cert) and added to
Scriberr's ALLOWED_ORIGINS; the Homepage link points at it. The plain
http://10.251.50.54:8080 URL keeps working except for recording.
2026-10-01 13:07:18 -07:00
vh