Commit Graph
569 Commits
Author SHA1 Message Date
vh 11174ffea1 memory: GPU 0 room can come from trimming gen-small KV (Prime); donor analysis 2026-09-30 16:08:10 -07:00
vh aa17f64bd1 memory: warm-up cache audit passed; parakeet seat A/B in flight 2026-09-30 16:05:56 -07:00
vh 8c68bacf2e scriberr: carry patch 0002 (gap retry + PARAKEET_MODEL_PATH), live as dropout2
Prime: Scriberr gets the basic fix, v3 stays (no NeMo 3.0.0 surgery). 0002 moves
from proposed/ into the carried set; scriberr-rebuild now applies 0001+0002 by
default (suffix dropout2) and its memory budget becomes a 5,600 MiB regression
guard (Scriberr is on GPU 3). Live on fv-ml1 1602: scripts rewritten from the
patched embed, a 20-min file at 5,502 MiB with retried_gaps reported.
2026-09-30 16:03:07 -07:00
vh 38015a1977 docs(scriberr): Parakeet dropout investigation; proposed 0002 (gap retry + model path)
Prime's ask (via the coordinator): investigate the "Parakeet skips
stretches of speech" finding, including other Parakeet weights.
Investigation only; nothing deployed.

Against ground truth (official SCOTUS transcript, Gutenberg #38916) the
drops are real: production v3 loses 140 / 66 clean words per transcript on
the two public files and ~50 on each private one (Whisper-referenced,
Canary-confirmed; adjudicator 129/129 correct on the calibration). Cause:
the v2/v3 0.6B weights collapse deep inside long full-attention windows;
the encoder output is degraded, the audio alone transcribes fine, and
1.1B TDT/RNNT/CTC and CTC-0.6B never do it. Decoding (CUDA graphs, greedy
variants, max_symbols, beam), slice length, local attention, loudness,
resampling and a noise floor do not fix it. Controls: A-vs-A, silence
positive control (>=15 words 36/36), null control, bootstrap floor.

Proposed patch 0002 re-transcribes >=3 s stretches where the audio holds
speech but no word came out (-80 to -90 % lost words on all four
recordings, lower WER, no invented text, +10 MiB) and adds an explicit
PARAKEET_MODEL_PATH with the loaded model recorded in JSON and ModelUsed.
Reviewed at high effort, all findings fixed; built and tested as
scriberr:local-blackwell-a353078-dropout2, not deployed.

scriberr-rebuild: --patches takes DIR[:DIR...]; embeds and seam-checks
both Parakeet scripts (seam-check --standard for the short-audio one).
2026-09-30 15:48:02 -07:00
vh 1189adbf18 memory: intern-decision warm-up cache tasked to infra-hermes 2026-09-30 15:44:45 -07:00
vh af450f4ef7 docs(intern-decision): Triton warm state survives restart, lost on recreate; ~16 x 2048-token buckets, ~2 min full warm-up 2026-09-30 15:41:55 -07:00
vh 6b201e1d4a scriberr to fv-ml1 GPU 3 (on-demand, steps aside to irv-ml1 A6000); intern-decision 32k-token calls (cap 14.4 GiB)
Prime 2026-09-30: move scriberr to GPU 3 and extend the Jev endpoint to 32k tokens.
Scriberr holds 0 VRAM idle; verified a 20-min job on GPU 3 at 5,496 MiB. With GPU 1
freed, intern-decision's measured card peak at MAX_TOKENS=32768 is 15,220 MiB against
a 15,437 MiB budget (n=3, 1 and 16 questions); 32,769 tokens is refused 422 up front.
JevBench v1.2.16 via /v1/systemone unchanged: 202/231, 0 diffs vs the bench.
2026-09-30 13:35:32 -07:00
vh e3dbb08d84 memory: true Jev is text-only (official docs); our real Jev gap is context (7,168 vs 64k tokens) 2026-09-30 13:09:44 -07:00
vh 1faadb45f6 memory: intern-decision 0.1.2 live, re-audit passed 2026-09-30 13:06:53 -07:00
vh da50696fe8 memory: intern-decision /v1/systemone live (0.1.1), infra-ops audit passed; 2 low findings to infra-hermes 2026-09-30 13:01:05 -07:00
vh 579b1f5d42 memory: Jev /v1/systemone tasked to infra-hermes; infra-ops audits 2026-09-30 12:39:16 -07:00
vh 2734cbf575 memory: losing Jev candidate weights deleted; intern-decision Jev-API status (model yes, service no) 2026-09-30 12:34:49 -07:00
vh 3c5f1ea803 docs(scriberr): slicer patch live on fv-ml1 as local-blackwell-a353078-slicer1; live GPU 1 peak 5,496 MiB
Deployed 2026-09-30 1211 PT by pointing SCRIBERR_IMAGE at the patched tag (.env backed up as .env.bak-20260930-pre-slicer1; rollback is the unpatched scriberr:local-blackwell). PrepareEnvironment rewrote the env's parakeet_transcribe_buffered.py from the embed (sha256 matches the patched source). One live run on GPU 1 beside intern-decision peaked at 5,496 MiB. Memory records the open Parakeet mid-chunk dropout finding and the held upstream PR.
2026-09-30 12:14:09 -07:00
vh ab62644315 memory: scriberr pause-aware slicer build in flight 2026-09-30 10:53:03 -07:00
vh 128d1d847c semif: container removed after replacement by intern-decision; rollback is compose up -d 2026-09-30 09:49:30 -07:00
vh 66034cc69e scriberr: correct the GPU 1 budget — nvidia-smi Free is 15,442 MiB, not total−used; 70 MiB spare beside intern-decision at 9.0 GiB 2026-09-30 09:39:12 -07:00
vh cc211ef10c scripts: wt-h2-count.py — worldtree-dev's U11b H2 gate check (verbatim); rehearsed on copies, 0 on both 2026-09-30 09:07:50 -07:00
vh bb806e3596 memory: U11b step-5 auto-trigger (3 PASS) is mine; semif->intern-decision in flight; scriberr GPU budget 2026-09-30 09:03:21 -07:00
vh d9bbaa07b2 memory: Jev bench done — Intern-Decision-4B is the SemIf replacement candidate 2026-09-30 05:01:30 -07:00
vh 9a6ac59da7 memory: U11b legacy-memory archive done (dedicated restic repo, drill passed); destroy-by 2026-10-30 runbook 2026-09-30 03:01:54 -07:00
vh 37d0b34682 memory: U11 daily off batches live (infra-hermes), first PASS; U11b /embed usage + legacy inventory + archive plan 2026-09-30 02:24:28 -07:00
vh 6dd9ed2964 memory: U11a overnight log sweep clean but traffic-free; re-sweep TODO 2026-09-30 01:55:01 -07:00
vh 21e064b588 memory: scriberr 35-min retry succeeded with SemIf offline 2026-09-30 01:39:07 -07:00
vh 9daf43683d semif: offline by operator ruling — scriberr needs GPU 1 headroom
Stopped (not removed) 2026-09-30 0135 PT. Scriberr's Parakeet path hardcodes
5-minute slices; attention memory is quadratic in slice length, so a long file
needs >6 GB and hit CUDA OOM with SemIf resident (~6.7 GB free). GPU 1 now
81,806 MiB used. Durable fix (shorter scriberr slices) deferred to later.
2026-09-30 01:36:18 -07:00
vh 060fd92e4e memory: Worldtree U11a — personal flipped to legacy off and verified 2026-09-30 01:21:13 -07:00
vh 9905b76404 memory: Worldtree U11a — Prime ruled legacy off; demo flipped via config repo and verified; personal pending 2026-09-30 01:17:23 -07:00
vh f9fae53599 memory: snapshot — Worldtree U10 done, U11a prepped, U8 wrapper for infra-hermes in progress; Blender extensions live; Bonsai spike closed; 3 entries archived 2026-09-29 23:42:03 -07:00
vh 274b8175c0 memory: Worldtree U11a prepped (staged config on demo, U8 window harness facts) 2026-09-29 23:33:24 -07:00
vh 47d8fd6139 memory: Worldtree U10 backfill committed on personal (797 filed) 2026-09-29 10:07:45 -07:00
vh 40427b4822 memory: U10 personal presync; demo U9 forget-policy gap fixed; pinned chroma in container layer 2026-09-29 08:55:16 -07:00
vh b7b6e01d6e memory: Worldtree U10 backfill committed on demo; memory_tagger config sync needed on personal 2026-09-28 23:52:05 -07:00
vh ea5d8dfd4e docs(blender): desktop Blender + MCP registration per working session, not per task (Prime 2026-09-28) 2026-09-28 16:36:44 -07:00
vh 2296fbba12 memory: Bonsai fork pin has a second copy on the smithy NAS 2026-09-28 15:44:44 -07:00
vh 60e6ed5b4e docs(fv-ml1): /tank/aimodels/mlx and the PROVENANCE-beside convention; Bonsai weights acquired 2026-09-28 15:43:52 -07:00
vh fd20183cbb feat(blender): extension set live in the GUI; MCP acceptance 9/9, probe made safe-mode compliant 2026-09-28 15:08:22 -07:00
vh 056433555f memory: Bonsai MMVQ-threshold follow-up (N=8 1.06x -> 1.21x); build image removed 2026-09-28 14:33:39 -07:00
vh 841f05de7a memory: Bonsai spike result on fv-ml1 GPU 3; blender-run --cpu 2026-09-28 14:23:21 -07:00
vh 59d436cc55 memory: first nightly restic run on repository-file verified all-green on all 8 hosts 2026-09-28 12:54:29 -07:00
vh 83dc497b40 feat(blender): pinned extension set in a read-only System repo, for the GUI and blender-run --extensions
Blender is now a mandatory stage in draupnir's pipeline (Prime, 2026-09-28), and draupnir asked
for eight add-ons from extensions.blender.org: SurfacePsycho 0.10.4, CAD Sketcher 0.32.1,
3D-Print Toolbox 1.4.1, STEP Importer 1.2.1, Bool Tool 2.1.0, LoopTools 4.7.7, MeasureIt 1.8.4,
3MF Import/Export 2.7.7.

- stacks/blender/extensions.lock pins each by version and archive sha256.
- scripts/blender-extensions sync builds fv-ml1:/tank/blender-extensions/5.2/system with Blender's
  own install-file, pre-warms and byte-compiles it, checks a read-only enable, then swaps it in.
  It refuses while the GUI or a blender-run job holds the old directory.
- conf/scripts/startup/fleet_extensions.py enables every package in the System repo: in a timer
  in the GUI (after the prefs load), and as --python ahead of the caller's args in
  blender-run --extensions (a failed enable exits 1 before the caller's script).
- It also patches SurfacePsycho's sp_overwrite_segment_selection from eval() to literal_eval():
  the eval walked past MCP safe mode (control: unpatched ran code, patched refuses).
- blender-run: --extensions (bind mounts via --mount so a missing source fails instead of being
  created); USER/LOGNAME set, which CAD Sketcher's getpass needs.
- compose.yaml mounts the repo read-only and the hook into the GUI container. NOT yet deployed.
- scripts/blender-probes/extensions_acceptance.py: one operator run per add-on, safe-mode
  compliant. Headless 8/9 online and with --network none; CAD Sketcher sketching is GUI-only.
  A Python audit hook saw no network/process events (positive control fired).
2026-09-28 12:50:57 -07:00
vh 1756b891c6 memory: snapshot — Zigbee2MQTT + HA-MQTT route fix, Blender on GPU 3 (MCP per task + blender-run), semif 0.1.4, Worldtree reward config live and pushed; 3 entries archived; next: 2026-09-28 restic freshness check 2026-09-28 09:57:41 -07:00
vh 6f0c480693 memory: Blender access stays on the shared fleet login; batch path blender-run (Prime) 2026-09-28 08:50:09 -07:00
vh 2b38cfd1b0 memory: worldtree-instance-configs pushed (Prime) 2026-09-27 15:05:15 -07:00
vh 179309df3f memory: Worldtree reward config live on demo+personal; Blender MCP registration is per task (Prime) 2026-09-27 14:39:43 -07:00
vh ac1cd29afa feat(blender): agent control via mcp-for-blender (in-container, ssh stdio)
The MCP server (mcp-for-blender 2.1.1, frozen requirements) runs inside the Blender
container. Its add-on is vendored at upstream 41a18432 (MIT) and started by a
startup hook. scripts/blender-mcp carries the stdio over ssh + docker exec, so the
add-on socket, which runs arbitrary Python with no auth, stays on the container's
localhost with no published port. It also runs there because viewport screenshots
need a filesystem shared by server and Blender. Telemetry is off and safe mode is
on. The hook also defaults Cycles to OptiX on GPU 3, because safe mode forbids
agents from touching preferences.

Verified end to end from nh3-dev: 36 tools; a GPU render of an agent-built scene;
a viewport screenshot; and safe mode refusing 'import os'. Blender left down
(on demand).
2026-09-27 14:34:07 -07:00
vh 0b8632a7ed fix(esh-docker-vm): /32 route to Home Assistant over macvlan-shim
The shim holds 10.0.50.47/24, which gives two equal connected 10.0.50.0/24 routes,
and ens18's wins. Host-to-HA traffic therefore left via the macvlan parent and was
dropped. HA lost MQTT to the broker on this host on 2026-08-19, 2026-09-21 and
2026-09-25 (the last lasted two days). This adds an ifupdown if-up.d hook that
routes 10.0.50.46/32 via macvlan-shim; /etc/network/interfaces is not edited.
Verified: the route resolves via the shim, the host pings HA, HA reaches :1883,
and HA reconnected to the broker. Diagnosis by ha-dev.

Also corrects the zigbee2mqtt acceptance note, which had wrongly reported HA as
connected.
2026-09-27 12:50:05 -07:00
vh c7b32418e1 feat(zigbee2mqtt): Zigbee2MQTT 2.14.1 on esh-docker-vm, replacing HA's ZHA
Requested by ha-dev; approved by Prime in this session. Radio: SLZB-MR1U chip 0
(EFR32MG21, EmberZNet 8.0.2) at tcp://10.0.90.10:6638, adapter ember. A fresh
network was formed on channel 25, PAN 0xCFF4. The broker is reached as mosquitto
user zigbee2mqtt, with HA discovery on homeassistant/. The frontend on :8099 is
token-protected.

State and the network key stay host-only in /opt/docker/data/zigbee2mqtt
(root 0700, restic). The repo carries only compose, .env.example and the README.
configuration.yaml refers to the secrets as !secret.yaml, and those references
survived Z2M's v4->v5 settings migration.
2026-09-27 12:43:49 -07:00
vh 47cad33dd1 fix(semif): 0.1.4 — object states ending in ) ; } no longer 422 (INV-7)
SemIf's shared scorer trims one token at the state boundary. When an object
state's last value ends in ')', ';' or '}', the JSON that follows re-merges two
tokens back, so score_shared refused the request with 422. The engine now wraps
semif_phase1.shared._state_prefix to keep only the tokens the full prompts
share. Each row scores the same token sequence; only the prefill/suffix split
moves.

Startup proves the fix is in effect, not just installed (heid bug hunt SKAL,
folded). It checks that the hook is callable and is what score_shared resolves,
that an ordinary state keeps upstream's whole prefix, and that a merge-prone
state scores through the shared path.

Real tokenizer: 154 states, 23 refused before and 0 after, with no ordinary or
authored144 prefix changed. Acceptance: 144/144 parity. Shared vs direct 71/72;
the miss is a bf16 tie that flipped across a plain restart (see README).
2026-09-27 10:23:14 -07:00
vh 7e11cf247b spike(semif): SemIf as Cicada's mood source is slower and less apt (no service change)
Against talk /face's guided pose (first paragraph 246 ms median), SemIf in
parallel adds 32 ms and SemIf-first adds 94 ms (n=72 each, noise floor 16.5 ms).
Removing the pose header saves only ~31 ms, and SemIf shares GPU 1 with the LLM.
Acceptable pose 67% vs 92% on clear-emotion lines, and the mood carried through
mundane follow-ups 7/15 vs 14/15. SemIf gestures far less (13% vs 58%).

README: rotations cost options^2 in suffix tokens, and /decide/shared returns
422 when an object state's last value ends in ) ; or }.
2026-09-27 09:47:28 -07:00
vh e268ff7c99 spike(semif): consumer fit for Wyrd scene change and Cicada affect gate (no service change)
Harness consumer_fit.py runs a consumer's per-turn decisions over hand-labelled
cases, with rotations, a content-free null control and tagged positive controls.

Cicada: an input-only 'does this earn a visible reaction?' gate scored 30/31
with descriptive options and 19/31 with terse yes/no options. Scoped by
Cicada's 2026-09-20 ruling (affect is emitted once, no mood-ring classifier).

Wyrd: on 3 real seed graphs, the first place-change wording failed its
positive controls (1/6 moves). A location-anchored rewording scored 21/21, and
exit selection scored 18/21. Semif fits the choice, not writing the node.
2026-09-27 09:20:35 -07:00
vh 3048e4194f memory: snapshot — semif 0.1.3 live (averaging + fast kernels + bug-hunt folds), hermes-gateway restart for SVOS seat_up, overnight backups green; next: SemIf spike (scope to confirm with Prime) 2026-09-27 09:00:51 -07:00