memory: snapshot — 2026-06-11 Mac Pro migration framing + GLM thinking-off + R17 v2 corpus; archive TTS-streaming arc

archival-memory.md

@@ -295,6 +295,28 @@ _Entries moved out of persistent-memory.md to keep the active file scannable. Re
   `01KT2K2SY9N7AY69R9V0B4RXSW` to asset-engine-dev.
   _Archived 2026-06-04._
 
+- `[2026-06-01]` **Fish reference_id empty-dir fix shipped** (`c5bbb90`) — Populated
+  `references/<name>/<name>.wav`+`.lab` for all 32 voices; playbook gained
+  normalize-step + A/B smoke gate. glados got a real transcript (ASR'd via
+  Parakeet): the Portal "Welcome to test chamber 4" lines.
+  _Archived 2026-06-11._
+
+- `[2026-06-02]` **Chatterbox → main TTS engine; build custom `chatterbox-fast`
+  streaming container.** Workload = single-stream interactive. GPU placement:
+  3090 (device 0) if it fits else A6000 (device 1). Cutover: parallel catalog
+  entry, burn in beside live `chatterbox`, then flip. Streaming = adaptive
+  buffer-ratchet chunking (native frame-streaming abandoned). Since extracted to
+  its own repo `vh/chatterbox-fast` (auto-memory `reference_chatterbox_fast_repo`).
+  _Archived 2026-06-11._
+
+- `[2026-06-02]` **Sentence-splitting loses quality (operator-corrected).** Claimed
+  naive sentence-level streaming has "zero quality loss" — WRONG. The T3 AR backbone
+  conditions prosody on the WHOLE text; splitting loses cross-sentence prosodic
+  context even though voice timbre stays (reference-conditioned). No *artifacts* ≠
+  no *quality loss*. Hence adaptive-chunk design (maximize context per chunk subject
+  to latency budget), not fixed per-sentence splits.
+  _Archived 2026-06-11._
+
 ## Tried and abandoned (archived)
 
 - `[2026-04-30]` task-board workflow with
@@ -629,3 +651,33 @@ _Entries moved out of persistent-memory.md to keep the active file scannable. Re
   same clips gave the correct answer (clone 0.79 to real Imogen, 0.10 to
   default). Use ECAPA, not resemblyzer, for speaker-verification.
   _Archived 2026-06-04._
+
+- `[2026-06-02]` **Fish (fish-s2 / OpenAudio S1-mini) progressive streaming — SHELVED
+  (sub-realtime).** Benched RTF on A6000: 0.72x/0.82x/0.86x, mean 0.80x = sub-realtime,
+  so client-side chunking starves. Root cause: Fish only chunks on `<|speaker:X|>` tags;
+  plain text → batches=[whole text] → all semantic tokens generate before any audio
+  (chunk_length inert). Plus kui/ASGI StreamResponse doesn't flush (fix = anti-buffering
+  headers, kept on file). STANDING REVISIT TRIGGER: when an RTX Blackwell Pro lands → bench
+  fp4-quantized Fish; if RTF > ~1.5x give it the chatterbox-fast treatment. For now Fish is
+  a buffered catalog entry (great for SAVED gens, not live-audition).
+  _Archived 2026-06-11._
+
+- `[2026-06-02]` **Context-priming at chunk joins (chatterbox-fast §1.6) — ABANDONED
+  (discard-cut leaks the prefix).** Prepend prior sentence, generate `prefix+content`,
+  discard prefix audio. Built + shipped (`d707439`), live-A/B'd, reverted (`090e70a`).
+  Kill: `generate()` returns one waveform with NO marker where the prefix ends, and the
+  model renders the same prefix with different timing solo vs followed-by-content — so
+  locating the cut is a guess that left a whole clause in the output (operator caught
+  "...without a trace of sarcasm" spoken twice). Reliable cut needs token-level boundaries
+  or per-chunk forced-alignment (heavy). Coherence loss at joins = ACCEPTED limitation.
+  _Archived 2026-06-11._
+
+- `[2026-06-02]` **Native frame-level streaming on Chatterbox-TURBO — ABANDONED (turbo
+  isn't built for streaming).** (1) `S3GenStreamer` is docstring-only (unimplemented).
+  (2) `flow_inference(finalize=False)` is buggy (lookahead trim removes frames from `h`
+  not `h_masks`/conds → shape mismatch); a 1-line patch makes it RUN. (3) BUT the flow
+  encoder uses full-context attention (`static_chunk_size=0`) → incremental decode is
+  prefix-unstable (adding tokens re-attends, shifts earlier mel ~0.30-0.39 vs one-shot).
+  (4) Forcing `static_chunk_size>0` didn't stabilize it. True sub-second frame-streaming
+  needs deep attention surgery with quality risk — not worth it. → adaptive-chunking instead.
+  _Archived 2026-06-11._

persistent-memory.md

@@ -1,6 +1,6 @@
 # Persistent memory — eshpfi-management
 
-_Last updated: 2026-06-05_
+_Last updated: 2026-06-11_
 
 ## Repo purpose
 
@@ -24,9 +24,12 @@ Sister repos (separate gitea repos, deployed by playbooks here):
 | `vh/mead-hall` | Bifrost tool-provider sidecar (port 5173 on dev VM 10.100.10.50) | push-to-main → CI deploys (2026-05-16) |
 | `vh/skaldsong` | Wizard + reader surface (port 8300, ana-docker, registry-pull pattern) | push-to-main → CI deploys (2026-05-19) |
 | `vh/worldtree` | Conversation API (corviduo-dev demo :8080 / personal :8081 / pinned :8082) — Heimdall auth, Bifrost integration | push-to-main → CI deploys |
-| `vh/volva` | Codex peer agent on althing bus (single-turn oracle, systemd daemon on nh3-dev) | manual install via `deploy/volva.service` (2026-05-18) |
 | `vh/yt-voice-clipper` | YouTube → diarized voice-clip dataset builder + audition console (irv-ml1 :8000) | push-to-main → **gitea-webhook auto-deploy** to irv-ml1 (2026-06-03) — see `docs/runbooks/ytvc-autodeploy.md` |
 
+(`vh/volva` + Heid were re-architected from systemd daemons to Claude Code
+session orchestrators 2026-06-08; their nh3-dev `.service` units were removed —
+no longer deployed sidecars here. See Recent decisions.)
+
 - **Two-layer backups** — Backrest orchestrates restic for file+DB (5
   fleet repos, daily 01:00 PDT); PBS-ANA primary + PBS-NH3 DR mirror for
   VM images. ana-nas is the SPOF for postgres + PBS-ANA datastore +
@@ -88,29 +91,85 @@ Sister repos (separate gitea repos, deployed by playbooks here):
 
 ## Current state / in-flight
 
-_As of 2026-06-05:_
+_As of 2026-06-11:_
 
-- **Granite-FP8 + observability session — all LIVE & committed (`34a43a0`, `9171e6a`).**
+- **Mac Pro migration is the big open project** — `migration-plan.md` (repo root) frames a
-  - **Granite 4.1 8B FP8 is the production summarizer** (`vllm-granite` :8004, ana-ml2 GPU 1,
+  WORKSTATION-ONLY migration of nh3-dev's dev env to an M2 Ultra Mac Pro Rack, racked in NH3 on
-    50K ctx, CUDA graphs) — replaced phi4-mini, validated by brokkr (valid_format 1.0, FP8 stays).
+  the 10.100 subnet. nh3-dev wears two hats; only **Hat 1** (the ~40 repos, all of `~/.claude`,
-  - **LiteLLM gateway** (:4000) routes `granite-4.1-8b`→vLLM (explicit entry shadows the `*`
+  dotfiles, toolchain) moves. **Hat 2** fleet sidecars (egress SOCKS5 proxy, mead-hall, ttyd
-    wildcard) + **Langfuse v3 wired** (ana-docker:3001, "LLM Throughput (tok/s)" dashboard built).
+  seats, `althing-forseti`) STAY on the Linux VM. **Phase 0 is the only time-sensitive step**:
-  - **GPU-1 retuned** (trio over-provisioned KV trimmed) → granite runs with CUDA graphs + ~10 GB
+  push the pushable repos + confirm which of ~6 no-upstream repos are local-only. Cutover =
-    free as a future Granite-text-LoRA hedge. Streaming through the gateway confirmed (TTFT 0.24s).
+  `rsync` the working trees, **NOT re-clone** (re-clone would lose unsaved work in ~15 repos).
-  - **ana-docker pruned** 77 GB (unused images + build cache; disk 83%→49%) to fit ClickHouse.
+  macOS gotcha pre-flagged: `/home/lkraven` → `/Users/lkraven` repath. Everything else waits on
-- **Worldtree summarizer repoint — NO instance change now; DEFERRED to Worldtree #254** (see Recent
+  the hardware landing in the rack.
-  decisions). worldtree-dev will ping with the providers.yaml + consumer config when #254 un-holds;
+
-  infra-ops applies to the personal/demo/pinned bind mounts (vh@10.250.50.152, `/opt/worldtree*/config`).
+- **Two commits/branches committed-not-pushed** (operator's push call): `95b2701` (GLM thinking
-- **Commits unpushed** (`34a43a0`, `9171e6a`, nevermore `d3e19b8` in its repo) — operator's call to push.
+  config) on main; and the R17 v2 results branch `r17-v2-characterization` on the irv-ml1
-- **Operator flagged "new work to do"** for the next session — this snapshot is the handoff.
+  checkout (push HELD — see decisions).
-- **Disclosed-keys hygiene queue** (rotate at convenience): HF token `hf_HBl…` (lkraven's), `/tmp/
+
-  wt-personal-skaldsong-prod.key`, Worldtree `Z_AI_API_KEY`, Gitea runner reg token, `MINIFLUX_PASSWORD`
+- **R17 v2 corpus characterization DONE on irv-ml1** (brokkr, ~8.5h): 119 clips → **18 MULTI /
-  (leaked twice). (sk-corvid + the langfuse/vastblueai-gateway keys are dev-enclosed — leakage deprioritized.)
+  101 ELIGIBLE** (Stage-0 drop list) + 19936 candidates + ASR speech + blend verdicts. Outputs
-- **Still open from prior:** clean legacy `news-digest` on ana-docker; watch nh3-nas `/volume1`; **pin
+  on the local branch; **push HELD** — no-push rule AND the `speech.json` are ASR transcripts of
-  llama-swap to GPU 0** for clean GPU-1 separation; the `docker push 60s ceiling` mystery uninstrumented.
+  the intimate-audio batch (content-exposure call). Awaiting brokkr-collect or explicit push approval.
+
+- **pi coding agent + GLM 5.1 harness live on nh3-dev** — `glm` launcher runs pi against `glm-5.1`
+  (thinking-off) via the LiteLLM gateway; `glm-5.1-reasoning` for opt-in thinking.
+
+- **Worldtree config-propagation lane is mature + humming** — pre-merge delta-ping → sync-on-merge.
+  All of v0.33.x→v0.34.1 reconciled to the demo+personal bind-mounts; demo/personal both on v0.34.x,
+  pinned stays pre-cutover (full migration at its next re-image).
+
+- **Disclosed-keys hygiene queue** carries forward (rotate at convenience): HF token, the wt-personal
+  keys, Gitea runner reg token, MINIFLUX_PASSWORD. `ZAI_API_KEY` (z.ai MCP web-tools key) now also
+  lives in `~/.config/secrets/env.sh` (shared from Worldtree env for the Brokkr panel).
+
+- **Still open from prior:** clean legacy `news-digest` on ana-docker; pin llama-swap to GPU 0; the
+  `docker push 60s ceiling` mystery uninstrumented.
 
 ## Recent decisions
 
+- `[2026-06-11]` **GLM thinking inverted at the LiteLLM gateway** (operator call): `glm-5.1` defaults
+  thinking-OFF; new `glm-5.1-reasoning` alias = same z.ai upstream with thinking ON (opt-in).
+  Mechanism: `litellm_params.extra_body:{thinking:{type:disabled}}` — LiteLLM `drop_params` STRIPS a
+  top-level `thinking`/`reasoning_effort`, but forwards `extra_body` verbatim to z.ai (the only channel
+  that works; verified reasoning_tokens 0 vs >0). Shared-gateway change — affects ALL glm-5.1 callers
+  (brokkr's all-models key included). (`95b2701`, auto-memory `reference_litellm_gateway`)
+
+- `[2026-06-11]` **pi coding agent installed on nh3-dev as a GLM 5.1 harness** — `@earendil-works/
+  pi-coding-agent` via **bun** (user-level; npm's global prefix is `/usr` → needs sudo, bun avoids it).
+  Config `~/.pi/agent/models.json` (litellm provider → gateway), launcher `~/.local/bin/glm` sources
+  the gateway key + selects the model. pi is OpenAI-compatible; proxy-safe compat flags for the GLM path.
+
+- `[2026-06-11]` **z.ai web-tools (regin) = z.ai hosted MCP path, NOT the `/paas/v4` Tool API.** Direct
+  `api.z.ai/api/paas/v4/web_search` → 429/1113 "insufficient balance" (coding-plan keys bill tools on a
+  separate quota path); `open.bigmodel.cn` is the China platform (404, different account). WORKS: MCP
+  streamable-HTTP at `https://api.z.ai/api/mcp/{web_search_prime,web_reader}/mcp`, `Authorization: Bearer
+  $ZAI_API_KEY` (the **MCP** key — distinct from `Z_AI_API_KEY` the LLM key). Reference impl = Worldtree's
+  Leif agent (`tools/web/zai_client.py` + `core/clients/mcp.py`).
+
+- `[2026-06-10]` **Mac Pro migration framed: workstation-only** (M2 Ultra ARM, racked NH3 on-subnet);
+  sidecars stay Linux. `migration-plan.md`. (See in-flight.)
+
+- `[2026-06-10]` **Worldtree deployed-config propagation is infra-ops's OWNED lane** (operator ruling).
+  worldtree-dev pings the config delta on every config-touching commit (pre-merge); infra-ops syncs
+  `config/*.yaml` from MERGED canonical to the `/opt/worldtree*/config` bind-mounts on demo+personal
+  (Heimdall hot-reloads policies; defaults are code-defaulted). The **v0.33.8 9-HOUR demo outage** — a
+  `model_roles.yaml` hard-startup-dep that shipped in canonical 3 releases earlier but never reached the
+  VM — is the failure mode this lane prevents. providers.yaml stays hand-tuned (artemis graft on
+  personal). corviduo-dev emergency-ops = `ssh vh@10.250.50.152` (alias doesn't resolve, lkraven denied,
+  infra-ops key excluded), docker no-sudo. (auto-memory `reference_worldtree_deploys_cicd`,
+  `reference_corviduo_dev_emergency_ops`)
+
+- `[2026-06-09]` **LiteLLM scoped virtual keys issued to consumers** (operator-authorized): `brokkr-
+  smithy` (all-proxy-models), `arbo-prompt-enhance` (comfy-dev, granite-only). Pattern: mint via
+  `/key/generate` (master `sk-corvid`), scope-restricted + rotatable, value → 600 file never the bus.
+  (auto-memory `reference_litellm_gateway`)
+
+- `[2026-06-08]` **volva.service + heid.service removed from nh3-dev** — vestigial systemd daemons;
+  Heid/Volva re-architected from Python pollers to Claude Code session orchestrators (heid commit
+  `12aa5a9`); binaries+dirs gone, volva.service was crash-looping 203/EXEC. Cleanup at heid's request
+  (the bus-content-driven sudo got the harness guardrail; operator green-lit). (`6e2f80e`)
+
 - `[2026-06-05]` **Granite 4.1 8B FP8 replaced phi4-mini as the production summarizer** (supersedes
   the 2026-06-04 phi4 decision below). Beat phi4 on precision in brokkr's R15 P03. **Staying FP8, not
   Q4/AWQ** — primary workload (agent memory + summarization) is high-concurrency, where FP8-on-Ada
@@ -190,32 +249,22 @@ _As of 2026-06-05:_
   CI/CD lane (a developer `staging/vX.Y.Z` git-tag promote), not infra-ops. They self-corrected
   to the same conclusion independently.
 
-- `[2026-06-02]` **Chatterbox → main TTS engine; build custom `chatterbox-fast`
+_40 older entries archived to archival-memory.md._
-  streaming container.** Workload = single-stream interactive. **GPU placement:
-  3090 (device 0) if it fits else A6000 (device 1)** — shared dev stack, 20.5 GB
-  3090-idle is expected residency, not a blocker. **Cutover: parallel catalog
-  entry**, burn in beside live `chatterbox`, then flip. **Streaming approach:
-  adaptive buffer-ratchet chunking** (see in-flight). Native frame-streaming
-  abandoned (Tried/abandoned). Tracked: `docs/design/chatterbox-fast-plan.md`.
-
-- `[2026-06-02]` **Sentence-splitting loses quality (operator-corrected).** I
-  claimed naive sentence-level streaming has "zero quality loss" — WRONG. The
-  T3 AR backbone conditions prosody on the WHOLE text; splitting loses
-  cross-sentence prosodic context (contextual delivery, declination, affect
-  continuity) even though voice timbre stays (reference-conditioned). No
-  *artifacts* ≠ no *quality loss*. Hence the adaptive-chunk design (maximize
-  context per chunk subject to latency budget), not fixed per-sentence splits.
-
-- `[2026-06-01]` **Fish reference_id empty-dir fix shipped** (`c5bbb90`) — see
-  in-flight + Tried/abandoned. Populated `references/<name>/<name>.wav`+`.lab`
-  for all 32 voices; playbook gained normalize-step + A/B smoke gate. glados got
-  a real transcript (ASR'd via Parakeet): the Portal "Welcome to test chamber 4"
-  lines.
-
-_37 older entries archived to archival-memory.md._
 
 ## Tried and abandoned
 
+- `[2026-06-11]` **A completion-poll `while pgrep -f <scriptname>` SELF-MATCHES its own remote shell
+  argv** — the poll's command line contains the script name, so its own `pgrep -f` always finds itself
+  → the loop never exits, the poll never fires. Use a match pattern ABSENT from the poll command (pgrep
+  the python stage, or a sentinel file), not the driver's own name. (Caught only because the operator
+  asked "check status"; the job had already finished cleanly.)
+
+- `[2026-06-08]` **Demucs `uv pip install demucs` pulls torch 2.12/torchaudio 2.11 → `ta.save()`
+  requires torchcodec → dies AFTER separating (0 stems written, rc=1).** Same class as the torch-2.12
+  torchcodec foot-gun below. Fix = pin `torch==torchaudio==2.4.1` (pre-torchcodec save backend) +
+  `UV_LINK_MODE=copy` for the EPERM-hardlink quirk. Lesson restated: validate the SAVE path, not just
+  import + GPU inference, on a bleeding-edge torch.
+
 - `[2026-06-05]` **vLLM 0.19 CUDA-graph-capture OOMs on a SHARED GPU** — it fills the KV cache to the
   `--gpu-memory-utilization` budget WITHOUT reserving graph-capture memory, so `capture_model` OOMs
   AFTER weights+KV load (model/KV log looks healthy, then crash-loops; saw 11 restarts at util 0.36
@@ -291,43 +340,4 @@ _37 older entries archived to archival-memory.md._
   — it's analysis scripts + a poetry pyproject, no buildable package (setuptools
   package-discovery error). Install the pyproject deps directly, don't build the "package".
 
-- `[2026-06-02]` **Fish (fish-s2 / OpenAudio S1-mini) progressive streaming — SHELVED (sub-realtime).** Benched RTF on A6000: 0.72x (12w) / 0.82x (30w) / 0.86x (60w), **mean 0.80x = sub-realtime**, so client-side chunking would starve (same reason chatterbox-fast needs turbo's RTF>1). Root cause of the buffering (dvalin-smithy-dev deep research, verified in our code text2semantic/inference.py L600-607): Fish only chunks on `<|speaker:X|>` tags; **plain text -> batches=[whole text]** -> all semantic tokens generate before any audio (chunk_length inert). Plus a 2nd layer: kui/ASGI StreamResponse doesn't flush (header produced t=1s, delivered t=23s) -> fix = anti-buffering headers (X-Accel-Buffering:no / Transfer-Encoding:chunked) in tools/server/views.py (kept on file, not applied). A rebuild does NOT fix this (current main same logic). **STANDING REVISIT TRIGGER: when an RTX Blackwell Pro lands in the fleet -> bench fp4-quantized Fish; if RTF > ~1.5x, give it the chatterbox-fast treatment** (client-side adaptive buffer-ratchet chunker driving /v1/tts with small text pieces). Projection: fp4 (~1/4 weight bytes, memory-bound AR decode) + Blackwell (GDDR7 ~1.8TB/s vs A6000 0.77TB/s, native FP4 cores) ~ 2-3x RTF; validate fp4 voice quality (ear/ECAPA) before committing. For now Fish stays a buffered catalog entry (great for SAVED gens, not the live-audition lane).
+_44 older entries archived to archival-memory.md._
-
-
-- `[2026-06-02]` **Context-priming at chunk joins (chatterbox-fast §1.6) —
-  ABANDONED (discard-cut leaks the prefix).** To give a chunk backward prosodic
-  context, prepend the prior sentence, generate `prefix+content` together, then
-  discard the prefix audio. Built + opt-in shipped (commit d707439), live-A/B'd,
-  reverted (090e70a). The kill: `generate()` returns one finished waveform with
-  NO marker for where the prefix ends, and the model renders the same prefix with
-  different timing solo vs followed-by-content — so locating the cut (generate
-  prefix solo → measure duration → snap to nearest energy-min pause within ±0.4s)
-  is a guess that left a whole clause of prefix in the output ("...without a trace
-  of sarcasm," spoken twice; operator caught it). A reliable cut needs token-level
-  boundaries (= the abandoned native-streaming arc) or per-chunk ASR/forced-
-  alignment (heavy, imperfect, eats the latency budget). → Coherence loss at joins
-  stays an ACCEPTED limitation; cold adaptive-chunk streaming judged "really good".
-  Scheduler-side work that DID land + survive: affordability-gated priming math
-  (a 2nd pass can't starve the buffer) — sound, but moot without a working cut.
-
-- `[2026-06-02]` **Native frame-level streaming on Chatterbox-TURBO — ABANDONED
-  (turbo isn't built for streaming).** Long R&D arc; record so it's not
-  re-derived. (1) The model's flow is CosyVoice2-derived but `S3GenStreamer` is
-  referenced-in-docstring-only (not implemented). (2) The lib's
-  `flow_inference(finalize=False)` is BUGGY: the lookahead trim removes
-  `pre_lookahead_len(3)*token_mel_ratio(2)=6` frames from `h` but NOT from
-  `h_masks`/conds → decoder shape mismatch (e.g. 656 vs 662). A 1-line patch
-  (`h_masks = h_masks[:, :, :-pre*ratio]` after the `h` trim) + sizing the
-  meanflow noise to the trimmed length makes finalize=False RUN. (3) BUT the
-  flow encoder uses FULL-context attention (`static_chunk_size=0`), so
-  incremental/cumulative decode is **prefix-unstable** — adding tokens
-  re-attends and shifts earlier mel (maxdiff ~0.30-0.39 vs one-shot,
-  irrespective of fixed-noise slicing or emit-margin). (4) Forcing
-  `static_chunk_size>0` on the 2 modules that carry the attr did NOT stabilize
-  it (decoding_chunk_size is a forward-arg, not settable via attribute). Verdict:
-  true sub-second frame-streaming on turbo needs deep model-attention surgery
-  with quality risk — not worth it. Matches research ("turbo+streaming
-  unsolved"; vLLM-turbo outputs noise; davidbrowne17 streaming fork is
-  BASE-only). → Use adaptive-chunking instead.
-
-_41 older entries archived to archival-memory.md._