memory: snapshot — 2026-07-05 (T1 train venue = cloud-rec/operator-chose-ana-ml2-smoke; LitBench-RM up + comfyui displaced; character-rp shipped + #344; althing v2 herald/receiver systemd + PATH fix; glm-5.2 1M/128K; LiteLLM shared-param-mutation footgun; condensed R22 + several Recent-decisions entries)

persistent-memory.md

@@ -1,6 +1,6 @@
 # Persistent memory — eshpfi-management
 
-_Last updated: 2026-07-02_
+_Last updated: 2026-07-05_
 
 ## Repo purpose
 
@@ -22,12 +22,13 @@ Sister repos (separate gitea repos, deployed by playbooks here):
 | `vh/vor` | Inquisitor UI sidecar (port 7879) | push-to-main → CI deploys (2026-04-29) |
 | `vh/nevermore` | Twice-daily LLM-curated briefing (port 8181, replaces news-digest) | push-to-main → CI deploys (2026-04-30) |
 | `vh/asset-engine` | Internal control plane over inference services (port 8200, LAN-direct) | push-to-main → CI deploys (2026-05-12) |
-| `vh/althing` | Lean trusted inter-agent message bus — **v0.17 multi-machine** (re-architected 2026-06): each box runs a local-SQLite bus + a courier/receiver for P2P over the 10.x net. The v0.15 lean-bus cut RIPPED moderation / chamber (UI 7881) / forseti-daemon / agent-runner / redis-valkey. | per-box `uv tool install` (NOT CI-deploy); **nh3-extdev** added as a mesh peer 2026-06-25 (model B: althing-svc svc account + shared `/srv/althing`) |
+| `vh/althing` | Lean trusted inter-agent message bus — **v2 "email model" (v2.0.0b2, 2026-07)**: per-box local-SQLite bus + courier/receiver for P2P over the 10.x net; pillars = open-loops / per-box herald + wake-listener / roaming owner API `/owner/*` / `althing-mcp` stdio surface. The v0.15 lean-bus cut RIPPED moderation / chamber / forseti-daemon / agent-runner / redis-valkey. | per-box `uv tool install` (NOT CI-deploy); **nh3-dev = the DEV box** (editable install of `~/development/althing`, gets new versions first); **nh3-extdev** a mesh peer (model B: althing-svc + shared `/srv/althing`) |
 | `vh/mead-hall` | Bifrost tool-provider sidecar (port 5173 on dev VM 10.100.10.50) | push-to-main → CI deploys (2026-05-16) |
 | `vh/skaldsong` | Wizard + reader surface (port 8300, ana-docker, registry-pull pattern) | push-to-main → CI deploys (2026-05-19) |
-| `vh/Worldtree` | Conversation API (corviduo-dev demo :8080 / personal :8081 / pinned :8082) — Heimdall auth, Bifrost integration. **gitea-runner builds on ana-docker** (the image-bloat source); claude-bot now ADMIN collaborator (2026-06-20) | push-to-main → CI build-and-deploy (runner on ana-docker) |
+| `vh/Worldtree` | Conversation API (corviduo-dev demo :8080 / personal :8081 / pinned :8082) — Heimdall auth, Bifrost integration. **gitea-runner builds on ana-docker**; claude-bot ADMIN collaborator (2026-06-20). Now v1.0.0b19. | push-to-main → CI build-and-deploy (runner on ana-docker) |
 | `vh/yt-voice-clipper` | YouTube → diarized voice-clip dataset builder + audition console (irv-ml1 :8000) | push-to-main → **gitea-webhook auto-deploy** to irv-ml1 (2026-06-03) — see `docs/runbooks/ytvc-autodeploy.md` |
 | `vh/arbo` | Catalog-driven ComfyUI engine (irv-ml1 :8201, comfy-dev owns engine/catalog/image) | push-to-main → gitea Actions CI (deploy-engine.sh, build-local, health-gated) now LIVE; catalog via :9009 webhook |
+| `model-training-forge` (mtf-dev) | Fine-tuning recipe forge; **T1 = qwopus E-RP writing LoRA** (SFT→DPO, LitBench-RM reward) | training runs, not a deployed sidecar |
 
 (`vh/volva` + Heid were re-architected from systemd daemons to Claude Code
 session orchestrators 2026-06-08; their nh3-dev `.service` units were removed —
@@ -55,41 +56,34 @@ no longer deployed sidecars here. See Recent decisions.)
   auths against **demo only**. Personal-instance admin (the
   `~/.config/worldtree/personal-admin-token`, mode 600) POSTs
   `/admin/keys` (mints per-project keys; takes `user_id`+`label`, **no
-  scope param** — scopes are tier-derived, so admin-tier scopes like
-  `admin.events.read` must be granted WT-side by worldtree-dev).
+  scope param** — scopes are tier-derived). **On-instance mint recipe
+  (cleaner than DB-manip):** `docker exec worldtree-worldtree-api-1` POST
+  `/admin/keys` with the in-container `WORLDTREE_BOOTSTRAP_ADMIN_KEY`; cleartext
+  once in `.key`=`wt_live_+16hex`. auto-memory `reference_worldtree_demo_key_mint`.
 
 - **Per-project user keys against personal Worldtree** (issued
-  2026-05-19): `skaldsong:79744637` (nh3-dev iteration),
-  `skaldsong:7c1dbbbe` (ana-docker prod), `althing:50d85460`,
-  `mead-hall:a360822d`. Same `user_id=skaldsong` across both
-  skaldsong keys → shared Heimdall agent slot; different `key_id`
-  → independently rotatable. Pattern: mint via `/admin/keys`, drop
+  2026-05-19): `skaldsong:79744637`, `skaldsong:7c1dbbbe`,
+  `althing:50d85460`, `mead-hall:a360822d`. Mint via `/admin/keys`, drop
   value to `/tmp/wt-personal-<name>.key` mode 600, dev collects +
   shreds (DO NOT cat to chat transcript).
 
-- **Skaldsong CD pattern (registry-pull).** Differs from althing /
-  asset-engine which build-on-host. vh/skaldsong's CI builds and
+- **Skaldsong CD pattern (registry-pull).** vh/skaldsong's CI builds and
   pushes `gitea.phasefinal.com/vh/skaldsong:<sha>` + `:latest`;
   `playbooks/deploy-skaldsong.yaml` on ana-docker pulls + recreates.
-  SHA-pin only (no `:latest` health-gated advance yet). Prereq: host
-  needs `docker login gitea.phasefinal.com` once (read:package PAT) —
-  not currently in the workflow.
+  SHA-pin only. Prereq: host needs `docker login gitea.phasefinal.com` once.
 
 - **gitea internal route for fleet hosts.** gitea is a container on
   **ana-docker** — git-SSH `10.250.50.70:222`, HTTP `:3000`. Fleet/colo
   hosts must use this internal route, NOT public `gitea.phasefinal.com`
-  (`38.120.12.44`, ana-srv1) — the public path fail2bans the host egress
-  IP and wedges webhook deploys. `:22` on `10.250.50.70` is ana-docker's
-  HOST sshd, not gitea. Full gotcha in `docs/orientation.md` → Git/gitea.
+  (`38.120.12.44`) — the public path fail2bans the host egress IP. Full
+  gotcha in `docs/orientation.md` → Git/gitea.
 
 - **docker-as-root pattern** (for ops with no admin API, or to edit
   deploy-owned/root-owned files without sudo): `docker run --rm -v
-  <target-dir>:/wt [-v /var/run/docker.sock:/var/run/docker.sock]
-  docker:cli sh -c "..."` (or `alpine` for plain file ops). docker-group
-  membership is effectively root via bind-mount; treat as sudo-equivalent.
-  **Foot-gun: relative paths in compose.yaml resolve against the sandbox
-  CWD but the Docker daemon interprets them against the HOST fs — always
-  pass `-e VAR=/abs/path` for any relative-default config dir.**
+  <target-dir>:/wt docker:cli sh -c "..."`. docker-group membership is
+  effectively root via bind-mount. **Foot-gun: relative paths in compose.yaml
+  resolve against the sandbox CWD but the daemon interprets them against the
+  HOST fs — always pass `-e VAR=/abs/path` for any relative-default config dir.**
 
 - **`scripts/elway` sudo handling** — elway prompts for the sudo password
   ONCE via `getpass` before the first `sudo: true` step → can't run
@@ -98,247 +92,200 @@ no longer deployed sidecars here. See Recent decisions.)
 
 - **Per-host SSH identity matters for sudo.** infra-ops has NOPASSWD sudo
   on most PFI Linux boxes (corviduo-dev included since 2026-06-15). On
-  ana-docker there are TWO identities: the **default `ssh ana-docker` =
-  `lkraven`** (docker-group, NO passwordless sudo — docker works, root-file
-  edits don't); **but `ssh infra-ops@ana-docker` HAS NOPASSWD root** (verified
-  2026-06-20). **→ For any sudo op on ana-docker (mount, root-owned files,
-  service control), use `ssh infra-ops@ana-docker`, NOT the default session.**
-  lkraven-owned files (litellm config, most stack compose/conf) still take
-  plain `cp`/edit under either identity. **NEW (2026-06-25):** `ssh infra-ops@10.100.10.50`
-  (nh3-dev) ALSO has NOPASSWD sudo; on **nh3-extdev** infra-ops is sudo-LESS by
-  design and **`ssh lkraven@10.100.50.42` is the NOPASSWD path** there.
+  ana-docker: **default `ssh ana-docker` = `lkraven`** (docker-group, NO
+  passwordless sudo); **`ssh infra-ops@ana-docker` HAS NOPASSWD root**. **→
+  For any sudo op on ana-docker, use `ssh infra-ops@ana-docker`.** `ssh
+  infra-ops@10.100.10.50` (nh3-dev) ALSO NOPASSWD sudo; on **nh3-extdev** infra-ops
+  is sudo-LESS by design (`ssh lkraven@10.100.50.42` is the NOPASSWD path). **irv-ml1:
+  `ssh irv-ml1` = lkraven, docker-group (plain docker) but sudo needs a PASSWORD
+  (no NOPASSWD)** — stage model pulls to `/home`, not root-owned `/worktank`.
 
 ## Current state / in-flight
 
-_As of 2026-07-02:_
+_As of 2026-07-05:_
 
-- **`gen` model = qwopus (`qwen3.5-122-a10b`); the Deckard trial is CONCLUDED.** Trialed
-  `robbatt/Qwen3.6-40B-Deckard-NVFP4` behind the `gen` aliases — it won writing quality
-  decisively but lost on speed (~36 vs qwopus's ~90 tok/s); reverted (git `681eb70`).
-  Deckard kept STAGED on ana-ml2 as T1's quality benchmark
-  (`/tank/aimodels/qwen36-40b-deckard-{bf16,nvfp4}`; container `vllm-deckard-40b`
-  stopped-but-kept). Full arc in auto-memory `reference_gen_qwopus_122b`.
+- **T1 TRAIN — venue decision live; NEXT ACTION = the ana-ml2 SMOKE.** T1 (qwopus E-RP writing
+  LoRA) datasets + recipe + smoke set are all staged (mtf-dev, on `nh3-dev:data/export/qwen-3.5-122b-erp-lora/`
+  incl. `smoke/`); bf16 base is on ana-ml2 `/tank/aimodels/qwopus3.5-122b-a10b-bf16` (233G, PUBLIC
+  HF repo, zero-auth pull). On-prem RULED OUT except a full ana-ml2 shutdown → CPU-offload → ~1-day
+  fleet outage; cloud (Vast.ai 8×80GB, ~3-6h, ~$60-500, zero impact) is the clean alt. **Operator
+  chose smoke-first on ana-ml2** to measure real samples/sec before committing. ⚠️ the smoke ITSELF
+  needs both GPUs + the ~421G vLLM RAM freed (192G VRAM < 244G base → CPU offload) = a brief full
+  ana-ml2 serving pause; keeping any serving up drops it into the ~week NVMe regime. Full runbook +
+  both paths in `reference_t1_cloud_train_plan`. Post-train serve-path (swappable-LoRA-on-NVFP4
+  test) still queued: `reference_gen_qwopus_122b`.
 
-- **T1 (mtf-dev qwopus writing-LoRA) IN PROGRESS.** Attention-only LoRA (full-attn q/k/v/o
-  + the bf16 GDN `in_proj_qkv`/`out_proj`); serve via swappable-LoRA-on-NVFP4 (test QUEUED,
-  gated on the first T1 adapter) or merge+requant. Trainer harness seam MECHANICALLY
-  proven via a granite-8b spike (2026-07-02): the pipeline runs end-to-end, but efficacy
-  is NOT validated (mechanics-only synthetic fixture, by design; a granite adapter
-  wouldn't transfer to qwopus regardless). Real behaviour-shift efficacy is a T1-run
-  question. NVFP4 quant-structure + serve-path facts in `reference_gen_qwopus_122b`.
+- **LitBench-RM UP (irv-ml1 A6000); comfyui DISPLACED.** Respun the R19 creative-writing reward
+  (`vllm-litbench-rm` :8202, ~19.6G) for T1's reward ensemble; stopped comfyui to free the A6000
+  (operator is sole comfyui consumer + holding image-gen until LitBench done). Score scale
+  re-verified from nh3-dev (literary 0.95 > flat 0.44 > slop 0.12). **TEARDOWN on "litbench done":**
+  `ssh irv-ml1 'docker rm -f vllm-litbench-rm && docker start comfyui'` — ⚠️ then watch the mmartial
+  boot trap (root-owned venv → crash-loop; `chown -R 1000:1000 /comfy/mnt/venv`). `reference_litbench_rm_irv_ml1`.
 
-- **Worldtree #332 scoped-log view + tunnel = STANDING ASSET.** `wt_gateway_logs` view +
-  `wt_readonly` role on the litellm DB + `wt-db-tunnel` systemd on corviduo-dev, for
-  worldtree-dev's embed-recall regression-watches. #332 fix verified in prod (15×→1.01×
-  re-embed). Teardown steps + the IP-pin caveat in auto-memory
-  `reference_wt_gateway_scoped_log_view`.
+- **character-rp role — SHIPPED; one QUEUED spot-check.** Proved per-request `extra_body`
+  (top_k/repetition_penalty) forwards through the `gen-reasoning` alias to the vLLM sampler (no
+  gateway cap needed); pre-staged the `character-rp` role in demo+personal bind-mount
+  `model_roles.yaml` (byte-verified live on b18; caught the cached-registry ordering so the deploy's
+  own restart activates it). worldtree-dev shipped **#344 (v1.0.0b19)** fixing the durable-agent
+  override-drop (role now applies the RP profile). QUEUED: ratatoskr-dev pings a turn-timestamp → I
+  spot-check the gateway spend_logs to confirm the RP sampler params (temp 0.75 / top_p 0.85 /
+  presence 1.5 / top_k 20 / rep 1.0) reach vLLM.
 
-- **`/books` mounted (transient) on nh3-dev** for a books/corpus ingestion:
-  `10.0.50.50:/mnt/books` (ESH NAS) → `/mnt/books`, NFSv4 `ro,soft` (soft dodges the ESH
-  D-state hang). NOT fstab — re-mount via `infra-ops@10.100.10.50` if nh3-dev reboots.
+- **althing v2 (herald + receiver) = systemd-supervised on nh3-dev (the althing DEV box).**
+  Formalized this session: `althing-herald.service` (Restart=always, **Environment=PATH incl
+  ~/.cargo/bin** — the fix for the silent pane-dispatch outage) + `althing-receiver.service` (v2 →
+  pillar-3 `/owner/*` live). Stale forseti unit removed; both on v2.0.0b2, canonicalized by forseti.
+  `reference_nh3_dev_althing_herald`.
 
-- **granite-4.1-8b bf16 kept** at `irv-ml1:/home/lkraven/granite-4.1-8b-bf16` (17G,
-  reap-on-request; the T1-harness-spike base). comfyui was borrowed off the A6000 for the
-  spike + RESTORED healthy.
+- **glm-5.2 = z.ai 1M-in / 128K-out, NO gateway cap** (probed z.ai; recorded in the config comment
+  + `reference_litellm_gateway`; commit 624a07e). All GLM entries are pure z.ai passthrough — the
+  effective ceiling is z.ai's canonical, not a gateway limit.
 
-- **Recently completed (2026-06-22..25, now in Recent decisions):** Worldtree #314/#322/#317
-  persona-render config arc; nh3-extdev althing v0.17.1 mesh peer (Model B); zellij web-seat pilot.
+- **`/books` mounted (transient) on nh3-dev** (`10.0.50.50:/mnt/books` → NFSv4 ro,soft; re-mount via
+  `infra-ops@10.100.10.50` if it reboots).
 
-- **Backups — recovered + hardened (2026-06-20), STILL OPEN:** rotate the 5 disclosed
-  rest-server creds (operator, offline); confirm esh-vm-db's resticprofile includes DB
-  dumps (esh-pve-nas restic-only). Topology + 2-min check: `docs/runbooks/backups.md`.
+- **Backups — recovered + hardened (2026-06-20), STILL OPEN:** rotate the 5 disclosed rest-server
+  creds (operator, offline); confirm esh-vm-db's resticprofile includes DB dumps. `docs/runbooks/backups.md`.
 
-- **R22 (brokkr/dwarves) — CONCLUDED, gateway-only.** Phase B (internal-model hooks)
-  CANCELLED (Worldtree model-agnostic → no deploy path). Whole footprint = gateway calls
-  to free `qwen3.5-122-a10b` via a PERSISTENT full-access key at
-  `/home/lkraven/.r22-gateway-key` (mode 600, carries paid GLM, do NOT delete). Detail in
-  Recent decisions.
+- **Open follow-ups (low-priority):** clean phantom `qwen3.6-35b-a3b` off the gateway (lists in
+  /v1/models but 400s); docker-daemon default log-cap on ana-docker; drop the now-inert
+  `mood.decay_rate`/`mood.stale_hours` keys from the deployed `/opt/worldtree*/config` bind-mounts
+  (harmless-present per worldtree-dev; edit-only/no-restart at a no-deploy window).
 
-- **Open follow-ups (low-priority):** clean phantom `qwen3.6-35b-a3b` off the gateway
-  (lists in /v1/models but 400s — backend displaced); docker-daemon default log-cap on
-  ana-docker (systemic fix behind the 94 GB disk incident); drop the now-inert
-  `mood.decay_rate`/`mood.stale_hours` keys from the deployed `/opt/worldtree*/config`
-  bind-mounts (removed from canonical in R30 b15–b17; harmless-present per worldtree-dev
-  since `CharacterSchema.mood` is `dict[str,Any]` — clean edit-only/no-restart at a
-  no-deploy window to restore byte-identity; new optional `mood.tau_base` is unset→derived).
+- **Standing / parked:** Mac Pro migration (hardware-gated); R17 v2 corpus HELD; disclosed-keys
+  rotation queue; clean legacy `news-digest`; R22 gateway-only full-access key at
+  `/home/lkraven/.r22-gateway-key` (mode 600, carries paid GLM, do NOT delete); granite-4.1-8b bf16
+  at `irv-ml1:/home/lkraven/granite-4.1-8b-bf16` (reap-on-request); Deckard staged on ana-ml2 as
+  T1's writing benchmark.
 
-- **Standing / parked (from prior):** Mac Pro migration (`migration-plan.md`,
-  hardware-gated); R17 v2 corpus push HELD (distribution barred); disclosed-keys rotation
-  queue; clean legacy `news-digest` on ana-docker.
-
-- **Worldtree config-propagation (reference):** demo+personal bind-mount config
-  (`providers.yaml`/`model_roles.yaml`/`policies.yaml`/`defaults.yaml`, now incl. the
-  #314/#322/#317 additions) from `/opt/worldtree{,-personal}/config`
-  (infra-ops-deployable, byte-identical from worldtree-dev canonical sha); pinned is
-  IN-IMAGE. Reload via `docker restart <container>`, NEVER `compose up` (stale-`:latest`
-  footgun); WT PDP is rule-based (a tier needs allow RULES, not just scopes). **#317
-  lesson: config REMOVALS are NOT backward-compatible with the still-running image — push
-  promptly, don't bounce the instance in the window.**
+- **Worldtree config-propagation (reference):** demo+personal bind-mount config from
+  `/opt/worldtree{,-personal}/config` (infra-ops-deployable, byte-identical from canonical); reload
+  via `docker restart <container>`, NEVER `compose up` (stale-`:latest` footgun). **The role registry
+  is loaded ONCE + CACHED at startup** (`ConversationService.role_registry`) → a bind-mount
+  `model_roles.yaml` change needs a container restart to take effect; pre-stage the bind-mount
+  BEFORE the activating deploy's restart (else it needs a second bounce). config REMOVALS are NOT
+  backward-compatible with the still-running image.
 
 ## Recent decisions
 
-- `[2026-07-02]` **mtf-dev granite harness-spike provisioned on irv-ml1 (comfyui displaced
-  for the A6000) → ran GREEN.** Freed the A6000 by stopping comfyui (operator-coordinated
-  with comfy-dev), staged granite-4.1-8b bf16 to `irv-ml1:/home/lkraven`, mtf-dev's TRL
-  SFT→DPO→eval seam proved end-to-end (DPO genuinely learned, 0.833 acc; anti-slop ~0 = the
-  expected null on clean-writing granite-instruct); comfyui restored healthy. De-risks T1's
-  trainer harness ahead of the real qwopus train. **MECHANICAL green ONLY — efficacy NOT
-  validated, by design** (mtf-dev confirm 2026-07-02): the run used a 12-row/12-pair
-  SYNTHETIC writing fixture (clean-vs-sloppy generic prompts), NOT the E-RP corpus; rank 8,
-  1 epoch, ~2 SFT + ~6 DPO steps. The in-loop base-vs-adapter check (HoldoutEvaluator)
-  already ran and returned the expected null (anti_slop_improvement −0.002); the adapter was
-  then reaped (gone), so there is nothing to A/B — and on a mechanics-only synthetic adapter
-  an A/B would only reconfirm the null. Real efficacy = the T1 run (real recipe + E-RP data
-  on qwopus). **Operator DECIDED 2026-07-02: DEFER efficacy to the T1 run** — NO intermediate
-  real-efficacy granite spike (uninterpretable proxy: granite arch ≠ qwopus MoE+GDN AND
-  censored-instruct vs qwopus-abliterated, so a null could be refusal priors fighting E-RP
-  rather than a recipe bug). If any pre-T1 de-risk is later wanted, the right target is the
-  real-data/judge WIRING check (corpus tokenizes → sane DPO pairs → coherent judge scores;
-  arch-independent, validates LitBench-less) — NOT a granite train. mtf-dev holds; infra's
-  next owed item stays the queued swappable-LoRA-on-NVFP4 load test (gated on first T1 adapter).
+- `[2026-07-05]` **T1 training venue: CLOUD recommended; operator chose smoke-first on ana-ml2.**
+  On-prem ruled out (ana-ml2 full — both 96G GPUs ~93G used): keep-serving = NVMe offload ~6-8 DAYS;
+  full ana-ml2 shutdown = CPU offload ~1 DAY but a whole-fleet outage. Cloud Vast.ai 8×80GB (no
+  offload → ~3-6h, ~$60-500, zero fleet impact) is the clean alt (mtf-dev + infra-ops both rec;
+  Vast for its no-content-AUP marketplace + likely-existing VastBlue account). Operator's next step
+  = the ana-ml2 CPU-offload SMOKE (~60 steps) to get real samples/sec before the full-outage-vs-cloud
+  call. HF base verified public (zero-auth pull). Runbook + gotchas in `reference_t1_cloud_train_plan`.
 
-- `[2026-07-01]` **Worldtree #332 embed-recall diagnosed + scoped-log view/tunnel
-  provisioned + fix verified.** The persona-recitation gate re-embedded persona segments
-  every turn (15× re-embed, 95% cross-turn recurrence, one anchor ×206/min → worldtree-gateway
-  was ~95% of the embedding backend load); worldtree-dev's cross-turn content-hash cache
-  dropped it to 1.01× in prod. Built a boundary-safe read-only `wt_gateway_logs` view +
-  `wt_readonly` role + `wt-db-tunnel` (corviduo-dev) for their regression-watch. auto-memory
-  `reference_wt_gateway_scoped_log_view`.
+- `[2026-07-05]` **glm-5.2 canonical limits recorded** (probed live vs z.ai): **1,048,576 (1M) input
+  context / 131,072 (128K) max output**; NO gateway-side cap (pure passthrough → z.ai's limits are
+  effective). Written to the config comment (commit `624a07e`) + `reference_litellm_gateway`.
 
-- `[2026-07-01]` **qwopus native MTP speculative-decode tested on `gen` → NOT kept.** qwopus
-  HAS a full native MTP head (785 tensors, in NVFP4 + bf16). Measured +12% single-stream but
-  **−15–20% AGGREGATE at moderate concurrency** (N=4: 250→200 tok/s) + it silently drops
-  `min_p`/`logit_bias` → reverted to clean baseline. The win is banked for T1 (MTP as a
-  per-deployment option, preserved through requant). `reference_gen_qwopus_122b`.
+- `[2026-07-04]` **character-rp: gateway-forwarding proven + role pre-staged + #344 shipped.**
+  Empirically confirmed per-request `extra_body` (top_k/repetition_penalty) forwards through the
+  `gen-reasoning` LiteLLM alias to vLLM + standard params override the alias defaults — no gateway
+  cap needed (I over-built a dedicated alias, operator corrected, reverted with zero fleet impact).
+  Pre-staged the `character-rp` role into demo+personal bind-mount `model_roles.yaml` (byte-verified
+  on b18; caught the cached-registry ordering). worldtree-dev shipped **#344 (v1.0.0b19)** for the
+  durable-agent override-drop. spend_logs spot-check queued (ratatoskr's timestamp ping).
 
-- `[2026-07-01]` **Deckard trial → reverted to qwopus (`gen`).** `robbatt/Qwen3.6-40B-Deckard-NVFP4`
-  won writing "in every way" but at ~36 vs ~90 tok/s (dense-40B vs MoE-~10B-active);
-  spec-decode rescue ruled out (aeon DFlash image is arm64/DGX-Spark-only; Deckard's
-  NVFP4/bf16/base-27B all LACK MTP; EAGLE-head training a multi-week non-starter). git
-  `b63c48b` (repoint) + `681eb70` (revert). Deckard kept staged as T1's writing benchmark.
+- `[2026-07-04]` **althing v2 herald+receiver formalized as systemd on nh3-dev.** `althing-herald.service`
+  (Restart=always, **Environment=PATH incl ~/.cargo/bin** — the pane-dispatch fix) + `althing-receiver.service`
+  (v2 → pillar-3 `/owner/*` live); stale forseti unit removed; both on v2.0.0b2, canonicalized by
+  forseti. `reference_nh3_dev_althing_herald`.
 
-- `[2026-06-25]` **althing re-architected to the v0.17 lean multi-machine bus; nh3-extdev
-  stood up as a mesh peer (MODEL B).** v0.15.0's lean-bus cut RIPPED
-  moderation/chamber/forseti-daemon/agent-runner/redis-valkey; **v0.17 = per-box
-  local-SQLite bus + a courier/receiver for P2P over the 10.x net** (installed per-box via
-  `uv tool install`, NOT CI). Migrated nh3-dev's client v0.15→v0.17. nh3-extdev
-  provisioned as a full mesh peer with the operator's **MODEL B** (dedicated `althing-svc`
-  service account + group-shared `/srv/althing` root, so multiple OS users share one
-  config/DB; `ALTHING_ROOT` via /etc/profile.d). Verified multi-user concurrent rw.
-  Supersedes the old althing Tools-row (was chamber/daemons/valkey). auto-memory
-  `reference_nh3_extdev_althing_mesh`.
+- `[2026-07-04]` **LitBench-RM respun (irv-ml1 A6000, comfyui displaced)** for T1's reward ensemble;
+  operator sole comfyui consumer, holding image-gen until LitBench done. `reference_litbench_rm_irv_ml1`.
 
-- `[2026-06-23]` **zellij native web client piloted on nh3-dev** (`zellij-web.service`
-  :8443, native TLS), system-installed for all users, ALONGSIDE ttyd (kept as the no-auth
-  fallback). Bind left `0.0.0.0` per operator. auto-memory `reference_zellij_web_seat`.
+- `[2026-07-03]` **ratatoskr-dev DEMO Heimdall key provisioned (R30 φ0).** Minted a tier-user key on
+  the demo via `POST /admin/keys` (bootstrap admin key), mirroring their personal base consumer (no
+  character-binding); base-agent affect reads work ungated. `reference_worldtree_demo_key_mint`.
 
-- `[2026-06-22]` **Worldtree persona-render config arc (#314 relational-stance / #322
-  affect-mood gate / #317 valence teardown) pre-synced + deployed green on demo+personal**
-  — #317 a boot-blocking config REMOVAL (relational_valence). Config-sync recipe +
-  additions-vs-removals lesson in auto-memory `reference_corviduo_dev_emergency_ops`.
+- `[2026-07-02]` **mtf-dev granite harness-spike ran GREEN — MECHANICAL only, efficacy DEFERRED to
+  the T1 run.** Trainer TRL SFT→DPO→eval seam proven end-to-end on a synthetic fixture (not the E-RP
+  corpus); operator DECIDED no intermediate real-efficacy granite spike (uninterpretable proxy —
+  arch gap + abliteration axis). `reference_gen_qwopus_122b`.
 
-- `[2026-06-20]` **R22 (brokkr/eitri/dwarves) stood down to gateway; full-access R22
-  key minted; Phase B parked.** Operator caught the raw-weights/GPU ask as premature —
-  Phase A (P00 + NL-rendering) is PROMPT-LEVEL on the LiteLLM gateway, not raw weights
-  (no GPU, no Blackwell displacement). Real blocker was gateway ACCESS: the shared
-  `all-agents-local` key only serves granite chat, and granite-8b is too weak to be the
-  P00 model-under-test (capability-confound). Operator said "full access" → minted
-  `r22-brokkr-phaseA` LiteLLM key (scope **all-proxy-models**, incl PAID GLM), dropped
-  `~/.r22-gateway-key` (mode 600) for eitri — **re-minted PERSISTENT** after eitri's
-  stateless tool-env couldn't hold it in env + shredded the one-shot drop (old key
-  revoked); the live full-access key now lives at `/home/lkraven/.r22-gateway-key`,
-  read per-command, **don't delete** (cost surface — carries paid GLM; operator
-  reconfirmed full-open + declined to narrow, 2026-06-20 — settled, don't re-surface).
-  Lesson: stateless consumers need a persistent mode-600 file read per-call, NOT
-  shred-after-load. **P00 MUT = `qwen3.5-122-a10b` (alias
-  `gen`)** — 122B, capable, FREE-LOCAL (no per-token cost); told them keep frozen
-  calibration on it, NOT the paid GLM the key can now reach. **PHASE B** (raw-weights
-  hooks: Dvalin activation-steering on hidden states + Regin ECM logit-injection on a
-  7-8B Qwen/Llama via transformers LogitsProcessor) **CANCELLED 2026-06-20 (not
-  deferred)** — operator cut the internals arms: **Worldtree is model-agnostic** (regard/
-  affect renders via context manipulation = prompt-level only), so the internal-model
-  approach has no deploy path (the pragmatic-outcome gate firing). R22's ENTIRE compute
-  is now the gateway (free qwen3.5-122-a10b MUT); NO raw-weights/GPU ever; the A6000
-  slice reservation is released (was never provisioned). The full-scope R22 key stays
-  (pinned to the free qwen-122, GLM untouched). **OPERATOR STEER (2026-06-20): R22 is
-  research for a PRAGMATIC/deployable outcome, NOT advancing-the-art.** Before the dwarves
-  invest in Phase B, establish a deploy path (distilled steering-vector / fine-tune
-  applied to a model we already serve). Operator is taking the dwarf discussion directly.
+- `[2026-07-01]` **Worldtree #332 embed-recall diagnosed + scoped-log view/tunnel provisioned + fix
+  verified** (15×→1.01× re-embed). `reference_wt_gateway_scoped_log_view`.
 
-- `[2026-06-20]` **claude-bot issue-scope token minted for worldtree-dev self-serve**
-  (closes their last `tea`-as-vh fallback — issues; they already self-serve Actions +
-  deploys). Gitea PAT scopes are IMMUTABLE → minted a NEW claude-bot PAT
-  `worldtree-dev-actions-issues-20260621` (id 16) with `write:repository`+`write:issue`
-  via basic-auth (`claude-bot:gitea-password`, POST `/users/claude-bot/tokens`); dropped
-  mode-600 `~/.claude-bot-token-issues` → worldtree-dev swaps. **Gitea COLLAPSES
-  `read:issue` into `write:issue`** (write implies read). Old token (id 15) REVOKED after
-  verification → claude-bot carries ONE consolidated worldtree-dev token (id 16) + arbo-ci
-  (id 14). Advances the credential-migration directive.
+- `[2026-07-01]` **qwopus native MTP speculative-decode tested on `gen` → NOT kept** (+12% single-stream,
+  −15–20% aggregate at concurrency, silently drops min_p/logit_bias). Banked for T1. `reference_gen_qwopus_122b`.
 
-- `[2026-06-20]` **rest-server-ana recovered + backup prevention shipped + worldtree-dev
-  admin keys provisioned** (operator-directed). rest-server-ana fixed (mount-a +
-  force-recreate via `infra-ops@ana-docker`); freshness alert + fstab hardening landed
-  (see `docs/runbooks/backups.md`). Minted worldtree-dev **admin-tier Heimdall keys** on
-  demo (key_id d113207c) + personal (f4f75adb), dropped mode-600 to
-  `~/.wt-admin-{demo,personal}` on nh3-dev → worldtree-dev now self-serves key minting.
-  Cred rotation (5 rest-server pw) BELAYED per operator.
+- `[2026-07-01]` **Deckard trial → reverted to qwopus (`gen`)** (won writing "in every way" but ~36 vs
+  ~90 tok/s; spec-decode rescue ruled out). git `b63c48b`→`681eb70`. Deckard kept staged as T1's
+  writing benchmark.
 
-- `[2026-06-20]` **claude-bot → ADMIN on vh/Worldtree** (operator-authorized;
-  one-time use of operator vh-admin to enable the migration) — claude-bot
-  self-serves WT deploys/tokens henceforth, reducing personal-cred fallbacks.
+- `[2026-06-25]` **althing re-architected to the lean multi-machine bus; nh3-extdev stood up as a
+  MODEL B mesh peer** (dedicated `althing-svc` + group-shared `/srv/althing`). `reference_nh3_extdev_althing_mesh`.
 
-- `[2026-06-14]` **STANDING DIRECTIVE: migrate ALL infra access to Claude-specific credentials.** Stand up service accounts (claude-bot Gitea user + scoped tokens, distinct litellm admin key); re-mint consumer creds under them; flag personal-cred fallbacks until done. (auto-memory `project_migrate_infra_access_to_claude_credentials`)
+- `[2026-06-23]` **zellij native web client piloted on nh3-dev** (`zellij-web.service` :8443)
+  alongside ttyd. `reference_zellij_web_seat`.
+
+- `[2026-06-22]` **Worldtree persona-render config arc (#314/#322/#317) pre-synced + deployed green
+  on demo+personal** — #317 a boot-blocking config REMOVAL. `reference_corviduo_dev_emergency_ops`.
+
+- `[2026-06-20]` **R22 (brokkr/dwarves) stood down to gateway-only; full-access R22 key minted;
+  Phase B CANCELLED** (Worldtree model-agnostic → no deploy path). Key at `/home/lkraven/.r22-gateway-key`
+  (persistent mode-600, carries paid GLM, don't delete). MUT = free `qwen3.5-122-a10b` (`gen`).
+  Operator steer: R22 research is gated on a pragmatic/deployable outcome, not advancing-the-art.
+
+- `[2026-06-20]` **claude-bot issue-scope token minted for worldtree-dev self-serve** (id 16,
+  `write:repository`+`write:issue`); old token revoked. Advances the credential-migration directive.
+
+- `[2026-06-20]` **rest-server-ana recovered + backup prevention shipped + worldtree-dev admin keys
+  provisioned** (demo d113207c / personal f4f75adb). Cred rotation (5 rest-server pw) BELAYED.
+
+- `[2026-06-20]` **claude-bot → ADMIN on vh/Worldtree** (operator-authorized) — self-serves WT
+  deploys/tokens henceforth.
+
+- `[2026-06-14]` **STANDING DIRECTIVE: migrate ALL infra access to Claude-specific credentials.**
+  (auto-memory `project_migrate_infra_access_to_claude_credentials`)
 
 _118 older entries archived to archival-memory.md._
 
 ## Tried and abandoned
 
-- `[2026-07-01]` **A personal-Worldtree CI deploy that fails ~85s in with "not found /
-  unauthorized" is usually the pull-only-vs-build RACE, not registry-auth.** `deploy-personal.yml`
-  is PULL-ONLY ("image must already be built by a push to main") but fires on the
-  `staging/vX` tag push SIMULTANEOUSLY with `deploy.yml`'s main build → it tries to pull the
-  image ~2.5 min BEFORE the build finishes pushing it → step-4 "Verify image exists" aborts
-  "not found". Misattributed to registry-auth twice (the earlier cff3328 saga too). DIAGNOSE:
-  the image tag (12-char short-sha, NOT 7) exists in the registry + the VM login succeeds ⇒
-  it's the race. FIX: re-run once the build's done (image now present), OR gate deploy-personal
-  on `workflow_run: completed`.
+- `[2026-07-04]` **LiteLLM (this gateway version) mutates the SHARED deployment config in-place on
+  per-request sampler-param merge** → my deliberately-invalid `top_k=-5` forwarding-probe bled into a
+  param-less character-rp request (vLLM 400, ONE-OFF, self-cleared by a later valid probe). NOT
+  caching (none configured), NOT a config change. **Never fire invalid/distinctive sampler values at
+  a SHARED gateway alias with live consumers** — use a throwaway alias, or a `docker restart litellm`
+  flushes residual carryover. `feedback_litellm_shared_param_mutation`.
+
+- `[2026-07-04]` **A systemd `--user` daemon that shells out to `~/.cargo/bin`/`~/.local/bin` tools
+  needs an explicit `Environment=PATH`** — the minimal `--user` default silently drops them. The
+  althing herald lost `zellij` → silent `pane-miss` for ALL config-backed TUI/pane agents; CC + FIFO
+  routes were unaffected, so it was invisible from a CC session. `reference_nh3_dev_althing_herald`.
+
+- `[2026-07-04]` **On-prem T1 train that keeps ANY ana-ml2 serving up = ~6-8 DAYS** (1-GPU + NVMe
+  ZeRO-Infinity offload; MoE ~10B-active cuts FLOPs but NOT the 244G base's param I/O). The only fast
+  on-prem path is a FULL ana-ml2 shutdown (both GPUs + the ~421G vLLM RAM freed → base fits in the
+  566G CPU RAM) → CPU offload → ~1-day full-fleet outage. Cloud (no offload) = hours. `reference_t1_cloud_train_plan`.
+
+- `[2026-07-01]` **A personal-Worldtree CI deploy that fails ~85s in with "not found / unauthorized"
+  is usually the pull-only-vs-build RACE, not registry-auth.** `deploy-personal.yml` is PULL-ONLY but
+  fires on the `staging/vX` tag simultaneously with `deploy.yml`'s build → pulls before the push
+  finishes. FIX: re-run once built, or gate on `workflow_run: completed`.
 
 - `[2026-07-01]` **MTP/spec-decode on a SHARED serving model helps single-stream but HURTS
-  moderate-concurrency aggregate throughput + silently ignores `min_p`/`logit_bias`.**
-  Measured on qwopus `gen`: N=1 +12%, N=4 −20% aggregate. Don't bolt spec-decode onto the
-  fleet `gen` for a single-stream win — reserve it for dedicated/interactive deployments.
+  moderate-concurrency aggregate + silently ignores `min_p`/`logit_bias`** (qwopus `gen`: N=1 +12%,
+  N=4 −20%). Reserve for dedicated/interactive deployments.
 
-- `[2026-07-02]` **irv-ml1 `/worktank` ROOT is root-owned — lkraven can't write there (and
-  irv-ml1 sudo needs a password non-interactively) → stage model pulls to `/home`.** Also
-  PIN THE A6000 BY UUID for training runs: nvidia-smi index 1 is the A6000, but native-CUDA
-  ordering can differ vs docker, and the 3090 (index 0) is usually near-full → land there and
-  OOM. `CUDA_VISIBLE_DEVICES=GPU-<uuid>`.
+- `[2026-07-02]` **irv-ml1 `/worktank` ROOT is root-owned — lkraven can't write there (irv-ml1 sudo
+  needs a password) → stage model pulls to `/home`.** PIN THE A6000 BY UUID for training (native-CUDA
+  ordering differs vs docker; the 3090 index 0 is usually near-full → OOM). `CUDA_VISIBLE_DEVICES=GPU-<uuid>`.
 
-- `[2026-06-25]` **althing "unreachable: <machine> — retry later" can MASK an app-level
-  500.** A multi-hour "intermittent :8087 / firewall flap" hunt was a red herring: raw
-  network was always clean (curl POST to the receiver `:8087` worked; a connect probe = 0
-  fails). Root cause = the receiver's DB agents-table wasn't synced with the config roster,
-  so delivery 500'd `"unknown to: <handle>"`, which the courier MAPPED to "unreachable"
-  (looks like network/DNS). **Diagnose: raw curl to :8087 + a connect-probe pass ⇒ NOT
-  network — look up-stack.** Fixed in althing **v0.17.1** (receiver auto-ensures the
-  recipient on delivery). Stop-gap on older builds: `ALTHING_HANDLE=<h> althing-cli inbox`
-  as the receiver user syncs config→DB; re-run after any roster change. auto-memory
-  `reference_nh3_extdev_althing_mesh`.
+- `[2026-06-25]` **althing "unreachable: <machine>" can MASK an app-level 500.** Raw network was
+  clean; root cause = receiver DB agents-table not synced with the config roster → delivery 500'd
+  "unknown to: <handle>", MAPPED to "unreachable". Diagnose: raw curl to :8087 + connect-probe ⇒ NOT
+  network. Fixed in althing v0.17.1. `reference_nh3_extdev_althing_mesh`.
 
-- `[2026-06-20]` **rest-server `.htpasswd: permission denied` = the ana-nas
-  NFS mount FAILED (ghost file on the local mount point), NOT a decommission.**
-  `mnt-backup.mount` stuck `failed` (fstab bare `defaults`, no retry) → the
-  rest-server serves an empty local dir with a root:root 0-byte `.htpasswd`.
-  Documented recovery in disaster-recovery.md. Don't tear down what looks like
-  a crash-looping legacy container until you've checked fstab + the docs — it
-  was the live ana-side restic target.
+- `[2026-06-20]` **rest-server `.htpasswd: permission denied` = the ana-nas NFS mount FAILED (ghost
+  file on the local mount point), NOT a decommission.** `mnt-backup.mount` stuck `failed` (fstab bare
+  `defaults`) → rest-server serves an empty local dir. Recovery in disaster-recovery.md.
 
-- `[2026-06-20]` **The DEFAULT `ssh ana-docker` is `lkraven` (no NOPASSWD) —
-  but `ssh infra-ops@ana-docker` HAS NOPASSWD root** (corrected later same day).
-  Early on a `sudo cp` as lkraven silently failed (password prompt) → one
-  wasted gateway bounce; I then wrongly concluded "no sudo on ana-docker" and
-  nearly punted the rest-server-ana recovery to the operator. The real rule:
-  reach for `infra-ops@ana-docker` for sudo ops; lkraven-owned files (litellm
-  config, most stack compose/conf) take plain `cp` under either identity.
+- `[2026-06-20]` **The DEFAULT `ssh ana-docker` is `lkraven` (no NOPASSWD) — but `ssh
+  infra-ops@ana-docker` HAS NOPASSWD root.** A `sudo cp` as lkraven silently failed → nearly punted
+  the rest-server recovery. Reach for `infra-ops@ana-docker` for sudo ops.
 
 _98 older entries archived to archival-memory.md._