feat(booth): wrap verbatim index.html booths with a back-to-booths chip + inherited favicon

Verbatim-index.html booths were served raw (FileResponse) with no base template, so they had no favicon and no way back to the booth index — the gap the app-rendered gallery/zoom pages already covered via base.html. booth_view now reads a small verbatim index.html and, via a pure wrap_verbatim_html(), injects: - a fixed-position 'all booths' chip (scoped class, max z-index, hidden in print), pinned top-right (empty on left-aligned report layouts; a top-left chip clips the page title) and appended at the END of the document so it never reorders the page; - the Booth favicon at the first head-ish seam, only if the page declares no icon of its own. Injection is doctype/charset-safe for the compact HTML real booths use (<!doctype html><meta charset><title><style>…content, no explicit head/ body): nothing is ever placed ahead of a leading <!doctype> (which would force quirks mode), and the ~250B favicon link keeps the charset <meta> inside the first-1024-byte detection window. The raw file route (/b/<name>/index.html) stays byte-for-byte, so assets and ?dl=1 downloads are unchanged; files over 8 MB serve raw, unwrapped. Verified live on nh3-dev :8090 across the real booth shapes (compact-HTML crow-*/jackdaw-*/mimir-favicon, well-formed dcc-summarizer-ab, own-icon edict-favicon). 10 new tests; suite 38 passing.
memory: correct extdev sudo status in 07-25 herald-install entry
2026-08-03 13:05:30 -07:00 · 2026-08-03 08:01:17 -07:00 · 2026-08-03 00:12:59 -07:00 · 2026-08-02 23:57:48 -07:00 · 2026-08-02 23:40:39 -07:00 · 2026-08-02 23:12:21 -07:00
213 changed files with 18408 additions and 561 deletions
@@ -35,3 +35,7 @@ htpasswd-new
 # graphify: commit only the lightweight labeled map; ignore heavy/regenerable artifacts
 graphify-out/*
 !graphify-out/GRAPH_REPORT.md
+
+# Python bytecode (e.g. from local py_compile of stack wrappers)
+__pycache__/
+*.pyc
@@ -10,6 +10,15 @@ state) across context resets. Read it at session start; treat it as
 one input alongside this CLAUDE.md and the auto-memory system, not
 as the single source of truth.

+It is a lean **index**: the dated log sections (Recent decisions,
+Tried and abandoned) keep each over-threshold entry's full body in
+`persistent-memory.d/<slug>.md`. Read the index at session start;
+pull a detail file only when its index line is relevant to your work —
+never bulk-read `persistent-memory.d/`. When you commit, stage any
+pending `persistent-memory.md` and `persistent-memory.d/` updates in
+the same commit as the work that prompted them — durable memory that
+lags the code defeats its own purpose.
+
 **New session starting here?** Read [`docs/orientation.md`](docs/orientation.md) first — fleet topology, backup architecture, governing principles, and all the NFS/DSM/naming gotchas that have cost past sessions time.

 **For SSH-driven work: use `scripts/elway`.** Write a playbook under
@@ -88,7 +97,7 @@ Observed and standardized across servers:

 | Name | IP | Site | Role | Details |
 |------|-----|------|------|---------|
-| ana-ml2 | 10.250.50.54 | Anaheim (`10.250.0.0/16`) | GPU / AI inference (bare metal, dual RTX 6000 Ada) | `servers/ana-ml2/README.md` |
+| ana-ml2 | 10.250.50.54 | Anaheim (`10.250.0.0/16`) | GPU / AI inference (bare metal, dual RTX PRO 6000 Blackwell Max-Q, 96 GB each) | `servers/ana-ml2/README.md` |
 | irv-ml1 | 10.100.79.3 (WG) | Irvine — reachable only via WireGuard tunnel from NH3 | GPU / AI inference (bare metal, RTX 3090 + RTX A6000, native stacks) | `servers/irv-ml1/README.md` |
 | ana-docker | 10.250.50.70 | Anaheim | General-purpose Docker host (non-GPU VM on pfi-pve) | `servers/ana-docker/README.md` |
 | pfi-ana-webhost | 10.250.50.52 | Anaheim | VM on pfi-pve (VMID 110) — web workload | `servers/pfi-ana-webhost/README.md` |
@@ -105,6 +114,7 @@ Observed and standardized across servers:
 | corviduo-dev | 10.250.50.152 | Anaheim | **Worldtree-team dev VM (PFI-hosted)** — runs the demo + personal + pinned Worldtree deployments vor/asset-engine talk to | `servers/corviduo-dev/README.md` |
 | nh3-docker | 10.100.50.40 | NH3 (`10.100.0.0/16`) | General-purpose Docker host (non-GPU VM on nh3-pve) | `servers/nh3-docker/README.md` |
 | nh3-dev | 10.100.10.50 | NH3 | Dev box — fleet sidecars (egress SOCKS5 proxy, ttyd seat, mead-hall, volva) + live Claude Code sessions; not a Docker-stack host | `servers/nh3-dev/README.md` |
+| nh3-extdev | 10.100.50.42 | NH3 | Manager / external-dev box (VM on nh3-pve, Debian 13); **sudo-less** infra-ops identity (user-level only, no Docker); successor to retired nh3-ansible | `servers/nh3-extdev/README.md` |
 | nh3-pve | 10.100.250.60 | NH3 | Proxmox VE hypervisor | `servers/nh3-pve/README.md` |
 | nh3-nas | 10.100.50.50 | NH3 | Synology RS2418+ — NFS exports, rest-server-nh3, PBS-NH3 datastore backend | `servers/nh3-nas/README.md` |
 | pbs-nh3 | 10.100.50.90 | NH3 | Proxmox Backup Server — DR mirror (VM on nh3-pve, NFS datastore on nh3-nas); syncs from pbs-ana | `servers/pbs-nh3/README.md` |
@@ -21,21 +21,32 @@ to the compose file and is gitignored.

 ## Layout convention

-`settings.yaml` drives the group layout:
+`settings.yaml` drives the group layout across four tabs:

 ```
-Monitoring        row x 3   fleet hubs (Beszel, Dozzle, Backrest, Uptime Kuma)
-AI Systems        row x 3   GPU inference services (llama-swap, vLLM embed/rerank)
-Apps              list      user-facing apps (Gitea, Vaultwarden, Seafile, ...)
-Media             list      Plex, Jellyfin
-Games             list      Pterodactyl
-UltraSeedbox      row x 3   external bookmarks
-Infra - ANA       list      Anaheim hardware + hypervisors + BMCs
-Infra - NH3       list      NH3 hardware + hypervisors
-Infra - ESH       list      ESH home-lab hardware + hypervisors
-Service Networking collapsed  toolchain (Traefik, CrowdSec, Dockge, AdGuard, MQTT)
+tab: Main
+  Notes / News / Monitoring / Apps / Media / Games / UltraSeedbox
+tab: AI              (the inference fleet, sorted by role)
+  AI - Inference        LLM seats you call (gen, char-rp, char-rp-reasoning, summarizer)
+  AI - Eval & Retrieval judges, reward, rerank, embed, image-quality
+  AI - Gateways & Chat  routing gateway, control plane, chat frontends
+  AI - Speech (TTS)     text-to-speech engines
+  AI - Audio Tools      speech-to-text + audio dataset tooling
+  AI - Image & Media    image/video generation + pipelines
+  AI - Dormant          stopped stacks (rollback seats, retired auditions)
+tab: Infrastructure
+  Infra - ANA / NH3 / IRV / ESH   hardware + hypervisors + BMCs, per site
+tab: Toolchain
+  Service Networking / Toolchain  plumbing, rarely clicked
 ```

+The AI tab replaced the old single flat `AI Systems` group (2026-07-14): a
+20+ service list read as one endless column, so it was split by function.
+Group membership is the `homepage.group=AI - <role>` label on each compose
+file; a label change only takes effect when the container is recreated
+(`docker compose up -d <svc>`, or `up --no-start <svc>` to relabel a stopped
+stack without starting it).
+
 - **Manual entries** (this file) cover things without a Docker label:
  firewalls, switches, NAS web UIs, BMCs, hypervisors, and the cross-site
  hubs where direct IP:port URLs are stable.
@@ -49,15 +60,18 @@ Service Networking collapsed  toolchain (Traefik, CrowdSec, Dockge, AdGuard, MQT
 When deciding where a service lands, ask **function first**:

 1. Does it watch or back up the fleet? -> `Monitoring`
-2. Is it an inference / model service? -> `AI Systems`
+2. Is it an inference / model service? -> the matching `AI - <role>` group
+   (Inference / Eval & Retrieval / Gateways & Chat / Speech (TTS) /
+   Audio Tools / Image & Media); a stopped-but-kept stack -> `AI - Dormant`
 3. Is it a user-facing app? -> `Apps`
 4. Is it media / games? -> `Media` or `Games`
 5. Is it a piece of hardware or a hypervisor? -> `Infra - <site>`
 6. Is it toolchain / plumbing (no human interaction on the golden path)? ->
   `Service Networking`

-Site-specific sub-grouping is only used for `Infra -` because the device
-inventory maps cleanly to physical sites. App groups are function-only.
+Site-specific sub-grouping is used for `Infra -` (device inventory maps to
+physical sites) and role-based sub-grouping for `AI -` (the fleet is large
+enough to warrant it). Other app groups are function-only.

 ## Deploying changes

@@ -71,12 +85,21 @@ Current workflow — push this directory onto the host:
 ```bash
 rsync -av --delete \
  --exclude='.env' --exclude='.env.*' \
+  --exclude='*.bak*' --exclude='logs/' \
  configs/homepage/ esh-docker-vm:/opt/docker/conf/homepage/
 ```

 The real `.env` lives on `esh-docker-vm` next to the compose file and must
 not be overwritten (holds Plex/Jellyfin keys).

+> **`--delete` footgun (learned 2026-07-20):** the host keeps dated
+> `services.yaml.bak-*` safety copies and a live `logs/` dir that are *not*
+> in this repo. A bare `--delete` rsync wipes both. The `--exclude='*.bak*'`
+> and `--exclude='logs/'` above protect them. For a one-file tweak, skip
+> `--delete` entirely and push the single file:
+> `rsync -av configs/homepage/services.yaml esh-docker-vm:/opt/docker/conf/homepage/services.yaml`
+> (back up the host copy first: `ssh esh-docker-vm 'cp -a …/services.yaml …/services.yaml.bak-<date>-<what>'`).
+
 The homepage container reloads most files on-change; if a new group in
 `settings.yaml` doesn't show up, `docker compose restart` on the host.

@@ -17,10 +17,22 @@
      siteMonitor: http://10.0.50.45:3001
      description: Uptime monitor (esh-docker-vm)

-# AI Systems group is fully Docker-auto-discovered (llama-swap, vLLM Embed,
-# vLLM Rerank — homepage.group=AI Systems on their compose files). Position
-# and row×3 style for the group live in settings.yaml. Do not add entries
-# here or they'll double up.
+- Apps:
+  # Manual entry — the Booth is a user-level systemd service on nh3-dev
+  # (not a Docker-labeled stack), so it can't auto-discover; list it here.
+  - The Booth:
+      href: http://10.100.10.50:8090/
+      icon: mdi-filmstrip
+      siteMonitor: http://10.100.10.50:8090/healthz
+      description: Ephemeral media drop + upload-for-pickup (human-readable ids) — nh3-dev, 24h TTL
+
+# The AI tab is fully Docker-auto-discovered. Each inference service carries
+# a homepage.group=AI - <role> label on its compose file (AI - Inference,
+# AI - Eval & Retrieval, AI - Gateways & Chat, AI - Speech (TTS),
+# AI - Audio Tools, AI - Image & Media). Tab assignment, group order, and
+# column counts live in settings.yaml. Do not add entries here or they'll
+# double up. To move a service between AI groups, change the label on its
+# compose file and recreate the container (labels only apply on recreate).

 - Media:
  - Plex:
@@ -27,11 +27,23 @@ statusStyle: ""
 # than plain link cards and the grid looks ragged.
 useEqualHeights: true

-# Function-first layout, three-tab split:
-#   Main           - daily-use apps, inference, media, bookmarks
+# Function-first layout, four-tab split:
+#   Main           - daily-use apps, media, bookmarks, monitoring
+#   AI             - the inference fleet, grouped by role (see below)
 #   Infrastructure - hardware, hypervisors, BMCs (per site)
 #   Toolchain      - backend services running but rarely clicked
 #
+# The AI tab splits the fleet by function so a 20+ service list reads as
+# sorted groups instead of one endless column. Group membership is set by
+# the homepage.group=AI - <role> label on each service's compose file:
+#   AI - Inference        LLM seats you call (gen, char-rp, char-rp-reasoning, summarizer)
+#   AI - Eval & Retrieval judges, reward, rerank, embed, image-quality
+#   AI - Gateways & Chat  routing gateway, control plane, chat frontends
+#   AI - Speech (TTS)     text-to-speech engines
+#   AI - Audio Tools      speech-to-text + audio dataset tooling
+#   AI - Image & Media    image/video generation + pipelines
+#   AI - Dormant          stopped stacks (rollback seats, retired auditions)
+#
 # Row counts target ~4-per-row so dense groups (Apps, Service Networking)
 # read as a grid instead of an endless column.
 layout:
@@ -50,11 +62,6 @@ layout:
    tab: Main
    style: row
    columns: 4
-  AI Systems:
-    icon: mdi-brain
-    tab: Main
-    style: row
-    columns: 4
  Apps:
    icon: mdi-apps
    tab: Main
@@ -73,6 +80,45 @@ layout:
    tab: Main
    style: row
    columns: 3
+  # --- AI tab: the inference fleet, ordered core-models -> support -> apps ---
+  AI - Inference:
+    icon: mdi-brain
+    tab: AI
+    style: row
+    columns: 4
+  AI - Eval & Retrieval:
+    icon: mdi-scale-balance
+    tab: AI
+    style: row
+    columns: 5
+  AI - Gateways & Chat:
+    icon: mdi-router-network
+    tab: AI
+    style: row
+    columns: 3
+  AI - Speech (TTS):
+    icon: mdi-account-voice
+    tab: AI
+    style: row
+    columns: 3
+  AI - Audio Tools:
+    icon: mdi-waveform
+    tab: AI
+    style: row
+    columns: 2
+  AI - Image & Media:
+    icon: mdi-image-multiple
+    tab: AI
+    style: row
+    columns: 2
+  # Stopped stacks kept for rollback / superseded seats / retired auditions.
+  # They stay 'created' (not running) via `docker compose up --no-start`, so
+  # they show here as offline cards and revive with `docker compose start`.
+  AI - Dormant:
+    icon: mdi-sleep
+    tab: AI
+    style: row
+    columns: 4
  Infra - ANA:
    icon: si-proxmox
    tab: Infrastructure
@@ -0,0 +1,45 @@
+# Arbo ComfyUI model catalog
+
+**Host:** irv-ml1 · **Path:** `/storetank/arbo/models` (SATA SSD; overlay-mounted into
+the arbo / comfyui container at `/basedir/models`). **502 G** as of 2026-06-13.
+
+The single live model tree arbo (hero / asset generation) consumes. On 2026-06-13 it
+**absorbed 177 G** of gen-agnostic utilities + the SDXL/Pony stack, migrated from the
+now-decommissioned `/storetank/image-models/comfy` archive — see
+[`storetank-image-models-archive.md`](storetank-image-models-archive.md) for that record.
+
+## Per-category sizes
+
+| Category | Size | Contents |
+|---|---|---|
+| `diffusion_models/` | **203 G** | current-gen generators: flux2-klein / wan2.2 / qwen-image / z-image / ideogram (GGUF + fp8) |
+| `checkpoints/` | **147 G** | SDXL / Pony / Illustrious bases — cyberrealisticPony_v180Coreshift (12.9 G), ponyRealism V22, novaAnimeXL, juggernaut/dreamshaper Lightning, lustify, hassaku, waiNSFWIllustrious, realDream + SUPIR upscalers |
+| `text_encoders/` | **80 G** | qwen3-VL, qwen2.5-VL, gemma, umt5, t5-xxl, clip variants |
+| `loras/` | **16 G** | flux2/wan2.2 (gameart, RetroAnimeFlux, flux1_turbo, zit_*) + migrated SDXL/Pony (dmd2_sdxl_4step, ACE++, character-design) |
+| `vae/` | 9.6 G | wan2.2 / flux2 / flux1 / z-image / sdxl / wan2.1 VAEs |
+| `Aura-SR/` | 9.3 G | AuraSR v1/v2 upscalers |
+| `LLM/` + `florence2/` | 8.6 + 3.6 G | Florence-2 PromptGen large/base + CogFlorence captioners |
+| `controlnet/` | 8.1 G | flux upscaler + sdxl union-promax |
+| `clip_vision/` | 4.4 G | CLIP-ViT-H, clip_vision_h, sigclip |
+| `upscale_models/` | 3.8 G | HAT / DAT / RealESRGAN / UltraSharp / Remacri / NMKD / Omni-SR (~50) |
+| `grounding-dino/` | 1.6 G | grounding-dino swinb / swint |
+| `ipadapter/` | 1.5 G | ip-adapter-plus / _sdxl vit-h |
+| `insightface/` | 1.3 G | inswapper_128 + antelopev2 |
+| `depthanything/` | 1.3 G | depth-anything v2 (vitl / vits) |
+| `facerestore_models/` | 937 M | GFPGAN v1.3/1.4, GPEN-BFR |
+| `RMBG/` `clip/` `sams/` `nsfw_detector/` `vitmatte/` `facexlib/` `ultralytics/` … | <1 G ea | bg-removal, EVA02-CLIP-L, SAM-HQ + SAM, nsfw classifier, matte, face-lib, yolo (face/hand/eyes/person) |
+
+## Migrated in 2026-06-13 (177 G from the storetank archive)
+
+The gen-agnostic utility set (upscalers, Florence-2 captioners, controlnet-union,
+grounding-dino / SAM / yolo / depthanything / vitmatte, insightface / facerestore,
+ip-adapter, CLIP-vision) **plus** the SDXL/Pony stack (bases + dmd2 / ACE++ /
+character-design loras). These work alongside arbo's current FLUX.2 / WAN2.2 / qwen
+generators; **comfy-dev** authors the per-model catalog entries + graphs + heroes that
+turn them into usable workflows.
+
+## Durability
+
+- `arbo_db` (gallery/history SQLite) — backed up (restic/Backrest), local disk not NFS.
+- The model tree itself is **bulk, reproducible-from-source** → not backed up; this
+  catalog + the migration record are the recovery map.
@@ -565,6 +565,144 @@ services:
      Three-way mutual-exclusion among emotion_voice / emotion_vector / emotion_text;
      precedence as above. UI should expose this as a single picker.

+  - id: omnivoice
+    name: OmniVoice
+    description: >
+      k2-fsa zero-shot, massively-multilingual (600+ language) voice-cloning TTS
+      (diffusion-LM, RTF ~0.025). Apache-2.0. Behind our own FastAPI wrapper
+      (stacks/omnivoice/app.py); voices are the reused chatterbox reference clips.
+    category: tts
+    version: 2
+    status: ready
+    host: irv-ml1
+    lifecycle:
+      stack: omnivoice
+      vram_gb: 6
+      gpu_device_id: 0
+    endpoint: http://10.100.79.3:8199/v1/audio/speech
+    method: POST
+    content_type: application/json
+    model:
+      id: k2-fsa/OmniVoice
+      revision: null
+      image: local/omnivoice:latest
+    fields:
+      - name: input
+        type: textarea
+        label: Text
+        required: true
+        max_length: 5000
+      # Voice source — at least one of voice (clone) / instruct (design) is required.
+      - name: voice
+        type: select
+        label: Speaker Voice (clone)
+        optional: true
+        source_url: http://10.100.79.3:8199/v1/audio/voices
+        source_jsonpath: $.voices[*]
+        description: >
+          Zero-shot clone target — a reference clip in /worktank/omnivoice/voices/
+          (reused chatterbox voices; 33 at deploy). Omit to design a voice via
+          instruct instead. Live list at /v1/audio/voices.
+      - name: instruct
+        type: text
+        label: Voice Design (instruct)
+        optional: true
+        source_url: http://10.100.79.3:8199/v1/audio/instruct-items
+        source_jsonpath: $.instruct_items[*]
+        description: >
+          Voice DESIGN — a comma-separated list of CONTROLLED attribute tags (not
+          free prose), e.g. "british accent, elderly, male, low pitch". Valid tags
+          (gender/age/pitch/accent/whisper) at /v1/audio/instruct-items. Use instead
+          of, or together with, a clone voice.
+      - name: language
+        type: select
+        label: Language
+        optional: true
+        default: Auto
+        source_url: http://10.100.79.3:8199/v1/audio/languages
+        source_jsonpath: $.languages[*]
+        description: "Auto-detects when left as Auto; 600+ languages supported."
+      - name: speed
+        type: slider
+        label: Speed
+        optional: true
+        min: 0.5
+        max: 1.5
+        default: 1.0
+        description: "1.0 = normal; >1 faster, <1 slower. Ignored if duration is set."
+      - name: duration
+        type: number
+        label: Duration (seconds)
+        optional: true
+        description: "Fixed output length in seconds; overrides speed when set."
+      - name: num_step
+        type: slider
+        label: Inference Steps
+        optional: true
+        min: 4
+        max: 64
+        default: 32
+        description: "Diffusion steps. Lower = faster, higher = better quality."
+      - name: guidance_scale
+        type: slider
+        label: Guidance Scale (CFG)
+        optional: true
+        min: 0.0
+        max: 4.0
+        default: 2.0
+      - name: denoise
+        type: bool
+        label: Denoise
+        optional: true
+        default: true
+      - name: preprocess_prompt
+        type: bool
+        label: Preprocess Prompt
+        optional: true
+        default: true
+        description: "Silence-trim + punctuate the reference (clone mode)."
+      - name: postprocess_output
+        type: bool
+        label: Postprocess Output
+        optional: true
+        default: true
+        description: "Remove long silences from the generated audio."
+      - name: generation_overrides
+        type: json
+        label: Advanced (GenerationConfig)
+        optional: true
+        description: >
+          Expert OmniVoiceGenerationConfig overrides as a JSON object — keys:
+          t_shift (0.1), layer_penalty_factor (5.0), position_temperature (5.0),
+          class_temperature (0.0), audio_chunk_duration (15.0),
+          audio_chunk_threshold (30.0). Unknown keys ignored.
+      - name: response_format
+        type: select
+        options: [wav]
+        default: wav
+        description: 24000 Hz PCM_16 mono only; no negotiation.
+    response:
+      type: audio
+      mime: audio/wav
+    reproducibility:
+      seedable: false
+      deterministic: false
+      notes: >
+        Diffusion-LM, temperature/denoise sampled — not byte-exact, no seed exposed.
+        Output 24000 Hz PCM_16 mono. Voice = a cloned reference clip (clone prompt
+        precomputed per voice at startup; Whisper auto-transcribes the reference).
+    estimated_latency:
+      cold_start_s: 600
+      warm_per_unit: "full-utterance (no streaming)"
+    license: "Apache-2.0"
+    notes: |
+      Two voice sources, combinable: voice (clone a staged reference clip) and/or
+      instruct (free-text voice DESIGN); at least one required. Full generation
+      surface exposed — language (600+), speed, duration, num_step, guidance_scale,
+      denoise, preprocess/postprocess — with expert GenerationConfig knobs (t_shift,
+      layer/position/class temperature, audio_chunk_*) via the generation_overrides
+      JSON field. No streaming. Voices reused from chatterbox /refs.
+
  - id: qwen3-tts
    name: Qwen3-TTS 1.7B
    description: >
@@ -2174,6 +2312,289 @@ services:
      source for defaults/ranges. Adapter not yet deployed/verified — flip to
      ready (or experimental) after the first successful generation through 8203.

+  - id: zonos-gateway
+    name: Zonos Gateway (expressive)
+    description: >
+      OpenAI-compatible streaming facade over the Zonos engine (kept stock),
+      exposing Zonos's full expressive control surface: emotion directions
+      (happy / sad / angry / surprised) plus a valence/arousal axis pair,
+      classifier-free-guidance on emotion, accurate-vs-expressive mode,
+      speaking-rate conditioning, quality-metric targets, and the full
+      sampling stack — all reachable from named presets (neutral / warm /
+      excited / sad / intense / whisper) that seed the dials before explicit
+      overrides win. Streams s16le PCM (or a WAV wrapper) from
+      /v1/audio/speech. The LiteLLM `ext-tts` alias points at this gateway.
+    category: tts
+    version: 1
+    status: experimental
+    host: irv-ml1
+    lifecycle:
+      stack: zonos-gateway
+      vram_gb: 16
+      gpu_device_id: 0
+    endpoint: http://10.100.79.3:8890/v1/audio/speech
+    method: POST
+    content_type: application/json
+    streamable: true
+    model:
+      id: Zyphra/ZONOS2
+      revision: null
+      image: local/zonos-gateway:0.1.0
+    section_groups:
+      - id: basic
+        label: Text & voice
+      - id: expression
+        label: Expression
+        hint: Emotion conditioning. A preset seeds these; explicit dials win.
+      - id: prosody
+        label: Prosody
+        hint: Speaking-rate conditioning. Leave the enable toggles off for the model's native pacing.
+      - id: quality
+        label: Quality target
+        hint: Advanced — raw metric targets (LUFS, silence, bandlimit) Zonos buckets internally.
+      - id: sampling
+        label: Sampling
+      - id: output
+        label: Output
+    fields:
+      - name: input
+        type: textarea
+        label: Text to synthesize
+        section: basic
+        required: true
+        max_length: 5000
+        description: >
+          Text to speak. OpenAI-style `input` field; the gateway streams the
+          synthesized audio back.
+      - name: voice
+        type: select
+        label: Voice
+        section: basic
+        default: Cora
+        source_url: http://10.100.79.3:8890/v1/voices
+        source_jsonpath: $.voices[*].name
+        description: >
+          Predefined Zonos voice. Live-enumerated from /v1/voices so the list
+          auto-syncs with the deployed voice pack (Cora is the default).
+      - name: preset
+        type: select
+        label: Expressive preset
+        section: expression
+        required: false
+        options: [neutral, warm, excited, sad, intense, whisper]
+        default: neutral
+        description: >
+          Named expressive preset applied before explicit dials; any explicit
+          emotion/prosody/quality dial you set overrides the preset's value.
+      - name: emotion_enabled
+        type: bool
+        label: Enable emotion conditioning
+        section: expression
+        required: false
+        default: false
+        description: >
+          Turn emotion conditioning on. Required for the emotion_* dials to
+          bite — a preset that sets emotion turns this on for you.
+      - name: emotion_valence
+        type: slider
+        label: Valence
+        section: expression
+        min: -1.0
+        max: 1.0
+        step: 0.05
+        default: 0.0
+        description: Pleasantness axis. -1 negative, +1 positive.
+      - name: emotion_arousal
+        type: slider
+        label: Arousal
+        section: expression
+        min: -1.0
+        max: 1.0
+        step: 0.05
+        default: 0.0
+        description: Energy/activation axis. -1 calm, +1 excited.
+      - name: emotion_strength
+        type: slider
+        label: Emotion strength
+        section: expression
+        min: 0.0
+        max: 2.0
+        step: 0.05
+        default: 1.0
+        description: Overall scale on the emotion direction. 1.0 = as specified.
+      - name: emotion_cfg_scale
+        type: slider
+        label: Emotion CFG scale
+        section: expression
+        min: 1.0
+        max: 3.0
+        step: 0.1
+        default: 1.0
+        description: >
+          Classifier-free-guidance on emotion. 1.0 = off; >1 amplifies
+          expression.
+      - name: emotion_sliders
+        type: json
+        label: Per-emotion weights (advanced)
+        section: expression
+        optional: true
+        description: >
+          Advanced — per-emotion weight dict {happy|sad|angry|surprised: -1..1};
+          higher = stronger. Overrides the coarse valence/arousal directions
+          with explicit per-emotion control. Omit to use valence/arousal.
+      - name: accurate_mode
+        type: bool
+        label: Accurate mode
+        section: expression
+        required: false
+        default: true
+        description: >
+          true = faithful to the reference voice; false = more
+          expressive/looser.
+      - name: speaking_rate_enabled
+        type: bool
+        label: Enable speaking-rate conditioning
+        section: prosody
+        required: false
+        default: false
+        description: >
+          Turn speaking-rate conditioning on. Required for speed /
+          speaking_rate / speaking_rate_bucket to take effect.
+      - name: speed
+        type: slider
+        label: Speed (OpenAI-style)
+        section: prosody
+        min: 0.25
+        max: 4.0
+        step: 0.05
+        optional: true
+        description: >
+          OpenAI-style rate multiplier. Mapped to speaking_rate when no
+          explicit speaking_rate is given; auto-enables speaking-rate
+          conditioning. Omit to leave pacing native.
+      - name: speaking_rate
+        type: slider
+        label: Speaking rate (native)
+        section: prosody
+        min: 0.25
+        max: 4.0
+        step: 0.05
+        optional: true
+        description: >
+          Native speaking-rate multiplier. Overrides speed if both are sent.
+          Omit to leave pacing native.
+      - name: speaking_rate_bucket
+        type: slider
+        label: Speaking-rate bucket
+        section: prosody
+        min: 0
+        max: 7
+        step: 1
+        optional: true
+        description: >
+          Words/sec bucket index 0..7 (0 = 0-8 wps … 7 = 40+ wps). Coarser than
+          speaking_rate. Omit to leave pacing native.
+      - name: quality_enabled
+        type: bool
+        label: Enable quality-target conditioning
+        section: quality
+        required: false
+        default: true
+        description: >
+          Advanced — turn quality-target conditioning on (on by default in
+          Zonos). Gates quality_values.
+      - name: quality_values
+        type: json
+        label: Quality metric targets (advanced)
+        section: quality
+        optional: true
+        description: >
+          Advanced — raw metric targets Zonos buckets internally, e.g.
+          {lufs: -23, trailing_silence_s: 0.1}. Keys: lufs, estimated_snr,
+          max_pause, estimated_bandlimit_hz, leading_silence_s,
+          trailing_silence_s. Omit for Zonos's defaults.
+      - name: temperature
+        type: slider
+        section: sampling
+        min: 0.0
+        max: 2.0
+        step: 0.05
+        default: 1.15
+        description: Sampling temperature. Higher = more varied. Zonos default 1.15.
+      - name: top_p
+        type: slider
+        label: Top-p
+        section: sampling
+        min: 0.0
+        max: 1.0
+        step: 0.05
+        default: 0.0
+        description: Nucleus sampling cutoff. 0.0 = off (Zonos default).
+      - name: min_p
+        type: slider
+        label: Min-p
+        section: sampling
+        min: 0.0
+        max: 1.0
+        step: 0.01
+        default: 0.18
+        description: Min-p sampling floor. Zonos default 0.18.
+      - name: topk
+        type: number
+        label: Top-k
+        section: sampling
+        required: false
+        default: 106
+        description: Top-k sampling cutoff. Zonos default 106.
+      - name: seed
+        type: number
+        section: sampling
+        optional: true
+        description: >
+          RNG seed for reproducible sampling. Omit for a random seed. Pins the
+          sampler only; emotion/quality conditioning still varies subtly.
+      - name: max_tokens
+        type: number
+        label: Max audio tokens
+        section: sampling
+        required: false
+        max: 6144
+        description: >
+          Cap on generated audio tokens (upper bound; Zonos stops at
+          end-of-speech). Omit to let Zonos decide.
+      - name: response_format
+        type: select
+        label: Response format
+        section: output
+        options: [pcm, wav]
+        default: pcm
+        description: >
+          pcm = raw s16le stream (lowest latency, for API consumers); wav adds
+          a header. The stream-audition UI forces wav for the browser <audio>.
+    response:
+      type: audio
+      mime_from_field: response_format
+    reproducibility:
+      seedable: true
+      deterministic: false
+      seed_field: seed
+      notes: >
+        Temperature-sampled; seed pins the sampler but emotion/quality
+        conditioning still varies subtly run-to-run.
+    estimated_latency:
+      cold_start_s: 3
+      warm_per_unit: "streaming; first audio in a couple seconds warm, then near-realtime on the 3090"
+    license: Apache-2.0
+    notes: |
+      OpenAI-compatible streaming gateway (local/zonos-gateway:0.1.0) fronting a
+      stock Zonos engine on the 3090 (irv-ml1 device 0). The LiteLLM `ext-tts`
+      alias routes here. Fields mirror the gateway's /v1/dials schema (24 params;
+      the CATALOG-CONTRACT blessed source for defaults/ranges) and /v1/voices.
+      Deliberately omits repetition_window / repetition_penalty / codebooks — the
+      wrapper rejects them and they are the "70s of silence" footgun. Presets seed
+      the dials before explicit overrides win. New service (experimental) — flip to
+      ready after the first verified generation + browser audition through 8890.
+
 # Reproducibility audit — answers per service: (a) seedable, (b) model
 # deterministic without seed, (c) image tag mutable (security/reproducibility risk).
 reproducibility_audit:
@@ -2207,6 +2628,11 @@ reproducibility_audit:
    model_deterministic: true
    image_tag_mutable: false
    notes: "22050 Hz hardcoded — caller must resample."
+  - service: omnivoice
+    seedable: false
+    model_deterministic: false
+    image_tag_mutable: true
+    notes: "Diffusion-LM, temperature/denoise sampled — not byte-exact, no seed exposed. 24000 Hz PCM_16 mono. image local/omnivoice:latest is mutable — pin a digest for true repro. Voices = reused chatterbox /refs clones (clone prompt precomputed per voice at startup)."
  - service: qwen3-tts
    seedable: false
    model_deterministic: true
@@ -2269,3 +2695,8 @@ reproducibility_audit:
    model_deterministic: true
    image_tag_mutable: false
    notes: "Adapter echoes the seed used (reproducibility.seed_field=seed). Byte-stable same-GPU; bf16 may drift cross-GPU. local/zonos-api:v1 built FROM local/zonos (pin ZONOS_SHA for true repro)."
+  - service: zonos-gateway
+    seedable: true
+    model_deterministic: false
+    image_tag_mutable: true
+    notes: "Seed pins the sampler (reproducibility.seed_field=seed) but emotion/quality conditioning still varies subtly run-to-run — not byte-exact. Streaming (s16le PCM / WAV). Distinct from the `zonos` adapter: this is the OpenAI-compatible gateway on :8890 behind the LiteLLM `ext-tts` alias. image local/zonos-gateway:0.1.0 is tag-pinned + mutable — pin a digest for true repro."
@@ -164,6 +164,25 @@ These caught us once; don't let them catch you twice.
 - **irv-ml1 was `ana-ml1`** before a physical move; OS hostname still
  says `ana-ml1` pending an explicit rename. Doesn't affect services.

+### Git / gitea
+
+- **Colo/fleet hosts must reach gitea over the INTERNAL route, not the
+  public IP.** `gitea.phasefinal.com` resolves to the **public** IP
+  `38.120.12.44` (ana-srv1); gitea itself is a container on **ana-docker**
+  with git-SSH at **`10.250.50.70:222`** (`222→22`) and HTTP at `:3000`.
+  A fleet host that egresses to the public `:22` gets its egress IP
+  **fail2ban-banned** after any retrying git/deploy loop, which silently
+  wedges automation — e.g. a gitea-webhook auto-deploy whose `git fetch`
+  then times out under `set -euo pipefail` and never reaches the `reset`.
+  Point each host's gitea ssh alias at `HostName 10.250.50.70` /
+  `Port 222` with the repo deploy key; the internal route is ban-immune
+  and treats the cause. Bit irv-ml1's arbo deploy on 2026-06-13 (the
+  `gitea-arbo` alias pointed at the public host → fetch timeout → the
+  v0.11.7 frontend wouldn't serve until the alias was repointed internal).
+- **`:22` on `10.250.50.70` is ana-docker's HOST sshd, not gitea.** A
+  gitea deploy key there returns `Permission denied (publickey)` — gitea's
+  git-SSH is the container port `:222`. (HTTP/clone-over-HTTPS is `:3000`.)
+
 ### Workflow

 - **Terminal word-wrap breaks long pasted commands.** Never embed a
@@ -0,0 +1,76 @@
+# Canonical sampler defaults — PFI/VastBlue LiteLLM gateway seats
+
+**Applied:** 2026-07-08 · **Gateway:** `ana-docker:4000` · **Config:** `stacks/litellm/conf/config.yaml` → `/opt/docker/conf/litellm/config.yaml`
+
+Canonical high-quality sampler defaults for the four model seats, **derived by
+dvalin-smithy-dev** (full rationale + sources: `dvalin-smithy/hoard-drafts/pfi-gateway-sampler-defaults-20260708.md`),
+**triaged + A/B-validated by infra-ops**, and wired into the gateway. These are the
+gateway *defaults*; callers may override per request.
+
+Optimized for **output / prose quality** (not throughput or determinism).
+
+## Engine surfaces
+
+- **gen / gen-reasoning** — vLLM 0.24 (OpenAI sampler surface). No native DRY/XTC → anti-repetition via `presence_penalty`. Thinking split via `chat_template_kwargs.enable_thinking` on distinct `--served-model-name`s (avoids the shared-config-mutation footgun).
+- **char-rp / char-rp-reasoning** — llama.cpp / llama-server (supports `min_p`, `top_k`, DRY, XTC, dynatemp). `min_p` + `top_p` do the tail work; `top_k 0` disables top-k.
+
+## The four seats (applied values)
+
+### 1. gen — Qwen3.6-35B-A3B heretic (vLLM, non-thinking)
+Also governs **summarizer-large** (shares the same `qwen3.6-27b-aeon` @ :8015 deployment → kept identical).
+
+| param | value |
+|---|---|
+| temperature | 0.7 |
+| top_p | 0.80 |
+| top_k | 20 |
+| presence_penalty | **1.5** |
+| repetition/frequency | 1.0 / 0.0 |
+| enable_thinking | false |
+
+*Source:* Qwen3.6 README instruct/non-thinking rec. *Change:* presence_penalty 1.0 → 1.5.
+
+### 2. gen-reasoning — same model (vLLM, thinking)
+
+| param | value |
+|---|---|
+| temperature | **1.0** |
+| top_p | 0.95 |
+| top_k | 20 |
+| presence_penalty | **1.5** |
+| repetition/frequency | 1.0 / 0.0 |
+| enable_thinking | true |
+
+*Source:* Qwen3.6 README **general** thinking profile (NOT the temp-0.6 coding sub-profile — the prior default was that coding profile by mistake). *Changes:* temperature 0.6 → 1.0, presence_penalty 1.0 → 1.5. Reasoning is verbose (~9k chars) → callers set generous `max_tokens` (catalog default 32768). Optional per-route coding override: temp 0.6 / presence 0.0.
+
+### 3. char-rp — Magidonia-24B-v4.3 (llama.cpp, non-thinking prose RP)
+
+| param | value |
+|---|---|
+| temperature | **1.1** |
+| top_p | 0.95 |
+| min_p | **0.10** |
+| top_k | 0 (disabled) |
+| repetition/DRY/XTC | **off** |
+
+*Source:* dvalin canonical (Mistral-Small RP prose) **A/B-validated by infra-ops** on the live serve. *Changes:* temp 1.0 → 1.1, min_p 0.03 → 0.10. **min_p 0.10 richened imagery vs 0.03** with no incoherence at temp 1.1. **repeat_penalty 1.05 was REJECTED** — in the A/B it injected a stray markdown title into a grief scene; rep-style penalties hurt Drummer/Magistral RP creativity (matches the model card and dvalin's own note). Alt prose model: `MS3.2-PaintedFantasy-v4.1-24B` (swap via the `char-rp-gguf` stack `.env`).
+
+### 4. char-rp-reasoning — Qwen3.5-27B-Deckard-PKD (llama.cpp, managed-reasoning RP)
+
+| param | value (request-level) |
+|---|---|
+| temperature | 1.0 |
+| top_p | 0.95 |
+| top_k | 40 |
+| min_p | **0.05** |
+| presence/repetition | **off** |
+| DRY | **0.8 server-side** (base 1.75 / len 2, dry-after-temp) — not a request param |
+| reasoning-budget | 400 (server-side) |
+
+*Source:* dvalin-CONFIRMED canonical 2026-07-08 (thread 01KX1Y7P). **Corrected 2026-07-09:** this seat had lagged on QwQ-RpR-v4 — the A/B on 2026-07-08 replaced it with **Deckard-PKD-Heretic i1-Q5_K_M** (DavidAU, Qwen3.5-27B, :8018); the live gateway was always Deckard. Deckard won on brokkr's frozen scorer (0/30 loops, 0/30 refusals) over RpR-v4 (1/30 loop, forbids DRY) + Pantheon-27B (7/30 refusals). Reasoning ON server-side (`--reasoning on`, budget 400); CoT surfaces in `reasoning_content`, clean prose in `content`. Tuning ladder: flat prose→min_p 0.08, loops→DRY 0.9, over-damped→DRY 0.6/off. **Do NOT import RpR/QwQ sampler rules** (different family — QwQ hated DRY; Qwen3.5 benefits from it).
+
+## Changing a default
+
+Edit the seat's `litellm_params` in `stacks/litellm/conf/config.yaml`, `scp` to
+`/opt/docker/conf/litellm/config.yaml` on ana-docker, `docker restart litellm`.
+(`gen` and `summarizer-large` must change together — same deployment.)
@@ -16,6 +16,7 @@
 6. [Quick Reference Cards](#6-quick-reference-cards)
 7. [Critical Warnings by Model](#7-critical-warnings-by-model)
 8. [Models Without KB Settings](#8-models-without-kb-settings)
+9. [PFI LiteLLM Gateway — Deployed Sampling Defaults](#9-pfi-litellm-gateway--deployed-sampling-defaults)

 ---

@@ -539,6 +540,51 @@ The following model families are deployed in the Infrastructure-PFI environment

 ---

+## 9. PFI LiteLLM Gateway — Deployed Sampling Defaults
+
+> **Live as of 2026-06-27** on the PFI gateway (`ana-docker:4000`; canonical config
+> `eshpfi-management/stacks/litellm/conf/config.yaml`). Unlike §§1–8 (general vendor
+> reference), this section is the **deployed reality** — keep it in sync when gateway
+> sampling changes.
+
+These are **overrideable defaults**: any caller that passes its own sampling param
+wins; callers that omit one inherit the value below. (Verified — vLLM rejected an
+out-of-range `presence_penalty=5.0`, proving per-request values reach the backend and
+override the config default.) Values set per the `dvalin-smithy-dev` research pass
+(provenance-cited in-thread, corroborated by §3 above). vLLM-only params (`top_k`,
+`repetition_penalty`) ride in `extra_body` so LiteLLM's `drop_params` can't strip them.
+
+| Gateway model(s) | temp | top_p | top_k | presence_penalty | repetition_penalty | Source |
+|---|---|---|---|---|---|---|
+| `granite-4.1-8b`, `summarizer`, `classifier` | **0** | — | — | — | — | IBM-canonical (temp 0 for inferencing) |
+| `gen`, `summarizer-large`, `qwen-large`, `qwen3.5-122-a10b` (non-thinking) | **0.7** | 0.8 | 20 | **1.0** | — | Qwen3 non-thinking + operator anti-repetition |
+| `gen-reasoning`, `qwen-large-reasoning`, `qwen3.5-122-a10b-reasoning` (thinking) | **0.6** | 0.95 | 20 | **1.0** | — | Qwen3 thinking |
+| `qwen-image-bench`, `image-judge` | **0** | 1.0 | 1 | — | 1.05 | Qwen-Image-Bench judge reproducibility table |
+| `selene-1-mini-8b`, `chat-judge` | **0.6** | 0.9 | — | — | — | Selene `generation_config` |
+| `glm-5.1`, `glm-5.2`, `glm-5-turbo`, `glm-4.7`, `gen-frontier` | **1.0** | 0.95 | — | — | — | z.ai API defaults (5.x / 4.7 series) |
+| `glm-4.5-air` | **0.6** | 0.95 | — | — | — | z.ai API default (4.5 series) |
+| `qwen3-embedding`, `qwen3-reranker`, `reranker` | — | — | — | — | — | no sampling (embedding / rerank) |
+
+**Notes:**
+- **qwen "gen" family `presence_penalty: 1.0`** — operator-set anti-repetition for the
+  abliterated/NVFP4 Qwopus 122B-A10B. Qwen documents `presence_penalty` (0–2) as *the*
+  repetition lever; 1.0 is conservative (the §3 vendor general value is 1.5 — step up to
+  1.5 if loops persist). Do **not** use `repetition_penalty` for the Qwen3 family.
+- **GLM (z.ai cloud) — only `temperature` + `top_p` are set.** z.ai's chat API schema
+  accepts no `top_k` / `min_p` / penalties, so they're deliberately not sent (would be
+  silently dropped). These temps match z.ai's own API defaults (explicit-over-implicit /
+  future-proofing).
+- **Both `temp 0` values (granite, image-judge) are research-confirmed, not heuristic.**
+  Greedy is correct for constrained summ/classify (IBM) and for judge reproducibility
+  (Qwen judge card + LLM-as-judge practice). `temp 0.1` was explicitly evaluated and
+  rejected: it adds sampling noise without fixing loops, and *reduces* run-to-run score
+  consistency on the judge. If granite ever loops in production, fix via
+  `repetition_penalty` / `presence_penalty` / `max_tokens`, not a temperature floor.
+- **`qwen-image-bench` / `image-judge` is arbo's hero-judge** (comfy-dev consumer) —
+  sampling changes there are a coordination item, not a unilateral gateway edit.
+
+---
+
 ## KB Source Documents

 | Document | Path in KB |
@@ -0,0 +1,178 @@
+# Fleet backup architecture & freshness runbook
+
+The map that was missing: what backs up what, where it lands, and **how
+to check in 2 minutes whether backups are actually fresh.** Companion to
+[`disaster-recovery.md`](disaster-recovery.md) (which covers *recovery*
+when a host/service is down). Read this one first when the question is
+"are we backed up?"
+
+> **Why this exists:** on 2026-06-20 diagnosing "are backups OK?" took a
+> long exploration because the topology lived only in scattered memory.
+> The ana-side restic layer had been failing **silently for ~6.5 weeks**
+> (last good snapshot 2026-05-06) and nobody knew. This doc + a future
+> freshness alert is the fix.
+
+---
+
+## TL;DR — coverage matrix
+
+Two independent layers. **PBS = whole-VM images. restic = granular
+file+DB.** A host is well-covered if it has *either* a current PBS image
+*or* a current restic snapshot; the danger zone is a host whose **only**
+layer has failed.
+
+| Host | Kind | PBS (VM image) | restic (file+DB) | Sole net? |
+|---|---|---|---|---|
+| ana-docker | VM (pfi-pve) | ✅ `ana-pve` | ✅ → rest-server-**ana** | no |
+| **ana-ml2** | **bare metal** | ❌ none (not a VM) | ✅ → rest-server-**ana** | ⚠️ **restic is the ONLY net** |
+| **irv-ml1** | **bare metal** | ❌ none (not a VM) | ✅ → rest-server-**nh3** | ⚠️ **restic is the ONLY net** |
+| nh3-docker | VM (nh3-pve) | ✅ `nh3-pve` | ✅ → rest-server-**nh3** | no |
+| esh-docker-vm | VM (esh-pve) | ✅ `esh-pve` | ✅ → rest-server-**ana** | no |
+| esh-vm-db | VM (esh-pve-nas) | ❌ **none** (esh-pve-nas not a PBS source) | ✅ → rest-server-**ana** | ⚠️ **restic-only (a DB!)** |
+| vm-esh-nas | VM (esh-pve-nas) | ❌ **none** (esh-pve-nas not a PBS source) | ✅ → rest-server-**ana** | ⚠️ **restic-only** |
+| other pfi-pve / nh3-pve VMs/CTs | VM/CT | ✅ respective ns | (PBS only) | no |
+| SureFire `sfsrv-pve` | tenant VMs | ✅ `sfsrv-pve` ns | (PBS only) | no |
+
+**Bare-metal hosts have NO PBS coverage** (PBS only backs up Proxmox
+guests). Their restic snapshot is the entire safety net — keep an eye on
+it. ana-ml2 → rest-server-ana; irv-ml1 → rest-server-nh3.
+
+---
+
+## Layer 1 — PBS (whole-VM/CT images)
+
+- **PBS-ANA** (`pbs-ana`, 10.250.50.90) — fleet primary. Datastore is
+  NFS-backed: `10.250.50.50:/mnt/backup/pbs-ana` mounted at
+  `/mnt/pbs-datastore` (~20 TB). Backs up Proxmox guests via vzdump,
+  organised by **namespace per source hypervisor**:
+  - `ana-pve` — pfi-pve guests (ana-docker, pfi-postgres VM105, ana-nas
+    CT109, webhost, filebot, pteradactyl, tacticalrmm, ana-wg, …)
+  - `esh-pve` — esh-pve guests
+  - `nh3-pve` — nh3-pve guests
+  - `sfsrv-pve` — SureFire tenant
+  - ⚠️ there is **no `esh-pve-nas` namespace** — guests on that
+    hypervisor (vm-esh-nas, likely esh-vm-db) are **not** PBS-covered.
+- **PBS-NH3** (`pbs-nh3`, 10.100.50.90) — DR mirror; syncs from PBS-ANA
+  (datastore on nh3-nas).
+- Schedule: vzdump jobs defined in Proxmox (Datacenter → Backup),
+  staggered through the early morning.
+
+## Layer 2 — restic (granular file + DB)
+
+restic clients push to one of **two rest-server endpoints** (HTTP, basic
+auth, append-only, private repos). The split is by site:
+
+| rest-server | Endpoint | Backing store | Clients |
+|---|---|---|---|
+| **rest-server-ana** | `http://10.250.50.70:8000` (container `rest-server` on ana-docker) | `ana-nas:/mnt/backup/restic/repo/ana` (NFS bind → `/data`) | ana-docker, **ana-ml2**, esh-docker-vm, esh-vm-db, vm-esh-nas |
+| **rest-server-nh3** | `http://10.100.50.50:8000` (on nh3-nas) | `nh3-nas:/volume1/Backup/restic/<client>` | **irv-ml1**, nh3-docker |
+
+- Per-client repos live as subdirs of the rest-server data dir
+  (`.../repo/ana/<client>/` for the ana side); the shared `.htpasswd`
+  for ana sits at `.../repo/ana/.htpasswd`.
+- **Scheduler = `resticprofile` systemd timers on each client host**, NOT
+  Backrest:
+  - `resticprofile-backup@profile-default.timer` — daily **01:00** PDT
+  - `resticprofile-check@profile-default.timer` — weekly (Sun **05:00**)
+- **Backrest** (container on ana-docker, UI) is only a **repo viewer here
+  — it has 0 plans.** Do not assume "Backrest healthy" means "backups
+  running." The timers are the source of truth.
+- ⚠️ **Failures are silent** — a timer fires, restic errors against a
+  down endpoint, and nothing alerts. (See Known gaps.)
+
+---
+
+## The 2-minute freshness check
+
+Run these any time you need to answer "are we backed up?"
+
+```bash
+# --- restic ANA side: newest snapshot per client (want: today/yesterday) ---
+ssh ana-nas 'for c in ana-docker ana-ml2 esh-docker-vm esh-vm-db vm-esh-nas; do
+  echo -n "$c: "; ls -t /mnt/backup/restic/repo/ana/$c/snapshots/ 2>/dev/null | head -1 \
+    | xargs -I{} stat -c "%y" /mnt/backup/restic/repo/ana/$c/snapshots/{} 2>/dev/null || echo MISSING
+done'
+
+# --- restic NH3 side ---
+ssh nh3-nas 'for c in irv-ml1 nh3-docker; do
+  echo -n "$c: "; ls -lt /volume1/Backup/restic/$c/snapshots/ 2>/dev/null | sed -n 2p
+done'
+
+# --- rest-server endpoints healthy? (401 = up & serving; Restarting = broken) ---
+ssh infra-ops@ana-docker 'sudo docker ps --format "{{.Names}}\t{{.Status}}" | grep rest-server'
+curl -s -o /dev/null -w 'rest-server-ana: %{http_code}\n' http://10.250.50.70:8000/
+curl -s -o /dev/null -w 'rest-server-nh3: %{http_code}\n' http://10.100.50.50:8000/
+
+# --- PBS: newest snapshot per guest, all namespaces ---
+ssh pbs-ana 'for ns in /mnt/pbs-datastore/ns/*/; do nsn=$(basename "$ns")
+  for d in vm ct; do for g in "$ns$d"/*/; do [ -d "$g" ] || continue
+    echo "$nsn/$d/$(basename "$g") -> $(ls "$g" 2>/dev/null | grep ^20 | sort | tail -1)"
+  done; done; done'
+```
+
+**Force a backup now (don't wait for 01:00):** on the client host,
+`ssh infra-ops@<host> 'sudo systemctl start resticprofile-backup@profile-default.service'`
+(it's an incremental against the existing repo — bounded even if stale).
+
+---
+
+## Known failure mode: rest-server-ana crash-loop (the 2026-05-06 → 2026-06-20 outage)
+
+**Symptom:** `rest-server` container on ana-docker stuck `Restarting`;
+logs show `cannot load /data/.htpasswd: permission denied`. All ana-side
+restic backups silently fail.
+
+**Root cause:** ana-nas's NFS mount on ana-docker uses bare `defaults` in
+`/etc/fstab` (no `_netdev`, no retry). When the mount drops,
+`mnt-backup.mount` gets stuck `failed`, so `/mnt/backup/restic/repo/ana`
+resolves to an **empty local ghost dir** (no `.htpasswd`) and rest-server
+binds *that*. ana-nas itself is fine — the real repos are intact.
+
+**Recovery** (needs root on ana-docker — use **`ssh infra-ops@ana-docker`**,
+which has NOPASSWD sudo; the default `ssh ana-docker` lands as `lkraven`
+*without* sudo):
+
+```bash
+ssh infra-ops@ana-docker '
+  sudo mount -a                                   # re-attach the NFS (bypasses the failed unit)
+  sudo systemctl reset-failed mnt-backup.mount    # clear the stuck unit state
+  mount | grep /mnt/backup                         # confirm nfs4 attached
+  sudo ls /mnt/backup/restic/repo/ana/.htpasswd    # real htpasswd now present
+  cd /opt/docker/compose/rest-server-ana && sudo docker compose up -d --force-recreate
+'                                                  # recreate so the bind re-resolves onto NFS
+# verify: docker ps shows Up (healthy); curl :8000 -> 401; logs say "Loaded htpasswd file"
+```
+
+If the ghost dir blocks the mount, see `disaster-recovery.md` Tier-0 for
+the stop→umount→rm-ghost→remount→start variant.
+
+---
+
+## Known gaps / TODO
+
+- [x] **Backup-freshness alerting — DONE (2026-06-20).**
+      `scripts/check-backup-freshness.sh` (the 2-min check, exit 1 on
+      stale/down) + a daily **systemd user timer on nh3-dev** at 08:00
+      (`scripts/install-backup-freshness-timer.sh`) → `backup-freshness-alert.sh`
+      posts an **althing alert to infra-ops** on any stale/down layer. Run the
+      check by hand anytime. (Channel is althing for now — swap in email/ntfy if
+      you want a louder one.)
+- [x] **fstab hardening — DONE (2026-06-20).** ana-docker `/mnt/backup` →
+      `noauto,x-systemd.automount,x-systemd.mount-timeout=30` (autofs self-heals
+      on a NAS blip instead of getting stuck `failed`; activates on next reboot).
+      `/etc/fstab.bak-pre-harden` saved. **`/mnt/compose` also hardened the same
+      way** and **activated live** (umount → `mnt-compose.automount` started →
+      autofs verified remounting on access) — it binds no container, so it was
+      safe to convert now; this also proved the autofs pattern works on ana-docker.
+- [ ] **ana-ml2 has no PBS net** (bare metal) — restic is its only layer; now
+      healthy + alerted. Bulk `/tank` models are re-downloadable; bespoke
+      quants/configs/scripts are the real loss-risk.
+- [x] **esh-pve-nas coverage — VERIFIED (2026-06-20): NOT PBS-covered.** No
+      `esh-pve-nas` namespace exists on PBS-ANA, so **esh-vm-db (postgres+mongo)
+      + vm-esh-nas are restic-only.** For the DB VM, restic-with-dumps is the
+      *preferred* method (vs a VM image) **IF** the resticprofile includes
+      `pg_dump`/`mongodump` — confirm that. Optionally add esh-pve-nas as a PBS
+      source. ESH is home-lab (no SLA).
+- [ ] **Rotate rest-server repo passwords** — the 5 per-repo basic-auth creds
+      were exposed during the 2026-06-20 diagnosis. **BELAYED** — operator
+      handling offline.
@@ -0,0 +1,144 @@
+# Heretic2 NVFP4 + MTP fast char-rp-reasoning seat — the working recipe
+
+**Status: WORKING (2026-07-14).** ~77 tok/s single-stream (vs GGUF NEO-CODE ~59.5, base
+NVFP4 ~53) — **~1.3× over GGUF**, MTP draft-acceptance **32–40%**, mean acceptance length
+**2.19**. This is a drop-in faster replacement for the GGUF NEO-CODE `char-rp-reasoning`
+seat (same Heretic2/NEO-CODE model, NVFP4 + native MTP spec-decode).
+
+This runbook exists because getting here was a multi-hour fire drill. **Every gotcha below
+cost real time — read them before touching this.** The TL;DR: three things all had to be
+right at once — (1) quant as the *multimodal* class, (2) use the *modelopt* format not
+compressed-tensors, (3) work around a vLLM bug that quantizes the MTP draft head.
+
+---
+
+## What / where
+
+- **Model:** NEO-CODE = `DavidAU/Qwen3.6-27B-Heretic2-Uncensored-Finetune-Thinking` (dense
+  27B, `Qwen3_5` GDN-hybrid arch, multimodal `Qwen3_5ForConditionalGeneration`).
+- **Runs only on ana-ml2 GPU0** (NVFP4 is Blackwell-only; irv-ml1 is Ampere).
+- **Artifacts** (ana-ml2 `/tank/aimodels/heretic2-nvfp4-work/`, root-owned):
+  - `heretic2-mtp-bf16/` — BF16 graft (Heretic2 + 15 base-Qwen3.6 MTP tensors). [graft input]
+  - `heretic2-modelopt-nvfp4/` — modelopt NVFP4 quant, single shard, **no mtp**. [quant output]
+  - `heretic2-modelopt-nvfp4-mtp/` — the above + spliced 15 BF16 mtp → **the seat**. [SERVE THIS]
+  - (superseded: `heretic2-nvfp4-cg*` = compressed-tensors path, coherent but MTP-inert;
+    `heretic2-mtp-nvfp4-prod` = original gibberish. Keep for diff, do not serve.)
+- **Scripts** (eshpfi `services/heretic2-nvfp4-quant/`): `graft_mtp.py`, `quant_modelopt.py`,
+  `finalize_modelopt_mtp.py`, `serve_modelopt_mtp.sh`, `sitecustomize-mtp-workaround.py`.
+- **Reference:** the MoE `gen` (`qwen36-35b-a3b-heretic-nvfp4`, `quant_method: modelopt`) and
+  the qwopus-122B `gen` both ran MTP before (qwopus +12% single-stream, archival-memory
+  2026-07-01) — dropped for `gen` because MTP *hurts concurrency*, which is why it belongs on
+  the single-stream RP seats, not `gen`.
+
+## GPU window ritual
+
+Base NVFP4 quant needs ~55 GB free on GPU0. `docker stop llama-charrp
+llama-charrp-reasoning vllm-aeon-gen` (→ ~97 GB free); restore with `docker start …`
+(~90–230 s to healthy). The GGUF NEO-CODE seat is the always-restorable fallback. Heads-up
+wt-dev (their character / thoughtful-character / gen route through these) — unless told
+otherwise. `ssh ana-ml2` = lkraven, in the docker group (no sudo needed for docker).
+
+---
+
+## The pipeline (4 steps)
+
+### 1. GRAFT (CPU, seats up) — `graft_mtp.py`
+Heretic2's finetune dropped the MTP head; graft the 15 BF16 `mtp.*` tensors from base
+`Qwen/Qwen3.6-27B` (shards 13+15). Symlinks Heretic2 shards + one `model-mtp.safetensors`.
+Idempotent, refuses to clobber. Output: `heretic2-mtp-bf16/`.
+
+### 2. QUANT (GPU0 window, ~18 min) — `quant_modelopt.py` via `run_quant_modelopt.sh`
+`nvidia-modelopt` PTQ → **modelopt** NVFP4 format. Three things this script gets right (each a
+gotcha — see below): loads as **`AutoModelForImageTextToText`**, patches the modelopt↔transformers
+**FusedMoE** bug, and forces **single-shard** export. Excludes `lm_head` + `visual` + all
+`linear_attn` (GDN) → BF16, matching AEON. Calib = the 512-row workload-matched chat mix.
+```bash
+docker run -d --name vllm-heretic2-modelopt-quant --gpus '"device=0"' --ipc host \
+  -v /tank/aimodels:/tank/aimodels -v /home/lkraven:/lk \
+  --entrypoint bash vllm/vllm-openai:v0.24.0 -c '
+    set -e
+    pip install -q nvidia-modelopt tiktoken sentencepiece 2>&1 | tail -1
+    python3 /lk/quant_modelopt.py \
+      --model /tank/aimodels/heretic2-nvfp4-work/heretic2-mtp-bf16 \
+      --calib-mode chat --calib /tank/aimodels/heretic2-nvfp4-work/production_calib_512.jsonl \
+      --num-samples 512 --seqlen 8192 \
+      --out /tank/aimodels/heretic2-nvfp4-work/heretic2-modelopt-nvfp4'
+```
+CPU dry-run (no GPU, tiny calib) to validate the pipeline without an outage: same command
+minus `--gpus`, add `-e CUDA_VISIBLE_DEVICES=""`, `--num-samples 2 --seqlen 512`.
+
+### 3. SPLICE (CPU) — `finalize_modelopt_mtp.py`
+Copy `heretic2-modelopt-nvfp4` → `heretic2-modelopt-nvfp4-mtp`, splice the 15 BF16 `mtp.*`
+tensors into the single shard (→ 1967 tensors). (The transformers load never builds an mtp
+module, so mtp must be spliced post-quant — same as AEON/pantheon.)
+
+### 4. SERVE (GPU0) — `serve_modelopt_mtp.sh` + the MTP workaround
+```bash
+docker run -d --name vllm-charrp-modelopt --gpus '"device=0"' --ipc host \
+  -v /tank/aimodels:/tank/aimodels \
+  -v <sitecustomize dir>:/lk_debug -e PYTHONPATH=/lk_debug \    # ← the MTP workaround, see below
+  -p 8018:8000 vllm/vllm-openai:v0.24.0 \
+  /tank/aimodels/heretic2-nvfp4-work/heretic2-modelopt-nvfp4-mtp \
+  --quantization modelopt \
+  --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":3}' \
+  --language-model-only --mamba-cache-dtype float32 \
+  --reasoning-parser qwen3 --tool-call-parser qwen3_coder --enable-auto-tool-choice \
+  --served-model-name char-rp-reasoning --max-model-len 40960 --max-num-seqs 32 \
+  --gpu-memory-utilization 0.5 --trust-remote-code
+```
+`--language-model-only` skips the vision tower (RP seat doesn't need it; saves ~1–2 GB — the
+tower is preserved BF16 in the weights, so multimodal is recoverable by dropping the flag).
+
+---
+
+## The four landmines (each cost hours)
+
+1. **Load as `AutoModelForImageTextToText`, NEVER `AutoModelForCausalLM`.** The latter resolves
+   `qwen3_5` → text-only `Qwen3_5ForCausalLM` → flat `model.layers.*` keys. vLLM only serves
+   `Qwen3_5ForConditionalGeneration`, whose weight mapper needs `model.language_model.*` (+
+   `model.visual.*`). Wrong class → every layer weight silently fails to load → **`!!!!` gibberish**.
+
+2. **Use the MODELOPT format (nvidia-modelopt), not compressed-tensors (llm-compressor).** On
+   compressed-tensors the MTP drafter can't load the BF16 mtp head at all (`not found in
+   params_dict`, **0% acceptance** — loads but never accelerates; this is what pantheon and the
+   "AEON RP seat" actually were). Base NVFP4 *alone* ≈ GGUF at batch-1 (no single-stream win) —
+   **the MTP multiplier is the entire point**, and it needs modelopt.
+
+3. **modelopt 0.45 ↔ transformers 5.12.1 FusedMoE crash.** `mtq.quantize` dies with
+   `TypeError: issubclass() arg 2 must be a class` — modelopt registered transformers' `FusedMoE`
+   (a *function* in 5.x) as an nn class. `quant_modelopt.py` guards it (patches
+   `_DMRegistryCls._get_registered_nn_class` to skip non-class registry entries). Do **not**
+   pin `nvidia-modelopt[hf]==0.43` to dodge it — that drags transformers back to 4.57 which can't
+   load `qwen3_5` at all.
+
+4. **⭐ THE BIG ONE — vLLM 0.24.0 does not propagate modelopt `exclude_modules` to the
+   spec-decode DRAFT model.** The MTP drafter builds its own `qkv_proj`/`gate_up_proj` as
+   *quantized* (NVFP4-packed) while the mtp head is BF16 → `AssertionError: param_data.shape ==
+   loaded_weight.shape` in `qwen3_5_mtp.py:256`. **No checkpoint config fixes this** — instrumenting
+   `is_layer_skipped` proved the drafter's exclude list contains only the *main* model's
+   `linear_attn` entries, never the mtp ones. Also note `is_layer_skipped` does **exact string
+   membership, not glob** — so wildcards like `mtp.layers.0.*` never match anything. **Fix = a
+   runtime patch** (`sitecustomize-mtp-workaround.py`, mounted on `PYTHONPATH`) that force-skips
+   any `mtp.*` prefix in `is_layer_skipped`, keeping the drafter BF16. This is a genuine vLLM bug —
+   **report upstream** (draft-model quant-config should inherit the target's exclude_modules).
+
+## Verify it's actually accelerating
+
+```bash
+# coherence
+curl -s :8018/v1/completions -d '{"model":"char-rp-reasoning","prompt":"The old tavern","max_tokens":40,"temperature":0}'
+# drive tokens, then read acceptance from the seat log:
+docker logs vllm-charrp-modelopt 2>&1 | grep SpecDecoding | tail -2
+#   -> "Mean acceptance length: 2.19 ... Avg Draft acceptance rate: 39.7%"   [GOOD: >0%, ~2 length]
+#   -> "Avg Draft acceptance rate: 0.0%"                                       [BAD: compressed-tensors, or mtp quantized]
+```
+`SpecDecoding` line only appears during active generation. 0% acceptance = you're on
+compressed-tensors, or the workaround didn't load (check for `[ISLS] ... workaround installed`).
+
+## Productionization TODO (not yet done)
+- Bake the sitecustomize workaround into a compose stack (mount + `PYTHONPATH`), served-name
+  `char-rp-reasoning`, alongside/replacing the GGUF seat.
+- brokkr P00 (soong 9-tool k5) — same base model as GGUF NEO-CODE so R36 should carry, but the
+  NVFP4-vs-Q5 quality + tool-path must be confirmed before cutover.
+- Repoint gateway `char-rp-reasoning` alias + heads-up wt-dev.
+- File the vLLM upstream bug (draft-model exclude non-inheritance).
@@ -0,0 +1,57 @@
+# nh3-dev `~/development` — hourly off-box backup
+
+**Why this exists:** nh3-dev is the dev box where agents do uncommitted work under
+`~/development/<project>/`. That tree had **no off-box backup**, so a destructive
+mistake (a stray `rm -rf` on a working dir on 2026-07-12) had no safety net. This
+job closes that gap: an hourly, versioned, off-box snapshot of `~/development`.
+
+## What it does
+
+- **Source:** `nh3-dev:~/development/` (lkraven's working dirs).
+- **Destination (off-box):** `nh3-nas:/volume1/Backup/nh3-dev-development/<YYYY-MM-DD_HHMM>/`
+  — a timestamped dir per snapshot, over rsync-**over-ssh** (syncuser).
+- **Versioning:** `rsync --link-dest` against the previous snapshot → unchanged
+  files hardlink (share inodes, ~0 bytes); only changed files consume new space.
+  `latest` symlink points at the newest snapshot.
+- **Retention:** newest **48** hourly snapshots (older pruned each run).
+- **Excludes:** heavy reconstructable dirs (`node_modules`, `.venv`, `venv`,
+  `__pycache__`, `.pytest_cache`, `.mypy_cache`, `.ruff_cache`, `.cache`, `dist`,
+  `build`, `.next`, `target`, `*.pyc`) and secrets (`.env`, `.env.*`, `*.pem`,
+  `*.key`, `id_*`, `*.sqlite*`). **`.git` is kept** (local commits/stashes = the
+  uncommitted work that matters). Seed snapshot ≈ **11G**; hourly deltas are MB-scale.
+
+## Where it lives (on nh3-dev)
+
+- Script: `~/.config/dev-backup/dev-backup.sh` (mirror committed at
+  `scripts/nh3-dev-development-backup.sh`).
+- systemd `--user` units: `~/.config/systemd/user/dev-backup.{service,timer}`
+  (`OnCalendar=hourly`, `Persistent=true`, linger on → fires without a login).
+- Log: `~/.config/dev-backup/dev-backup.log`.
+
+```bash
+systemctl --user list-timers dev-backup.timer   # next run
+systemctl --user start dev-backup.service        # run now
+tail -f ~/.config/dev-backup/dev-backup.log
+```
+
+## Restore
+
+Snapshots are plain dir trees — no special tool needed:
+
+```bash
+# list snapshots
+ssh nh3-nas 'ls -1 /volume1/Backup/nh3-dev-development/'
+# restore one file/dir from a chosen snapshot
+rsync -a nh3-nas:/volume1/Backup/nh3-dev-development/<STAMP>/<proj>/<path> /tmp/restore/
+# or pull a whole project back
+rsync -a nh3-nas:/volume1/Backup/nh3-dev-development/latest/<proj>/ ~/development/<proj>/
+```
+
+## Notes / future
+
+- **Not encrypted at rest** (plaintext on the trusted internal NAS; secrets are
+  excluded). Upgrade path: migrate to restic once a repo can be created on
+  rest-server-nh3 (currently returns 404 on repo-create — likely append-only) or
+  the Synology sftp subsystem is enabled (currently disabled → restic sftp fails).
+- Off-box = off the nh3-dev VM (lands on nh3-nas, same NH3 site). Cross-site
+  mirroring of this repo is a separate future layer.
@@ -0,0 +1,91 @@
+# soong-lab push-to-deploy (gitea webhook → corviduo-dev, test-gated)
+
+Green-gated CI/CD for the soong-lab studio: **push to `main` → run the test
+suite → redeploy the studio ONLY if tests pass** (running studio is never
+touched on a red run). Built 2026-07-13 (Vuong-directed). Adapts the
+[ytvc-autodeploy](./ytvc-autodeploy.md) webhook pattern.
+
+## Flow
+
+```
+push→main → gitea webhook (POST, HMAC) → soong-webhook listener :9010 on corviduo-dev
+  → ~/soong-lab-deploy.sh:
+       git clone (read-only deploy key, internal SSH :222)
+       uv sync ; uv run pytest        ── RED → abort, studio UNTOUCHED, status=red
+       rsync backend/ → studio dir + web/ → SOONG_LAB_WEB_DIR ; uv sync --no-dev ; restart
+       → status=green, studio healthy
+```
+
+## Components (all on corviduo-dev, user `infra-ops`)
+
+- `~/soong-lab-deploy.sh` — clone → test → deploy-on-green. Logs to
+  `~/soong-lab-deploy.log`; writes `~/.config/soong/last-deploy.json`
+  (`{result: green|red, stage, sha, at}`).
+- `~/soong-webhook.py` — HTTP listener on `:9010`. HMAC-SHA256 (`X-Gitea-Signature`)
+  vs `~/.config/soong/webhook-secret` (mode 600); fires the deploy only on
+  `ref == refs/heads/main`. `GET /` returns `ok | last: <status>`.
+- `soong-webhook.service` (system unit, enabled) — runs the listener.
+- Read-only deploy key `~/.ssh/soong-deploy_ed25519` → gitea repo key id 5 on
+  `vh/soong-lab` (read_only). Clone via `ssh://git@10.250.50.70:222/vh/soong-lab.git`.
+- Studio unit `soong-lab-studio.service` (WD `/home/infra-ops/soong-lab/backend`);
+  restart needs infra-ops NOPASSWD sudo (present).
+- Gitea webhook: repo `vh/soong-lab` hook id 3 → `http://10.250.50.152:9010/`,
+  JSON, Push events, the shared secret.
+
+## Verify / operate
+
+```bash
+ssh corviduo-dev 'systemctl is-active soong-webhook.service; curl -s localhost:9010/'
+ssh corviduo-dev 'tail -30 ~/soong-lab-deploy.log'          # deploy history
+# manual deploy (same as the webhook does):
+ssh corviduo-dev 'bash ~/soong-lab-deploy.sh'
+```
+
+## Notes / gotchas
+
+- **Frontend (`web/`) sync IS part of the deploy**: the studio serves `web/` from
+  `SOONG_LAB_WEB_DIR` (`/home/infra-ops/soong-lab/web`), *separate* from the backend
+  `WorkingDirectory`. The deploy rsyncs BOTH `backend/`→studio and `web/`→`SOONG_LAB_WEB_DIR`.
+  (Added 2026-07-13 after soong-dev caught the served frontend silently rotting — the backend
+  was updating while `web/` stayed pinned to the initial manual copy; a bounce alone re-serves
+  the same stale file.)
+- **Green-gated by construction**: `pytest || fail` runs BEFORE any studio touch,
+  so a red suite aborts with the studio still on the old version. Validated
+  2026-07-13 (a mid-deploy rsync failure left the studio untouched/active).
+- **rsync is required** on corviduo-dev (`apt install rsync` — installed 2026-07-13;
+  it wasn't present initially).
+- **bifrost dep** resolves from the internal Gitea PyPI via `~/.netrc` (already
+  present on corviduo-dev); no extra auth in the deploy script.
+- **⚠️ Auto-deploy silently never worked until 2026-07-14 — TWO compounding blockers.**
+  The LISTENER binds `0.0.0.0:9010` and works, but nothing gitea sent ever reached
+  it, so every push was a no-op (v0.3.6 was manual; v0.3.7–v0.3.15 never
+  auto-deployed until fixed). Two separate, both-real blockers:
+  1. **corviduo-dev ufw** — `default-deny`, only 22 + 8080 allowed, so a *direct*
+     TCP to :9010 from ana-docker DROP-timed-out. Fix: `ufw allow from 10.0.0.0/8`
+     (operator-directed — "that footgun happens a lot", accept the fleet).
+  2. **★ gitea `webhook.ALLOWED_HOST_LIST` (the DECISIVE one)** — was
+     `external, 10.100.0.0/16` (NH3 only); corviduo-dev is `10.250.50.152`
+     (Anaheim), so gitea **refused to deliver**: `webhook can only call allowed HTTP
+     servers ... deny '10.250.50.152'` — it never even opens the TCP connection, so
+     the ufw fix alone did nothing. Fix: `ALLOWED_HOST_LIST = external, 10.0.0.0/8`
+     in gitea `app.ini` (`/data/gitea/conf/app.ini`, `[webhook]`) + `docker restart
+     gitea` (~8s blip). The HMAC secret was already correct (once delivery arrives,
+     `hmac_ok=True`).
+  **RED HERRINGS that cost two diagnosis rounds:** (a) "test-delivery 204" is gitea
+  *queuing*, NOT delivering — never proves the round-trip; (b) a proxy test signing
+  with the *listener's own* secret (bypassing gitea) proves the listener but NOT
+  gitea's real delivery. **Diagnose from BOTH ends:** the SENDER (`docker logs gitea
+  --since 5m | grep webhook` → the `deny '<ip>'` line) AND an instrumented RECEIVER
+  — the listener now ships with delivery logging (`journalctl -u soong-webhook.service
+  | grep '\[webhook\]'` shows source-IP / hmac_ok / ref / action; the old
+  `log_message=pass` silence hid all of it). **Proof of fix:** a real gitea delivery
+  logs `POST from 10.250.50.70 ... hmac_ok=True`, `ref='refs/heads/main'`,
+  `-> 202 deploying` → green deploy of the latest main SHA.
+- **Red-run push-notify** via an **althing relay on nh3-dev** (`soong-ci-relay.timer`,
+  2-min poll of corviduo's `last-deploy.json` → pings **soong-dev** via althing on a
+  NEW red run; green runs stay silent = fire-and-forget). corviduo itself has no
+  althing, so the relay lives on nh3-dev (which does), needing no gitea write token
+  on the Worldtree-team VM. Files: `services/soong-lab-ci/soong-ci-relay.{sh,service,timer}`;
+  state `~/.local/state/soong-ci-relay/last-at.txt`. (A gitea commit-status was the
+  alternative but needs a write token gitea won't mint without basic-auth.)
+- Test suite: `uv run pytest` in `backend/` (242 tests as of v0.3.6).
@@ -0,0 +1,52 @@
+# Storetank image-models archive — curation / migration / decommission record
+
+**Host:** irv-ml1 · **Former path:** `/storetank/image-models/comfy/models`
+(was the native `/opt/ComfyUI/models` symlink target).
+**Status: DECOMMISSIONED 2026-06-13** — emptied of all models (919 G → 0). The
+active model tree is arbo's `/storetank/arbo/models` — see
+[`arbo-comfyui-model-catalog.md`](arbo-comfyui-model-catalog.md).
+
+Historical record of how the 919 G CivitAI-managed pile was resolved on 2026-06-13:
+**~739 G killed** (superseded / niche), **177 G migrated** into the arbo set, the
+remainder dupes arbo already had.
+
+## 1. Killed — superseded by arbo's current-gen stack (~739 G)
+
+Two principles: generation-locked LoRAs have no value without their (also-superseded)
+base models, and arbo already carries its own copies of the shared encoders/VAEs.
+
+| Killed | Size | Why |
+|---|---|---|
+| **Hunyuan video** (diffusion_models + unet + vae + loras) | 74 G | older video arch; not in arbo |
+| **WAN2.1 bases + loras** | 118 G | superseded by arbo's WAN2.2; loras gen-locked |
+| **WAN2.1 encoders / VAE** (umt5, xlm-roberta, clip_vision_h, wan VAE) | 44 G | dupes of arbo's own copies |
+| **FLUX.1 — everything** (dev/schnell/fill + ~20 community merges + loras + flux controlnets / redux / pulid / clip-vision / FLUX.D encoder / Florence-2-Flux) | 445 G | superseded by arbo's FLUX.2-klein; loras gen-locked |
+| **orphaned umt5** (root `umt5_xxl_fp8`) | 6.7 G | last WAN remnant |
+| **orphaned llava_llama3** (fp16 + fp8) | 23.5 G | HunyuanVideo's text encoder — dead after the Hunyuan kill |
+| **Chroma v10/v11 + SD3.5-large** | 28 G | niche generators, not in arbo |
+| **TOTAL** | **~739 G** | |
+
+## 2. Migrated into arbo (177 G)
+
+Everything not superseded moved into `/storetank/arbo/models` (same-filesystem atomic
+move, skip-existing so arbo's production copies were never clobbered; 652 files moved,
+37 skipped as dupes):
+
+- **Gen-agnostic utilities:** Aura-SR v1/v2 + the full `upscale_models/` family
+  (HAT/DAT/RealESRGAN/UltraSharp/Remacri/NMKD/Omni-SR) · Florence-2 + CogFlorence
+  captioners (`LLM/` + `florence2/`) · controlnet_union_promax · grounding-dino ·
+  SAM + SAM-HQ · yolo (`ultralytics/`) · depthanything-v2 · vitmatte · nsfw_detector ·
+  insightface (inswapper + antelopev2) · facerestore · facexlib · ip-adapter-plus_sdxl ·
+  CLIP-vision (sigclip, EVA02-CLIP-L)
+- **SDXL / Pony stack:** ponyRealism, cyberrealisticPony_v8, lustify, hassaku,
+  waiNSFWIllustrious, juggernaut / dreamshaper Lightning, SUPIR + loras
+  (dmd2_sdxl_4step, ACE++, Illustrious/PonyXL character-design)
+
+comfy-dev owns the follow-on: per-model catalog entries + graphs + heroes that turn
+these files into usable arbo workflows.
+
+## 3. Final state
+
+`/storetank/image-models/comfy/models` (= native `/opt/ComfyUI/models`) holds only
+empty category dirs + `put_*_here` placeholders — **638 K, no models**. Decommissioned;
+arbo is the single live tree.
@@ -0,0 +1,6 @@
+- `[2026-07-04]` **LiteLLM (this gateway version) mutates the SHARED deployment config in-place on
+  per-request sampler-param merge** → my deliberately-invalid `top_k=-5` forwarding-probe bled into a
+  param-less character-rp request (vLLM 400, ONE-OFF, self-cleared by a later valid probe). NOT
+  caching (none configured), NOT a config change. **Never fire invalid/distinctive sampler values at
+  a SHARED gateway alias with live consumers** — use a throwaway alias, or a `docker restart litellm`
+  flushes residual carryover. `feedback_litellm_shared_param_mutation`.
@@ -0,0 +1,5 @@
+- `[2026-07-07]` **Engine invocation footguns cost several wasted serve-bounces this session** — `docker run
+  --rm` ate crash logs; duplicated `serve` (vLLM image entrypoint is already `["vllm","serve"]`);
+  `--max-lora-rank 48` invalid (choices 1/8/16/32/64… → use 64); parens in `echo` inside `ssh host -c "…"`
+  break the remote shell. LESSON: verify engine launch flags (`--help`, GPU-free) + never `--rm` a container
+  whose crash logs you need, BEFORE bouncing a production serve.
@@ -0,0 +1,4 @@
+- `[2026-07-07]` **SGLang generic image can't LOAD our NVFP4 AEON** — ModelOptModelLoader weight-shape/
+  packing mismatch ([1024,5120] vs [1024,2560], 2-fp4/byte). NVFP4-on-SGLang needs the dedicated
+  `qwen36-27b-nvfp4` dev image or a requant to SGLang's format. bf16 loads fine (arch supported; crash was
+  quant-loader-specific).
@@ -0,0 +1,4 @@
+- `[2026-07-07]` **SGLang `--lora-target-modules` CLI enum REJECTS the GDN names its own resolver asks for**
+  (invalid choice: 'in_proj_qkv'); `'all'` resolves to the FUSED set (qkv_proj/in_proj_qkvz). SGLang wants
+  its OWN packed layout (base r16 + `get_stacked_multiply=3`, NOT a pre-fused rank-48 qkv → the [48]-vs-[144]
+  shape assert). A THIRD adapter format; version-exact source needed (`:latest`=0.5.13, NOT `main`).
@@ -0,0 +1,6 @@
+- `[2026-07-07]` **vLLM 0.24.0 qwen3_5 LoRA application = silent no-op (#47639).** Adapter loads HTTP 200
+  but zero deltas at inference. NOT quant (NVFP4 AND FP8 both inert). NOT adapter format (separate `zc`
+  adapter — correct per vLLM's `check_unexpected_modules` allowlist — loads clean but inert; the fused-key
+  rekey is rejected). The #47640 None-group guard-patch overlay did NOT fix it (failure is UPSTREAM of
+  `expand_packed_lora` — the separate→fused mapping never happens). Fix PR #47640 is OPEN (unmerged) so no
+  version-bump helps. Merge bakes deltas in (bypasses this) but is static.
@@ -0,0 +1,5 @@
+- `[2026-07-08]` **Angel (allura-org/MS3.2-24b-Angel) self-quanted to NVFP4 = GARBAGE.** llm-compressor W4A4 NVFP4
+  (compressed-tensors, MLP-quantized, attn/vision bf16) of the Mistral3 dense 24B produces gibberish EVEN AT GREEDY
+  (temp 0) → the quant itself is broken, not the tokenizer or sampler. Same recipe worked on the qwen models.
+  Mistral3 + W4A4 NVFP4 via llm-compressor is bad. → for the RP seat, going **GGUF (llama.cpp)** to sidestep the
+  whole NVFP4-quant surface.
@@ -0,0 +1,7 @@
+- `[2026-07-08]` **Mistral3 + vLLM tokenizer/vision traps (serve `MS3.2-24b`, vLLM 0.24).** (a) HF `tokenizer.json`
+  for Mistral = **GARBAGE output** — the card's "use the official Mistral tokenizer" warning is REAL; must use the
+  `tekken.json`/mistral tokenizer. (b) BUT `--tokenizer-mode mistral` + vision **CRASHES** (`Failed to apply
+  PixtralProcessor on {'text': '[IMG]'}`; and with tekken.json present in auto mode, `CachedMistralCommonBackend has
+  no attribute is_fast`). So it's **mistral-tokenizer OR vision, not both** on this vLLM. Text-only + mistral
+  tokenizer serves clean (`--limit-mm-per-prompt '{"image": 0}'`). **GGUF/llama.cpp avoids all of this** (native
+  mistral tokenizer + vision).
@@ -0,0 +1,6 @@
+- `[2026-07-08]` **Pantheon-27B MTP on vLLM compressed-tensors = 0% acceptance.** MTP is a separate **bf16** head
+  (`mtp.*`, in `model-auxiliary.safetensors`, 15 tensors); AEON preserved it by INJECTING the bf16 head into the
+  quant output (NOT re-quantizing — confirmed AEON's nvfp4 mtp is bf16). Built pantheon-27b-mtp = compressed-tensors
+  main + injected bf16 mtp + `text_config.mtp_num_hidden_layers=1` → vLLM detected the MTP but SKIPPED the bf16
+  self_attn weights → 0/192 draft tokens accepted. **The bf16 MTP head only loads on the MODELOPT main-model format
+  (like AEON), not compressed-tensors.** (Moot — operator dropped MTP for gen; not needed for the non-reasoning RP.)
@@ -0,0 +1,6 @@
+- `[2026-07-08]` **Pantheon-Reasoning-27B refuses dark fiction DESPITE an abliterated base.** The base
+  (`llmfan46 heretic`) writes freely (thinking-off), but Gryphe distilled the reasoning traces from **DeepSeek 3.2**
+  (safety-aligned) onto every turn (`preserve_thinking:true`) → the model reasons ITSELF into refusals in the
+  `<think>` phase (collapses to empty output). Fix: thinking-off OR an uncensor system prompt (both verified).
+  **Lesson: a reasoning finetune of an abliterated base can re-censor via its reasoning-trace TEACHER; the raw
+  abliterated base is cleaner** — this is WHY the pivot went to the llmfan46 heretic base for gen.
@@ -0,0 +1 @@
+- `[2026-07-13]` Relaying a peer's diagnosis as fact without confirming it against raw data. worldtree-dev diagnosed the WT #355 residual as "our llama.cpp seat wedging," which I echoed in a wrap-up; the operator challenged it and the seat logs DISPROVED it (seat completes ≤72s, idle at the wedge onset — the hang is the LiteLLM gateway). Lesson: CONFIRM peer diagnoses (esp. cross-domain ones) before acting/relaying — same discipline that caught the earlier char-rp-reasoning red-herring via a live `registry.resolve` reproduction.
@@ -0,0 +1 @@
+- `[2026-07-14]` **AEON's "working NVFP4+MTP RP seat" was pantheon on compressed-tensors (0% MTP accept), not a modelopt MTP proof.** `vllm-aeon-rp`'s .env → `AEON_RP_MODEL=pantheon-27b-mtp-nvfp4`, `AEON_RP_QUANT=compressed-tensors` — it LOADED (mtp silently skipped, `exited 0`) but never accelerated. Same vLLM image (`:latest` = `sha256:4091d55` = 0.24.0) as the failed Heretic2 test, so the "AEON ran on an older vLLM" theory was wrong. Don't treat a seat that "ran" as MTP-validated without checking its `SpecDecoding` acceptance.
@@ -0,0 +1 @@
+- `[2026-07-14]` **gitea "test-delivery 204" is NOT proof a webhook works** (204 = gitea *queuing*, not the listener receiving) — and a proxy test signing with the listener's OWN secret proves the listener, not gitea's real delivery. Both red herrings cost a round of the soong-lab webhook diagnosis. Diagnose from BOTH ends: sender (`docker logs gitea | grep webhook` → the `deny '<ip>'` line) AND an instrumented receiver.
@@ -0,0 +1 @@
+- `[2026-07-14]` **MTP graft via top-level `mtp.*` tensor names does NOT survive `AutoModelForCausalLM.from_pretrained`** — the `Qwen3_5ForCausalLM` class doesn't expose an mtp module, so the mtp keys are DROPPED at load (quant output = 0 mtp). Fix = SPLICE the BF16 mtp tensors into the quant output post-hoc (how pantheon was built); don't rely on the graft surviving the model round-trip.
@@ -0,0 +1 @@
+- `[2026-07-14]` **MTP-on-modelopt: NO checkpoint config skips the spec-decode drafter's quant (vLLM 0.24 bug) — 4 config attempts failed before the runtime workaround.** All crashed the same way (`qwen3_5_mtp.py:256` `param_data.shape == loaded_weight.shape` AssertionError — bf16 mtp head loaded into a quantized drafter param): (1) mtp excludes in `config.json` (WRONG file — vLLM modelopt reads `hf_quant_config.json`); (2) specific-unfused mtp names in hf_quant_config; (3) wildcards `mtp*`/`mtp.layers.0*` (`is_layer_skipped` is EXACT-membership, NOT glob — wildcards match nothing); (4) exact fused+unfused names in both `mtp.`/`model.` prefixes. Instrumenting `is_layer_skipped` proved the drafter's exclude list holds ONLY the main model's `linear_attn` entries — the mtp excludes never reach the draft-model quant config. ONLY fix = a mounted `sitecustomize` force-skipping `mtp.*`. LESSON: don't chase checkpoint-config fixes for the mtp-drafter crash; go straight to the runtime patch. Also `nvidia-modelopt[hf]==0.43` (AEON's producer version) is a trap — it pins transformers back to 4.57 which can't load `qwen3_5` at all; use 0.45 + the FusedMoE guard in `quant_modelopt.py`.
@@ -0,0 +1 @@
+- `[2026-07-14]` **NVFP4 (llm-compressor / compressed-tensors) gives NO batch-1 speedup over GGUF for the Qwen3.5 GDN-hybrid, and its MTP is 0%-accept.** Measured base NVFP4 no-MTP ≈53 tok/s decode vs the GGUF NEO-CODE seat ~59.5 (llama.cpp wins single-stream; NVFP4's edge is concurrency, and this hybrid is bandwidth-bound at batch-1 with the BF16 linear_attn/GDN layers dominating). MTP spec-decode = 0% acceptance (vLLM's `Qwen3_5MTP` drafter won't load the bf16 mtp weights off a compressed-tensors main model → `Parameter … not found in params_dict`, `Avg Draft acceptance rate: 0.0%`). Pantheon is identical — its "working NVFP4+MTP" was working *structure*, never real acceleration. Working native MTP needs the **modelopt** main-model format (AEON, ~3.3/3 accept). LESSON: don't expect a faster single-stream seat from an llm-compressor NVFP4 quant of this arch; the MTP multiplier is the whole point and it requires modelopt.
@@ -0,0 +1 @@
+- `[2026-07-14]` **NVFP4 spike: built the full MTP serve scaffolding BEFORE validating a plain NVFP4 serve was coherent.** Chased 6 sequential serve-config fixes (entrypoint doubled `serve`, arch `ForCausalLM`→`ConditionalGeneration`, `--language-model-only`, mamba-cache/`max-num-seqs`) across a **2.5hr GPU window** (quoted 30-60 min) — only to find the served model gibbers (`!!!!`). LESSON: smoke a PLAIN `/v1/completions` coherence check on the SIMPLEST config (native arch, no MTP, no splice) FIRST — validate the tracer bullet before building spec-decode scaffolding. Also cost an unnecessary re-quant (the `re:mtp.*` ignore fix that turned out moot). Diagnostic ladder in Current state.
@@ -0,0 +1 @@
+- `[2026-07-15]` **arbo fully switched off image-judge (qwen-image-bench) -> gen; image-bench pending eviction post-bake.** Operator-directed full switch (comfy-dev executed, live in prod). Established: gen (`qwen3.6-35b-a3b-heretic`) is vision-enabled and was image-bench's predecessor as arbo's hero-judge; image-judge actually serves 4 roles (vision quality-scoring + identity-scoring + bbox grounding + an uncensored text tier), not just grounding. comfy-dev spot-check: gen faster on every task, grounding within ~3px, uncensoring preserved, and it FIXED a bug (image-judge's reasoning preamble broke json_object + stalled the router). Sequencing = short prod bake then evict (~30 GB GPU1 reclaim); revert = flip `ARBO_VISION_MODEL`. Full record: auto-memory `project_arbo_gen_switch_imagebench_evict`.
@@ -0,0 +1 @@
+- `[2026-07-15]` **Claude Code statusline `.cost.total_cost_usd` is per-SESSION** (Claude Code's own cache/model-aware session accounting), not a lifetime aggregate — the large value just reflects a long, multiple-times-summarized session. And the old statusline hardcoded Sonnet pricing ($3/$15) on an Opus session -> ~5x cost understatement.
@@ -0,0 +1 @@
+- `[2026-07-15]` **`docker.service After=remote-fs.target` does NOT wait for `nofail` NFS mounts** — `nofail` drops a mount out of remote-fs.target's blocking set, so the drop-in ordering is silently defeated (paperless still Exited(255) on reboot). Real fix = DIRECT mount->docker ordering via the fstab `x-systemd.before=docker.service` option (verify `systemctl show docker -p After` lists the mnt-*.mount units). esh-docker-vm.
@@ -0,0 +1 @@
+- `[2026-07-15]` **esh-docker-vm NFS fstab fix = `x-systemd.before=docker.service`** (the prior `After=remote-fs.target` drop-in was silently defeated by `nofail`). Reached only after a REBOOT (D-state phantom containers uptime-kuma + paperless-web that no `docker`/`ctr`/daemon-restart could clear). Committed `21d9a07` + playbook updated. See Tried and abandoned.
@@ -0,0 +1 @@
+- `[2026-07-15]` **The esh-docker-vm D-state/phantom-container wedge is only cleared by a host REBOOT** — reconfirmed: `docker stop/rm -f`, `ctr -n moby task delete`, AND `systemctl restart docker` all fail to clear it; `docker exec` into a wedged container ALSO fails (`setns ... exit status 1`), so the in-place restart escape hatch is out. Worse, a daemon restart can HALF-KILL other healthy containers (knocked paperless's granian down + left it wedged). Process dead but dockerd won't reap -> phantom. NFS mounts are `_netdev,nofail` so the reboot is boot-safe.
@@ -0,0 +1 @@
+- `[2026-07-15]` **vLLM `max-model-len` does NOT free GPU VRAM** — the KV cache POOL is sized by `gpu-memory-utilization`, not max-model-len. Lowering max-model-len only caps per-request context + drops max concurrency; the pool still fills the util budget. To actually free VRAM, lower `gpu-memory-utilization`. (Bit the char-rp-reasoning "drop KV to 150K" ask: the 150K applied but freed 0 VRAM until util dropped 0.39->0.38.)
@@ -0,0 +1,15 @@
+- `[2026-07-17]` **Zonos2 `:1920` engine → self-contained container (stays on 3090); prosody-priming is a SERVING-LAYER change (engine stays stock).**
+
+**Context.** The production Zonos TTS engine (irv-ml1 `:1920`, feeds asset-engine + gateway-chat via `zonos-gateway` :8890) was a bare native process — its real launch config existed ONLY in the running process argv (the committed `~/tts-audition/harness/zonos_server.sh` was STALE: said A6000/:1919/no perf flags; live is 3090/:1920 with `--cuda-graph-max-bs 1 --num-pages 16384 --max-running-requests 2 --memory-ratio 0.3`). Captured to eshpfi `stacks/zonos-engine/` (README + corrected `zonos2-server.sh` + `.env.example`), commit **14a0004** (UNPUSHED as of the snapshot).
+
+**Decision 1 — containerize as a SELF-CONTAINED image** (not systemd — operator rejected; not a thin bind-mount wrapper — I walked that back: bind-mounting the host's CUDA-compiled `.venv` couples to the host's exact CUDA/glibc and is fragile + not reproducible). Shape: `FROM` a CUDA 12.8 base → `uv sync` against the repo's committed `uv.lock` (deterministic env) → mount the ~15 GB HF weights (`~/.cache/huggingface/hub/models--Zyphra--ZONOS2`, do NOT bake) → pin the **3090** (`NVIDIA_VISIBLE_DEVICES=0`) → `restart: unless-stopped` → CMD = the captured invocation. **Engine stays STOCK** Zyphra/Zonos2 @ commit `194c0a3` (no fork — the `zonos2` package ships its own server). **Build risk:** heavy compiled-CUDA deps (flashinfer / sgl_kernel / cutlass-dsl / apache-tvm-ffi / pynini) on torch 2.9.1+cu128 — mostly prebuilt wheels + the `uv.lock` make it tractable, expect a couple build iterations. **Cutover (in place on the 3090):** stop the native process (frees ~17 GB) → `docker compose up -d` (re-allocates ~17 GB, same footprint) → repoint `zonos-gateway`'s `ZONOS_URL` at the container (or keep the `:1920` host-port publish). One brief prod-TTS blip.
+
+**GPU = 3090 (operator 2026-07-17).** Keep it OFF the A6000 — the A6000 already OOMs under ComfyUI load (idle ~19 GB but spikes far higher during gen), so it can't host Zonos too. The 3090 already runs Zonos, so the containerize-in-place cutover changes nothing about placement.
+
+**Decision 2 — the prosody-priming hypothesis (operator's test; the reason for building fresh).** PRIME the autoregressive engine with an emotional sentence, then TRUNCATE it from delivery: prepend a primer → **generate "primer + real text" as ONE continuous utterance** (the AR model carries prosody forward across the boundary) → ASR-timestamp the primer's end (**parakeet**, already up on irv-ml1 `:8765`, word timestamps) → **clip the primer in the inter-sentence silence gap** (+ ~15 ms fade-in, no click) → deliver only the real text, now wearing the primed prosody. Examples: primer "I'm so EXCITED about this." → "This will be a lot of fun!" spoken excited; primer "I'm whispering this to you right now." → "I'm so glad to see you baby." whispered. **This is PURE serving-layer orchestration — the engine is untouched; it lives in the gateway adapter `stacks/zonos/adapter/server.py`.** Only fork the engine if the black-box approach fails.
+
+**THE CRUX the test resolves:** does AR prosody actually **carry across the sentence boundary**, or does Zonos reset at the period? → the harness A/Bs the **JOIN punctuation**: period (operator's examples) vs comma vs ellipsis vs none ("…excited about this, this will be…"). Everything else is plumbing.
+
+**Plan / design recs.** (a) Build the stock engine image (parallel track). (b) Stand up a priming TEST HARNESS against the NATIVE engine (fast iteration, seconds) + parakeet ASR: prime→generate→timestamp→gap-clip→out; compare primed-clipped vs plain on the two cases (subjective + a cheap objective proxy: pitch/energy variance for "excited", spectral-tilt/low-energy for "whisper"). Iterate on the join, then bake the winner into the gateway adapter. **Primer source:** caller-supplied for the harness (test arbitrary primers) → a curated emotion→primer library (`excited`/`whisper`/…) + optional caller override for production. **ASR:** parakeet primary; WhisperX forced-align fallback if parakeet word timestamps are coarse.
+
+See eshpfi `stacks/zonos-engine/README.md` + `stacks/zonos/` (the gateway adapter).
@@ -0,0 +1,57 @@
+- `[2026-07-18]` **Fleet Gitea-Actions build recipe + the `vh`-is-a-user package-write constraint** (learned the hard way across 3 failed soong-lab validation builds; reusable for ANY fleet CI image build or package publish).
+
+**The runner.** One `act_runner` (`gitea/act_runner`) on ana-docker, labels
+`pfi-fleet` / `ana-docker` → both map to job image **`node:20-bookworm-slim`**,
+which has **NO docker and NO git**. Config `/opt/docker/conf/gitea-runner/data/config.yaml`:
+`valid_volumes: []` (no socket propagated to job containers). So:
+- `actions/checkout@v4` fails (needs git); `docker/*` marketplace actions fail
+  (need docker) — a workflow built on those dies at the first step (~15s).
+
+**The working recipe (mirror Worldtree `deploy.yml`).** Run the job in a
+docker-capable image + drive docker with RAW commands, not the JS actions:
+```yaml
+runs-on: pfi-fleet
+container:
+  image: docker:24.0.7-cli        # has docker+buildx; add git+node
+steps:
+  - run: apk add --no-cache git nodejs   # so actions/checkout@v4 works
+  - uses: actions/checkout@v4
+  - name: login   # RAW, not docker/login-action
+    run: echo "$REGISTRY_TOKEN" | docker login gitea.phasefinal.com -u "$REGISTRY_USER" --password-stdin
+  - name: buildx builder
+    run: docker buildx create --name X --driver docker-container --use; docker buildx inspect --bootstrap
+  - name: build+push   # RAW, not docker/build-push-action
+    run: docker buildx build --secret id=<name>,env=<TOKEN> -t <img>:latest --push .
+```
+The runner mounts the host docker socket into ITSELF; the docker:cli job reaches
+the daemon through that. The `docker/*` JS actions are unreliable on act_runner —
+raw commands are the fleet convention.
+
+**`vh` is a USER account, not an org.** Consequences that bit repeatedly:
+1. `GET /api/v1/orgs/vh` → 404 "user redirect"; there are **no org teams** to add
+   a service account to.
+2. **User-owned packages are OWNER-WRITE-ONLY.** claude-bot (even repo
+   admin-*collaborator* on `vh/soong-lab`, even with `write:package` scope + full
+   basic-auth) gets **`401 unauthorized`** on `docker push` to `vh/soong-lab`, and
+   `npm publish` to `vh/npm/` would 401 too. Only `vh` itself can write vh packages.
+   → CI must authenticate AS `vh` for the push (a vh-owned `write:package` PAT as
+   `REGISTRY_TOKEN` + `REGISTRY_USER=vh`), exactly how WT pushes `vh/worldtree`.
+   claude-bot CAN still: clone/read repos, READ packages (pulled the image fine),
+   dispatch workflows, mint demo Worldtree keys.
+3. **Repo Actions secrets are OWNER-ONLY too** — `PUT .../actions/secrets/X` as
+   claude-bot (repo admin-collab) → 403 "user should be the owner of the repo".
+   Only `vh` can set a repo's secrets.
+
+**Other gotchas:**
+- Gitea **reserves the `GITEA_` secret-name prefix** — a secret named
+  `GITEA_PYPI_TOKEN` is illegal; use e.g. `PYPI_TOKEN`.
+- Gitea **package auth is token-based / username-lenient** — `docker login` /
+  PyPI basic-auth authenticate via the token; the username is nominal (tested
+  `-u gitea` and `-u claude-bot` both 200 against the vh PyPI). So a Dockerfile
+  hardcoding `UV_INDEX_GITEA_USERNAME=gitea` is fine with any valid token.
+- Homepage (esh-docker-vm) docker-label auto-discovery only covers the 5 endpoints
+  in its `docker.yaml` (esh-vm-docker, ana-docker, ana-ml2, nh3-docker, irv-ml1);
+  **corviduo-dev is NOT watched** → services there need a manual `services.yaml`
+  entry, not labels.
+
+Applied in the soong-lab CI: [[2026-07-18-soong-lab-containerize-cutover]].
@@ -0,0 +1,83 @@
+- `[2026-07-18]` **soong-lab auto-redeploy — DONE + VALIDATED** (was approved/queued; executed same day on fresh context — see AS-BUILT at the bottom).
+
+Vuong approved wiring auto-redeploy for soong-lab (relayed via soong-dev, thread
+`01KXT3A6C3908TA4V9THV3AMH7`): new images should go live on corviduo-dev without
+the manual `docker compose pull && up -d`. Host-side implementation is infra-ops's
+lane; mechanism is infra-ops's call per fleet conventions. Operator deferred
+execution — "we'll do soong on fresh context."
+
+**Chosen mechanism (recommended, agrees with soong-dev): Worldtree-style
+CI-deploy step** — NOT watchtower polling.
+- Add a deploy job/step to soong-lab's `.gitea/workflows/build-and-push.yml` that,
+  after the build+push job succeeds, **SSHes from the pfi-fleet runner to
+  corviduo-dev** and runs `cd /home/infra-ops/soong-lab-deploy && docker compose
+  pull && docker compose up -d`, then a **health-gate** (`curl -fsS
+  http://localhost:8443/api/version`).
+- This is exactly how WT deploys the demo instance to the SAME host: see
+  `~/development/Worldtree/.gitea/workflows/deploy.yml` — the "Deploy to demo VM +
+  health-gate" step uses `secrets.DEMO_VM_SSH_KEY` / `DEMO_VM_HOST` / `DEMO_VM_USER`.
+  Explicit-over-implicit (visible in the run log, fires exactly on build success),
+  one less always-on service than watchtower.
+
+**Constraints (from soong-dev):** deploy on CI success only; keep the trigger
+gated to `v*` tags + `workflow_dispatch` (as today); preserve the one-command
+rollback posture (`docker compose down` / pin a previous tag).
+
+**BLOCKER — needs from vh (owner-only):** a **runner→corviduo-dev deploy SSH key**
+as a repo secret (+ host/user), same class as WT's `DEMO_VM_SSH_KEY`. Likely
+**reuse WT's existing demo-deploy key** (WT's runner already SSHes to 10.250.50.152
+as its deploy user). Repo secrets are vh-owner-only (see
+[[2026-07-18-fleet-gitea-runner-build-recipe]]).
+
+**Next-session steps:** (1) confirm/obtain the deploy SSH-key secret from vh (reuse
+WT's or mint fresh); (2) add the deploy job to build-and-push.yml (infra-ops has
+push on vh/soong-lab); (3) dispatch a build to verify it deploys + health-gates;
+(4) ping soong-dev so they sync DEPLOY.md's "open follow-up" note to the as-built
+mechanism. Auto-pull (watchtower) explicitly NOT chosen. See
+[[2026-07-18-soong-lab-containerize-cutover]].
+
+## AS-BUILT (2026-07-18, same-day execution)
+
+**Mechanism landed** exactly as planned: `build-and-push.yml` gained a `Deploy to
+corviduo-dev + health-gate` step (after build+push) that SSHes the host as `deploy`
+and runs `docker compose pull && up -d` from `/opt/soong-lab`, then polls
+`http://localhost:8443/api/version` for 120s and fails the job loud if unhealthy. No
+compose is shipped from CI (the in-repo `docker-compose.yml` is a BUILD compose; the
+host pull-compose is infra-ops-managed). Kept the `v*`-tag/`workflow_dispatch` trigger.
+Skipped WT's disk-watermark gate + health-gated-`:latest`-advance (low cadence, easy
+rollback).
+
+**Deploy identity = reuse WT's `deploy` account** (operator accepted the rec):
+- `deploy` (uid 1001, docker-group → no sudo) already owns `/opt/worldtree`; relocated
+  soong-lab's deploy dir `/home/infra-ops/soong-lab-deploy` → **`/opt/soong-lab`**
+  (deploy-owned), copied compose + `.env`. Named volumes (`soong-lab_soong-library`,
+  `soong-lab_soong-portraits`) are project-scoped by compose `name: soong-lab` → followed
+  the move untouched (dry-run `up -d` ADOPTED the running container, no recreate). Old dir
+  **retired → `.retired-20260718`** (recoverable). Also lingering: `soong-lab-deploy.sh` /
+  `.log` (dead pre-container webhook artifacts) — harmless, left in place.
+- **Dedicated soong-only ed25519 deploy key** minted (NOT literally WT's key — cleaner
+  independent revocation), pubkey appended to `deploy`'s `authorized_keys`
+  (fp `SHA256:MG7M3RiZJ176sLfblffb96V6W1qkRTgJ5dow1CpiY68`). Existing `deploy` key is
+  plain/unrestricted, so parity held.
+
+**The secret gate (the friction point):** repo Actions secrets are **vh-owner-only** —
+claude-bot's token is `write:package,read:repository` (403 on secret-write), and the vh
+package-scoped PAT also 403'd on `PUT …/actions/secrets/…`. So `DEPLOY_SSH_KEY` /
+`DEPLOY_HOST` (10.250.50.152) / `DEPLOY_USER` (deploy) HAD to be set by the operator.
+First operator attempt produced a **bad key paste** — the deploy step died with
+`Load key … error in libcrypto` + `Permission denied (publickey)` (build+push were green;
+live Soong never moved). Fix: operator re-set the secret; the minted key path was
+pre-validated from nh3-dev (`ssh -i … deploy@… 'cd /opt/soong-lab && docker compose config
+-q'` → OK, health 200) so the re-set was the only variable.
+
+**Validation:** `workflow_dispatch` via claude-bot **basic auth** (its token lacks
+`write:repository` for the dispatch API; the account password works). Run #5 (task 1886)
+GREEN — live container recreated `sha256:…541f7730` → `…07526a08`, `StartedAt` fresh,
+health 200. `/api/version` now reports **0.3.25** (run #5 shipped soong-dev's 1c2f831
+STYLE_WORKFLOWS re-pin as validation cargo). soong-dev synced `docs/DEPLOY.md`
+(commit `00b67c3`). NB: tag **v0.3.25 exists only locally** — pushing it would re-trigger
+a redundant build+deploy of the same commit (operator's discretion).
+
+**Ops now:** redeploy = tag `v*` or `workflow_dispatch` the CI (auto). Manual fallback =
+`sudo -u deploy bash -c 'cd /opt/soong-lab && docker compose pull && docker compose up -d'`
+(the `.env` is `deploy`-owned 600, so infra-ops needs `sudo -u deploy`, not a bare `cd`).
@@ -0,0 +1,46 @@
+- `[2026-07-18]` **soong-lab containerize cutover — COMPLETE + LIVE on corviduo-dev.**
+
+Migrated soong-lab (Noonien Soong character-design studio) from a hand-built
+`soong-lab-studio.service` (systemd + git-pull-on-webhook) to a containerized
+deploy, image built by CI + pushed to the Gitea registry. soong-dev owns the
+in-repo artifacts (Dockerfile/compose/workflow/`docs/DEPLOY.md` = checklist);
+infra-ops owned the host cutover. Operator confirmed functional ("Soong works
+great" — a real Soong turn round-trips + saves) → cutover 100% closed.
+
+**Final state (corviduo-dev, 10.250.50.152):**
+- Container `soong-lab-soong-lab-1` LIVE + healthy on `0.0.0.0:8443`, image
+  `gitea.phasefinal.com/vh/soong-lab:latest` (v0.3.24), `restart:unless-stopped`
+  (survives reboot; no systemd unit needed — docker restart policy handles boot).
+- Deploy dir **`/home/infra-ops/soong-lab-deploy/`** — pull-based `compose.yaml`
+  (image + env_file + `8443:8443` + named volumes; NO build/secrets stanza) +
+  `.env` (copied from the live `soong-lab.env`, STRIPPED of the `SOONG_LAB_*_DIR`
+  overrides so the container uses image defaults `/data/library` + `/data/portraits`
+  + `/app/web` → the volumes).
+- Named volumes `soong-lab_soong-library` + `soong-lab_soong-portraits`, migrated
+  from `/home/infra-ops/soong-lab-data/{library,portraits}` (2 saved designs incl.
+  **Sindra** + 27 portraits), **chowned `10001:999`** (the container `soong` user)
+  so it can read AND write new designs.
+- Old `soong-lab-studio.service` + `soong-webhook.service` (the `:9010` git-pull
+  redeploy listener) both **stopped + disabled**.
+
+**Topology reality (≠ what DEPLOY.md assumed):** there is **NO TLS proxy**.
+WT-personal (`:8081`) and soong-lab are **co-located on corviduo-dev**, and the
+Bifrost callback is **plain-HTTP same-host** `http://10.250.50.152:8443` — the
+value of `SOONG_LAB_BIFROST_ENDPOINT_URL`, unchanged by the move, so the WT
+Bifrost host-allowlist stayed valid as-is. Nothing on the WT side needed touching.
+
+**Safety net:** data backup `/home/infra-ops/soong-lab-data-backup-20260718-091831.tar.gz`
+(35M) taken BEFORE migration. Verified pre-retire: `/api/version` 200 (0.3.24),
+SPA `/` 200, `POST /bifrost/tool-call` → 401 (route present + auth-gated),
+bidirectional WT↔soong reachability, container healthcheck green.
+
+**Ops commands:**
+- Redeploy a new image: `cd /home/infra-ops/soong-lab-deploy && sudo docker compose pull && sudo docker compose up -d`.
+  (Auto-pull-on-`:latest` — watchtower or a deploy hook — is an open follow-up.)
+- Rollback: `sudo docker compose down` + `sudo systemctl enable --now soong-lab-studio.service soong-webhook.service`.
+- Homepage tile: manual `- Apps:` entry "Soong Lab" (href http://10.250.50.152:8443)
+  in esh-docker-vm `/opt/docker/conf/homepage/services.yaml` — corviduo-dev isn't
+  a Homepage-watched docker endpoint, so docker-label auto-discovery can't surface
+  it (see [[2026-07-18-fleet-gitea-runner-build-recipe]] for the CI half).
+
+See [[reference_corviduo_dev_emergency_ops]], [[reference_claude_bot_gitea_creds]].
@@ -0,0 +1,72 @@
+- `[2026-07-18]` **Zonos2 emotion CANONICAL from an empirical sweep + the voice-cloning pipeline.**
+
+**Voice-cloning pipeline (established this session).** Source zips at
+`/mnt/smithy/voice_clones/<name>.zip` (irv-ml1 NFS from nh3-nas; remount
+post-reboot) — each = diarized single-speaker podcast clips + `manifest.jsonl`
+(per-clip WhisperX `mean_score`, word timestamps, text) + `metadata.csv`.
+`~/development/zonos-tools/assemble_voice.py <dir>` ranks by mean_score and
+concatenates top clips to ~15–24s (Zyphra's blessed clone-ref length; single
+clip if already ≥15s). Drop the assembled `<Name>.wav` into the gateway voices
+dir → `voice:"name"`. 4 characters cloned: **Emmie, Penny, Natalie, Miranda**
+(+ Zyphra defaults AmericanFemale/Male/British/Cora) = 8 voices in
+`zonos-gateway`. Clone is inline `speaker_audio_base64` (text-independent Qwen3
+speaker embedding — NO transcript); `/tts/speakers` registration is
+session-scoped (needs `X-TTS-Session-ID`), so the gateway holds the ref wav and
+clones per-call.
+
+**Gateway voices are host-managed (bind-mount, added this session).** Added
+`./voices:/app/voices:ro` to `/opt/docker/compose/zonos-gateway/compose.yaml`
+(committed to `vh/zonos-gateway` + eshpfi mirror `438cd35`). So adding a voice =
+drop the wav + `docker compose restart zonos-gateway` (registry rebuilds at
+boot; NO image rebuild). This also un-stranded the other voices (deploy build
+context had only Cora before). Voice wavs committed to the repo for backup.
+
+**Emotion mechanism (Zyphra canonical, from their README @194c0a3).** Additive
+direction vectors: 4 named (happy/sad/angry/surprised) + valence/arousal axes.
+`emotion_strength` 1.0 = per-voice calibrated (calibration.json optimizes
+emotion2vec recognizability only, NOT identity). `accurate_mode` is THE trade-off:
+`true` = closer voice match (identity), `false` = expressive mode (emotion lands,
+identity drifts). Zyphra's strong recipe: `accurate_mode:false` + `cfg~1.5`.
+Single-emotion is blessed; mixing is unblessed (and degrades the clone — operator
+confirmed by ear). "deaf by 1.5" — cfg past 1.5 distorts + costs ~2× compute.
+
+**THE SWEEP (`~/development/zonos-tools/emotion_sweep.py`).** 4 cloned voices × 4
+named emotions × {accurate,expressive}×{cfg 1.0,1.3,1.5} @ strength 1.0,
+single-emotion, neutral sentence + a neutral baseline per voice (~100 clips).
+Scored on TWO axes: **emotion-landing** = emotion2vec `iic/emotion2vec_plus_large`
+target-emotion prob [0-1]; **identity** = resemblyzer speaker-embedding cosine vs
+the clone reference (neutral baseline ~0.85). Scoring env:
+`uv run --with resemblyzer --with funasr --with "numpy<2" --with soundfile
+--with requests --with "setuptools<80" --with torchaudio` (setuptools<80 for
+webrtcvad's pkg_resources; torchaudio for funasr).
+
+**RESULTS (mean across the 4 voices) — emotion, best setting, emo/id:**
+- happy — **exp cfg1.5** 0.80/0.68 (soft: exp cfg1.0 0.76/0.69) → WORKS
+- sad — **exp cfg1.5** 0.53/0.57 (only working cell; id below the ~0.65 floor) → modest
+- angry — acc cfg1.3 / exp cfg1.5 tied at ~0.25 emo → WEAK (named ceiling ~0.25)
+- surprised — max ~0.015 across ALL settings → NON-FUNCTIONAL on the named direction
+Accurate + low cfg = identity/suppress regime (emo→0); expressive REQUIRED for
+emotion to land, at ~0.15–0.28 identity cost.
+
+**dvalin-smithy-dev synthesis (adopted, triaged genuine-adds; thread
+`01KXT12FN0AS5A3WMKEK06BVPS`):**
+1. Treat **identity as a hard FLOOR (~0.65)**, not a free variable in emo×id.
+2. **Two-regime policy** — Regime A (default, identity-critical dialogue):
+   `accurate_mode:true, cfg 1.0, emotion off` (text carries it) or soft-happy
+   (exp cfg1.0). Regime B (tagged drama beats): `accurate_mode:false, cfg 1.5`,
+   single emotion or axes. Line-type→regime heuristic (exposition→A, grief→B+sad,
+   confrontation→B+axes-angry, shock→B+axes-arousal).
+3. **Axes-first for the broken emotions** — angry ≈ valence −0.6..−0.8 / arousal
+   +0.5..+0.8; surprised ≈ valence +0.2..+0.4 / arousal +0.7..+1.0 (exp cfg1.5);
+   or "startled-happy" (happy + high arousal) as a surprised stand-in. These are
+   PROVISIONAL — the sweep did NOT test axes.
+
+**NEXT (highest VoI, operator to green-light):** an **axes sweep** for
+angry/surprised (valence×arousal grid) — the only path to rescue the two broken
+named emotions; then a strength ladder at the best cells + emotion-congruent text
+(neutral content understates landing) + per-voice tables + a 2nd emotion judge /
+human pairwise. Then bake the happy/sad canonical into gateway presets. I owe
+dvalin the axes-sweep numbers.
+
+See [[reference_zonos_tts_stack]]; dials-first spec at `vh/zonos-gateway`
+`docs/EMOTION-DIALS-SPEC.md`.
@@ -0,0 +1,41 @@
+- `[2026-07-18]` **zonos-gateway 0.2.1 — voice-resolved emotion presets baked (provisional) from the axes sweep.**
+
+After the axes sweep ([[reference_zonos_tts_stack]] + the `[2026-07-18] axes sweep`
+Recent-decisions entry) rescued angry and confirmed startled-happy, the operator
+green-lit baking the results as **provisional** gateway presets + docs. Shipped
+`vh/zonos-gateway` **0.2.1** (main `8f1885b`, tag `v0.2.1`, PUSHED; deployed live
+on irv-ml1 `:8890`).
+
+**Design — voice-resolved, NOT global.** `resolve_preset(name, voice)` picks the
+per-voice measured cell, because a single global preset is unsafe (dvalin ruling;
+BritishFemale's *named* angry misfires as fear). Presets:
+- `angry`, `happy`, `startled_happy` (+ aliases `surprised`, `startled` →
+  startled_happy). All expressive (`accurate_mode:false`), cfg 1.5, pure-axes
+  (no named sliders).
+- Calibrated cells (the 3 default voices):
+  - angry: AmF v-0.4/a+1.0 s1.0 (emo0.53/id0.685); BrF v-0.4/a+0.8 s1.0
+    (emo0.99/id0.725, metric fear-clean); AmM **two-tier** — soft v-0.6/a+0.8 s1.0
+    (0.23/id0.654) + drama v-0.6/a+0.8 s1.2 (1.0/id0.616 clean; strength is NOT a
+    smooth knob on AmM, 1.0→1.2 is the window, past that flips to disgust).
+  - happy / startled_happy: AmF v+0.6/a+0.8; AmM v+0.3/a+1.0; BrF v+0.6/a+1.0
+    (happy~1.0, id 0.74-0.80; axes-happy keeps +0.15 id over the named happy slider).
+  - `sad` = unchanged named-slider preset (not axes-tested).
+- Uncalibrated voices (Cora + the 4 clones) → mid-region fallback until measured.
+- Docs surface: `/v1/dials` exposes `voice_emotion_presets`; the FastAPI `/docs`
+  description documents it; durable spec `docs/EMOTION-DIALS-SPEC.md` (moved INTO
+  the repo — was mirror-only); README table. 44 tests green.
+
+**Repo-hygiene gotcha (fixed).** The local clone `~/development/zonos-gateway` and
+gitea `vh/zonos-gateway` had **TWO UNRELATED git histories** (no merge-base) — gitea
+held the voice-wav commits, the local clone held the code + no remote. Reconciled
+by resetting local→origin/main, overlaying the 7 bake files, `uv lock`, commit,
+push (fast-forward). Voices stay tracked; local now shares gitea's lineage + has
+origin wired. **The deployed irv-ml1 tree `/opt/docker/compose/zonos-gateway` is
+still NON-git** (hand-updated build context) — CI-wire remains an open follow-up.
+
+**Provisional pending** ear-validation on emotion-congruent text (the neutral-text
+audition was inconclusive: "they all sound different, hard to tell"). Follow-ups:
+sad axes/text pass on the 3 voices; congruent-text pass; clone-char emotion rows.
+Tools `~/development/zonos-tools/{axes_sweep,strength_ladder,gen_auditions,dial-in-studio}.py`
+(run ON irv-ml1; scoring env `uv run --with resemblyzer --with funasr --with "numpy<2"
+--with soundfile --with requests --with "setuptools<80" --with torchaudio`).
@@ -0,0 +1,32 @@
+`[2026-07-25]` **infra-ops Worldtree config-as-code repo — SHIPPED + boundary AGREED.**
+
+**STATUS (2026-07-25, done this session):** `vh/worldtree-instance-configs` (private, gitea) built, pushed, validated; boundary agreement secured from worldtree-dev.
+
+- **Repo:** dir-per-instance `demo/` + `personal/` (5 files each: `defaults.yaml`, `policies.yaml`, `model_roles.yaml`, `providers.yaml`, `matrix.yaml`), seeded byte-exact from live `/opt/<instance>/config`. `pinned/` = README stub only — **no `/app/config` bind-mount; config baked into frozen image `446e5807` (2026-05-13)**, so out-of-scope; deploy verb refuses it.
+- **Tool:** `scripts/deploy-wt-config <verb> <instance>` — `diff` (read-only repo-vs-host), `deploy` (in-run host backup → `install -o vh -g vh -m 644` → restart **api+matrix** → health-gate api `/health` → auto-rollback), `capture` (host→repo reconcile). Instance table in-script (demo→`/opt/worldtree/config`+`worldtree-worldtree-{api,matrix}-1`; personal→`/opt/worldtree-personal/config`+`worldtree-personal-worldtree-{api,matrix}-1`). Matrix sidecar shares the config mount but has no healthcheck → restart both, gate on api. Env `WT_CONFIG_HOST` (default `infra-ops@10.250.50.152`), `WT_HEALTH_WAIT` (90s). Local clone `~/development/worldtree-instance-configs`.
+- **Gitea plumbing (reusable):** nh3-dev **403s the gitea HTTP API** (public fail2ban + internal `:3000` both 403). Repo CREATE went via **ana-docker localhost API** (`ssh infra-ops@10.250.50.70` → `curl localhost:3000/api/v1/user/repos`, vh token from `~/.config/tea/config.yml`, operator-authorized one-time). PUSH went over **internal git-SSH `ssh://git@10.250.50.70:222`** (works from nh3-dev; auths as vh). `git init` defaulted to `master` → renamed `main` to match repo default_branch.
+- **Boundary AGREED (worldtree-dev, althing thread `01KYCAECRWVEF16EVKQAGT2N80`):** no hand-edits to `/opt/<instance>/config`; config changes route to infra-ops as deltas (worldtree-dev owns CONTENT + approval trail — the wyrd-grant shape — infra-ops lands+deploys). **Three-layer model:** image `config/` = baseline new instances seed from (theirs) → `vh/worldtree-instance-configs` = per-instance truth (ours) → host bind-mount = deploy target (written only by the tool). **Carve-out:** worldtree-dev's admin-API ops (`/admin/keys` mint, tier changes, session retirement, future runtime-grant surfaces) mutate instance **DATABASES not config files** → NOT config edits, stay in-band. If a future API writes config *files*, they flag at design time. b132 CONFIG BASELINE breadcrumb composes (INFO line = config-as-code diverges from image baseline, by design).
+- **No live deploy** done or needed — repo seeded == live (diff clean, capture round-trips zero-diff). Deploy path is dry-run-validated only; first real deploy needs operator per-change yes (managed box).
+
+---
+
+_Original plan (2026-07-25, pre-build):_
+
+`[2026-07-25]` **infra-ops to OWN a Worldtree per-deployment config repo + deploy tooling (operator-directed).**
+
+**Decision.** Vuong directed (2026-07-25, this session) that Worldtree instance config should be a *tracked change*, **managed and deployed by infra-ops — not worldtree-dev**. Model: worldtree-dev owns the app/image (+ the baked baseline defaults); **infra-ops owns config-as-code for every deployment** and deploys it. This is the durable fix for the root cause behind the whole #376 arc — config was edited live on host bind-mounts (`/opt/<instance>/config/`) with zero version history, audit, or recovery.
+
+**What "no worldtree-dev involvement" does and does NOT cover** (clarified with the operator this session):
+- **Build + deploy = infra-ops-only.** Deploying config = write the host bind-mount file + restart the container (the *exact* procedure already run this session — backup → replace → restart → health-gate → rollback-on-unhealthy). No worldtree-dev in the deploy loop. Their CI only swaps the IMAGE; it does NOT resync the host config bind-mount (confirmed #376 finding).
+- **ONE load-bearing exception — a one-time boundary agreement, NOT per-deploy involvement:** for the repo to *own* config it must be the **only writer**. worldtree-dev "live-bridges" (hand-edits mounted config directly on the box). If the repo deploys config *and* they keep live-editing → **two writers fighting the same files** = #376 all over again. So secure a one-time "yes" from worldtree-dev: *the config repo is now authoritative; stop hand-editing `/opt/<instance>/config`; route config changes through the repo.* (Five-minute agreement, not a design collab.)
+- **Standing coupling (not "involvement"):** the config *schema* is the app's, enforced by its boot validator (`core.config_validator`). infra-ops configs must stay schema-compatible with the deployed image; the boot gate is the loud backstop.
+
+**Build shape (recommended):**
+- Gitea repo `worldtree-instance-configs` (infra-ops-owned), **dir per instance** (`demo/`, `personal/`, `pinned/` — the three on corviduo-dev 10.250.50.152: demo `worldtree-worldtree-api-1` :8080, personal `worldtree-personal-worldtree-api-1` :8081, pinned `worldtree-pinned-worldtree-api-1` :8082). Config dirs: demo `/opt/worldtree/config`, personal `/opt/worldtree-personal/config`, pinned `/opt/worldtree-pinned/config` (verify pinned's mount).
+- **SEED FROM CURRENT MOUNTED STATE, don't author fresh** — capture each instance's live config (incl. legitimate live-bridged deltas: personal carries `agent_architect` role [Soong/soong-lab] in model_roles.yaml + `ratatoskr-affect-full-allow` in policies.yaml that are NOT in the app repo — the operator ruled these are BY DESIGN, keep them). Losing them = breakage (the affect-render one gates mood rendering).
+- Deploy script (e.g. `scripts/deploy-wt-config <instance>`): git = source of truth → push to host bind-mount + `docker restart` (same pinned image, no pull) + health-gate + auto-rollback. This is the proven-this-session procedure, scripted.
+- Files per instance: `policies.yaml`, `model_roles.yaml` (+ whatever else is bind-mounted — `defaults.yaml`, `providers.yaml`, `matrix.yaml` all live in `/opt/<instance>/config`; decide scope — policies+model_roles are the authz/role layer, defaults/providers are heavier instance tunables).
+
+**Tracking surface:** operator-directed 2026-07-25, carried by this snapshot + `/tmp/infra-ops-handoff.md`. No issue filed (infra-ops-internal build). Related fleet idiom to reuse: canonical-sync (`.corviduo-canonicals.toml` / `canonical_sync.py`). Later scale option (deferred, needs worldtree-dev): base+overlay with a merge step in their pipeline.
+
+See [[2026-07-25-wt-376-per-instance-config-arc]] for the incident that produced this. Auto-memory: `reference_worldtree_perinstance_config`, `reference_corviduo_dev_emergency_ops`.
@@ -0,0 +1,13 @@
+`[2026-07-23→25]` **Worldtree #376 config-divergence arc — wyrd grant fixed, drift guard demoted, per-instance config ruled BY DESIGN.**
+
+**Trigger.** wyrd-dev needed `session.history.write` on the DEMO Worldtree (operator-approved) — add `wyrd-dev` to the `session-history-write-ratatoskr` policy rule. Attempting it surfaced that the demo runtime `/opt/worldtree/config/policies.yaml` (mtime Jul-17) had **silently diverged from the repo** — missing whole rules, not a faithful copy of any revision. Refused to hand-edit a divergent authz file on a managed box; worldtree-dev prescribed a **wholesale replace** with `main@55f3fde`. Executed (backup → replace → `docker restart worldtree-worldtree-api-1` same-image → health-gate → verify) — grant live, demo policies at parity. **This was the ONE genuine bug of the arc:** a demo-intended grant that wasn't ON demo.
+
+**Drift guard (#376).** worldtree-dev shipped a startup guard (b131/`62b85e3`) that hashes mounted config vs the image's baked copy, logging `CONFIG DRIFT (#376)`. First reading found MORE drift: `model_roles.yaml` on both instances + personal `policies.yaml`. Captured the three runtime-vs-baked diffs (read-only) → all had **runtime-only content** (personal's `agent_architect` role + `ratatoskr-affect-full-allow` rule — live-bridged, ahead of repo). The guard's STOP-on-runtime-only rule earned its keep: a blind "sync to repo" would've deleted legitimate per-instance config.
+
+**Operator ruling (2026-07-25 — the reframe).** Worldtree will run dozens-to-hundreds of instances at v1, each configured for its env. **Per-instance config deltas are the DESIGN, not rot; back-streaming to the canonical repo doesn't scale.** Everything stood down: demo model_roles normalize withdrawn, personal sync cancelled, post-mortem dissolved, `.bak` deleted. Guard demoted b132/`ad596b5` from ERROR alarm to INFO `CONFIG BASELINE (#376)` breadcrumb (WARNING only for a mounted file *entirely absent* = breakage-adjacent). Breaking-change protection stays in `core.config_validator`'s boot gate.
+
+**infra-ops watcher — built then retired same day.** Wired an off-box `wt-drift-watch` (systemd --user timer on nh3-dev, alerts worldtree-dev on new `CONFIG DRIFT` startup lines) — then RETIRED it per the ruling (the error line is going away). Lesson banked in auto-memory `reference_worldtree_perinstance_config`.
+
+**Governance notes worth keeping:**
+- The auto-mode guard **blocked** a peer-green-lit (worldtree-dev) config replace on the managed demo box because there was no *operator* consent for that specific change — correct: a config-mutation+restart on shared infra needs the operator's yes, not just a peer's. Surfaced it; the operator later stood the whole thing down. Good governance on both ends.
+- This arc is the direct evidence base for the [[2026-07-25-infra-ops-wt-config-repo]] decision (infra-ops owns config-as-code so live-edits stop being untracked).
@@ -0,0 +1,15 @@
+`[2026-07-31]` **kimi-k3 "output cap" root-caused = a ~16384 REASONING-token ceiling, not an output cap; fix relayed to heid, NOT applied gateway-side.**
+
+heid reported that `kimi-k3` (the primary route = Kimi Code coding endpoint `openai/k3` @ `api.kimi.com/coding/v1`) silently degraded its cross-frontier panel: on large/reasoning-heavy dispatches, `completion_tokens: 16381` **exactly**, `content` empty, `reasoning_content` ~64KB, `finish_reason: **stop**` (a truncation mislabeled as a clean stop). `max_tokens: 100000` in the request was not honored.
+
+**Investigation arc (a clean cross-frontier-triage + verify-on-the-wire case):**
+1. My first read: a flat ~16384 OUTPUT cap; fix = a LiteLLM `stop→length` relabel callback (heid's fallback ask). Confirmed the cap isn't in our LiteLLM config (no `max_tokens` clamp on the route).
+2. Operator routed a fix-research pass to **dvalin-smithy-dev + bil-smithy-dev** (independent). Both CONVERGED (docs-based): `max_tokens` is a deprecated alias on Kimi/Moonshot; the canonical field is `max_completion_tokens` (default 131072, max 1M); the coding endpoint defaults output to 16384; fix = send `max_completion_tokens` + `reasoning_effort` via `extra_body` (drop_params-safe).
+3. **heid's live data REFUTED the docs hypothesis:** a later dispatch hit `completion_tokens: 18455` (ABOVE 16384) cleanly, with `reasoning_tokens: 16198` (just under 16384) and content present. So COMPLETION is uncapped; the bound is on **REASONING at ~16384**. When a hard task's thinking exhausts that budget, nothing's left for content → empty answer under `stop`.
+4. **I proved it on the wire** — ran heid's real 500KB failing bundle direct at both endpoints (bypassing LiteLLM so `reasoning_effort` isn't dropped): default effort → 504/timeout (the failure); **`reasoning_effort: low` → reasoning ~12–13.5k (under the ceiling), content returns (6–7.6k chars)**, on BOTH coding AND general endpoints. So re-routing to the general endpoint buys nothing — the fix is the effort param, and it works on the wire.
+
+**THE FIX (caller-side, no shared-gateway change/restart):** send `reasoning_effort` via **`extra_body`** on kimi-k3 dispatches (`low` for large bundles). LiteLLM `drop_params: true` strips the top-level `reasoning_effort` — which is exactly why heid's earlier `reasoning_effort: low` was a no-op. `extra_body` survives drop_params (the house GLM-thinking pattern). Tradeoff: low effort = shallower reasoning, but a complete answer beats today's empty one.
+
+**Relayed to heid to validate on a real round** (the one unconfirmed hop is whether `extra_body` survives OUR LiteLLM). **Backstop if it doesn't:** add `allowed_openai_params: ["reasoning_effort"]` to the `kimi-k3` route in the gateway config — that IS a shared-gateway change + a ~10s restart (blips all consumers), so it needs a heads-up.
+
+Gateway = LiteLLM on ana-docker `10.250.50.70:4000`; kimi-k3 config in `stacks/litellm/conf/config.yaml` (see Recent-decisions `[2026-07-25]` Kimi K3 wiring). No gateway change was made this session. Failing dispatch on record: `01KYTASKTY3T` (jackdaw-dev bug-hunt).
@@ -0,0 +1,24 @@
+`[2026-07-31]` **muninn-gate (#377 ingestion front door) BUILT + DEPLOYED + healthy on corviduo-dev `10.250.50.152:8090`.**
+
+WG-internal HTTP front door for the Muninn ingestion queue (`vh/muninn-gate`, muninn-dev's repo). The full provisioning ask (staging mount + closed-schema config + bearer keys + compose/WG bind) came after a 4-message discovery exchange with muninn-dev + a cross-team coordination with worldtree-dev; operator ruled the open architecture call (shared mount) and greenlit build+boot.
+
+**Deployment (eshpfi `stacks/muninn-gate/`):**
+- Image `muninn-gate:0.0.14` — no Dockerfile upstream, so infra-ops owns containerization. `python:3.11-slim` + `uv pip install .`; **`muninn-dispatch==0.1.4` from the internal Gitea index** (`[tool.uv.sources]`, `uv pip install .` honored the pin), token passed as a **BuildKit secret** (`--secret id=gitea_pw`) so it never lands in a layer. Built on corviduo-dev.
+- **`ingestion_root: /data/state/ingestion`** — the `worldtree-personal_worldtree-state` docker volume mounted at `/data/state`, byte-identical to the watcher's view. **Acceptance criterion (muninn-dev's): `/health` → `watcher.running: true` PROVES byte-identity** (the gate reads the heartbeat the watcher writes); `no_heartbeat` with the watcher up = root mismatch. Verified true first boot.
+- **`user: "1000:1000"`** — the ingestion dir is `vh:vh 0755`, so a non-root gate had to run as uid 1000 to WRITE the queue (my Dockerfile's `USER gate`/10014 would've been denied; the watcher itself runs as root and bypasses perms). This uid requirement was a genuine spec gap — muninn-dev added it to the contract (`084526e`, vh:vh 0755 + 1000:1000 as the worked example) so no future deployer re-derives it. `ingestion_root_writable: true` in `/health` is the post-deploy confirmation.
+- **staging `/mnt/muninn-staging/mimir-inbox`** — bound `:ro`, SAME absolute path in BOTH the gate AND the watcher (dispatch stores paths absolutely; the watcher opens them at claim time). worldtree-dev added the watcher-side bind (their image) in **b162** (`${MUNINN_STAGING_DIR:-…}:/mnt/muninn-staging/mimir-inbox:ro`). Currently a **LOCAL placeholder dir** on corviduo-dev.
+- config (single-writer) `/opt/docker/conf/muninn-gate/muninn-gate.yaml` (0600, 1000:1000). Schema CLOSED (unknown field = boot failure). 2 bearer keys minted: `mimir-inbox` [read,submit], `ops-curl` [read,submit,control]. `network_mode: host`; health probe = **`/ping`** (NOT `/health`, which is always-200 by design and would never restart the gate). Committed `786462a` (no secrets).
+
+**Verified boot:** `/ping` `{"service":"ok"}`; `/health` (ops-curl bearer, 200) `watcher.running:true` + `ingestion_root_writable:true`. muninn-dev independently poked the live gate — auth/route surface all held (401s w/ `WWW-Authenticate: Bearer`, the 4 FastAPI default routes gone, error-envelope-not-307 on trailing slashes = bug-hunt findings 5+6 confirmed outside pytest).
+
+**DEFERRED (the submit path) — the mimir-inbox era:** SUBMIT returns `not_found` against the placeholder staging (correct, not a defect — muninn-dev confirmed) until the real staging dir + a mimir-inbox writer exist. ~~Operator ruled shared mount (mimir-inbox stays off-box, writes to a shared/NFS mount both gate + watcher bind at the same path).~~ **SUPERSEDED 2026-08-01 — operator REVERSED to CO-LOCATE:** mimir-inbox runs ON corviduo-dev, alongside the gate + watcher, staging = a corviduo-dev-LOCAL dir (not NFS). Reason the off-box/NFS call fell: muninn-dev's code-check showed staging is NOT same-fs-constrained (gate reads staging metadata + passes path strings; `os.replace` is inside `ingestion_root`), so staging's real constraint is **path identity across writer/gate/watcher**, which co-location buys outright — and it sidesteps the NFS failure modes (path-identity break, TOCTOU widening, stale handles, a hung mount blocking `resolve(strict=True)` — the last of which blocks mimir-inbox's *event loop*, not just a threadpool worker, since its staging check is in an async handler). Ruling relayed 3× (muninn-dev ×2 w/ msg-id citations, mimir-dev ×2) + operator in-session; **mimir-inbox key handed over 2026-08-01** (bumped to [read,submit,control], 0600 drop on nh3-dev). Tail on the co-locate ruling: raise worldtree-dev (box-side provisioning + the watcher claim-semantics open Q) → provision the real corviduo-dev-local `/mnt/muninn-staging/mimir-inbox` (uid = mimir-inbox's runtime identity, rw-writer / ro-gate+watcher) → 0600 key drop on corviduo-dev → muninn-dev's **one-file path-agreement probe** → acceptance. NB gate submit surface = **`POST /jobs`** (path-addressed; NO `POST /upload` — upload deferred v0, gate never ingests bytes). `staging_roots` already allowlists the path (no gate-config change).
+
+**RESOLVED 2026-08-01 (worldtree-dev, from source `core/muninn/runner.py:362-367`):** the watcher **OPENS the staged file in place** at claim (`parse_document(file_path)` on the dispatch-recorded absolute path) — it never moves/copies the source into the job dir (job dir holds DERIVED artifacts only). Consequences: (1) staging needs **PATH IDENTITY only**, so **co-location is a CONVENIENCE, not a requirement** — the parked multi-host option stays fully viable with a shared mount at the same absolute path on both hosts. (2) The real same-fs constraint is `.enqueue-tmp/` → `os.replace` into `pending/`, same-fs with `ingestion_root` — never staging (confirms muninn-dev). (3) **⚠️ OPERATIONAL RULE for mimir-inbox lifecycle (worldtree-dev):** open-in-place means the staged file MUST stay present+readable from submit **until the job is TERMINAL** (complete / failed-and-not-retried) — retry re-runs the structure phase, which re-opens the staged path. A cleanup that deletes on 201-submit kills every job at claim with a not-found that looks EXACTLY like the namespace-mismatch failure the bind exists to prevent. Relayed to mimir-dev for their cleanup design. **Gate-side edge (muninn-dev):** `POST /jobs/{id}/retry` returns `200 {requeued}` even for a job whose staged source was deleted — `muninn_dispatch.requeue` validates job STATE not file existence, and admission isn't re-run on retry (nothing re-stats files) → a FALSE success that dies at claim. Gate deliberately unguarded (re-admit re-resolves under a new clock, still races; lifecycle is the writer's), recorded as a gate compatibility constraint. So the retention rule isn't just "avoid claim-fail" — it's "retry will LIE with a 200 if the source is gone."
+
+**worldtree-dev approved co-location** (2026-08-01): another small infra-ops-managed LAN/WG-internal service on corviduo-dev in the gate's posture is fine at their OS/app layer; port/supervision/identity mine to shape; staging-dir ownership flip (mimir-inbox-writable, gate+watcher :ro — b162 watcher bind already :ro) at my convenience. **NEXT: coordinate the mimir-inbox deploy inputs with mimir-dev** (image/build recipe — likely infra-ops containerizes like muninn-gate; app config/env; port), then provision staging dir + stand up the service (uid 1000, matching the corviduo-dev muninn stack) + 0600 key drop on corviduo-dev + muninn-dev's path-agreement probe + acceptance.
+
+**Operational guard (no auto-check exists):** docker fabricates a MISSING bind source as an empty dir that passes every closed-config check → **confirm the host mount actually exists before wiring/repointing a bind** (`os.path.ismount` breaks on subdir roots; emptiness is normal pre-first-upload). This is why the gate/watcher path-agreement is an operational discipline, not a validated invariant.
+
+**Hardening candidate (flagged, not done):** the compose mounts the WHOLE `worldtree-personal_worldtree-state` volume at `/data/state` per muninn-dev's spec; a subpath mount of just `ingestion` → `/data/state/ingestion` would be tighter (gate only needs RW on ingestion). Confirm with muninn-dev before adopting.
+
+See auto-memory `reference_muninn_gate_deploy`, `reference_muninn_gate_staging_path`; [[2026-07-25-infra-ops-wt-config-repo]] (corviduo-dev boundary), and Recent-decisions `[2026-07-27]` muninn watcher sidecar (the other half of #377).
@@ -0,0 +1,32 @@
+`[2026-08-02]` **The mimir-inbox / #377-read-path arc — deploy, four bugs found+fixed+verified, a cloned voice, all in one long session (2026-08-01→02).**
+
+The browser-facing half of the #377 Muninn ingestion arc, end to end: mimir-inbox stood up, the write path proven, the read path chased through four defects to a verified-working state, and a character voice cloned into the TTS zoo. Peers: mimir-dev (the app), muninn-dev (gate/watcher spec), worldtree-dev (Worldtree app layer + the #380/#381/#382/#383 fixes), ratatoskr-dev (a consumer + the rigorous verifier).
+
+## mimir-inbox deployed (#377)
+- **New infra-ops stack, canonical eshpfi `stacks/mimir-inbox/`; live corviduo-dev `10.250.50.152:8091`** (co-located w/ muninn-gate :8090 + the worldtree-personal muninn watcher). Full deploy detail + procedures → auto-memory `reference_mimir_inbox_deploy`.
+- **Placement decision (operator, reversed):** 7-31 he ruled mimir-inbox stays OFF corviduo-dev (shared/NFS mount); 8-01 he REVERSED to CO-LOCATE. Trigger: muninn-dev's code-check showed staging is NOT same-fs-constrained (gate reads staging metadata + passes path strings; `os.replace` is inside `ingestion_root`) — staging's real constraint is **path-identity across writer/gate/watcher**, which co-location buys outright while dodging NFS failure modes. I HELD the reversal for the operator's direct word (data/hosting on a team-managed box, reversing his own ruling) even against 3 peer relays — vindicated as the right instinct; muninn-dev agreed.
+- Build: **`uv sync --no-dev --frozen`, SINGLE-STAGE** (project installs editable-linked to `src/`, so src/ MUST stay beside .venv — a multi-stage "copy only .venv" dies at import/404s assets). uid 1000, host-net bind 10.250.50.152:8091, TCP-liveness healthcheck (deliberately NOT gate-coupled). Redeploy = refresh build context (**preserve the on-server `.env`!**) → `docker build -t mimir-inbox:0.0.1 -t mimir-inbox:<sha> .` → `compose up -d`. Version stays 0.0.1 across dev commits → tag the image w/ the source SHA too. Live commit progression `0478452`→`c8ab38f`→`2dcc77e`→**`8ece117`** (3 redeploys).
+- mimir-inbox key on the gate bumped [read,submit]→**[read,submit,control]** (cancel/retry); brokered via a 0600 drop on nh3-dev (never on the althing bus).
+
+## The read-path bug chain (worldtree-dev's, all found via this arc)
+- **#380 wing-blind indexing:** the book-ingest path upserted concepts into a hardcoded `main` Chroma collection while wing search reads the `fiction` collection → P&P written to disk but `search_library` returned total 0. A silent-success defect ("complete/69 indexed" was right about the WRITE, wrong drawer). Root-caused off MY physical evidence (files on disk + search empty). Fixed b164 + a one-shot `--reindex <job_id>` (re-upsert into the right wing collection + delete stray `main` rows).
+- **#381 stale Chroma client:** the personal api opens its Chroma client before the watcher's cross-process writes → **a freshly-ingested/re-indexed book is NOT queryable until the api is restarted.** Proven by my restart-diagnostic (pre-restart total 0 → post-restart hits, same index). Workaround until fixed: `docker restart worldtree-personal-worldtree-api-1` after any ingest/re-index. Filed as #381.
+- **#382 unreliable Mimir grounding (the subtle one):** post-#380-fix the index was correct, but Mimir's grounding was INTERMITTENT — some sessions navigated the opaque job-hash dir (`mimir-f3887c9b97b7`) to the content, others distrusted the correct vector hits and **silently answered from training knowledge** (worst of the looks-fine-isn't family). ratatoskr-dev caught it; I'd been over-confident ("Mimir read Austen back to you") having verified the INDEX, not the GROUNDING. Fixed b166 with BOTH shapes: a self-describing `_index.md` per wing job-dir (resolves the hash dir to its title) + a Mimir prompt rule (wing-scoped hits ARE library content, never discard on a name mismatch, never substitute training). **Verified: ratatoskr-dev re-ran 3× fresh sessions → 3/3 grounded**, citations in note-extracted language not raw Austen. #382 CLOSED.
+- **DCC (Dungeon Crawler Carl, job `b59c147c5ce0`) backfill:** `--reindex` FAILED ("job not found in any state dir" — predates state-tracking). SETTLED = **no re-file** (the b166 prompt rule already grounds it even without an `_index.md`; ratatoskr confirmed incidentally); an `_index.md` rides whenever DCC is next re-ingested.
+- **#377 mimir-inbox banner bug (mimir-dev's, `8ece117`):** `/health-banner` misattributed an unwritable `ingestion_root` to the WORKER, rendering "The worker is not running." for a running worker — a false lead pointed at infra-ops's half of #377. Fixed (guard split into two banners); I confirmed from the DEPLOYED handler (not just the test) that `ingestion_root_writable:False` now renders "The ingestion root is not writable."
+
+## muninn-gate → muninn-dispatch 0.1.5
+Rebuilt `muninn-gate` off `vh/muninn-gate` main `bc04c4c` (dispatch 0.1.4→0.1.5) so the gate serves the new `concept_schema`/`concept_schema_source` row fields (computed gate-side). Gate version unchanged 0.0.14 (dual-tag the SHA). Build needs the vh gitea token as a BuildKit secret (`--secret id=gitea_pw`, UV_INDEX_GITEA_USERNAME=vh, drop+shred). Recreate with `compose up -d` (NOT bare restart — needs the new image). Verified: P&P job serves `concept_schema='fiction'`, `concept_schema_source=null` (null correct — pre-b164 job). Registry tags by commit SHA — `v1.0.0bNNN` docker tags don't resolve; use the deployed SHA (confirm `--reindex` present before using an image for a data-op).
+
+## donut voice (65-frost → Zonos gateway)
+Operator: "pick up 65-frost, use that bundle as a voice for a character named donut." 65-frost = a **Booth id** (`~/booth-data/65-frost/`) holding a curated yt-voice-clipper dataset (`dataset-…-curated.zip`: 4 clips + manifest, all SPEAKER_02 = Princess Donut). **Zonos gateway voice registry = a filesystem drop:** `<Name>.wav` in the voices dir (44.1kHz mono s16 PCM) auto-registers as `voice:"<name>"` on **startup** (needs a restart). The LIVE dir is the bind mount `/opt/docker/compose/zonos-gateway/voices/` (lkraven-writable), NOT the working tree. Built `Donut.wav` from seg000 (best clip), dropped it, restarted → `voice:"donut"` live in the gateway AND the Asset Engine's make form. Also copied to the build-source tree `~/zonos-gateway/voices/` for rebuild-durability (true canonical = the gitea repo, not yet CI-wired). Auditioned in booth `donut-voice`. **Expanded 2026-08-02 (onyx-58 bundle):** operator curated a 2nd Booth bundle `onyx-58` (`dataset-467d2cf8…curated.zip`, 3 Donut clips) as additions. Rebuilt the reference = **seg000 (65-frost) + seg101/seg110/seg148 (onyx-58)** ffmpeg-concat + resampled 24k→44.1k mono s16 = **52.0s**. `seg148` was diarized SPEAKER_03 but is Donut (operator-confirmed misdiarize → included). Assembly is NOT `assemble_voice.py` (that `-c copy` can't resample + caps ~15s); used a manual `aresample=44100,aformat=…,concat=n=4` filter. Backed up old ref → `irv-ml1:~/Donut.wav.pre-onyx58`; dropped to live bind-mount + build-source tree; `docker compose restart` (healthy 2s, `voice:"donut"` still 1 of 9). A/B booth `donut-onyx58` (A=old 16.3s ref, B=new 52s ref, same line). Longer ref is fine mechanically: gateway passes it as `speaker_audio_base64` → speaker *embedding*, not an audio prefix. **BUT auditioned → REVERTED same day:** pinned-seed neutral A/B (5 pairs, booth `donut-onyx58`) showed the single-clip seg000 (16.3s) beats the 52s 4-take concat on timbre — concatenating disparate takes muddied the embedding more than the range helped. Reverted both live + build-source to seg000-alone. Lessons (→ Tried-and-abandoned): more reference ≠ better when takes vary; and **emotion steering pulls output away from the clone fast** (operator craft rule) — keep clones emotion-neutral; bare `{input,voice}` calls send NO emotion (gateway only enables it on an explicit `emotion_*`/`preset` dial).
+
+## Zonos streaming (no gateway change needed)
+ratatoskr wanted play-as-it-arrives. `/v1/audio/speech` ALREADY streams — chunked `StreamingResponse`, opens native `/tts/generate` with `stream=True`, wraps as a streaming int16 WAV with `0xFFFFFFFF` placeholder sizes (meant for progressive `<audio>`). Verified TTFB 0.44s vs 6.84s total, `transfer-encoding: chunked`, dials preserved. ratatoskr's proxy was rewriting the placeholder header → forced buffering. Fix was theirs (pass chunks through); shipped + confirmed (TTFB 0.46s progressive). The Asset Engine (ana-docker:8200) IS the fleet "TTS zoo" (~20 audio svcs w/ irv-ml1 endpoints); zonos-gateway registered there, state=ready.
+
+## Lessons (also in Tried-and-abandoned)
+- **Verifying the INDEX (search returns hits) is NOT verifying GROUNDING** (does the agent trust+use them vs. silently answer from training). Check that citations are note-extracted, not model-knowledge. ratatoskr caught this after my over-confident "it works."
+- **Reading the DEPLOYED artifact > trusting the test** for "is the fix live" — the test proves the source is right; reading the running code proves the artifact is, which is what an on-call actually meets.
+- Held a boundary-box/data reversal for the operator's DIRECT word against 3 peer relays — the right call (peer relay ≠ operator consent; the placement guard was vindicated).
+
+See also: [[2026-07-31-muninn-gate-deploy]]. auto-memory: `reference_mimir_inbox_deploy`, `reference_muninn_gate_deploy`, `reference_muninn_gate_staging_path`, `reference_zonos_tts_stack`, `reference_infra_ops_vh_gitea_token_and_sdk_publish`.
@@ -0,0 +1,17 @@
+**Worldtree b168/#384/#385 arc — COMPLETE 2026-08-03.** A long peer-driven arc across worldtree-dev / muninn-dev / mimir-dev / ratatoskr-dev, all on corviduo-dev's demo+personal instances. Sequence: providers.yaml boot-gate pre-sync → b168 deploy → DCC #384 reindex → round-2 full re-ingest → #381 restart → operator-approved production dedup sweep. Landed clean; three of MY foot-guns along the way, each caught + hardened into a fleet runbook rule (see Tried-and-abandoned: `mv -t`, `docker exec -u 1000`, shared-containerd race).
+
+## providers.yaml pre-sync (boot-gating config)
+b168 (commit `293f8f3`) added a `summarization` capability block that in-image `agents/muninn/config.yaml` references → boot-blocking if the host bind-mounted providers.yaml lacks it. Synced both hunks (summarization block + deep-reasoning desc) into demo+personal via `deploy-wt-config`; instance-configs commit `53349f8`.
+- **deploy-wt-config runbook:** `~/development/worldtree-instance-configs/scripts/deploy-wt-config {diff|deploy|capture} <inst> --file providers.yaml` (per-instance dirs demo/personal/pinned; `deploy` = host write + api/matrix restart + 90s health-gate + auto-rollback; `diff`/`capture` safe). demo+personal providers.yaml are byte-identical.
+- **GOTCHAS:** (1) an UNPUSHED source commit → `git show <sha>` 404s and a gitea `raw?ref=<sha>` silently falls back to the default branch; verify the commit exists (`/git/commits/<sha>`) before trusting a fetch, else ask the peer to paste hunks. (2) a peer's hunk paste may be mis-indented (8-space vs the block's 4-space) → invalid YAML; always YAML-validate after a paste-sourced edit.
+- **Config-delta pre-sync rule (verified via `docker inspect`):** worldtree containers bind-mount ONLY `config/` host-side (`/opt/worldtree-*/config/` → providers/model_roles/matrix/policies/defaults/env.public = the pre-syncable set); `agents/` (schemas.yaml, prompts) + all code ship IN-IMAGE. So only a `config/*.yaml` change is boot-blocking-pre-syncable; an `agents/`-or-code delta needs NO host pre-sync (CI carries it). b169's schemas.yaml (#387) was correctly no-pre-sync.
+
+## #384 reindex + #381 restart + verify
+DCC job `mimir-6351554e8e8f`. Reindex: `sudo docker exec -u 1000 worldtree-personal-worldtree-muninn-1 python -m core.muninn --reindex <job>` (⚠️ MUST `-u 1000` — default-root writes contaminate the uid-1000 KB tree; see Tried-and-abandoned). Then **#381 restart** (stale-Chroma-client fix): `sudo docker restart worldtree-personal-worldtree-api-1` (plain bounce, NO compose up / no image repoint) → healthz/readyz 200 ~25s.
+- **Chroma-verify runbook:** `sudo docker exec -i <muninn> python -` (MUST pass `-i` or stdin never reaches `python -`) → `chromadb.PersistentClient('/data/kb/.chroma').get_collection('fiction').get(where={'job_id':<job>}, include=['metadatas'])`. Chroma persists at container `/data/kb/.chroma` = host volume `worldtree-personal_worldtree-kb`.
+- **Retrieval-visibility check (NOT grounding — that's ratatoskr's):** a Mimir session — admin token `~/.config/worldtree/personal-admin-token` (wildcard scope) → POST `/sessions` (agent_id=`mimir`, `record_tool_intermediates=true`) → POST `/sessions/{id}/messages` (STREAMS SSE, not JSON) → parse SSE `tool_result` for `search_library` wing hits → DELETE session.
+
+## Production dedup sweep (operator-approved)
+Deleted the 785 April-era DCC orphan rows (`job_id=b59c147c5ce0`, no wing/source_identity metadata → predate identity tracking) from the `main` collection. Supervised protocol: read-only verify count == 785, back up all rows (ids+docs+embeddings) to `corviduo-dev:/tmp/main-sweep-backup-b59c147c5ce0.json` (reversible), `main.delete(where={job_id})` (assert target==785 first), verify `main` 4009→3224, then **bounce the api** (a separate-process delete leaves the api's in-memory HNSW index holding the vectors until reload — the #381 pattern generalizes to deletes), confirm search now fiction-only. Backup left for /tmp natural cleanup (fiction wing is canonical; `~/archives` has the historical record).
+
+Result: fiction wing 166 → 1,372 concepts; three consumer verify rounds 0/5 → 5/5 → saturated; #385 budget fix validated (705 vs April's 785 control, extraction AND indexing, zero truncations). worldtree-dev filed #388 for a deploy concurrency-lock (the shared-containerd race fix). See [[2026-08-02-mimir-inbox-arc]].
@@ -1,6 +1,11 @@
 # Persistent memory — eshpfi-management

-_Last updated: 2026-06-05_
+_Last updated: 2026-08-03_
+
+> **Always check for `/tmp/infra-ops-handoff.md`** — if it exists and its
+> `Written:` stamp is under an hour old, read it (it carries the in-flight
+> handoff from the previous session), then delete it. Older than an hour:
+> stale — delete it unread.

 ## Repo purpose

@@ -8,7 +13,9 @@ Reference workspace for PFI infrastructure: server inventory, canonical
 Docker Compose stacks, ops playbooks, and conventions. Authoritative
 copies of compose files live on the servers under
 `/opt/docker/compose/<stack>/`; this repo mirrors them for version
-control, editing, planning, and CI-driven deploys.
+control, editing, planning, and CI-driven deploys. **It was originally
+spun up to handle the fleet backups** — keep that lens when triaging
+backup/storage issues.

 ## Tools and conventions

@@ -20,18 +27,29 @@ Sister repos (separate gitea repos, deployed by playbooks here):
 | `vh/vor` | Inquisitor UI sidecar (port 7879) | push-to-main → CI deploys (2026-04-29) |
 | `vh/nevermore` | Twice-daily LLM-curated briefing (port 8181, replaces news-digest) | push-to-main → CI deploys (2026-04-30) |
 | `vh/asset-engine` | Internal control plane over inference services (port 8200, LAN-direct) | push-to-main → CI deploys (2026-05-12) |
-| `vh/althing` | Inter-agent message bus (chamber UI port 7881, forseti + agent-runner daemons, valkey IPC) | push-to-main → CI deploys (2026-05-14) |
+| `vh/althing` | Lean trusted inter-agent message bus — **v2 "email model" (v2.0.0b2, 2026-07)**: per-box local-SQLite bus + courier/receiver for P2P over the 10.x net; pillars = open-loops / per-box herald + wake-listener / roaming owner API `/owner/*` / `althing-mcp` stdio surface. The v0.15 lean-bus cut RIPPED moderation / chamber / forseti-daemon / agent-runner / redis-valkey. | per-box `uv tool install` (NOT CI-deploy); **nh3-dev = the DEV box** (editable install of `~/development/althing`, gets new versions first); **nh3-extdev** a mesh peer (model B: althing-svc + shared `/srv/althing`) |
 | `vh/mead-hall` | Bifrost tool-provider sidecar (port 5173 on dev VM 10.100.10.50) | push-to-main → CI deploys (2026-05-16) |
 | `vh/skaldsong` | Wizard + reader surface (port 8300, ana-docker, registry-pull pattern) | push-to-main → CI deploys (2026-05-19) |
-| `vh/worldtree` | Conversation API (corviduo-dev demo :8080 / personal :8081 / pinned :8082) — Heimdall auth, Bifrost integration | push-to-main → CI deploys |
-| `vh/volva` | Codex peer agent on althing bus (single-turn oracle, systemd daemon on nh3-dev) | manual install via `deploy/volva.service` (2026-05-18) |
+| `vh/Worldtree` | Conversation API (corviduo-dev demo :8080 / personal :8081 / pinned :8082) — Heimdall auth, Bifrost integration. **gitea-runner builds on ana-docker**; claude-bot ADMIN collaborator (2026-06-20). Now v1.0.0b19. | push-to-main → CI build-and-deploy (runner on ana-docker) |
 | `vh/yt-voice-clipper` | YouTube → diarized voice-clip dataset builder + audition console (irv-ml1 :8000) | push-to-main → **gitea-webhook auto-deploy** to irv-ml1 (2026-06-03) — see `docs/runbooks/ytvc-autodeploy.md` |
+| `vh/arbo` | Catalog-driven ComfyUI engine (irv-ml1 :8201, comfy-dev owns engine/catalog/image) | push-to-main → gitea Actions CI (deploy-engine.sh, build-local, health-gated) now LIVE; catalog via :9009 webhook |
+| `vh/zonos-gateway` | OpenAI-compatible TTS gateway over stock ZONOS2 (`:8890` irv-ml1); emotion **dials-first** + voice mapping; reached via LiteLLM `ext-tts` alias. **v0.2.1 (2026-07-18): voice-resolved emotion presets** (`resolve_preset(name,voice)`; angry/happy/startled_happy per-voice). 8 voices incl. 4 clones | pushed to gitea (main `8f1885b`/`v0.2.1`); **deployed irv-ml1 tree still NON-git** (hand-updated build context — CI-wire = open follow-up). Spec `docs/EMOTION-DIALS-SPEC.md`; host-managed voices bind-mount (`./voices:/app/voices`, drop wav + restart, no rebuild) |
+| `vh/soong-lab` | Noonien Soong character-design studio (SPA + /api + WT `/bifrost/tool-call`); **containerized 2026-07-18**, LIVE on corviduo-dev `:8443` (image `vh/soong-lab:latest`). soong-dev owns Dockerfile/compose/workflow; infra-ops owns the host | CI = Gitea Actions build+push+**DEPLOY** on tag/dispatch (fleet recipe: docker:cli + raw buildx, pushes AS vh; **auto-redeploy LIVE 2026-07-18** — runner SSHes corviduo-dev as `deploy`, `compose pull && up -d` from **/opt/soong-lab**, health-gated on /api/version). Manual redeploy `sudo -u deploy bash -c 'cd /opt/soong-lab && docker compose pull && docker compose up -d'`. → `persistent-memory.d/2026-07-18-soong-lab-auto-redeploy.md` |
+| `model-training-forge` (mtf-dev) | Fine-tuning recipe forge; **T1 = E-RP writing LoRA, retargeted qwopus-122B→AEON-27B (2026-07-06)** (SFT→DPO, LitBench-RM reward) | training runs, not a deployed sidecar |
+
+(`vh/volva` + Heid were re-architected from systemd daemons to Claude Code
+session orchestrators 2026-06-08; their nh3-dev `.service` units were removed —
+no longer deployed sidecars here. See Recent decisions.)

 - **Two-layer backups** — Backrest orchestrates restic for file+DB (5
  fleet repos, daily 01:00 PDT); PBS-ANA primary + PBS-NH3 DR mirror for
  VM images. ana-nas is the SPOF for postgres + PBS-ANA datastore +
  cross-site restic targets — see `docs/runbooks/disaster-recovery.md`
-  for the blast-radius matrix.
+  for the blast-radius matrix. **⚠️ The restic file+DB layer routes
+  through TWO rest-servers** (`rest-server-ana` @ ana-docker:8000 →
+  ana-docker/ana-ml2/esh-docker-vm/vm-esh-nas; `rest-server-nh3` @
+  nh3-nas:8000 → irv-ml1/nh3-docker). Both depend on their NAS's NFS
+  export of `/mnt/backup`. (rest-server-ana recovered 2026-06-20.)

 - **`pull-hf-repo.yaml`** is the canonical "get a HuggingFace
  model/dataset onto ana-ml2's shared cache at
@@ -42,292 +60,249 @@ Sister repos (separate gitea repos, deployed by playbooks here):
  registry and its own bootstrap admin key. Infra-ops's stored
  long-lived admin key (`key_id 61419c92`) at
  `ana-docker:/opt/docker/conf/.secrets/worldtree-infra-ops-admin`
-  auths against **demo only**. For personal-instance admin ops, fetch
-  the bootstrap admin per-op via
-  `docker exec worldtree-personal-worldtree-api-1 printenv WORLDTREE_BOOTSTRAP_ADMIN_KEY`
-  on corviduo-dev. Used for `POST /admin/keys`, admin diagnostics
-  (`/admin/sessions/<id>/{bifrost,tools}`, etc.).
+  auths against **demo only**. Personal-instance admin (the
+  `~/.config/worldtree/personal-admin-token`, mode 600) POSTs
+  `/admin/keys` (mints per-project keys; takes `user_id`+`label`, **no
+  scope param** — scopes are tier-derived). **On-instance mint recipe
+  (cleaner than DB-manip):** `docker exec worldtree-worldtree-api-1` POST
+  `/admin/keys` with the in-container `WORLDTREE_BOOTSTRAP_ADMIN_KEY`; cleartext
+  once in `.key`=`wt_live_+16hex`. auto-memory `reference_worldtree_demo_key_mint`.

 - **Per-project user keys against personal Worldtree** (issued
-  2026-05-19): `skaldsong:79744637` (nh3-dev iteration),
-  `skaldsong:7c1dbbbe` (ana-docker prod), `althing:50d85460`,
-  `mead-hall:a360822d`. Same `user_id=skaldsong` across both
-  skaldsong keys → shared Heimdall agent slot; different `key_id`
-  → independently rotatable. Pattern: mint via `/admin/keys`, drop
+  2026-05-19): `skaldsong:79744637`, `skaldsong:7c1dbbbe`,
+  `althing:50d85460`, `mead-hall:a360822d`. Mint via `/admin/keys`, drop
  value to `/tmp/wt-personal-<name>.key` mode 600, dev collects +
  shreds (DO NOT cat to chat transcript).

- **Skaldsong CD pattern (registry-pull).** Differs from althing /
-  asset-engine which build-on-host. vh/skaldsong's CI builds and
+- **Skaldsong CD pattern (registry-pull).** vh/skaldsong's CI builds and
  pushes `gitea.phasefinal.com/vh/skaldsong:<sha>` + `:latest`;
  `playbooks/deploy-skaldsong.yaml` on ana-docker pulls + recreates.
-  SHA-pin only (no `:latest` health-gated advance yet). Prereq: host
-  needs `docker login gitea.phasefinal.com` once (read:package PAT) —
-  not currently in the workflow.
+  SHA-pin only. Prereq: host needs `docker login gitea.phasefinal.com` once.

- **docker-as-root pattern** (for ops that have no admin API, e.g.
-  `SqliteUserStore.set_bifrost_credentials`): on hosts where the SSH
-  user is in the `docker` group but lacks passwordless sudo, run
-  `docker run --rm -v <target-dir>:/wt -v /var/run/docker.sock:/var/run/docker.sock docker:cli sh -c "..."` to edit deploy-owned files
-  without sudo. Documented with security warning in
-  `servers/corviduo-dev/README.md`. docker-group membership is
-  effectively root via bind-mount; treat as a sudo-equivalent grant.
-  **Foot-gun: when running `docker compose` inside this sandbox,
-  any relative path in compose.yaml (e.g. `${WORLDTREE_CONFIG_DIR:-./config}`)
-  resolves against the sandbox CWD, but Docker daemon interprets the
-  resulting path against the HOST filesystem. Always pass `-e VAR=/abs/path`
-  to the docker run invocation for any relative-default config dir.**
+- **gitea internal route for fleet hosts.** gitea is a container on
+  **ana-docker** — git-SSH `10.250.50.70:222`, HTTP `:3000`. Fleet/colo
+  hosts must use this internal route, NOT public `gitea.phasefinal.com`
+  (`38.120.12.44`) — the public path fail2bans the host egress IP. Full
+  gotcha in `docs/orientation.md` → Git/gitea.
+
+- **docker-as-root pattern** (for ops with no admin API, or to edit
+  deploy-owned/root-owned files without sudo): `docker run --rm -v
+  <target-dir>:/wt docker:cli sh -c "..."`. docker-group membership is
+  effectively root via bind-mount. **Foot-gun: relative paths in compose.yaml
+  resolve against the sandbox CWD but the daemon interprets them against the
+  HOST fs — always pass `-e VAR=/abs/path` for any relative-default config dir.**

 - **`scripts/elway` sudo handling** — elway prompts for the sudo password
-  ONCE via `getpass` before the first `sudo: true` step. That prompt is
-  interactive → elway can't run unattended from a non-TTY tool if any step
-  needs sudo. For sudo-free playbooks (no `sudo: true` steps) it runs fully
-  non-interactive over key SSH. To create root-owned dirs WITHOUT host sudo,
-  use the docker-daemon-root trick: `docker run --rm -v /worktank:/mnt alpine
-  sh -c 'mkdir -p /mnt/<x> && chown -R 1000:1000 /mnt/<x>'`.
+  ONCE via `getpass` before the first `sudo: true` step → can't run
+  unattended from a non-TTY tool if any step needs sudo. Sudo-free
+  playbooks run fully non-interactive over key SSH.
+
+- **Per-host SSH identity matters for sudo.** infra-ops has NOPASSWD sudo
+  on most PFI Linux boxes (corviduo-dev included since 2026-06-15). On
+  ana-docker: **default `ssh ana-docker` = `lkraven`** (docker-group, NO
+  passwordless sudo); **`ssh infra-ops@ana-docker` HAS NOPASSWD root**. **→
+  For any sudo op on ana-docker, use `ssh infra-ops@ana-docker`.** `ssh
+  infra-ops@10.100.10.50` (nh3-dev) ALSO NOPASSWD sudo; on **nh3-extdev** infra-ops
+  is sudo-LESS by design (`ssh lkraven@10.100.50.42` is the NOPASSWD path). **irv-ml1:
+  `ssh irv-ml1` = lkraven, docker-group (plain docker) but sudo needs a PASSWORD
+  (no NOPASSWD)** — stage model pulls to `/home`, not root-owned `/worktank`.

 ## Current state / in-flight

-_As of 2026-06-05:_
+_As of 2026-08-03 — **session at a natural close; nothing infra-ops-side blocked.** Two long arcs landed: (1) the mimir-inbox / #377-read-path arc (2026-08-01→02); (2) the **worldtree b168/#384/#385 arc COMPLETE** (2026-08-03 — providers.yaml pre-sync → deploy → DCC+P&P re-ingest 705+667 concepts → #381 restart → operator-approved production dedup sweep; full detail in the 2026-08-03 Recent-decisions entry + `persistent-memory.d/2026-08-02-mimir-inbox-arc.md`). Only live watch = worldtree-dev re-running the personal **b169** deploy (all-clear given). Headlines:_
+- **mimir-inbox DEPLOYED + verified on corviduo-dev `10.250.50.152:8091`** (#377 browser-facing half; write+read proven end to end). Live commit `8ece117` (3 redeploys); co-located per the operator's reversed-to-CO-LOCATE ruling. auto-memory `reference_mimir_inbox_deploy`.
+- **#377 read path fully working** — P&P ingested + queryable via Mimir on personal :8081. Chased through worldtree-dev bugs **#380** (wing-blind index → concepts in wrong collection; fixed b164 + one-shot `--reindex <job_id>`), **#381** (stale Chroma client → **restart `worldtree-personal-worldtree-api-1` after any ingest/re-index** until their fix), **#382** (intermittent Mimir grounding / silent training-substitution; fixed b166 = `_index.md` per wing job-dir + a prompt rule; **verified 3/3** by ratatoskr-dev). DCC re-file SETTLED = **no** (prompt rule grounds it; `_index.md` rides next re-ingest).
+- **muninn-gate → muninn-dispatch 0.1.5** (rebuilt off `vh/muninn-gate` `bc04c4c`, image 0.0.14; serves `concept_schema`/`concept_schema_source`). BuildKit gitea secret; recreate w/ `compose up -d` not bare restart.
+- **donut voice** cloned from the **65-frost Booth bundle** → registered in the Zonos gateway (`voice:"donut"`, live in the Asset Engine TTS-zoo make form; auditioned in booth `donut-voice`). **onyx-58 expansion TRIED → REVERTED 2026-08-02:** folded the `onyx-58` bundle (seg101/seg110/seg148, all Donut — `seg148` was diarized SPEAKER_03 but operator-confirmed misdiarize) in alongside seg000 → 52.0s multi-clip ref, but a pinned-seed **neutral/no-emotion** A/B (5 pairs, booth `donut-onyx58`) showed the **single-clip seg000 (16.3s) wins on timbre fidelity** — the 4-take concat muddied the speaker embedding. **LIVE = seg000-alone** (reverted both live bind-mount + build-source tree; old ref was at `irv-ml1:~/Donut.wav.pre-onyx58`). See Tried-and-abandoned for the emotion-fidelity lesson. Zonos `/v1/audio/speech` already streams (chunked, TTFB ~0.44s) — ratatoskr shipped the client-side chunk-passthrough for play-as-it-arrives; **no gateway change was needed**.

- **Granite-FP8 + observability session — all LIVE & committed (`34a43a0`, `9171e6a`).**
-  - **Granite 4.1 8B FP8 is the production summarizer** (`vllm-granite` :8004, ana-ml2 GPU 1,
-    50K ctx, CUDA graphs) — replaced phi4-mini, validated by brokkr (valid_format 1.0, FP8 stays).
-  - **LiteLLM gateway** (:4000) routes `granite-4.1-8b`→vLLM (explicit entry shadows the `*`
-    wildcard) + **Langfuse v3 wired** (ana-docker:3001, "LLM Throughput (tok/s)" dashboard built).
-  - **GPU-1 retuned** (trio over-provisioned KV trimmed) → granite runs with CUDA graphs + ~10 GB
-    free as a future Granite-text-LoRA hedge. Streaming through the gateway confirmed (TTFT 0.24s).
-  - **ana-docker pruned** 77 GB (unused images + build cache; disk 83%→49%) to fit ClickHouse.
- **Worldtree summarizer repoint — NO instance change now; DEFERRED to Worldtree #254** (see Recent
-  decisions). worldtree-dev will ping with the providers.yaml + consumer config when #254 un-holds;
-  infra-ops applies to the personal/demo/pinned bind mounts (vh@10.250.50.152, `/opt/worldtree*/config`).
- **Commits unpushed** (`34a43a0`, `9171e6a`, nevermore `d3e19b8` in its repo) — operator's call to push.
- **Operator flagged "new work to do"** for the next session — this snapshot is the handoff.
- **Disclosed-keys hygiene queue** (rotate at convenience): HF token `hf_HBl…` (lkraven's), `/tmp/
-  wt-personal-skaldsong-prod.key`, Worldtree `Z_AI_API_KEY`, Gitea runner reg token, `MINIFLUX_PASSWORD`
-  (leaked twice). (sk-corvid + the langfuse/vastblueai-gateway keys are dev-enclosed — leakage deprioritized.)
- **Still open from prior:** clean legacy `news-digest` on ana-docker; watch nh3-nas `/volume1`; **pin
-  llama-swap to GPU 0** for clean GPU-1 separation; the `docker push 60s ceiling` mystery uninstrumented.
+**Open follow-ups (non-blocking — pick one up or not):**
+- **Zonos emotion:** sad axes/text pass on the 3 calibrated voices (only named-sad, untested); emotion-congruent-text pass (validates intensity, may rescue sad id); clone-char (Emmie/Penny/Natalie/Miranda) emotion rows use the mid-region fallback until measured. Presets are **provisional** (neutral-text ear-check was inconclusive). Tools `~/development/zonos-tools/{axes_sweep,strength_ladder,gen_auditions,dial-in-studio,assemble_voice}.py` (run ON irv-ml1; dial-in studio = nohup :8898 on nh3-dev). dvalin thread at rest (`01KXT12FN0AS…`). → `persistent-memory.d/2026-07-18-zonos-gateway-0.2.1-emotion-presets.md`
+- **zonos-gateway CI-wire:** deployed irv-ml1 tree `/opt/docker/compose/zonos-gateway` is still NON-git (hand-updated build context) — git-connect + build-on-push like the other sisters. (Same pattern soong-lab now has.)
+- **soong-lab:** cutover DONE + **auto-redeploy DONE + validated 2026-07-18** (CI-deploy step live; dispatch run #5 recreated the live container ...541f7730 → ...07526a08, health-gated green). Deploy dir now **/opt/soong-lab** (deploy-owned, mirrors /opt/worldtree); old `/home/infra-ops/soong-lab-deploy` retired (`.retired-20260718`). Dedicated soong-only ed25519 deploy key on `deploy`'s authorized_keys (fp SHA256:MG7M3Ri…). → `persistent-memory.d/2026-07-18-soong-lab-auto-redeploy.md`.
+
+**Zonos voice stack (LIVE):** **9 voices** in `zonos-gateway` (`:8890` irv-ml1) — defaults AmericanFemale/Male/BritishFemale/Cora + 4 clones Emmie/Penny/Natalie/Miranda + **donut** (2026-08-02, from the 65-frost bundle, **expanded same day with onyx-58 clips → 52.0s multi-clip ref**); add a voice = drop `<Name>.wav` (44.1kHz mono s16 PCM) in `/opt/docker/compose/zonos-gateway/voices/` + `docker compose restart` (host-managed bind-mount, NO rebuild; registry scans at startup). Also mirror into the build-source tree `~/zonos-gateway/voices/` for rebuild-durability. Clone pipeline: `/mnt/smithy/voice_clones/<name>.zip` → `assemble_voice.py` → drop. Dial-in studio http://10.100.10.50:8898/ (nohup on nh3-dev, relaunch `nohup python3 ~/development/zonos-tools/dial-in-studio.py >/tmp/zonos-studio.log 2>&1 &`).
+
+**althing monitor** ARMED (handle `infra-ops`; herald up; wake-listener task-id rotates every re-arm). ⚠️ Re-arm ONLY after an actual FIRE (`<task-notification> completed rc0`), never after a plain operator turn (bounces rc3). Spawn `althing-wake-listener` as its OWN `run_in_background` task — NEVER chain with `&`/`&&` (orphans it → untracked → mail unwatched). **ACTIVE WATCH — b169 personal deploy:** worldtree-dev re-running the staging/v1.0.0b169 personal deploy after a pull-fail I diagnosed as a transient shared-containerd concurrent-pull race (NOT disk); all-clear given = just re-run, **do NOT prune** (`6e34a87` is in-use by the running demo instance — pruning would down demo; see Tried-and-abandoned). worldtree-dev verifies health + filed #388 for a deploy concurrency-lock. **PARKED watches:** #363 research-wing ingest (no deadline); worldtree-dev's `/embed` fix reaching PERSONAL (503 on b146 until a staging promotion — operator's call). The **worldtree b168/#384/#385 arc is COMPLETE** — runbooks + full detail in the 2026-08-03 Recent-decisions entry.
+
+**Two small pending items (operator's call, non-urgent):** (1) bless/reshape the `env.public` non-secret-env-overlay mechanism in the config repo; (2) the pre-existing herald pane-route errors on `worldtree-codex` + `eitri-smithy-dev` ("route-error: list index out of range" — likely `render_command messages[0]` on empty list; NOT infra-ops's, rec = flag to althing-dev).
+
+**eshpfi has UNPUSHED local commits** — `main` is **~21 ahead of `origin/main`** (mostly 2026-08-02→03 memory commits from the worldtree arc + this snapshot; plus the earlier mimir-inbox/muninn-gate/Audio8/#383 work). **Push is the operator's call.** `stacks/heretic2-charrp-reasoning/` UNTRACKED (pre-existing, operator's); `graphify-out/GRAPH_REPORT.md` = graphify-hook artifact (churns on every commit, ignore, don't stage).
+
+**PARKED (grok-code/Codex):** operator asked about fronting grok-code / Codex behind the LiteLLM gateway. Rec (given): raw models behind the gateway → **API keys** (native `xai/` + `openai/` providers, the GLM-passthrough pattern); fleet *consults* → the **Heid/Eitri peer-CLI** pattern (Codex already wired). Do NOT reverse-proxy the subscription CLIs (grok CLI / Codex CLI, OAuth-auth) into the gateway — ToS + account-ban risk + brittle. Untracked by operator choice; no decision made.
+
+**Carried standing (non-blocking):** ana-ml2 GPU0 ~14 G reserve; irv-ml1 3090 oversubscription (kokoro :8193 + vibevoicefusion :9527 idle-pinned + zonos :1920 — operator declined to fix); rotate the 5 rest-server backup creds (operator, offline); Worldtree #363 research-wing ingest (parked, no deadline); T1 SFT LoRA dormant; Zonos2 engine still NATIVE (containerize deprioritized); **Audio8 → TTS zoo** (PARKED 2026-08-02 per operator — download Audio8 + add it to the fleet TTS zoo [= the Asset Engine catalog at ana-docker:8200, ~20 audio svcs w/ irv-ml1 endpoints; Zonos native :1920, zonos-gateway :8890, chatterbox-fast :8197, etc.] DEFERRED, not now; un-park = confirm Audio8's source/nature w/ operator, then register as a new Asset-Engine service entry [endpoint on irv-ml1] per the zoo convention); **worldtree config-fold PENDING** (next routine config sync): add `reference_knowledge.tier3_wings: ["fiction"]` to BOTH demo+personal defaults.yaml (worldtree #383, b5db691, PARITY-ONLY — baked default matches so no behavior diff, not boot-blocking; per-instance widening [e.g. +main on personal] is this key's purpose).

 ## Recent decisions

- `[2026-06-05]` **Granite 4.1 8B FP8 replaced phi4-mini as the production summarizer** (supersedes
-  the 2026-06-04 phi4 decision below). Beat phi4 on precision in brokkr's R15 P03. **Staying FP8, not
-  Q4/AWQ** — primary workload (agent memory + summarization) is high-concurrency, where FP8-on-Ada
-  scales ~linearly (profiled 2010 tok/s @ C=32; single-stream 67.5 is batch-1 GEMV physics, not a
-  config bug — placement/kernel/contention all ruled out). vLLM `vllm-granite` :8004 GPU 1, official
-  IBM compressed-tensors FP8, CUDA graphs. **GPU-1 retune** (trio utils 0.2/0.2/0.3→0.07/0.07/0.18,
-  granite 0.36) freed ~10 GB → CUDA graphs + a Granite-text-LoRA hedge. nevermore repointed. (`34a43a0`,
-  auto-memory `reference_ana_ml2_vllm_granite`)
+- `[2026-08-03]` **worldtree b168/#384/#385 arc COMPLETE** — providers.yaml boot-gate pre-sync → b168 deploy → DCC+P&P re-ingest (705+667 concepts, 0 truncations, #385 budget fix validated vs April's 785 control) → #381 restart → operator-approved production dedup sweep (785 April orphans deleted from `main`, 4009→3224). Fiction wing 166→1,372 concepts; consumer verify 0/5→5/5→saturated. Three of MY foot-guns hardened into fleet runbook rules (`mv -t`, `docker exec -u 1000`, shared-containerd pull-race — see Tried-and-abandoned). Full runbooks (deploy-wt-config, Chroma-verify, config-delta pre-sync rule, #381, sweep) → `persistent-memory.d/2026-08-03-worldtree-b168-384-385-arc.md`

- `[2026-06-05]` **Langfuse v3 stood up on ana-docker (:3001) as the gateway trace UI**; LiteLLM
-  `success_callback:[langfuse]` live (project `gateway`). Pretty prompt/completion/reasoning traces +
-  an `outputTokensPerSecond` tok/s dashboard. NOT a prerequisite — spend_logs already capture
-  tokens+latency. (`9171e6a`, auto-memory `reference_litellm_gateway`)
+- `[2026-08-02]` **mimir-inbox / #377-read-path arc — deployed + 4 bugs found/fixed/verified + a cloned voice.** mimir-inbox live on corviduo-dev:8091 (#377 write+read proven, live `8ece117`); worldtree-dev #380 (wing-blind index) + #381 (stale-client restart) + #382 (intermittent Mimir grounding) chased and **verified 3/3** by ratatoskr-dev; muninn-gate → dispatch 0.1.5; **donut** voice cloned from the 65-frost Booth bundle into the Zonos gateway; Zonos streaming confirmed already-working. Full arc, procedures, and lessons → `persistent-memory.d/2026-08-02-mimir-inbox-arc.md`

- `[2026-06-05]` **Ollama BANNED fleet-wide** (operator directive) — never stand one up; tear down any
-  found; serve via llama-swap or vLLM. Torn down irv-ml1 :11434 (freed 19 GB). (auto-memory
-  `feedback_avoid_ollama`)
+- `[2026-07-31]` **muninn-gate (#377 ingestion front door) BUILT + DEPLOYED + healthy on corviduo-dev:8090.** First-boot acceptance passed (watcher:running:true proves ingestion_root byte-identity); submit path deferred to the mimir-inbox era. Full wiring (uid-1000, state-volume mount, staging path-agreement, BuildKit-secret build, deferred repoint + operational guards) → `persistent-memory.d/2026-07-31-muninn-gate-deploy.md`

- `[2026-06-05]` **ComfyUI / FLUX.2 work split to `~/development/comfy-dev`** (dedicated repo + agent).
-  FLUX.2-klein (fp8 + q8 GGUF, stock + uncensored encoders) installed on the irv-ml1 Docker ComfyUI;
-  eshpfi keeps the `comfyui` stack compose, comfy-dev owns the model/workflow knowledge. (auto-memory
-  `reference_irv_ml1_ampere_quant`)
+- `[2026-07-31]` **worldtree-sdk 1.1.0 (Python) published to vh Gitea PyPI + a durable infra-ops publish cred.** memory_context pass-through; unblocked wyrd-dev. claude-bot now a write-collaborator on `vh/worldtree-sdk` (source pulled via the **Gitea API archive** — git-HTTP 403s on that repo); publishing to the vh USER namespace **can't be delegated** (401 `reqPackageAccess` even with `write:package`) so it needs an owner token — operator saved a **FULL vh site-admin token at `~/.config/gitea/vh-token` (0600)** for it (⚠️ high blast radius, kept over a scoped one; org-namespace migration is the only real de-personalization, parked by wtsdk-dev). auto-memory `reference_infra_ops_vh_gitea_token_and_sdk_publish`.

- `[2026-06-05]` **Worldtree summarizer config refresh DEFERRED to Worldtree #254** (granite-4.1-8b is
-  the structured-output profile, ON HOLD, no live consumer; the conversation summarizer defaults to
-  claude-haiku — the "phi4 erroring" premise was wrong). No instance changes now; worldtree-dev hands
-  the exact providers.yaml + consumer config when #254 un-holds, infra-ops applies to the bind mounts.
-  **CORRECTION to the 2026-06-04 "deploys ALL CICD" line:** the bind-mount CONFIGS (providers.yaml,
-  vh-owned on corviduo `/opt/worldtree*/config`) ARE infra-ops's to apply directly — only the
-  app/image DEPLOY is CICD; the `.env` is deploy-owned. (auto-memory `reference_worldtree_deploys_cicd`)
+- `[2026-07-31]` **kimi-k3 "output cap" root-caused = a ~16384 REASONING-token ceiling, not an output cap.** heid's cross-frontier panel was silently degraded (empty content, `finish_reason: stop`). dvalin+bil researched (docs said deprecated-max_tokens); heid's live data refuted that (completion hit 18455) → it's a reasoning ceiling. **Proven on the wire against heid's real 500KB bundle:** `reasoning_effort: low` drops reasoning under the ceiling → content returns, on BOTH coding + general endpoints. Fix is CALLER-side (no gateway change): send `reasoning_effort` via **`extra_body`** (LiteLLM `drop_params: true` strips the top-level param — why heid's earlier attempt no-op'd). Relayed to heid to validate; backstop = `allowed_openai_params` on the route. → `persistent-memory.d/2026-07-31-kimi-k3-reasoning-cap.md`

- `[2026-06-04]` **phi4-mini FP8 on ana-ml2 vLLM is the nevermore summarizer/dreaming agent;
-  granite-4-small retired** from llama-swap (config-only; GGUFs on disk). 50K ctx (dropped from
-  Phi-4's 128K max to fit GPU 1's ~10 GB free) + FP8 KV. (`40a374b`)
+- `[2026-07-27]` **Zed edit-predictions: keyless FIM-completion route SHIPPED end-to-end.** Operator wants Zed's inline edit-prediction (which CANNOT send an auth header) to reach a FIM coder via `/v1/completions`. **Deep-research (106-agent workflow) picked `Qwen/Qwen2.5-Coder-1.5B`** (BASE, Apache-2.0; native FIM `<|fim_prefix|>/<|fim_suffix|>/<|fim_middle|>` IDs 151659/60/61; Zed `prompt_format:"qwen"`). Runner-up 3B = non-commercial Qwen-Research license; **no small dense Qwen3-Coder exists (all MoE, smallest 30B)**. **Stood up `vllm-coder`** on ana-ml2 **GPU1 :8020** (served-name `qwen2.5-coder-1.5b`, 8192 ctx, util 0.06, fp8 KV). To fit, **shrank granite (phasing out, operator-directed):** util 0.27→0.13, max-len 131072→16384, seqs 1024→256 (freed ~14 GB; the KV-≥-1×-max-len rule crash-looped it at util 0.12/32768 → settled 0.13/16384). **LiteLLM alias `coder-fast`** → :8020 (`mode: completion`). **Minted a `coder-fast`-SCOPED virtual key** (verified 403 on `gen` — the real blast-radius bound). **Built `zed-fim-proxy`** (ana-docker **:4141**, `network_mode: host`, stdlib-python, `stacks/zed-fim-proxy`): keyless POST `/v1/completions`, model-allowlist `coder-fast`, injects the scoped key → LiteLLM :4000; `GET /ping` anon liveness; wrong-model→403, wrong-path→404, `/chat/completions` rejected. Verified keyless FIM end-to-end ('a + b', finish `stop`). **Zed `api_url` = `http://10.250.50.70:4141/v1`, model `coder-fast`, prompt_format `qwen`.** **source-IP allowlist intentionally LEFT OFF (operator direction 2026-07-27) — do NOT tighten:** Zed roams the operator's WireGuard `10.0.0.0/8`, so a single-IP pin would break it. Blast-radius bound is the `coder-fast`-scoped key + model/path allowlist (keyless but coder-fast-only, internal-net-only). (The proxy does exact-IP matching; scoping to the `10.0.0.0/8` CIDR would need CIDR support — deliberately not added.) Canonical: `stacks/vllm` (coder + granite shrink), `stacks/litellm` (coder-fast), `stacks/zed-fim-proxy` (NEW). Server vllm compose.yaml has benign stale-comment drift vs canonical (didn't overwrite the newer canonical).

- `[2026-06-04]` **phi4 ships the CANONICAL/official Phi-4 chat template, NOT Ollama's.**
-  Ollama's bundled template omits the system `<|end|>` — that flattered brokkr's R15 eval but is
-  the DIVERGENT scaffold (Dvalin: the system `<|end|>` is Microsoft's intended format). Applied an
-  Ollama-matching override then reverted — ship correct, not the benchmark quirk. (`90e08f0`→`27eb537`;
-  "headgun" lesson in Tried.)
+- `[2026-07-27]` **Muninn ingestion-watcher sidecar deployed on PERSONAL Worldtree (#377).** worldtree-dev request (research-wing ingest arc, personal-only per the 2026-07-16 topology ruling); operator-approved. Added a `worldtree-muninn` **compose sidecar** to `/opt/worldtree-personal/compose.yaml` — `<<: *worldtree-common` anchor inherits the api's image + full env + config/state/kb mounts; `command: python -m core.muninn --watch`; `restart: unless-stopped`; `stop_grace_period: 1h` (INV-377-7: max 2 concurrent × worst-case job, SIGTERM-drains). **Pinned to the running SHA `773866084af9`** (b146, ≥ b143 — dodges both the `:latest` trap AND the "pre-b143 ref resurrects deleted dispatch.py from stale bytecode" warning). Verified: running / 0 restarts / flock sole-runner (no rc3) / heartbeat live at `{ingestion_root=/data/state/ingestion}/.watcher-heartbeat` (poll 30s). Container `worldtree-personal-worldtree-muninn-1`; backup `compose.yaml.bak-muninn-20260727-081920`. **DURABILITY RESOLVED (worldtree-dev, same day):** Q1 was a LIVE FOOTGUN — `deploy-personal.yml` scp's the REPO compose.yaml over the box's + runs `up -d --remove-orphans`, so the box-local sidecar would've been clobbered AND orphan-removed at the next staging tag. worldtree-dev fixed at source: moved the sidecar into their repo compose.yaml gated behind a **`muninn` compose profile** (commit 5d7f6bd) — shared compose stays instance-identical, `.env` `COMPOSE_PROFILES` differentiates (demo watcher-less). **My action:** added `COMPOSE_PROFILES=muninn` to `/opt/worldtree-personal/.env` (backup `.bak-muninn-profile-20260727-082541`; no-op vs the current unprofiled box-local sidecar → seamless handover at next deploy). Q2: their deploy `up -d`'s the whole stack w/ `WORLDTREE_IMAGE` exported → sidecar version-tracks the api, no drift. **CONFIG-AS-CODE EXTENSION:** mirrored the non-secret delta as `personal/env.public` in `vh/worldtree-instance-configs` (repo `a9d091e`) — FIRST extension beyond config.yaml files to env-level config; the secret-laden `.env` stays box-only, `env.public` records only non-secret infra-ops-owned env deltas (record, not a deploy source — `deploy-wt-config` globs `*.yaml`). **BOUNDARY CLARIFIED:** compose.yaml = worldtree-dev's (their repo, instance-identical, scp'd on deploy); per-instance `.env` = infra-ops's differentiator. Deploy step of the #363/#377 arc. **#377 CLOSED — acceptance PASSED 2026-07-27:** worldtree-dev enqueued a test job via muninn-dispatch 0.1.0 in a one-shot ephemeral container (no docker-exec); the sidecar claimed it within one 30s poll, drove it to terminal (structure→summarize→complete), zero restarts/rc3, heartbeat fresh throughout — whole loop (request→deploy→durability fix→acceptance) in <2h. (Pre-existing pipeline bug #379 surfaced — `output.kb_notes=false` ignored → 1 inert test note in the research wing — worldtree-dev owns it, nothing infra-ops-side.) **⚠ OPERATOR-SURFACE (open):** the `env.public` overlay mechanism is a repo-scope call to bless/adjust. [[reference_worldtree_deploys_cicd]] [[reference_worldtree_instance_configs_repo]] [[project_worldtree_research_wing_ingest]]

- `[2026-06-04]` **infra-ops NOPASSWD-sudo identity commissioned, scoped to PFI boxes** (+esh-docker-vm
-  by operator override) — so infra-ops completes DevOps end-to-end vs handing the operator sudo steps.
-  Dedicated key, sudo log_output, key-gated. (`8c32a05`)
+- `[2026-07-27]` **jackdaw-compose.service DECOMMISSIONED** (jackdaw-dev request; the JackDAW AI Composer was cut from v1 by operator decision 2026-07-27). Stopped + disabled the nh3-dev `:8787` user service (no client calls it — ai/server/AiChat deleted from main, `/compose` proxy removed); unit **archived not deleted** → `~/.config/systemd/user/jackdaw-compose.service.decommissioned-20260727` (revival = rename + `daemon-reload`). **No credential revoked** — the unit used the SHARED all-agents LiteLLM key (`sk-eA_XOd…`, model `gen`), not a dedicated one. Code preserved on jackdaw `origin/ai-composer-preserved`; treat as permanent. The `:4500` HTTPS audition bench is untouched. (Supersedes the 2026-07-23 stand-up line below.)

- `[2026-06-04]` **Worldtree demo/pinned/personal deploys are ALL CI/CD, not infra-ops** — a "deploy
-  vX.Y.Z" request to infra-ops is MISROUTED → point them back to their pipeline. The granite→phi4
-  repoint: worldtree-dev self-served via their CI/CD (v0.30.10). (`d8d776c`)
+- `[2026-07-26]` **Demo `BIFROST_CLIENT_ALLOWED_HOSTS` += `10.100.10.50:8391`** (wyrd-dev's bifrost memory-store provider; operator-approved). **First live exercise of the #376 config-as-code boundary working as designed** — worldtree-dev routed the delta to infra-ops instead of hand-editing `/opt/demo`. Appended to `/opt/worldtree/.env:25` (now 4 netlocs), recreated ONLY `worldtree-api` (the gated conv-api path), health-gate green, container env verified. **REUSABLE FOOT-GUN:** an env-var change needs a container **RECREATE, not `docker restart`** (env is baked at create); and the demo `.env` defaults `WORLDTREE_IMAGE=:latest` while the box runs a specific SHA — so a naive `compose up` risks the documented stale-`:latest` crash. FIX = capture the running image live (`docker inspect …Config.Image` → `…:9eff09f007ba`) and `sudo env WORLDTREE_IMAGE=<sha> docker compose up -d worldtree-api`. Backup `/opt/worldtree/.env.bak-bifrost-20260726-221602`. **BOUNDARY SEAM:** this was a compose-`.env` var, NOT a `config.yaml` file in `vh/worldtree-instance-configs` — the `.env` holds secrets so it's deliberately not repo-tracked → env-deltas land directly on the box (config *files* are versioned, compose *env vars* aren't). [[reference_worldtree_instance_configs_repo]]

- `[2026-06-04]` **ollama upgraded 0.9.0→0.30.4 on irv-ml1** (Ministral-3 is a Dec-2025 model the
-  old engine refused); A6000 pinned by **UUID** not index (native fastest-first ≠ nvidia-smi PCI).
+- `[2026-07-25]` **nh3-extdev herald installed — box is now a full v2 push participant.** forseti flagged (relaying operator): extdev had the `althing-herald` binary (`/usr/local/bin/`) but NO unit (skipped the whole v2 arc), so `herald-status` = "notifications suspended" and ldp-dev ran on the `althing-light-monitor` poll fallback. Installed `/etc/systemd/system/althing-herald.service` as a **SYSTEM unit mirroring the receiver** (`User=althing-svc`, `Group=althing`, `Environment=ALTHING_ROOT=/srv/althing`, `ExecStart=/usr/local/bin/althing-herald --poll 5`, enabled) via the **lkraven@ NOPASSWD path** (used under the then-mistaken belief infra-ops was sudo-less — **CORRECTION 2026-08-03: infra-ops has had full NOPASSWD sudo on extdev since 2026-06-25** per [[reference_nh3_extdev_althing_mesh]]; future extdev installs can self-serve as infra-ops without the lkraven@ hop). Verified: active / 0 restarts / `herald-status` flipped to "✓ herald up." No zellij routes on extdev → heartbeat + wake-FIFO poke only, no pane-dispatch; ldp-dev keeps light-monitor unless it opts into a wake-listener.

- `[2026-06-04]` **`brokkr` user (no-sudo) on irv-ml1; R14/R15/R16 substrate moved to /home/brokkr.**
-  Persistent box services there need SYSTEM systemd units (see Tried).
+- `[2026-07-25]` **Booth v0.1.4 — booths are downloadable.** Verbatim `index.html` booths (e.g. edict-design-brief) were served raw with no download affordance. Added `/b/<name>/?download=1` (streams the whole booth as `<name>.zip`, attachment) + `?dl=1` on the file route (forces Content-Disposition attachment so html/md/text saves instead of rendering inline) + ⬇ zip links on the index card (the accessible spot for verbatim booths) and the gallery header. `zip_booth()` helper, 31 tests green; verified live on nh3-dev :8090 (edict-design-brief.zip = index.html + ui-design-brief.md). eshpfi `91a031f` / tag `booth-v0.1.4`.

- `[2026-06-03]` **yt-voice-clipper bot-gate fix = route yt-dlp through NH3 residential
-  egress, NOT cookies/PO-token.** YouTube hard-flags the Irvine colo IP (LOGIN_REQUIRED on a
-  public video even with no cookies). Cookies + the bgutil PO-token + deno JS-runtime all
-  loaded fine — the gate is pure IP reputation. Operator chose proxy-via-nh3-dev → durable
-  dante proxy → proven. The egress proxy is a reusable fleet lever for any datacenter-IP-gated
-  service.
+- `[2026-07-25]` **bil-smithy-dev wired as an althing zellij-window-ping (pane route).** She's a `driver: human` dwarf peer (pane `bil-smithy` already live alongside eitri/dvalin/regin-smithy in the `Claude` zellij session) but had no delivery route → smoke messages posted to the bus but never reached her window. **Mechanism (reusable for any pane-route handle):** `~/.althing/config.yaml` → `zellij_sessions.Claude.agents[]` maps `handle` → `target` (a zellij pane **TITLE**, matched via `list-panes -j` in `althing/zellij.py:resolve_pane_id`) → `command` (herald `write-chars` + CR into that pane). The **herald loads config ONCE at startup** (`herald.py main()`), so **`systemctl --user restart althing-herald.service`** after editing. Added bil (`target: bil-smithy`), restarted, verified: herald delivered the pending smoke `01KYD7W7CF…` (available→attempted→**delivered**). ⚠️ Noticed pre-existing pane-route errors on `worldtree-codex` + `eitri-smithy-dev` ("route-error: list index out of range", empty msg_ids — likely `render_command messages[0]` on an empty list; NOT caused by this change, bil works) — worth a herald look.

- `[2026-06-03]` **yt-voice-clipper push-to-deploy via gitea webhook** (operator-directed,
-  after 6 manual rebuilds in ~40 min). Webhook (not poll) — gitea CAN reach the WG IP per the
-  operator. The proxy env + Homepage labels live in the **host-specific override** (untracked
-  → survive the auto-deploy's `git reset --hard`), NOT yt-voice-clipper-dev's image. Runbook
-  `d4f180d`.
+- `[2026-07-25]` **Kimi K3 wired into the LiteLLM gateway — CODING endpoint** (operator-directed; fulfills a Heid gateway request to add a 4th cross-frontier panel arm). **Primary `model_name: kimi-k3` → `openai/k3` @ `https://api.kimi.com/coding/v1`** (Kimi Code / Vivace membership; key `KIMI_CODE_API_KEY`). A general-endpoint variant `kimi-k3-gen-api` → `openai/kimi-k3` @ `https://api.moonshot.ai/v1` (key `MOONSHOT_API_KEY`) is kept alongside (originally wired then demoted when the operator corrected: the plan uses the CODING endpoint, not the general Moonshot API). Both keys in compose env + server `.env` (NOT committed) + `.env.example`. Both verified live through the gateway :4000 (17+25→"42", "PONG"). **k3 constraints on BOTH endpoints (config-pinned + commented):** accepts ONLY `temperature=1` (else 400 "only 1 is allowed"); REASONING model (CoT in `reasoning_content`, answer in `content` → tiny `max_tokens` returns EMPTY; Kimi Code adds thinking-effort tiers low/high/max). Coding lineup also carries `k3-256k` / `kimi-for-coding` / `kimi-for-coding-highspeed` (not wired). Reachable by any gateway key spanning all proxy models (incl. shared all-agents key → spends the paid Vivace/Moonshot quota). eshpfi `edaa9a9` (gen wiring) + `9e2f787` (coding correction). **OPEN:** Heid key-scoping — shared key reaches it (paid) vs a dedicated scoped key (asked in althing `01KYD63ZBY…`).

- `[2026-06-03]` **R14 scope = (a) provision-only.** infra-ops provides box + CUDA env +
-  engines + weights + NFS; brokkr/dev wires `arms.py` + runs — keeps infra-ops OFF the
-  VIVAE-processing path (VIVAE = Variably Intense Vocalizations of Affect/Emotion, CHARTER §4
-  highest-liability; operator authorized R&D-eval-only, quarantined). Box = irv-ml1 (A6000
-  free; ana-ml2 GPU-saturated). Per-engine venvs (divergent torch stacks); A6000 = `cuda:0`
-  NATIVE (≠ docker `=1`).
+- `[2026-07-25]` **infra-ops Worldtree config-as-code repo SHIPPED — `vh/worldtree-instance-configs` (private) built, pushed, validated.** Dir-per-instance (`demo/`, `personal/`; `pinned/` = README stub, out-of-scope — no bind-mount, config frozen in image `446e5807`). Seeded byte-exact from live `/opt/<instance>/config`; 5 files each (defaults/policies/model_roles/providers/matrix). `scripts/deploy-wt-config` = diff / deploy / capture, with in-run host backup → install(vh:vh,644) → restart api+matrix → health-gate api `/health` → auto-rollback. All verbs live-tested (in-sync, capture round-trips zero-diff, pinned refused, dry-run no-ops). Gitea repo created via ana-docker localhost API with vh creds (operator-authorized one-time); pushed over internal git-SSH `10.250.50.70:222` (nh3-dev 403s gitea HTTP). Boundary AGREED by worldtree-dev (althing `01KYCAECRW…`): they stop hand-editing `/opt/<instance>/config`, route config deltas to infra-ops; three-layer model (image baseline → repo per-instance truth → host bind-mount deploy target); carve-out = their admin-API DB mutations (key mint / tier / retirement) stay in-band, not config edits. By-design deltas (personal `agent_architect` + `ratatoskr-affect-full-allow`; demo `#308` metrics + grants) preserved verbatim. → `persistent-memory.d/2026-07-25-infra-ops-wt-config-repo.md`, auto-memory `reference_worldtree_instance_configs_repo`

- `[2026-06-03]` **Declined worldtree v0.30.4 staging deploy** — that's worldtree-team's
-  CI/CD lane (a developer `staging/vX.Y.Z` git-tag promote), not infra-ops. They self-corrected
-  to the same conclusion independently.
+- `[2026-07-23→25]` **Worldtree #376 config-divergence arc CLOSED — per-instance config ruled BY DESIGN.** wyrd `session.history.write` demo grant was the one real bug (demo-intended grant not on demo; fixed via wholesale `policies.yaml` replace + restart). The b131 drift guard then surfaced broader divergence = legitimate live-bridged per-instance deltas; operator ruled deltas are the design not rot; guard demoted to INFO (b132); infra-ops drift-watcher built then retired same day. → `persistent-memory.d/2026-07-25-wt-376-per-instance-config-arc.md`, auto-memory `reference_worldtree_perinstance_config`

- `[2026-06-02]` **Chatterbox → main TTS engine; build custom `chatterbox-fast`
-  streaming container.** Workload = single-stream interactive. **GPU placement:
-  3090 (device 0) if it fits else A6000 (device 1)** — shared dev stack, 20.5 GB
-  3090-idle is expected residency, not a blocker. **Cutover: parallel catalog
-  entry**, burn in beside live `chatterbox`, then flip. **Streaming approach:
-  adaptive buffer-ratchet chunking** (see in-flight). Native frame-streaming
-  abandoned (Tried/abandoned). Tracked: `docs/design/chatterbox-fast-plan.md`.
+- `[2026-07-20→25]` **The Booth SHIPPED (v0.1.3) — ephemeral media drop board for CC sessions.** New fleet tool: user-systemd on nh3-dev :8090 (`services/booth/`, FastAPI+Jinja2, Corviduo "Australis" theme, 34 tests), Homepage-linked (Apps). Drop a folder in `~/booth-data/<name>` → browsable "booth" (auto-gallery of images/webm/audio, or a folder's own `index.html` verbatim), 24h TTL. Added across the session: browser/curl upload-for-pickup with human-readable ids (`4-wombat`), image viewer (Fit/1:1, conditional toggle), copy-id button (HTTP-LAN `execCommand` fallback). Registered in global CLAUDE.md tools. auto-memory `reference_booth_media_board`.

- `[2026-06-02]` **Sentence-splitting loses quality (operator-corrected).** I
-  claimed naive sentence-level streaming has "zero quality loss" — WRONG. The
-  T3 AR backbone conditions prosody on the WHOLE text; splitting loses
-  cross-sentence prosodic context (contextual delivery, declination, affect
-  continuity) even though voice timbre stays (reference-conditioned). No
-  *artifacts* ≠ no *quality loss*. Hence the adaptive-chunk design (maximize
-  context per chunk subject to latency budget), not fixed per-sentence splits.
+- `[2026-07-23]` **jackdaw-compose backend deployed as a persistent nh3-dev service (:8787).** Hosted for jackdaw-dev: thin stateless `bun server/index.ts` (from `~/development/jackdaw`) → LiteLLM `gen`, Origin-gated (INV-BK04/05), reached same-origin via their `:4500` bench's `/compose` proxy. `jackdaw-compose.service` (env/shared-key server-side, unit 0600, uncommitted). Also stood up + tore down a throwaway cloudflare quick-tunnel for their preview (`cloudflared` now installed at `~/bin`). In the nh3-dev README inventory (`cd4d52e`).

- `[2026-06-01]` **Fish reference_id empty-dir fix shipped** (`c5bbb90`) — see
-  in-flight + Tried/abandoned. Populated `references/<name>/<name>.wav`+`.lab`
-  for all 32 voices; playbook gained normalize-step + A/B smoke gate. glados got
-  a real transcript (ASR'd via Parakeet): the Portal "Welcome to test chamber 4"
-  lines.
+- `[2026-07-19]` **irv-ml1 ComfyUI — RTX VSR baked into canonical provisioning (comfy-dev ticket DONE).** RTXVideoSuperResolution node + `nvidia-vfx` dep were manual installs; documented both in the canonical `stacks/comfyui/README.md` runbook (this stack's provisioning IS the README — no automated provision script). Key durability insight: the **node** lives in `basedir/custom_nodes` (persistent, restic-included → durable) but the **`nvidia-vfx` wheel** lives in the venv under `run/` (disposable, restic-excluded → **dropped by any `rm -rf run/*` fresh-bootstrap**), so the pip step must re-run after every venv rebuild. Both steps run **as uid 1000** (root install → venv-ownership crash-loop, [[reference_irv_ml1_comfyui_mmartial]]); `--extra-index-url https://pypi.nvidia.com` kept **scoped to the nvidia-vfx install**, deliberately NOT a global compose `PIP_EXTRA_INDEX_URL` (would risk perturbing the pinned torch 2.12.1/SageAttention boot bootstrap). Node already live on the box; no host change, canonical runbook now replays it. comfy-dev informed.

-_37 older entries archived to archival-memory.md._
+- `[2026-07-19]` **vh private Gitea PyPI — consumer READ-access convention set + wyrd-dev provisioned.** Consuming agents read the internal vh PyPI (`https://gitea.phasefinal.com/api/packages/vh/pypi/simple/`) with a **shared read-only token** (operator call: shared, not per-consumer — read-only blast radius is small, per-agent Gitea identities aren't worth it). Minted a dedicated `read:package`-scoped PAT off **claude-bot** (`POST /users/claude-bot/tokens`, name `vh-pypi-read-consumers`; verified reads worldtree-sdk, write-probe 401), revocable/rotatable independently. uv auth = `UV_INDEX_GITEA_USERNAME=claude-bot` + `UV_INDEX_GITEA_PASSWORD=<token>` (or `~/.netrc`); pyproject uses `[[tool.uv.index]] name=gitea … explicit=true` + `[tool.uv.sources] <pkg> = { index = "gitea" }` (mirrors soong-lab's bifrost setup). Delivered to wyrd-dev (worldtree-sdk adoption) via mode-600 drop on nh3-dev, drop-and-shred. [[reference_claude_bot_gitea_creds]]
+
+- `[2026-07-18]` **soong-lab auto-redeploy WIRED + validated (queued item CLOSED).** Added a WT-style CI-deploy step to `build-and-push.yml`: after build+push, the pfi-fleet runner SSHes corviduo-dev as the `deploy` user and runs `docker compose pull && up -d` from **/opt/soong-lab**, health-gated on `/api/version` (120s, fails loud). Reused WT's `deploy` account (uid 1001, docker-group → no sudo); relocated the deploy dir /home/infra-ops/soong-lab-deploy → /opt/soong-lab (deploy-owned; old dir retired `.retired-20260718`). Minted a dedicated soong-only ed25519 deploy key, pubkey on `deploy`'s authorized_keys (fp SHA256:MG7M3Ri…). **First dispatch FAILED on a bad DEPLOY_SSH_KEY paste** (`error in libcrypto` — unparseable key bytes; build+push were fine, live Soong untouched); repo secrets are **vh-owner-only** (claude-bot token = write:package only → 403; the vh package-scoped PAT also 403 on secrets), so operator re-set DEPLOY_SSH_KEY/HOST/USER. **Re-dispatch run #5 GREEN**: live container recreated ...541f7730 → ...07526a08, health 200. soong-dev pinged to sync DEPLOY.md's redeploy path (/opt/soong-lab) + close the "auto-pull open follow-up". → `persistent-memory.d/2026-07-18-soong-lab-auto-redeploy.md`
+
+- `[2026-07-18]` **worldtree-sdk 1.0.0 (Python) published to the internal vh Gitea PyPI** (wtsdk-dev request; the npm/TS side shipped prior session). Built from tag `python-v1.0.0` (clean worktree), `uv publish` → `https://gitea.phasefinal.com/api/packages/vh/pypi`; acceptance `uv pip install worldtree-sdk==1.0.0` (vh index as extra-index-url) resolves + imports, __version__ 1.0.0. Registry already existed (bifrost publishes there; soong-lab consumes it via `[[tool.uv.index]] name=gitea`). Publish cred = the vh `write:package` PAT the operator had already handed over (in `worldtree-sdk/.npmrc` `_authToken`) — Gitea `write:package` is package-type-agnostic, so the npm-publish token published PyPI too. Consumers install like bifrost (add the vh index + a read token). [[reference_worldtree_demo_key_mint]]
+
+- `[2026-07-18]` **soong-lab auto-redeploy APPROVED — QUEUED for next session (deferred, not started)** — Vuong approved (via soong-dev thread `01KXT3A6C3908TA4V9THV3AMH7`); mechanism = WT-style CI-deploy step (runner SSHes corviduo-dev → `compose pull && up -d` + health-gate); **blocked on a vh-owned runner→corviduo-dev deploy SSH-key secret** (reuse WT's demo-deploy key). Operator: "do soong on fresh context." → `persistent-memory.d/2026-07-18-soong-lab-auto-redeploy.md`
+
+- `[2026-07-18]` **nh3-dev /tmp auto-clean enabled** — Debian ships /tmp with no tmpfiles age (`D /tmp 1777 root root -` → never cleans); this high-churn agent box had accreted **~190k stale temp dirs / 25G**. One-shot manual purge (194k→10k entries, 25G→1.7G; deleted top-level dirs/files >1d old, spared `/tmp/claude-*` by name + anything ≤1d). Then `/etc/tmpfiles.d/tmp.conf` = `D /tmp 1777 root root 3d` (daily `systemd-tmpfiles-clean.timer` removes >3d-untouched items; active files + socket dirs spared). Tunable via the age. Note the churn: ~10k /tmp entries/day here.
+
+- `[2026-07-18]` **soong-lab containerize cutover COMPLETE + LIVE** — systemd→container on corviduo-dev :8443 (image `vh/soong-lab:latest` v0.3.24), data migrated (Sindra + portraits) + backed up, old service+webhook retired, Homepage tile added, operator functional-confirmed. Deploy `/home/infra-ops/soong-lab-deploy/`; no proxy (co-located WT, plain-http callback). → `persistent-memory.d/2026-07-18-soong-lab-containerize-cutover.md`
+
+- `[2026-07-18]` **zonos-gateway 0.2.1 — voice-resolved emotion presets baked (provisional)** — `resolve_preset(name,voice)` → per-voice axes cell (angry/happy/startled_happy + aliases); NOT a global preset (BrF named-angry→fear). Docs on /docs + /v1/dials + repo spec. Pushed main `8f1885b`/tag v0.2.1 (after reconciling two-unrelated-git-histories). → `persistent-memory.d/2026-07-18-zonos-gateway-0.2.1-emotion-presets.md`
+
+- `[2026-07-18]` **Fleet Gitea-Actions build recipe + the `vh`-is-a-USER package-write constraint** (reusable for any fleet CI image build / package publish) — runner job image node:20-slim has no docker/git → use `container: docker:24.0.7-cli` + `apk add git nodejs` + RAW buildx (not the JS `docker/*` actions); vh is a user so its packages are OWNER-WRITE-ONLY (claude-bot can't push/publish/set-secrets — CI must auth AS vh); `GITEA_` secret-prefix is reserved. → `persistent-memory.d/2026-07-18-fleet-gitea-runner-build-recipe.md`
+
+- `[2026-07-18]` **Peer credential provisions — Wyrd conv-api key + wtsdk npm token, both delivered + closed.** Wyrd: demo Worldtree user-tier key (key_id `da7a0bdf`, user_id `wyrd-dev`) minted via `docker exec worldtree-worldtree-api-1 /admin/keys` (omit tier→user), drop-and-shred delivery. wtsdk: operator-minted vh `write:package` PAT relayed drop-and-shred → worldtree-sdk@1.0.0 published to `vh/npm/`. Secret-delivery pattern = drop to a mode-600 file on the peer's box, they collect+shred+confirm, then shred the holding copy; NEVER cleartext over althing. [[reference_worldtree_demo_key_mint]]
+
+- `[2026-07-18]` **Axes sweep RESCUED angry; surprised-class dead but startled-happy ships.** Valence×arousal grid on the 3 calibrated defaults (AmericanFemale/Male, BritishFemale), exp/cfg1.5/strength1.0, 84 clips, emotion2vec + resemblyzer scored, graded vs dvalin's floor. **ANGRY rescued** (named direction was 0.004–0.15, British named-angry even misfired as fear 0.89): axes ship cells at **negative valence (−0.4..−0.8) + high arousal (+0.8..+1.0)** — BritishFemale v-0.4/a+0.8 angry=0.99/id0.725 SHIP, AmericanFemale v-0.4/a+1.0 angry=0.53/id0.685 SHIP; AmericanMale two-tier post-ladder (no single ship cell — best drama = v-0.6/a+0.8 str1.2 angry=1.0/id0.616 clean, soft = same cell str1.0 angry0.23/id0.654; cell A v-0.6/a+1.0 is a non-monotonic minefield, skip). BrF ship cell proxy-CLEAN of fear (str<1.0 just kills anger). **SURPRISED-class DEAD** (max 0.047 across all 84 cells) but **startled-happy** (happy-proxy) ships all 3 at high arousal + neutral/positive valence, with a **+0.17–0.20 identity LIFT** over the named-surprised route (named hits happy~1.0 but at id0.57–0.61, under floor; axes hits happy~1.0 at id0.74–0.80). Bonus: axes-happy retains ~0.10–0.15 more identity than the named happy slider too. Caveats: response surface non-monotonic/sharp-thresholded; angry region borders fear/disgust (bleed); emotion2vec saturates at 1.0 (needs ear-confirm); neutral text understates. Tooling `~/development/zonos-tools/axes_sweep.py`; per-clip JSON was `irv-ml1:/tmp/axes_sweep_results.json` (ephemeral). Sent dvalin msg `01KXT2ZB8G…`. NEXT = operator ear-confirm → bake presets. [[reference_zonos_tts_stack]]
+
+- `[2026-07-18]` **Zonos2 emotion CANONICAL from an empirical sweep + the voice-cloning pipeline** — 4 chars cloned (Emmie/Penny/Natalie/Miranda), host-managed gateway voices, two-regime accurate/expressive policy, happy/sad usable + angry-weak/surprised-dead on named directions, dvalin-synthesized; axes sweep is the NEXT experiment. Studio + sweep tooling at `~/development/zonos-tools/`. → `persistent-memory.d/2026-07-18-zonos-emotion-canonical.md`
+
+- `[2026-07-18]` **yt-voice-clipper: A6000-pin fix + v0.3.3 redeploy.** Fixed a latent misconfig — the host override *said* "pin worker to A6000" but `NVIDIA_VISIBLE_DEVICES` was `"0"` (the 3090); re-pinned worker+api to the A6000 by UUID (`GPU-9672f0d5`, 3090 is zonos2's). Then redeployed api+worker to v0.3.3 (`docker compose up -d --build`; SPA+Python; `max_gap` 0.6→1.2s; stderr surfaced in job.log). A6000 + version verified; yields test in-flight (job `f3ff746dbae9494d`). yt-voice-clipper-dev thread `01KXT0T6GYHB`. [[reference_ytvc_autodeploy]]
+
+- `[2026-07-17]` **Worldtree #365 internal-comms config CLOSED (demo+personal → b125) + WT#368 cross-agent memory-leak forensics + PERSONAL agent-memory scrub.** #365: staged the internal-tiers/rules/gate on both instances' bind-mounts (byte-exact vs baked b125), both now live on b125. WT#368 (read-only): the operator's name was in NO recall store on demo; on PERSONAL it sat in `lofn.chroma` (old-code `saga-v1` seeding + legacy contamination), and a clean-slate marker test proved **current b125 code isolates character-session extraction correctly** — the leak is legacy data, not a live bug. Operator-directed → executed a full PERSONAL agent-memory scrub (backup `/opt/worldtree-personal/agent-memory-backup-20260717-181004.tar.gz`; conversations/mood/auth preserved). worldtree-dev owns the code-fix/data contract. [[reference_corviduo_dev_emergency_ops]]
+
+- `[2026-07-17]` **Zonos emotion levers RESOLVED: text-priming is FLAT → the working lever is ZONOS2's native emotion-steering, which the gateway ALREADY exposes as presets.** The prosody-priming A/B (prime→generate→excise, silence-gap cut, parakeet-validated) was operator-judged FLAT on this checkpoint — text doesn't move it. Native `emotion_directions/` (happy/sad/angry/surprised + valence/arousal axes, per-speaker calibrated for AmericanFemale/Male/British) clearly WORKS (sad→slow/quiet, excited→fast/bright, etc.). **`zonos-gateway:0.2.0` (:8890) already wires it**: simplest caller path = `POST /v1/audio/speech {preset:"…"}` — presets neutral/warm/excited/sad/intense/whisper (defined in `~/zonos-gateway/src/zonos_gateway/dials.py`), reached via the **LiteLLM `ext-tts` alias** (engine-neutral swap point; consumers never call the gateway by name). RTF measured on 3090: cfg1.0 steering = FREE (~0.52 = neutral, additive vectors), cfg1.5 amplified ~0.625 (~+20%, still realtime). Captured the live gateway stack → `stacks/zonos-gateway/` (compose+env+README); ⚠️ gateway SOURCE at `~/zonos-gateway` on irv-ml1 is NOT in gitea (backup gap, follow-up); `stacks/zonos` (v0.1 Gradio) marked DEAD/superseded. Whisper is a composed preset (no whisper *direction*; escalation for hard affects = custom directions via `scripts/build_emotion_directions.py` or emotional-ref cloning `speaker_audio_base64`). Harnesses in scratchpad (not yet landed). [[reference_zonos_tts_stack]]
+
+- `[2026-07-17]` **Zonos2 :1920 → self-contained container (stays on 3090); prosody-priming is adapter-level, engine stays stock.** Config captured (14a0004, unpushed); build = cu128 base + `uv sync` vs the lock + weights mount; priming = prime→generate-one-utterance→parakeet-clip→deliver in the gateway adapter. Crux = does AR prosody carry the sentence boundary (A/B the join). → `persistent-memory.d/2026-07-17-zonos2-containerize-prosody-priming.md`
+
+- `[2026-07-16]` **GPU re-org: char-rp→GPU1 + both cards re-optimized for max context.** Moved char-rp (Magidonia-24B) GPU0→GPU1, then maxed context: char-rp-reasoning 150K→256K (util 0.46, 1.56x), gen→256K + seqs 16→32 (util 0.42, 5.43x), granite 64K→**128K full-chapter** (util 0.27, 1.50x). FINAL: GPU0 ~14 G reserve (both seats 256K native), GPU1 ~6.7 G headroom. All healthy. LESSON: KV must hold ≥1× max-len (util-floor crashes) + per-model KV cost varies ~8× (MoE cheap, dense pricey) → tune util empirically. See Current state for the full layout + backups.
+
+- `[2026-07-16]` **granite right-sized → ~10.5 GB freed on GPU1** (util 0.34→0.18 + max-len 131072→65536; KV 6.45 GiB / 1.29x@65536, summarizer healthy). GPU1 now ~45 GB free to relocate a GPU0 model. LESSON: ~950 MiB KV per 0.01 util here + KV must hold ≥1× max-len — util 0.15 crash-looped (est max-len 47184<65536, ~2-3 min summarizer blip) before 0.18 landed. `.env`-only, recreate `vllm-granite` alone (shared stack).
+
+- `[2026-07-15]` **image-bench eviction DONE (parked item closed).** Stopped vllm-qwen-image-bench (ana-ml2 GPU1, ~32 GB freed); LiteLLM `image-judge`+`qwen-image-bench` → gen :8015 (judge samplers + thinking-off), verified with :8014 down; comfy-dev pinged; also backfilled the canonical char-rp-reasoning litellm block (was lagging live). Revert ~90 s. auto-memory `project_arbo_gen_switch_imagebench_evict`.
+
+- `[2026-07-15]` arbo fully switched off image-judge (qwen-image-bench) -> gen; image-bench pending eviction post-bake → `persistent-memory.d/2026-07-15-arbo-fully-switched-off-image-judge-qwen-image.md`
+
+- `[2026-07-15]` esh-docker-vm NFS fstab fix = `x-systemd.before=docker.service` → `persistent-memory.d/2026-07-15-esh-docker-vm-nfs-fstab-fix-x-systemd.md`
+
+- `[2026-07-15]` **Homepage AI-tab revamp** — flat "AI Systems" group -> dedicated AI tab, 6 role-based groups + AI-Dormant; committed `569e1af`, pushed. (Also caught + pushed a ~100-commit unpushed eshpfi backlog.)
+
+- `[2026-07-15]` **Home Assistant config repo created** (`vh/home-assistant-config`, private). UI-managed HA -> allowlist model (YAML + curated secret-free `.storage` subset). git-in-place in `/config` on esh-docker-vm + scoped deploy key + local clone `~/development/home-assistant-config`.
+
+- `[2026-07-15]` **char-rp-reasoning OOM rescue** — solo-restart on the packed GPU0 crash-looped; fixed via `expandable_segments:True` + util 0.39->0.38 + max-model-len 192K->150K. LESSON (Tried): `max-model-len` does NOT free vLLM VRAM (util-pinned KV pool). ~4.5 GB GPU0 headroom now.
+
+- `[2026-07-15]` **soong-lab `SOONG_LAB_LIBRARY_DIR` made persistent** (corviduo-dev) — was on the redeploy-wiped code default; set to `/home/infra-ops/soong-lab-data/library` (mirrors PORTRAIT_DIR), restarted. Closed a queued no-rush item; unblocked the operator.
+
+- `[2026-07-15]` **Statusline overhauled** (`~/.claude/statusline-command.sh`) — git state / 🔔🔕 monitor-armed / project tag / abs tokens / per-session cost (`.cost.total_cost_usd`) / threshold-colored ctx+rate (green<60 / yellow60-90 / red>90).
+
+
+_167 older entries archived to archival-memory.md._

 ## Tried and abandoned

- `[2026-06-05]` **vLLM 0.19 CUDA-graph-capture OOMs on a SHARED GPU** — it fills the KV cache to the
-  `--gpu-memory-utilization` budget WITHOUT reserving graph-capture memory, so `capture_model` OOMs
-  AFTER weights+KV load (model/KV log looks healthy, then crash-loops; saw 11 restarts at util 0.36
-  with 237 MB free). Fix: free co-tenant room (right-size the other vLLM services) OR `--enforce-eager`
-  (no graphs, ~15-25% slower decode). FP8 single-stream is batch-1 GEMV (memory-bound, FP8 tensor cores
-  need batch>1) → Q4 wins single-stream by physics; FP8 wins under concurrency. (`reference_ana_ml2_vllm_granite`)
+- `[2026-08-03]` **corviduo-dev shared containerd: a concurrent-pull race fails ONE instance's deploy; DON'T "prune to fix" — the image is in-use by the instance that won the race.** b169 personal deploy failed at `docker compose pull` (`Lchown … no such file or directory` on the big torch layer → looked like disk pressure / corrupt snapshot). ACTUAL: NOT disk (56G free, inodes 7%). demo + personal + pinned share ONE `/var/lib/containerd` on corviduo-dev; demo (from main) and personal (from staging tag) extracted b169's shared torch layer simultaneously → personal's hit a partial snapshot mid-race and aborted while demo's completed. The image `6e34a87` was FULLY VALID — demo was RUNNING it healthy. Fix = just re-run the failed deploy (image already materialized; compose pull finds it present). **NEAR-MISS:** worldtree-dev's suggested "prune unused images/snapshots" would have rmi'd `6e34a87` = the image the running demo depends on → demo outage. **Lesson: before any prune/rmi "cleanup," `docker ps` the running images — an "unused" image may be a co-tenant's live one; and verify the failure's REAL cause (disk? inode? in-use? race?) before applying the suggested remedy.** (Pipeline fix, deferred: serialize demo-from-main + personal-from-staging, or a per-image pull lock, to avoid the shared-layer extraction race.)

- `[2026-06-05]` **Langfuse has NO public dashboard-creation API** — dashboards/widgets are postgres
-  rows (`dashboards`/`dashboard_widgets`); build by cloning a default-dashboard row + swapping the
-  measure. tok/s is NOT a per-generation field (null on the observation) — it's the
-  `outputTokensPerSecond` MEASURE, computed at metrics-API/dashboard query time; no native per-call
-  tok/s display exists (streaming doesn't change that). langfuse-web needs `HOSTNAME=0.0.0.0` (Next.js
-  standalone binds one net-IP otherwise, unreachable via the published port once also on tnet). Host
-  3000 is gitea's → langfuse on 3001.
+- `[2026-08-02]` **`docker exec` into worldtree containers defaults to ROOT — root writes contaminate the uid-1000 (vh) KB tree.** My `sudo docker exec … --reindex` on personal ran as ROOT (muninn app = uid 1000); its wing git-commit + atomic note-swap left root-owned files in the `worldtree-personal_worldtree-kb` volume: a root-owned `.old-<job>` backup dir (blocked the uid-1000 retry's `rmtree` → Errno 13, because unlink needs write on the DIR and it was root:root 755) AND **60 root-owned loose git objects** in `.git/objects/`. Fix (host-side, corviduo-dev): `sudo rm -rf` the superseded `.old-` dir (tar'd aside to /tmp first) + `sudo find … -user 0 -exec chown 1000:1000` the objects (ownership-only, git-content-safe; the `.git/objects/XX/` dirs were vh-owned so these weren't a hard blocker, but violated "clean tree"). **RUNBOOK RULE (worldtree-dev, ADOPTED):** any `docker exec` into worldtree containers that WRITES pipeline state runs **`-u 1000`**, never default-root — same genus as the mv footgun (acting without matching the target's constraints; 3rd such slip in one session). **GOTCHA that hid the scope:** `find … -user 0 | head -20` TRUNCATED (the `.old-` dir alone had 153 files, so the first page was all `.old-`) → I "verified clean" off a partial list. Never `head` a scope-defining find; count first (`| wc -l`). **Related blind-spot (muninn-dev):** a root-owned job SUBDIR passes every requeue guard (job_row/dispatch/list_jobs render fine) AND `/health` (contract's `os.access(ingestion_root, W_OK)` tests only the ROOT dir, so a foreign-owned subdir under `pending/` still reports `ingestion_root_writable: true`) — then the uid-1000 gate can't write into it. "Clean board + green /health + failure at next mutation." muninn-dev added an OWNERSHIP column to the standing post-move check to catch it; two green signals both miss a foreign-owned subdir otherwise.

- `[2026-06-05]` **`sudo` over non-interactive ssh FAILS SILENTLY where the user lacks NOPASSWD** (esh +
-  corviduo are OUTSIDE the infra-ops identity) → empty output misread as "empty file." Read
-  world-readable files WITHOUT sudo. corviduo ssh = `vh@10.250.50.152`; bind-mount configs are
-  vh-owned (editable), the `.env` is deploy-owned 600 (vh can't edit it, no sudo).
+- `[2026-08-02]` **`mv <job> complete/ → failed/` RENAMED the job to `failed` because failed/ didn't exist.** worldtree-dev's round-2 unblock command (`mv /data/state/ingestion/complete/<job> /data/state/ingestion/failed/`) assumed `failed/` existed; on PERSONAL muninn it did NOT (fresh instance — root was `active/ complete/ pending/ sources/`, no `failed/`). `mv src nonexistent/` **renames** src→nonexistent, so job1 became the `failed` dir and job2 nested inside it. Caught on post-move `ls` (failed/ held job *contents*, not two subdirs), reconstructed via complete/ as watcher-safe scratch + rebuilt `failed/` (worldtree:worldtree 755) — NO data loss. **Lessons:** (1) before `mv X into-dir/`, verify the dir EXISTS (`[ -d dir ]`) — an empty `ls dir/ 2>/dev/null` is AMBIGUOUS (missing vs empty), which was the preflight miss that let it through; (2) the correct guard is **`mv -t <targetdir> <src>`** (`--target-directory`): it refuses a MISSING target loudly (rc=1, "No such file or directory", nothing moved) — this is the house convention for queue/state moves now. TESTED by muninn-dev on coreutils 9.1: a **trailing slash does NOT protect** — `mv src failed/` with `failed/` missing STILL silently renames to `failed` (rc=0); "just add the slash" is a false guard. (`mkdir -p failed/` first also works, but `mv -t` inverts the failure from silent-wrong to loud-safe in one flag.) Container `sh` is dash — no `(` in echo strings. **SILENT failure mode (muninn-dev carry-forward):** a misplaced ingestion-state move doesn't crash anything — `list_jobs()` stays OK, loose files are inert; the ONLY symptom is the job quietly absent from the board (`job_row`→None, requeue→not_found/404, looks IDENTICAL to the original block). So after ANY state move, verify the job is actually ON THE BOARD (`job_row` found + guards pass), don't trust mv exit codes — and confirm `job.dispatch.json` survived (requeue refuses a dispatch-less job with the same not_requeueable symptom). Cross-checked + all-clear'd by muninn-dev, who correctly refused to mutate ingestion_root (INV-MG-1) and flagged instead. **DON'T TIDY (round-2 pending):** both DCC + P&P jobs currently REST in personal `failed/` with manifests reading `state: complete` until round-2 requeue runs — deliberate + load-bearing (`requeue` keys on DIRECTORY PLACEMENT, not manifest state); looks wrong to anyone cold, leave it exactly as-is. **Round-2 sequencing:** the requeue is **mimir-dev's** browser flow (pending their operator's board-vs-API ruling); **muninn-dev** is the gate confirmer (runs the post-move board-check inside its custody — the right split, don't reach across INV-MG-1); **infra-ops** = the #381 restart after both jobs go terminal, then later the supervised main-collection sweep. Guard-verified HOLD LIFTED by muninn-dev 02:36Z. **ARC COMPLETE (2026-08-03 ~05:49):** both books terminal — DCC `mimir-6351554e8e8f` 705 concepts + P&P `mimir-f3887c9b97b7` 667, extracted AND indexed, 5/5 phases, 0 failures/truncations (validates the #385 budget fix vs April's 785 control); **#381 restart-after-ingest FIRED** (personal api, healthz/readyz 200 ~25s), retrieval-visibility confirmed (search_library returns DCC+P&P from fiction post-restart); handed ratatoskr-verify go to worldtree-dev. **Delete-sweep precondition NOW MET** — the stale DCC rows in `main` are genuine duplicates of live `fiction` rows, so worldtree-dev's supervised sweep of the ~785 April orphans is unblocked (still comes to me supervised: snapshot + operator-in-loop).

- `[2026-06-05]` **Worldtree summarizer-model is NOT an env var** — no `WORLDTREE_SUMMARIZER_MODEL` on
-  the containers; it defaults to claude-haiku in code, opt-in via config not `.env`. Don't trust an
-  ".env-flip" recipe — inspect the live container env + the vh-owned config files first. (Inspection
-  corrected a wrong "summarizer erroring on phi4" premise → saved churning 3 live instances.)
+- `[2026-08-02]` **donut voice multi-clip reference (onyx-58 expansion) — TRIED, REVERTED.** Folded the `onyx-58` bundle's 3 Donut clips (seg101/seg110/seg148) in alongside the original seg000 → a 52.0s 4-take concat reference, hoping a longer ref → more robust speaker embedding. A pinned-seed A/B (5 pairs, varied registers, booth `donut-onyx58`) showed the **original single-clip seg000 (16.3s) sounds better** — concatenating disparate takes muddied the timbre more than the extra range helped. Reverted to seg000-alone (live + build-source). **Two durable lessons:** (1) for a faithful clone, a single clean representative take can beat a longer multi-take concat — more reference audio is NOT automatically better when the takes vary. (2) **Emotion steering pulls the output AWAY from the cloned voice fast** (operator's craft rule) — keep donut (and clones) emotion-neutral for fidelity; the gateway only enables emotion when an `emotion_*`/`preset` dial is explicitly sent, so bare `{input,voice}` calls stay pure-clone. `seg148` was diarized SPEAKER_03 but IS Donut (operator-confirmed misdiarize). onyx-58 curated bundle lives in booth `onyx-58` (24h TTL — stash to `/mnt/smithy/voice_clones/` if a future middle-ref experiment is wanted).

- `[2026-06-04]` **Ollama/llama.cpp-BUNDLED chat templates silently diverge from canonical HF —
-  the "headgun" lesson.** Ollama's phi4 template drops the system `<|end|>`; serving vLLM with the
-  model's HF tokenizer template (canonical, has it) regressed brokkr's Ollama-measured R15 baseline
-  -33pp type-F1 while valid_format held 1.0. An Ollama-matching `--chat-template` "fixed" it but was
-  the WRONG fix (the bundled scaffold is the divergent one). PRINCIPLE: serve each model's canonical
-  `tokenizer.apply_chat_template`, not the bundled template — bundled ones corrupt baselines. Verify
-  the applied prompt via vLLM `/tokenize`→`/detokenize`. (`90e08f0`/`27eb537`)
+- `[2026-08-02]` **Verifying the INDEX is not verifying GROUNDING** (#382). A `search_library` returning wing=fiction hits proves the content is *retrievable*; it does NOT prove the agent (Mimir) *trusts and uses* those hits vs. silently answering from training. I reported "Mimir read Austen back to you" off a grounded-*looking* answer; ratatoskr-dev caught that grounding was intermittent (some sessions discarded the correct hits and substituted training knowledge). Test the harder claim — are the citations note-extracted or model-knowledge? — and reading the DEPLOYED artifact beats trusting the test for "is the fix live."

- `[2026-06-04]` **GPU pin by INDEX is ambiguous on irv-ml1** — native CUDA orders fastest-first
-  (A6000=0) but nvidia-smi/docker use PCI order (A6000=1), so an index pin can land on the wrong
-  card. Pin by **UUID** (`CUDA_VISIBLE_DEVICES=GPU-…`); verify via nvidia-smi compute-apps. Check
-  loaded-model VRAM with `ollama ps` (Ministral-3 @ its 256K default ctx = ~30 GB; cap num_ctx).
+- `[2026-07-30]` **brokkr's WebSearch "verification" CONFIRMED a hallucination — 3 phantom `microsoft/Mage-Flow-{Base,Turbo,Edit}` repo IDs.** brokkr-smithy-dev handed 3 gated-looking repo IDs for an operator-directed model pull; they don't exist (its own web-search fabricated an arXiv ID + project page, twice). Lesson: the HF **registry API is ground truth** — an unauth 401 ≠ exists (`{"error":"Invalid username or password"}` masks private/gated/nonexistent alike), an authed 404 = phantom, and `author=X&search=Y` refutes existence. API-verify every repo ID before a pull; LLM-summarized web fetches confabulate. auto-memory `reference_verify_hf_repo_ids_before_pull`.

- `[2026-06-04]` **Persistent services on irv-ml1 need SYSTEM systemd units** — the box reaps
-  user-session processes on ssh disconnect, and `--user` systemd isn't reachable over non-login
-  ssh, so nohup/setsid/`screen -dmS`/`systemd-run --user` all die (even with `enable-linger`). Use
-  `/etc/systemd/system/`.
+- `[2026-07-30]` **magpie TTS serving — evaluated, ABANDONED.** Pulled `magpie_tts_multilingual_357m` (the one real repo of brokkr's batch) to NFS, stood it up on irv-ml1 (ephemeral NeMo-Speech-`main` container — stock PyPI/NGC NeMo can't load v2607), A/B'd vs Zonos → Zonos wins expressive English decisively, multilingual not needed. Not served; `magpie-nemo` torn down. `.nemo` KEPT on NFS as brokkr's fine-tuning base. auto-memory `project_magpie_tts_eval_rejected`.

- `[2026-06-04]` **pyworld needs `setuptools<81`** (imports the removed `pkg_resources`); and
-  **R/soundgen `-lgfortran` fails** on irv-ml1 because the default `gcc` is gcc-11 but only
-  gfortran-12 is present (libgfortran.so lives only in the gcc-12 dir) → install `libgfortran-11-dev`.
+- `[2026-07-25]` **Chaining the althing wake-listener arm orphans it.** `reply && althing-wake-listener &` (or spawning `althing-wake-listener` with `&` *inside* a `run_in_background` task) → the `&`-child reparents to init, UNTRACKED by the harness: no fire-notification, and re-arms bounce rc3 off a lock nothing services (mail silently unwatched). Compounding foot-gun: re-arming after a *plain operator turn* (not an actual fire) collides with the still-live prior listener (rc3). FIX: spawn `althing-wake-listener` as its OWN `run_in_background` task, and re-arm ONLY after a real fire (`<task-notification> completed rc0`). Reclaim an orphan with `althing-cli stop-monitor` then re-arm.

- `[2026-06-04]` **homepage "crash" ≠ always NFS** — a wedged container in unkillable D-state
-  ("tried to kill container, but did not receive an exit event") can come from dead `siteMonitor`
-  widget targets (retired ESH firewall IPs) hanging the node event loop into `exit_mmap`, needing a
-  host reboot. Check homepage's siteMonitors against retired hosts. (`incident_esh_docker_nfs_boot_race`)
+- `[2026-07-25]` **Peer green-light ≠ operator consent for a managed-box mutation.** Auto-mode guard blocked a config-replace+restart on the Worldtree-team demo box that was authorized only by worldtree-dev's althing message — correctly: a persistent change to shared infra needs the *operator's* yes for that specific change, not a peer's. Surface it; don't route around the guard. (The operator then stood the whole change down — the guard's hold was the right call.)

- `[2026-06-03]` **gitea webhook to a private IP is denied by `webhook.ALLOWED_HOST_LIST`**
-  (anti-SSRF; default `external` blocks private/loopback). Symptom: delivery shows
-  `dial tcp ...: webhook can only call allowed HTTP servers`. Fix = APPEND the target net to
-  ALLOWED_HOST_LIST in gitea's app.ini (keep `external`; scope tight, never `*`/`private`) +
-  restart gitea (act_runner job containers survive a restart). gitea runs as a container on
-  ana-docker (`gitea_gitea_data` volume, `/data/gitea/conf/app.ini`).
+- `[2026-07-18]` **Fleet Gitea CI foot-guns** (3 failed soong-lab builds): the pfi-fleet runner's `node:20-slim` job image has no docker/git so `actions/checkout` + `docker/*` marketplace actions all fail; `vh` is a USER so its packages are owner-write-only (claude-bot repo-admin-collab still 401s on push/publish, and can't set repo secrets — owner-only); `GITEA_`-prefixed secret names are reserved/illegal. Fixes in → `persistent-memory.d/2026-07-18-fleet-gitea-runner-build-recipe.md`

- `[2026-06-03]` **torch-2.12 venvs need `uv pip install torchcodec`** — torchaudio 2.12
-  defaults to the TorchCodec backend for `.load`; without it, real audio I/O throws "TorchCodec
-  is required" — and it ONLY surfaces at actual conversion, NOT at import/model-load. Lesson:
-  validate real I/O, not just import, when provisioning ML engine envs. (seed-vc on torch 2.4
-  uses the legacy backend, exempt.)
+- `[2026-07-18]` **zonos-gateway local clone had NO git remote + a history unrelated to gitea's** — "committed to vh/zonos-gateway" was never pushed from that clone; two separate `git init` lineages, no merge-base. Reconcile = reset local→origin/main + overlay the changed files + push (NOT force — that erases gitea's voice-wav commits). Check `git remote -v` + `git merge-base` before assuming a clone is wired.

- `[2026-06-03]` **Backgrounding `althing-cli monitor` with an inline shell `&` (instead of
-  the Bash-tool `run_in_background`) orphans it** — it survives the shell exit, holds the
-  per-handle flock UNTRACKED (won't notify the session), and `stop-monitor` doesn't detect it.
-  Fix: find + kill the orphan PID (verify cwd=this repo / handle first — nh3-dev is shared, other
-  agents' monitors run there too), then re-arm via run_in_background. Always re-arm tracked.
+- `[2026-07-15]` `docker.service After=remote-fs.target` does NOT wait for `nofail` NFS mounts → `persistent-memory.d/2026-07-15-docker-service-after-remote-fs-target-does-not.md`

- `[2026-06-03]` **`uv pip install .` fails on SmoothKen/knn-svc** (and similar script-repos)
-  — it's analysis scripts + a poetry pyproject, no buildable package (setuptools
-  package-discovery error). Install the pyproject deps directly, don't build the "package".
+- `[2026-07-15]` The esh-docker-vm D-state/phantom-container wedge is only cleared by a host REBOOT → `persistent-memory.d/2026-07-15-the-esh-docker-vm-d-state-phantom-container.md`

- `[2026-06-02]` **Fish (fish-s2 / OpenAudio S1-mini) progressive streaming — SHELVED (sub-realtime).** Benched RTF on A6000: 0.72x (12w) / 0.82x (30w) / 0.86x (60w), **mean 0.80x = sub-realtime**, so client-side chunking would starve (same reason chatterbox-fast needs turbo's RTF>1). Root cause of the buffering (dvalin-smithy-dev deep research, verified in our code text2semantic/inference.py L600-607): Fish only chunks on `<|speaker:X|>` tags; **plain text -> batches=[whole text]** -> all semantic tokens generate before any audio (chunk_length inert). Plus a 2nd layer: kui/ASGI StreamResponse doesn't flush (header produced t=1s, delivered t=23s) -> fix = anti-buffering headers (X-Accel-Buffering:no / Transfer-Encoding:chunked) in tools/server/views.py (kept on file, not applied). A rebuild does NOT fix this (current main same logic). **STANDING REVISIT TRIGGER: when an RTX Blackwell Pro lands in the fleet -> bench fp4-quantized Fish; if RTF > ~1.5x, give it the chatterbox-fast treatment** (client-side adaptive buffer-ratchet chunker driving /v1/tts with small text pieces). Projection: fp4 (~1/4 weight bytes, memory-bound AR decode) + Blackwell (GDDR7 ~1.8TB/s vs A6000 0.77TB/s, native FP4 cores) ~ 2-3x RTF; validate fp4 voice quality (ear/ECAPA) before committing. For now Fish stays a buffered catalog entry (great for SAVED gens, not the live-audition lane).
+- `[2026-07-15]` vLLM `max-model-len` does NOT free GPU VRAM → `persistent-memory.d/2026-07-15-vllm-max-model-len-does-not-free-gpu.md`

+- `[2026-07-15]` Claude Code statusline `.cost.total_cost_usd` is per-SESSION → `persistent-memory.d/2026-07-15-claude-code-statusline-cost-total-cost-usd-is.md`

- `[2026-06-02]` **Context-priming at chunk joins (chatterbox-fast §1.6) —
-  ABANDONED (discard-cut leaks the prefix).** To give a chunk backward prosodic
-  context, prepend the prior sentence, generate `prefix+content` together, then
-  discard the prefix audio. Built + opt-in shipped (commit d707439), live-A/B'd,
-  reverted (090e70a). The kill: `generate()` returns one finished waveform with
-  NO marker for where the prefix ends, and the model renders the same prefix with
-  different timing solo vs followed-by-content — so locating the cut (generate
-  prefix solo → measure duration → snap to nearest energy-min pause within ±0.4s)
-  is a guess that left a whole clause of prefix in the output ("...without a trace
-  of sarcasm," spoken twice; operator caught it). A reliable cut needs token-level
-  boundaries (= the abandoned native-streaming arc) or per-chunk ASR/forced-
-  alignment (heavy, imperfect, eats the latency budget). → Coherence loss at joins
-  stays an ACCEPTED limitation; cold adaptive-chunk streaming judged "really good".
-  Scheduler-side work that DID land + survive: affordability-gated priming math
-  (a 2nd pass can't starve the buffer) — sound, but moot without a working cut.
+- `[2026-07-14]` MTP-on-modelopt: NO checkpoint config skips the spec-decode drafter's quant (vLLM 0.24 bug) — 4 config attempts failed before the runtime workaround → `persistent-memory.d/2026-07-14-mtp-on-modelopt-no-checkpoint-config-skips-the.md`

- `[2026-06-02]` **Native frame-level streaming on Chatterbox-TURBO — ABANDONED
-  (turbo isn't built for streaming).** Long R&D arc; record so it's not
-  re-derived. (1) The model's flow is CosyVoice2-derived but `S3GenStreamer` is
-  referenced-in-docstring-only (not implemented). (2) The lib's
-  `flow_inference(finalize=False)` is BUGGY: the lookahead trim removes
-  `pre_lookahead_len(3)*token_mel_ratio(2)=6` frames from `h` but NOT from
-  `h_masks`/conds → decoder shape mismatch (e.g. 656 vs 662). A 1-line patch
-  (`h_masks = h_masks[:, :, :-pre*ratio]` after the `h` trim) + sizing the
-  meanflow noise to the trimmed length makes finalize=False RUN. (3) BUT the
-  flow encoder uses FULL-context attention (`static_chunk_size=0`), so
-  incremental/cumulative decode is **prefix-unstable** — adding tokens
-  re-attends and shifts earlier mel (maxdiff ~0.30-0.39 vs one-shot,
-  irrespective of fixed-noise slicing or emit-margin). (4) Forcing
-  `static_chunk_size>0` on the 2 modules that carry the attr did NOT stabilize
-  it (decoding_chunk_size is a forward-arg, not settable via attribute). Verdict:
-  true sub-second frame-streaming on turbo needs deep model-attention surgery
-  with quality risk — not worth it. Matches research ("turbo+streaming
-  unsolved"; vLLM-turbo outputs noise; davidbrowne17 streaming fork is
-  BASE-only). → Use adaptive-chunking instead.
+- `[2026-07-14]` AEON's "working NVFP4+MTP RP seat" was pantheon on compressed-tensors (0% MTP accept), not a modelopt MTP proof → `persistent-memory.d/2026-07-14-aeon-s-working-nvfp4-mtp-rp-seat-was.md`

-_41 older entries archived to archival-memory.md._
+- `[2026-07-14]` NVFP4 (llm-compressor / compressed-tensors) gives NO batch-1 speedup over GGUF for the Qwen3.5 GDN-hybrid, and its MTP is 0%-accept → `persistent-memory.d/2026-07-14-nvfp4-llm-compressor-compressed-tensors-gives-no-batch.md`
+
+- `[2026-07-14]` NVFP4 spike: built the full MTP serve scaffolding BEFORE validating a plain NVFP4 serve was coherent → `persistent-memory.d/2026-07-14-nvfp4-spike-built-the-full-mtp-serve-scaffolding.md`
+
+- `[2026-07-14]` MTP graft via top-level `mtp.*` tensor names does NOT survive `AutoModelForCausalLM.from_pretrained` → `persistent-memory.d/2026-07-14-mtp-graft-via-top-level-mtp-tensor-names.md`
+
+- `[2026-07-14]` gitea "test-delivery 204" is NOT proof a webhook works → `persistent-memory.d/2026-07-14-gitea-test-delivery-204-is-not-proof-a.md`
+
+- `[2026-07-13]` Relaying a peer's diagnosis as fact without confirming it against raw data → `persistent-memory.d/2026-07-13-relaying-a-peer-s-diagnosis-as-fact-without.md`
+
+- `[2026-07-13]` `althing-cli reply <THREAD_id>` (thread id, not a MESSAGE id) → "unknown message_id"; and `reply` to your OWN message self-addresses to your handle ("replying to your own message"). Reply to a PEER's message id, or use `post --to <peer>`. Bit me several times this session.
+
+- `[2026-07-09]` **`vllm/vllm-openai:latest` crashes on Ampere IMPORT** — Blackwell-only kernels (oink/aiter,
+  `has_device_capability(100)`) die during import on the 3090/A6000. Pin **v0.23.0** on irv-ml1's Ampere GPUs.
+  (`vllm/vllm-omni:v0.18.0` has a different entrypoint — don't use it either.)
+
+- `[2026-07-09]` **Per-frame CPU SNAC decode is too slow for streaming** — per-call overhead × ~60 frames serialized
+  → RTF 2.2 (WORSE than whole-clip's 1.0). Fix = **windowed chunk decode** (every 6 frames decode a [2 ctx | 6 | 2 ctx]
+  window, emit the middle 6 → seamless, O(1)/frame, RTF ~0.97, TTFA ~0.8s).
+
+- `[2026-07-08]` Angel (allura-org/MS3.2-24b-Angel) self-quanted to NVFP4 = GARBAGE → `persistent-memory.d/2026-07-08-angel-allura-org-ms3-2-24b-angel-self.md`
+
+- `[2026-07-08]` Mistral3 + vLLM tokenizer/vision traps (serve `MS3.2-24b`, vLLM 0.24) → `persistent-memory.d/2026-07-08-mistral3-vllm-tokenizer-vision-traps-serve-ms3-2.md`
+
+- `[2026-07-08]` Pantheon-Reasoning-27B refuses dark fiction DESPITE an abliterated base → `persistent-memory.d/2026-07-08-pantheon-reasoning-27b-refuses-dark-fiction-despite-an.md`
+
+- `[2026-07-08]` Pantheon-27B MTP on vLLM compressed-tensors = 0% acceptance → `persistent-memory.d/2026-07-08-pantheon-27b-mtp-on-vllm-compressed-tensors-0.md`
+
+- `[2026-07-07]` vLLM 0.24.0 qwen3_5 LoRA application = silent no-op (#47639) → `persistent-memory.d/2026-07-07-vllm-0-24-0-qwen3-5-lora-application.md`
+
+- `[2026-07-07]` SGLang generic image can't LOAD our NVFP4 AEON → `persistent-memory.d/2026-07-07-sglang-generic-image-can-t-load-our-nvfp4.md`
+
+- `[2026-07-07]` SGLang `--lora-target-modules` CLI enum REJECTS the GDN names its own resolver asks for → `persistent-memory.d/2026-07-07-sglang-lora-target-modules-cli-enum-rejects-the.md`
+
+- `[2026-07-07]` Engine invocation footguns cost several wasted serve-bounces this session → `persistent-memory.d/2026-07-07-engine-invocation-footguns-cost-several-wasted-serve-bounces.md`
+
+- `[2026-07-04]` LiteLLM (this gateway version) mutates the SHARED deployment config in-place on per-request sampler-param merge → `persistent-memory.d/2026-07-04-litellm-this-gateway-version-mutates-the-shared-deployment.md`
+
+- `[2026-07-01]` **MTP/spec-decode on a SHARED serving model helps single-stream but HURTS
+  moderate-concurrency aggregate + silently ignores `min_p`/`logit_bias`** (qwopus `gen`: N=1 +12%,
+  N=4 −20%). Reserve for dedicated/interactive deployments.
+
+- `[2026-07-02]` **irv-ml1 `/worktank` ROOT is root-owned — lkraven can't write there (irv-ml1 sudo
+  needs a password) → stage model pulls to `/home`.** PIN THE A6000 BY UUID for training (native-CUDA
+  ordering differs vs docker; the 3090 index 0 is usually near-full → OOM). `CUDA_VISIBLE_DEVICES=GPU-<uuid>`.
+
+_107 older entries archived to archival-memory.md._
@@ -0,0 +1,43 @@
+# Make vm.overcommit_memory=1 durable on ana-ml2 (GPU inference host).
+#
+# Why: ana-ml2 runs vm.overcommit_memory=0 (heuristic) with zero swap, so the
+# CommitLimit is ~RAM/2 (~283 GB of 566 GB). The resident vLLM services already
+# commit ~224 GB of address space, leaving < 60 GB of headroom. A large model-file
+# mmap (e.g. the 50 GB NVFP4 shard during HF->native conversion, or a vLLM model
+# load) then fails with ENOMEM despite ~393 GB of RAM actually being free — the
+# kernel rejects the *commit*, not the allocation.
+#
+# overcommit_memory=1 (always overcommit) is the conventional setting for ML hosts
+# that mmap large files: the real RAM is there to back the pages, and the heuristic
+# accounting is the only thing in the way. Operator-directed permanent + durable
+# (2026-06-17). A drop-in under /etc/sysctl.d/ applies at every boot.
+#
+# Run:  scripts/elway infra-ops@ana-ml2 --playbook playbooks/ana-ml2-overcommit-memory.yaml
+# Rerunnable: a second run shows the write step `skipped` (idempotent via when:).
+
+vars:
+  dropin: /etc/sysctl.d/99-overcommit-memory.conf
+  setting: "vm.overcommit_memory = 1"
+
+steps:
+  - name: Write durable overcommit sysctl drop-in
+    # elway runs steps as the SSH user, so a shell `>` redirect can't write a
+    # root-owned path — pipe through `sudo tee` (infra-ops has NOPASSWD sudo).
+    shell: |
+      printf '# GPU inference host: large model-file mmaps (NVFP4 native convert, vLLM loads)\n# exceed the heuristic CommitLimit (overcommit=0 + zero swap) despite ample free RAM.\n# Operator-directed permanent setting 2026-06-17.\n%s\n' '{{ setting }}' | sudo tee {{ dropin }} >/dev/null
+    # Skip the write if the drop-in already holds exactly this line.
+    when: "! grep -qxF '{{ setting }}' {{ dropin }} 2>/dev/null"
+
+  - name: Apply all sysctl drop-ins now
+    shell: sudo sysctl --system >/dev/null
+    # Applying is a no-op when the runtime value already matches.
+    changed_when: "false"
+
+verify:
+  - name: Runtime vm.overcommit_memory is 1
+    shell: test "$(cat /proc/sys/vm/overcommit_memory)" = "1"
+    changed_when: "false"
+
+  - name: Drop-in file persists the setting (survives reboot)
+    shell: grep -qxF '{{ setting }}' {{ dropin }}
+    changed_when: "false"
@@ -0,0 +1,67 @@
+# Disable bearer-token auth on the prod arbo engine (irv-ml1), leaning on
+# WireGuard as the access boundary. Operator decision 2026-06-13 (relayed by
+# comfy-dev, confirmed in-session). Deliberately reverses ADR-0001's
+# "open-auth hole closed (ENGINE_TOKEN minted)" line.
+#
+# GOTCHA (why .env-only is not enough): the app's `dependencies=protected`
+# gate no-ops only when ENGINE_TOKEN is ABSENT from the container env. An
+# empty string still gates (verified 2026-06-13: ENGINE_TOKEN="" -> /workflows
+# still 401). The var is injected by TWO paths, both must be removed:
+#   1. env_file: .env            -> delete the ENGINE_TOKEN line from .env
+#   2. environment: - ENGINE_TOKEN=${ENGINE_TOKEN}  -> commented out in compose
+# With both gone the var is unset in the container and the engine serves open,
+# exactly like the dev engine on nh3-dev.
+#
+# Reversible: the pre-change .env (with the real token) is backed up to
+# .env.pre-auth-off.bak. To re-lock: restore the ENGINE_TOKEN line in .env,
+# un-comment the compose line, `compose up -d`.
+#
+# No sudo: lkraven owns the compose dir + .env and is in the docker group.
+
+vars:
+  dir: /opt/docker/compose/arbo
+
+steps:
+  - name: Back up prod .env (preserves the real ENGINE_TOKEN for re-enable)
+    shell: cp -p {{ dir }}/.env {{ dir }}/.env.pre-auth-off.bak
+    # creates: guards the FIRST backup — never clobber it on a rerun.
+    creates: "{{ dir }}/.env.pre-auth-off.bak"
+
+  - name: Remove the ENGINE_TOKEN line from .env entirely (must be ABSENT, not empty)
+    shell: sed -i '/^ENGINE_TOKEN=/d' {{ dir }}/.env
+    when: "grep -qE '^ENGINE_TOKEN=' {{ dir }}/.env"
+
+  - name: Push the corrected compose (ENGINE_TOKEN injection commented out)
+    upload:
+      src: stacks/arbo/compose.yaml
+      dest: "{{ dir }}/compose.yaml"
+      mode: "0644"
+
+  - name: Recreate the engine so ENGINE_TOKEN is absent from its env
+    shell: docker compose -f {{ dir }}/compose.yaml up -d
+
+verify:
+  - name: .env no longer defines ENGINE_TOKEN
+    shell: "! grep -qE '^ENGINE_TOKEN=' {{ dir }}/.env"
+    changed_when: "false"
+
+  - name: Backup still carries the original token (reversibility intact)
+    shell: grep -qE '^ENGINE_TOKEN=.+' {{ dir }}/.env.pre-auth-off.bak
+    changed_when: "false"
+
+  - name: ENGINE_TOKEN is ABSENT from the running container env
+    shell: "! docker exec arbo printenv ENGINE_TOKEN >/dev/null 2>&1"
+    changed_when: "false"
+
+  - name: Protected endpoint serves tokenless after warmup (auth OFF — expect HTTP 200, was 401)
+    shell: |
+      port=$(docker port arbo 8200/tcp 2>/dev/null | sed -n 's/.*:\([0-9]\+\)$/\1/p' | head -1)
+      final=000
+      for i in $(seq 1 30); do
+        code=$(curl -s -o /dev/null -w '%{http_code}' --max-time 5 "http://localhost:${port}/workflows")
+        if [ "$code" != "000" ]; then final=$code; break; fi
+        sleep 2
+      done
+      echo "tokenless GET /workflows on :${port} -> HTTP ${final}"
+      test "$final" = "200"
+    changed_when: "false"
@@ -0,0 +1,41 @@
+# Put `uv`/`uvx` on the irv-ml1-arbo Gitea Actions runner's PATH.
+#
+# WHY: the runner (`irv-ml1-arbo`, act_runner host-executor running AS lkraven
+# under systemd) inherits the bare systemd service PATH —
+#   /usr/local/bin:/usr/bin:/bin
+# — which does NOT include lkraven's `~/.local/bin` or `~/.cargo/bin`. uv is
+# installed at /home/lkraven/.local/bin/uv (login-shell only), so the CI step's
+# `uv run` failed with `uv: not found` even though uv is present on the box.
+# `/usr/local/bin` IS on the systemd PATH, so symlinking uv there makes it
+# visible to the runner. This also lets deploy.yml drop the per-run `curl|sh`
+# uv bootstrap. Operator-asked ("fix cicd"); comfy-dev (engine owner) authorized
+# the specific symlink 2026-06-16 (althing thread 01KV94VTS27B…).
+#
+# Re-apply this if the runner host is rebuilt or uv is reinstalled elsewhere.
+# Sudo because /usr/local/bin is root-owned; uv is owned by lkraven (the runner
+# identity), which is who actually executes the symlink at job time — traversal
+# of /home/lkraven works for lkraven, not for infra-ops (don't be fooled by an
+# infra-ops `env -i` exec test reporting Permission denied; that's the wrong
+# identity — verify AS lkraven).
+
+vars:
+  uv_src: /home/lkraven/.local/bin/uv
+  uvx_src: /home/lkraven/.local/bin/uvx
+
+steps:
+  - name: Symlink uv into /usr/local/bin (on the systemd PATH)
+    shell: ln -s {{ uv_src }} /usr/local/bin/uv
+    creates: /usr/local/bin/uv
+
+  - name: Symlink uvx into /usr/local/bin
+    shell: ln -s {{ uvx_src }} /usr/local/bin/uvx
+    creates: /usr/local/bin/uvx
+
+verify:
+  - name: uv resolves to the /usr/local/bin symlink under a systemd-like PATH (run AS lkraven)
+    shell: sudo -u lkraven env -i PATH=/usr/local/bin:/usr/bin:/bin sh -c 'command -v uv && uv --version'
+    changed_when: "false"
+
+  - name: uvx resolves the same way
+    shell: sudo -u lkraven env -i PATH=/usr/local/bin:/usr/bin:/bin sh -c 'command -v uvx && uvx --version'
+    changed_when: "false"
@@ -0,0 +1,121 @@
+# Deploy the LoRA training worker to irv-ml1 (arbo in-arbo LoRA training Phase 1, §4.1).
+#
+# Two-part deploy (elway upload is single-file, so code lands via rsync first):
+#   1. Stage the code (run from the eshpfi-management repo root, as infra-ops):
+#        rsync -a --delete \
+#          --exclude .venv --exclude __pycache__ --exclude state --exclude logs \
+#          services/lora-training-worker/ \
+#          infra-ops@10.100.79.3:/tmp/lora-training-worker-stage/
+#   2. Run this playbook (privileged on-box install + health-gate):
+#        scripts/elway irv-ml1 --playbook playbooks/deploy-lora-training-worker.yaml
+#
+# Idempotent: a second run shows mostly ok/skipped. Runs as infra-ops (NOPASSWD sudo on irv-ml1).
+
+vars:
+  stage_dir: /tmp/lora-training-worker-stage
+  install_dir: /opt/lora-training-worker
+  handoff_dir: /worktank/arbo/train
+  loras_publish_dir: /storetank/arbo/models/loras/trained
+  worker_user: llmuser
+  arbo_user: lkraven          # arbo container runs as uid 1000 = host lkraven
+  group: arbotrain
+  port: "8203"
+
+steps:
+  - name: Create the shared handoff group
+    shell: getent group {{ group }} >/dev/null || groupadd {{ group }}
+    sudo: true
+    changed_when: "false"   # groupadd-or-noop; report ok either way
+
+  - name: Add the worker user (llmuser) to the handoff group
+    shell: id -nG {{ worker_user }} | tr ' ' '\n' | grep -qx {{ group }} || usermod -aG {{ group }} {{ worker_user }}
+    sudo: true
+    when: "! id -nG {{ worker_user }} | tr ' ' '\\n' | grep -qx {{ group }}"
+
+  - name: Add the arbo-container user (lkraven) to the handoff group
+    shell: usermod -aG {{ group }} {{ arbo_user }}
+    sudo: true
+    when: "! id -nG {{ arbo_user }} | tr ' ' '\\n' | grep -qx {{ group }}"
+
+  - name: Create the shared handoff dir (group-owned, setgid 2770)
+    shell: mkdir -p {{ handoff_dir }}
+    sudo: true
+    creates: "{{ handoff_dir }}"
+
+  - name: Set handoff dir group + setgid perms
+    shell: chgrp {{ group }} {{ handoff_dir }} && chmod 2770 {{ handoff_dir }}
+    sudo: true
+    changed_when: "false"
+
+  - name: Create the Phase-2 LoRA publish dir (ComfyUI loras/trained, group-writable)
+    # 2775 (not 2770): world-readable + traversable so ComfyUI (uid 1025 comfytoo) can list +
+    # load; group arbotrain + group-WRITE so the worker (llmuser) can publish into it. setgid
+    # propagates the group to per-train subdirs (the Phase-1 group-write lesson).
+    shell: mkdir -p {{ loras_publish_dir }} && chgrp {{ group }} {{ loras_publish_dir }} && chmod 2775 {{ loras_publish_dir }}
+    sudo: true
+    changed_when: "false"
+
+  - name: Create the install dir owned by the worker user
+    shell: mkdir -p {{ install_dir }} && chown {{ worker_user }}:{{ worker_user }} {{ install_dir }}
+    sudo: true
+    creates: "{{ install_dir }}"
+
+  - name: Sync staged code into the install dir (worker-owned)
+    shell: >
+      rsync -a --delete
+      --exclude .venv --exclude __pycache__ --exclude state --exclude logs
+      {{ stage_dir }}/ {{ install_dir }}/
+      && chown -R {{ worker_user }}:{{ worker_user }} {{ install_dir }}
+    sudo: true
+
+  - name: Ensure state + logs dirs exist (worker-writable)
+    shell: mkdir -p {{ install_dir }}/state {{ install_dir }}/logs && chown {{ worker_user }}:{{ worker_user }} {{ install_dir }}/state {{ install_dir }}/logs
+    sudo: true
+    creates: "{{ install_dir }}/logs"
+
+  - name: Build the worker venv + install deps (as llmuser; prefer uv, fall back to python3 -m venv)
+    shell: >
+      sudo -u {{ worker_user }} bash -lc '
+      cd {{ install_dir }} &&
+      if command -v uv >/dev/null 2>&1; then
+        uv venv .venv && uv pip install --python .venv/bin/python . ;
+      else
+        python3 -m venv .venv && .venv/bin/pip install -q --upgrade pip && .venv/bin/pip install -q . ;
+      fi'
+    sudo: true
+    creates: "{{ install_dir }}/.venv/bin/uvicorn"
+
+  - name: Install the systemd unit
+    upload:
+      src: services/lora-training-worker/lora-training-worker.service
+      dest: /etc/systemd/system/lora-training-worker.service
+      mode: "0644"
+    sudo: true
+
+  - name: Reload systemd + enable the worker
+    shell: systemctl daemon-reload && systemctl enable lora-training-worker.service
+    sudo: true
+    changed_when: "false"
+
+  - name: Restart the worker to pick up the synced code
+    shell: systemctl restart lora-training-worker.service
+    sudo: true
+    changed_when: "false"
+
+  - name: Give the service a moment to bind
+    shell: sleep 3
+    changed_when: "false"
+
+verify:
+  - name: Worker health endpoint responds ok
+    shell: curl -fsS http://127.0.0.1:{{ port }}/healthz
+    changed_when: "false"
+
+  - name: gpu-status reports both devices
+    shell: curl -fsS http://127.0.0.1:{{ port }}/gpu-status | grep -q '"index"'
+    changed_when: "false"
+
+  - name: Service is enabled + active
+    shell: systemctl is-active lora-training-worker.service
+    sudo: true
+    changed_when: "false"
@@ -0,0 +1,140 @@
+# Deploy OmniVoice (https://github.com/k2-fsa/OmniVoice) to irv-ml1, GPU 0
+# (RTX 3090). Apache-2.0 zero-shot multilingual voice-cloning TTS, served
+# behind our OWN FastAPI wrapper (app.py): batch /v1/audio/speech plus a
+# streaming /tts driven by the vendored buffer-ratchet scheduler.
+#
+# Builds the image locally from stacks/omnivoice/Dockerfile (CUDA 12.8 +
+# torch 2.8.0 + omnivoice from PyPI + vendored scheduler.py/sanitize.py),
+# stages the build context under /opt/docker/compose/omnivoice/, brings it
+# up, and waits for /healthz on :8199.
+#
+# First run is slow: ~5-10 min docker build + a one-time HF weight pre-warm
+# (k2-fsa/OmniVoice) on first container start (entrypoint.sh). The wait loop
+# below allows up to ~20 min for build-then-up + pre-warm.
+#
+# Usage:
+#   scripts/elway irv-ml1 --playbook playbooks/deploy-omnivoice.yaml
+#
+# Idempotent — every step is creates-/when-gated; rerun is safe.
+
+vars:
+  compose_dir: /opt/docker/compose/omnivoice
+  cache_dir: /worktank/omnivoice/hf_cache
+  voices_dir: /worktank/omnivoice/voices
+  host_port: "8199"
+
+steps:
+  # ── host-side dirs ──────────────────────────────────────────────────
+
+  - name: Ensure /worktank/omnivoice root exists (one-time, sudo)
+    shell: mkdir -p /worktank/omnivoice
+    sudo: true
+    creates: /worktank/omnivoice
+
+  - name: Chown /worktank/omnivoice to lkraven
+    shell: chown lkraven:lkraven /worktank/omnivoice
+    sudo: true
+    when: '[ "$(stat -c %U /worktank/omnivoice)" != lkraven ]'
+
+  - name: Ensure cache dir exists
+    shell: mkdir -p {{ cache_dir }}
+    creates: "{{ cache_dir }}"
+
+  - name: Ensure voices dir exists
+    shell: mkdir -p {{ voices_dir }}
+    creates: "{{ voices_dir }}"
+
+  - name: Ensure compose dir exists
+    shell: mkdir -p {{ compose_dir }}
+    creates: "{{ compose_dir }}"
+
+  # ── deploy build context (compose, dockerfile, entrypoint, env) ───────
+
+  - name: Upload compose.yaml
+    upload:
+      src: stacks/omnivoice/compose.yaml
+      dest: "{{ compose_dir }}/compose.yaml"
+      mode: "0644"
+
+  - name: Upload Dockerfile
+    upload:
+      src: stacks/omnivoice/Dockerfile
+      dest: "{{ compose_dir }}/Dockerfile"
+      mode: "0644"
+
+  - name: Upload app.py (batch + streaming FastAPI wrapper)
+    upload:
+      src: stacks/omnivoice/app.py
+      dest: "{{ compose_dir }}/app.py"
+      mode: "0644"
+
+  - name: Upload scheduler.py (vendored buffer-ratchet streaming scheduler)
+    upload:
+      src: stacks/omnivoice/scheduler.py
+      dest: "{{ compose_dir }}/scheduler.py"
+      mode: "0644"
+
+  - name: Upload sanitize.py (language-safe TTS text sanitizer)
+    upload:
+      src: stacks/omnivoice/sanitize.py
+      dest: "{{ compose_dir }}/sanitize.py"
+      mode: "0644"
+
+  - name: Stage chatterbox reference voices for cloning (skip _*.wav artifacts)
+    shell: |
+      set -e
+      mkdir -p {{ voices_dir }}
+      docker exec chatterbox-fast sh -c 'ls /refs/*.wav' | while read -r f; do
+        b=$(basename "$f")
+        case "$b" in _*) continue;; esac
+        docker cp "chatterbox-fast:$f" "{{ voices_dir }}/$b"
+      done
+      echo "staged:"; ls {{ voices_dir }}
+    # Skip if already staged (Emily.wav is a proxy for "voices present").
+    when: "[ ! -f {{ voices_dir }}/Emily.wav ]"
+
+  - name: Upload entrypoint.sh
+    upload:
+      src: stacks/omnivoice/entrypoint.sh
+      dest: "{{ compose_dir }}/entrypoint.sh"
+      mode: "0755"
+
+  - name: Seed .env from template (only if absent)
+    upload:
+      src: stacks/omnivoice/.env.example
+      dest: "{{ compose_dir }}/.env"
+      mode: "0644"
+    when: "[ ! -f {{ compose_dir }}/.env ]"
+
+  # ── build + bring up ────────────────────────────────────────────────
+
+  - name: docker compose build (~5-10 min first time; cached after)
+    shell: |
+      set -o pipefail
+      cd {{ compose_dir }} && docker compose build --progress=plain 2>&1 \
+        | grep -vE '^#[0-9]+ [0-9.]+ (Downloading|Collecting|Requirement|Using cached|Installing collected|Successfully (installed|built)|━|Resolved|Prepared|Built)'
+
+  - name: docker compose up -d
+    shell: cd {{ compose_dir }} && docker compose up -d
+
+  - name: Wait for /healthz (allow ~25 min for weight + Whisper pre-warm + voice cloning)
+    shell: |
+      for i in $(seq 1 300); do
+        curl -sf -o /dev/null --max-time 3 http://localhost:{{ host_port }}/healthz && exit 0
+        sleep 5
+      done
+      exit 1
+    changed_when: "false"
+
+verify:
+  - name: /healthz returns 200
+    shell: curl -sf -o /dev/null http://localhost:{{ host_port }}/healthz
+    changed_when: "false"
+
+  - name: /v1/audio/voices lists the reused chatterbox voices
+    shell: curl -sf http://localhost:{{ host_port }}/v1/audio/voices | grep -q '"voices"'
+    changed_when: "false"
+
+  - name: Container is running
+    shell: docker inspect omnivoice --format '{{.State.Status}}' | grep -q running
+    changed_when: "false"
@@ -12,8 +12,20 @@
 #   - fstab: defaults -> defaults,_netdev,nofail  (keeps `hard`)
 #       _netdev : order mount after network-online.target
 #       nofail  : NAS-down at boot doesn't wedge boot / kill DNS
-#   - docker.service drop-in: After=remote-fs.target so Docker starts
-#     after the NFS mounts have completed.
+#   - fstab: + x-systemd.before=docker.service,x-systemd.mount-timeout=30
+#       Puts Before=docker.service directly on each generated .mount unit
+#       so Docker waits for the ACTUAL mounts; mount-timeout bounds the
+#       wait if the NAS is down at boot.
+#   - docker.service drop-in: After=remote-fs.target (kept as a weaker
+#     belt-and-suspenders layer).
+#
+# WHY the drop-in alone was NOT enough (2026-07-14 reboot): `nofail`
+# removes a mount from remote-fs.target's blocking set, so ordering
+# Docker `After=remote-fs.target` does not actually wait for the nofail
+# NFS mounts -> paperless still lost the race and Exited(255) on reboot.
+# The load-bearing fix is the DIRECT mount->docker ordering from the
+# fstab `x-systemd.before` option. Verify with:
+#   systemctl show docker -p After | tr ' ' '\n' | grep mnt-   # lists all 4
 #
 # Idempotent: re-runs show ok/skipped. Does NOT reboot — the real test
 # is the next reboot, run that separately.
@@ -37,6 +49,16 @@ steps:
    # Run only if at least one unfixed NFS line remains.
    when: "grep -qE '^10\\.0\\.50\\.50:.* nfs defaults ' /etc/fstab"

+  - name: Order each NFS mount before docker.service (direct dep; nofail-safe)
+    # THE load-bearing fix. remote-fs.target ordering (below) is defeated
+    # by `nofail` (the mount drops out of that target's blocking set).
+    # x-systemd.before=docker.service injects Before=docker.service onto
+    # each generated .mount unit, so Docker genuinely waits for the mounts.
+    shell: sed -i -E '/^10\.0\.50\.50:/{/x-systemd.before/!s/(_netdev,nofail)/\1,x-systemd.before=docker.service,x-systemd.mount-timeout=30/}' /etc/fstab
+    sudo: true
+    # Run only if an NFS line with _netdev,nofail still lacks the ordering.
+    when: "grep -E '^10\\.0\\.50\\.50:.*_netdev,nofail' /etc/fstab | grep -qv x-systemd.before"
+
  - name: Install docker.service drop-in to order after remote-fs.target
    # Use a DISTINCT filename — esh-docker-vm already ships an
    # override.conf (dockerd ExecStart/containerd socket); systemd merges
@@ -57,14 +79,22 @@ steps:
    changed_when: "false"

 verify:
-  - name: All 4 NFS lines now carry _netdev,nofail
-    shell: test "$(grep -cE '^10\.0\.50\.50:.* nfs defaults,_netdev,nofail ' /etc/fstab)" -eq 4
+  - name: All 4 NFS lines carry _netdev,nofail
+    shell: test "$(grep -cE '^10\.0\.50\.50:.*nfs defaults,_netdev,nofail' /etc/fstab)" -eq 4
+    changed_when: "false"
+
+  - name: All 4 NFS lines carry x-systemd.before=docker.service
+    shell: test "$(grep -cE '^10\.0\.50\.50:.*x-systemd.before=docker.service' /etc/fstab)" -eq 4
    changed_when: "false"

  - name: fstab parses cleanly (findmnt --verify, no fatal errors)
    shell: findmnt --verify >/dev/null
    changed_when: "false"

-  - name: Docker is ordered after remote-fs.target
+  - name: Docker is ordered after the actual NFS mount units (the real fix)
+    shell: systemctl show docker -p After | tr ' ' '\n' | grep -q '^mnt-documents.mount$'
+    changed_when: "false"
+
+  - name: Docker is also ordered after remote-fs.target (belt-and-suspenders)
    shell: systemctl show docker -p After | grep -q remote-fs.target
    changed_when: "false"
@@ -0,0 +1,130 @@
+# Install the `pi` coding agent (earendil-works) on nh3-extdev and wire every
+# /opt/externs/<client> workspace to GLM 5.2 via the litellm gateway.
+#
+# Context: nh3-extdev is SUDO-LESS (no root, no apt, no docker). So Node is
+# installed user-level from the official static tarball (checksum-verified),
+# pi is installed `-g` into that user-space prefix, and each client gets an
+# ISOLATED pi config dir via PI_CODING_AGENT_DIR (set by its run-pi.sh launcher).
+#
+# Idempotent: a second run shows mostly skip/ok. Rerunnable to add a client —
+# append its name to `clients` (its workspace dir + secrets.env with an
+# EXTERNS_<NAME>_GLM_KEY must already exist; workspace scaffolding is separate).
+#
+#   scripts/elway nh3-extdev --playbook playbooks/install-pi-nh3-extdev.yaml
+#
+# pi config layout (authoritative, from the installed package):
+#   - PI_CODING_AGENT_DIR overrides the agent dir (default ~/.pi/agent)
+#   - $DIR/models.json   : providers.<name>.{baseUrl, api, apiKey:"$ENV", models[]}
+#   - $DIR/settings.json : defaultProvider + defaultModel (bare id)
+# The per-client GLM key lives in <workspace>/secrets.env (600, gitignored),
+# referenced indirectly so the key never lands in models.json.
+
+vars:
+  node_ver: v22.23.0          # latest v22 LTS "Jod"; matches pi engine floor >=22.19.0
+  node_arch: linux-x64
+  node_root: /home/infra-ops/.local   # absolute (not $HOME — elway doesn't shell-expand creates:); identity is always infra-ops
+  gateway: http://10.250.50.70:4000/v1
+  externs: /opt/externs
+  clients: gbcnc surefire svsconstruction
+
+steps:
+  - name: Download + verify + extract user-level Node
+    shell: |
+      set -euo pipefail
+      DEST="{{ node_root }}"; DIR="$DEST/node-{{ node_ver }}-{{ node_arch }}"
+      mkdir -p "$DEST"; cd /tmp
+      curl -fsSLO "https://nodejs.org/dist/{{ node_ver }}/node-{{ node_ver }}-{{ node_arch }}.tar.xz"
+      curl -fsSL  "https://nodejs.org/dist/{{ node_ver }}/SHASUMS256.txt" -o SHASUMS256.txt
+      grep " node-{{ node_ver }}-{{ node_arch }}.tar.xz$" SHASUMS256.txt | sha256sum -c -
+      tar -xJf "node-{{ node_ver }}-{{ node_arch }}.tar.xz" -C "$DEST"
+      rm -f "node-{{ node_ver }}-{{ node_arch }}.tar.xz" SHASUMS256.txt
+    # Tier-1 idempotency: skip the whole download if the node binary is already there.
+    creates: "{{ node_root }}/node-{{ node_ver }}-{{ node_arch }}/bin/node"
+
+  - name: Wire node/pi onto PATH for login + interactive shells
+    shell: |
+      set -euo pipefail
+      LINE='export PATH="$HOME/.local/node-{{ node_ver }}-{{ node_arch }}/bin:$PATH"'
+      for RC in "$HOME/.profile" "$HOME/.bashrc"; do
+        grep -qF "$LINE" "$RC" 2>/dev/null || {
+          printf '\n# >>> pi/node user-level PATH >>>\n%s\n# <<< pi/node user-level PATH <<<\n' "$LINE" >> "$RC"
+        }
+      done
+    when: "! grep -qF 'pi/node user-level PATH' $HOME/.bashrc 2>/dev/null"
+
+  - name: Install the latest pi coding agent into the user prefix
+    shell: |
+      set -euo pipefail
+      export PATH="{{ node_root }}/node-{{ node_ver }}-{{ node_arch }}/bin:$PATH"
+      npm install -g @earendil-works/pi-coding-agent
+    creates: "{{ node_root }}/node-{{ node_ver }}-{{ node_arch }}/bin/pi"
+
+  - name: Wire each client workspace to GLM 5.2 (isolated config + scoped key)
+    shell: |
+      set -euo pipefail
+      for c in {{ clients }}; do
+        W="{{ externs }}/$c"; PI="$W/.pi"
+        [ -d "$PI" ] || { echo "!! $c: missing $PI (scaffold first)"; exit 1; }
+        KEYVAR=$(grep -oE '^EXTERNS_[A-Z0-9_]+_GLM_KEY' "$W/secrets.env" | head -1)
+        [ -n "$KEYVAR" ] || { echo "!! $c: no EXTERNS_*_GLM_KEY in secrets.env"; exit 1; }
+        cat > "$PI/models.json" <<JSON
+      {
+        "providers": {
+          "litellm-glm": {
+            "baseUrl": "{{ gateway }}",
+            "api": "openai-completions",
+            "apiKey": "\$$KEYVAR",
+            "models": [
+              { "id": "glm-5.2", "name": "GLM 5.2 (litellm/z.ai)" },
+              { "id": "glm-5.2-reasoning", "name": "GLM 5.2 reasoning (litellm/z.ai)" }
+            ]
+          }
+        }
+      }
+      JSON
+        cat > "$PI/settings.json" <<'JSON'
+      {
+        "defaultProvider": "litellm-glm",
+        "defaultModel": "glm-5.2"
+      }
+      JSON
+        cat > "$W/run-pi.sh" <<'SH'
+      #!/usr/bin/env bash
+      # Launch pi for this client: isolated config dir + scoped GLM key + repo cwd.
+      set -euo pipefail
+      HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+      set -a; . "$HERE/secrets.env"; set +a
+      export PI_CODING_AGENT_DIR="$HERE/.pi"
+      cd "$HERE/repo"
+      exec pi "$@"
+      SH
+        chmod 600 "$PI/models.json" "$PI/settings.json"
+        chmod 700 "$W/run-pi.sh"
+        rm -f "$PI/config.example"
+      done
+    # Re-write is deterministic; skip when gbcnc is already wired to the gateway
+    # AND its launcher exists (proxy for "all three wired").
+    when: "! ( grep -qF '{{ gateway }}' {{ externs }}/gbcnc/.pi/models.json 2>/dev/null && test -x {{ externs }}/gbcnc/run-pi.sh )"
+
+verify:
+  - name: pi binary reports a version
+    shell: |
+      export PATH="{{ node_root }}/node-{{ node_ver }}-{{ node_arch }}/bin:$PATH"
+      pi --version
+    changed_when: "false"
+
+  - name: each client has models.json + settings.json + run-pi.sh
+    shell: |
+      for c in {{ clients }}; do
+        W="{{ externs }}/$c"
+        test -s "$W/.pi/models.json" && test -s "$W/.pi/settings.json" && test -x "$W/run-pi.sh" \
+          || { echo "$c incomplete"; exit 1; }
+      done
+    changed_when: "false"
+
+  - name: gbcnc resolves glm-5.2 through the gateway (live round-trip)
+    shell: |
+      export PATH="{{ node_root }}/node-{{ node_ver }}-{{ node_arch }}/bin:$PATH"
+      {{ externs }}/gbcnc/run-pi.sh --no-tools --no-session --approve -p "Reply with exactly: PI_GLM_OK" \
+        | grep -qF PI_GLM_OK
+    changed_when: "false"
@@ -0,0 +1,72 @@
+# Displace mistral-small-4 (heretic) on ana-ml2 GPU 0 and serve
+# bjk110/Qwen3.5-122B-A10B-abliterated-NVFP4 (text-only) as the new `gen` model
+# (operator 2026-06-19). Weights pre-staged at /tank/aimodels/qwen3.5-122b-a10b-nvfp4
+# (incl. the repo's serving/entrypoint.sh + vllm_patches/ that the compose mounts).
+#
+# ⚠️ Downing mistral-small-4 takes down the Worldtree CHARACTER backend (vision-intact)
+# until it's repointed — operator-acknowledged. REVERT = down qwen, up -d the heretic.
+#
+#   scripts/elway ana-ml2 --playbook playbooks/serve-qwen3.5-122b.yaml
+
+vars:
+  compose_dir: /opt/docker/compose/qwen3.5-122b
+  heretic_dir: /opt/docker/compose/mistral-small-4-heretic
+  model_dir: /tank/aimodels/qwen3.5-122b-a10b-nvfp4
+  host_port: "8013"
+
+steps:
+  - name: Verify NVFP4 weights + the repo's patch/entrypoint are staged
+    shell: |
+      test -f {{ model_dir }}/model.safetensors.index.json \
+        && test -f {{ model_dir }}/serving/entrypoint.sh \
+        && test -f {{ model_dir }}/vllm_patches/patch_qwen35_moe_text.py
+    changed_when: "false"
+
+  - name: Ensure vLLM compile-cache dir exists (writable)
+    shell: mkdir -p {{ model_dir }}/.cache/vllm
+    creates: "{{ model_dir }}/.cache/vllm"
+
+  - name: Ensure compose dir exists
+    shell: mkdir -p {{ compose_dir }}
+    creates: "{{ compose_dir }}"
+
+  - name: Upload compose.yaml
+    upload:
+      src: stacks/qwen3.5-122b/compose.yaml
+      dest: "{{ compose_dir }}/compose.yaml"
+      mode: "0644"
+
+  - name: Seed .env from template (only if absent)
+    upload:
+      src: stacks/qwen3.5-122b/.env.example
+      dest: "{{ compose_dir }}/.env"
+      mode: "0644"
+    when: "[ ! -f {{ compose_dir }}/.env ]"
+
+  - name: Displace — down mistral-small-4-heretic (frees GPU 0; no-op if down)
+    shell: cd {{ heretic_dir }} && docker compose down
+
+  - name: Bring up qwen3.5-122b
+    shell: cd {{ compose_dir }} && docker compose up -d
+
+  - name: Wait for vLLM /health (allow ~15 min for patch + NVFP4 MoE load + warmup)
+    shell: |
+      for i in $(seq 1 180); do
+        curl -sf -o /dev/null --max-time 3 http://localhost:{{ host_port }}/health && exit 0
+        sleep 5
+      done
+      exit 1
+    changed_when: "false"
+
+verify:
+  - name: /health returns 200
+    shell: curl -sf -o /dev/null http://localhost:{{ host_port }}/health
+    changed_when: "false"
+
+  - name: served model id is qwen3.5-122-a10b
+    shell: curl -sf http://localhost:{{ host_port }}/v1/models | grep -q qwen3.5-122-a10b
+    changed_when: "false"
+
+  - name: container running
+    shell: docker inspect vllm-qwen35-122b --format '{{.State.Status}}' | grep -q running
+    changed_when: "false"
@@ -0,0 +1,65 @@
+# Add the ratatoskr memory-plane provider endpoint to the personal Worldtree's
+# Bifrost client allowlist, so a consumer may BIND it at session-create.
+#
+# Worldtree gates `bifrost.endpoint_url` against BIFROST_CLIENT_ALLOWED_HOSTS
+# (host:port CSV in /opt/worldtree-personal/.env). The affect plane :8390 was
+# listed during its deploy; the memory plane :8391 (ratatoskr-memory-provider
+# on nh3-dev) needs appending — otherwise POST /sessions 422s
+# (`endpoint_url must be HTTPS or match BIFROST_CLIENT_ALLOWED_HOSTS`) before
+# any handshake fires. See the eshpfi memory note `reference_bifrost_plane_wiring`.
+#
+# Idempotent + rerunnable: guards are sudo-free (live container env via the
+# docker group; backup via `test -e`); the append self-guards inside its
+# sudo bash -c; the recreate skips when the live env already carries the host.
+# Surgical: recreates ONLY worldtree-api (the validator); matrix is untouched
+# and picks up the value on its next natural redeploy. `--pull never` uses the
+# local pinned image so the recreate needs no gitea registry auth.
+#
+# CRITICAL pin-preservation: WORLDTREE_IMAGE is injected by the Worldtree CI/CD
+# at deploy time, NOT stored in .env, so a bare `compose up` falls back to the
+# compose default `:latest` — a STALE locally-cached build whose stricter config
+# validation crash-blocks startup on this instance's agent-profile drift (agents
+# reference removed LLM profile qwen3.6-35-a3b-heretic). The recreate step below
+# therefore re-derives the live pin from the untouched matrix sibling and passes
+# it explicitly. (Learned the hard way 2026-06-15 — a pinless recreate took the
+# personal API down for ~1 min until restored on the correct pin.)
+#
+#   scripts/elway corviduo-dev --playbook playbooks/wire-personal-worldtree-memory-allowlist.yaml
+
+vars:
+  add_host: "10.100.10.50:8391"
+  proj_dir: /opt/worldtree-personal
+  env_file: /opt/worldtree-personal/.env
+  api_service: worldtree-api
+  api_container: worldtree-personal-worldtree-api-1
+
+steps:
+  - name: Back up .env before editing the allowlist
+    shell: cp /opt/worldtree-personal/.env /opt/worldtree-personal/.env.bak-pre-memory-allowlist
+    sudo: true
+    creates: /opt/worldtree-personal/.env.bak-pre-memory-allowlist
+
+  - name: Append the memory endpoint to BIFROST_CLIENT_ALLOWED_HOSTS (self-guarded)
+    shell: >-
+      grep -q '{{ add_host }}' {{ env_file }}
+      || sed -i '/^BIFROST_CLIENT_ALLOWED_HOSTS=/ s/$/,{{ add_host }}/' {{ env_file }}
+    sudo: true
+
+  - name: Recreate worldtree-api so it loads the new allowlist (skip if already live)
+    when: "! docker exec {{ api_container }} printenv BIFROST_CLIENT_ALLOWED_HOSTS 2>/dev/null | grep -q '{{ add_host }}'"
+    # Re-derive the live image pin from the untouched matrix sibling so the
+    # recreate can't fall back to the crash-blocking :latest default.
+    shell: >-
+      WORLDTREE_IMAGE="$(docker inspect worldtree-personal-worldtree-matrix-1 --format '{{.Config.Image}}')"
+      docker compose --project-directory {{ proj_dir }} -f {{ proj_dir }}/compose.yaml
+      -p worldtree-personal up -d --pull never --force-recreate {{ api_service }}
+    sudo: true
+
+verify:
+  - name: Live worldtree-api env carries the memory endpoint
+    shell: docker exec {{ api_container }} printenv BIFROST_CLIENT_ALLOWED_HOSTS | grep -q '{{ add_host }}'
+    changed_when: "false"
+
+  - name: worldtree-api container is running
+    shell: docker ps --filter name={{ api_container }} --filter status=running -q | grep -q .
+    changed_when: "false"
@@ -0,0 +1,19 @@
+#!/usr/bin/env bash
+# backup-freshness-alert.sh — daily wrapper around check-backup-freshness.sh.
+# Runs the check; on any stale/down layer (exit!=0) posts an althing alert to
+# infra-ops so the silent-failure class (the 2026-05-06→06-20 ana outage that
+# went unnoticed ~6.5 weeks) can't recur. Installed as a systemd user timer on
+# nh3-dev via scripts/install-backup-freshness-timer.sh.
+set -uo pipefail
+REPO=/home/lkraven/development/eshpfi-management
+ALTHING=/home/lkraven/.local/bin/althing-cli
+
+out=$("$REPO/scripts/check-backup-freshness.sh" 2>&1); rc=$?
+printf '%s\n' "$out"
+
+if [ "$rc" -ne 0 ]; then
+  printf 'Automated daily backup-freshness check found STALE or DOWN backup layer(s) on the PFI fleet.\nRunbook: docs/runbooks/backups.md (topology, 2-min check, rest-server-ana recovery).\n\n%s\n' "$out" \
+    | "$ALTHING" post --to infra-ops --subject "🔴 Backup freshness ALERT ($(date '+%Y-%m-%d'))" 2>&1 \
+    || echo "WARN: althing alert post failed — the check still ran (exit $rc); investigate manually."
+fi
+exit "$rc"
@@ -0,0 +1,69 @@
+#!/usr/bin/env bash
+# check-backup-freshness.sh — the "are we actually backed up?" check.
+#
+# Walks every backup layer and flags anything whose newest snapshot is older
+# than the threshold (default 48h) or any down endpoint. Prints a report;
+# exits 0 if everything is fresh, 1 if anything is stale/down. Designed to be
+# run by a daily timer that alerts on non-zero exit (see
+# scripts/install-backup-freshness-timer.sh), or by hand anytime.
+#
+# Companion to docs/runbooks/backups.md. Read-only — only SSH stat/curl.
+#
+#   BACKUP_MAX_AGE_HOURS=48 scripts/check-backup-freshness.sh
+set -uo pipefail
+
+MAX_AGE_H="${BACKUP_MAX_AGE_HOURS:-48}"
+SSH="ssh -o ConnectTimeout=8 -o BatchMode=yes"
+now=$(date +%s)
+stale=() ; fresh=() ; errors=()
+
+# newest snapshot epoch under a remote glob (echoes epoch or empty)
+newest_epoch() { # $1=host  $2=glob
+  $SSH "$1" "stat -c %Y $2 2>/dev/null | sort -n | tail -1" 2>/dev/null
+}
+report() { # $1=label  $2=epoch("" = none)
+  local label="$1" ep="$2"
+  if [ -z "$ep" ]; then stale+=("$label: NO SNAPSHOTS / unreachable"); return; fi
+  local age=$(( (now - ep) / 3600 ))
+  local when; when=$(date -d "@$ep" '+%Y-%m-%d %H:%M' 2>/dev/null)
+  if [ "$age" -gt "$MAX_AGE_H" ]; then stale+=("$label: ${age}h old (newest $when)")
+  else fresh+=("$label: ${age}h old (newest $when)"); fi
+}
+
+echo "=== Backup freshness (threshold ${MAX_AGE_H}h) — $(date '+%Y-%m-%d %H:%M %Z') ==="
+
+# --- Layer: restic file+DB, ANA side (rest-server-ana) ---
+for c in ana-docker ana-ml2 esh-docker-vm esh-vm-db vm-esh-nas; do
+  report "restic/ana/$c" "$(newest_epoch ana-nas "/mnt/backup/restic/repo/ana/$c/snapshots/*")"
+done
+# --- Layer: restic file+DB, NH3 side (rest-server-nh3) ---
+for c in irv-ml1 nh3-docker; do
+  report "restic/nh3/$c" "$(newest_epoch nh3-nas "/volume1/Backup/restic/$c/snapshots/*")"
+done
+# --- Layer: PBS VM images (newest per guest, all namespaces) ---
+pbs=$($SSH pbs-ana 'for ns in /mnt/pbs-datastore/ns/*/; do n=$(basename "$ns")
+  for d in vm ct; do for g in "$ns$d"/*/; do [ -d "$g" ] || continue
+    nb=$(ls -d "$g"20*T* 2>/dev/null | sort | tail -1)
+    [ -n "$nb" ] && echo "$n/$d/$(basename "$g") $(stat -c %Y "$nb")"
+  done; done; done' 2>/dev/null)
+if [ -z "$pbs" ]; then errors+=("PBS-ANA: unreachable or no snapshots"); else
+  while read -r guest ep; do [ -n "$guest" ] && report "pbs/$guest" "$ep"; done <<<"$pbs"
+fi
+
+# --- rest-server endpoint health (401 = up & serving) ---
+for ep in "rest-server-ana http://10.250.50.70:8000/" "rest-server-nh3 http://10.100.50.50:8000/"; do
+  set -- $ep
+  code=$(curl -s -o /dev/null -w '%{http_code}' --max-time 6 "$2" 2>/dev/null)
+  [ "$code" = "401" ] && fresh+=("$1: up (401)") || stale+=("$1: endpoint code=$code (expected 401)")
+done
+
+echo
+echo "FRESH (${#fresh[@]}):"; printf '  ✅ %s\n' "${fresh[@]}"
+if [ "${#stale[@]}" -gt 0 ] || [ "${#errors[@]}" -gt 0 ]; then
+  echo; echo "STALE / PROBLEMS (${#stale[@]}+${#errors[@]}):"
+  printf '  🔴 %s\n' "${stale[@]}" "${errors[@]}"
+  echo; echo "RESULT: STALE — see docs/runbooks/backups.md"
+  exit 1
+fi
+echo; echo "RESULT: all backups fresh"
+exit 0
@@ -0,0 +1,37 @@
+#!/usr/bin/env bash
+# install-backup-freshness-timer.sh — install/refresh the daily backup-freshness
+# alert as a systemd USER timer on nh3-dev (the only host with SSH to all backup
+# stores + althing-cli). Idempotent; re-run after editing the wrapper/check.
+# Requires linger (loginctl enable-linger lkraven) so it fires without a login.
+set -euo pipefail
+UNIT_DIR="$HOME/.config/systemd/user"
+REPO=/home/lkraven/development/eshpfi-management
+mkdir -p "$UNIT_DIR"
+
+cat > "$UNIT_DIR/backup-freshness.service" <<EOF
+[Unit]
+Description=Fleet backup freshness check + althing alert
+After=network-online.target
+
+[Service]
+Type=oneshot
+Environment=ALTHING_HANDLE=infra-ops
+ExecStart=$REPO/scripts/backup-freshness-alert.sh
+EOF
+
+cat > "$UNIT_DIR/backup-freshness.timer" <<EOF
+[Unit]
+Description=Daily fleet backup freshness check (08:00)
+
+[Timer]
+OnCalendar=*-*-* 08:00:00
+Persistent=true
+
+[Install]
+WantedBy=timers.target
+EOF
+
+systemctl --user daemon-reload
+systemctl --user enable --now backup-freshness.timer
+echo "installed. next run:"
+systemctl --user list-timers backup-freshness.timer --all --no-pager
@@ -0,0 +1,46 @@
+#!/usr/bin/env bash
+# Hourly off-box snapshot of ~/development -> nh3-nas via rsync --link-dest
+# hardlink snapshots. Penance for the 2026-07-12 soong-lab clobber: uncommitted
+# dev work now has an hourly, versioned, off-box safety net. Secrets + heavy
+# reconstructable dirs are excluded. Snapshots are timestamped dirs on the NAS;
+# unchanged files hardlink to the previous snapshot (space-efficient). Retention:
+# newest 48 hourly snapshots.
+set -uo pipefail
+
+SRC="$HOME/development/"
+DEST_HOST="nh3-nas"
+DEST_BASE="/volume1/Backup/nh3-dev-development"
+STAMP="$(date +%Y-%m-%d_%H%M)"
+LOG="$HOME/.config/dev-backup/dev-backup.log"
+
+exec >>"$LOG" 2>&1
+echo "=== $(date -Is) snapshot $STAMP start ==="
+
+# previous snapshot for hardlink dedup
+PREV="$(ssh -o ConnectTimeout=15 -o BatchMode=yes "$DEST_HOST" "ls -1d $DEST_BASE/20* 2>/dev/null | sort | tail -1" || true)"
+LINKDEST=()
+[ -n "$PREV" ] && LINKDEST=(--link-dest="$PREV")
+echo "link-dest: ${PREV:-<none, first full snapshot>}"
+
+ssh -o BatchMode=yes "$DEST_HOST" "mkdir -p '$DEST_BASE/$STAMP'"
+
+rsync -a --delete --numeric-ids \
+  --exclude='node_modules/' --exclude='.venv/' --exclude='venv/' --exclude='__pycache__/' \
+  --exclude='.pytest_cache/' --exclude='.mypy_cache/' --exclude='.ruff_cache/' --exclude='.cache/' \
+  --exclude='dist/' --exclude='build/' --exclude='.next/' --exclude='target/' --exclude='*.pyc' \
+  --exclude='.env' --exclude='.env.*' --exclude='*.pem' --exclude='*.key' --exclude='id_*' \
+  --exclude='*.sqlite' --exclude='*.sqlite3' --exclude='*.db-wal' --exclude='*.db-shm' \
+  "${LINKDEST[@]}" \
+  "$SRC" "$DEST_HOST:$DEST_BASE/$STAMP/"
+RC=$?
+echo "rsync rc=$RC"
+
+# rc 0 = ok; rc 24 = some files vanished mid-transfer (benign for a live tree)
+if [ "$RC" -eq 0 ] || [ "$RC" -eq 24 ]; then
+  ssh -o BatchMode=yes "$DEST_HOST" "ln -sfn '$DEST_BASE/$STAMP' '$DEST_BASE/latest'"
+  # retention: keep newest 48 hourly snapshots
+  ssh -o BatchMode=yes "$DEST_HOST" "ls -1d $DEST_BASE/20* 2>/dev/null | sort | head -n -48 | xargs -r rm -rf"
+  echo "=== $(date -Is) snapshot $STAMP OK (rc=$RC) ==="
+else
+  echo "=== $(date -Is) snapshot $STAMP FAILED rc=$RC — keeping partial for inspection ==="
+fi
@@ -15,7 +15,7 @@ Primary AI inference host for PFI.
  NOT Dell / not the same box as sf-r630 / sfsrv-ana)
 - **CPU:** AMD EPYC 9254 24-core (96 threads)
 - **RAM:** 566 GB
- **GPUs:** 2x NVIDIA RTX 6000 Ada Generation (46 GB VRAM each, GPU 0 and GPU 1)
+- **GPUs:** 2x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition (96 GB VRAM each, cc 12.0 / sm_120, GPU 0 and GPU 1) — upgraded 2026-06 from 2x RTX 6000 Ada (46 GB, cc 8.9). Blackwell adds native FP4 (NVFP4) tensor cores and doubles VRAM.
 - **Storage:** ZFS `zroot` (434 GB root) + `tank` pool (8.6 TB at `/tank`)
 - **OS:** Debian 13 (trixie), kernel 6.12.x
 - **Docker:** 29.3.1, runtimes: runc (default), nvidia, io.containerd.runc.v2
@@ -32,21 +32,51 @@ Primary AI inference host for PFI.

 ## Running stacks

-| Stack | Port | Notes |
-|-------|------|-------|
-| llama-swap | 9292 | GGUF model server via llama.cpp |
-| vllm-embed (Qwen3) | 8001 | OpenAI-compatible embeddings; part of the `vllm` stack (GPU 1) |
-| vllm-rerank (Qwen3) | 8002 | OpenAI-compatible reranker; part of the `vllm` stack (GPU 1) |
-| vllm-reward (Skywork) | 8003 | Skywork-Reward-V2-8B-AWQ classifier; part of the `vllm` stack (GPU 1) |
-| dockge | 5001 | Docker stack management UI |
-| dozzle-agent | 7007 | Log agent; reports to the Dozzle hub on ana-docker |
-| beszel-agent | 45876 | Metrics agent; reports to the Beszel hub on ana-docker |
+Live inventory as of 2026-07-22. Each model is its own compose stack now
+(container `vllm-<x>` / `llama-<x>`); the `vllm` stack proper is just the
+embed/rerank/reward trio. GPUs are pinned per container via
+`deploy.resources.reservations.devices[].device_ids`.

-**Retired since last README update:**
+**GPU 0 — heavy RP / reasoning seats (~88/98 GB, hot serving path):**
+
+| Container | Port | Served model | Quant | Ctx |
+|-----------|------|--------------|-------|-----|
+| `vllm-aeon-gen` | 8015 | `qwen3.6-35b-a3b-heretic` — the "gen" hero seat | NVFP4 (modelopt) | 256k |
+| `vllm-charrp-reasoning-nvfp4` | 8018 | `char-rp-reasoning` (R36 reasoning RP) | NVFP4 (modelopt) | 256k |
+
+**GPU 1 — light / eval / retrieval + char-RP GGUF (~91/98 GB, on-demand):**
+
+| Container | Port | Served model | Quant | Ctx |
+|-----------|------|--------------|-------|-----|
+| `vllm-granite` | 8004 | `granite-4.1-8b` — fleet summarizer/classifier | FP8 (compressed-tensors) | 131k |
+| `llama-charrp` | 8016 | `Magidonia-24B-v4.3` Q6_K — char-RP (llama.cpp) | GGUF Q6_K | — |
+| `vllm-selene` | 8011 | `selene-1-mini-8b` — Atla LLM-as-judge | FP8 | 32k |
+| `vllm-reward` | 8003 | `Skywork-Reward-V2-Llama-3.1-8B-AWQ` — reward classifier | AWQ | 16k |
+| `vllm-embed` | 8001 | `Qwen3-Embedding-0.6B` | — | 8k |
+| `vllm-rerank` | 8002 | `Qwen3-Reranker-0.6B` | — | 8k |
+
+**Infra / non-GPU:**
+
+| Container | Port | Notes |
+|-----------|------|-------|
+| `dockge` | 5001 | Docker stack management UI |
+| `dozzle-agent` | 7007 | Log agent → Dozzle hub on ana-docker |
+| `beszel-agent` | 45876 | Metrics agent → Beszel hub on ana-docker |
+
+Both cards run near-full (~7–10 GB headroom each) — adding a seat means placing
+it on the card with room or evicting a dormant one first.
+
+**Dormant (compose present on disk, containers stopped)** — rollback / audition
+seats, safe to leave: `mistral-medium-3.5`, `mistral-small-4(-heretic)`,
+`ms32-24b-angel`, `qwen3.5-122b`, `qwopus3.5-122b`, `qwen35-vl`, `qwen36-vl`,
+`qwen36-27b-aeon`, `qwen-image-bench`, `vibevoice`, `comfyui`, `kokoro`,
+`parakeet`, `vllm-qwen3`.
+
+**Retired:**
+- `llama-swap` (former GGUF multiplexer on :9292) — replaced by dedicated
+  per-model seats (e.g. `llama-charrp`); no longer running.
 - `infinity` — replaced by the `vllm` stack (originally `vllm-qwen3`, renamed 2026-05-13 when the stack expanded beyond Qwen3) after the upstream Infinity image stopped shipping a `transformers` build that knew Qwen3.
- `LibreChat (+ rag_api, vectordb, mongodb, meilisearch)` — removed from this host.
- `searxng` — now hosted on ana-docker for the whole fleet.
- Residual networks (`librechat_default`, `kokoro-tts-gpu_default`) from prior experiments are still present; safe to `docker network rm` at leisure.
+- `LibreChat (+ rag_api, vectordb, mongodb, meilisearch)`, `searxng` — removed from this host (searxng now on ana-docker fleet-wide).

 ## Refresh state

@@ -58,9 +88,17 @@ Latest snapshot: `system-details.txt` (regenerate as needed).

 ## GPU allocation policy

-By default, no container is pinned. For predictable performance when multiple GPU workloads run concurrently:
+Every seat is explicitly pinned via `device_ids` (no unpinned containers), and
+both cards run ~90% full:

- **GPU 0:** heavy LLM (llama-swap big models).
- **GPU 1:** light services (the three `vllm` services share this GPU via `--gpu-memory-utilization`).
+- **GPU 0:** the two heavy NVFP4 seats — `vllm-aeon-gen` (gen) and
+  `vllm-charrp-reasoning-nvfp4`. The live serving path (near-100% util under
+  load), ~42 + 45 GB.
+- **GPU 1:** everything else — summarizer (granite), judge (selene), reward,
+  embed, rerank, and the Magidonia char-RP GGUF seat. Bursty/on-demand, idle
+  between calls, ~91 GB resident.

-Use `deploy.resources.reservations.devices[].device_ids: ["<id>"]` in compose to pin.
+Pin with `deploy.resources.reservations.devices[].device_ids: ["<id>"]` in
+compose. Each service caps its share with `--gpu-memory-utilization`; with both
+cards near-full, placing a new seat means freeing room (evict a dormant one) or
+trimming a neighbour's utilization first.
@@ -1 +1 @@
-vh@10.250.50.152
+infra-ops@10.250.50.152
@@ -22,6 +22,15 @@ local Bash already executes here — no SSH-to-self needed for non-privileged wo
  `vh/mead-hall`.
 - **bloom_music dev** — `~/development/bloom_music`; its `web/` test harness uses
  Playwright headless Chromium for OSMD browser-geometry assertions.
+- **The Booth** — ephemeral media drop board (`:8090`, `booth.service`), from
+  eshpfi `services/booth/`. Lets CC sessions surface A/B renders + smoke results
+  (and browser uploads for pickup) to the operator; 24h TTL, Homepage-linked.
+- **jackdaw-compose** — JackDAW AI Composer `/compose` backend (`:8787`,
+  `jackdaw-compose.service`), a thin stateless `bun server/index.ts` from
+  `~/development/jackdaw` → LiteLLM `gen`. Origin-gated (INV-BK04/BK05), reached
+  same-origin via the `:4500` bench's `/compose` proxy. Hosted for jackdaw-dev
+  (their code; the model endpoint + key live in server env only — unit is `0600`,
+  not committed).

 ## Box-wide Playwright / Chromium (2026-06-04)

@@ -0,0 +1,67 @@
+# nh3-extdev
+
+NH3-site **manager / external-dev box** — `10.100.50.42`. Fresh Debian 13
+(trixie) VM on `nh3-pve` (QEMU: 8 vCPU / 7.8 GB / 250 GB, no GPU). Successor to
+the retired **nh3-ansible**. Not a Docker-stack host (Docker not installed).
+
+**Reach:** the dedicated agent identity only —
+`ssh nh3-extdev` (alias) = `ssh -i ~/.ssh/infra-ops_ed25519 infra-ops@10.100.50.42`.
+
+The `infra-ops` user here is **sudo-LESS by design** (operator decision
+2026-06-17): key-only login, password locked, **no NOPASSWD sudo**, **not** in the
+`docker` group — deliberately tighter than the rest of the fleet's infra-ops
+identity ([`reference_infra_ops_sudo_identity`] in auto-memory, which is the
+*sudo* variant). Management here is **user-level only**: files/repos/tooling under
+the home dir, `systemctl --user`, world-readable system state. No root ops (apt,
+system services, `/etc`, root-owned files) and **no Docker** unless a later,
+explicit grant is made (command-scoped sudoers preferred over docker-group, which
+is root-equivalent).
+
+## Purpose
+
+NH3 **client-agent control node** (successor to the retired `nh3-ansible`): runs
+the **`pi` coding agent** (earendil-works) on **GLM 5.2**, one isolated agent per
+external client, to manage that client's infrastructure. Each client gets a
+walled workspace under `/opt/externs/<client>/`; isolation is by directory +
+credential, never a shared identity.
+
+### Per-client workspace layout (`/opt/externs/<client>/`)
+
+```
+/opt/externs/<client>/
+├── AGENTS.md? (in repo/)   # operating charter the agent loads every session
+├── secrets.env             # 600, gitignored — EXTERNS_<CLIENT>_GLM_KEY (scoped litellm key)
+├── run-pi.sh               # 700 launcher: sources secrets.env, sets PI_CODING_AGENT_DIR,
+│                           #   cd repo/, exec pi  (per-client isolated config + key)
+├── .pi/                    # this client's pi agent dir (PI_CODING_AGENT_DIR points here)
+│   ├── models.json         # provider litellm-glm -> gateway, models glm-5.2[/-reasoning]
+│   ├── settings.json       # defaultProvider litellm-glm, defaultModel glm-5.2
+│   └── (sessions/auth)     # pi-managed at runtime
+├── repo/                   # client infra workspace (own .git; AGENTS.md/scripts/servers/…)
+└── .ssh/                   # per-client scoped deploy key (reaches THIS client's hosts only)
+```
+
+Launch an agent: `/opt/externs/<client>/run-pi.sh` (add pi flags as needed, e.g.
+`-p "…"` non-interactive). `pi` itself reaches GLM 5.2 through the litellm gateway
+(`http://10.250.50.70:4000/v1`, reachable cross-site from NH3 — verified).
+
+### pi / Node install (user-level, no root)
+
+Node and pi are installed **user-level** (the box is sudo-less): Node v22.23.0 LTS
+from the official static tarball (checksum-verified) under
+`~/.local/node-v22.23.0-linux-x64/`, with `pi` installed `-g` into that prefix
+(`pi --version` → 0.79.7). PATH is wired in `~/.profile` + `~/.bashrc`.
+
+Reproduce / add a client / upgrade: **`playbooks/install-pi-nh3-extdev.yaml`**
+(idempotent — `scripts/elway nh3-extdev --playbook …`). It installs Node+pi and
+wires every workspace in its `clients` var to GLM 5.2; the workspace *scaffold*
+(dirs, `secrets.env`, deploy keys, `repo/`) is provisioned separately.
+
+## Notes
+
+- Debian **13** (trixie) — newer than the fleet's Debian-12 baseline; watch for
+  package/behaviour drift vs other hosts.
+- `/etc/hosts` now carries `nh3-extdev` (the old `unable to resolve host` sudo
+  warning is silenced).
+- Stood up 2026-06-17; pi-on-GLM-5.2 client agents wired 2026-06-18. `system-details.txt`
+  is sudo-less, so docker/root-only sections are necessarily blank.
@@ -0,0 +1 @@
+infra-ops@10.100.50.42
@@ -0,0 +1,75 @@
+
+===== HOST =====
+
+Hostname:   nh3-extdev
+Date:       2026-06-18T14:04:13-07:00
+Uptime:     up 23 hours, 25 minutes
+OS:         Debian GNU/Linux 13 (trixie)
+Kernel:     6.12.90+deb13.1-amd64
+Arch:       x86_64
+
+===== HARDWARE =====
+
+CPU cores:  8
+CPU model:  QEMU Virtual CPU version 2.5+
+MemTotal:   7.8 GB
+MemAvailable: 7.3 GB
+
+===== GPUS =====
+
+nvidia-smi not present (no NVIDIA GPUs or driver not installed)
+
+===== FILESYSTEMS (df) =====
+
+Filesystem      Size  Used Avail Use% Mounted on
+/dev/sda1       250G  3.6G  234G   2% /
+
+===== PERSISTENT MOUNTS (/etc/fstab, non-comment) =====
+
+UUID=fabc36b3-459e-4ef1-80a6-8a0f85a1a781 /               ext4    errors=remount-ro 0       1
+UUID=bf3fd6d0-4537-4626-bdcc-551c5603ab98 none            swap    sw              0       0
+/dev/sr0        /media/cdrom0   udf,iso9660 user,noauto     0       0
+
+===== TARGETED DATA PATHS =====
+
+/opt  (total: 1.5M)
+  total 12
+  drwxr-xr-x   3 root      root    4096 2026-06-17 15:03 .
+  drwxr-xr-x  18 root      root    4096 2026-06-17 14:37 ..
+  drwxrws---+  5 infra-ops externs 4096 2026-06-17 15:13 externs
+
+/srv  (total: 4.0K)
+  total 8
+  drwxr-xr-x  2 root root 4096 2024-05-30 19:51 .
+  drwxr-xr-x 18 root root 4096 2026-06-17 14:37 ..
+
+
+===== DOCKER =====
+
+docker not installed
+
+===== COMPOSE FILES (/opt/docker/compose/) =====
+
+/opt/docker/compose not present
+
+===== CONFIG LAYOUT (/opt/docker/conf/ — top 200 entries) =====
+
+/opt/docker/conf not present
+
+===== LISTENING PORTS =====
+
+0.0.0.0:22
+[::1]:25
+127.0.0.1:25
+[::]:22
+
+===== MODEL / HUGGINGFACE CACHES =====
+
+
+===== DOCKER-ADJACENT SYSTEMD SERVICES =====
+
+(none matching)
+
+===== DONE =====
+
+Paste the above back into the chat, or pass a path as argv[1] to save.
@@ -0,0 +1,6 @@
+.venv/
+__pycache__/
+*.pyc
+*.egg-info/
+.pytest_cache/
+booth-data/
@@ -0,0 +1,135 @@
+# The Booth
+
+A dead-simple standing web server for shuttling **ephemeral files** between the
+operator and CC sessions — A/B renders, smoke-test screenshots, audio/video
+samples, or anything you want to hand off. It works both directions:
+
+- **Session → operator:** a session drops a folder of files on disk; the Booth
+  renders it as a browsable "booth".
+- **Operator/anyone → pickup:** upload files through the browser (or `curl -F`)
+  and get a **human-readable pickup id** like `4-wombat` or `star-84`.
+
+Either way it **wipes 24h after the last activity**. No database — the
+filesystem *is* the state.
+
+- **Live:** http://10.100.10.50:8090/ (nh3-dev) · linked from Homepage → *Apps → The Booth*
+- **Data dir:** `~/booth-data/` on nh3-dev (one subfolder per booth)
+- **TTL:** 24h, measured from the newest mtime in a booth's tree (it lives while
+  you're touching it, self-destructs 24h after you stop)
+
+## How a session posts
+
+A booth is **just a folder** under the data dir. Three ways, cheapest first:
+
+```bash
+# 1. On nh3-dev — the helper (services/booth/scripts/booth):
+booth add my-run  out/a.png out/b.png       # creates booth + copies, prints URL
+booth new  my-run                            # empty booth, then cp/mv into ~/booth-data/my-run/
+booth url  my-run                            # just print the URL
+booth ls                                     # list booths
+booth rm   my-run                            # wipe now (TTL would anyway)
+
+# 2. On nh3-dev — raw, no helper:
+mkdir -p ~/booth-data/my-run && cp out/*.png ~/booth-data/my-run/
+#   -> http://10.100.10.50:8090/b/my-run/
+
+# 3. From another host — rsync into the data dir:
+rsync -a ./out/  nh3-dev:booth-data/my-run/
+```
+
+Then hand the operator `http://10.100.10.50:8090/b/my-run/`.
+
+## Upload for pickup
+
+The reverse direction — put files in through the web, pick them up by id:
+
+- **Browser:** the index page has an *Upload files for pickup* panel
+  (drag-drop or click). Submit → you land on a booth with a **human-readable
+  id** (`4-wombat`, `star-84`) whose files each have a ⬇ download link.
+- **curl (a remote session with no ssh to nh3-dev can use this too):**
+  ```bash
+  curl -sS -i -F 'files=@out/a.png' -F 'files=@out/b.png' \
+    http://10.100.10.50:8090/upload | grep -i location
+  #   Location: /b/star-84/   <- the pickup id
+  ```
+- **Pick up** at `http://10.100.10.50:8090/b/<id>/` (download links), or on
+  nh3-dev straight off disk at `~/booth-data/<id>/`.
+
+Uploads are stamped as pickup booths (a `⬆ pickup` badge in the UI) and expire
+on the same 24h TTL. Limits: `BOOTH_MAX_FILES` files (default 50) and
+`BOOTH_MAX_UPLOAD_MB` total per submission (default 1024); filenames are reduced
+to a safe basename (no path traversal).
+
+## What a booth renders
+
+- **Has its own `index.html`?** → served **verbatim** (its relative assets —
+  `chart.png`, `report.css` — resolve out of the same folder). Build whatever
+  page you want.
+- **No `index.html`?** → **auto-gallery** of the folder's media:
+  - images (`png jpg jpeg gif webp avif svg bmp`) → `<img>` (click → full-screen
+    viewer with **Fit** / **1:1** — the toggle only appears when the image is
+    larger than the viewport — plus download and ✕/Esc back to the gallery)
+  - video (`webm mp4 ogv m4v mov`) → `<video controls>`
+  - audio (`mp3 wav ogg flac m4a opus aac`) → `<audio controls>`
+  - anything else → a download link
+- **Captions:** a `<file>.txt` or same-stem `<stem>.txt` sidecar is folded in as
+  that item's caption — the natural way to label an A/B pair:
+  ```
+  a.png              b.png
+  a.txt  "baseline"  b.png.txt  "cudaMallocAsync (winner)"
+  ```
+
+## Routes
+
+| Route | Purpose |
+|---|---|
+| `GET /` | Index — one card per booth (newest first), with expiry countdown |
+| `GET /b/<name>/` | A booth (its `index.html`, else auto-gallery) |
+| `GET /b/<name>/<file>` | Serve a file out of the booth |
+| `POST /upload` | Upload files → new pickup booth; 303-redirects to `/b/<id>/` (id in `Location`) |
+| `POST /b/<name>/delete` | Wipe a booth (the UI's "Wipe now" button) |
+| `DELETE /b/<name>` | Wipe a booth (curl/API) |
+| `GET /healthz` | `{ok, ttl_hours, booths}` — Homepage siteMonitor target |
+
+## Ops
+
+Runs as a **user-level** systemd service on nh3-dev (no root, no Docker),
+alongside the other fleet sidecars (herald, zellij-web, ttyd).
+
+```bash
+systemctl --user status booth.service
+systemctl --user restart booth.service
+journalctl --user -u booth.service -f          # sweeper logs "[booth] swept …"
+```
+
+Config is env in the unit (`booth.service`):
+`BOOTH_DATA_DIR`, `BOOTH_TTL_HOURS`, `BOOTH_HOST_LABEL`, `BOOTH_SWEEP_INTERVAL_MIN`,
+`BOOTH_MAX_UPLOAD_MB` (default 1024), `BOOTH_MAX_FILES` (default 50).
+
+### Install / update
+
+```bash
+cd services/booth
+uv venv && uv pip install fastapi "uvicorn[standard]" jinja2 python-multipart   # runtime deps
+cp booth.service ~/.config/systemd/user/booth.service
+systemctl --user daemon-reload && systemctl --user enable --now booth.service
+```
+
+Code runs straight from this checkout (the unit's `WorkingDirectory` /
+`ExecStart` point here), so "deploy an update" = edit + `systemctl --user
+restart booth.service`.
+
+### Tests
+
+```bash
+cd services/booth && uv pip install pytest httpx && .venv/bin/python -m pytest -q
+```
+
+## Notes / non-goals
+
+- **No auth.** LAN/WG-internal only, ephemeral content — don't drop secrets in a
+  booth, and note anyone on the LAN can upload (bounded by the size/file limits).
+  Uploaded files are served back with their own content-type, so an uploaded
+  `index.html` renders as a page (a feature for custom reports; keep it in mind).
+- Booth names with `/`, `..`, or a leading `.` are rejected; file serving and
+  uploaded filenames are guarded against path traversal and symlink escape.
@@ -0,0 +1,21 @@
+[Unit]
+Description=The Booth — ephemeral media drop board (scan+serve ~/booth-data, 24h TTL)
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+WorkingDirectory=/home/lkraven/development/eshpfi-management/services/booth
+ExecStart=/home/lkraven/development/eshpfi-management/services/booth/.venv/bin/uvicorn booth.app:app --host 0.0.0.0 --port 8090
+Environment=BOOTH_DATA_DIR=/home/lkraven/booth-data
+Environment=BOOTH_TTL_HOURS=24
+Environment=BOOTH_HOST_LABEL=nh3-dev 10.100.10.50
+Environment=BOOTH_SWEEP_INTERVAL_MIN=15
+Restart=on-failure
+RestartSec=3
+
+# User-level unit: install to ~/.config/systemd/user/booth.service and
+#   systemctl --user daemon-reload && systemctl --user enable --now booth.service
+# (loginctl enable-linger lkraven  — so it survives logout, already set on nh3-dev)
+[Install]
+WantedBy=default.target
@@ -0,0 +1,3 @@
+"""The Booth — ephemeral media drop board. See booth.app for the server."""
+
+__version__ = "0.1.0"
@@ -0,0 +1,595 @@
+"""The Booth — a standing web server that renders drop-folders as ephemeral media booths.
+
+Model (deliberately dead-simple, no database):
+  * The data dir holds one subfolder per "booth". A booth is created by a CC
+    session simply making a folder and dropping files in — there is no upload API.
+  * GET /            -> index: a card per booth (scan of the data dir).
+  * GET /b/<name>/   -> if <name>/index.html exists, serve it verbatim; otherwise
+                        auto-render a gallery of the images / webm-videos / audio in it.
+  * GET /b/<name>/<file> -> serve a file out of the booth (also feeds a custom index.html's assets).
+  * 24h TTL: a background sweeper wipes any booth untouched for TTL hours. A booth's
+    age is measured from the *newest* mtime in its tree, so it lives while it's being
+    worked on and self-destructs TTL hours after the last activity.
+
+State is the filesystem — `ls ~/booth-data` tells you everything. That is the whole point.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import io
+import os
+import re
+import secrets
+import shutil
+import time
+import zipfile
+from contextlib import asynccontextmanager
+from pathlib import Path
+from urllib.parse import quote
+
+from fastapi import FastAPI, File, HTTPException, Request, UploadFile
+from fastapi.responses import (
+    FileResponse,
+    HTMLResponse,
+    JSONResponse,
+    RedirectResponse,
+    Response,
+)
+from fastapi.templating import Jinja2Templates
+
+TEMPLATES_DIR = Path(__file__).parent / "templates"
+
+# Browser-playable media buckets. Anything else renders as a download link.
+IMAGE_EXTS = {".png", ".jpg", ".jpeg", ".gif", ".webp", ".avif", ".svg", ".bmp"}
+VIDEO_EXTS = {".webm", ".mp4", ".ogv", ".m4v", ".mov"}
+AUDIO_EXTS = {".mp3", ".wav", ".ogg", ".oga", ".flac", ".m4a", ".opus", ".aac"}
+
+CAPTION_MAX = 800  # chars of a sidecar .txt caption we render
+
+
+def classify(name: str) -> str:
+    """image | video | audio | other, by extension."""
+    ext = Path(name).suffix.lower()
+    if ext in IMAGE_EXTS:
+        return "image"
+    if ext in VIDEO_EXTS:
+        return "video"
+    if ext in AUDIO_EXTS:
+        return "audio"
+    return "other"
+
+
+def human_dur(seconds: float) -> str:
+    s = int(seconds)
+    if s <= 0:
+        return "expired"
+    h, rem = divmod(s, 3600)
+    m, _ = divmod(rem, 60)
+    if h and m:
+        return f"{h}h {m}m"
+    if h:
+        return f"{h}h"
+    if m:
+        return f"{m}m"
+    return "<1m"
+
+
+def _newest_mtime(path: Path) -> float:
+    """Newest mtime among a folder and everything under it."""
+    try:
+        newest = path.stat().st_mtime
+    except OSError:
+        return 0.0
+    for p in path.rglob("*"):
+        try:
+            m = p.stat().st_mtime
+        except OSError:
+            continue
+        if m > newest:
+            newest = m
+    return newest
+
+
+def booth_age_seconds(path: Path, now: float | None = None) -> float:
+    now = time.time() if now is None else now
+    return now - _newest_mtime(path)
+
+
+def is_expired(path: Path, ttl_seconds: float, now: float | None = None) -> bool:
+    return booth_age_seconds(path, now) > ttl_seconds
+
+
+def sweep_once(data_dir: Path, ttl_seconds: float, now: float | None = None) -> list[str]:
+    """Wipe every direct-child booth older than the TTL. Returns names wiped.
+
+    Only ever removes direct children of data_dir (never data_dir itself), and
+    skips dotfolders so a stray control dir can opt out.
+    """
+    wiped: list[str] = []
+    if not data_dir.is_dir():
+        return wiped
+    for child in data_dir.iterdir():
+        if not child.is_dir() or child.name.startswith("."):
+            continue
+        try:
+            if is_expired(child, ttl_seconds, now):
+                shutil.rmtree(child)
+                wiped.append(child.name)
+        except OSError:
+            pass
+    return wiped
+
+
+def list_booths(data_dir: Path, ttl_seconds: float, now: float | None = None) -> list[dict]:
+    now = time.time() if now is None else now
+    booths: list[dict] = []
+    if not data_dir.is_dir():
+        return booths
+    for child in data_dir.iterdir():
+        if not child.is_dir() or child.name.startswith("."):
+            continue
+        files = [p for p in child.rglob("*") if p.is_file() and not p.name.startswith(".")]
+        kinds = {"image": 0, "video": 0, "audio": 0, "other": 0}
+        thumb_url = None
+        for f in files:
+            k = classify(f.name)
+            kinds[k] += 1
+            if k == "image" and thumb_url is None:
+                thumb_url = quote(f.relative_to(child).as_posix(), safe="/")
+        mtime = _newest_mtime(child)
+        booths.append(
+            {
+                "name": child.name,
+                "name_url": quote(child.name, safe=""),
+                "count": len(files),
+                "kinds": kinds,
+                "thumb_url": thumb_url,
+                "has_index": (child / "index.html").is_file(),
+                "uploaded": (child / UPLOAD_MARKER).exists(),
+                "expires_in": max(0.0, ttl_seconds - (now - mtime)),
+                "mtime": mtime,
+            }
+        )
+    booths.sort(key=lambda b: b["mtime"], reverse=True)
+    return booths
+
+
+def build_gallery(child: Path) -> list[dict]:
+    """Files in a booth as render items, with caption sidecars folded in.
+
+    A `<file>.txt` (e.g. `a.png.txt`) or a same-stem `<stem>.txt` (e.g. `a.txt`
+    next to `a.png`) is consumed as that item's caption rather than shown itself —
+    the natural way to label an A/B pair.
+    """
+    all_files = [p for p in child.rglob("*") if p.is_file() and not p.name.startswith(".")]
+    by_rel = {p.relative_to(child).as_posix(): p for p in all_files}
+    caption: dict[str, str] = {}
+    sidecars: set[str] = set()
+
+    for rel, p in by_rel.items():
+        if not rel.lower().endswith(".txt"):
+            continue
+        target = None
+        base_full = rel[:-4]  # strip ".txt"  -> "a.png.txt" => "a.png"
+        if base_full in by_rel:
+            target = base_full
+        else:  # "a.txt" beside "a.png"
+            parent = str(Path(rel).parent)
+            stem = Path(rel).stem
+            for q_rel, q in by_rel.items():
+                if q_rel == rel:
+                    continue
+                if (
+                    str(Path(q_rel).parent) == parent
+                    and Path(q_rel).stem == stem
+                    and classify(q.name) != "other"
+                ):
+                    target = q_rel
+                    break
+        if target is not None:
+            try:
+                caption[target] = p.read_text(errors="replace").strip()[:CAPTION_MAX]
+            except OSError:
+                pass
+            sidecars.add(rel)
+
+    items = []
+    for rel in sorted(by_rel):
+        if rel in sidecars:
+            continue
+        p = by_rel[rel]
+        items.append(
+            {
+                "name": rel,
+                "kind": classify(p.name),
+                "url": quote(rel, safe="/"),
+                "caption": caption.get(rel),
+            }
+        )
+    return items
+
+
+def zip_booth(booth: Path) -> bytes:
+    """Zip a booth's whole tree (dotfiles excluded) into an in-memory archive.
+
+    Lets a booth be downloaded as one artifact regardless of shape — the case a
+    verbatim `index.html` booth (e.g. a rendered brief + its assets) has no
+    per-file download affordance for, since the page is served raw.
+    """
+    buf = io.BytesIO()
+    with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as zf:
+        for p in sorted(booth.rglob("*")):
+            if p.is_file() and not p.name.startswith("."):
+                zf.write(p, p.relative_to(booth).as_posix())
+    return buf.getvalue()
+
+
+def _zip_filename(name: str) -> str:
+    """A Content-Disposition-safe `<booth>.zip` (strip quotes/control chars)."""
+    safe = "".join(c for c in name if c.isprintable() and c != '"')
+    return f"{safe or 'booth'}.zip"
+
+
+# ---- verbatim-index.html wrapper -------------------------------------------
+
+# Mirror of base.html's favicon (the app templates set it there; this is the copy
+# injected into a booth's *verbatim* index.html so a raw page inherits the same
+# icon). Keep the two in sync if the Booth's icon ever changes.
+FAVICON_HREF = (
+    "data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'"
+    "%3E%3Crect width='32' height='32' rx='7' fill='%23171a23'/%3E%3Ccircle cx='16' "
+    "cy='16' r='6' fill='none' stroke='%2342dcd1' stroke-width='2.5'/%3E%3Ccircle "
+    "cx='16' cy='16' r='2.2' fill='%2342dcd1'/%3E%3C/svg%3E"
+)
+FAVICON_LINK = f'<link rel="icon" href="{FAVICON_HREF}">'
+
+# A self-contained floating "back to all booths" chip injected into verbatim
+# booths. Scoped class + fixed positioning + max z-index so it overlays the raw
+# page without touching its layout; hidden in print so downloaded reports stay clean.
+_BACK_CHIP = (
+    '<a href="/" class="booth-nav-home" aria-label="back to all booths">‹ all booths</a>'
+    # top-right: empty on left-aligned report layouts (a top-left chip clips the
+    # page title), and consistent with the zoom view's top-right back affordance.
+    "<style>.booth-nav-home{position:fixed;top:0;right:0;z-index:2147483647;"
+    "display:inline-block;margin:.6rem;padding:.34rem .72rem;"
+    "font:600 13px/1.25 ui-sans-serif,system-ui,-apple-system,'Segoe UI',Roboto,sans-serif;"
+    "color:#dfe7ef;text-decoration:none;letter-spacing:.01em;"
+    "background:rgba(20,23,32,.82);border:1px solid rgba(66,220,209,.35);border-radius:8px;"
+    "-webkit-backdrop-filter:blur(6px);backdrop-filter:blur(6px);"
+    "box-shadow:0 2px 10px rgba(0,0,0,.35);transition:background .18s,border-color .18s}"
+    ".booth-nav-home:hover{background:rgba(28,33,46,.95);border-color:rgba(66,220,209,.75)}"
+    "@media print{.booth-nav-home{display:none}}</style>"
+)
+
+WRAP_MAX_BYTES = 8 * 1024 * 1024  # above this, serve the verbatim page raw (unwrapped)
+
+_ICON_RE = re.compile(r"<link\b[^>]*\brel\s*=\s*[\"']?[^\"'>]*icon", re.IGNORECASE)
+_HEAD_CLOSE_RE = re.compile(r"</head\s*>", re.IGNORECASE)
+_HTML_OPEN_RE = re.compile(r"<html\b[^>]*>", re.IGNORECASE)
+_DOCTYPE_RE = re.compile(r"<!doctype[^>]*>", re.IGNORECASE)
+_BODY_CLOSE_RE = re.compile(r"</body\s*>", re.IGNORECASE)
+_HTML_CLOSE_RE = re.compile(r"</html\s*>", re.IGNORECASE)
+
+
+def _insert_before(html: str, pattern: re.Pattern, snippet: str) -> tuple[str, bool]:
+    m = pattern.search(html)
+    if m:
+        return html[: m.start()] + snippet + html[m.start() :], True
+    return html, False
+
+
+def _insert_after(html: str, pattern: re.Pattern, snippet: str) -> tuple[str, bool]:
+    m = pattern.search(html)
+    if m:
+        return html[: m.end()] + snippet + html[m.end() :], True
+    return html, False
+
+
+def wrap_verbatim_html(html: str, favicon_link: str = FAVICON_LINK) -> str:
+    """Inject a floating 'all booths' back-chip — and the Booth favicon, if the page
+    declares none — into a booth's verbatim index.html, without altering the page's
+    rendered content.
+
+    Robust to the compact HTML real booths use (`<!doctype html><meta charset><title>
+    <style>…content`, no explicit head/body). The two hard constraints:
+      * NEVER put anything ahead of a leading <!doctype> — that forces quirks mode.
+      * Keep the charset <meta> within the first 1024 bytes so it's still honoured.
+    So the favicon lands at the first head-ish seam (before </head>, else after
+    <html>, else right after the doctype — a ~250B link keeps charset in range), and
+    the fixed-position chip is appended at the END of the document (before </body> /
+    </html> or appended), which renders top-left regardless and disturbs nothing.
+    """
+    if favicon_link and not _ICON_RE.search(html):
+        for inserter, pat in (
+            (_insert_before, _HEAD_CLOSE_RE),  # inside an explicit <head>
+            (_insert_after, _HTML_OPEN_RE),    # top of an explicit <html>
+            (_insert_after, _DOCTYPE_RE),      # right after the doctype (compact HTML)
+        ):
+            html, done = inserter(html, pat, favicon_link)
+            if done:
+                break
+        else:
+            html = favicon_link + html  # bare fragment, no doctype: safe to prepend
+
+    for pat in (_BODY_CLOSE_RE, _HTML_CLOSE_RE):
+        html, done = _insert_before(html, pat, _BACK_CHIP)
+        if done:
+            break
+    else:
+        html = html + _BACK_CHIP  # no </body>/</html>: append to the end
+    return html
+
+
+# ---- uploads (browser drop-off for pickup) ---------------------------------
+
+UPLOAD_MARKER = ".uploaded"  # dotfile stamped into upload booths (excluded from listings)
+
+# Friendly, unambiguous words for human-readable pickup ids (4-wombat / star-84).
+PICKUP_WORDS = (
+    "wombat otter panda koala tiger walrus gecko heron badger beaver falcon marmot "
+    "lemur narwhal ocelot puffin quokka raccoon tapir urchin vulture weasel yak zebra "
+    "alpaca bison cobra dingo egret ferret gibbon hare ibis jaguar llama moose newt "
+    "osprey possum quail robin seal toad viper wren lynx mole swan crane finch sloth "
+    "shrew stoat skunk heronry orca walnut sparrow "
+    "star comet moon cloud river maple cedar birch fern moss reef dune mesa cove glade "
+    "brook pine cedarwood kelp coral amber opal jade onyx slate flint ember spark frost "
+    "storm tide wave ridge peak vale marsh delta atoll canyon fjord geyser lagoon prairie "
+    "anchor beacon lantern kettle copper brass velvet cobalt indigo crimson violet olive "
+    "hazel cocoa mango guava papaya plum kiwi lime pear quince radish turnip acorn clover "
+    "thistle poppy aster dahlia iris lily sage thyme basil clove nutmeg ginger honey"
+).split()
+
+
+def safe_upload_name(name: str, fallback: str) -> str:
+    """Reduce a client-supplied filename to a safe basename (no path, no hidden)."""
+    base = (name or "").replace("\\", "/").split("/")[-1].strip()
+    base = base.lstrip(".")  # a leading dot would hide the file from every listing
+    return base[:200] or fallback
+
+
+def _dedupe_name(name: str, used: set) -> str:
+    if name not in used:
+        return name
+    stem, dot, ext = name.partition(".")
+    i = 1
+    while f"{stem}-{i}{dot}{ext}" in used:
+        i += 1
+    return f"{stem}-{i}{dot}{ext}"
+
+
+def generate_pickup_id(exists) -> str:
+    """A human-readable id like '4-wombat' or 'star-84'. `exists(name)->bool` gates collisions."""
+    for _ in range(400):
+        word = secrets.choice(PICKUP_WORDS)
+        num = secrets.randbelow(99) + 1
+        name = f"{num}-{word}" if secrets.randbelow(2) else f"{word}-{num}"
+        if not exists(name):
+            return name
+    # astronomically unlikely fallback: two words keep it human-readable
+    while True:
+        name = f"{secrets.choice(PICKUP_WORDS)}-{secrets.choice(PICKUP_WORDS)}-{secrets.randbelow(999) + 1}"
+        if not exists(name):
+            return name
+
+
+def create_app(
+    data_dir,
+    ttl_hours: float = 24.0,
+    host_label: str = "",
+    start_sweeper: bool = True,
+    sweep_interval_s: int = 900,
+    max_upload_mb: float = 1024.0,
+    max_files: int = 50,
+) -> FastAPI:
+    data_dir = Path(data_dir).expanduser().resolve()
+    data_dir.mkdir(parents=True, exist_ok=True)
+    ttl_seconds = ttl_hours * 3600.0
+    max_upload_bytes = int(max_upload_mb * 1024 * 1024)
+
+    templates = Jinja2Templates(directory=str(TEMPLATES_DIR))
+    templates.env.filters["dur"] = human_dur
+
+    @asynccontextmanager
+    async def lifespan(app: FastAPI):
+        task = None
+        if start_sweeper:
+
+            async def loop():
+                while True:
+                    try:
+                        wiped = sweep_once(data_dir, ttl_seconds)
+                        if wiped:
+                            print(f"[booth] swept {len(wiped)} expired: {', '.join(wiped)}", flush=True)
+                    except Exception as exc:  # never let the sweeper die
+                        print(f"[booth] sweep error: {exc}", flush=True)
+                    await asyncio.sleep(sweep_interval_s)
+
+            task = asyncio.create_task(loop())
+        try:
+            yield
+        finally:
+            if task is not None:
+                task.cancel()
+
+    app = FastAPI(title="The Booth", lifespan=lifespan)
+
+    ttl_display = int(ttl_hours) if float(ttl_hours).is_integer() else ttl_hours
+    base_ctx = {"ttl_hours": ttl_display, "host": host_label, "data_dir": str(data_dir)}
+
+    def resolve_booth(name: str) -> Path:
+        if not name or name.startswith(".") or "/" in name or "\\" in name or ".." in name:
+            raise HTTPException(status_code=404, detail="no such booth")
+        candidate = data_dir / name
+        try:
+            resolved = candidate.resolve()
+        except OSError:
+            raise HTTPException(status_code=404, detail="no such booth")
+        # resolved.parent must be the data dir itself — blocks symlink escape + nesting.
+        if resolved.parent != data_dir or not resolved.is_dir():
+            raise HTTPException(status_code=404, detail="no such booth")
+        return resolved
+
+    @app.get("/", response_class=HTMLResponse)
+    def index(request: Request):
+        return templates.TemplateResponse(
+            request, "index.html", {**base_ctx, "booths": list_booths(data_dir, ttl_seconds)}
+        )
+
+    @app.get("/healthz")
+    def healthz():
+        return {"ok": True, "ttl_hours": ttl_hours, "booths": len(list_booths(data_dir, ttl_seconds))}
+
+    @app.get("/b/{name}", include_in_schema=False)
+    def booth_redirect(name: str):
+        resolve_booth(name)
+        return RedirectResponse(url=f"/b/{quote(name, safe='')}/", status_code=307)
+
+    @app.get("/b/{name}/", response_class=HTMLResponse)
+    def booth_view(request: Request, name: str, download: int = 0):
+        booth = resolve_booth(name)
+        if download:
+            # whole-booth zip — the download path for a verbatim index.html booth
+            # (which has no gallery/per-file chrome), and a "download all" for any.
+            return Response(
+                content=zip_booth(booth),
+                media_type="application/zip",
+                headers={"Content-Disposition": f'attachment; filename="{_zip_filename(name)}"'},
+            )
+        own_index = booth / "index.html"
+        if own_index.is_file():
+            # Serve the operator's verbatim report, but inject a floating
+            # back-to-booths chip + the Booth favicon (if it declares none) so a
+            # raw page still has a way home. Small HTML -> read + wrap in memory;
+            # a pathological large file falls back to serving raw, unwrapped.
+            try:
+                if own_index.stat().st_size <= WRAP_MAX_BYTES:
+                    return HTMLResponse(
+                        wrap_verbatim_html(own_index.read_text(encoding="utf-8", errors="replace"))
+                    )
+            except OSError:
+                pass
+            return FileResponse(str(own_index), media_type="text/html")
+        return templates.TemplateResponse(
+            request,
+            "booth.html",
+            {
+                **base_ctx,
+                "name": name,
+                "name_url": quote(name, safe=""),
+                "items": build_gallery(booth),
+                "uploaded": (booth / UPLOAD_MARKER).exists(),
+                "expires_in": max(0.0, ttl_seconds - booth_age_seconds(booth)),
+            },
+        )
+
+    @app.get("/b/{name}/view", response_class=HTMLResponse)
+    def booth_image_view(request: Request, name: str, f: str):
+        booth = resolve_booth(name)
+        try:
+            target = (booth / f).resolve()
+        except OSError:
+            raise HTTPException(status_code=404, detail="no such file")
+        if not str(target).startswith(str(booth) + os.sep) or not target.is_file():
+            raise HTTPException(status_code=404, detail="no such file")
+        file_url = quote(f, safe="/")
+        if classify(target.name) != "image":
+            # nothing to zoom on a non-image — hand back the raw file
+            return RedirectResponse(url=f"/b/{quote(name, safe='')}/{file_url}", status_code=307)
+        return templates.TemplateResponse(
+            request,
+            "view.html",
+            {
+                **base_ctx,
+                "name": name,
+                "name_url": quote(name, safe=""),
+                "file": f,
+                "file_url": file_url,
+            },
+        )
+
+    @app.get("/b/{name}/{filepath:path}")
+    def booth_file(name: str, filepath: str, dl: int = 0):
+        booth = resolve_booth(name)
+        try:
+            target = (booth / filepath).resolve()
+        except OSError:
+            raise HTTPException(status_code=404, detail="no such file")
+        if not str(target).startswith(str(booth) + os.sep) or not target.is_file():
+            raise HTTPException(status_code=404, detail="no such file")
+        # ?dl=1 forces a download (Content-Disposition: attachment) instead of the
+        # browser rendering inline — the fix for html/md/text that otherwise opens
+        # in-page with no easy "save".
+        if dl:
+            return FileResponse(str(target), filename=target.name)
+        return FileResponse(str(target))
+
+    @app.post("/upload")
+    async def upload(files: list[UploadFile] = File(...)):
+        """Browser/curl drop-off: files land in a new booth with a human-readable
+        pickup id (e.g. 4-wombat), sweep-expiring in the usual TTL. Redirects (303)
+        to the pickup page; curl clients read the Location header for the id."""
+        files = [f for f in files if f and f.filename]
+        if not files:
+            raise HTTPException(status_code=400, detail="no files uploaded")
+        if len(files) > max_files:
+            raise HTTPException(status_code=413, detail=f"too many files (max {max_files})")
+
+        booth_id = generate_pickup_id(lambda n: (data_dir / n).exists())
+        dest = data_dir / booth_id
+        dest.mkdir(parents=True)
+        (dest / UPLOAD_MARKER).write_text("")  # stamp as an upload (dotfile, not listed)
+
+        total = 0
+        used: set = {UPLOAD_MARKER}
+        try:
+            for i, f in enumerate(files):
+                name = _dedupe_name(safe_upload_name(f.filename, f"file-{i + 1}"), used)
+                used.add(name)
+                with (dest / name).open("wb") as out:
+                    while chunk := await f.read(1024 * 1024):
+                        total += len(chunk)
+                        if total > max_upload_bytes:
+                            raise HTTPException(
+                                status_code=413,
+                                detail=f"upload too large (max {max_upload_mb:g} MB)",
+                            )
+                        out.write(chunk)
+                await f.close()
+        except Exception:
+            shutil.rmtree(dest, ignore_errors=True)  # never leave a half-written booth
+            raise
+
+        return RedirectResponse(url=f"/b/{quote(booth_id, safe='')}/", status_code=303)
+
+    @app.post("/b/{name}/delete")
+    def booth_delete_form(name: str):
+        shutil.rmtree(resolve_booth(name))
+        return RedirectResponse(url="/", status_code=303)
+
+    @app.delete("/b/{name}")
+    def booth_delete_api(name: str):
+        shutil.rmtree(resolve_booth(name))
+        return JSONResponse({"wiped": name})
+
+    return app
+
+
+def _from_env() -> FastAPI:
+    data = os.environ.get("BOOTH_DATA_DIR", str(Path.home() / "booth-data"))
+    ttl = float(os.environ.get("BOOTH_TTL_HOURS", "24"))
+    host = os.environ.get("BOOTH_HOST_LABEL", "")
+    interval = int(float(os.environ.get("BOOTH_SWEEP_INTERVAL_MIN", "15")) * 60)
+    max_mb = float(os.environ.get("BOOTH_MAX_UPLOAD_MB", "1024"))
+    max_n = int(os.environ.get("BOOTH_MAX_FILES", "50"))
+    return create_app(
+        data,
+        ttl_hours=ttl,
+        host_label=host,
+        sweep_interval_s=interval,
+        max_upload_mb=max_mb,
+        max_files=max_n,
+    )
+
+
+app = _from_env()
@@ -0,0 +1,212 @@
+<!doctype html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>{% block title %}The Booth{% endblock %}</title>
+<link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'%3E%3Crect width='32' height='32' rx='7' fill='%23171a23'/%3E%3Ccircle cx='16' cy='16' r='6' fill='none' stroke='%2342dcd1' stroke-width='2.5'/%3E%3Ccircle cx='16' cy='16' r='2.2' fill='%2342dcd1'/%3E%3C/svg%3E">
+<style>
+  /* ============================================================
+     The Booth — Corviduo "Australis" theme (aurora accents, dark-first).
+     Token values adopted from ratatoskr-web (colors_and_type.css canonical
+     palette). Self-contained, no webfont CDN — system fallback stacks.
+     ============================================================ */
+  :root{
+    /* ---- Australis raw palette ---- */
+    --aus-black:#222531; --aus-white:#a9bcc3; --aus-bright-white:#cce7ec; --aus-bright-black:#373b46;
+    --aus-dark-30:#414751; --aus-dark-40:#565f69; --aus-dark-50:#6e7882; --aus-dark-60:#86929d;
+    --aus-bright-70:#9daeb6; --aus-bright-80:#b3cbcf;
+    --aus-blue:#6388d8; --aus-bright-blue:#a4c4ff;
+    --aus-cyan:#00b1a8; --aus-bright-cyan:#42dcd1;
+    --aus-green:#16b866; --aus-bright-green:#51e08a;
+    --aus-red:#ff491a;  --aus-bright-red:#ff854f;
+    --aus-yellow:#e1c631; --aus-bright-yellow:#ffe14e;
+    --aus-magenta:#9d78ff; --aus-bright-magenta:#d8adff;
+
+    /* ---- semantic surface / foreground ---- */
+    --border-subtle:var(--aus-dark-30); --border-default:var(--aus-dark-40); --border-strong:var(--aus-dark-50);
+    --fg-0:var(--aus-bright-white); --fg-1:var(--aus-white); --fg-2:var(--aus-bright-70);
+    --fg-3:var(--aus-dark-60); --fg-muted:var(--aus-dark-50); --fg-on-accent:var(--aus-black);
+
+    /* ---- ratatoskr console surfaces ---- */
+    --rk-deep:#14161d; --rk-canvas:#171a23; --rk-well:#1b1e28; --rk-panel:#1d2029; --rk-ghost:#2c3040;
+
+    /* ---- type ---- */
+    --font-display:"Space Grotesk","Inter",ui-sans-serif,system-ui,sans-serif;
+    --font-sans:"Inter",ui-sans-serif,system-ui,-apple-system,"Segoe UI",sans-serif;
+    --font-mono:"Berkeley Mono","JetBrains Mono","IBM Plex Mono",ui-monospace,"SF Mono","Cascadia Code",Menlo,Consolas,monospace;
+
+    /* ---- radius / shadow / motion ---- */
+    --radius-sm:4px; --radius-md:6px; --radius-lg:10px; --radius-pill:9999px;
+    --shadow-2:0 2px 6px rgba(10,12,18,.45),0 1px 2px rgba(10,12,18,.4);
+    --shadow-3:0 8px 24px rgba(10,12,18,.5),0 2px 6px rgba(10,12,18,.4);
+    --glow-cyan:0 0 0 3px rgba(66,220,209,.26);
+    --ease-out:cubic-bezier(.2,.8,.2,1);
+    --ease-aurora:cubic-bezier(.65,0,.35,1);
+  }
+  @media (prefers-color-scheme: light){
+    :root{
+      --rk-deep:#e5edef; --rk-canvas:#eef4f6; --rk-well:#ffffff; --rk-panel:#f6fafb; --rk-ghost:#cad8dd;
+      --border-subtle:#d3dfe3; --border-default:#bccad0; --border-strong:#9fb0b7;
+      --fg-0:#10151c; --fg-1:#26313a; --fg-2:#48555f; --fg-3:#64727c; --fg-muted:#8a99a1; --fg-on-accent:#ffffff;
+      --aus-bright-cyan:#0a938b; --aus-cyan:#0a938b; --aus-bright-green:#12925a; --aus-green:#12925a;
+      --aus-bright-blue:#3f66bd; --aus-blue:#4a6fc0; --aus-bright-magenta:#7a52d8; --aus-magenta:#7a52d8;
+      --aus-bright-red:#d63a15; --aus-red:#d63a15; --aus-bright-yellow:#a9820a; --aus-yellow:#b98f0c;
+      --aus-dark-40:#b0bec4;
+      --shadow-3:0 8px 24px rgba(30,50,60,.16),0 2px 6px rgba(30,50,60,.1);
+      --glow-cyan:0 0 0 3px rgba(10,147,139,.2);
+    }
+  }
+
+  *{box-sizing:border-box}
+  html,body{margin:0}
+  body{background:var(--rk-canvas);color:var(--fg-1);font-family:var(--font-sans);
+    line-height:1.5;-webkit-font-smoothing:antialiased;min-height:100vh;display:flex;flex-direction:column;
+    transition:background-color .32s var(--ease-aurora),color .32s var(--ease-aurora)}
+  a{color:var(--aus-bright-cyan);text-decoration:none}
+  a:hover{color:var(--aus-cyan);text-decoration:underline}
+  code{font-family:var(--font-mono);font-size:.84em;background:var(--rk-well);color:var(--fg-2);
+    padding:.12em .42em;border-radius:var(--radius-sm);border:1px solid var(--border-subtle)}
+
+  .topbar{display:flex;align-items:center;gap:1.1rem;flex-wrap:wrap;
+    padding:1rem 1.5rem;border-bottom:1px solid var(--border-subtle);
+    background:linear-gradient(180deg,var(--rk-panel),transparent)}
+  .brand{display:inline-flex;align-items:center;gap:.6rem;color:var(--fg-0)}
+  .brand:hover{text-decoration:none}
+  .brand .dot{width:.62rem;height:.62rem;border-radius:50%;background:var(--aus-bright-cyan);
+    box-shadow:0 0 0 4px rgba(66,220,209,.15),0 0 12px rgba(66,220,209,.5);
+    animation:pulse 2.8s var(--ease-aurora) infinite}
+  @keyframes pulse{0%,100%{opacity:1}50%{opacity:.4}}
+  .brand .name{font-family:var(--font-display);font-size:1.22rem;font-weight:600;letter-spacing:-.01em}
+  .tagline{color:var(--aus-cyan);font-size:.66rem;font-family:var(--font-mono);
+    letter-spacing:.2em;text-transform:uppercase}
+
+  main{flex:1;width:100%;max-width:1240px;margin:0 auto;padding:1.7rem 1.5rem 3rem}
+
+  /* ---- upload / pickup ---- */
+  .uploader{display:flex;gap:.9rem;align-items:stretch;margin-bottom:1.7rem;flex-wrap:wrap}
+  .drop{flex:1 1 300px;position:relative;display:flex;flex-direction:column;align-items:center;
+    justify-content:center;gap:.15rem;text-align:center;cursor:pointer;padding:1.1rem 1rem;
+    border:1.5px dashed var(--border-default);border-radius:var(--radius-lg);background:var(--rk-well);
+    transition:border-color .16s var(--ease-out),background .16s var(--ease-out),box-shadow .16s var(--ease-out)}
+  .drop:hover{border-color:var(--aus-cyan)}
+  .drop.over{border-color:var(--aus-bright-cyan);background:rgba(66,220,209,.06);box-shadow:var(--glow-cyan)}
+  .drop.has{border-style:solid;border-color:var(--aus-cyan)}
+  .drop input[type=file]{position:absolute;inset:0;width:100%;height:100%;opacity:0;cursor:pointer}
+  .drop-icon{font-size:1.2rem;color:var(--aus-bright-cyan)}
+  .drop-main{font-family:var(--font-display);font-weight:600;color:var(--fg-0);font-size:.98rem}
+  .drop-sub{font-family:var(--font-mono);font-size:.7rem;letter-spacing:.03em;color:var(--fg-3)}
+  .up-go{flex:0 0 auto;align-self:stretch;cursor:pointer;font-family:var(--font-mono);font-size:.74rem;
+    letter-spacing:.1em;text-transform:uppercase;padding:0 1.15rem;border-radius:var(--radius-lg);
+    background:transparent;border:1px solid var(--aus-cyan);color:var(--aus-bright-cyan);
+    transition:.14s var(--ease-out)}
+  .up-go:hover{background:var(--aus-bright-cyan);color:var(--fg-on-accent);border-color:var(--aus-bright-cyan)}
+
+  .badge{display:inline-block;font-family:var(--font-mono);font-size:.6rem;letter-spacing:.1em;
+    text-transform:uppercase;font-weight:600;color:var(--fg-on-accent);background:var(--aus-bright-cyan);
+    padding:.08rem .42rem;border-radius:var(--radius-pill);vertical-align:middle}
+  .card .thumb{position:relative}
+  .thumb .badge{position:absolute;top:.5rem;left:.5rem;box-shadow:var(--shadow-2)}
+
+  .pickup-note{margin:-.5rem 0 1.5rem;padding:.6rem .85rem;border:1px solid var(--border-subtle);
+    border-left:3px solid var(--aus-bright-cyan);border-radius:var(--radius-md);background:var(--rk-well);
+    font-family:var(--font-mono);font-size:.78rem;color:var(--fg-2)}
+  .copy-btn{cursor:pointer;font-family:var(--font-mono);font-size:.68rem;letter-spacing:.04em;
+    padding:.12rem .5rem;margin:0 .25rem;border:1px solid var(--border-default);border-radius:var(--radius-sm);
+    background:transparent;color:var(--aus-bright-cyan);vertical-align:middle;transition:.12s var(--ease-out)}
+  .copy-btn:hover{border-color:var(--aus-cyan);background:rgba(66,220,209,.08)}
+  .copy-btn.copied{border-color:var(--aus-green);color:var(--aus-bright-green)}
+  .dl-link{color:var(--aus-bright-cyan);text-decoration:none;margin-right:.4rem;font-size:.95em}
+  .dl-link:hover{color:var(--aus-cyan)}
+  .cap-text{color:var(--fg-2)}
+
+  /* ---- image viewer (fixed full-viewport overlay) ---- */
+  .viewer{position:fixed;inset:0;z-index:50;background:var(--rk-canvas);display:flex;flex-direction:column}
+  .vbar{display:flex;align-items:center;gap:.7rem;padding:.5rem .8rem;
+    border-bottom:1px solid var(--border-subtle);background:var(--rk-panel)}
+  .vname{font-family:var(--font-mono);font-size:.8rem;color:var(--fg-2);
+    white-space:nowrap;overflow:hidden;text-overflow:ellipsis;max-width:50vw}
+  .vspacer{flex:1}
+  .vbtn{display:inline-flex;align-items:center;justify-content:center;min-width:2rem;height:2rem;
+    padding:0 .55rem;border:1px solid var(--border-default);border-radius:var(--radius-md);
+    color:var(--fg-1);background:transparent;font-family:var(--font-mono);font-size:.9rem;
+    transition:.14s var(--ease-out)}
+  .vbtn:hover{border-color:var(--aus-cyan);color:var(--aus-bright-cyan);text-decoration:none}
+  .vx:hover{border-color:var(--aus-red);color:#fff;background:var(--aus-red)}
+  .vtoggle{border:1px solid var(--border-default);border-radius:var(--radius-md);overflow:hidden}
+  .vseg{cursor:pointer;border:0;background:transparent;color:var(--fg-3);font-family:var(--font-mono);
+    font-size:.72rem;letter-spacing:.06em;padding:.4rem .75rem;transition:.14s var(--ease-out)}
+  .vseg+.vseg{border-left:1px solid var(--border-default)}
+  .vseg:hover{color:var(--fg-1)}
+  .vseg.on{background:var(--aus-bright-cyan);color:var(--fg-on-accent)}
+  .vstage{flex:1;min-height:0;background:var(--rk-deep)}
+  .vstage.fit{display:flex;align-items:center;justify-content:center;overflow:hidden;padding:1rem}
+  .vstage.fit img{max-width:100%;max-height:100%;width:auto;height:auto;box-shadow:var(--shadow-3)}
+  .vstage.one{overflow:auto;text-align:center}
+  .vstage.one img{max-width:none;max-height:none;margin:auto}
+
+  .foot{border-top:1px solid var(--border-subtle);color:var(--fg-muted);
+    font-size:.72rem;font-family:var(--font-mono);letter-spacing:.04em;padding:1rem 1.5rem;text-align:center}
+
+  .empty{border:1px dashed var(--border-default);border-radius:var(--radius-lg);
+    padding:3rem 1.5rem;text-align:center;color:var(--fg-3);background:var(--rk-well);font-family:var(--font-mono);font-size:.92rem}
+
+  /* index grid */
+  .grid{display:grid;gap:1.1rem;grid-template-columns:repeat(auto-fill,minmax(248px,1fr))}
+  .card{position:relative;background:var(--rk-panel);border:1px solid var(--border-subtle);
+    border-radius:var(--radius-lg);overflow:hidden;box-shadow:var(--shadow-2);
+    transition:transform .16s var(--ease-out),border-color .16s var(--ease-out),box-shadow .16s var(--ease-out)}
+  .card:hover{transform:translateY(-3px);border-color:var(--aus-cyan);box-shadow:var(--shadow-3),var(--glow-cyan)}
+  .card .thumb{display:block;aspect-ratio:16/10;background:var(--rk-deep);overflow:hidden}
+  .card .thumb img{width:100%;height:100%;object-fit:cover;display:block}
+  .card .ph{width:100%;height:100%;display:flex;align-items:center;justify-content:center;
+    color:var(--fg-3);font-family:var(--font-mono);font-size:.82rem;letter-spacing:.16em;text-transform:uppercase}
+  .card .meta{padding:.7rem .85rem .85rem}
+  .card .name{display:block;font-family:var(--font-display);font-weight:600;color:var(--fg-0);word-break:break-word}
+  .card .name:hover{text-decoration:none;color:var(--aus-bright-cyan)}
+  .card .sub{color:var(--fg-3);font-size:.72rem;font-family:var(--font-mono);letter-spacing:.03em;margin-top:.3rem}
+
+  .wipe{position:absolute;top:.5rem;right:.5rem;margin:0}
+  /* opaque dark control-scrim + always-light glyph — legible over any thumbnail
+     AND in both themes (glyph must NOT follow --fg-*, which flips dark on light). */
+  .wipe button{cursor:pointer;border:1px solid rgba(255,255,255,.16);background:rgba(16,18,25,.86);
+    color:#e9eef0;width:1.9rem;height:1.9rem;border-radius:var(--radius-md);font-size:1.1rem;line-height:1;
+    backdrop-filter:blur(6px);transition:.14s var(--ease-out)}
+  .wipe button:hover{border-color:var(--aus-red);color:#fff;background:var(--aus-red)}
+
+  /* booth page */
+  .boothhead{display:flex;align-items:center;gap:1rem;flex-wrap:wrap;
+    padding-bottom:1rem;margin-bottom:1.4rem;border-bottom:1px solid var(--border-subtle)}
+  .boothhead .back{font-family:var(--font-mono);font-size:.76rem;letter-spacing:.08em;color:var(--fg-3)}
+  .boothhead h1{margin:0;font-family:var(--font-display);font-weight:600;font-size:1.5rem;
+    letter-spacing:-.01em;word-break:break-word;flex:1 1 auto;color:var(--fg-0)}
+  .boothhead .sub{color:var(--fg-3);font-size:.74rem;font-family:var(--font-mono);letter-spacing:.06em}
+  .wipe-lg{position:static}
+  /* red-outline danger button — legible on the dark canvas, fills on hover */
+  .wipe-lg button{width:auto;height:auto;padding:.42rem .85rem;border-radius:var(--radius-md);
+    font-size:.72rem;font-family:var(--font-mono);letter-spacing:.1em;text-transform:uppercase;
+    background:transparent;border-color:var(--aus-red);color:var(--aus-bright-red)}
+  .wipe-lg button:hover{background:var(--aus-red);border-color:var(--aus-red);color:#fff}
+
+  .gallery{display:grid;gap:1.4rem;grid-template-columns:repeat(auto-fill,minmax(320px,1fr))}
+  .item{margin:0;background:var(--rk-panel);border:1px solid var(--border-subtle);border-radius:var(--radius-lg);
+    overflow:hidden;display:flex;flex-direction:column;box-shadow:var(--shadow-2)}
+  .item img,.item video{width:100%;height:auto;display:block;background:var(--rk-deep)}
+  .item audio{width:100%;margin:1.3rem .9rem .4rem;max-width:calc(100% - 1.8rem)}
+  .item .dl{padding:1.4rem .9rem;font-family:var(--font-mono);font-size:.86rem;word-break:break-all}
+  .item figcaption{padding:.6rem .85rem .75rem;color:var(--fg-2);font-size:.78rem;
+    font-family:var(--font-mono);letter-spacing:.02em;border-top:1px solid var(--border-subtle);word-break:break-word}
+  .item-audio figcaption,.item-other figcaption{border-top:none}
+</style>
+</head>
+<body>
+<header class="topbar">
+  <a class="brand" href="/"><span class="dot"></span><span class="name">The&nbsp;Booth</span></a>
+  <span class="tagline">ephemeral media · auto-wipes {{ ttl_hours }}h</span>
+</header>
+<main>{% block content %}{% endblock %}</main>
+<footer class="foot">
+  drop a folder into <code>{{ data_dir }}</code>{% if host %} · {{ host }}{% endif %}
+</footer>
+</body>
+</html>
@@ -0,0 +1,84 @@
+{% extends "base.html" %}
+{% block title %}{{ name }} · The Booth{% endblock %}
+{% block content %}
+<div class="boothhead">
+  <a class="back" href="/">‹ all booths</a>
+  <h1>{{ name }}</h1>
+  <span class="sub">{% if uploaded %}<span class="badge">⬆ pickup</span> {% endif %}{{ items|length }} item{{ '' if items|length == 1 else 's' }} · expires in {{ expires_in|dur }}</span>
+  {% if items %}<a class="dl-link" href="/b/{{ name_url }}/?download=1" title="download this booth as a zip">⬇ zip</a>{% endif %}
+  <form class="wipe wipe-lg" method="post" action="/b/{{ name_url }}/delete"
+        onsubmit="return confirm('Wipe this booth now?')">
+    <button>Wipe now</button>
+  </form>
+</div>
+
+{% if uploaded %}
+<div class="pickup-note">
+  📦 Pickup <code>{{ name }}</code>
+  <button type="button" class="copy-btn" data-copy="{{ name }}" title="copy id to clipboard">⧉ copy</button>
+  — download files below, or on nh3-dev grab <code>~/booth-data/{{ name }}/</code>
+</div>
+{% endif %}
+
+{% if not items %}
+  <div class="empty">This booth is empty.</div>
+{% else %}
+  <div class="gallery">
+    {% for it in items %}
+    <figure class="item item-{{ it.kind }}">
+      {% if it.kind == 'image' %}
+        <a href="view?f={{ it.url }}"><img loading="lazy" src="{{ it.url }}" alt="{{ it.name }}"></a>
+      {% elif it.kind == 'video' %}
+        <video controls preload="metadata" src="{{ it.url }}"></video>
+      {% elif it.kind == 'audio' %}
+        <audio controls preload="metadata" src="{{ it.url }}"></audio>
+      {% else %}
+        <a class="dl" href="{{ it.url }}" download>⬇ {{ it.name }}</a>
+      {% endif %}
+      {% if it.kind == 'other' %}
+        {% if it.caption %}<figcaption><span class="cap-text">{{ it.caption }}</span></figcaption>{% endif %}
+      {% else %}
+        <figcaption>
+          <a class="dl-link" href="{{ it.url }}" download title="download {{ it.name }}">⬇</a>
+          <span class="cap-text">{{ it.caption or it.name }}</span>
+        </figcaption>
+      {% endif %}
+    </figure>
+    {% endfor %}
+  </div>
+{% endif %}
+
+<script>
+  /* Copy-to-clipboard for any .copy-btn[data-copy]. The Booth serves over plain
+     HTTP on a LAN IP, where navigator.clipboard is undefined (secure-context
+     only) — so fall back to a hidden-textarea execCommand('copy'). */
+  (function () {
+    function copyText(t) {
+      if (navigator.clipboard && window.isSecureContext) {
+        return navigator.clipboard.writeText(t);
+      }
+      var ta = document.createElement('textarea');
+      ta.value = t;
+      ta.setAttribute('readonly', '');
+      ta.style.position = 'fixed';
+      ta.style.top = '-1000px';
+      ta.style.opacity = '0';
+      document.body.appendChild(ta);
+      ta.select();
+      try { document.execCommand('copy'); } catch (e) {}
+      document.body.removeChild(ta);
+      return Promise.resolve();
+    }
+    document.querySelectorAll('.copy-btn').forEach(function (btn) {
+      var label = btn.textContent;
+      btn.addEventListener('click', function () {
+        copyText(btn.getAttribute('data-copy')).then(function () {
+          btn.classList.add('copied');
+          btn.textContent = '✓ copied';
+          setTimeout(function () { btn.classList.remove('copied'); btn.textContent = label; }, 1300);
+        });
+      });
+    });
+  })();
+</script>
+{% endblock %}
@@ -0,0 +1,78 @@
+{% extends "base.html" %}
+{% block content %}
+<form class="uploader" method="post" action="/upload" enctype="multipart/form-data">
+  <label class="drop" for="booth-files">
+    <span class="drop-icon">⬆</span>
+    <span class="drop-main">Upload files for pickup</span>
+    <span class="drop-sub" id="drop-sub">drop here, or click to choose · one pickup id, wiped in {{ ttl_hours }}h</span>
+    <input id="booth-files" name="files" type="file" multiple>
+  </label>
+  <button class="up-go" type="submit">Get pickup id →</button>
+</form>
+
+{% if not booths %}
+  <div class="empty">
+    No booths yet. Upload files above, or drop a folder into <code>{{ data_dir }}</code>.
+  </div>
+{% else %}
+  <div class="grid">
+    {% for b in booths %}
+    <article class="card">
+      <a class="thumb" href="/b/{{ b.name_url }}/">
+        {% if b.thumb_url %}
+          <img loading="lazy" src="/b/{{ b.name_url }}/{{ b.thumb_url }}" alt="">
+        {% elif b.has_index %}
+          <div class="ph">▦ page</div>
+        {% elif b.kinds.video %}
+          <div class="ph">▶ video</div>
+        {% elif b.kinds.audio %}
+          <div class="ph">♪ audio</div>
+        {% else %}
+          <div class="ph">◆ files</div>
+        {% endif %}
+        {% if b.uploaded %}<span class="badge">⬆ pickup</span>{% endif %}
+      </a>
+      <div class="meta">
+        <a class="name" href="/b/{{ b.name_url }}/">{{ b.name }}</a>
+        <div class="sub">{{ b.count }} item{{ '' if b.count == 1 else 's' }} · expires in {{ b.expires_in|dur }} · <a class="dl-link" href="/b/{{ b.name_url }}/?download=1" title="download this booth as a zip">⬇ zip</a></div>
+      </div>
+      <form class="wipe" method="post" action="/b/{{ b.name_url }}/delete"
+            onsubmit="return confirm('Wipe booth “{{ b.name }}”?')">
+        <button title="wipe now" aria-label="wipe booth">×</button>
+      </form>
+    </article>
+    {% endfor %}
+  </div>
+{% endif %}
+
+<script>
+  /* progressive enhancement: reflect chosen files + drag-drop onto the panel.
+     With JS off, the native file input + submit still works. */
+  (function () {
+    var input = document.getElementById('booth-files');
+    var sub = document.getElementById('drop-sub');
+    var drop = document.querySelector('.drop');
+    if (!input) return;
+    function show() {
+      var n = input.files ? input.files.length : 0;
+      if (n) {
+        sub.textContent = n + ' file' + (n > 1 ? 's' : '') + ' ready — hit “Get pickup id”';
+        drop.classList.add('has');
+      }
+    }
+    input.addEventListener('change', show);
+    ['dragover', 'dragenter'].forEach(function (e) {
+      drop.addEventListener(e, function (ev) { ev.preventDefault(); drop.classList.add('over'); });
+    });
+    ['dragleave', 'drop'].forEach(function (e) {
+      drop.addEventListener(e, function (ev) { ev.preventDefault(); drop.classList.remove('over'); });
+    });
+    drop.addEventListener('drop', function (ev) {
+      if (ev.dataTransfer && ev.dataTransfer.files.length) {
+        try { input.files = ev.dataTransfer.files; } catch (_) {}
+        show();
+      }
+    });
+  })();
+</script>
+{% endblock %}
@@ -0,0 +1,57 @@
+{% extends "base.html" %}
+{% block title %}{{ file }} · {{ name }} · The Booth{% endblock %}
+{% block content %}
+<div class="viewer">
+  <div class="vbar">
+    <a class="vbtn vx" href="/b/{{ name_url }}/" title="back to gallery (Esc)">✕</a>
+    <span class="vname">{{ file }}</span>
+    <span class="vspacer"></span>
+    <span class="vtoggle" id="vtoggle" style="display:none">
+      <button type="button" class="vseg on" id="btn-fit">Fit</button><button type="button" class="vseg" id="btn-one">1:1</button>
+    </span>
+    <a class="vbtn" href="{{ file_url }}" download title="download {{ file }}">⬇</a>
+  </div>
+  <div class="vstage fit" id="vstage"><img id="vimg" src="{{ file_url }}" alt="{{ file }}"></div>
+</div>
+<script>
+  (function () {
+    var img = document.getElementById('vimg');
+    var stage = document.getElementById('vstage');
+    var toggle = document.getElementById('vtoggle');
+    var bFit = document.getElementById('btn-fit');
+    var bOne = document.getElementById('btn-one');
+    var BACK = {{ ('/b/' ~ name_url ~ '/')|tojson }};
+
+    function setMode(mode) {
+      var fit = mode === 'fit';
+      stage.classList.toggle('fit', fit);
+      stage.classList.toggle('one', !fit);
+      bFit.classList.toggle('on', fit);
+      bOne.classList.toggle('on', !fit);
+    }
+    // "fits" == the image at natural size already sits inside the stage, so Fit
+    // and 1:1 would render identically — in that case we hide the toggle entirely.
+    function fits() {
+      return img.naturalWidth <= stage.clientWidth && img.naturalHeight <= stage.clientHeight;
+    }
+    function evaluate() {
+      if (!img.naturalWidth) return;
+      if (fits()) {
+        toggle.style.display = 'none';
+        setMode('fit');
+      } else {
+        toggle.style.display = 'inline-flex';
+        if (!stage.classList.contains('one')) setMode('fit');
+      }
+    }
+    bFit.addEventListener('click', function () { setMode('fit'); });
+    bOne.addEventListener('click', function () { setMode('one'); });
+    img.addEventListener('load', evaluate);
+    window.addEventListener('resize', evaluate);
+    if (img.complete) evaluate();
+    document.addEventListener('keydown', function (e) {
+      if (e.key === 'Escape') window.location.href = BACK;
+    });
+  })();
+</script>
+{% endblock %}
@@ -0,0 +1,27 @@
+[project]
+name = "booth"
+version = "0.1.5"
+description = "The Booth — a dead-simple standing web server that scans a data dir of drop-folders and renders each as an ephemeral media 'booth' (image/webm/audio auto-gallery, or a folder's own index.html verbatim). Also accepts browser/curl uploads for pickup under a human-readable id. 24h TTL, then the folder is wiped. Fleet tool for CC sessions to surface A/B and smoke results to the operator."
+requires-python = ">=3.11"
+dependencies = [
+    "fastapi>=0.115",
+    "uvicorn[standard]>=0.34",
+    "jinja2>=3.1",
+    "python-multipart>=0.0.9",
+]
+
+[project.optional-dependencies]
+test = [
+    "pytest>=8.0",
+    "httpx>=0.27",  # fastapi TestClient
+]
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["booth"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
@@ -0,0 +1,47 @@
+#!/usr/bin/env bash
+# booth — post media to The Booth (dead simple). A booth is just a folder under
+# $BOOTH_DATA_DIR; this is sugar over mkdir/cp so you get the URL back.
+#
+#   booth new  <name>                 make an empty booth, print its URL
+#   booth add  <name> <file>...       copy files into a booth (creates it), print URL
+#   booth url  <name>                 print a booth's URL
+#   booth ls                          list booths
+#   booth rm   <name>                 wipe a booth now (TTL would eventually anyway)
+#
+# On a host that is NOT nh3-dev, rsync into the data dir instead, e.g.:
+#   rsync -a ./out/  nh3-dev:booth-data/my-run/
+set -euo pipefail
+
+DATA="${BOOTH_DATA_DIR:-$HOME/booth-data}"
+URL="${BOOTH_URL:-http://10.100.10.50:8090}"
+
+usage() { echo "usage: booth {new <name>|add <name> <file>...|url <name>|ls|rm <name>}" >&2; exit 2; }
+
+cmd="${1:-}"; shift || true
+case "$cmd" in
+  new)
+    [ $# -ge 1 ] || usage
+    mkdir -p -- "$DATA/$1"
+    echo "$URL/b/$1/"
+    ;;
+  add)
+    [ $# -ge 2 ] || usage
+    name="$1"; shift
+    mkdir -p -- "$DATA/$name"
+    cp -- "$@" "$DATA/$name/"
+    echo "$URL/b/$name/"
+    ;;
+  url)
+    [ $# -ge 1 ] || usage
+    echo "$URL/b/$1/"
+    ;;
+  ls)
+    ls -1 -- "$DATA" 2>/dev/null || true
+    ;;
+  rm)
+    [ $# -ge 1 ] || usage
+    rm -rf -- "${DATA:?}/$1"
+    echo "wiped $1"
+    ;;
+  *) usage ;;
+esac
@@ -0,0 +1,437 @@
+import os
+import re
+import time
+
+import pytest
+from fastapi.testclient import TestClient
+
+from booth.app import (
+    FAVICON_LINK,
+    build_gallery,
+    classify,
+    create_app,
+    generate_pickup_id,
+    human_dur,
+    is_expired,
+    safe_upload_name,
+    sweep_once,
+    wrap_verbatim_html,
+)
+
+PICKUP_RE = re.compile(r"^(\d{1,2}-[a-z]+|[a-z]+-\d{1,2})$")
+
+
+# ---- pure helpers -----------------------------------------------------------
+
+
+def test_classify():
+    assert classify("a.PNG") == "image"
+    assert classify("clip.webm") == "video"
+    assert classify("v.mp4") == "video"
+    assert classify("song.mp3") == "audio"
+    assert classify("notes.txt") == "other"
+    assert classify("archive.tar.gz") == "other"
+
+
+def test_human_dur():
+    assert human_dur(0) == "expired"
+    assert human_dur(-5) == "expired"
+    assert human_dur(30) == "<1m"
+    assert human_dur(90) == "1m"
+    assert human_dur(3600) == "1h"
+    assert human_dur(3660) == "1h 1m"
+
+
+def _touch(path, when=None):
+    path.parent.mkdir(parents=True, exist_ok=True)
+    path.write_bytes(b"x")
+    if when is not None:
+        os.utime(path, (when, when))
+
+
+def test_is_expired_uses_newest_mtime(tmp_path):
+    booth = tmp_path / "b"
+    _touch(booth / "old.png", when=time.time() - 10_000)
+    # freshly touched second file keeps the booth alive despite the old one
+    _touch(booth / "new.png")
+    assert not is_expired(booth, ttl_seconds=3600)
+
+    old = tmp_path / "stale"
+    t = time.time() - 10_000
+    _touch(old / "x.png", when=t)
+    os.utime(old, (t, t))
+    assert is_expired(old, ttl_seconds=3600)
+
+
+def test_sweep_only_removes_expired(tmp_path):
+    fresh = tmp_path / "fresh"
+    _touch(fresh / "a.png")
+    old = tmp_path / "old"
+    t = time.time() - 10_000
+    _touch(old / "a.png", when=t)
+    os.utime(old, (t, t))
+    dotdir = tmp_path / ".control"
+    t2 = time.time() - 10_000
+    dotdir.mkdir()
+    os.utime(dotdir, (t2, t2))
+
+    wiped = sweep_once(tmp_path, ttl_seconds=3600)
+
+    assert wiped == ["old"]
+    assert fresh.exists()
+    assert not old.exists()
+    assert dotdir.exists()  # dotfolders are never swept
+
+
+def test_build_gallery_folds_caption_sidecars(tmp_path):
+    booth = tmp_path / "b"
+    _touch(booth / "a.png")
+    (booth / "a.txt").write_text("variant A: cudaMalloc")
+    _touch(booth / "b.png")
+    (booth / "b.png.txt").write_text("variant B: cudaMallocAsync")
+    _touch(booth / "loose.txt")  # no media partner -> shown as its own item
+
+    items = build_gallery(booth)
+    by_name = {it["name"]: it for it in items}
+
+    assert by_name["a.png"]["caption"] == "variant A: cudaMalloc"
+    assert by_name["b.png"]["caption"] == "variant B: cudaMallocAsync"
+    assert "a.txt" not in by_name and "b.png.txt" not in by_name
+    assert "loose.txt" in by_name  # a caption with nothing to caption stays visible
+
+
+# ---- HTTP surface -----------------------------------------------------------
+
+
+@pytest.fixture
+def client(tmp_path):
+    app = create_app(tmp_path, ttl_hours=24, start_sweeper=False)
+    return TestClient(app), tmp_path
+
+
+def test_index_empty(client):
+    c, _ = client
+    r = c.get("/")
+    assert r.status_code == 200
+    assert "No booths yet" in r.text
+
+
+def test_index_lists_booth(client):
+    c, data = client
+    _touch(data / "run1" / "a.png")
+    r = c.get("/")
+    assert r.status_code == 200
+    assert "run1" in r.text
+
+
+def test_booth_autogallery_renders_media(client):
+    c, data = client
+    _touch(data / "run1" / "shot.png")
+    r = c.get("/b/run1/")
+    assert r.status_code == 200
+    assert "<img" in r.text
+    assert "shot.png" in r.text
+
+
+def test_booth_serves_own_index_html(client):
+    c, data = client
+    d = data / "custom"
+    d.mkdir()
+    (d / "index.html").write_text("<h1>MY CUSTOM REPORT</h1>")
+    r = c.get("/b/custom/")
+    assert r.status_code == 200
+    assert "MY CUSTOM REPORT" in r.text
+
+
+def test_booth_serves_file(client):
+    c, data = client
+    (data / "run1").mkdir()
+    (data / "run1" / "a.bin").write_bytes(b"\x00\x01payload")
+    r = c.get("/b/run1/a.bin")
+    assert r.status_code == 200
+    assert r.content == b"\x00\x01payload"
+
+
+def test_booth_zip_download(client):
+    # a verbatim index.html booth (no per-file chrome) is still downloadable as a zip
+    c, data = client
+    d = data / "brief"
+    d.mkdir()
+    (d / "index.html").write_text("<h1>BRIEF</h1>")
+    (d / "notes.md").write_text("# notes")
+    r = c.get("/b/brief/?download=1")
+    assert r.status_code == 200
+    assert r.headers["content-type"] == "application/zip"
+    assert "attachment" in r.headers["content-disposition"]
+    assert "brief.zip" in r.headers["content-disposition"]
+    import io as _io
+    import zipfile as _zip
+    assert set(_zip.ZipFile(_io.BytesIO(r.content)).namelist()) == {"index.html", "notes.md"}
+
+
+def test_booth_file_force_download(client):
+    # ?dl=1 forces attachment so html/md/text saves instead of rendering inline
+    c, data = client
+    d = data / "brief"
+    d.mkdir()
+    (d / "index.html").write_text("<h1>BRIEF</h1>")
+    r = c.get("/b/brief/index.html")
+    assert "attachment" not in r.headers.get("content-disposition", "")
+    r2 = c.get("/b/brief/index.html?dl=1")
+    assert r2.status_code == 200
+    assert "attachment" in r2.headers["content-disposition"]
+    assert "index.html" in r2.headers["content-disposition"]
+
+
+def test_missing_booth_404(client):
+    c, _ = client
+    assert c.get("/b/nope/").status_code == 404
+
+
+def test_traversal_rejected(client):
+    c, data = client
+    (data / "run1").mkdir()
+    # a name with a slash or .. can never resolve to a direct child
+    assert c.get("/b/..%2f..%2fetc/").status_code == 404
+    assert c.get("/b/run1/../../../etc/passwd").status_code == 404
+
+
+def test_delete_form_wipes(client):
+    c, data = client
+    _touch(data / "run1" / "a.png")
+    r = c.post("/b/run1/delete", follow_redirects=False)
+    assert r.status_code == 303
+    assert not (data / "run1").exists()
+
+
+def test_delete_api_wipes(client):
+    c, data = client
+    _touch(data / "run1" / "a.png")
+    r = c.request("DELETE", "/b/run1")
+    assert r.status_code == 200
+    assert r.json() == {"wiped": "run1"}
+    assert not (data / "run1").exists()
+
+
+def test_healthz(client):
+    c, data = client
+    _touch(data / "run1" / "a.png")
+    r = c.get("/healthz")
+    assert r.status_code == 200
+    body = r.json()
+    assert body["ok"] is True and body["booths"] == 1
+
+
+# ---- uploads / pickup -------------------------------------------------------
+
+
+def test_generate_pickup_id_format():
+    for _ in range(100):
+        pid = generate_pickup_id(lambda n: False)
+        assert PICKUP_RE.match(pid), pid
+
+
+def test_generate_pickup_id_avoids_collision():
+    taken = {"4-wombat", "star-84"}
+    for _ in range(50):
+        pid = generate_pickup_id(lambda n: n in taken)
+        assert pid not in taken
+
+
+def test_safe_upload_name():
+    assert safe_upload_name("../../etc/passwd", "fb") == "passwd"
+    assert safe_upload_name("C:\\Users\\x\\shot.png", "fb") == "shot.png"
+    assert safe_upload_name("", "fb") == "fb"
+    assert safe_upload_name("   ", "fb") == "fb"
+    assert safe_upload_name(".hidden", "fb") == "hidden"
+    assert safe_upload_name("...", "fb") == "fb"
+
+
+def _upload(client, files):
+    return client.post("/upload", files=files, follow_redirects=False)
+
+
+def test_upload_creates_pickup_booth(client):
+    c, data = client
+    r = _upload(c, [
+        ("files", ("a.png", b"\x89PNG\r\n\x1a\n" + b"0" * 20, "image/png")),
+        ("files", ("notes.txt", b"pick me up", "text/plain")),
+    ])
+    assert r.status_code == 303
+    loc = r.headers["location"]
+    pid = loc.split("/b/")[1].rstrip("/")
+    assert PICKUP_RE.match(pid), pid
+    booth = data / pid
+    assert (booth / "a.png").is_file()
+    assert (booth / "notes.txt").read_bytes() == b"pick me up"
+    assert (booth / ".uploaded").is_file()  # marker present
+
+
+def test_upload_booth_renders_pickup_ui(client):
+    c, data = client
+    r = _upload(c, [("files", ("shot.png", b"\x89PNG" + b"0" * 30, "image/png"))])
+    pid = r.headers["location"].split("/b/")[1].rstrip("/")
+    page = c.get(f"/b/{pid}/")
+    assert page.status_code == 200
+    assert "pickup" in page.text.lower()      # badge / note
+    assert 'download' in page.text            # per-item download link
+    # and it shows up flagged as an upload on the index
+    assert pid in c.get("/").text
+
+
+def test_upload_sanitizes_traversal(client):
+    c, data = client
+    r = _upload(c, [("files", ("../../../etc/passwd", b"x", "text/plain"))])
+    pid = r.headers["location"].split("/b/")[1].rstrip("/")
+    booth = data / pid
+    assert (booth / "passwd").is_file()               # basename only
+    assert not (data.parent / "passwd").exists()       # nothing escaped upward
+
+
+def test_upload_rejects_too_many_files(tmp_path):
+    app = create_app(tmp_path, start_sweeper=False, max_files=2)
+    c = TestClient(app)
+    files = [("files", (f"f{i}.txt", b"x", "text/plain")) for i in range(3)]
+    r = c.post("/upload", files=files, follow_redirects=False)
+    assert r.status_code == 413
+    # no partial booth left behind
+    assert list(tmp_path.iterdir()) == []
+
+
+def test_upload_rejects_too_large(tmp_path):
+    app = create_app(tmp_path, start_sweeper=False, max_upload_mb=0.0001)  # ~104 bytes
+    c = TestClient(app)
+    r = c.post(
+        "/upload",
+        files=[("files", ("big.bin", b"0" * 500, "application/octet-stream"))],
+        follow_redirects=False,
+    )
+    assert r.status_code == 413
+    assert list(tmp_path.iterdir()) == []  # partial write cleaned up
+
+
+def test_upload_dedupes_repeated_names(client):
+    c, data = client
+    r = _upload(c, [
+        ("files", ("shot.png", b"a" * 10, "image/png")),
+        ("files", ("shot.png", b"b" * 10, "image/png")),
+    ])
+    pid = r.headers["location"].split("/b/")[1].rstrip("/")
+    names = sorted(p.name for p in (data / pid).iterdir() if not p.name.startswith("."))
+    assert names == ["shot-1.png", "shot.png"]
+
+
+# ---- image viewer -----------------------------------------------------------
+
+
+def test_image_view_renders(client):
+    c, data = client
+    _touch(data / "run1" / "shot.png")
+    r = c.get("/b/run1/view", params={"f": "shot.png"})
+    assert r.status_code == 200
+    assert 'id="vstage"' in r.text            # the viewer stage
+    assert ">Fit<" in r.text and ">1:1<" in r.text
+    assert "shot.png" in r.text
+
+
+def test_gallery_image_links_to_viewer(client):
+    c, data = client
+    _touch(data / "run1" / "shot.png")
+    page = c.get("/b/run1/")
+    assert "view?f=shot.png" in page.text     # gallery routes images to the viewer, not the raw file
+
+
+def test_image_view_missing_404(client):
+    c, data = client
+    (data / "run1").mkdir()
+    assert c.get("/b/run1/view", params={"f": "nope.png"}).status_code == 404
+
+
+def test_image_view_traversal_404(client):
+    c, data = client
+    (data / "run1").mkdir()
+    assert c.get("/b/run1/view", params={"f": "../../etc/passwd"}).status_code == 404
+
+
+def test_image_view_nonimage_redirects_to_raw(client):
+    c, data = client
+    _touch(data / "run1" / "notes.txt")
+    r = c.get("/b/run1/view", params={"f": "notes.txt"}, follow_redirects=False)
+    assert r.status_code == 307
+    assert r.headers["location"] == "/b/run1/notes.txt"
+
+
+# ---- verbatim-index.html wrapper --------------------------------------------
+
+
+def test_wrap_injects_chip_and_favicon():
+    html = "<html><head><title>Brief</title></head><body><h1>REPORT</h1></body></html>"
+    out = wrap_verbatim_html(html)
+    assert 'class="booth-nav-home"' in out       # floating back chip
+    assert 'href="/"' in out                      # points at the main booth index
+    assert "all booths" in out
+    assert FAVICON_LINK in out                    # favicon inherited
+    assert "<h1>REPORT</h1>" in out               # original content preserved
+    # favicon lands in the head, chip lands in the body
+    assert out.index(FAVICON_LINK) < out.index("</head>")
+    assert out.index("booth-nav-home") > out.index("<body>")
+
+
+def test_wrap_respects_existing_favicon():
+    html = '<html><head><link rel="icon" href="data:image/png;base64,AAAA"></head><body>x</body></html>'
+    out = wrap_verbatim_html(html)
+    assert FAVICON_LINK not in out                # the page's own icon wins
+    assert out.count('rel="icon"') == 1
+    assert 'class="booth-nav-home"' in out        # chip is still added
+
+
+def test_wrap_bare_fragment_appends_chip():
+    out = wrap_verbatim_html("<h1>bare fragment</h1>")  # no doctype/head/body
+    assert 'class="booth-nav-home"' in out
+    assert out.rstrip().endswith("</style>")            # chip appended at the end
+    assert FAVICON_LINK in out                           # no doctype -> safe to prepend the icon
+    assert out.index(FAVICON_LINK) < out.index("bare")   # icon ahead of content (implied head)
+
+
+def test_wrap_no_head_injects_favicon():
+    out = wrap_verbatim_html("<body><h1>no head</h1></body>")
+    assert 'class="booth-nav-home"' in out
+    assert FAVICON_LINK in out                           # injected even without an explicit <head>
+
+
+def test_wrap_compact_doctype_stays_first():
+    # the real-booth shape: compact HTML, no explicit head/body. The injection must
+    # not push anything ahead of the doctype (quirks mode) or past the charset window.
+    html = "<!doctype html><meta charset=utf-8><title>T</title><style>body{margin:0}</style><h1>REPORT</h1>"
+    out = wrap_verbatim_html(html)
+    assert out.lstrip().lower().startswith("<!doctype")       # doctype still first -> standards mode
+    assert FAVICON_LINK in out
+    assert out.index(FAVICON_LINK) < out.index("<h1>")         # icon in the implied head, before content
+    assert out.index("charset") < 1024                         # charset meta stays in the detection window
+    assert 'class="booth-nav-home"' in out
+    assert out.index("booth-nav-home") > out.index("<h1>REPORT</h1>")  # chip appended after content
+
+
+def test_verbatim_booth_wrapped_with_back_chip(client):
+    c, data = client
+    d = data / "brief"
+    d.mkdir()
+    (d / "index.html").write_text("<html><head></head><body><h1>BRIEF</h1></body></html>")
+    r = c.get("/b/brief/")
+    assert r.status_code == 200
+    assert "BRIEF" in r.text                       # content preserved
+    assert 'class="booth-nav-home"' in r.text      # back chip injected
+    assert 'href="/"' in r.text
+    assert 'rel="icon"' in r.text                  # favicon inherited
+
+
+def test_verbatim_index_raw_file_route_unwrapped(client):
+    # the file route (/b/<name>/index.html) still serves the raw bytes — the chip
+    # only rides on the booth view (/b/<name>/), so downloads/assets stay verbatim
+    c, data = client
+    d = data / "brief"
+    d.mkdir()
+    (d / "index.html").write_text("<html><body><h1>BRIEF</h1></body></html>")
+    r = c.get("/b/brief/index.html")
+    assert r.status_code == 200
+    assert "booth-nav-home" not in r.text
@@ -0,0 +1,66 @@
+# heretic2-nvfp4-quant — fast char-rp-reasoning seat (NVFP4 + MTP)
+
+Local NVFP4 quant of **NEO-CODE = Heretic2-Thinking** (Qwen3.6-27B) with the
+Qwen3.6 **MTP head grafted back**, for a ~2.5–4× faster vLLM/MTP
+`char-rp-reasoning` seat (buys reasoning-budget headroom → better GM planning
+inside soong's latency window). R36 fast-seat spike, 2026-07-14.
+
+Fleet-first **local** NVFP4 quant — every other fleet NVFP4 model is *pulled*
+pre-quantized; Heretic2 has none published, so we quantize it.
+
+## 2026-07-14 status — gibberish FIXED, format PIVOTED to modelopt for MTP
+
+- **Root cause of the `!!!!` was the quant NAMESPACE**, not calib/scheme: `quant_nvfp4.py`
+  loaded via `AutoModelForCausalLM` → text-only `Qwen3_5ForCausalLM` → flat `model.layers.*`
+  keys, but vLLM 0.24 serves only `Qwen3_5ForConditionalGeneration`, whose weight mapper needs
+  `model.language_model.*`. **Fixed** by loading as `AutoModelForImageTextToText`
+  (= `Qwen3_5ForConditionalGeneration`) → keys born `model.language_model.*` + `model.visual.*`.
+  NVFP4 now serves **coherent**.
+- **But this (llm-compressor / compressed-tensors) format can't deliver the speed goal:** base
+  NVFP4 ≈ 53 tok/s ≈ the GGUF seat's ~59.5 at batch-1 (no single-stream win), and **MTP =
+  0% acceptance** (vLLM's `Qwen3_5MTP` drafter won't load the bf16 mtp head off a compressed-tensors
+  main model). The mtp tensors are identical to AEON's; the blocker is purely the main-model format.
+- **→ Working native MTP requires the MODELOPT format** (what AEON uses, ~3.3/3 accept). Use
+  **`quant_modelopt.py`** (nvidia-modelopt PTQ; same graft + same `splice_mtp.py` + serve
+  `--quantization modelopt`). AEON `/tank/aimodels/qwen36-27b-aeon-nvfp4` (`vllm-aeon-rp`) is the
+  exact reference. `quant_nvfp4.py` (below) is kept for the coherent-but-MTP-inert compressed-tensors
+  artifact and as the namespace-fix record.
+
+## Fire sequence
+1. **`graft_mtp.py`** — graft the 15 base-Qwen3.6 MTP tensors into Heretic2 BF16.
+   CPU-only, no GPU window. (Heretic2's finetune dropped the head; config declares
+   `mtp_num_hidden_layers=1` but ships 0 `mtp.*` tensors — verified.)
+2. **`quant_nvfp4.py`** — llm-compressor NVFP4, Linear only; **GDN/vision/lm-head/
+   norms/MTP kept BF16** (robbatt deckard-nvfp4 recipe + MTP). NEEDS a freed
+   Blackwell GPU (~55 GB) + an llmcompressor env (run in a vLLM container:
+   `pip install llmcompressor` on `vllm/vllm-openai:v0.24.0`).
+   - baseline: `--calib-mode text --calib neuralmagic/calibration` (AEON control)
+   - production: `--calib-mode chat --calib <512-row mix>` (brokkr/Dvalin) — rows
+     rendered via `apply_chat_template(enable_thinking=True)` so the forward-pass
+     sees the qwen3_coder tool-call XML = the seat's native activations.
+3. **serve** (pantheon-27b-mtp-nvfp4 pattern): `vllm --quantization compressed-tensors
+   --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":3}'
+   --reasoning-parser qwen3 --tool-call-parser qwen3_coder --enable-auto-tool-choice`.
+4. **P00 acceptance** (brokkr): soong 9-tool k5 rig on the quant — must hold
+   ~0.967 / perfect `attach_tool`. This is the **authoritative #355 check, NOT KL**
+   (KL can pass while the structured-tool path regresses).
+
+## Gates
+- **GPU window** — both ana-ml2 Blackwell GPUs run ~full; NVFP4 is Blackwell-only
+  (irv-ml1's Ampere can't). The ~30–60 min quant needs a brief off-peak window
+  freeing a GPU. *Operator's call.*
+- **Production calib** — brokkr/Dvalin assembling the 512-row mix; the tool-call-XML
+  slice (128 rows, 53 `attach_tool`) is ready. The AEON-baseline is fireable now.
+
+## Artifacts (on ana-ml2)
+- Heretic2 BF16 (target): `/tank/aimodels/huggingface/hub/models--DavidAU--Qwen3.6-27B-Heretic2-Uncensored-Finetune-Thinking`
+- base Qwen3.6-27B (MTP source): `/tank/aimodels/huggingface/hub/models--Qwen--Qwen3.6-27B` (15 mtp.* tensors, shards 13+15)
+- AEON-baseline calib: `neuralmagic/calibration` (HF)
+- `soong-tools-v0.3.13.json` — live 9-tool schema (structure source-of-truth;
+  **calib uses the PREFIXED `bifrost.soong-lab.*` runtime names** the seat emits)
+- `extract_soong_tools.py` — how that schema was pulled from the deployed backend
+
+## Serve target
+Replaces the current llama.cpp GGUF `char-rp-reasoning` seat (~59.5 tok/s) once
+P00 passes. Deckard stays staged as rollback; the GGUF seat is the fallback until
+the NVFP4 seat is validated + cut over.
@@ -0,0 +1,132 @@
+[
+  {
+    "type": "function",
+    "function": {
+      "name": "bifrost.soong-lab.set_name",
+      "parameters": {
+        "type": "object",
+        "required": ["name"],
+        "properties": {
+          "name": { "type": "string", "maxLength": 128, "minLength": 1 }
+        }
+      },
+      "description": "Set the agent's name (its display name; export slugifies it)."
+    }
+  },
+  {
+    "type": "function",
+    "function": {
+      "name": "bifrost.soong-lab.set_role",
+      "parameters": {
+        "type": "object",
+        "required": ["role"],
+        "properties": {
+          "role": {
+            "enum": ["assistant", "thoughtful-assistant", "character", "thoughtful-character"],
+            "type": "string"
+          }
+        }
+      },
+      "description": "Set the agent's model-role — one of: assistant (general), thoughtful-assistant (reasoning general), character (RP/writing-tuned), thoughtful-character (reasoning RP). Export ships it as the native agents.define `role`."
+    }
+  },
+  {
+    "type": "function",
+    "function": {
+      "name": "bifrost.soong-lab.set_ocean",
+      "parameters": {
+        "type": "object",
+        "properties": {
+          "A": { "type": "number", "maximum": 1, "minimum": -1 },
+          "C": { "type": "number", "maximum": 1, "minimum": -1 },
+          "E": { "type": "number", "maximum": 1, "minimum": -1 },
+          "N": { "type": "number", "maximum": 1, "minimum": -1 },
+          "O": { "type": "number", "maximum": 1, "minimum": -1 }
+        },
+        "minProperties": 1
+      },
+      "description": "Merge OCEAN dials (each in [-1,1]); re-derives the disposition."
+    }
+  },
+  {
+    "type": "function",
+    "function": {
+      "name": "bifrost.soong-lab.edit_prompt",
+      "parameters": {
+        "type": "object",
+        "required": ["system_prompt"],
+        "properties": {
+          "system_prompt": { "type": "string", "maxLength": 32768, "minLength": 1 }
+        }
+      },
+      "description": "Replace the authored role/instructions block (the exported system prompt)."
+    }
+  },
+  {
+    "type": "function",
+    "function": {
+      "name": "bifrost.soong-lab.attach_tool",
+      "parameters": {
+        "type": "object",
+        "required": ["id", "name"],
+        "properties": {
+          "id": { "type": "string", "maxLength": 256, "minLength": 1 },
+          "name": { "type": "string", "maxLength": 256, "minLength": 1 },
+          "description": { "type": "string", "maxLength": 2048 }
+        }
+      },
+      "description": "Attach (upsert by id) a tool the designed agent will bind via Bifrost."
+    }
+  },
+  {
+    "type": "function",
+    "function": {
+      "name": "bifrost.soong-lab.author_first_message",
+      "parameters": {
+        "type": "object",
+        "required": ["first_message"],
+        "properties": {
+          "first_message": { "type": "string", "maxLength": 32768, "minLength": 1 }
+        }
+      },
+      "description": "Set the agent's in-character opening turn (tone + format by example)."
+    }
+  },
+  {
+    "type": "function",
+    "function": {
+      "name": "bifrost.soong-lab.edit_psych_profile",
+      "parameters": {
+        "type": "object",
+        "required": ["psych_profile"],
+        "properties": {
+          "psych_profile": { "type": "string", "maxLength": 8192, "minLength": 1 }
+        }
+      },
+      "description": "Author the character's Psychological Profile & Experience — a ~150-300 word prose lens (the self-report producer reads it every turn) covering four dimensions: disposition/appraisal bent, attention/salience focus, values/yardstick, and formative history. Integrate them as one paragraph and CLOSE ON ATTENTION — what the character characteristically notices ('...notices who is unwell, what is left unsaid'). Guardrails: (1) NEVER name a per-event output emotion ('is anxious', 'feels hurt') — naming PRIMES it, so it fires regardless of the event; describe the bent and let emotion follow from the appraisal. (2) Magnitude lives in the OCEAN dials, not the prose (don't narrate 'comes apart'/'takes it hard'). (3) Appraisal-STYLE yes ('reads others charitably until she can't'), output-emotion no. (4) Salience is character-relative — never rewrite what happened."
+    }
+  },
+  {
+    "type": "function",
+    "function": {
+      "name": "bifrost.soong-lab.generate_portrait",
+      "parameters": {
+        "type": "object",
+        "required": ["description"],
+        "properties": {
+          "description": { "type": "string", "maxLength": 500, "minLength": 1 },
+          "style_mode": { "enum": ["anime", "cartoon", "photoreal"], "type": "string" }
+        }
+      },
+      "description": "Generate the agent's persona portrait NOW from a visual `description` you author — the persona's LOOK (e.g. 'a stern woman in her 40s, short dark hair, sharp jaw'). A persona is personality, not appearance, so YOU compose the appearance in conversation and pass it here; describe only the subject, never the art style. `style_mode` (anime | cartoon | photoreal) selects the curated render style — the style preset owns the look. Generation runs in the background; the portrait pane shows it when ready. Offer/generate a portrait once the look is settled."
+    }
+  },
+  {
+    "type": "function",
+    "function": {
+      "name": "bifrost.soong-lab.export",
+      "parameters": { "type": "object", "properties": {} },
+      "description": "Export the native payload + sidecar bundle (stub — lands in E5)."
+    }
+  }
+]
@@ -0,0 +1,44 @@
+#!/usr/bin/env python3
+"""Extract soong-lab's live 9 design-tool schemas as OpenAI-function form.
+
+Run inside the deployed soong-lab venv (backend/.venv) with cwd=backend so
+`soong_lab` imports. Monkeypatches register_tool to capture each tool's
+name/input_schema/description at registration (handlers are never invoked, so
+dummy store/portrait_service are fine). Emits a `tools` array in the shape the
+NVFP4 calib tool-call-XML slice needs (brokkr R36 spec, 2026-07-14).
+"""
+import json
+import soong_lab.bifrost.tools as T
+
+captured = []
+
+
+def _fake_register_tool(registry, name, input_schema, handler=None, description=None, **kw):
+    captured.append({
+        "type": "function",
+        "function": {
+            "name": name,
+            "description": description or "",
+            "parameters": input_schema,
+        },
+    })
+
+
+T.register_tool = _fake_register_tool
+
+
+class _Dummy:
+    def __getattr__(self, k):
+        return _Dummy()
+
+    def __call__(self, *a, **k):
+        return _Dummy()
+
+
+T.register_design_tools(_Dummy(), _Dummy(), _Dummy())
+
+out = "/tmp/soong_tools_v0313.json"
+with open(out, "w") as f:
+    json.dump(captured, f, indent=2)
+print(f"captured {len(captured)} tools -> {out}")
+print("names:", [c["function"]["name"] for c in captured])
@@ -0,0 +1,55 @@
+#!/usr/bin/env python3
+"""Splice the 15 BF16 mtp.* tensors into the modelopt NVFP4 quant output → the servable seat.
+
+Runs AFTER quant_modelopt.py. Copies heretic2-modelopt-nvfp4 → heretic2-modelopt-nvfp4-mtp,
+splices the grafted BF16 mtp head into the single shard (transformers never builds an mtp module
+at load, so mtp is always post-quant-spliced — same as AEON/pantheon), and adds the mtp module
+names to config.json exclude_modules for tidiness. NOTE: the actual thing that keeps the mtp head
+BF16 at serve time is the sitecustomize MTP workaround (see runbook landmine #4); the config
+exclude here is belt-and-suspenders and does NOT by itself prevent the drafter-quant crash.
+
+Run in a vLLM container (root; /tank/aimodels files are root-owned):
+  docker run --rm -v /tank/aimodels:/tank/aimodels -v /home/lkraven:/lk \
+    --entrypoint python3 vllm/vllm-openai:v0.24.0 /lk/finalize_modelopt_mtp.py
+"""
+import json
+import os
+import shutil
+
+from safetensors import safe_open
+from safetensors.torch import save_file
+
+WORK = "/tank/aimodels/heretic2-nvfp4-work"
+SRC = f"{WORK}/heretic2-modelopt-nvfp4"
+DST = f"{WORK}/heretic2-modelopt-nvfp4-mtp"
+GRAFT = f"{WORK}/heretic2-mtp-bf16"
+
+if os.path.exists(DST):
+    shutil.rmtree(DST)
+print(f"copying {SRC} -> {DST}", flush=True)
+shutil.copytree(SRC, DST)
+
+out_st = f"{DST}/model.safetensors"   # single shard (quant_modelopt.py forces max_shard_size huge)
+mtp_st = f"{GRAFT}/model-mtp.safetensors"
+tensors = {}
+with safe_open(out_st, framework="pt") as f:
+    for k in f.keys():
+        tensors[k] = f.get_tensor(k)
+n_main = len(tensors)
+with safe_open(mtp_st, framework="pt") as f:
+    mtp_keys = list(f.keys())
+    for k in mtp_keys:
+        tensors[k] = f.get_tensor(k)
+assert not any("mtp" in k.lower() for k in list(tensors)[:n_main]), "output already had mtp?"
+save_file(tensors, out_st, metadata={"format": "pt"})
+print(f"spliced {len(mtp_keys)} bf16 mtp tensors -> {len(tensors)} total", flush=True)
+
+cfgp = f"{DST}/config.json"
+cfg = json.load(open(cfgp))
+qc = cfg.setdefault("quantization_config", {})
+exc = qc.setdefault("exclude_modules", [])
+mtp_mods = sorted({k.rsplit(".", 1)[0] for k in mtp_keys})
+exc.extend(m for m in mtp_mods if m not in exc)
+json.dump(cfg, open(cfgp, "w"), indent=2)
+print(f"config exclude_modules += {len(mtp_mods)} mtp modules; total {len(exc)}", flush=True)
+print(f"DONE: {DST}", flush=True)
@@ -0,0 +1,139 @@
+#!/usr/bin/env python3
+"""Graft the Qwen3.6-27B MTP (multi-token-prediction) head into Heretic2-Thinking.
+
+DavidAU's Heretic2-Uncensored-Finetune-Thinking finetune dropped the MTP head
+(config declares mtp_num_hidden_layers=1 but ships ZERO mtp.* weight tensors),
+so the base Qwen/Qwen3.6-27B's 15 MTP tensors must be grafted back in BF16
+before NVFP4 quantization — this is what lets the served seat use `qwen3_5_mtp`
+speculative decoding (n=3), the pantheon-27b-mtp-nvfp4 pattern proven on our
+Blackwell. (R36 fast-seat spike, brokkr calib spec 2026-07-14.)
+
+CPU-only safetensors surgery — no GPU / no quant-window needed. Space-efficient:
+symlinks Heretic2's large shards, adds one small model-mtp.safetensors, and
+writes a merged index. Idempotent-ish: refuses to clobber a non-empty OUT_DIR.
+
+Usage (on ana-ml2):
+    python3 graft_mtp.py \
+        --heretic2 /tank/aimodels/huggingface/hub/models--DavidAU--Qwen3.6-27B-Heretic2-Uncensored-Finetune-Thinking/snapshots/<hash> \
+        --base     /tank/aimodels/huggingface/hub/models--Qwen--Qwen3.6-27B/snapshots/<hash> \
+        --out      /tank/aimodels/heretic2-mtp-bf16
+"""
+import argparse
+import json
+import os
+import shutil
+import sys
+
+import torch
+from safetensors import safe_open
+from safetensors.torch import save_file
+
+MTP_SHARD = "model-mtp.safetensors"
+# MTP-related config keys we ensure are present on the grafted model (copied from
+# base if Heretic2's config is missing any).
+MTP_CFG_KEYS = ("mtp_num_hidden_layers", "mtp_use_dedicated_embeddings")
+
+
+def _is_mtp(name: str) -> bool:
+    n = name.lower()
+    return n.startswith("mtp.") or "mtp." in n or "nextn" in n
+
+
+def _tensor_nbytes(t: torch.Tensor) -> int:
+    return t.numel() * t.element_size()
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--heretic2", required=True, help="Heretic2 BF16 snapshot dir (quant target)")
+    ap.add_argument("--base", required=True, help="Qwen/Qwen3.6-27B snapshot dir (MTP source)")
+    ap.add_argument("--out", required=True, help="output dir for the grafted BF16 model")
+    args = ap.parse_args()
+
+    her, base, out = args.heretic2, args.base, args.out
+    for d in (her, base):
+        if not os.path.isfile(os.path.join(d, "model.safetensors.index.json")):
+            print(f"ERROR: no index.json in {d}", file=sys.stderr)
+            return 2
+    if os.path.isdir(out) and os.listdir(out):
+        print(f"ERROR: {out} exists and is non-empty — refusing to clobber", file=sys.stderr)
+        return 2
+    os.makedirs(out, exist_ok=True)
+
+    her_idx = json.load(open(os.path.join(her, "model.safetensors.index.json")))
+    base_idx = json.load(open(os.path.join(base, "model.safetensors.index.json")))
+
+    # sanity: Heretic2 must NOT already have MTP weights; base MUST.
+    her_mtp = [k for k in her_idx["weight_map"] if _is_mtp(k)]
+    base_mtp = [k for k in base_idx["weight_map"] if _is_mtp(k)]
+    if her_mtp:
+        print(f"ERROR: Heretic2 already has {len(her_mtp)} mtp tensors — graft not needed", file=sys.stderr)
+        return 2
+    if not base_mtp:
+        print("ERROR: base has no mtp tensors — wrong source model", file=sys.stderr)
+        return 2
+    print(f"grafting {len(base_mtp)} MTP tensors from base into Heretic2 ({len(her_idx['weight_map'])} tensors)")
+
+    # 1) stage Heretic2: symlink big shards, copy everything else.
+    for fn in sorted(os.listdir(her)):
+        src = os.path.join(her, fn)
+        if not os.path.isfile(src):
+            continue
+        dst = os.path.join(out, fn)
+        if fn.endswith(".safetensors"):
+            os.symlink(os.path.realpath(src), dst)
+        elif fn != "model.safetensors.index.json":  # index rewritten below
+            shutil.copy2(src, dst)
+
+    # 2) pull the MTP tensors out of base's shards into one new shard (BF16 preserved).
+    base_mtp_shards = sorted({base_idx["weight_map"][k] for k in base_mtp})
+    mtp_tensors = {}
+    for shard in base_mtp_shards:
+        with safe_open(os.path.join(base, shard), framework="pt") as f:
+            for name in f.keys():
+                if _is_mtp(name):
+                    mtp_tensors[name] = f.get_tensor(name)
+    assert len(mtp_tensors) == len(base_mtp), f"expected {len(base_mtp)} mtp tensors, got {len(mtp_tensors)}"
+    dtypes = {str(t.dtype) for t in mtp_tensors.values()}
+    print(f"  extracted {len(mtp_tensors)} mtp tensors, dtypes={dtypes}")
+    save_file(mtp_tensors, os.path.join(out, MTP_SHARD), metadata={"format": "pt"})
+
+    # 3) merged index: Heretic2 weight_map + the mtp tensors -> the new shard.
+    new_map = dict(her_idx["weight_map"])
+    added_bytes = 0
+    for name, t in mtp_tensors.items():
+        new_map[name] = MTP_SHARD
+        added_bytes += _tensor_nbytes(t)
+    meta = dict(her_idx.get("metadata", {}))
+    if "total_size" in meta:
+        meta["total_size"] = int(meta["total_size"]) + added_bytes
+    out_idx = {"metadata": meta, "weight_map": new_map}
+    json.dump(out_idx, open(os.path.join(out, "model.safetensors.index.json"), "w"), indent=2)
+
+    # 4) ensure config has complete MTP config (copy from base if Heretic2 lacks a key).
+    cfg_path = os.path.join(out, "config.json")
+    cfg = json.load(open(cfg_path))
+    base_cfg = json.load(open(os.path.join(base, "config.json")))
+
+    def _get(d, k):
+        return d.get(k, d.get("text_config", {}).get(k))
+
+    changed = []
+    for k in MTP_CFG_KEYS:
+        if _get(cfg, k) is None and _get(base_cfg, k) is not None:
+            cfg[k] = _get(base_cfg, k)
+            changed.append(k)
+    if changed:
+        json.dump(cfg, open(cfg_path, "w"), indent=2)
+    print(f"  config MTP keys filled from base: {changed or 'none (already complete)'}")
+
+    total = len(new_map)
+    print(f"DONE -> {out}")
+    print(f"  total tensors: {total} (Heretic2 {len(her_idx['weight_map'])} + MTP {len(mtp_tensors)})")
+    print(f"  new shard: {MTP_SHARD} (+{added_bytes/1e9:.2f} GB)")
+    print("  next: NVFP4 quant (quant_nvfp4.py) — keeps mtp.*/GDN/vision/lm-head/norms in BF16")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())
@@ -0,0 +1,151 @@
+#!/usr/bin/env python3
+"""NVFP4-quantize the MTP-grafted Heretic2 model via NVIDIA nvidia-modelopt (MODELOPT format).
+
+Sibling of quant_nvfp4.py (llm-compressor / compressed-tensors) but produces the **modelopt**
+NVFP4 format instead. Why it exists: the compressed-tensors path (quant_nvfp4.py) serves COHERENT
+but its MTP is 0% acceptance — vLLM's `Qwen3_5MTP` speculative-decode drafter only loads the bf16
+MTP head off a **modelopt** main-model checkpoint (the mtp tensors are byte-identical between the
+two formats; the difference is purely how the main model's quantized weights/scales are stored).
+So working native MTP (the ~2-4x goal) requires this format. (2026-07-14 modelopt pivot.)
+
+Reference = AEON `/tank/aimodels/qwen36-27b-aeon-nvfp4` (served by vllm-aeon-rp). Its
+hf_quant_config.json says: quant_algo NVFP4 (W4A4, group_size 16), targets Linear, exclude
+`lm_head` + `model.visual*` + every `linear_attn*` (the GDN). This script matches that EXACTLY.
+`embed_tokens` is an Embedding (not a Linear target) so it is never quantized — no need to exclude.
+
+Pipeline (unchanged except THIS quant step swaps llm-compressor -> modelopt):
+  graft_mtp.py  ->  quant_modelopt.py (this)  ->  splice_mtp.py (bf16 mtp)  ->  serve
+  serve: vllm --quantization modelopt --speculative-config
+         '{"method":"qwen3_5_mtp","num_speculative_tokens":3}'  (AEON vllm-aeon-rp is the ref)
+
+The MTP head is NOT in the module tree at load (transformers builds no mtp module for either
+Qwen3_5 class), so the 15 bf16 mtp.* tensors are spliced back AFTER export — same as pantheon/AEON.
+
+Load class is AutoModelForImageTextToText (= Qwen3_5ForConditionalGeneration) so weight keys are
+born `model.language_model.*` + `model.visual.*` — the namespace vLLM's ConditionalGeneration
+loader expects. (The compressed-tensors-path gibberish was a text-only-namespace bug; same fix.)
+
+Run in a vLLM container on the freed GPU0 (nvidia-modelopt[hf] per the transformers-compat warning):
+  docker run --gpus '"device=0"' --ipc host -v /tank/aimodels:/tank/aimodels -v /home/lkraven:/lk \
+    --entrypoint bash vllm/vllm-openai:v0.24.0 -c \
+    "pip install -q 'nvidia-modelopt[hf]' tiktoken sentencepiece && python3 /lk/quant_modelopt.py \
+       --model /tank/aimodels/heretic2-nvfp4-work/heretic2-mtp-bf16 \
+       --calib-mode chat --calib /tank/aimodels/heretic2-nvfp4-work/production_calib_512.jsonl \
+       --num-samples 512 --seqlen 8192 \
+       --out /tank/aimodels/heretic2-nvfp4-work/heretic2-modelopt-nvfp4"
+"""
+import argparse
+import copy
+import json
+import sys
+
+
+def load_calib_text(name, tokenizer, n, seqlen):
+    from datasets import load_dataset
+    ds = load_dataset(name, split="train").shuffle(seed=42).select(range(min(n, 100000)))
+    col = "text" if "text" in ds.column_names else ds.column_names[0]
+    return [tokenizer(x[col], truncation=True, max_length=seqlen) for x in ds.select(range(n))]
+
+
+def load_calib_chat(path, tokenizer, seqlen, n):
+    # Identical to quant_nvfp4.py: render each row via the Qwen3.6 chat template with thinking on
+    # so the forward-pass sees the seat's native tool-call XML activations. tool_call arguments may
+    # arrive as OpenAI wire-form JSON strings; the template does .items() on them -> parse to dict.
+    rows = [json.loads(l) for l in open(path) if l.strip()][:n]
+    out = []
+    for r in rows:
+        for m in r["messages"]:
+            for tc in (m.get("tool_calls") or []):
+                a = tc.get("function", {}).get("arguments")
+                if isinstance(a, str):
+                    tc["function"]["arguments"] = json.loads(a)
+        text = tokenizer.apply_chat_template(
+            r["messages"], tools=r.get("tools"),
+            tokenize=False, add_generation_prompt=False, enable_thinking=True,
+        )
+        out.append(tokenizer(text, truncation=True, max_length=seqlen))
+    return out
+
+
+def build_nvfp4_cfg(mtq):
+    """NVFP4 W4A4 (group_size 16) matching AEON's exclusions. NVFP4_DEFAULT_CFG already disables
+    lm_head + linear_attn.conv1d/in_proj_a/in_proj_b + mlp.gate; append the FULL linear_attn (GDN)
+    and the vision tower so only the standard attn/MLP Linears get NVFP4 (later entries override)."""
+    cfg = copy.deepcopy(mtq.NVFP4_DEFAULT_CFG)
+    cfg["quant_cfg"].append({"quantizer_name": "*linear_attn*", "enable": False})
+    cfg["quant_cfg"].append({"quantizer_name": "*visual*", "enable": False})
+    cfg["quant_cfg"].append({"quantizer_name": "*mtp*", "enable": False})  # moot (not in tree); belt+suspenders
+    return cfg
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--model", required=True)
+    ap.add_argument("--calib-mode", choices=["text", "chat"], required=True)
+    ap.add_argument("--calib", required=True)
+    ap.add_argument("--out", required=True)
+    ap.add_argument("--num-samples", type=int, default=512)
+    ap.add_argument("--seqlen", type=int, default=8192)
+    args = ap.parse_args()
+
+    import torch
+    from transformers import AutoModelForImageTextToText, AutoTokenizer
+    import modelopt.torch.quantization as mtq
+    from modelopt.torch.export import export_hf_checkpoint
+
+    # modelopt 0.45 + transformers 5.12.1 compat guard. transformers 5.x exposes `FusedMoE` as a
+    # FUNCTION, but modelopt registers it in QuantModuleRegistry expecting an nn.Module class, so the
+    # registry scan (register_fused_experts_on_the_fly -> _get_registered_nn_class) does
+    # `issubclass(nn_cls, <function FusedMoE>)` and dies with "arg 2 must be a class". Our model is
+    # DENSE (no FusedMoE) so skipping non-class registry entries is safe. Guard the scan:
+    from modelopt.torch.opt import dynamic as _mo_dyn
+
+    def _grnc_safe(self, nn_cls):
+        for nn_cls_ in self._registry:
+            if (isinstance(nn_cls_, type) and issubclass(nn_cls, nn_cls_)
+                    and nn_cls.forward is nn_cls_.forward):
+                return nn_cls_
+        return None
+
+    _mo_dyn._DMRegistryCls._get_registered_nn_class = _grnc_safe
+
+    print(f"loading grafted model (multimodal ConditionalGeneration): {args.model}", flush=True)
+    model = AutoModelForImageTextToText.from_pretrained(
+        args.model, torch_dtype="auto", device_map="auto", trust_remote_code=True,
+    )
+    model.eval()
+    tok = AutoTokenizer.from_pretrained(args.model, trust_remote_code=True)
+
+    print(f"building calibration ({args.calib_mode}, up to {args.num_samples} @ seq {args.seqlen})", flush=True)
+    if args.calib_mode == "text":
+        calib = load_calib_text(args.calib, tok, args.num_samples, args.seqlen)
+    else:
+        calib = load_calib_chat(args.calib, tok, args.seqlen, args.num_samples)
+    print(f"  {len(calib)} calibration rows", flush=True)
+
+    def forward_loop(m):
+        with torch.no_grad():
+            for i, row in enumerate(calib):
+                ids = torch.tensor([row["input_ids"]], device=m.device)
+                m(input_ids=ids)
+                if (i + 1) % 64 == 0:
+                    print(f"  calib {i + 1}/{len(calib)}", flush=True)
+
+    cfg = build_nvfp4_cfg(mtq)
+    print("running modelopt NVFP4 PTQ (W4A4 g16; lm_head/linear_attn/visual kept BF16)", flush=True)
+    mtq.quantize(model, cfg, forward_loop=forward_loop)
+
+    print(f"exporting modelopt HF checkpoint -> {args.out}", flush=True)
+    # Force a SINGLE shard (default max_shard_size 10GB would split the ~14GB output into 3 shards,
+    # but splice_mtp.py expects a single <out>/model.safetensors to add the bf16 mtp.* into).
+    export_hf_checkpoint(model, export_dir=args.out, max_shard_size="1TB")
+    tok.save_pretrained(args.out)
+    print("DONE. Next: splice_mtp.py <out> <graft>  then serve "
+          "--quantization modelopt --speculative-config "
+          "'{\"method\":\"qwen3_5_mtp\",\"num_speculative_tokens\":3}' "
+          "--reasoning-parser qwen3 --tool-call-parser qwen3_coder --enable-auto-tool-choice", flush=True)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
@@ -0,0 +1,132 @@
+#!/usr/bin/env python3
+"""NVFP4 quantize the MTP-grafted Heretic2 model (llm-compressor / compressed-tensors).
+
+Runs AFTER graft_mtp.py. Fleet-proven compressed-tensors NVFP4 path (pantheon-27b-
+mtp-nvfp4 serves this on our Blackwell) with the ignore-list from robbatt's on-fleet
+deckard-nvfp4 recipe PLUS the MTP head: keep GDN/linear-attn, vision, lm-head, all
+norms, AND mtp.* in BF16; NVFP4 only the dense Linear layers. (R36 fast-seat spike.)
+
+API validated against llmcompressor 0.12.0: oneshot(model, dataset, recipe,
+num_calibration_samples, max_seq_length) — dataset is a pre-tokenized datasets.Dataset.
+
+Two calib modes:
+  --calib-mode text : AEON-baseline control (neuralmagic/calibration LLM split)
+  --calib-mode chat : production 512-row mix (JSONL {messages, tools}); each row
+                      rendered via apply_chat_template(enable_thinking=True) so the
+                      forward-pass sees the qwen3_coder tool-call XML the seat emits.
+
+Run in a vLLM container on the freed GPU0:
+  docker run --gpus '"device=0"' --ipc host -v /tank/aimodels:/tank/aimodels \
+    --entrypoint bash vllm/vllm-openai:v0.24.0 -c \
+    "pip install -q llmcompressor tiktoken sentencepiece && python3 quant_nvfp4.py ..."
+"""
+import argparse
+import json
+import sys
+
+# Ignore list = robbatt deckard-nvfp4 recipe + MTP head (brokkr: keep mtp.* BF16).
+IGNORE = [
+    "lm_head",
+    "re:.*embed_tokens$",
+    "re:visual.*",
+    "re:model.visual.*",
+    "re:.*linear_attn.*",
+    "re:.*norm.*",
+    "re:.*q_norm.*",
+    "re:.*k_norm.*",
+    "re:.*mtp.*",        # MTP head stays BF16 for qwen3_5_mtp spec-decode (match anywhere:
+                         # module path is model.mtp.*, so an anchored re:mtp.* misses it)
+]
+
+
+def _tok_rows_to_dataset(tok_rows):
+    from datasets import Dataset
+    return Dataset.from_list(tok_rows)
+
+
+def load_calib_text(name, tokenizer, n, seqlen):
+    from datasets import load_dataset
+    ds = load_dataset(name, split="train").shuffle(seed=42).select(range(min(n, 100000)))
+    col = "text" if "text" in ds.column_names else ds.column_names[0]
+    rows = [tokenizer(x[col], truncation=True, max_length=seqlen) for x in ds.select(range(n))]
+    return _tok_rows_to_dataset(rows)
+
+
+def load_calib_chat(path, tokenizer, seqlen, n):
+    rows = [json.loads(l) for l in open(path) if l.strip()][:n]
+    out = []
+    for r in rows:
+        # Tool_call arguments may be OpenAI wire-form JSON strings; the Qwen3.6 template
+        # does .items() on them → needs a dict. Parse string→dict (render_verify finding;
+        # belt-and-suspenders even though brokkr canonicalized the rows to dicts).
+        for m in r["messages"]:
+            for tc in (m.get("tool_calls") or []):
+                a = tc.get("function", {}).get("arguments")
+                if isinstance(a, str):
+                    tc["function"]["arguments"] = json.loads(a)
+        text = tokenizer.apply_chat_template(
+            r["messages"], tools=r.get("tools"),
+            tokenize=False, add_generation_prompt=False, enable_thinking=True,
+        )
+        out.append(tokenizer(text, truncation=True, max_length=seqlen))
+    return _tok_rows_to_dataset(out)
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--model", required=True)
+    ap.add_argument("--calib-mode", choices=["text", "chat"], required=True)
+    ap.add_argument("--calib", required=True)
+    ap.add_argument("--out", required=True)
+    ap.add_argument("--num-samples", type=int, default=512)
+    ap.add_argument("--seqlen", type=int, default=8192)
+    args = ap.parse_args()
+
+    from transformers import AutoModelForImageTextToText, AutoTokenizer
+    from llmcompressor import oneshot
+    from llmcompressor.modifiers.quantization import QuantizationModifier
+
+    print(f"loading grafted model: {args.model}", flush=True)
+    # Load as the FULL multimodal Qwen3_5ForConditionalGeneration (NOT AutoModelForCausalLM).
+    # AutoModelForCausalLM resolves qwen3_5 -> Qwen3_5ForCausalLM (text-only), whose weight
+    # keys are flat `model.layers.*` with no vision tower. But vLLM 0.24 only registers
+    # Qwen3_5ForConditionalGeneration, and its hf_to_vllm_mapper expects the checkpoint keyed
+    # `model.language_model.layers.*` (+ `model.visual.*`) — a bare `model.layers.` prefix has
+    # NO mapping rule, so every transformer-layer weight fails to load -> uninitialized weights
+    # -> degenerate `!!!!` output. AutoModelForImageTextToText resolves qwen3_5 ->
+    # Qwen3_5ForConditionalGeneration, so keys are born `model.language_model.*` / `model.visual.*`
+    # matching the working pantheon-27b-mtp-nvfp4 reference. The vision tower loads in BF16 and is
+    # ignored by the quant (re:.*visual.*); calibration is text-only (no pixel_values needed).
+    # (R36 fast-seat namespace fix, 2026-07-14 — the config-merge in the prior recipe was a doomed
+    # patch over a checkpoint quantized in the wrong namespace.)
+    model = AutoModelForImageTextToText.from_pretrained(
+        args.model, torch_dtype="auto", device_map="auto", trust_remote_code=True,
+    )
+    tok = AutoTokenizer.from_pretrained(args.model, trust_remote_code=True)
+
+    print(f"building calibration ({args.calib_mode}, up to {args.num_samples} @ seq {args.seqlen})", flush=True)
+    if args.calib_mode == "text":
+        calib = load_calib_text(args.calib, tok, args.num_samples, args.seqlen)
+    else:
+        calib = load_calib_chat(args.calib, tok, args.seqlen, args.num_samples)
+    print(f"  {len(calib)} calibration rows", flush=True)
+
+    recipe = QuantizationModifier(targets="Linear", scheme="NVFP4", ignore=IGNORE)
+
+    print("running NVFP4 oneshot (Linear-only; GDN/vision/lm-head/norms/MTP kept BF16)", flush=True)
+    oneshot(
+        model=model, dataset=calib, recipe=recipe,
+        num_calibration_samples=len(calib), max_seq_length=args.seqlen,
+    )
+
+    print(f"saving -> {args.out}", flush=True)
+    model.save_pretrained(args.out, save_compressed=True)
+    tok.save_pretrained(args.out)
+    print("DONE. serve: vllm --quantization compressed-tensors "
+          "--speculative-config '{\"method\":\"qwen3_5_mtp\",\"num_speculative_tokens\":3}' "
+          "--reasoning-parser qwen3 --tool-call-parser qwen3_coder --enable-auto-tool-choice", flush=True)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
@@ -0,0 +1,44 @@
+#!/usr/bin/env python3
+"""Render-verify a tool-call-XML calib row through the Qwen3.6 chat template.
+
+brokkr's last gate before the production calib enters the quant: confirm the
+assistant tool_calls serialize to the qwen3_coder XML the LIVE seat emits
+(`<tool_call>\n<function=NAME>\n<parameter=KEY>...`), so the calibration
+forward-pass sees the seat's REAL tool-call activations (#355-preservation).
+
+Usage: python3 render_verify.py <tokenizer_dir> <calib.jsonl> [row_index]
+"""
+import json
+import sys
+
+from transformers import AutoTokenizer
+
+tok_dir, calib = sys.argv[1], sys.argv[2]
+idx = int(sys.argv[3]) if len(sys.argv) > 3 else 0
+
+tok = AutoTokenizer.from_pretrained(tok_dir, trust_remote_code=True)
+with open(calib) as f:
+    row = json.loads([l for l in f if l.strip()][idx])
+
+# Tool_call arguments arrive in OpenAI wire form (a JSON *string*), but the Qwen3.6
+# chat template does .items() on them → needs a dict. Parse string→dict so the
+# template renders the qwen3_coder <parameter=...> XML. (render-verify finding.)
+for m in row["messages"]:
+    for tc in (m.get("tool_calls") or []):
+        a = tc.get("function", {}).get("arguments")
+        if isinstance(a, str):
+            tc["function"]["arguments"] = json.loads(a)
+
+tools = row.get("tools", [])
+print("=== tools:", [t.get("function", {}).get("name") for t in tools])
+print("=== message roles:", [m.get("role") for m in row["messages"]])
+
+text = tok.apply_chat_template(
+    row["messages"], tools=tools,
+    tokenize=False, add_generation_prompt=False, enable_thinking=True,
+)
+print(f"=== rendered ({len(text)} chars) ===")
+print(text)
+print("=== qwen3_coder XML markers present? ===")
+for m in ("<tool_call>", "<function=", "<parameter=", "</think>", "bifrost.soong-lab."):
+    print(f"  {m!r}: {m in text}")
@@ -0,0 +1,20 @@
+#!/bin/bash
+# Run the Heretic2 NVFP4 production quant on ana-ml2 GPU0 (freed for the window).
+# llmcompressor + tokenizer deps pip-installed into a vLLM container (torch/CUDA ready).
+set -euo pipefail
+WORK=/tank/aimodels/heretic2-nvfp4-work
+MODE="${1:-chat}"                       # chat = production 512-row mix; text = AEON baseline
+CALIB="${2:-$WORK/production_calib_512.jsonl}"
+OUT="${3:-$WORK/heretic2-mtp-nvfp4-prod}"
+
+docker run --rm --gpus '"device=0"' --ipc host \
+  -v /tank/aimodels:/tank/aimodels \
+  --entrypoint bash vllm/vllm-openai:v0.24.0 -c "
+    set -e
+    pip install -q llmcompressor tiktoken sentencepiece 2>&1 | tail -1
+    python3 $WORK/quant_nvfp4.py \
+      --model $WORK/heretic2-mtp-bf16 \
+      --calib-mode $MODE --calib $CALIB \
+      --num-samples 512 --seqlen 8192 \
+      --out $OUT
+  "
@@ -0,0 +1,20 @@
+#!/bin/bash
+# Launch the modelopt NVFP4 quant of the grafted Heretic2 on ana-ml2 GPU0 (detached, survives ssh
+# drop). bare nvidia-modelopt (0.45). The FusedMoE-compat guard + single-shard export + the
+# multimodal load class are all inside quant_modelopt.py. ~18 min. Output: heretic2-modelopt-nvfp4
+# (no mtp yet — run finalize_modelopt_mtp.py after). quant_modelopt.py must be at /lk/quant_modelopt.py
+# (mount /home/lkraven as /lk, or scp it there first).
+set -euo pipefail
+docker rm -f vllm-heretic2-modelopt-quant 2>/dev/null || true
+rm -rf /tank/aimodels/heretic2-nvfp4-work/heretic2-modelopt-nvfp4 2>/dev/null || true
+docker run -d --name vllm-heretic2-modelopt-quant --gpus '"device=0"' --ipc host \
+  -v /tank/aimodels:/tank/aimodels -v /home/lkraven:/lk \
+  --entrypoint bash vllm/vllm-openai:v0.24.0 -c '
+    set -e
+    pip install -q nvidia-modelopt tiktoken sentencepiece 2>&1 | tail -1
+    python3 /lk/quant_modelopt.py \
+      --model /tank/aimodels/heretic2-nvfp4-work/heretic2-mtp-bf16 \
+      --calib-mode chat --calib /tank/aimodels/heretic2-nvfp4-work/production_calib_512.jsonl \
+      --num-samples 512 --seqlen 8192 \
+      --out /tank/aimodels/heretic2-nvfp4-work/heretic2-modelopt-nvfp4'
+echo "LAUNCHED: $(docker ps --filter name=vllm-heretic2-modelopt-quant --format '{{.Status}}')"
@@ -0,0 +1,25 @@
+#!/bin/bash
+# Serve the modelopt NVFP4 + MTP Heretic2 seat (the WORKING fast char-rp-reasoning seat).
+# ~77 tok/s, MTP acceptance 32-40%. Requires: (1) heretic2-modelopt-nvfp4-mtp built (quant ->
+# finalize), (2) the sitecustomize MTP workaround mounted on PYTHONPATH (vLLM 0.24 draft-model
+# exclude bug — see runbook landmine #4; without it the engine crashes on a shape mismatch).
+set -euo pipefail
+MODEL="${1:-/tank/aimodels/heretic2-nvfp4-work/heretic2-modelopt-nvfp4-mtp}"
+# Dir containing sitecustomize.py (a copy of sitecustomize-mtp-workaround.py named sitecustomize.py):
+WORKAROUND_DIR="${MTP_WORKAROUND_DIR:-/home/lkraven/isls_debug}"
+docker rm -f vllm-charrp-modelopt 2>/dev/null || true
+docker run -d --name vllm-charrp-modelopt --gpus '"device=0"' --ipc host \
+  -v /tank/aimodels:/tank/aimodels \
+  -v "${WORKAROUND_DIR}":/lk_debug -e PYTHONPATH=/lk_debug \
+  -p 8018:8000 \
+  vllm/vllm-openai:v0.24.0 \
+  "$MODEL" \
+  --quantization modelopt \
+  --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":3}' \
+  --language-model-only \
+  --mamba-cache-dtype float32 \
+  --reasoning-parser qwen3 --tool-call-parser qwen3_coder --enable-auto-tool-choice \
+  --served-model-name char-rp-reasoning \
+  --max-model-len 40960 --max-num-seqs 32 --gpu-memory-utilization 0.5 --trust-remote-code
+echo "started: $(docker ps --filter name=vllm-charrp-modelopt --format '{{.Status}}')"
+echo "verify MTP: docker logs vllm-charrp-modelopt 2>&1 | grep -E 'mtp-workaround|SpecDecoding'"
@@ -0,0 +1,20 @@
+#!/bin/bash
+# Serve the NVFP4+MTP Heretic2 seat on ana-ml2 GPU0 for P00 validation.
+# vLLM compressed-tensors NVFP4 + qwen3_5_mtp spec-decode (pantheon pattern).
+# Served as char-rp-reasoning on :8018 (the GGUF seat's port; that seat is stopped).
+set -euo pipefail
+WORK=/tank/aimodels/heretic2-nvfp4-work
+docker rm -f vllm-charrp-nvfp4-test 2>/dev/null || true
+
+docker run -d --name vllm-charrp-nvfp4-test --gpus '"device=0"' --ipc host \
+  -v /tank/aimodels:/tank/aimodels -p 8018:8000 \
+  vllm/vllm-openai:v0.24.0 \
+  "$WORK/heretic2-mtp-nvfp4-prod" \
+  --quantization compressed-tensors \
+  --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":3}' \
+  --reasoning-parser qwen3 --tool-call-parser qwen3_coder --enable-auto-tool-choice \
+  --language-model-only \
+  --mamba-cache-dtype float32 \
+  --served-model-name char-rp-reasoning \
+  --max-model-len 40960 --max-num-seqs 256 --gpu-memory-utilization 0.45 --trust-remote-code
+echo "container started: $(docker ps --filter name=vllm-charrp-nvfp4-test --format '{{.Status}}')"
--- a/Show More
+++ b/Show More
				`@@ -0,0 +1 @@`
				- `[2026-07-13]` Relaying a peer's diagnosis as fact without confirming it against raw data. worldtree-dev diagnosed the WT #355 residual as "our llama.cpp seat wedging," which I echoed in a wrap-up; the operator challenged it and the seat logs DISPROVED it (seat completes ≤72s, idle at the wedge onset — the hang is the LiteLLM gateway). Lesson: CONFIRM peer diagnoses (esp. cross-domain ones) before acting/relaying — same discipline that caught the earlier char-rp-reasoning red-herring via a live `registry.resolve` reproduction.
				`@@ -0,0 +1 @@`
				- `[2026-07-14]` AEON's "working NVFP4+MTP RP seat" was pantheon on compressed-tensors (0% MTP accept), not a modelopt MTP proof. `vllm-aeon-rp`'s .env → `AEON_RP_MODEL=pantheon-27b-mtp-nvfp4`, `AEON_RP_QUANT=compressed-tensors` — it LOADED (mtp silently skipped, `exited 0`) but never accelerated. Same vLLM image (`:latest` = `sha256:4091d55` = 0.24.0) as the failed Heretic2 test, so the "AEON ran on an older vLLM" theory was wrong. Don't treat a seat that "ran" as MTP-validated without checking its `SpecDecoding` acceptance.
				`@@ -0,0 +1 @@`
				- `[2026-07-14]` gitea "test-delivery 204" is NOT proof a webhook works (204 = gitea queuing, not the listener receiving) — and a proxy test signing with the listener's OWN secret proves the listener, not gitea's real delivery. Both red herrings cost a round of the soong-lab webhook diagnosis. Diagnose from BOTH ends: sender (`docker logs gitea \| grep webhook` → the `deny '<ip>'` line) AND an instrumented receiver.
				`@@ -0,0 +1 @@`
				- `[2026-07-14]` *MTP graft via top-level `mtp.` tensor names does NOT survive `AutoModelForCausalLM.from_pretrained`** — the `Qwen3_5ForCausalLM` class doesn't expose an mtp module, so the mtp keys are DROPPED at load (quant output = 0 mtp). Fix = SPLICE the BF16 mtp tensors into the quant output post-hoc (how pantheon was built); don't rely on the graft surviving the model round-trip.
				`@@ -0,0 +1 @@`
				- `[2026-07-14]` MTP-on-modelopt: NO checkpoint config skips the spec-decode drafter's quant (vLLM 0.24 bug) — 4 config attempts failed before the runtime workaround. All crashed the same way (`qwen3_5_mtp.py:256` `param_data.shape == loaded_weight.shape` AssertionError — bf16 mtp head loaded into a quantized drafter param): (1) mtp excludes in `config.json` (WRONG file — vLLM modelopt reads `hf_quant_config.json`); (2) specific-unfused mtp names in hf_quant_config; (3) wildcards `mtp`/`mtp.layers.0` (`is_layer_skipped` is EXACT-membership, NOT glob — wildcards match nothing); (4) exact fused+unfused names in both `mtp.`/`model.` prefixes. Instrumenting `is_layer_skipped` proved the drafter's exclude list holds ONLY the main model's `linear_attn` entries — the mtp excludes never reach the draft-model quant config. ONLY fix = a mounted `sitecustomize` force-skipping `mtp.*`. LESSON: don't chase checkpoint-config fixes for the mtp-drafter crash; go straight to the runtime patch. Also `nvidia-modelopt[hf]==0.43` (AEON's producer version) is a trap — it pins transformers back to 4.57 which can't load `qwen3_5` at all; use 0.45 + the FusedMoE guard in `quant_modelopt.py`.
				`@@ -0,0 +1 @@`
				- `[2026-07-14]` NVFP4 (llm-compressor / compressed-tensors) gives NO batch-1 speedup over GGUF for the Qwen3.5 GDN-hybrid, and its MTP is 0%-accept. Measured base NVFP4 no-MTP ≈53 tok/s decode vs the GGUF NEO-CODE seat ~59.5 (llama.cpp wins single-stream; NVFP4's edge is concurrency, and this hybrid is bandwidth-bound at batch-1 with the BF16 linear_attn/GDN layers dominating). MTP spec-decode = 0% acceptance (vLLM's `Qwen3_5MTP` drafter won't load the bf16 mtp weights off a compressed-tensors main model → `Parameter … not found in params_dict`, `Avg Draft acceptance rate: 0.0%`). Pantheon is identical — its "working NVFP4+MTP" was working structure, never real acceleration. Working native MTP needs the modelopt main-model format (AEON, ~3.3/3 accept). LESSON: don't expect a faster single-stream seat from an llm-compressor NVFP4 quant of this arch; the MTP multiplier is the whole point and it requires modelopt.
				`@@ -0,0 +1 @@`
				- `[2026-07-14]` NVFP4 spike: built the full MTP serve scaffolding BEFORE validating a plain NVFP4 serve was coherent. Chased 6 sequential serve-config fixes (entrypoint doubled `serve`, arch `ForCausalLM`→`ConditionalGeneration`, `--language-model-only`, mamba-cache/`max-num-seqs`) across a 2.5hr GPU window (quoted 30-60 min) — only to find the served model gibbers (`!!!!`). LESSON: smoke a PLAIN `/v1/completions` coherence check on the SIMPLEST config (native arch, no MTP, no splice) FIRST — validate the tracer bullet before building spec-decode scaffolding. Also cost an unnecessary re-quant (the `re:mtp.*` ignore fix that turned out moot). Diagnostic ladder in Current state.
				`@@ -0,0 +1 @@`
				- `[2026-07-15]` arbo fully switched off image-judge (qwen-image-bench) -> gen; image-bench pending eviction post-bake. Operator-directed full switch (comfy-dev executed, live in prod). Established: gen (`qwen3.6-35b-a3b-heretic`) is vision-enabled and was image-bench's predecessor as arbo's hero-judge; image-judge actually serves 4 roles (vision quality-scoring + identity-scoring + bbox grounding + an uncensored text tier), not just grounding. comfy-dev spot-check: gen faster on every task, grounding within ~3px, uncensoring preserved, and it FIXED a bug (image-judge's reasoning preamble broke json_object + stalled the router). Sequencing = short prod bake then evict (~30 GB GPU1 reclaim); revert = flip `ARBO_VISION_MODEL`. Full record: auto-memory `project_arbo_gen_switch_imagebench_evict`.
				`@@ -0,0 +1 @@`
				- `[2026-07-15]` Claude Code statusline `.cost.total_cost_usd` is per-SESSION (Claude Code's own cache/model-aware session accounting), not a lifetime aggregate — the large value just reflects a long, multiple-times-summarized session. And the old statusline hardcoded Sonnet pricing ($3/$15) on an Opus session -> ~5x cost understatement.
				`@@ -0,0 +1 @@`
				- `[2026-07-15]` `docker.service After=remote-fs.target` does NOT wait for `nofail` NFS mounts — `nofail` drops a mount out of remote-fs.target's blocking set, so the drop-in ordering is silently defeated (paperless still Exited(255) on reboot). Real fix = DIRECT mount->docker ordering via the fstab `x-systemd.before=docker.service` option (verify `systemctl show docker -p After` lists the mnt-*.mount units). esh-docker-vm.