Commit Graph
1224 Commits
Author SHA1 Message Date
vh 0a9fb85a52 docs(zonos-gateway): dials-first emotion spec (operator ruling)
Canonical direction: emotion set by twisting raw dials per-utterance, not preset
selection. Presets demoted to optional examples. Spec covers the dial vocabulary
+ ranges, the emotion_cfg_scale 'deaf by 1.5' rule (NO cap — documented ceiling,
explicit over implicit), measured RTF cost, starting-point dial-sets, an LLM
client system-prompt snippet, and clone reference guidance (~15-24s, no
transcript). Follow-ups flagged: align dials.py cfg help/metadata, trim in-code
PRESETS pending usage check, stand up vh/zonos-gateway for version control.
2026-07-17 18:33:26 -07:00
vh ba0ec64ac3 docs(zonos-gateway): capture the live :8890 gateway stack + emotion-preset system
Captures the production Zonos TTS path that the repo was blind to: stock ZONOS2
:1920 engine (stacks/zonos-engine) fronted by zonos-gateway:0.2.0 :8890, reached
via the LiteLLM ext-tts alias. Documents the emotion-preset system (neutral/warm/
excited/sad/intense/whisper, the simple preset: caller path), the API, and the
measured real-time cost (calibrated steering free at RTF~0.52, cfg1.5 ~0.625 —
still realtime). Flags the gateway source (~/zonos-gateway on irv-ml1) as not yet
in gitea. Marks stacks/zonos (v0.1 Gradio) dead/superseded. Records the emotion-
lever finding (text-priming flat -> native steering works) in persistent memory.
2026-07-17 15:56:11 -07:00
vh 196a0c1e4c memory: #365 CLOSED — personal live on b125, byte-exact, both instances confirmed
worldtree-dev ruled the four other personal deltas code-default-optional (none
boot-required), cut staging/v1.0.0b125 (workflow 1859 green). Personal now on
b125 (877bb577ad36), healthy, booted off the staged bind-mount, live #365 blocks
byte-exact vs baked, clean boot. Demo + personal both confirmed. .bak-pre-365
rollback on both.
2026-07-17 13:38:47 -07:00
vh bb7d38d6dc memory: #365 demo confirmed on b125 (byte-exact) + personal staged; b125 tag held on config-scope ruling
Demo auto-deployed to b125 and passed the boot-FATAL validator live against
the staged bind-mount; staged demo blocks byte-exact vs the baked image.
Personal staged (operator-directed) with the same verbatim blocks + backups,
all b125 checks pass, no restart. worldtree-dev holds staging/v1.0.0b125 until
they rule whether personal's other missing b107-b125 blocks (metrics/kb_audit/
reference_knowledge + affect-debug-observability-allow) are boot-required.
2026-07-17 13:34:46 -07:00
vh dbf511851c memory: Worldtree #365 demo config pre-sync STAGED (internal-comms tiers/rules + gate)
Staged the boot-blocking #365 delta on corviduo-dev DEMO /opt/worldtree/config
per worldtree-dev thread 01KXRE59 (verbatim from their 6a60fe3/b123 blocks):
internal + internal-restricted tiers, four internal-* rules, and the
internal_agent_comms:{enabled:false} gate. Validated via the demo container's
own PyYAML + the exact-set uniqueness guard; no restart (b123 activates off the
bind-mount). Backups .bak-pre-365. Flagged a pre-existing free==pro scope-set
dup to worldtree-dev as a b123 boot-block risk.
2026-07-17 13:10:14 -07:00
vh 069b3c2020 memory: capture Worldtree #365 boot-blocking demo config pre-sync (unblocked, deferred)
worldtree-dev flagged a boot-blocking config delta (commit 6a60fe3/b123, #365
internal-badge layer) needing a pre-sync of the demo policies.yaml + defaults.yaml
before its eventual deploy. Verbatim blocks received (althing 01KXRE59); the two
new scope-sets pre-verified unique vs the demo's 7 existing tiers. No urgency (push
held for heid gates) + FATAL-on-malformed, so deferred to a focused pass and captured
as an open loop (persistent-memory + /tmp handoff) rather than staged at the snapshot tail.
2026-07-17 09:29:42 -07:00
vh 2941158c70 memory: snapshot — Zonos2 containerize + prosody-priming (for /clear to build fresh)
Current state rewritten to the active build task: containerize the production
Zonos2 :1920 TTS engine as a self-contained image (config captured in 14a0004)
and prototype the operator's prosody-priming hypothesis (prime→generate-one-
utterance→parakeet-clip→deliver, adapter-level, engine stays stock). New detail
file + Recent-decisions pointer capture the full plan + the crux (does AR prosody
carry the sentence boundary — A/B the join). /tmp handoff written for the fresh session.
2026-07-17 09:16:43 -07:00
vh 14a0004a47 docs(zonos-engine): capture the production ZONOS2 :1920 launch config (was live-only)
The Zonos TTS engine that zonos-gateway fronts (irv-ml1 3090, feeds asset-engine +
gateway-chat) ran as a bare native process with its real invocation existing ONLY
in the running process argv — the committed harness/zonos_server.sh on irv-ml1 was
STALE (said A6000/:1919, no perf flags; live is 3090/:1920 with cuda-graph/num-pages/
max-running-requests/memory-ratio). Captured the corrected canonical invocation +
tunables + the containerization plan here so the config survives a process death.

Engine = stock Zyphra/Zonos2 @ 194c0a3 (no custom PFI server code); torch 2.9.1+cu128;
15 GB HF weights. Next: containerize in-place on the 3090 (operator: keep off the
A6000, it OOMs under ComfyUI). Not yet built — this commit is the config capture only.
2026-07-17 09:03:05 -07:00
vh 9e69639482 fix(vllm): pin granite --max-num-seqs=1024 (was implicit default 128)
granite (fleet fan-out summarizer/classifier) had no explicit --max-num-seqs,
so vLLM V1 resolved it to 128 — which caps concurrency BELOW granite's own KV
bound (~192 concurrent @ 1K-token calls, more for shorter classify calls).
Pinned it very high (1024) so the KV pool is the only bound; VRAM-neutral
(the KV pool is util-bound, unchanged). Added the flag to the granite command
+ GRANITE_MAX_NUM_SEQS to the env template. Live applied + verified
(resolved max_num_seqs=1024, seat healthy).
2026-07-16 10:47:15 -07:00
vh a2b026d499 feat(gpu): relocate char-rp to GPU1 + re-optimize both ana-ml2 cards for max context
Operator-directed 2026-07-16. Moved the char-rp prose seat (Magidonia-24B,
llama-charrp) from GPU0 to GPU1 (CHARRP_GPU_ID 0->1; recreate llama-charrp
only -- the var is shared with the retired GGUF reasoning service), then
re-optimized every context-relevant seat on both cards to native/max context
with acceptable headroom:

  GPU0 (both seats now 256K native, ~14 GB reserve):
    - char-rp-reasoning 150K -> 256K  (heretic2 stack, util 0.38->0.46, 1.56x)
    - gen 256K, max-num-seqs 16 -> 32 (qwen36-27b-aeon, util 0.30->0.42, 5.43x)
  GPU1 (~6.7 GB headroom):
    - granite 64K -> 128K full-chapter (vllm stack, util 0.18->0.27, 1.50x)
    - char-rp 128K native (4 slots), selene/reward/embed/rerank unchanged

All seats gateway-verified healthy. Live .env changes on ana-ml2 with per-stack
backups (*-20260716). Templates updated to match; the qwen36-27b-aeon template
carries a NOTE that its served-name/model still lag the 2026-07-08 gen model swap
(35B-A3B-heretic) -- separate reconciliation. persistent-memory records the full
layout + the util-floor / per-model-KV-cost lessons.

Note: the heretic2-charrp-reasoning stack (char-rp-reasoning's live config) is
still untracked in git -- standing open-loop, its .env change lives server-side only.
2026-07-16 09:28:07 -07:00
vh f25f494f07 fix(vllm): right-size granite util 0.34->0.18 + max-len ->65536 (free ~10.5GB GPU1)
Operator-directed 2026-07-16: free ~10GB on ana-ml2 GPU1 to relocate a GPU0
model onto GPU1. granite-4.1-8b (fleet summarizer) was over-provisioned at
util 0.34 / max-model-len 131072 with a flat 0.0% KV usage.

Set GRANITE_GPU_MEM_UTIL 0.34 -> 0.18 and GRANITE_MAX_MODEL_LEN 131072 -> 65536
on the live /opt/docker/compose/vllm/.env (backup .env.bak-pre-granite-rightsize-
20260716), recreated vllm-granite ONLY (shared stack). Result: GPU1 62,641 ->
51,897 MiB used (~10.5GB freed, ~45GB free now); KV 6.45 GiB / 84,528 tok /
1.29x concurrency @ 65536; summarizer verified healthy.

The util drop required the max-len drop: on this shared card the effective KV
slope is ~950 MiB per 0.01 util, and vLLM refuses to start unless the KV pool
holds >= 1x max-model-len -- util 0.15 undershot (est max-len 47184 < 65536,
crash-loop, ~2-3 min summarizer outage) before 0.18 landed. 65536 is granite's
precedented summarizer ctx; a summarizer doesn't need 131072.

.env.example updated to the new util (max-len was already 65536 in the template;
live had drifted to 131072). persistent-memory.md updated (parked item closed).
2026-07-16 08:51:05 -07:00
vh 925947c71e fix(litellm): retire image-bench backend, repoint image-judge + qwen-image-bench aliases to gen
Operator-directed 2026-07-15. The dedicated Qwen-Image-Bench NVFP4 judge
backend on ana-ml2 GPU1 (:8014) was stopped to reclaim ~32GB after the
arbo -> gen hero-judge switch. Both LiteLLM gateway aliases that pointed at
it -- image-judge and qwen-image-bench -- now repoint to the gen backend
(:8015, qwen3.6-35b-a3b-heretic, vision-intact), held at deterministic
judge sampling (temp 0 / top_k 1 / rep_pen 1.05) with enable_thinking:false
(a reasoning preamble breaks json_object). Verified live: both answer with
:8014 down, so they are definitively on gen.

Incidental: backfilled the canonical char-rp-reasoning litellm block, which
had lagged live since the 2026-07-14 NVFP4+MTP seat repoint (model
deckard-pkd-27b -> char-rp-reasoning, top_k 40 -> 20, min_p dropped,
enable_thinking:true added). Required so pushing the canonical would not
clobber the correct live block.

Live changes applied out-of-band (config push + litellm restart + stack
stop on ana-ml2); live config backup at
config.yaml.bak-pre-imagejudge-20260715. Revert path documented in the
config comment. persistent-memory.md updated (parked item closed).
2026-07-15 23:07:13 -07:00
vh d710e56aca memory: migrate persistent-memory.md to two-tier index (53 detail files)
Split the 53 over-threshold dated log entries (Recent decisions, Tried
and abandoned) into per-entry persistent-memory.d/<slug>.md detail files,
leaving one-line pointers in the index; the 10 short entries stay inline.
Startup index drops 60,527 -> 25,256 bytes (492 -> 270 lines); entry
bodies move verbatim to on-demand detail files, so a fresh session loads
~25 KB instead of ~60 KB and pulls a detail file only when its pointer is
relevant. Top matter (Repo purpose, Tools & conventions, Current state)
is unchanged; both archival back-references preserved.

CLAUDE.md persistent-memory section now documents the index<->detail read
discipline (read the index, pull details on demand, never bulk-read the
dir, commit both together).

Auto-archival still held every dated entry back (all <30 days old); the
July burst begins aging past the 30-day guard ~2026-07-31.
2026-07-15 13:33:42 -07:00
vh 05a4f54a2a memory: snapshot — 2026-07-15 fleet-ops session
Rewrote Current state / in-flight to reflect the session's landed work +
parked items (in-flight compressed from the now-done NVFP4/#355/deploy-speed
history). Added Recent decisions (homepage AI-tab revamp 569e1af; esh-docker-vm
reboot + NFS fstab fix 21d9a07; HA config repo; char-rp-reasoning OOM rescue;
arbo->gen switch with image-bench eviction parked post-bake; soong-lab library
persistence; statusline overhaul) and Tried-and-abandoned lessons (nofail
defeats After=remote-fs.target; D-state wedge needs a reboot; max-model-len
doesn't free vLLM VRAM; statusline cost is per-session). Nothing archivable
(all dated entries <30 days).
2026-07-15 11:04:13 -07:00
vh 21d9a07bc3 fix(esh-nfs): order docker after the NFS mount units directly
The prior fix (_netdev,nofail + docker.service After=remote-fs.target) looked
correct but silently failed — paperless still Exited(255) on the 2026-07-14
reboot. Root cause: `nofail` drops a mount out of remote-fs.target's blocking
set, so ordering docker After=remote-fs.target does NOT wait for the nofail
NFS mounts. Fix: add x-systemd.before=docker.service,x-systemd.mount-timeout=30
to the 4 NFS fstab lines (direct mount->docker ordering, nofail-safe). Applied
+ verified live (systemctl show docker -p After now lists all 4 mnt-*.mount).
Playbook + verify updated to canonicalize.
2026-07-14 21:53:33 -07:00
vh 569e1af9ca feat(homepage): split AI fleet into role-based groups on a dedicated AI tab
Move the ~22-service flat "AI Systems" group off the Main tab into a new
four-tab layout (Main / AI / Infrastructure / Toolchain). The AI tab sorts
the inference fleet by function into seven groups:

  AI - Inference        gen, char-rp, char-rp-reasoning, Granite summarizer
  AI - Eval & Retrieval Selene, Skywork Reward, Qwen3 rerank/embed, image-bench
  AI - Gateways & Chat  LiteLLM, Asset Engine, Gateway Chat, Open WebUI, ...
  AI - Speech (TTS)     Chatterbox Fast, Kokoro, mOrpheus
  AI - Audio Tools      Parakeet ASR, YT Voice Clipper
  AI - Image & Media    ComfyUI, Arbo
  AI - Dormant          stopped rollback seats + retired auditions

Relabel each stack's homepage.group so canonical stacks/ matches the live
containers on ana-ml2, ana-docker, and irv-ml1. Dormant stacks were refreshed
with `docker compose up --no-start` so they carry the new label while staying
stopped (compose-start rollback preserved). settings.yaml drives tab/order/
columns; services.yaml and README updated to the new scheme.
2026-07-14 20:05:50 -07:00
vh 982c319d9f feat(heretic2-nvfp4): WORKING modelopt NVFP4+MTP seat + full recipe runbook
The fast char-rp-reasoning seat works: ~77 tok/s (vs GGUF ~59.5, base NVFP4 ~53),
MTP draft-acceptance 32-40%, mean acceptance length 2.19. Same Heretic2/NEO-CODE
model, NVFP4 + native qwen3_5_mtp spec-decode.

Full end-to-end recipe + the four landmines in docs/runbooks/heretic2-nvfp4-mtp-seat.md:
(1) load as AutoModelForImageTextToText not AutoModelForCausalLM (namespace/gibberish);
(2) modelopt format not compressed-tensors (compressed-tensors MTP = 0% accept);
(3) modelopt 0.45 <-> transformers 5.12.1 FusedMoE crash (guarded in quant_modelopt.py);
(4) vLLM 0.24.0 does NOT propagate modelopt exclude_modules to the spec-decode draft
model -> BF16 mtp head gets quantized -> shape crash; no checkpoint config fixes it
(is_layer_skipped is exact-membership not glob) -> fix is a mounted sitecustomize that
force-skips mtp.* in is_layer_skipped (upstream vLLM bug to report).

Scripts: quant_modelopt.py (FusedMoE guard + single-shard export + multimodal load),
finalize_modelopt_mtp.py (splice bf16 mtp), serve_modelopt_mtp.sh, run_quant_modelopt.sh,
sitecustomize-mtp-workaround.py.
2026-07-14 14:41:48 -07:00
vh aca45393c2 fix(heretic2-nvfp4): quant as ConditionalGeneration (namespace fix) + modelopt recipe for working MTP
Root-caused the NVFP4 gibberish to a quant-namespace bug: quant_nvfp4.py loaded
via AutoModelForCausalLM -> text-only Qwen3_5ForCausalLM -> flat model.layers.* keys,
but vLLM 0.24 serves only Qwen3_5ForConditionalGeneration (whose weight mapper needs
model.language_model.*). Fixed by loading as AutoModelForImageTextToText; NVFP4 now
serves coherent (validated greedy on ana-ml2 GPU0).

Base NVFP4 (compressed-tensors) measured ~53 tok/s (~= GGUF at batch-1, no single-stream
win) and its MTP is 0% acceptance (vLLM's Qwen3_5MTP drafter loads the bf16 mtp head only
off a modelopt main-model checkpoint). Added quant_modelopt.py (nvidia-modelopt PTQ,
matches AEON's NVFP4 W4A4 g16 + lm_head/linear_attn/visual exclusions) as the path to
working native MTP; graft + splice + serve otherwise unchanged.
2026-07-14 13:10:25 -07:00
vh b972bef10e snapshot: NVFP4+MTP fast-seat quant recipe + failure state (gibberish, unisolated)
Captures the full pipeline recipe (graft->quant->splice->config->serve) with every
gotcha found this session, the 3 gibberish suspects, and the diagnostic ladder
(validate native-config no-MTP coherence FIRST) for a fresh session to finish the
chase. Also stages the NVFP4 scripts + 512-row calib. Recent decisions: NEO-CODE
seat swap (R36), webhook ALLOWED_HOST_LIST fix. Lessons: validate-tracer-bullet-first,
mtp-graft-dropped-at-load, gitea-204-red-herring.
2026-07-14 11:31:57 -07:00
vh 462d528bef fix(soong-lab-ci): webhook auto-deploy real root cause = gitea ALLOWED_HOST_LIST + add listener logging
The ufw fix (prior commit) was necessary but insufficient. The DECISIVE blocker
was gitea webhook.ALLOWED_HOST_LIST = 'external, 10.100.0.0/16' (NH3 only) —
corviduo-dev is 10.250.50.152 (Anaheim), so gitea refused to deliver ('deny
10.250.50.152') and never opened the TCP connection. Fixed to 'external,
10.0.0.0/8' (whole fleet, matches the ufw choice) + gitea restart.

Listener now logs every delivery (source-IP/hmac_ok/ref/action) — the old
log_message=pass silence hid the whole failure. Proven end-to-end: real gitea
delivery -> hmac_ok=True, ref=main, 202 deploying -> green deploy.
2026-07-14 09:19:38 -07:00
vh 4fc0c27485 fix(heretic2-nvfp4): parse tool_call arguments string->dict for Qwen3.6 template
render-verify caught it: Dvalin's calib tool_calls carry OpenAI wire-form JSON
string arguments, but the Qwen3.6 chat template does .items() on arguments (needs
a dict) → jinja TypeError. Parse string->dict in render_verify + the quant's
load_calib_chat. Confirmed: renders the exact qwen3_coder XML the seat emits
(prefixed bifrost.soong-lab.*, v0.3.13 generate_portrait, <think>, <tool_response>).
2026-07-14 09:04:19 -07:00
vh 920f9a3709 feat(heretic2-nvfp4): MTP-graft + NVFP4 quant scripts + pipeline README (fire-ready)
graft_mtp.py: grafts the 15 base-Qwen3.6 MTP tensors into Heretic2 BF16 (CPU-only).
quant_nvfp4.py: llm-compressor NVFP4 (Linear only; GDN/vision/lm-head/norms/MTP
kept BF16 per robbatt's deckard recipe + brokkr's spec); text (AEON-baseline) or
chat (production, apply_chat_template renders qwen3_coder XML) calib modes.
README: fire sequence + gates (GPU window, production calib) + artifacts.

Spike gated only on: (1) off-peak Blackwell GPU window, (2) brokkr's production calib.
2026-07-14 08:57:25 -07:00
vh bbbfe5502e feat(heretic2-nvfp4): stage soong-lab v0.3.13 live 9-tool schema for the NVFP4 calib tool-call-XML slice
Extracted from the deployed backend (bifrost/tools.py) via the venv with a
capturing mock register_tool — the LIVE schema, not a stale copy. OpenAI-function
form for brokkr/Dvalin's ~128-row tool-call-XML calib slice (R36 #355 anchor).
2026-07-14 08:46:49 -07:00
vh b195815586 docs(soong-lab-ci): correct webhook runbook — root cause was ufw firewall (not SSRF)
The auto-deploy silently never worked: corviduo-dev's ufw is default-deny and
port 9010 was never allowed, so gitea's webhook deliveries timed out (DROP).
v0.3.6 was a manual deploy; v0.3.7-v0.3.13 never auto-deployed. The setup-time
'test-delivery 204' was gitea queuing, not the listener receiving. Fixed by
'ufw allow from 10.0.0.0/8' (operator-directed). Confirmed end-to-end.
2026-07-14 08:29:28 -07:00
vh f960a73a79 feat(char-rp-gguf): swap reasoning seat Deckard-PKD → NEO-CODE (Heretic2-Thinking Qwen3.6-27B)
R36 gate (2026-07-14) validated NEO-CODE ships on all axes: tool-calling 0.967
(attach_tool 1.00, 0 runaways — #355 eliminated), prose genre-artifact-fine
(less clichéd than gen), refusal uncensored-as-spec + CSAM-clean.

#355 root cause was MODEL-level, not the reasoning-budget-forcing bug: Deckard
emitted Qwen's native qwen3_coder XML tool format malformed -> llama.cpp leaked
the closing tags into the arg value -> Bifrost attach_tool schema error -> retry
-> reasoning runaway to max_tokens. NEO-CODE emits the same native format cleanly
on the same seat/parser -> no schema error -> no runaway. The fix was the model
swap; there was never a wrong parser (the XML is Qwen3.5/3.6-native).

- reasoning seat: Deckard-PKD (Qwen3.5) -> NEO-CODE=Heretic2-Thinking (Qwen3.6-27B) Q5
- samplers: card defaults (temp 1.0 / top_p 0.95 / top_k 20 / min_p 0.0), DRY dropped
- ctx: 256K max; custom llama.cpp kept (qwen3_coder parse + PR#25544 belt-and-suspenders)
- persistent-memory ACTIVE 1 marked resolved
2026-07-13 22:21:10 -07:00
vh e0f1dbfae6 fix(soong-lab-ci): sync web/ frontend on deploy (was serving stale web)
soong-dev found the studio serving a stale web/ (52015 vs 55025 bytes — missing the
01-Role section, favicon, thinking-status): the deploy rsynced backend/ but never web/,
so SOONG_LAB_WEB_DIR stayed pinned to the initial manual copy while the backend updated.
Deploy now rsyncs BOTH backend/->studio AND web/->SOONG_LAB_WEB_DIR (read from the env)
on every green run. Verified: served frontend now 55025 bytes, current.
2026-07-13 15:09:09 -07:00
vh cc0e3af87d feat(soong-lab-ci): red-run althing relay (nh3-dev poll -> ping soong-dev)
Per operator call (no gitea write token on the Worldtree-team VM): a 2-min systemd
--user timer on nh3-dev polls corviduo's last-deploy.json and pings soong-dev via
althing on a NEW red deploy (green stays silent). Delivers soong-dev's red-run
visibility without a credential on corviduo. Tested (red detect+format DRY, green quiet).
2026-07-13 14:28:39 -07:00
vh fb556586e3 feat(soong-lab-ci): green-gated push-to-deploy CI/CD for the soong-lab studio
Vuong-directed. gitea webhook (push→main) → HMAC listener on corviduo-dev:9010 →
clone (read-only deploy key) → uv sync + pytest → redeploy soong-lab-studio.service
ONLY on green (running studio untouched on red). Validated end-to-end 2026-07-13.
Canonical copies of the deploy script + listener + unit; runbook in docs/runbooks.
2026-07-13 14:16:26 -07:00
vh 85792f4b55 feat(char-rp-gguf): swap reasoning seat to custom llama.cpp (master 6eddde0 + PR #25544)
The char-rp-reasoning (Deckard) seat now runs llamacpp-charrp:custom-latest via a
new LLAMA_REASONING_IMAGE var (Magidonia char-rp stays on stock — no reasoning bug).
Fixes Worldtree #355 at the source (budget multi-terminator handles Qwen3.5's
<tool_call> reasoning end-tag). Live 2026-07-13: Deckard loads on Blackwell, serves
coherent, reasoning bounds at the 400 budget. Rollback via .env LLAMA_REASONING_IMAGE.
2026-07-13 13:47:42 -07:00
vh 6cf3e78973 docs(char-rp-gguf): record custom llama.cpp build (master 6eddde0 + unmerged PR #25544)
Durable record of the custom llama.cpp the char-rp-reasoning seat will run to
fix Worldtree #355 (reasoning-budget forcing broken in stock b8840 — single
end-tag </think> can't match Qwen3.5's <tool_call> reasoning terminator, so the
budget never force-closes and reasoning runs away to max_tokens). PR #25544
adds multiple terminating sequences; unmerged upstream, so we build it.

- build.sh: reproducible recipe (clone master@6eddde0 + merge PR #25544 +
  resolve the 1 server-common.cpp conflict + CUDA build for Blackwell sm_120)
- README.md: why + acceptance test + rollback + REMOVE-WHEN-MERGED tracking

Image llamacpp-charrp:6eddde0-pr25544 BUILT + smoke-tested on ana-ml2; seat
swap pending. See also auto-memory reference_charrp_custom_llamacpp_pr25544.
2026-07-13 13:37:00 -07:00
vh 26b30d8231 memory: archive 17 spent decisions (2026-06-14..2026-07-07) to archival-memory.md; cumulative 142 2026-07-13 10:54:17 -07:00
vh 5df4edc5dc memory: snapshot — LiteLLM #355-residual investigation + BuildKit deploy-speed cache (both in-flight); #355 fix validated, Ledger tier-3 provisioned, assistant/thoughtful-assistant roles added, 3 zombie sessions retired 2026-07-13 10:47:26 -07:00
vh b95802efa4 docs(backup): hourly off-box ~/development backup to nh3-nas (runbook + script)
Adds the rsync --link-dest hourly snapshot job (nh3-dev:~/development ->
nh3-nas, 48-snapshot retention, secrets/build-dirs excluded) that closes the
no-off-box-backup gap exposed by the 2026-07-12 working-dir clobber. Script
mirrors the live ~/.config/dev-backup/dev-backup.sh; runbook covers restore.
2026-07-12 01:40:43 -07:00
vh 8d055a78b6 memory: snapshot — R17 audition concluded → Zonos productionized (zonos-gateway shipped + online in asset-engine); omnivoice retired, vllm-morpheus killed, ComfyUI down/parked 2026-07-11 09:36:40 -07:00
vh 6a90a70ad8 feat(gateway-chat): repoint TTS to zonos-gateway (OpenAI /v1/audio/speech)
Point the in-page TTS at the zonos-gateway wrapper on irv-ml1:8890 (direct,
so streaming isn't buffered by LiteLLM): OpenAI-shape body (model: ext-tts,
input, voice), Cora default voice, float32@44.1kHz PCM decode. Quotes are
joined into a single stream call (prosody — no per-sentence chunking).
Migration regex rewrites stale saved endpoints (:8299/:8210, /tts[/stream],
:4000) to the new one.
2026-07-11 09:24:35 -07:00
vh 6d384dd361 catalog(asset-engine): add zonos-gateway service (full expressive dials)
New TTS service entry + reproducibility_audit row for the zonos-gateway
wrapper (irv-ml1:8890) — the ext-tts-aliased OpenAI facade over Zonos.
23 fields across Text&voice / Expression / Prosody / Quality / Sampling /
Output section groups; live voice dropdown from /v1/voices; response
format pcm|wav (audition UI forces wav). Distinct from the older down
zonos :8203 entry. jsonschema-validated.
2026-07-11 01:02:51 -07:00
vh 80f839a0f4 memory: snapshot — TTS audition (delegated) + ComfyUI v0.27.1 bump (comfy-dev GO) + scan cron settled 2026-07-10 07:34:28 -07:00
vh 8894854127 docs(sampler-defaults): fix char-rp-reasoning seat → Deckard-PKD (was stale QwQ-RpR-v4)
The live gateway config has served char-rp-reasoning as deckard-pkd-27b (:8018)
since the 2026-07-08 A/B; the standalone doc had frozen on QwQ-RpR-v4. Corrects
seat 4 (backend + samplers + server-side DRY/reasoning-budget notes).

Also snapshots session state in persistent-memory.md: phantom-qwen verified
already-clean, ana-docker docker log-cap (logrotate copytruncate, no bounce),
and the granite→gen memory_extractor bind live on demo+personal.
2026-07-09 14:09:13 -07:00
vh 6081319743 memory: snapshot — mOrpheus TTS voice pipeline shipped (irv-ml1 stack + gateway-chat voicing)
Captured the session's mOrpheus arc: permanent 2-container stack (vLLM bf16 v0.23.0 on the
3090 + CPU SNAC/FastAPI wrapper), gateway-chat auto-voicing of quoted dialogue (streaming,
per-quote chunking, clone voices baddy/beatrice/whisper), and the load-bearing lessons
(FP8 breaks audio, latest-vLLM crashes Ampere, per-frame decode too slow, sentence-chunking
kills prosody, HF whisper datasets not whispered → kokoro af_nicole). Plus the granite→gen
memory_extractor bind green-lit for worldtree-dev (#335 Slice 4).
2026-07-09 10:48:30 -07:00
vh a5735147d4 docs(morpheus): agent system prompt for quoted-text voicing + tag discipline
Drop-in system prompt for an agent whose quoted output is voiced by mOrpheus: speak in
double quotes (only quoted text is voiced), phrase each quoted line as one coherent
utterance (per-quote prosody), and the sparse/boundary/no-stack tag rules (measured
stability on the early checkpoint). Lists the honored tag set.
2026-07-09 02:02:40 -07:00
vh f295cc1f46 fix(gateway-chat): chunk by quoted section, not sentence (prosody)
Per-sentence chunking generated each sentence cold, flattening intonation/prosody that
spans the whole quoted line. Chunk by QUOTED SECTION instead — each contiguous quote is
generated whole (max_tokens 2400) so its prosody stays intact; multiple quotes in a reply
still play serially on the shared clock. extractQuotes already returns exactly these spans;
dropped splitSentences.
2026-07-09 01:54:39 -07:00
vh a1f3023f70 feat(gateway-chat): pre-chunk quoted text by sentence, play serially
Split the quoted dialogue into sentences and stream each as its own short /tts/stream
request (max_tokens 900), queued back-to-back on one shared AudioContext clock (speechHead)
so playback is gapless and in order. First sentence starts fast; each chunk is short so it
generates cleanly (no ramble/cap risk); the next sentence generates while the current plays.
A newer reply supersedes via the ttsGen counter; 🔊 replays.
2026-07-09 01:49:45 -07:00
vh 033f3685f5 fix(gateway-chat): resume AudioContext on user gesture (no-sound / autoplay)
Browsers suspend the Web Audio AudioContext until a user gesture; speakQuotes fires on
reply-complete (no active gesture), so a suspended context played silently. Prime/resume
the context on any click or keydown (capture phase) so it's running before playback.
Server side was fine throughout (/tts + /tts/stream both 200 with valid audio).
2026-07-09 01:41:26 -07:00
vh 0655a37bf6 feat(morpheus): staged clone voices + max_tokens 3500 (context-clamped)
- max_tokens default 2400->3500 (~42s) in wrapper + gateway-chat client, with a _cap()
  clamp so prompt+gen never exceeds MAX_CTX (4096) — a cloning ref block is ~1100 tokens,
  so an unclamped 3500 would overflow context on the clone path.
- Staged clone voices: /voices dir of <name>.wav + <name>.txt, each encoded to its Orpheus
  reference block at startup; voice="<name>" zero-shot clones it. Beatrice (a chatterbox
  reference) staged as the first normal-voice clone. GET /voices lists baddy + clones.
- compose: mount voices dir + pass MORPHEUS_MAX_LEN to the wrapper (clamp must match engine).

vLLM concurrency (measured, --max-num-seqs 8, 250-tok reqs): near-linear batching — 8
concurrent finish in the same ~2.8s as 1 (707 tok/s, 8.1x single, flat per-req latency).
Chunked-sentence production can fan out for ~8x throughput; CPU SNAC decode is the scale
bottleneck, not generation.
2026-07-09 01:35:14 -07:00
vh f363fe6c84 fix(morpheus): raise TTS max_tokens 1200->2400 (long lines clipped at ~14.6s)
Cut-offs were the max_tokens=1200 ceiling (~14.6s of audio), not memory (~1250 tokens
<< 4096 context). Diagnosis: the repetition penalty is load-bearing for clean stops —
rep 1.0 => the model never emits end-of-speech and rambles to the cap; rep 1.1 (the
wrapper default) => clean natural stop. So normal lines already complete; only genuinely
long dialogue (>~14.6s, ~25+ words) hit the cap. Raised default + client max_tokens to
2400 (~29s), still within the 4096 context (no memory cost). Verified: a 49-word line
now finishes at 16.73s (was clipped at 14.6s).
2026-07-09 01:25:56 -07:00
vh da7682969b feat(morpheus,gateway-chat): streaming decode — TTFA ~4.5s -> ~0.8s
Wrapper gains POST /tts/stream: reads the vLLM token stream, decodes SNAC in WINDOWED
CHUNKS (every 6 frames, decode [2 ctx | 6 | 2 ctx] and emit only the middle 6 — context
both sides => seamless), and streams raw PCM16 (24kHz mono) as it generates. Windowed
(not per-frame) because per-frame CPU decode's per-call overhead x ~60 frames serialized
to ~7s (RTF 2.2); windowed keeps up (RTF ~0.97). Whole-clip /tts kept for non-browser use.

gateway-chat plays the stream via the Web Audio API (fetch reader -> int16->float32 ->
scheduled AudioBufferSourceNodes on a running clock; a new reply supersedes the prior
stream via a generation counter; 🔊 replays). Measured: TTFA 0.80s (was ~4.5s whole-clip),
RTF 0.97, full-duration match. CORS already covers the new route.

Deployed: tts rebuilt on irv-ml1, page pushed to ana-docker.
2026-07-09 01:14:25 -07:00
vh c948013a36 feat(gateway-chat): auto-voice quoted dialogue via mOrpheus TTS
Gateway-chat now auto-plays quoted text from each assistant reply through the mOrpheus
TTS endpoint. Sidebar gains a 🔊 toggle + endpoint/voice fields (persist in localStorage,
prefilled to irv-ml1:8299 / baddy). On reply-complete, straight and typographic double
quotes are extracted, joined, POSTed to /tts, and the returned WAV plays (click 🔊 to
replay; a new reply interrupts the prior clip).

Requires CORS on the wrapper (page served from ana-docker:8091 fetches irv-ml1:8299
cross-origin) — added CORSMiddleware(allow_origins=[*]) to the mOrpheus tts app (internal-
only endpoint). Verified end-to-end: preflight + POST return ACAO=*, valid 24kHz WAV.

Deployed: tts container rebuilt/recreated on irv-ml1; page pushed to ana-docker conf
(bind-mounted, live on next request).
2026-07-09 00:58:13 -07:00
vh 01eedd8d27 feat(morpheus): permanent mOrpheus TTS stack (vLLM bf16 + SNAC/FastAPI wrapper) on irv-ml1
Two-container stack serving MrDragonFox/mOrpheus (uncensored Orpheus TTS, Llama-3.2-3B
-> SNAC 24kHz). vllm-morpheus (GPU/3090) emits Orpheus audio tokens; morpheus-tts (CPU)
SNAC-decodes them to WAV and exposes POST /tts (baddy voice + zero-shot cloning). Deployed
+ tested end-to-end (28/28 valid frames, valid WAV, reachable over WG).

Hard-won config, all encoded in compose/README:
- bf16 REQUIRED: --quantization fp8 destroys audio-token generation (0 valid SNAC frames
  even at greedy). Footprint ~7.9GB.
- Image PINNED to v0.23.0: 'latest' ships Blackwell oink/aiter kernels that crash on Ampere
  import.
- 3090 (not the comfy-contended A6000); --enforce-eager to fit the shared card.
- RTF ~1.0 end-to-end (gen ~98 tok/s / RTF 0.84 + CPU decode + HTTP).

INTERNAL RESEARCH ONLY (CC-BY-NC-4.0); do not expose externally.
2026-07-09 00:49:25 -07:00
vh 99a4a1721f config(litellm): name gen backend by real model (aeon → qwen3.6-35b-a3b-heretic)
The gen seat's vLLM served-name was still qwen3.6-27b-aeon, a stale skin
left over from the AEON-27B → 35B-A3B-heretic swap — it named neither the
right family (aeon) nor size (27b vs 35B-A3B). Renamed the served-name to
qwen3.6-35b-a3b-heretic (+ -thinking) on ana-ml2 :8015 via the stack .env,
and repointed litellm's gen / gen-reasoning / summarizer-large model refs +
comments to match, so /v1/models, the gateway config, and spend-logs all
reveal the actual model in the request path.

Verified end-to-end: gen -> 'PIPELINE OK', gen-reasoning -> content + reasoning
surfaced, all three aliases healthy. char-rp / char-rp-reasoning untouched.
2026-07-08 18:40:08 -07:00
vh b889c55229 memory: archive 10 spent June entries (7 Recent decisions + 3 Tried) to archival-memory.md
Kept the [2026-06-14] standing credential-migration directive. persistent-memory 546->510 lines;
back-ref counts 118->125 (Recent decisions), 98->101 (Tried). Non-destructive move; archival-memory
is append-only with _Archived 2026-07-08._ stamps.
2026-07-08 15:55:09 -07:00