The comment still described TheDrummer Magidonia-24B-v4.3 Q6_K on
llama.cpp, which was replaced by the vLLM MeroMero-v2 NVFP4A16 seat on
2026-08-12. Routing was already correct (:8016 is MeroMero); only the
prose was wrong, so anyone reading the config got the wrong model family
entirely.
Records why the seat exists: char-rp-reasoning is a Qwen3.x derivative and
emits ~5-6k chars of CoT per turn regardless of which Qwen RP tune is
loaded. Measured 2026-08-16 on identical prompts -- Dark-Scarlett 6036 ch
vs Fable-Fusion 5323 ch -- so that is the base family, not the finetune,
and no swap within it fixes it. Gemma-4 is the non-thinking seat.
Also pins the mandatory --default-chat-template-kwargs
'{"enable_thinking": false}' rationale from b8f0f4c, and flags that the
temp 1.1 / min_p 0.10 samplers were tuned against the retired
Mistral-family seat and never re-tuned for Gemma-4.
Docs-only: no litellm_params touched, no routing change.
Operator-directed evaluation window. char-rp-reasoning now resolves to
Fable-Fusion 711 on :8019 instead of Dark-Scarlett on :8018; DS is DOWN
because GPU1 is zero-sum and FF occupies her slot.
This is an EXPLICIT substitution, not a silent alias swap: the config
block says so in place, carries the measured justification, and names the
rollback. char-rp-fable is added as the seat's honest name so the
evaluation can address it without depending on the temporary repoint, and
as a distinct model_name it gets its own litellm_params object rather than
sharing one (which is what bleeds sampler overrides between variants).
Samplers are unchanged from the DS entry and match the model card's
thinking-mode recommendation (temp 1.0 / top_p 0.95 / top_k 20). Verified
the FF chat template actually honours enable_thinking
(chat_template.jinja:44) rather than ignoring it -- the mismatch that
returned null content on the MeroMero seat.
Verified end-to-end through the gateway on both aliases: prose in content,
CoT in reasoning_content, finish=stop.
CONSUMER HAZARD: FF reasons heavily (2.1-4.6k chars). At max_tokens=1200
one of seven calls returned EMPTY content with finish_reason=length --
reasoning ate the whole budget. Not a refusal and not an alias fault. Use
max_tokens >= 3072; 6/6 clean there. No default is baked into the alias
because that would override caller intent silently.
Dark-Scarlett v1.0 refuses too much on the char-rp-reasoning seat. Root
cause is visible on its card: ReadyArt/Dark-Scarlett-v1.0-27B is a plain
finetune of stock Qwen/Qwen3.6-27B, tagged unaligned/nsfw/erp but carrying
no abliteration -- the base model's refusal machinery is intact, so
off-distribution prompts revert to safety-tuned Qwen3.6 behaviour.
Candidate kkuspa/Qwen3.6-27B-Fable-Fusion-711-...-NVFP4A16 is refusal-ablated
(Heretic), a structural edit rather than a behavioural preference. Verified
before pulling: Qwen3_5ForConditionalGeneration wrapper class, 15 mtp.*
tensors in a separate bf16 shard AND individually enumerated in
quantization_config.ignore, NVFP4A16 with null input_activations, FP8 KV
scales shipped, 262K context, Apache-2.0. Staged byte-verified at
/tank/aimodels/fable-fusion-711-nvfp4a16 (28.55 GB).
services/refusal-probe: deterministic marker-based classifier (LLM judge
only breaks AMBIGUOUS ties, never overrides), intensity-graded battery so
the report renders a refusal curve rather than an average, benign controls
that gate run validity, and explicit handling of the thinking-budget trap
-- empty content with finish_reason=length is reasoning exhausting the
budget, not a refusal, and is excluded from the denominator.
stacks/fablefusion-charrp-probe: throwaway :8019 seat serving as
char-rp-probe, never aliased to char-rp-reasoning. MTP depth 3 rather than
the card's 5 -- its 1.56x was measured greedy, and acceptance degrades at
the temp 1.0 this seat is probed at. GPU1 is zero-sum at 94.9/97.9 GB, so
this seat takes Dark-Scarlett's vacated slot; the A/B is sequential.
Re-quantizes the fleet `gen` seat from weight-only NVFP4A16 to a
mixed-precision build: NVFP4 W4A4 for layers 0-55 MLPs, FP8 W8A8 for the
attention projections / linear_attn / lm_head / layers 56-63 MLPs, FP8 KV
cache. Replicates the scheme of unsloth/Qwen3.8-27B-NVFP4 on the
abliterated weights.
The queued task named this "W4A8" (NVFP4 weights + FP8 activations). That
checkpoint cannot be served: vLLM 0.24's compressed-tensors dispatcher
(compressed_tensors.py:704-713) accepts NVFP4 weights with either no input
quantization (W4A16, which forces the Marlin kernel) or NVFP4 input
quantization (W4A4) -- anything else, FP8 included, raises ValueError at
load. CompressedTensorsW4A8Fp8 is INT4 weights gated on an exact-sm90
check, so it is closed on Blackwell twice over. The ~20% intuition was
correct; the scheme name was not. Getting FP8 into the mix has to be done
per-layer-group.
Established the gain before spending GPU time: unsloth's build was already
on-box, so serving it as a probe measured +19.1% over our seat at identical
MTP acceptance -- a kernel-level result, no requant needed to learn it.
Measured, cache-busted, bs=1:
decode 80.12 -> 94.53 tok/s (+18.0%)
MTP acceptance 47.8% -> 47.7% (unchanged)
perplexity (n=6) 6.941 -> 7.059 (+1.7%)
abliteration 4/4 -> 4/4 (preserved)
weights on disk 27.7 -> 22.5 GB (-19%)
Surface test green on the live seat: plain chat, vision, tool calling,
thinking split, 36K-token needle retrieval, streaming. All 7 LiteLLM
aliases verified routing.
GEN_GPU_MEM_UTIL 0.45 -> 0.43: the new weights are 5.2 GB smaller, and at
0.45 the seat absorbed that slack as KV, leaving meromero-charrp 0.18 GiB
short of its budget on the shared GPU0 -- it crash-looped. Handing the
space back leaves gen 422K tokens of KV (1.6x its 262K context) and both
seats co-resident at 89.8/97.9 GB.
Also records two measured negatives so they are not re-chased:
GEN_SPEC_TOKENS is already optimal at 3 (swept 2/3/4/5 -> 77.1/80.1/78.7/
75.9 tok/s), and vLLM's prompt_logprobs are ~uniform while speculative
decoding is on, so perplexity must be measured with spec off.
Pipeline, acceptance harness and raw measurements land in
services/gen-seat-mixed-quant/. Rollback is one .env line; the previous
build is untouched at /tank/aimodels/qwen38-27b-uncensored-nvfp4.
The char-rp seat shipped with no tool-call parser at all, so every
tools-bearing request was rejected outright:
400 "auto" tool choice requires --enable-auto-tool-choice and
--tool-call-parser to be set
MeroMero-v2 is Gemma-4, which emits its own native
`<|tool_call>call:name{...}<tool_call|>` syntax rather than the
qwen3_coder XML the Qwen-family seats use. vLLM 0.24 ships a matching
`gemma4` parser whose TOOL_CALL_START/END, CHANNEL_START/END and escape
token constants line up with this tokenizer's etc/eoc/escape tokens
exactly.
Three flags, and they are a set:
- --tool-call-parser gemma4 + --enable-auto-tool-choice: the fix proper.
- --reasoning-parser gemma4: without it the post-tool-response turn
leaks a literal `<|channel>thought\n<channel|>` prefix into content
(upstream vllm #45834 — the chat template leaves the prompt inside an
open channel block).
- --default-chat-template-kwargs '{"enable_thinking": false}': MANDATORY
companion to the reasoning parser. The parser reads enable_thinking
from chat_template_kwargs and defaults it to True
(vllm/parser/gemma4.py:439); True makes is_reasoning_end() return
False at a new turn, pre-initialising the engine to REASONING, which
routes ALL plain RP prose into reasoning_content and returns a null
content — breaking every char-rp consumer. This template already
defaults enable_thinking to false (chat_template.jinja:350), so
passing it explicitly renders a byte-identical prompt (verified across
plain / tools / post-tool-response / system-prompt shapes). It changes
generation not at all; it only corrects the parser state machine.
Verified green on the live seat after deploy: tool call streaming and
non-streaming, tool-result round-trip (leak gone), plain prose in
content with reasoning null, vision unchanged.
Operator-directed (Vuong 2026-08-13): migrated stonehenge-park off the nh3-docker
beta deployment to a permanent fixture on ana-docker (10.250.50.70) before the
v1.0.0 final cut. SQLite (park-data) migrated consistently (stop -> tar-copy ->
start; byte-identical). restic auto-covered by ana-docker's /var/lib/docker/volumes
source. Stable name park.phasefinal.com -> 10.250.50.70 (Cloudflare DNS-only) so
clients decouple from the host IP. Homepage tile 'The Henge' added. nh3-docker stack
left stopped as rollback pending park-dev cutover verification.
New self-contained stack (FastAPI + SQLite + in-process scheduler) from
vh/stonehenge-park tag v1.0.0-beta.1, deployed to nh3-docker per park-dev's
operator-approved request. Port 8420, LAN/WG-internal; park-data volume (SQLite
sole source of truth) covered by the host's /var/lib/docker/volumes restic source.
Image built locally (no registry yet); .env carries PARK_API_KEY from the vault.
althing push to henge-crow deferred (PARK_ALTHING_CHANNEL empty) until althing-cli
is wired into the container.
The TTS path was hardwired to the parked zonos-gateway: it force-reverted the
endpoint field back to zonos :8890 on load, hardcoded model=ext-tts, and decoded
the response as Zonos-specific raw float32 PCM @ 44.1kHz. Result: quoted-text TTS
was dead once zonos was parked, and pointing the field elsewhere silently failed.
- Honor the interface: set endpoint/model/voice defaults only when a field is
empty; never rewrite a user-typed value (removed the zonos auto-revert regex).
- Add a TTS model field (ttsModel); send the UI's model instead of hardcoding.
- Playback: request standard OpenAI /v1/audio/speech mp3 and decode via
audioCtx.decodeAudioData (handles wav/mp3/ogg/flac from any endpoint).
- Defaults: endpoint = LiteLLM ext-tts alias (fleet TTS gateway), voice = nova.
wgtunnel deployed + accepted end-to-end (tunnel-dev): erebe/wstunnel v10.6.2 behind
traefik on ana-docker, Host boring.phasefinal.com (Mode A anaprod cert), --restrict-to
ana-wg:31337 (not an open relay). Mirror per fleet convention; full project in vh/wgtunnel.
TTS development moves to a dedicated repo (~/development/tts-stack) so a separate
agent can own tuning/dev. Mirrors the chatterbox-fast extraction:
- stacks/dots-tts/ reduced to a pointer README (code/Dockerfile/compose/tests/env
now canonical in tts-stack).
- voices/ canonical corpus moved out to tts-stack/voices/. Blast-radius checked:
no eshpfi playbook/script reads the corpus (other voices/ refs are unrelated
host paths under /worktank/...).
- persistent-memory updated: TTS dev extracted + stood down; reverses the earlier
"corpus home = eshpfi voices/" call.
The ~15 experimental TTS compose wrappers stay here as reference (catalogued in
tts-stack/KNOWLEDGE.md). Live service on irv-ml1:8198 is unaffected (runs from a
copy on the host).
dots' prosody honors a pause only for ellipsis (~+0.43s) and period (~+0.3s);
comma/semicolon/colon/dash all run flat (~+0.03s vs no-punct), measured via a
duration-over-N-runs pause probe against the live service. Two sub-causes for
the flat clause reads: em-dashes regressed in v2 (the —→- fold made them read
as word-joiners), and semicolons were never honored by dots at all.
Operator ruled ellipsis "too much" → map semicolon, clause colon, and em-dash
to a period in _sanitize (believable ~0.3s clause pause). Guards, pinned by
tests: digit-guarded colon so times (3:45) and ratios (2:1) keep their colon;
en-dash kept folding to hyphen so numeric ranges (10–20) don't become "10.20";
a genuine ellipsis retains its strong pause.
Deployed to irv-ml1:8198 as local/dots-tts:v3 via the redeploy2 build →
:8199-test → pause-gate → cutover pattern (gate measured +0.427s, live healthy).
Curly apostrophes (ratatoskr's LLM emits typographic punctuation) made dots
mispronounce contractions ("Donut's"->"donut ess"); fold curly->ASCII before
synth, keep normalize_text on. Add server-side sentence-chunking so long turns
stop truncating at dots' ~40s single-generate cap (verified full 160s Zev).
Dockerfile: pin dots.tts==0.2.1 + torch/torchaudio==2.8.0 (upstream constraints
now pin a phantom gradio==6.17.0; float torchaudio->2.11.0 crashes the load).
LFM2.5 is </think>-delimited (opening tag in prompt); deepseek_r1 splits
reasoning into reasoning_content so content is the clean post-</think>
answer. Re-smoke: content valid JSON + reasoning_content populated. License
production-cleared (operator <$10M ruling), still out of routing per the
measurement gate.
vllm-lfm25 on ana-ml2 GPU1 :8021 (LiquidAI/LFM2.5-2.6B, util 0.09 into
unreserved slack, max-len 16384, no reasoning-parser so content is non-empty).
LiteLLM alias lfm2.5-2.6b with vendor sampling baked as default (temp 0.1;
top_k 50 + repetition_penalty 1.1 via extra_body). Eval-only, not in any
routing chain, pending operator ruling on LFM Open License production use.
The incumbent Qwen3-Reranker-0.6B was measured actively harming 80/90
fleet queries on main+knowledge_base (and inverting the bare-name region
behind Worldtree #389) — no-reranker beat it 89/90 vs 56/90. Brokkr's R43
bake-off selected BAAI/bge-reranker-v2-m3 (A3): 90/90 top-10, mean rank
0.19, multilingual (XLM-R), ~1.2 GB lighter than the incumbent.
Control arm (A2 = same Qwen weights, seq-cls head) scored identical to the
incumbent, proving the fault is a training prior, not the serving head —
which cancelled the expensive Qwen3-4B arm before it cost a GPU seat.
Cutover boundary 2026-08-06T17:37:48Z. The qwen3-reranker alias and the
:8002 backend are retained for one-edit rollback. Adds the process audit
trail at docs/pfi/reranker-selection-ledger.md.
Gate rebuilt off vh/muninn-gate main (bc04c4c) for the muninn-dispatch
0.1.4->0.1.5 bump (concept_schema/concept_schema_source row fields). Gate
version unchanged at 0.0.14; note to also tag the image with the source SHA
for traceability, and that the pin is authoritative in pyproject.
The mimir-inbox deploy changed two things the committed example documents:
the mimir-inbox key gained the control scope (2026-08-01, for cancel/retry),
and the staging root is no longer a placeholder — it's the real shared dir on
corviduo-dev, path-agreement probe PASS (muninn-dev). Bind was already correct
at :8090 (the stale :8080 was only in the gate repo's own example).
WG-internal FastAPI+HTMX front end for large-document ingestion into the
Muninn KB, over the muninn-gate API (browser -> mimir-inbox -> staging ->
path-addressed POST /jobs). Co-located on corviduo-dev with the gate (:8090)
and the worldtree-personal muninn watcher per the operator's 2026-08-01
co-location ruling (reversing the earlier off-box/NFS plan; worldtree-dev
approved the box placement).
- Dockerfile: python:3.11-slim + uv sync --no-dev --frozen (--no-dev is
load-bearing; the dev group's muninn-dispatch path source is absent in-image
and INV-MI-7 forbids importing it). Single-stage by design — src/ stays in
the final image (uv installs the project editable-linked to src/).
- compose.yaml: uid 1000, host-net bind 10.250.50.152:8091, staging :rw,
TCP-liveness healthcheck (deliberately not coupled to gate reachability).
- Built from vh/mimir-inbox HEAD c8ab38f; deployed + healthy.
Records the open-in-place claim semantics (worldtree-dev, runner.py:362-367)
and the INV-MI-19 retention rule (staged files persist until job terminal;
gate retry returns a false-200 on a swept source) in persistent memory.
Deployed on corviduo-dev, co-located with the worldtree-personal muninn
watcher. ingestion_root=/data/state/ingestion (shared state volume, byte-
identical to the watcher); runs as uid 1000 to write the queue; staging
bound :ro at the ratified /mnt/muninn-staging/mimir-inbox (local placeholder
until the shared mount + mimir-inbox writer land). Boot verified: /ping
{"service":"ok"}, /health watcher.running=true (byte-identity proven).
Image built out-of-band with the Gitea read token as a BuildKit secret.
Real config (bearer keys) lives on-server at /opt/docker/conf 0600.
The Heid panel plan uses Kimi's coding endpoint, not the general Moonshot API.
kimi-k3 now → openai/k3 @ https://api.kimi.com/coding/v1 (KIMI_CODE_API_KEY,
Vivace); the original general-endpoint entry is kept as kimi-k3-gen-api
(api.moonshot.ai, MOONSHOT_API_KEY). Both verified live through the gateway.
Same k3 constraints on both: temperature MUST be 1 (else 400), reasoning model
(reasoning_content vs content, needs adequate max_tokens).
Adds model_name kimi-k3 → openai/kimi-k3 @ https://api.moonshot.ai/v1
(OpenAI-compatible), keyed by MOONSHOT_API_KEY (compose env + .env.example
placeholder; real key on server only). Verified live through the gateway.
Two Moonshot constraints captured in the config comment + pinned: K3 accepts
ONLY temperature=1 (else 400), and it is a reasoning model (CoT in
reasoning_content, answer in content — needs adequate max_tokens or content
returns empty). Model id confirmed via /v1/models.
Mirror comfy-dev's operator-run allocator A/B result off irv-ml1: drop
--disable-cuda-malloc (ComfyUI keeps CUDA's default async allocator) and
remove PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True (native-allocator-only,
inert under cudaMallocAsync). The native+expandable_segments combo was
fragmenting/over-reserving (~45 GB allocated-but-unused) and OOMing the LTX-2.3
v1.5.0 LoRA stack at Gemma TE load; cudaMallocAsync packs tighter + returns
freed blocks so the job fits (stress test peaks ~82% VRAM, 0 OOM). The
shared-A6000 phantom-OOM that --disable-cuda-malloc guarded is gone since TTS
moved to the 3090 (2026-06-18).
Record the RTXVideoSuperResolution node clone + the nvidia-vfx pip install
(scoped --extra-index-url, uid 1000) in the stacks/comfyui runbook. Flag the
durability split: the node is persistent (basedir/custom_nodes) but nvidia-vfx
lives in the disposable venv (run/), so it must re-run after every fresh
bootstrap. Deliberately not a global PIP_EXTRA_INDEX_URL (torch-pin safety).
Closes the comfy-dev provisioning ticket.
Mirror the canonical EMOTION-DIALS-SPEC.md from vh/zonos-gateway (now carries
the provisional per-voice emotion presets baked as gateway 0.2.1) and capture
the axes-sweep → bake arc in persistent memory.
Created private vh/zonos-gateway on gitea, imported the previously-unversioned
~/zonos-gateway working tree (source + dials-first spec + voices). Updated the
stack README, spec §8, and the sister-repos table to point at the repo. Remaining
follow-up: CI + deploy key to wire the irv-ml1 deploy tree to the repo.
Canonical direction: emotion set by twisting raw dials per-utterance, not preset
selection. Presets demoted to optional examples. Spec covers the dial vocabulary
+ ranges, the emotion_cfg_scale 'deaf by 1.5' rule (NO cap — documented ceiling,
explicit over implicit), measured RTF cost, starting-point dial-sets, an LLM
client system-prompt snippet, and clone reference guidance (~15-24s, no
transcript). Follow-ups flagged: align dials.py cfg help/metadata, trim in-code
PRESETS pending usage check, stand up vh/zonos-gateway for version control.
Captures the production Zonos TTS path that the repo was blind to: stock ZONOS2
:1920 engine (stacks/zonos-engine) fronted by zonos-gateway:0.2.0 :8890, reached
via the LiteLLM ext-tts alias. Documents the emotion-preset system (neutral/warm/
excited/sad/intense/whisper, the simple preset: caller path), the API, and the
measured real-time cost (calibrated steering free at RTF~0.52, cfg1.5 ~0.625 —
still realtime). Flags the gateway source (~/zonos-gateway on irv-ml1) as not yet
in gitea. Marks stacks/zonos (v0.1 Gradio) dead/superseded. Records the emotion-
lever finding (text-priming flat -> native steering works) in persistent memory.
The Zonos TTS engine that zonos-gateway fronts (irv-ml1 3090, feeds asset-engine +
gateway-chat) ran as a bare native process with its real invocation existing ONLY
in the running process argv — the committed harness/zonos_server.sh on irv-ml1 was
STALE (said A6000/:1919, no perf flags; live is 3090/:1920 with cuda-graph/num-pages/
max-running-requests/memory-ratio). Captured the corrected canonical invocation +
tunables + the containerization plan here so the config survives a process death.
Engine = stock Zyphra/Zonos2 @ 194c0a3 (no custom PFI server code); torch 2.9.1+cu128;
15 GB HF weights. Next: containerize in-place on the 3090 (operator: keep off the
A6000, it OOMs under ComfyUI). Not yet built — this commit is the config capture only.
granite (fleet fan-out summarizer/classifier) had no explicit --max-num-seqs,
so vLLM V1 resolved it to 128 — which caps concurrency BELOW granite's own KV
bound (~192 concurrent @ 1K-token calls, more for shorter classify calls).
Pinned it very high (1024) so the KV pool is the only bound; VRAM-neutral
(the KV pool is util-bound, unchanged). Added the flag to the granite command
+ GRANITE_MAX_NUM_SEQS to the env template. Live applied + verified
(resolved max_num_seqs=1024, seat healthy).
Operator-directed 2026-07-16. Moved the char-rp prose seat (Magidonia-24B,
llama-charrp) from GPU0 to GPU1 (CHARRP_GPU_ID 0->1; recreate llama-charrp
only -- the var is shared with the retired GGUF reasoning service), then
re-optimized every context-relevant seat on both cards to native/max context
with acceptable headroom:
GPU0 (both seats now 256K native, ~14 GB reserve):
- char-rp-reasoning 150K -> 256K (heretic2 stack, util 0.38->0.46, 1.56x)
- gen 256K, max-num-seqs 16 -> 32 (qwen36-27b-aeon, util 0.30->0.42, 5.43x)
GPU1 (~6.7 GB headroom):
- granite 64K -> 128K full-chapter (vllm stack, util 0.18->0.27, 1.50x)
- char-rp 128K native (4 slots), selene/reward/embed/rerank unchanged
All seats gateway-verified healthy. Live .env changes on ana-ml2 with per-stack
backups (*-20260716). Templates updated to match; the qwen36-27b-aeon template
carries a NOTE that its served-name/model still lag the 2026-07-08 gen model swap
(35B-A3B-heretic) -- separate reconciliation. persistent-memory records the full
layout + the util-floor / per-model-KV-cost lessons.
Note: the heretic2-charrp-reasoning stack (char-rp-reasoning's live config) is
still untracked in git -- standing open-loop, its .env change lives server-side only.
Operator-directed 2026-07-16: free ~10GB on ana-ml2 GPU1 to relocate a GPU0
model onto GPU1. granite-4.1-8b (fleet summarizer) was over-provisioned at
util 0.34 / max-model-len 131072 with a flat 0.0% KV usage.
Set GRANITE_GPU_MEM_UTIL 0.34 -> 0.18 and GRANITE_MAX_MODEL_LEN 131072 -> 65536
on the live /opt/docker/compose/vllm/.env (backup .env.bak-pre-granite-rightsize-
20260716), recreated vllm-granite ONLY (shared stack). Result: GPU1 62,641 ->
51,897 MiB used (~10.5GB freed, ~45GB free now); KV 6.45 GiB / 84,528 tok /
1.29x concurrency @ 65536; summarizer verified healthy.
The util drop required the max-len drop: on this shared card the effective KV
slope is ~950 MiB per 0.01 util, and vLLM refuses to start unless the KV pool
holds >= 1x max-model-len -- util 0.15 undershot (est max-len 47184 < 65536,
crash-loop, ~2-3 min summarizer outage) before 0.18 landed. 65536 is granite's
precedented summarizer ctx; a summarizer doesn't need 131072.
.env.example updated to the new util (max-len was already 65536 in the template;
live had drifted to 131072). persistent-memory.md updated (parked item closed).
Operator-directed 2026-07-15. The dedicated Qwen-Image-Bench NVFP4 judge
backend on ana-ml2 GPU1 (:8014) was stopped to reclaim ~32GB after the
arbo -> gen hero-judge switch. Both LiteLLM gateway aliases that pointed at
it -- image-judge and qwen-image-bench -- now repoint to the gen backend
(:8015, qwen3.6-35b-a3b-heretic, vision-intact), held at deterministic
judge sampling (temp 0 / top_k 1 / rep_pen 1.05) with enable_thinking:false
(a reasoning preamble breaks json_object). Verified live: both answer with
:8014 down, so they are definitively on gen.
Incidental: backfilled the canonical char-rp-reasoning litellm block, which
had lagged live since the 2026-07-14 NVFP4+MTP seat repoint (model
deckard-pkd-27b -> char-rp-reasoning, top_k 40 -> 20, min_p dropped,
enable_thinking:true added). Required so pushing the canonical would not
clobber the correct live block.
Live changes applied out-of-band (config push + litellm restart + stack
stop on ana-ml2); live config backup at
config.yaml.bak-pre-imagejudge-20260715. Revert path documented in the
config comment. persistent-memory.md updated (parked item closed).
Move the ~22-service flat "AI Systems" group off the Main tab into a new
four-tab layout (Main / AI / Infrastructure / Toolchain). The AI tab sorts
the inference fleet by function into seven groups:
AI - Inference gen, char-rp, char-rp-reasoning, Granite summarizer
AI - Eval & Retrieval Selene, Skywork Reward, Qwen3 rerank/embed, image-bench
AI - Gateways & Chat LiteLLM, Asset Engine, Gateway Chat, Open WebUI, ...
AI - Speech (TTS) Chatterbox Fast, Kokoro, mOrpheus
AI - Audio Tools Parakeet ASR, YT Voice Clipper
AI - Image & Media ComfyUI, Arbo
AI - Dormant stopped rollback seats + retired auditions
Relabel each stack's homepage.group so canonical stacks/ matches the live
containers on ana-ml2, ana-docker, and irv-ml1. Dormant stacks were refreshed
with `docker compose up --no-start` so they carry the new label while staying
stopped (compose-start rollback preserved). settings.yaml drives tab/order/
columns; services.yaml and README updated to the new scheme.
R36 gate (2026-07-14) validated NEO-CODE ships on all axes: tool-calling 0.967
(attach_tool 1.00, 0 runaways — #355 eliminated), prose genre-artifact-fine
(less clichéd than gen), refusal uncensored-as-spec + CSAM-clean.
#355 root cause was MODEL-level, not the reasoning-budget-forcing bug: Deckard
emitted Qwen's native qwen3_coder XML tool format malformed -> llama.cpp leaked
the closing tags into the arg value -> Bifrost attach_tool schema error -> retry
-> reasoning runaway to max_tokens. NEO-CODE emits the same native format cleanly
on the same seat/parser -> no schema error -> no runaway. The fix was the model
swap; there was never a wrong parser (the XML is Qwen3.5/3.6-native).
- reasoning seat: Deckard-PKD (Qwen3.5) -> NEO-CODE=Heretic2-Thinking (Qwen3.6-27B) Q5
- samplers: card defaults (temp 1.0 / top_p 0.95 / top_k 20 / min_p 0.0), DRY dropped
- ctx: 256K max; custom llama.cpp kept (qwen3_coder parse + PR#25544 belt-and-suspenders)
- persistent-memory ACTIVE 1 marked resolved
The char-rp-reasoning (Deckard) seat now runs llamacpp-charrp:custom-latest via a
new LLAMA_REASONING_IMAGE var (Magidonia char-rp stays on stock — no reasoning bug).
Fixes Worldtree #355 at the source (budget multi-terminator handles Qwen3.5's
<tool_call> reasoning end-tag). Live 2026-07-13: Deckard loads on Blackwell, serves
coherent, reasoning bounds at the 400 budget. Rollback via .env LLAMA_REASONING_IMAGE.
Durable record of the custom llama.cpp the char-rp-reasoning seat will run to
fix Worldtree #355 (reasoning-budget forcing broken in stock b8840 — single
end-tag </think> can't match Qwen3.5's <tool_call> reasoning terminator, so the
budget never force-closes and reasoning runs away to max_tokens). PR #25544
adds multiple terminating sequences; unmerged upstream, so we build it.
- build.sh: reproducible recipe (clone master@6eddde0 + merge PR #25544 +
resolve the 1 server-common.cpp conflict + CUDA build for Blackwell sm_120)
- README.md: why + acceptance test + rollback + REMOVE-WHEN-MERGED tracking
Image llamacpp-charrp:6eddde0-pr25544 BUILT + smoke-tested on ana-ml2; seat
swap pending. See also auto-memory reference_charrp_custom_llamacpp_pr25544.
Drop-in system prompt for an agent whose quoted output is voiced by mOrpheus: speak in
double quotes (only quoted text is voiced), phrase each quoted line as one coherent
utterance (per-quote prosody), and the sparse/boundary/no-stack tag rules (measured
stability on the early checkpoint). Lists the honored tag set.
Per-sentence chunking generated each sentence cold, flattening intonation/prosody that
spans the whole quoted line. Chunk by QUOTED SECTION instead — each contiguous quote is
generated whole (max_tokens 2400) so its prosody stays intact; multiple quotes in a reply
still play serially on the shared clock. extractQuotes already returns exactly these spans;
dropped splitSentences.
Split the quoted dialogue into sentences and stream each as its own short /tts/stream
request (max_tokens 900), queued back-to-back on one shared AudioContext clock (speechHead)
so playback is gapless and in order. First sentence starts fast; each chunk is short so it
generates cleanly (no ramble/cap risk); the next sentence generates while the current plays.
A newer reply supersedes via the ttsGen counter; 🔊 replays.
Browsers suspend the Web Audio AudioContext until a user gesture; speakQuotes fires on
reply-complete (no active gesture), so a suspended context played silently. Prime/resume
the context on any click or keydown (capture phase) so it's running before playback.
Server side was fine throughout (/tts + /tts/stream both 200 with valid audio).
- max_tokens default 2400->3500 (~42s) in wrapper + gateway-chat client, with a _cap()
clamp so prompt+gen never exceeds MAX_CTX (4096) — a cloning ref block is ~1100 tokens,
so an unclamped 3500 would overflow context on the clone path.
- Staged clone voices: /voices dir of <name>.wav + <name>.txt, each encoded to its Orpheus
reference block at startup; voice="<name>" zero-shot clones it. Beatrice (a chatterbox
reference) staged as the first normal-voice clone. GET /voices lists baddy + clones.
- compose: mount voices dir + pass MORPHEUS_MAX_LEN to the wrapper (clamp must match engine).
vLLM concurrency (measured, --max-num-seqs 8, 250-tok reqs): near-linear batching — 8
concurrent finish in the same ~2.8s as 1 (707 tok/s, 8.1x single, flat per-req latency).
Chunked-sentence production can fan out for ~8x throughput; CPU SNAC decode is the scale
bottleneck, not generation.
Cut-offs were the max_tokens=1200 ceiling (~14.6s of audio), not memory (~1250 tokens
<< 4096 context). Diagnosis: the repetition penalty is load-bearing for clean stops —
rep 1.0 => the model never emits end-of-speech and rambles to the cap; rep 1.1 (the
wrapper default) => clean natural stop. So normal lines already complete; only genuinely
long dialogue (>~14.6s, ~25+ words) hit the cap. Raised default + client max_tokens to
2400 (~29s), still within the 4096 context (no memory cost). Verified: a 49-word line
now finishes at 16.73s (was clipped at 14.6s).
Wrapper gains POST /tts/stream: reads the vLLM token stream, decodes SNAC in WINDOWED
CHUNKS (every 6 frames, decode [2 ctx | 6 | 2 ctx] and emit only the middle 6 — context
both sides => seamless), and streams raw PCM16 (24kHz mono) as it generates. Windowed
(not per-frame) because per-frame CPU decode's per-call overhead x ~60 frames serialized
to ~7s (RTF 2.2); windowed keeps up (RTF ~0.97). Whole-clip /tts kept for non-browser use.
gateway-chat plays the stream via the Web Audio API (fetch reader -> int16->float32 ->
scheduled AudioBufferSourceNodes on a running clock; a new reply supersedes the prior
stream via a generation counter; 🔊 replays). Measured: TTFA 0.80s (was ~4.5s whole-clip),
RTF 0.97, full-duration match. CORS already covers the new route.
Deployed: tts rebuilt on irv-ml1, page pushed to ana-docker.
Gateway-chat now auto-plays quoted text from each assistant reply through the mOrpheus
TTS endpoint. Sidebar gains a 🔊 toggle + endpoint/voice fields (persist in localStorage,
prefilled to irv-ml1:8299 / baddy). On reply-complete, straight and typographic double
quotes are extracted, joined, POSTed to /tts, and the returned WAV plays (click 🔊 to
replay; a new reply interrupts the prior clip).
Requires CORS on the wrapper (page served from ana-docker:8091 fetches irv-ml1:8299
cross-origin) — added CORSMiddleware(allow_origins=[*]) to the mOrpheus tts app (internal-
only endpoint). Verified end-to-end: preflight + POST return ACAO=*, valid 24kHz WAV.
Deployed: tts container rebuilt/recreated on irv-ml1; page pushed to ana-docker conf
(bind-mounted, live on next request).
vh