dvalin confirmed the live A/B-proven set IS canonical for Deckard as a dark-RP reasoning seat:
temp 1.0/top_p 0.95/top_k 40/min_p 0.05, no presence/rep penalty, DRY 0.8 server-side. Endorsed
over the card's base-thinking (top_k 20/min_p 0/presence 1.5). No value change; comment + memory
record the confirmation + tuning ladder (flat->min_p 0.08, loops->DRY 0.9, over-damped->DRY 0.6/off).
Operator wanted a reasoning-RP model that tolerates DRY (RpR-v4 forbids rep/DRY -> a
1/30 loop tail). Ran the full A/B on brokkr's 30-prompt D1 suite (content-only, slop-scored):
- Deckard-PKD (Qwen3.5-27B, DavidAU creative tune) WON: 0/30 loops, 0/30 refusals, clean
managed reasoning (native Qwen3.5 <think>/enable_thinking), DRY-tolerant, ~57 tok/s,
runs on the base llama-swap b8840 image. -> now the char-rp-reasoning seat (:8018).
- RpR-v4: 0 refusals but 1/30 loop (no-DRY). Pantheon-27B: clean slop but 7/30 explicit
refusals + needs the newer ggml-org/llama.cpp image (Qwen3.6 won't load on b8840).
Snowdrop + Gembrain (Gemma-4): floored (llama.cpp can't manage their reasoning without
the vetoed template hacks). Losers kept on disk as alternates.
- char-rp (Magidonia) unchanged; gen unchanged. gateway char-rp-reasoning -> Deckard
sampler (temp 1.0/top_p 0.95/top_k 40/min_p 0.05; DRY server-side).
dvalin-smithy offered a follow-up sampler pass for char-rp / char-rp-reasoning after
they accumulate real Worldtree/SillyTavern character-role traffic. Parked as a future
option (thread 01KX1DS6...) — nothing to tune until there's live-session data.
char-rp -> TheDrummer Magidonia-24B-v4.3 Q6_K (Magistral prose, ~65 tok/s,
zero refusal, tight POV) via llama.cpp (:8016).
char-rp-reasoning -> ArliAI QwQ-32B-RpR-v4 Q5_K_M (abliterated managed reasoning,
~52 tok/s, reasoning surfaces in reasoning_content) via llama.cpp (:8018).
- New canonical stack stacks/char-rp-gguf/ (llama-server x2, GPU0-pinned, ~86/97G
co-resident with gen). GGUF sidesteps the vLLM-NVFP4 + Mistral-tokenizer traps that
killed the Angel serve. Never Ollama.
- Best-of-breed per seat: no single dense 24-32B is both an elite non-thinking prose
seat AND a clean managed-reasoning seat on llama.cpp (Magidonia [THINK] boundary is
loose; Cydonia-R1 <think> runs away; QwQ is template-managed). Pantheon-Reasoning-27B
stays rejected (re-censors in <think>; RpR-v4 abliterated reasoning is the fix).
- Gateway rewired: char-rp->:8016, char-rp-reasoning->:8018, Mistral/QwQ samplers,
dropped the Qwen enable_thinking kwarg. One-model Magidonia fallback documented.
- Retired the ms32-24b-angel stack.
comfy-dev's explicit-over-implicit call: arbo now sends train_id, so the
worker no longer derives the loras/trained/{train_id}/ namespace from
output_dir.parent (which coupled it to arbo's handoff layout). train_id is
optional + path-safe-validated; when present it wins, else the path
derivation remains as the fallback. Wired through TrainRequest ->
validate_request -> published_relative_path -> _publish_lora. 18 tests green.
On a train reaching succeeded, IN ADDITION to output/{name}.safetensors
(unchanged download source), COPY it into ComfyUI's loras search path at
/storetank/arbo/models/loras/trained/{train_id}/{name}.safetensors and
return published_lora_name (the ComfyUI-relative LoraLoader string) in the
terminal GET /train/{id} payload (arbo Phase 2 auto-registration, §4.1/§7).
- Copy not move; a publish failure NEVER fails the train (keeps succeeded,
omits published_lora_name, logs the reason to the tailable run log).
- INV-T7-safe: a copy to a fixed computed path, no new free-form args.
- train_id derived from the handoff layout (output_dir.parent.name).
- Provisions loras/trained/ (arbotrain 2775, group-write per the Phase-1
lesson; world-readable/traversable for ComfyUI) via the deploy playbook.
- ComfyUI verified to resolve nested loras subfolders (no flat fallback).
- Pure path helper unit-tested; 16 tests green.
The first real arbo train 422'd: SDXL checkpoints live at
/storetank/arbo/models/checkpoints/ (the 2026-06-13 move to the 1.8TB
/storetank volume), which wasn't in ALLOWED_MODEL_ROOTS — the old roots
predated the move (/worktank/models is gone, /worktank/comfyui host path
is empty; ComfyUI mounts /storetank/arbo/models -> /basedir/models inside
its container). Allowlist /storetank/arbo/models (llmuser-readable,
world-readable tree), drop the two stale roots. Regression test added (15 green).
comfy-dev cross-check: Sindra v1/v2 used alpha=dim/2 (0.5 LoRA scaling),
which produced the validated likeness; the initial alpha=dim (1.0) was a
stronger, unvalidated default. Align the default to the proven value
(operator/per-request can still override). Tests updated (14 green).
Host service (runs as llmuser, owns /opt/fluxgym + GPU access) that runs
sd-scripts SDXL LoRA training on demand for arbo — the infra-ops half of the
in-arbo LoRA training Phase 1 ownership split (vh/arbo
docs/contracts/in-arbo-lora-training-phase1.contract.md §4.1/§2).
- Fixed-invocation only (INV-T7): bounded params -> one sd-scripts command
shape; every param range/allowlist/path-containment checked before spawn;
bad request = 422, never a silent downgrade. 14 unit tests green.
- Thin supervisor: never imports torch; subprocesses the fluxgym venv's
accelerate. 1-job-at-a-time (arbo lease is the serializer, 409 is backstop).
Durable job records + boot reconciliation (§4.3).
- API: POST /train, GET /train/{id}[/log], POST /train/{id}/cancel,
GET /gpu-status (per-device VRAM + tts_on_3090 co-OOM signal), GET /healthz.
- Wire-shape (§7 resolved with comfy-dev): shared /worktank/arbo/train handoff
(group arbotrain, setgid 2770); worker binds 0.0.0.0:8203, arbo reaches via
host.docker.internal:host-gateway (reachability proven on 172.20.0.1:8203);
device-aware TTS steering via /gpu-status.
Deployed to irv-ml1 via playbooks/deploy-lora-training-worker.yaml (elway,
idempotent); systemd unit active; /healthz + /gpu-status verified live.
- gen := AEON dual NVFP4 serves (vLLM 0.24 + LiteLLM v1.91.0); reasoning-trace bug
was the LiteLLM shared-config mutation, fixed durably via distinct -thinking served-names.
- Worldtree personal character/thoughtful-character repointed to char-rp/char-rp-reasoning.
- LitBench-RM torn down, comfyui restored on irv-ml1.
New stacks/qwen36-27b-aeon: two co-located vLLM serves on ana-ml2 GPU0 —
gen (:8015, MTP off) and an RP seat (:8016, native MTP) — dense Qwen3.6-27B
(qwen3_5 GDN-hybrid, uncensored/abliterated), ModelOpt-NVFP4, multimodal,
256K context, depends_on-sequenced util split (~0.50/0.45). Each serve
carries a base + `-thinking` served-name so the `-reasoning` gateway records
target distinct LiteLLM deployments — otherwise a thinking-off request mutates
the shared litellm_params and clobbers enable_thinking (the shared-config
footgun that silently disabled char-rp-reasoning).
Gateway (stacks/litellm/conf/config.yaml): gen / gen-reasoning /
summarizer-large -> AEON :8015; char-rp / char-rp-reasoning added -> RP seat
:8016 (Qwen-RP sampler recs); gen-reasoning -> `-thinking`, char-rp-reasoning
-> `-rp-thinking`. Retired qwen3.5-122-a10b[-reasoning] + qwen-large[-reasoning]
(qwopus displaced; those named a 122B that no longer serves gen).
Probed live vs z.ai 2026-07-05: glm-5.2 = 1,048,576-token (1M) input context,
131,072 (128K) max output; no gateway-side cap (pure z.ai passthrough). Comment-only,
no runtime effect.
R30 b15-b17 removed mood.decay_rate/mood.stale_hours from canonical (OCEAN
wall-clock OU replaced per-turn decay); deployed /opt/worldtree*/config
bind-mounts still carry them, harmless (CharacterSchema.mood is dict[str,Any]).
Tracked as an opportunistic edit-only/no-restart cleanup to restore byte-identity;
noted new optional mood.tau_base (unset->derived). Config-delta acked to worldtree-dev.
Records Vuong's 2026-07-02 call closing the granite-efficacy thread: no
intermediate real-efficacy granite spike (uninterpretable proxy — arch
gap + abliteration axis), efficacy validated on the real T1 run. Notes
the LitBench-less data/judge WIRING check as the correct pre-T1 de-risk
IF one is ever wanted, and that infra's remaining owed item is the queued
swappable-LoRA-on-NVFP4 load test (gated on the first T1 adapter).
The granite-8b harness spike proved the TRL SFT->DPO->eval seam (incl. the
in-loop HoldoutEvaluator base-vs-adapter leg) runs end-to-end, but used a
12-row/12-pair synthetic writing fixture — NOT the E-RP corpus — so the
~0 anti-slop delta (-0.002) is the expected null, not an efficacy signal.
Adapter reaped; nothing to A/B. Real behaviour-shift efficacy is a T1-run
question. Sharpen both the T1 in-flight bullet and the Recent-decisions
entry so 'green' no longer reads as efficacy-validated.
Displaced qwopus-122B on ana-ml2 GPU0:8013 with robbatt/Qwen3.6-40B-Deckard-NVFP4
(stock vLLM 0.23.0, loaded clean: hybrid attn + multimodal + fp4_gemm all green).
Repointed the 5 role aliases (gen, gen-reasoning, qwen-large, qwen-large-reasoning,
summarizer-large); added the qwen3.6-40b-deckard true-name record; left the true
names qwen3.5-122-a10b[-reasoning] to 404 (no-false-alias). Operator-directed
trial-by-fleet-traffic; revert path in the config banner + live backup
config.yaml.bak-pre-deckard-20260701-001036.
Add §9 "PFI LiteLLM Gateway — Deployed Sampling Defaults": the live fleet
sampling table (granite/qwen/judges/GLM) with provenance, overrideable-default
semantics, the GLM API-accepted-subset caveat, and the research-confirmed temp-0
rationale for granite + image-judge. Accepts the dvalin-smithy-dev recommendations
as deployed. §§1-8 vendor reference left intact.
dvalin evidence pass: IBM canonical is temp 0; greedy-loop risk is an
open-ended-generation phenomenon, not summ/classify; temp 0.1 reduces
classification reproducibility without fixing loops (use repetition/presence
penalty if loops appear). image-judge stays 0 (Qwen judge card + W&B judge
practice = temp 0 for reproducibility; NVFP4-needs-0.1 unsupported). Both
gateway temps now 0, vendor-canonical.
Operator call: avoid pure-greedy rigidity/loop-risk on granite summ/classify
while staying near-deterministic; matches the house nonzero-temp-floor lean.
image-judge held at temp 0 (scoring reproducibility) pending operator review.
- granite-4.1-8b (+ summarizer/classifier): temperature 0 (IBM vendor-canonical
"temp 0 for inferencing"; top_p/top_k no-ops at temp 0, omitted). Deterministic
baseline for summ/classify; creative callers override.
- GLM family (z.ai cloud): temperature + top_p 0.95 only (the ONLY params z.ai
chat API accepts per its OpenAPI schema; top_k/min_p/penalties absent -> not set).
temp 1.0 for glm-5.1/5.2/5-turbo/4.7 + gen-frontier; temp 0.6 for glm-4.5-air.
Matches z.ai API defaults -> explicit-over-implicit, future-proofs vs vendor drift.
Round-2 dvalin-researched (provenance-labeled), verified live, granite+glm smoked 200.
Embeddings/rerankers excluded (no sampling). Fleet-wide canonical-defaults sweep complete.
The abliterated/NVFP4 Qwopus 122B "gen" model (+ qwen-large / summarizer-large
aliases) had no repetition control in its sampling defaults, causing degenerate
repetition loops. Add presence_penalty: 1.0 (Qwen-documented anti-repetition
lever, range 0-2) to all 7 qwen3.5-122-a10b gateway records. Overrideable
default; bake into the vLLM serving def once the value is validated.
Native-allocator expandable segments to cut Qwen-Image-Edit fragmentation
OOMs on the A6000 (a ~2 GB alloc failing with 1.75 GB free while 45 GB sat
allocated + reserved-but-unallocated). Cache-preserving — packs better
without unloading the checkpoint, so no edit-latency hit. Paired with the
existing --disable-cuda-malloc (incompatible with cudaMallocAsync).
Deployed + recreated on irv-ml1; verified env present, PyTorch reads it,
container healthy. comfy-dev request 2026-06-25.
Langfuse's ClickHouse member spewed ~94 GB of unrotated logs and filled ana-docker's
root disk (took the fleet host to 100%, 28/48 containers unhealthy). Its trace UI was
redundant with LiteLLM's native logging — store_prompts_in_spend_logs:true already
captures full prompts/responses/tokens/cost/latency at :4000/ui — and nothing used its
unique trace-grouping/eval features (it only received flat gateway success_callbacks).
Removed the callbacks (gateway observability stays fully native) and tore down the
6-container langfuse stack + volumes on ana-docker. Re-add the callbacks if it returns.
vh