vh
5f79b40982
feat(char-rp-reasoning): Deckard-PKD (Qwen3.5) replaces RpR-v4 after autonomous A/B
...
Operator wanted a reasoning-RP model that tolerates DRY (RpR-v4 forbids rep/DRY -> a
1/30 loop tail). Ran the full A/B on brokkr's 30-prompt D1 suite (content-only, slop-scored):
- Deckard-PKD (Qwen3.5-27B, DavidAU creative tune) WON: 0/30 loops, 0/30 refusals, clean
managed reasoning (native Qwen3.5 <think>/enable_thinking), DRY-tolerant, ~57 tok/s,
runs on the base llama-swap b8840 image. -> now the char-rp-reasoning seat (:8018).
- RpR-v4: 0 refusals but 1/30 loop (no-DRY). Pantheon-27B: clean slop but 7/30 explicit
refusals + needs the newer ggml-org/llama.cpp image (Qwen3.6 won't load on b8840).
Snowdrop + Gembrain (Gemma-4): floored (llama.cpp can't manage their reasoning without
the vetoed template hacks). Losers kept on disk as alternates.
- char-rp (Magidonia) unchanged; gen unchanged. gateway char-rp-reasoning -> Deckard
sampler (temp 1.0/top_p 0.95/top_k 40/min_p 0.05; DRY server-side).
2026-07-08 15:07:04 -07:00
vh
f49c4e40a3
config(char-rp-gguf): char-rp to full 128K, funded by gen util 0.40->0.37
...
Completes the GPU0 rebalance discussed with the operator:
- gen util 0.40 -> 0.37 (qwen36-27b-aeon stack .env) — frees ~2.9G of gen's IDLE KV
headroom (gen KV usage runs 0-2%; concurrency-at-256K 4.74x -> 3.66x, invisible).
- char-rp (Magidonia) 96K -> 131072 (full native 128K), q8_0 KV ~10.9G.
- char-rp-reasoning unchanged at 40K (QwQ native max).
- GPU0 ~4.4G margin, all 3 seats healthy, verified live.
Deployed .env values already set on ana-ml2; this canonicalizes the intent + rationale.
2026-07-08 13:01:22 -07:00
vh
d085604825
memory: park dvalin post-live-ST-sessions sampler re-tune follow-up
...
dvalin-smithy offered a follow-up sampler pass for char-rp / char-rp-reasoning after
they accumulate real Worldtree/SillyTavern character-role traffic. Parked as a future
option (thread 01KX1DS6...) — nothing to tune until there's live-session data.
2026-07-08 11:19:24 -07:00
vh
aac4bcfa3e
feat(litellm): wire canonical sampler defaults for all 4 gateway seats
...
dvalin-smithy canonical set, infra-ops triaged + char-rp A/B-validated on the live serve.
- gen (+summarizer-large twin): presence_penalty 1.0 -> 1.5 (Qwen3.6 non-thinking rec).
- gen-reasoning: temp 0.6 -> 1.0, presence 1.0 -> 1.5 (Qwen general-thinking profile;
the old 0.6 was the coding sub-profile).
- char-rp: temp 1.0 -> 1.1, min_p 0.03 -> 0.10, top_k 0, NO rep. A/B on 2 dark-romantasy
prompts: min_p 0.10 richened imagery; repeat_penalty 1.05 REJECTED (injected a stray
markdown title, hurts Drummer/Magistral RP creativity per the card + dvalin's own note).
- char-rp-reasoning: add explicit top_p 0.95 (else per the RpR card: no rep/DRY/XTC).
Canonical reference: docs/pfi/model-sampler-defaults.md (mirrors dvalin's derivation).
2026-07-08 11:10:28 -07:00
vh
f5706046b1
feat(char-rp-gguf): max context — char-rp 96K, char-rp-reasoning 40K, q8_0 KV
...
Raise both RP seats to near-max context within the GPU0 budget using q8_0 KV cache
(near-lossless 8-bit, ~2x context/GB, flash-attn-backed). Verified coherent on both
(no Qwen KV-quant gibberish) at 64/50 tok/s.
- char-rp (Magidonia): 16K -> 96K (native 128K; 128K would starve the reasoning seat).
- char-rp-reasoning (QwQ): 16K -> 40960 (QwQ native max; beyond needs YaRN).
- kv_unified=true -> a single conversation gets the full n_ctx (slots share the pool).
- GPU0 ~93/97G, ~4.3G margin (gen fixed-util + static KV = stable, no OOM risk).
- New .env knobs: CHARRP_CTX / CHARRP_REASONING_CTX / CHARRP_KV_TYPE / CHARRP_REASONING_KV_TYPE.
2026-07-08 07:29:49 -07:00
vh
b268f93035
feat(char-rp-gguf): replace broken Angel NVFP4 with dual GGUF RP seat on ana-ml2 GPU0
...
char-rp -> TheDrummer Magidonia-24B-v4.3 Q6_K (Magistral prose, ~65 tok/s,
zero refusal, tight POV) via llama.cpp (:8016).
char-rp-reasoning -> ArliAI QwQ-32B-RpR-v4 Q5_K_M (abliterated managed reasoning,
~52 tok/s, reasoning surfaces in reasoning_content) via llama.cpp (:8018).
- New canonical stack stacks/char-rp-gguf/ (llama-server x2, GPU0-pinned, ~86/97G
co-resident with gen). GGUF sidesteps the vLLM-NVFP4 + Mistral-tokenizer traps that
killed the Angel serve. Never Ollama.
- Best-of-breed per seat: no single dense 24-32B is both an elite non-thinking prose
seat AND a clean managed-reasoning seat on llama.cpp (Magidonia [THINK] boundary is
loose; Cydonia-R1 <think> runs away; QwQ is template-managed). Pantheon-Reasoning-27B
stays rejected (re-censors in <think>; RpR-v4 abliterated reasoning is the fix).
- Gateway rewired: char-rp->:8016, char-rp-reasoning->:8018, Mistral/QwQ samplers,
dropped the Qwen enable_thinking kwarg. One-model Magidonia fallback documented.
- Retired the ms32-24b-angel stack.
2026-07-08 02:43:00 -07:00
vh
75851c2837
memory: snapshot — T1 SFT done (AEON-27B, adapter banked) + hot-swap BLOCKED (vLLM qwen3_5 LoRA no-op #47639 ), SGLang last-shot = smoke sft_adapter_zc; comfy-dev LoRA worker done (Phases 1/2/2.5)
2026-07-07 16:02:45 -07:00
vh
5c64d31094
memory: snapshot — AEON Qwen3.6-27B is now gen (qwopus displaced), litbench torn down/comfyui restored
...
- gen := AEON dual NVFP4 serves (vLLM 0.24 + LiteLLM v1.91.0); reasoning-trace bug
was the LiteLLM shared-config mutation, fixed durably via distinct -thinking served-names.
- Worldtree personal character/thoughtful-character repointed to char-rp/char-rp-reasoning.
- LitBench-RM torn down, comfyui restored on irv-ml1.
2026-07-06 00:57:01 -07:00
vh
993decf3eb
memory: snapshot — 2026-07-05 (T1 train venue = cloud-rec/operator-chose-ana-ml2-smoke; LitBench-RM up + comfyui displaced; character-rp shipped + #344 ; althing v2 herald/receiver systemd + PATH fix; glm-5.2 1M/128K; LiteLLM shared-param-mutation footgun; condensed R22 + several Recent-decisions entries)
2026-07-05 16:14:37 -07:00
vh
3c966b2631
memory: track low-pri cleanup of inert mood.decay_rate/stale_hours from deployed WT bind-mounts
...
R30 b15-b17 removed mood.decay_rate/mood.stale_hours from canonical (OCEAN
wall-clock OU replaced per-turn decay); deployed /opt/worldtree*/config
bind-mounts still carry them, harmless (CharacterSchema.mood is dict[str,Any]).
Tracked as an opportunistic edit-only/no-restart cleanup to restore byte-identity;
noted new optional mood.tau_base (unset->derived). Config-delta acked to worldtree-dev.
2026-07-03 22:14:58 -07:00
vh
3a627c6c26
memory: operator decided DEFER granite efficacy to the T1 run (no intermediate spike)
...
Records Vuong's 2026-07-02 call closing the granite-efficacy thread: no
intermediate real-efficacy granite spike (uninterpretable proxy — arch
gap + abliteration axis), efficacy validated on the real T1 run. Notes
the LitBench-less data/judge WIRING check as the correct pre-T1 de-risk
IF one is ever wanted, and that infra's remaining owed item is the queued
swappable-LoRA-on-NVFP4 load test (gated on the first T1 adapter).
2026-07-02 10:33:15 -07:00
vh
7fdda2de53
memory: granite spike = mechanical-green ONLY, efficacy not validated by design (mtf-dev confirm)
...
The granite-8b harness spike proved the TRL SFT->DPO->eval seam (incl. the
in-loop HoldoutEvaluator base-vs-adapter leg) runs end-to-end, but used a
12-row/12-pair synthetic writing fixture — NOT the E-RP corpus — so the
~0 anti-slop delta (-0.002) is the expected null, not an efficacy signal.
Adapter reaped; nothing to A/B. Real behaviour-shift efficacy is a T1-run
question. Sharpen both the T1 in-flight bullet and the Recent-decisions
entry so 'green' no longer reads as efficacy-validated.
2026-07-02 10:25:46 -07:00
vh
5b52673b75
memory: snapshot — 2026-07-02 (Deckard trial→revert to qwopus; MTP concurrency verdict = not-kept-on-shared-gen; worldtree #332 diagnosis + scoped-view/tunnel + CI-race lesson; mtf-dev granite harness spike; /books ESH mount)
2026-07-02 08:26:35 -07:00
vh
30c883be5d
memory: snapshot — 2026-06-25 (althing v0.17 / nh3-extdev model-B mesh + zellij web pilot + Worldtree #314/#322/#317 config arc; rewrite stale althing Tools-row to v0.17; archive 10 pre-session 2026-06-20 entries)
2026-06-25 00:14:02 -07:00
vh
13bfa4a621
memory: snapshot — 2026-06-21 (compress in-flight to current; archive 21 pre-session entries to archival-memory.md)
2026-06-21 00:07:57 -07:00
vh
db2953e690
memory: R22 Phase B CANCELLED (Worldtree model-agnostic → no deploy path); gateway-only
2026-06-20 22:04:37 -07:00
vh
245b217372
memory: R22 key full-open confirmed by operator (settled)
2026-06-20 21:59:40 -07:00
vh
3cb54efd59
memory: R22 key re-minted persistent (stateless consumer); old orphan revoked
2026-06-20 21:55:00 -07:00
vh
b33049ce3e
memory: R22 operator steer — research gated on pragmatic/deployable outcome, not art
2026-06-20 21:52:17 -07:00
vh
9d65339fb2
memory: R22 stood down to gateway, full-access key minted, Phase B parked; phantom qwen3.6 entry to clean
2026-06-20 21:49:48 -07:00
vh
d9ebe8d0f4
memory: old claude-bot token (id 15) revoked — worldtree-dev fully self-serve on one token
2026-06-20 17:27:47 -07:00
vh
6430c01dad
memory: claude-bot issue-scope token (id 16) minted for worldtree-dev self-serve
2026-06-20 17:25:54 -07:00
vh
ec671e86c5
memory: cb2a79a readonly-admin allow-rules re-staged to demo+personal (PDP is rule-based)
2026-06-20 16:43:44 -07:00
vh
b13f66aea9
memory: ratatoskr flipped to readonly — re-staged 439bebf policies.yaml to personal + safe restart reload
2026-06-20 16:29:15 -07:00
vh
2e993ac3df
memory: ratatoskr resolved (operator chose admin; worldtree-dev self-served key 90db1fbd)
2026-06-20 16:24:21 -07:00
vh
5a3a75b73d
docs(backups): harden + live-activate /mnt/compose automount on ana-docker
2026-06-20 16:21:05 -07:00
vh
76b317ce3e
feat(backups): freshness check + daily alert timer; record rest-server-ana recovery, fstab hardening, esh-pve-nas gap, worldtree admin-key provisioning
2026-06-20 16:13:13 -07:00
vh
a7b4a82dec
docs(backups): add backup architecture + freshness runbook; record rest-server-ana recovery + correct ana-docker sudo path
2026-06-20 15:56:41 -07:00
vh
8f15f6bb0d
memory: capture 2026-06-20 session — Worldtree v0.37.7 demo fix, gitea notifier recovery, backup diagnosis
2026-06-20 15:46:14 -07:00
vh
58ec80d58a
memory: snapshot — 2026-06-20
...
WT capability-gateway migration (aliases + swap-transparency rule + wildcard
removal + gen-frontier + v0.37.5 staged/triggered), OmniVoice streaming /tts
(diffusion TTFA floor), arbo v0.14.9, ratatoskr :8392 + admin.events.read
brokered, claude-bot admin on vh/Worldtree. Disk incident root-caused to a
94GB unrotated langfuse-clickhouse log (+ secondary image bloat) → fixed,
image/buildx prune cron added, Langfuse retired. Backup gap: rest-server-ana
is the ACTIVE ana-side restic target down ~months on a failed ana-nas NFS
mount (NOT decommissioned — docs corrected) → recovery deferred to clean
context, along with the docker-daemon log cap. Archived the [2026-06-16]
cluster (13 Recent + 8 Tried) to archival-memory.md.
2026-06-20 13:36:54 -07:00
vh
826c2a6a64
memory: archive 15 pre-2026-06-16 entries to archival-memory.md
...
9 Recent decisions + 6 Tried-and-abandoned (dates [2026-06-14]/[2026-06-15]) moved
non-destructively to archival-memory.md, each stamped _Archived 2026-06-19._. Kept
the active [2026-06-14] 'migrate ALL infra access to Claude-specific credentials'
standing directive. Back-ref counts: Recent decisions 79->88, Tried-and-abandoned
70->76. persistent-memory.md 397->363 lines.
2026-06-19 21:54:45 -07:00
vh
dfda60fac7
memory: snapshot — 2026-06-19 (pt2) litellm task-aliases (classifier->granite, summarizer-large->gen; gen-nt/gen-reasoning-nt added-then-removed as redundant with strip_empty_tools) + gateway-chat model-smoking web chat enhanced (auto-discover /v1/models + image upload) and stood up as a PERSISTENT nginx container on ana-docker :8091 + pi on nh3-dev wired to gen (vision, ~/.pi models.json + gen launcher, local box config) + foot-guns: litellm config-loaded models can't be hot-removed (/model/delete is DB-only; /model/new live-adds work no-bounce but dup on restart) and the * wildcard routes stale/typo'd names to decommissioned llama-swap -> misleading 'Connection error' not 'model not found' (bit a brokkr call to the renamed-away qwen-image-judge).
2026-06-19 18:51:03 -07:00
vh
378261763c
memory: snapshot — 2026-06-19 gen model = Qwopus3.5-122B vision-intact NVFP4 LIVE on ana-ml2 GPU 0 (full 256K @ fp8 KV + CUDA graphs, util 0.95 + expandable_segments, 92.7 tok/s warm, 3.32x concurrency, text+image+video, tool-calling qwen3_coder; nightly+turboquant-4bit-KV proven UNNECESSARY — stable fp8 reaches 256K) replacing the bjk110 text-only qwen3.5-122b (which displaced mistral-small-4 → Worldtree character backend DARK until repointed, operator-acknowledged) + qwen-image-bench T2I judge replaced qwen3.6-35b-a3b on GPU 1 (alias image-judge) + TP=2 across both Blackwells REJECTED (PCIe-only PIX, no NVLink → all-reduce-bound, one-model-per-card is optimal; PP=2 only if a >96GB model is ever wanted) + foot-guns: MoE FusedMoE workspace is the ~3.1GB un-budgeted floor (can't fill to 0), discard cold tok/s reads (24.8 cold vs 92.7 warm).
2026-06-19 11:15:32 -07:00
vh
632124c8fb
memory: nh3-extdev pi-on-GLM-5.2 wired (mark in-flight item done + residual gates)
2026-06-18 14:05:02 -07:00
vh
a67d4950d0
memory: snapshot — 2026-06-18 heretic abliterated Mistral Small 4 NVFP4 BUILT + LIVE as mistral-small-4 (in-house quant device_map=cpu → native-format convert → drop-in stack under same served-name, A/B'd vs official, operator "heretic stays"; byte-equivalent to official NVFP4) + irv-ml1 VRAM consolidation (ComfyUI pinned to A6000 exclusive/48GB, audio zoo→3090, downed dia/ace-step/csm, comfy-dev torch-pin DISABLE_UPGRADES@2.12.1 + SageAttention rebuilt) + ComfyUI 9-node accel set installed for comfy-dev + ana-ml2 durable vm.overcommit_memory=1 + GLM5.2 wired + nh3-extdev sudo-less manager box + /opt/externs pi-on-GLM client workspaces.
...
Lessons: mmartial-comfyui root-install-leaves-root-owned-venv-files → boot-script crash-loop (chown -R 1000:1000 fix) + torch-upgrade-on-boot (DISABLE_UPGRADES); mistral HF→NVFP4 quant device_map=cpu (auto OOMs, constrained→meta-tensor) + non-mmap shard reads (safe_open mmap ENOMEMs on /tank ZFS) + NVFP4 keeps the model. prefix; HF-format Mistral4 UNSERVEABLE on vLLM (native mandatory); ComfyUI 0.24.1-not-0.19.3 version-drift kills module-level node imports + tensorrt-defaults-cu13-vs-cu12.9.
Archived the [2026-06-14] cluster (11 entries: 6 decisions + 5 foot-guns; kept the still-active credential-migration directive, infra-ops litellm key, gitea-internal-route).
2026-06-18 13:48:12 -07:00
vh
8cca365b78
memory: correct gitea action-log API note (per-job endpoint works)
...
Proven 2026-06-16 diagnosing arbo run #5/task 1175: the RUN-level
/runs/{id}/logs 404s, but the per-JOB endpoint
GET /api/v1/repos/{o}/{r}/actions/jobs/{job_id}/logs returns the full
plain-text log (claude-bot basic-auth, internal :3000) — no UI needed.
Also noted gitea's misleading per-step conclusions (every step shows
failure once any fails; trust the log + timestamps).
2026-06-16 14:56:12 -07:00
vh
47c30e85f9
memory: snapshot — 2026-06-16 litellm strip_empty_tools hook ( d1bea13) + single-file gateway-chat.html playground ( 984ca3d) + claude-bot ADMIN on vh/arbo (arbo CI/CD via service account) + LitBench-RM reward judge served on irv-ml1 A6000 then taken to on-demand (held comfyui's slot) + #295 recall root-cause FLIPPED (score_breakdown-shape DISPROVEN → cold-recall agent_self scope axis vs ratatoskr conjunctive INV-005; Worldtree #297 ); lessons: litellm-500-Router.acompletion-missing-messages = a request missing Content-Type (NOT a gateway outage — cost 4 needless restarts), litellm-admin-UI-playground-cant-test-vLLM ( #6228 empty-tools, proxy-hook-cant-reach-in-process-call), gitea-run-looks-like-never-fired-but-fired-then-skipped/failed-fast (check run list not runner). Archived the 06-09→06-13 cluster (24 entries: 17 decisions + 7 foot-guns).
2026-06-16 14:14:36 -07:00
vh
f9277f5440
memory: snapshot — 2026-06-16 ratatoskr Tier-3 MEMORY plane wired (allowlist :8391, key reused, persist+dispatch GREEN; recall-injection root-caused to the score_breakdown shape seam → worldtree-dev #295 ) + infra-ops durable admin on corviduo (ssh alias + ssh-target) + demo/personal character model qwen→mistral (first-bind-is-default reorder, pin-safe recreate); lessons: bifrost-allowlist-is-per-port, promotion-gate=consumer-agent-memory-block-not-agent_self_enabled, WORLDTREE_IMAGE-pin-from-matrix-sibling
2026-06-16 00:13:14 -07:00
vh
aeea377749
memory: snapshot — 2026-06-16 ana-ml2 dual-NVFP4 reshape (GPU0 Mistral Small 4 256K/v0.22.0-vision + GPU1 qwen36 FP8→NVFP4 + Selene FP8 judge + GPU1 grows) + NVFP4-MoE-loads-on-0.23.0 (supersedes blocked) + claude-bot service account (corviduo-org tabled) + arbo→comfy-dev ownership + gitea runner on irv-ml1 + Worldtree demo/personal capability-profile migration (pre-sync-first); lessons: vLLM-0.23-breaks-Mistral-vision ( #44911 ), Mistral-TTFT=Triton-JIT-spikes, vh-user-not-org blocks scoped package-write, old-baseline-instances-need-full-config-set; archived the 2026-06-05/08 cluster (11 entries)
2026-06-15 22:26:11 -07:00
vh
0943d145fb
memory: snapshot — 2026-06-15 (cont.) arbo v0.11.22 engine rebuild + catalog v0.11.23 (curated /workflows footer live) + althing-core v0.14.1 box-wide refresh (monitor lock fix) + comfyui VAE-decode SEGFAULT diagnosis (aimdo 0.4.8 cuda-hooks vs torch cu129/cu130 mismatch, NOT OOM); lessons: comfyui-segfault-not-OOM diagnostic, never blanket-kill peer light-monitors
2026-06-15 13:35:50 -07:00
vh
12bcd06442
memory: snapshot — 2026-06-15 ratatoskr affect smoke GREEN (Heimdall key mint+inject, allowlist, handshake+emit) + infra-ops bootstrapped on corviduo + dense Qwen3-VL-32B-NVFP4 judge A/B (lost, torn down) + MastMed cloudflared public + R18 clip+caption staged (stub smoke passed; real-voice gate pending) + LiteLLM infra-ops key; lessons: corviduo stale-:latest recreate crash, .claude.json ENOSPC repair, pkill self-match
2026-06-15 00:17:44 -07:00
vh
10f346b39e
memory: snapshot — 2026-06-14 FP8 vision cutover (qwen35-vl→qwen36-vl, truthful naming, GPU-1 rebalance) + llama-swap pin drop + R16 yield probe executed + standing credential-migration directive; NVFP4-on-vLLM-blocked + sampler-warmup/profiling-race/embed-rerank-waste lessons
2026-06-14 14:47:01 -07:00
vh
b45d0cd86d
memory: snapshot — 2026-06-14 arbo auth-off + deploy-pipeline fix (v0.11.6, internal gitea route, catalog-only restart, scripts tracked) + storetank archive decommission (919G -> arbo 502G) + R16 inline arc closed (v1 final); archive 11 (2026-06-04 cluster)
2026-06-14 08:56:12 -07:00
vh
0fc9083d16
memory: snapshot — 2026-06-13 ana-ml2 Ada→Blackwell + NVFP4-infeasible + Qwen3.5-VL FP8 + comfyui→arbo + GPU-1 rebalance + prefix-caching; archive 8 (2026-06-03 cluster)
2026-06-13 13:30:19 -07:00
vh
005effd664
memory: snapshot — 2026-06-11 Mac Pro migration framing + GLM thinking-off + R17 v2 corpus; archive TTS-streaming arc
2026-06-11 08:27:06 -07:00
vh
5349da567c
memory: snapshot — 2026-06-05 granite-FP8 cutover + Langfuse observability + worldtree #254 deferral
2026-06-05 14:14:44 -07:00
vh
1dba05d8c8
memory: snapshot — 2026-06-04 infra session (phi4/vLLM, infra-ops identity, R15/R16, brokkr svc)
...
Captures: phi4-mini FP8 on ana-ml2 vLLM as nevermore summarizer (granite retired,
canonical Phi-4 template after apply→revert); infra-ops NOPASSWD-sudo identity across
PFI boxes; R15/R16 brokkr-smithy stand-up + ollama 0.30.4 + A6000 UUID-pin; brokkr user
+ substrate migration + audition service; homepage incident; observability roadmap.
Foot-guns: headgun chat-template lesson, UUID GPU-pin, system-unit reaping, pyworld/
gfortran, homepage D-state. Archived 11 settled 2026-06-01 TTS-arc entries (Recent 5 +
Tried 6) to archival-memory.md; pruned 145 lines of DONE 2026-06-03 in-flight bulk.
File back under the 300-line cap (299).
2026-06-04 01:08:50 -07:00
vh
9090fb59d7
memory: archive 13 settled 2026-05-30/31 entries (Recent 7 + Tried 6) to archival-memory.md
2026-06-03 10:49:09 -07:00
vh
23f2b63884
memory: snapshot — 2026-06-03 infra session (NH3 egress proxy, R14 provisioning, yt-voice-clipper push-to-deploy, gitea allowlist, irv-ml1 disk)
2026-06-03 10:38:47 -07:00
vh
fa56718ba0
memory: shelve fish-streaming (sub-realtime 0.80x); standing fp4+Blackwell revisit trigger
2026-06-02 08:59:17 -07:00