vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(char-rp-gguf): swap reasoning seat to custom llama.cpp (master 6eddde0 + PR #25544)

Browse Source 
The char-rp-reasoning (Deckard) seat now runs llamacpp-charrp:custom-latest via a
new LLAMA_REASONING_IMAGE var (Magidonia char-rp stays on stock — no reasoning bug).
Fixes Worldtree #355 at the source (budget multi-terminator handles Qwen3.5's
<tool_call> reasoning end-tag). Live 2026-07-13: Deckard loads on Blackwell, serves
coherent, reasoning bounds at the 400 budget. Rollback via .env LLAMA_REASONING_IMAGE.
This commit is contained in:
vh
2026-07-13 13:47:42 -07:00
parent 6cf3e78973
commit 85792f4b55
3 changed files with 13 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
persistent-memory.md
+1 -1
View File
File diff suppressed because one or more lines are too long
stacks/char-rp-gguf/compose.yaml
+7 -1
View File
@@ -96,7 +96,13 @@ services:
# ── REASONING seat — Deckard-PKD (Qwen3.5) managed thinking + DRY. gateway char-rp-reasoning. ── # ── REASONING seat — Deckard-PKD (Qwen3.5) managed thinking + DRY. gateway char-rp-reasoning. ──
llama-charrp-reasoning: llama-charrp-reasoning:
image: ${LLAMA_IMAGE:-ghcr.io/mostlygeek/llama-swap:cuda} # ⚠️ CUSTOM llama.cpp build (master 6eddde0 + unmerged PR #25544) — the Worldtree
# #355 reasoning-budget-forcing fix (multi-terminator: handles Qwen3.5's <tool_call>
# reasoning end-tag, which stock b8840's single-</think> budget can't match → the seat
# loops in reasoning to max_tokens ≈ 22 min). DO NOT revert to a stock image until
# #25544 merges upstream. Build recipe + why + rollback: ./llamacpp-custom/README.md.
# Rollback: set LLAMA_REASONING_IMAGE=ghcr.io/mostlygeek/llama-swap:cuda in .env + recreate.
image: ${LLAMA_REASONING_IMAGE:-llamacpp-charrp:custom-latest}
container_name: ${CHARRP_REASONING_CONTAINER:-llama-charrp-reasoning} container_name: ${CHARRP_REASONING_CONTAINER:-llama-charrp-reasoning}
restart: unless-stopped restart: unless-stopped
runtime: nvidia runtime: nvidia
stacks/char-rp-gguf/llamacpp-custom/README.md
+5 -3
View File
@@ -2,9 +2,11 @@
> **STATUS (2026-07-13):** image `llamacpp-charrp:6eddde0-pr25544` (+ `:custom-latest`) > **STATUS (2026-07-13):** image `llamacpp-charrp:6eddde0-pr25544` (+ `:custom-latest`)
> BUILT on ana-ml2 (reports `version: 10001 (6ce128d0b)`, CUDA 12.8.1, sm_120) and > BUILT on ana-ml2 (reports `version: 10001 (6ce128d0b)`, CUDA 12.8.1, sm_120) and
> binary-smoke-tested OK. **Seat swap PENDING** — the live `char-rp-gguf` compose > binary-smoke-tested OK. **DEPLOYED 2026-07-13** — the live `char-rp-gguf`
> still points the reasoning seat at stock 8840 until the swap + on-seat forced-close > reasoning seat runs `llamacpp-charrp:custom-latest` (via `LLAMA_REASONING_IMAGE`);
> validation. Once swapped, update this line to DEPLOYED. > Deckard loads + serves coherent output + reasoning bounds at the 400 budget.
> Definitive tool-path proof = production monitoring (spend_logs `completion_tokens=32768`
> runaways should stop; baseline was ~13%/~4 today pre-swap). b61's 300s wall-clock backstops meanwhile.
**The `llama-charrp-reasoning` seat (Deckard-PKD, ana-ml2:8018) runs a **The `llama-charrp-reasoning` seat (Deckard-PKD, ana-ml2:8018) runs a
CUSTOM-BUILT llama.cpp, NOT a stock upstream image.** If you are upgrading, CUSTOM-BUILT llama.cpp, NOT a stock upstream image.** If you are upgrading,