# ⚠️ CUSTOM llama.cpp build for the char-rp-reasoning seat

> **STATUS (2026-07-13):** image `llamacpp-charrp:6eddde0-pr25544` (+ `:custom-latest`)
> BUILT on ana-ml2 (reports `version: 10001 (6ce128d0b)`, CUDA 12.8.1, sm_120) and
> binary-smoke-tested OK. **DEPLOYED 2026-07-13** — the live `char-rp-gguf`
> reasoning seat runs `llamacpp-charrp:custom-latest` (via `LLAMA_REASONING_IMAGE`);
> Deckard loads + serves coherent output + reasoning bounds at the 400 budget.
> Definitive tool-path proof = production monitoring (spend_logs `completion_tokens=32768`
> runaways should stop; baseline was ~13%/~4 today pre-swap). b61's 300s wall-clock backstops meanwhile.

**The `llama-charrp-reasoning` seat (Deckard-PKD, ana-ml2:8018) runs a
CUSTOM-BUILT llama.cpp, NOT a stock upstream image.** If you are upgrading,
rebuilding, or "cleaning up" the seat, read this first — a naive bump to a
stock image will silently reintroduce Worldtree #355.

## What it is

`llamacpp-charrp:custom-latest` = **llama.cpp master `6eddde0`** (2026-07-13,
~b9990 era) **+ unmerged upstream PR [#25544]** (`aldehir:reasoning-budget-multi-seq`),
built for Blackwell (sm_120) via `build.sh` in this directory.

Merged tree HEAD (on ana-ml2 `/home/lkraven/llamacpp-build`): `6ce128d0b`.

[#25544]: https://github.com/ggml-org/llama.cpp/pull/25544

## Why it exists (Worldtree #355 root cause)

Stock llama.cpp **b8840** (the previous seat build, `ghcr.io/mostlygeek/llama-swap:cuda`)
— and every *released* build to date — has a reasoning-budget sampler that
recognises only **one** end-of-thinking tag, `</think>`. But Qwen3.5's
tool-call path ends its reasoning block with **`<tool_call>`**, a terminator
the single-tag sampler can't match. So on the agentic tool-retry path,
`--reasoning-budget 400`'s forced-close **never fires**, and the model loops
in reasoning all the way to `max_tokens` (32768) ≈ **22 min per turn** (~13%
of char-rp-reasoning turns). That is #355's residual hang.

Evidence that pinned it (all three agree): LiteLLM spend_logs (4 requests at
exactly `completion_tokens=32768`), the seat's own log (`158 activated / 0
forced-close / 110 natural-end`), and the WT↔gateway pcap (runaway stream is
100 % `reasoning_content`, coherent tool-retry planning, never exits `<think>`).

**PR #25544 is the fix**: it extends the budget sampler to **multiple
terminating sequences** (`</think>` *or* `<tool_call>`) and replays the matched
end-sequence when the budget expires. It is **OPEN / unmerged** upstream
(the underlying trigger #22684 is closed-as-not-planned; the tool-parsing PR
#24202 is also open), so there is **no released build that fixes our bug** —
hence this local build.

## Conflict resolution (the one thing to re-verify on any rebump)

The PR (based on master of 2026-07-10) is 25 commits behind `6eddde0`; the
merge has a single conflict in `tools/server/server-common.cpp`. Resolution
(baked into `build.sh`): keep the **PR's plural `reasoning_budget_end_tags`**
(the fix) **and master's per-request body-read of `reasoning_budget_message`**
(a newer master feature). Both improvements retained.

## Build / rebuild

```bash
# on ana-ml2 (has the Blackwell GPUs + docker + disk):
scp stacks/char-rp-gguf/llamacpp-custom/build.sh ana-ml2:/home/lkraven/
ssh ana-ml2 'bash /home/lkraven/build-charrp.sh'   # ~20-40 min, sm_120 compile
```

Then deploy via the seat compose (`stacks/char-rp-gguf/compose.yaml`, service
`llama-charrp-reasoning`), whose `image:` is pinned to `llamacpp-charrp:custom-latest`.

## Acceptance test before trusting a build (do NOT skip)

The bug is specifically that forcing *never fires*. A build is only good if it
demonstrably **force-closes** — not just "loads + serves":

1. `tests/test-reasoning-budget.cpp` passes (`ctest -R reasoning-budget`).
2. Live low-budget check on Deckard: launch with `--reasoning-budget 20`, send a
   tools-bearing request that induces a tool-retry loop, and confirm the seat
   logs a **forced** close (budget-reached), not just `deactivated (natural end)`,
   and the response stops near the budget instead of running to `max_tokens`.

Then restore `--reasoning-budget 400` for production.

## Rollback

The previous stock image `ghcr.io/mostlygeek/llama-swap:cuda` (build 8840) is
preserved on ana-ml2 (imageID `d6c39f55…`). To roll back: point the seat's
`image:` back to it and recreate. 8840 works (RP is fine) — it only has the
forcing bug.

## 🔔 Retiring this custom build (REMOVE-WHEN-MERGED)

This is a temporary local build carrying an unmerged PR. **When PR #25544
merges upstream and lands in a stock release:**

1. Confirm the released build's `tests/test-reasoning-budget.cpp` covers the
   `<tool_call>` terminator and passes on Qwen3.5.
2. Switch the seat `image:` to that stock image (e.g. a
   `ghcr.io/ggml-org/llama.cpp:server-cuda` tag ≥ the merge) and run the
   acceptance test above.
3. Delete this directory + the compose `⚠️ CUSTOM BUILD` note, and update
   `persistent-memory.md` + the `reference_charrp_custom_llamacpp_pr25544`
   auto-memory.

Check status: <https://github.com/ggml-org/llama.cpp/pull/25544>.