vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

config(char-rp-gguf): char-rp to full 128K, funded by gen util 0.40->0.37

Browse Source 
Completes the GPU0 rebalance discussed with the operator:
- gen util 0.40 -> 0.37 (qwen36-27b-aeon stack .env) — frees ~2.9G of gen's IDLE KV
  headroom (gen KV usage runs 0-2%; concurrency-at-256K 4.74x -> 3.66x, invisible).
- char-rp (Magidonia) 96K -> 131072 (full native 128K), q8_0 KV ~10.9G.
- char-rp-reasoning unchanged at 40K (QwQ native max).
- GPU0 ~4.4G margin, all 3 seats healthy, verified live.

Deployed .env values already set on ana-ml2; this canonicalizes the intent + rationale.
This commit is contained in:
vh
2026-07-08 13:01:22 -07:00
parent d085604825
commit f49c4e40a3
2 changed files with 12 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
stacks/char-rp-gguf/.env.example
+5 -3
View File
@@ -24,9 +24,11 @@ CHARRP_PORT=8016
CHARRP_MODEL=rp/TheDrummer_Magidonia-24B-v4.3-Q6_K.gguf
# Alternate prose model (more literary flair, looser POV adherence) — uncomment to swap:
# CHARRP_MODEL=rp/MS3.2-PaintedFantasy-v4.1-24B-Q6_K.gguf
# Context. Magidonia native = 131072 (128K). 96K @ q8_0 KV ~ 8.4G VRAM; f16 KV doubles
# that. Raise toward 131072 only if you drop the reasoning seat's ctx (shared GPU-0 budget).
CHARRP_CTX=98304
# Context. Magidonia native = 131072 (128K) — now at FULL 128K. 128K @ q8_0 KV ~ 10.9G VRAM.
# Funded 2026-07-08 by dropping gen's util 0.40->0.37 (AEON_GEN_GPU_MEM_UTIL in the
# qwen36-27b-aeon stack), which freed ~2.9G of gen's IDLE KV headroom (gen KV usage runs
# 0-2%). GPU0 ~4.4G margin at: char-rp 128K + char-rp-reasoning 40K + gen@0.37, all q8_0 KV.
CHARRP_CTX=131072
# KV cache dtype: q8_0 (~half VRAM, near-lossless) or f16 (max fidelity, 2x VRAM).
CHARRP_KV_TYPE=q8_0