feat(gpu): relocate char-rp to GPU1 + re-optimize both ana-ml2 cards for max context

Operator-directed 2026-07-16. Moved the char-rp prose seat (Magidonia-24B,
llama-charrp) from GPU0 to GPU1 (CHARRP_GPU_ID 0->1; recreate llama-charrp
only -- the var is shared with the retired GGUF reasoning service), then
re-optimized every context-relevant seat on both cards to native/max context
with acceptable headroom:

  GPU0 (both seats now 256K native, ~14 GB reserve):
    - char-rp-reasoning 150K -> 256K  (heretic2 stack, util 0.38->0.46, 1.56x)
    - gen 256K, max-num-seqs 16 -> 32 (qwen36-27b-aeon, util 0.30->0.42, 5.43x)
  GPU1 (~6.7 GB headroom):
    - granite 64K -> 128K full-chapter (vllm stack, util 0.18->0.27, 1.50x)
    - char-rp 128K native (4 slots), selene/reward/embed/rerank unchanged

All seats gateway-verified healthy. Live .env changes on ana-ml2 with per-stack
backups (*-20260716). Templates updated to match; the qwen36-27b-aeon template
carries a NOTE that its served-name/model still lag the 2026-07-08 gen model swap
(35B-A3B-heretic) -- separate reconciliation. persistent-memory records the full
layout + the util-floor / per-model-KV-cost lessons.

Note: the heretic2-charrp-reasoning stack (char-rp-reasoning's live config) is
still untracked in git -- standing open-loop, its .env change lives server-side only.

persistent-memory.md

@@ -119,7 +119,7 @@ _As of 2026-07-15 — a long fleet-ops + infra session. Everything below either
 
 **PARKED (operator-directed holds):**
 - **image-bench (qwen-image-bench, GPU1) eviction — DONE 2026-07-15 (operator-directed, same-day as the switch).** Stopped `vllm-qwen-image-bench` on ana-ml2 (Exited(0), revert via `docker compose start`) → GPU1 freed ~32 GB (95→63 GB used). LiteLLM `image-judge` + `qwen-image-bench` aliases repointed to gen (:8015, qwen3.6-35b-a3b-heretic), deterministic judge sampling (temp0/top_k1/rep_pen1.05) + `enable_thinking:false` kept; verified live with :8014 down. comfy-dev pinged as regression watcher (NSFW-anatomy + tight-grounding; thread 01KXMRDJ). Backstop = the stopped stack (revert ~90 s) + live config backup `config.yaml.bak-pre-imagejudge-20260715`. Full record: auto-memory `project_arbo_gen_switch_imagebench_evict`.
-- **granite right-size — DONE 2026-07-16 (~10.5 GB freed).** util 0.34→0.18 + max-model-len 131072→65536 on `vllm-granite` (GPU1, `/opt/docker/compose/vllm` .env — recreate `vllm-granite` ONLY, it's a shared stack w/ embed/rerank/reward). GPU1 62,641→51,897 MiB (~45 GB free now); KV 6.45 GiB / 84,528 tok / 1.29x@65536, summarizer healthy. Effective slope ~950 MiB KV per 0.01 util + KV must hold ≥1× max-len; util 0.15 undershot (crash-loop, ~2-3 min summarizer outage) before 0.18 landed. Frees GPU1 to relocate a GPU0 model — operator's next call: which (gen :8015 / char-rp :8016 / char-rp-reasoning :8018). Backup `.env.bak-pre-granite-rightsize-20260716`.
+- **GPU re-org COMPLETE 2026-07-16 — char-rp relocated GPU0→GPU1 + both cards re-optimized for max context.** Full arc: image-bench evicted (GPU1 −32 G) → granite right-sized 0.34→0.18/64K → **char-rp** (Magidonia-24B, `llama-charrp`) moved to GPU1 (`CHARRP_GPU_ID` 0→1, ~29.9 G; recreate llama-charrp ONLY — shared var w/ retired GGUF reasoning svc) → then MAXED context: **char-rp-reasoning 150K→256K** (heretic2 stack, util 0.38→0.46 → 1.56x@256K), **gen 256K** (qwen36-27b-aeon stack — live model is 35B-A3B-heretic; util 0.30→0.42 + seqs 16→32 → 5.43x@256K), **granite 64K→128K** full-chapter (util 0.18→0.27 → 1.50x@128K). FINAL: **GPU0** 83.1/97.9 G (gen + char-rp-reasoning, both 256K native, ~14 G reserve — both native so no more ctx to add; reserve is for a future tenant), **GPU1** 90.5/97.9 G (char-rp 128K + granite 128K + selene 32K + reward/embed/rerank, ~6.7 G headroom). All seats healthy + gateway-verified. Backups (all `-20260716`): char-rp-gguf `.env.bak-pre-gpu1move`, heretic2 `.env.bak-preopt`, qwen36-aeon `.env.bak-preopt`, vllm `.env.bak-pre-granite-rightsize`. LESSON: KV pool must hold ≥1× max-model-len (util-floor → startup crash, e.g. granite 0.15); slope ~950 MiB/0.01 util but per-model KV cost varies ~8× (gen MoE ~99K tok/GiB, granite dense ~13K tok/GiB) so tune util empirically per seat off the `Maximum concurrency for N tokens` log line.
 
 **Open loops for the fresh session:**
 - **`stacks/heretic2-charrp-reasoning/` is UNTRACKED in git** (`?? stacks/heretic2-charrp-reasoning/`) — the char-rp-reasoning compose stack is deployed but never committed; commit it (canonical mirror of the live seat).
@@ -128,6 +128,8 @@ _As of 2026-07-15 — a long fleet-ops + infra session. Everything below either
 
 ## Recent decisions
 
+- `[2026-07-16]` **GPU re-org: char-rp→GPU1 + both cards re-optimized for max context.** Moved char-rp (Magidonia-24B) GPU0→GPU1, then maxed context: char-rp-reasoning 150K→256K (util 0.46, 1.56x), gen→256K + seqs 16→32 (util 0.42, 5.43x), granite 64K→**128K full-chapter** (util 0.27, 1.50x). FINAL: GPU0 ~14 G reserve (both seats 256K native), GPU1 ~6.7 G headroom. All healthy. LESSON: KV must hold ≥1× max-len (util-floor crashes) + per-model KV cost varies ~8× (MoE cheap, dense pricey) → tune util empirically. See Current state for the full layout + backups.
+
 - `[2026-07-16]` **granite right-sized → ~10.5 GB freed on GPU1** (util 0.34→0.18 + max-len 131072→65536; KV 6.45 GiB / 1.29x@65536, summarizer healthy). GPU1 now ~45 GB free to relocate a GPU0 model. LESSON: ~950 MiB KV per 0.01 util here + KV must hold ≥1× max-len — util 0.15 crash-looped (est max-len 47184<65536, ~2-3 min summarizer blip) before 0.18 landed. `.env`-only, recreate `vllm-granite` alone (shared stack).
 
 - `[2026-07-15]` **image-bench eviction DONE (parked item closed).** Stopped vllm-qwen-image-bench (ana-ml2 GPU1, ~32 GB freed); LiteLLM `image-judge`+`qwen-image-bench` → gen :8015 (judge samplers + thinking-off), verified with :8014 down; comfy-dev pinged; also backfilled the canonical char-rp-reasoning litellm block (was lagging live). Revert ~90 s. auto-memory `project_arbo_gen_switch_imagebench_evict`.

stacks/char-rp-gguf/.env.example

@@ -8,9 +8,11 @@
 # llama.cpp server image (bundles /app/llama-server; already present on ana-ml2).
 LLAMA_IMAGE=ghcr.io/mostlygeek/llama-swap:cuda
 
-# GPU 0 = the on-demand large-model card (co-resident with `gen`). GPU 1 is the
-# always-on vLLM trio (granite/embed/rerank/reward) — do NOT put this stack there.
-CHARRP_GPU_ID=0
+# GPU pin for the char-rp PROSE seat (llama-charrp). MOVED to GPU 1 on 2026-07-16 to
+# decongest GPU 0 (freed ~29 GB there for gen + char-rp-reasoning to grow to 256K).
+# NOTE: this var is ALSO read by the (retired, stopped) llama-charrp-reasoning service
+# below — recreate ONLY llama-charrp when changing it, or split into a dedicated var.
+CHARRP_GPU_ID=1
 
 # GGUF store on the host (mounted read-only at /models). Candidate GGUFs live in
 # /tank/aimodels/llm/rp/ (pre-pulled).

stacks/qwen36-27b-aeon/.env.example

@@ -19,9 +19,13 @@ AEON_GEN_PORT=8015
 AEON_GEN_SERVED_NAME=qwen3.6-27b-aeon
 AEON_GEN_SERVED_NAME_THINK=qwen3.6-27b-aeon-thinking
 AEON_GEN_MODEL=/tank/aimodels/qwen36-27b-aeon-nvfp4
-AEON_GEN_GPU_MEM_UTIL=0.45
-AEON_GEN_MAX_MODEL_LEN=131072
-AEON_GEN_MAX_NUM_SEQS=16
+# NOTE: live gen serves qwen3.6-35b-a3b-heretic since 2026-07-08 (aeon-27b displaced);
+# SERVED_NAME/MODEL above lag live — pending reconciliation with the model swap.
+# util/ctx/seqs RE-TUNED 2026-07-16 (char-rp left GPU-0 → room): util 0.30->0.42,
+# ctx 131072->262144 (256K native), seqs 16->32. gen MoE KV is cheap → 5.43x @ 256K.
+AEON_GEN_GPU_MEM_UTIL=0.42
+AEON_GEN_MAX_MODEL_LEN=262144
+AEON_GEN_MAX_NUM_SEQS=32
 AEON_GEN_KV_CACHE_DTYPE=fp8
 # Reasoning parser — KEEP qwen3. This checkpoint's chat_template.jinja injects the opening
 # <think> into the PROMPT (output has only </think>). qwen3 handles the non-thinking path

stacks/vllm/.env.example

@@ -96,7 +96,9 @@ GRANITE_SERVED_NAME=granite-4.1-8b
 # 65536 — REDUCED 2026-06-14 (was 131072) to free GPU-1 room for the FP8 vision
 # model (Qwen3.6-35B-A3B, stacks/qwen36-vl, ~34 GB weights). Summarizer load is
 # short parallel calls, so the 64K cap is ample.
-GRANITE_MAX_MODEL_LEN=65536
+# 131072 — RESTORED 2026-07-16 (native max) for full-chapter summarization; GPU-1
+# freed by the image-bench evict + qwen36-vl gone, so the 128K ctx fits again.
+GRANITE_MAX_MODEL_LEN=131072
 # FP8 KV cache (native on Blackwell cc 12.0). At 50K ≈ ~4.2 GB (vs ~8.4 GB at fp16).
 GRANITE_KV_CACHE_DTYPE=fp8
 # util 0.35 (~33.6 GB) — tuned 2026-06-13 to leave ~3.5 GB free on GPU 1 alongside
@@ -113,4 +115,7 @@ GRANITE_KV_CACHE_DTYPE=fp8
 # calls; ample). Effective slope on this shared card ≈ 950 MiB KV per 0.01 util, and
 # KV must hold >= 1x max-model-len — util 0.15 undershot (crash: est max-len 47184 <
 # 65536), 0.18 lands the target cleanly.
-GRANITE_GPU_MEM_UTIL=0.18
+# 0.27 — RE-GROWN 2026-07-16 (same session) after char-rp moved onto GPU-1: spend the
+# leftover room on full-chapter context (max-len 131072). KV 15.0 GiB = 196,560 tokens
+# = 1.50x @ 131072; GPU-1 lands ~6.7 GB headroom (char-rp 30 + granite 27 + selene 17 + trio).
+GRANITE_GPU_MEM_UTIL=0.27