vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(gpu): relocate char-rp to GPU1 + re-optimize both ana-ml2 cards for max context

Browse Source 
Operator-directed 2026-07-16. Moved the char-rp prose seat (Magidonia-24B,
llama-charrp) from GPU0 to GPU1 (CHARRP_GPU_ID 0->1; recreate llama-charrp
only -- the var is shared with the retired GGUF reasoning service), then
re-optimized every context-relevant seat on both cards to native/max context
with acceptable headroom:

  GPU0 (both seats now 256K native, ~14 GB reserve):
    - char-rp-reasoning 150K -> 256K  (heretic2 stack, util 0.38->0.46, 1.56x)
    - gen 256K, max-num-seqs 16 -> 32 (qwen36-27b-aeon, util 0.30->0.42, 5.43x)
  GPU1 (~6.7 GB headroom):
    - granite 64K -> 128K full-chapter (vllm stack, util 0.18->0.27, 1.50x)
    - char-rp 128K native (4 slots), selene/reward/embed/rerank unchanged

All seats gateway-verified healthy. Live .env changes on ana-ml2 with per-stack
backups (*-20260716). Templates updated to match; the qwen36-27b-aeon template
carries a NOTE that its served-name/model still lag the 2026-07-08 gen model swap
(35B-A3B-heretic) -- separate reconciliation. persistent-memory records the full
layout + the util-floor / per-model-KV-cost lessons.

Note: the heretic2-charrp-reasoning stack (char-rp-reasoning's live config) is
still untracked in git -- standing open-loop, its .env change lives server-side only.
This commit is contained in:
vh
2026-07-16 09:28:07 -07:00
parent f25f494f07
commit a2b026d499
4 changed files with 22 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
stacks/qwen36-27b-aeon/.env.example
+7 -3
View File
@@ -19,9 +19,13 @@ AEON_GEN_PORT=8015
AEON_GEN_SERVED_NAME=qwen3.6-27b-aeon
AEON_GEN_SERVED_NAME_THINK=qwen3.6-27b-aeon-thinking
AEON_GEN_MODEL=/tank/aimodels/qwen36-27b-aeon-nvfp4
AEON_GEN_GPU_MEM_UTIL=0.45
AEON_GEN_MAX_MODEL_LEN=131072
AEON_GEN_MAX_NUM_SEQS=16
# NOTE: live gen serves qwen3.6-35b-a3b-heretic since 2026-07-08 (aeon-27b displaced);
# SERVED_NAME/MODEL above lag live — pending reconciliation with the model swap.
# util/ctx/seqs RE-TUNED 2026-07-16 (char-rp left GPU-0 → room): util 0.30->0.42,
# ctx 131072->262144 (256K native), seqs 16->32. gen MoE KV is cheap → 5.43x @ 256K.
AEON_GEN_GPU_MEM_UTIL=0.42
AEON_GEN_MAX_MODEL_LEN=262144
AEON_GEN_MAX_NUM_SEQS=32
AEON_GEN_KV_CACHE_DTYPE=fp8
# Reasoning parser — KEEP qwen3. This checkpoint's chat_template.jinja injects the opening
# <think> into the PROMPT (output has only </think>). qwen3 handles the non-thinking path