vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(litellm): retire image-bench backend, repoint image-judge + qwen-image-bench aliases to gen

Browse Source 
Operator-directed 2026-07-15. The dedicated Qwen-Image-Bench NVFP4 judge
backend on ana-ml2 GPU1 (:8014) was stopped to reclaim ~32GB after the
arbo -> gen hero-judge switch. Both LiteLLM gateway aliases that pointed at
it -- image-judge and qwen-image-bench -- now repoint to the gen backend
(:8015, qwen3.6-35b-a3b-heretic, vision-intact), held at deterministic
judge sampling (temp 0 / top_k 1 / rep_pen 1.05) with enable_thinking:false
(a reasoning preamble breaks json_object). Verified live: both answer with
:8014 down, so they are definitively on gen.

Incidental: backfilled the canonical char-rp-reasoning litellm block, which
had lagged live since the 2026-07-14 NVFP4+MTP seat repoint (model
deckard-pkd-27b -> char-rp-reasoning, top_k 40 -> 20, min_p dropped,
enable_thinking:true added). Required so pushing the canonical would not
clobber the correct live block.

Live changes applied out-of-band (config push + litellm restart + stack
stop on ana-ml2); live config backup at
config.yaml.bak-pre-imagejudge-20260715. Revert path documented in the
config comment. persistent-memory.md updated (parked item closed).
This commit is contained in:
vh
2026-07-15 23:07:13 -07:00
parent d710e56aca
commit 925947c71e
2 changed files with 23 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
stacks/litellm/conf/config.yaml
+20 -14
View File
@@ -72,35 +72,40 @@ model_list:
model_info:
mode: chat
# --- Qwen-Image-Bench (NVFP4) — text-to-image quality JUDGE, vision-intact. vLLM
# on ana-ml2 GPU 1, :8014. REPLACED qwen3.6-35b-a3b (operator 2026-06-19; the
# qwen36 VL judge was downed + its FP8 weights freed ~33GB). Multimodal — image
# + prompt in, a 5-dim verdict out (overall quality / prompt match / aesthetic /
# LoRA activation / confidence). image-judge is a duplicate-entry alias.
# ⚠️ arbo's hero-judge was bound to qwen3.6-35b-a3b — comfy-dev notified to
# repoint (different I/O than the old general VL judge). ---
# --- image-judge / qwen-image-bench — T2I quality JUDGE aliases. The dedicated
# Qwen-Image-Bench NVFP4 backend (ana-ml2 GPU 1, :8014) was RETIRED 2026-07-15
# (operator: reclaim ~30GB GPU1) after the arbo→gen hero-judge switch. Both
# aliases now REPOINT to the gen backend (:8015, qwen3.6-35b-a3b-heretic,
# vision-intact), held at deterministic judge sampling (temp 0 / top_k 1) with
# enable_thinking:false (a reasoning preamble breaks json_object). Revert =
# `docker compose start` stacks/qwen-image-bench on ana-ml2 + repoint api_base
# back to :8014 + model hosted_vllm/qwen-image-bench. ---
- model_name: qwen-image-bench
litellm_params:
model: hosted_vllm/qwen-image-bench
api_base: http://10.250.50.54:8014/v1
model: hosted_vllm/qwen3.6-35b-a3b-heretic
api_base: http://10.250.50.54:8015/v1
api_key: os.environ/VLLM_API_KEY
temperature: 0
top_p: 1.0
extra_body:
top_k: 1
repetition_penalty: 1.05
chat_template_kwargs:
enable_thinking: false
model_info:
mode: chat
- model_name: image-judge
litellm_params:
model: hosted_vllm/qwen-image-bench
api_base: http://10.250.50.54:8014/v1
model: hosted_vllm/qwen3.6-35b-a3b-heretic
api_base: http://10.250.50.54:8015/v1
api_key: os.environ/VLLM_API_KEY
temperature: 0
top_p: 1.0
extra_body:
top_k: 1
repetition_penalty: 1.05
chat_template_kwargs:
enable_thinking: false
model_info:
mode: chat
@@ -184,14 +189,15 @@ model_list:
# char-rp (best-of-breed per seat) — see stacks/char-rp-gguf/README.md.
- model_name: char-rp-reasoning
litellm_params:
model: hosted_vllm/deckard-pkd-27b
model: hosted_vllm/char-rp-reasoning
api_base: http://10.250.50.54:8018/v1
api_key: os.environ/VLLM_API_KEY
temperature: 1.0
top_p: 0.95
extra_body:
top_k: 40
min_p: 0.05
top_k: 20
chat_template_kwargs:
enable_thinking: true
model_info:
mode: chat