fix(litellm): retire image-bench backend, repoint image-judge + qwen-image-bench aliases to gen

Operator-directed 2026-07-15. The dedicated Qwen-Image-Bench NVFP4 judge
backend on ana-ml2 GPU1 (:8014) was stopped to reclaim ~32GB after the
arbo -> gen hero-judge switch. Both LiteLLM gateway aliases that pointed at
it -- image-judge and qwen-image-bench -- now repoint to the gen backend
(:8015, qwen3.6-35b-a3b-heretic, vision-intact), held at deterministic
judge sampling (temp 0 / top_k 1 / rep_pen 1.05) with enable_thinking:false
(a reasoning preamble breaks json_object). Verified live: both answer with
:8014 down, so they are definitively on gen.

Incidental: backfilled the canonical char-rp-reasoning litellm block, which
had lagged live since the 2026-07-14 NVFP4+MTP seat repoint (model
deckard-pkd-27b -> char-rp-reasoning, top_k 40 -> 20, min_p dropped,
enable_thinking:true added). Required so pushing the canonical would not
clobber the correct live block.

Live changes applied out-of-band (config push + litellm restart + stack
stop on ana-ml2); live config backup at
config.yaml.bak-pre-imagejudge-20260715. Revert path documented in the
config comment. persistent-memory.md updated (parked item closed).
This commit is contained in:
vh
2026-07-15 23:07:13 -07:00
parent d710e56aca
commit 925947c71e
2 changed files with 23 additions and 15 deletions
+3 -1
View File
@@ -118,7 +118,7 @@ _As of 2026-07-15 — a long fleet-ops + infra session. Everything below either
- **Statusline overhaul** (`~/.claude/statusline-command.sh`, NOT this repo) — git state (branch/dirty/unpushed), 🔔/🔕 althing-monitor-armed, project tag, absolute tokens, **per-session cost** (`.cost.total_cost_usd`), threshold-colored ctx% + rate %s (green<60 / yellow60-90 / red>90).
**PARKED (operator-directed holds):**
- **image-bench (qwen-image-bench, GPU1) eviction — PENDING post-bake.** Operator directed a full arbo -> gen switch (comfy-dev executed it, LIVE in prod: `DEFAULT_LLM_MODEL=gen` + `ARBO_VISION_MODEL=gen`); image-judge now has ZERO arbo tenancy. Sequencing = **short prod bake on gen (~few days) THEN evict** -> reclaim ~30 GB GPU1. Keep `vllm-qwen-image-bench` WARM as the instant-revert backstop; comfy-dev watches NSFW-anatomy scoring + tight-grounding (flip `ARBO_VISION_MODEL` back on regression). On a clean bake: evict + ping comfy-dev. **Fully tracked in auto-memory `project_arbo_gen_switch_imagebench_evict`.**
- **image-bench (qwen-image-bench, GPU1) eviction — DONE 2026-07-15 (operator-directed, same-day as the switch).** Stopped `vllm-qwen-image-bench` on ana-ml2 (Exited(0), revert via `docker compose start`) → GPU1 freed ~32 GB (95→63 GB used). LiteLLM `image-judge` + `qwen-image-bench` aliases repointed to gen (:8015, qwen3.6-35b-a3b-heretic), deterministic judge sampling (temp0/top_k1/rep_pen1.05) + `enable_thinking:false` kept; verified live with :8014 down. comfy-dev pinged as regression watcher (NSFW-anatomy + tight-grounding; thread 01KXMRDJ). Backstop = the stopped stack (revert ~90 s) + live config backup `config.yaml.bak-pre-imagejudge-20260715`. Full record: auto-memory `project_arbo_gen_switch_imagebench_evict`.
- **granite right-size — un-started GPU1 win.** granite (GPU1) is ~29 GB (util 0.34 -> ~16.5 GB oversized KV for an 8B summarizer). Right-sizing util 0.34 -> ~0.13 frees ~16 GB WHILE keeping the summarizer (fleet's dedicated summarizer/classifier; aliases granite-4.1-8b/summarizer/classifier). Independent of image-bench; operator was freeing GPU1 for a TBD "something else".
**Open loops for the fresh session:**
@@ -128,6 +128,8 @@ _As of 2026-07-15 — a long fleet-ops + infra session. Everything below either
## Recent decisions
- `[2026-07-15]` **image-bench eviction DONE (parked item closed).** Stopped vllm-qwen-image-bench (ana-ml2 GPU1, ~32 GB freed); LiteLLM `image-judge`+`qwen-image-bench` → gen :8015 (judge samplers + thinking-off), verified with :8014 down; comfy-dev pinged; also backfilled the canonical char-rp-reasoning litellm block (was lagging live). Revert ~90 s. auto-memory `project_arbo_gen_switch_imagebench_evict`.
- `[2026-07-15]` arbo fully switched off image-judge (qwen-image-bench) -> gen; image-bench pending eviction post-bake → `persistent-memory.d/2026-07-15-arbo-fully-switched-off-image-judge-qwen-image.md`
- `[2026-07-15]` esh-docker-vm NFS fstab fix = `x-systemd.before=docker.service` → `persistent-memory.d/2026-07-15-esh-docker-vm-nfs-fstab-fix-x-systemd.md`