diff --git a/persistent-memory.md b/persistent-memory.md index 8f60a3e..32c1bcb 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -118,7 +118,7 @@ _As of 2026-07-15 β€” a long fleet-ops + infra session. Everything below either - **Statusline overhaul** (`~/.claude/statusline-command.sh`, NOT this repo) β€” git state (branch/dirty/unpushed), πŸ””/πŸ”• althing-monitor-armed, project tag, absolute tokens, **per-session cost** (`.cost.total_cost_usd`), threshold-colored ctx% + rate %s (green<60 / yellow60-90 / red>90). **PARKED (operator-directed holds):** -- **image-bench (qwen-image-bench, GPU1) eviction β€” PENDING post-bake.** Operator directed a full arbo -> gen switch (comfy-dev executed it, LIVE in prod: `DEFAULT_LLM_MODEL=gen` + `ARBO_VISION_MODEL=gen`); image-judge now has ZERO arbo tenancy. Sequencing = **short prod bake on gen (~few days) THEN evict** -> reclaim ~30 GB GPU1. Keep `vllm-qwen-image-bench` WARM as the instant-revert backstop; comfy-dev watches NSFW-anatomy scoring + tight-grounding (flip `ARBO_VISION_MODEL` back on regression). On a clean bake: evict + ping comfy-dev. **Fully tracked in auto-memory `project_arbo_gen_switch_imagebench_evict`.** +- **image-bench (qwen-image-bench, GPU1) eviction β€” DONE 2026-07-15 (operator-directed, same-day as the switch).** Stopped `vllm-qwen-image-bench` on ana-ml2 (Exited(0), revert via `docker compose start`) β†’ GPU1 freed ~32 GB (95β†’63 GB used). LiteLLM `image-judge` + `qwen-image-bench` aliases repointed to gen (:8015, qwen3.6-35b-a3b-heretic), deterministic judge sampling (temp0/top_k1/rep_pen1.05) + `enable_thinking:false` kept; verified live with :8014 down. comfy-dev pinged as regression watcher (NSFW-anatomy + tight-grounding; thread 01KXMRDJ). Backstop = the stopped stack (revert ~90 s) + live config backup `config.yaml.bak-pre-imagejudge-20260715`. Full record: auto-memory `project_arbo_gen_switch_imagebench_evict`. - **granite right-size β€” un-started GPU1 win.** granite (GPU1) is ~29 GB (util 0.34 -> ~16.5 GB oversized KV for an 8B summarizer). Right-sizing util 0.34 -> ~0.13 frees ~16 GB WHILE keeping the summarizer (fleet's dedicated summarizer/classifier; aliases granite-4.1-8b/summarizer/classifier). Independent of image-bench; operator was freeing GPU1 for a TBD "something else". **Open loops for the fresh session:** @@ -128,6 +128,8 @@ _As of 2026-07-15 β€” a long fleet-ops + infra session. Everything below either ## Recent decisions +- `[2026-07-15]` **image-bench eviction DONE (parked item closed).** Stopped vllm-qwen-image-bench (ana-ml2 GPU1, ~32 GB freed); LiteLLM `image-judge`+`qwen-image-bench` β†’ gen :8015 (judge samplers + thinking-off), verified with :8014 down; comfy-dev pinged; also backfilled the canonical char-rp-reasoning litellm block (was lagging live). Revert ~90 s. auto-memory `project_arbo_gen_switch_imagebench_evict`. + - `[2026-07-15]` arbo fully switched off image-judge (qwen-image-bench) -> gen; image-bench pending eviction post-bake β†’ `persistent-memory.d/2026-07-15-arbo-fully-switched-off-image-judge-qwen-image.md` - `[2026-07-15]` esh-docker-vm NFS fstab fix = `x-systemd.before=docker.service` β†’ `persistent-memory.d/2026-07-15-esh-docker-vm-nfs-fstab-fix-x-systemd.md` diff --git a/stacks/litellm/conf/config.yaml b/stacks/litellm/conf/config.yaml index 2f5e931..9ed4915 100644 --- a/stacks/litellm/conf/config.yaml +++ b/stacks/litellm/conf/config.yaml @@ -72,35 +72,40 @@ model_list: model_info: mode: chat - # --- Qwen-Image-Bench (NVFP4) β€” text-to-image quality JUDGE, vision-intact. vLLM - # on ana-ml2 GPU 1, :8014. REPLACED qwen3.6-35b-a3b (operator 2026-06-19; the - # qwen36 VL judge was downed + its FP8 weights freed ~33GB). Multimodal β€” image - # + prompt in, a 5-dim verdict out (overall quality / prompt match / aesthetic / - # LoRA activation / confidence). image-judge is a duplicate-entry alias. - # ⚠️ arbo's hero-judge was bound to qwen3.6-35b-a3b β€” comfy-dev notified to - # repoint (different I/O than the old general VL judge). --- + # --- image-judge / qwen-image-bench β€” T2I quality JUDGE aliases. The dedicated + # Qwen-Image-Bench NVFP4 backend (ana-ml2 GPU 1, :8014) was RETIRED 2026-07-15 + # (operator: reclaim ~30GB GPU1) after the arboβ†’gen hero-judge switch. Both + # aliases now REPOINT to the gen backend (:8015, qwen3.6-35b-a3b-heretic, + # vision-intact), held at deterministic judge sampling (temp 0 / top_k 1) with + # enable_thinking:false (a reasoning preamble breaks json_object). Revert = + # `docker compose start` stacks/qwen-image-bench on ana-ml2 + repoint api_base + # back to :8014 + model hosted_vllm/qwen-image-bench. --- - model_name: qwen-image-bench litellm_params: - model: hosted_vllm/qwen-image-bench - api_base: http://10.250.50.54:8014/v1 + model: hosted_vllm/qwen3.6-35b-a3b-heretic + api_base: http://10.250.50.54:8015/v1 api_key: os.environ/VLLM_API_KEY temperature: 0 top_p: 1.0 extra_body: top_k: 1 repetition_penalty: 1.05 + chat_template_kwargs: + enable_thinking: false model_info: mode: chat - model_name: image-judge litellm_params: - model: hosted_vllm/qwen-image-bench - api_base: http://10.250.50.54:8014/v1 + model: hosted_vllm/qwen3.6-35b-a3b-heretic + api_base: http://10.250.50.54:8015/v1 api_key: os.environ/VLLM_API_KEY temperature: 0 top_p: 1.0 extra_body: top_k: 1 repetition_penalty: 1.05 + chat_template_kwargs: + enable_thinking: false model_info: mode: chat @@ -184,14 +189,15 @@ model_list: # char-rp (best-of-breed per seat) β€” see stacks/char-rp-gguf/README.md. - model_name: char-rp-reasoning litellm_params: - model: hosted_vllm/deckard-pkd-27b + model: hosted_vllm/char-rp-reasoning api_base: http://10.250.50.54:8018/v1 api_key: os.environ/VLLM_API_KEY temperature: 1.0 top_p: 0.95 extra_body: - top_k: 40 - min_p: 0.05 + top_k: 20 + chat_template_kwargs: + enable_thinking: true model_info: mode: chat