vh
bfae924048
flukethoughts/Qwen-Image-Bench-NVFP4 — Qwen's text-to-image quality JUDGE (vision-intact, NVFP4) on ana-ml2 GPU 1, replacing qwen3.6-35b-a3b: - stacks/qwen-image-bench/ — vLLM multimodal serve (Qwen3_5ForConditionalGeneration, no text-only patch — vision wanted), GPU1 device pin, :8014. util 0.32 (0.22 KV-starved → crash-loop "no available memory for cache blocks"; util is a fraction of TOTAL so it must clear the ~20GB weight floor). - litellm: removed qwen3.6-35b-a3b + -thinking; added qwen-image-bench + qwen-image-judge alias. Verified live: healthy (KV 9.4GB / 133K tokens), text + IMAGE (vision path) both respond. NOTE: arbo's hero-judge was bound to qwen3.6-35b-a3b — comfy-dev notified to repoint.
21 lines
814 B
Bash
21 lines
814 B
Bash
# qwen-image-bench (flukethoughts/Qwen-Image-Bench-NVFP4) — ana-ml2 GPU 1 tunables.
|
|
# Real .env lives at /opt/docker/compose/qwen-image-bench/.env.
|
|
|
|
QIB_IMAGE=vllm/vllm-openai:latest
|
|
QIB_CONTAINER_NAME=vllm-qwen-image-bench
|
|
|
|
# Own port (8007=qwen36 [downed], 8010=qwen-gen, 8011=selene — 8014 free).
|
|
QIB_PORT=8014
|
|
QIB_GPU_ID=1
|
|
|
|
# Shared GPU 1 (granite/selene/embed/rerank/reward + this). util is a fraction of
|
|
# TOTAL (96GB), so it must clear the ~20GB weight floor + leave KV: 0.22 (~21GB) left
|
|
# ~nothing for KV and crash-looped ("no available memory for cache blocks"). 0.32
|
|
# (~31GB → ~9GB KV) fits the ~37GB freed by downing qwen36 (cf. qwen36's own 0.34).
|
|
QIB_GPU_MEM_UTIL=0.32
|
|
QIB_MAX_MODEL_LEN=32768
|
|
QIB_MAX_NUM_SEQS=8
|
|
|
|
# Optional upstream vLLM API key (empty = no auth; internal net only).
|
|
API_KEY=
|