gen-seat: deploy Qwen3.8-27B-Uncensored gen seat, rename qwen36-27b-aeon->gen-seat

- New uncensored gen seat: JonathanColetti/Qwen3.8-27B-Uncensored, in-house NVFP4
  W4A16 (compressed-tensors) + grafted bf16 MTP (config ignore re:^mtp.*), vision-intact,
  262K ctx, MTP n=3 (~42% accept, ~68 tok/s). Replaces the qwen3.6-35b-a3b-heretic MoE.
- Rename compose project qwen36-27b-aeon -> gen-seat, container vllm-aeon-gen -> vllm-gen,
  env vars AEON_GEN_* -> GEN_*; drop the dormant vllm-aeon-rp service.
- litellm: repoint 7 aliases (gen/summarizer/summarizer-large/classifier/image-judge/
  qwen-image-bench -> qwen3.8-27b-uncensored; gen-reasoning -> -thinking).
- servers/ana-ml2/README: refresh the gen hero-seat row.
This commit is contained in:
vh
2026-08-15 00:19:20 -07:00
parent dac4acf0c5
commit 680c30e778
6 changed files with 148 additions and 295 deletions
+18
View File
@@ -0,0 +1,18 @@
# gen-seat tunables — fleet `gen` seat (ana-ml2 GPU 0, :8015). Edit here, never commit.
GEN_IMAGE=vllm/vllm-openai:latest
API_KEY=
GEN_GPU_ID=0
GEN_CONTAINER_NAME=vllm-gen
GEN_PORT=8015
GEN_SERVED_NAME=qwen3.8-27b-uncensored
GEN_SERVED_NAME_THINK=qwen3.8-27b-uncensored-thinking
GEN_MODEL=/tank/aimodels/qwen38-27b-uncensored-nvfp4
GEN_QUANT=compressed-tensors
GEN_GPU_MEM_UTIL=0.45
GEN_MAX_MODEL_LEN=262144
GEN_MAX_NUM_SEQS=16
GEN_KV_CACHE_DTYPE=fp8
GEN_REASONING_PARSER=qwen3
GEN_SPEC_METHOD=qwen3_5_mtp
GEN_SPEC_TOKENS=3