gen-seat: deploy Qwen3.8-27B-Uncensored gen seat, rename qwen36-27b-aeon->gen-seat
- New uncensored gen seat: JonathanColetti/Qwen3.8-27B-Uncensored, in-house NVFP4 W4A16 (compressed-tensors) + grafted bf16 MTP (config ignore re:^mtp.*), vision-intact, 262K ctx, MTP n=3 (~42% accept, ~68 tok/s). Replaces the qwen3.6-35b-a3b-heretic MoE. - Rename compose project qwen36-27b-aeon -> gen-seat, container vllm-aeon-gen -> vllm-gen, env vars AEON_GEN_* -> GEN_*; drop the dormant vllm-aeon-rp service. - litellm: repoint 7 aliases (gen/summarizer/summarizer-large/classifier/image-judge/ qwen-image-bench -> qwen3.8-27b-uncensored; gen-reasoning -> -thinking). - servers/ana-ml2/README: refresh the gen hero-seat row.
This commit is contained in:
@@ -0,0 +1,18 @@
|
||||
# gen-seat tunables — fleet `gen` seat (ana-ml2 GPU 0, :8015). Edit here, never commit.
|
||||
GEN_IMAGE=vllm/vllm-openai:latest
|
||||
API_KEY=
|
||||
GEN_GPU_ID=0
|
||||
|
||||
GEN_CONTAINER_NAME=vllm-gen
|
||||
GEN_PORT=8015
|
||||
GEN_SERVED_NAME=qwen3.8-27b-uncensored
|
||||
GEN_SERVED_NAME_THINK=qwen3.8-27b-uncensored-thinking
|
||||
GEN_MODEL=/tank/aimodels/qwen38-27b-uncensored-nvfp4
|
||||
GEN_QUANT=compressed-tensors
|
||||
GEN_GPU_MEM_UTIL=0.45
|
||||
GEN_MAX_MODEL_LEN=262144
|
||||
GEN_MAX_NUM_SEQS=16
|
||||
GEN_KV_CACHE_DTYPE=fp8
|
||||
GEN_REASONING_PARSER=qwen3
|
||||
GEN_SPEC_METHOD=qwen3_5_mtp
|
||||
GEN_SPEC_TOKENS=3
|
||||
Reference in New Issue
Block a user