feat(litellm): promote cyberprev into the sec / sec-reasoning seats; retire mog-sec
Operator instruction: take down the existing sec seat (mog-sec) and promote hotdogs (cyberprev) into the sec and sec-reasoning gateway seats. - mog-sec container (vllm-mog-sec, :8019, fv-ml1 GPU0) taken down; ~48 GB freed on GPU0 (cyberprev, already co-resident there, is now the sole GPU0 chat seat). - Gateway sec -> hosted_vllm/cyberprev-27b @ :8025; sec-reasoning -> hosted_vllm/cyberprev-27b-thinking @ :8025. sec/sec-reasoning are ROLE aliases, so this is a promotion, not silent substitution (samplers were already identical between the sec blocks and cyberprev, so only model+api_base changed). - Removed the standalone cyberprev-27b / cyberprev-reasoning gateway aliases added in the prior commit -- now redundant with sec/sec-reasoning, and the fleet convention is a role alias on the gateway with the model's served-name only at the vLLM layer (as mog-sec had). cyberprev's vLLM served-names are unchanged. - Verified e2e through the gateway: sec answers (nmap -sV version detection), sec-reasoning answers with a thinking split (127 reasoning tokens); retired mog-sec-27b now 400s. Note: mog-sec was the fleet's only offense+defense/blue-team seat; the sec role is now offense-only (cyberprev tool-calling). Operator-directed after reviewing the capability comparison. mog-sec stack files retained for a future restore.
This commit is contained in:
@@ -376,8 +376,8 @@ model_list:
|
||||
# structure (and upstream warns it can cause language mixing). Non-thinking. ---
|
||||
- model_name: sec
|
||||
litellm_params:
|
||||
model: hosted_vllm/mog-sec-27b
|
||||
api_base: http://10.251.50.54:8019/v1
|
||||
model: hosted_vllm/cyberprev-27b
|
||||
api_base: http://10.251.50.54:8025/v1
|
||||
api_key: os.environ/VLLM_API_KEY
|
||||
temperature: 0.7
|
||||
top_p: 0.8
|
||||
@@ -395,8 +395,8 @@ model_list:
|
||||
# shared-config clobber). Canonical Qwen3.8 thinking samplers (temp 1.0/top_p 0.95).
|
||||
- model_name: sec-reasoning
|
||||
litellm_params:
|
||||
model: hosted_vllm/mog-sec-27b-thinking
|
||||
api_base: http://10.251.50.54:8019/v1
|
||||
model: hosted_vllm/cyberprev-27b-thinking
|
||||
api_base: http://10.251.50.54:8025/v1
|
||||
api_key: os.environ/VLLM_API_KEY
|
||||
temperature: 1.0
|
||||
top_p: 0.95
|
||||
@@ -970,44 +970,6 @@ model_list:
|
||||
mode: chat
|
||||
|
||||
# cyberprev-27b -> hotdogs/Qwen3.8-27B-abliterated-cyber-preview (:8025, fv-ml1 GPU0).
|
||||
# Cyber offensive-security tool-calling SFT (LoRA finetune on an abliterated base), in-house
|
||||
# mixed NVFP4 quant, dflash k=7. Replaced the retired sentinel-r3 seat 2026-09-14.
|
||||
# Sampler mirrors sec/sec-reasoning so an A/B isolates weights, not sampling. The retired
|
||||
# sentinel-r3 aliases were DELETED, not repointed — cyberprev weights under sentinel's name
|
||||
# would be silent substitution.
|
||||
- model_name: cyberprev-27b
|
||||
litellm_params:
|
||||
model: hosted_vllm/cyberprev-27b
|
||||
api_base: http://10.251.50.54:8025/v1
|
||||
api_key: os.environ/VLLM_API_KEY
|
||||
temperature: 0.7
|
||||
top_p: 0.8
|
||||
presence_penalty: 0.0
|
||||
extra_body:
|
||||
top_k: 20
|
||||
min_p: 0.0
|
||||
repetition_penalty: 1.0
|
||||
chat_template_kwargs:
|
||||
enable_thinking: false
|
||||
model_info:
|
||||
mode: chat
|
||||
- model_name: cyberprev-reasoning
|
||||
litellm_params:
|
||||
model: hosted_vllm/cyberprev-27b-thinking
|
||||
api_base: http://10.251.50.54:8025/v1
|
||||
api_key: os.environ/VLLM_API_KEY
|
||||
temperature: 1.0
|
||||
top_p: 0.95
|
||||
presence_penalty: 0.0
|
||||
extra_body:
|
||||
top_k: 20
|
||||
min_p: 0.0
|
||||
repetition_penalty: 1.0
|
||||
chat_template_kwargs:
|
||||
enable_thinking: true
|
||||
model_info:
|
||||
mode: chat
|
||||
|
||||
general_settings:
|
||||
master_key: os.environ/LITELLM_MASTER_KEY
|
||||
database_url: os.environ/DATABASE_URL
|
||||
|
||||
Reference in New Issue
Block a user