feat(litellm): add mistral-medium-3.5 (RecViking NVFP4 :8012, temporary GPU0 tenant)
This commit is contained in:
@@ -95,6 +95,19 @@ model_list:
|
|||||||
model_info:
|
model_info:
|
||||||
mode: chat
|
mode: chat
|
||||||
|
|
||||||
|
# --- Mistral Medium 3.5 (RecViking NVFP4, vLLM nightly) — TEMPORARY speed-check
|
||||||
|
# tenant DISPLACING mistral-small-4 on GPU 0 (2026-06-19). Loads the HF-format
|
||||||
|
# NVFP4 directly on :8012; served-name mistral-medium-3.5. NOTE: while this is
|
||||||
|
# up, mistral-small-4 (above) points at a DOWN backend — revert is the
|
||||||
|
# mistral-small-4-heretic stack. ---
|
||||||
|
- model_name: mistral-medium-3.5
|
||||||
|
litellm_params:
|
||||||
|
model: hosted_vllm/mistral-medium-3.5
|
||||||
|
api_base: http://10.250.50.54:8012/v1
|
||||||
|
api_key: os.environ/VLLM_API_KEY
|
||||||
|
model_info:
|
||||||
|
mode: chat
|
||||||
|
|
||||||
# --- Selene 1 Mini 8B (AtlaAI judge, FP8) — restored on GPU1 after the
|
# --- Selene 1 Mini 8B (AtlaAI judge, FP8) — restored on GPU1 after the
|
||||||
# llama-swap teardown (was the Q6_K GGUF in the swap zoo). vLLM dynamic fp8,
|
# llama-swap teardown (was the Q6_K GGUF in the swap zoo). vLLM dynamic fp8,
|
||||||
# :8011. Explicit entry shadows the "*" wildcard (which used to reach it via
|
# :8011. Explicit entry shadows the "*" wildcard (which used to reach it via
|
||||||
|
|||||||
Reference in New Issue
Block a user