feat(vllm,litellm): lfm2.5-2.6b non-prod bake-off alias for brokkr

vllm-lfm25 on ana-ml2 GPU1 :8021 (LiquidAI/LFM2.5-2.6B, util 0.09 into
unreserved slack, max-len 16384, no reasoning-parser so content is non-empty).
LiteLLM alias lfm2.5-2.6b with vendor sampling baked as default (temp 0.1;
top_k 50 + repetition_penalty 1.1 via extra_body). Eval-only, not in any
routing chain, pending operator ruling on LFM Open License production use.
This commit is contained in:
vh
2026-08-10 07:13:37 -07:00
parent c8acf60449
commit edc9f42da1
4 changed files with 100 additions and 1 deletions
+17
View File
@@ -426,6 +426,23 @@ model_list:
# model names. Removed so unknown models now fail loudly (404). Re-add an
# explicit per-model entry if a swappable zoo ever returns. ---
# --- lfm2.5-2.6b -> LiquidAI LFM2.5-2.6B (ana-ml2 GPU1 :8021, vLLM). NON-PROD bake-off
# vs granite-4.1-8b (brokkr R-target 2026-08-10). LFM Open License v1.0 (<USD 10M-rev
# commercial) - EVAL-ONLY pending operator ruling; NOT in any default/fallback chain.
# Reasoning model served raw (no vLLM reasoning-parser) so content is non-empty.
# Vendor sampling (temp 0.1 / top_k 50 / rep_pen 1.1) baked as the alias default. ---
- model_name: lfm2.5-2.6b
litellm_params:
model: hosted_vllm/lfm2.5-2.6b
api_base: http://10.250.50.54:8021/v1
api_key: os.environ/VLLM_API_KEY
temperature: 0.1
extra_body:
top_k: 50
repetition_penalty: 1.1
model_info:
mode: chat
general_settings:
master_key: os.environ/LITELLM_MASTER_KEY
database_url: os.environ/DATABASE_URL