feat(vllm,litellm): lfm2.5-2.6b non-prod bake-off alias for brokkr

vllm-lfm25 on ana-ml2 GPU1 :8021 (LiquidAI/LFM2.5-2.6B, util 0.09 into
unreserved slack, max-len 16384, no reasoning-parser so content is non-empty).
LiteLLM alias lfm2.5-2.6b with vendor sampling baked as default (temp 0.1;
top_k 50 + repetition_penalty 1.1 via extra_body). Eval-only, not in any
routing chain, pending operator ruling on LFM Open License production use.
This commit is contained in:
vh
2026-08-10 07:13:37 -07:00
parent c8acf60449
commit edc9f42da1
4 changed files with 100 additions and 1 deletions
+15
View File
@@ -142,3 +142,18 @@ CODER_MAX_MODEL_LEN=8192
CODER_KV_CACHE_DTYPE=fp8
CODER_GPU_MEM_UTIL=0.06
CODER_MAX_NUM_SEQS=32
# LFM2.5-2.6B — NON-PRODUCTION bake-off alias vs granite (brokkr R-target 2026-08-10).
# LiquidAI LFM Open License v1.0 (<USD 10M-rev commercial, not OSI) — eval-only pending
# an operator production ruling. Reasoning model (emits <think>); served raw (no vLLM
# reasoning-parser) so content is non-empty. util 0.09 (~8.6GB) fits GPU1's free slack
# without touching production reservations; max-len 16384 keeps KV small (bake-off
# doesn't need the 131k ceiling). Vendor sampling lives in the LiteLLM alias.
LFM25_PORT=8021
LFM25_GPU_ID=1
LFM25_MODEL=LiquidAI/LFM2.5-2.6B
LFM25_SERVED_NAME=lfm2.5-2.6b
LFM25_MAX_MODEL_LEN=16384
LFM25_KV_CACHE_DTYPE=auto
LFM25_GPU_MEM_UTIL=0.09
LFM25_MAX_NUM_SEQS=8