feat(vllm,litellm): lfm2.5-2.6b non-prod bake-off alias for brokkr
vllm-lfm25 on ana-ml2 GPU1 :8021 (LiquidAI/LFM2.5-2.6B, util 0.09 into unreserved slack, max-len 16384, no reasoning-parser so content is non-empty). LiteLLM alias lfm2.5-2.6b with vendor sampling baked as default (temp 0.1; top_k 50 + repetition_penalty 1.1 via extra_body). Eval-only, not in any routing chain, pending operator ruling on LFM Open License production use.
This commit is contained in:
@@ -142,3 +142,18 @@ CODER_MAX_MODEL_LEN=8192
|
||||
CODER_KV_CACHE_DTYPE=fp8
|
||||
CODER_GPU_MEM_UTIL=0.06
|
||||
CODER_MAX_NUM_SEQS=32
|
||||
|
||||
# LFM2.5-2.6B — NON-PRODUCTION bake-off alias vs granite (brokkr R-target 2026-08-10).
|
||||
# LiquidAI LFM Open License v1.0 (<USD 10M-rev commercial, not OSI) — eval-only pending
|
||||
# an operator production ruling. Reasoning model (emits <think>); served raw (no vLLM
|
||||
# reasoning-parser) so content is non-empty. util 0.09 (~8.6GB) fits GPU1's free slack
|
||||
# without touching production reservations; max-len 16384 keeps KV small (bake-off
|
||||
# doesn't need the 131k ceiling). Vendor sampling lives in the LiteLLM alias.
|
||||
LFM25_PORT=8021
|
||||
LFM25_GPU_ID=1
|
||||
LFM25_MODEL=LiquidAI/LFM2.5-2.6B
|
||||
LFM25_SERVED_NAME=lfm2.5-2.6b
|
||||
LFM25_MAX_MODEL_LEN=16384
|
||||
LFM25_KV_CACHE_DTYPE=auto
|
||||
LFM25_GPU_MEM_UTIL=0.09
|
||||
LFM25_MAX_NUM_SEQS=8
|
||||
|
||||
Reference in New Issue
Block a user