vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(vllm,litellm): lfm2.5-2.6b non-prod bake-off alias for brokkr

Browse Source 
vllm-lfm25 on ana-ml2 GPU1 :8021 (LiquidAI/LFM2.5-2.6B, util 0.09 into
unreserved slack, max-len 16384, no reasoning-parser so content is non-empty).
LiteLLM alias lfm2.5-2.6b with vendor sampling baked as default (temp 0.1;
top_k 50 + repetition_penalty 1.1 via extra_body). Eval-only, not in any
routing chain, pending operator ruling on LFM Open License production use.
This commit is contained in:
vh
2026-08-10 07:13:37 -07:00
parent c8acf60449
commit edc9f42da1
4 changed files with 100 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
persistent-memory.md
+1 -1
View File
@@ -119,7 +119,7 @@ _As of 2026-08-08 — long session; all major arcs LANDED (full detail per arc i
- **EVIDENCE HOLD (partial):** WT #394 index-row half LIFTED + swept (kb 8,230→2,876); the **FILE half STILL STANDS** — do NOT delete the on-disk generation dirs (`fiction/rex390-dcc`, `rex392-dcc`, `b59c147c5ce0`). Also un-actioned worldtree-side: the `reconcile --repair` rc=1 git-staging bug (phantom `docs/specs/saga-taxonomy-redesign-plan.md`) — flagged, their code.
- **OPEN LOOPS (2026-08-10):** (1) **dots-tts ratatoskr cutover** — draft the ask to repoint ratatoskr's RP TTS to irv-ml1:8198 `/v1/audio/speech` (Phase-2 tail; dots container LIVE, see Recent decisions). (2) **LFM2.5-2.6b bake-off alias for brokkr-smithy-dev** — arch confirmed (Lfm2ForCausalLM, vLLM 0.24.0 on ana-ml2 supports it, no bump), replied w/ alias `lfm2.5-2.6b` + plan (non-prod, vendor sampling temp0.1/topk50/reppen1.1, reasoning-channel content-non-empty smoke, LFM Open License = eval-only pending operator ruling); **stand-up QUEUED** (thread `01KZNA4MKPDR…`, brokkr said no-rush).
- **OPEN LOOPS (2026-08-10):** (1) **dots-tts ratatoskr cutover** — handoff SENT to ratatoskr-dev (thread `01KZNZF1…`: endpoint :8198, OpenAI `/v1/audio/speech` shape, 4 voices, streaming WAV format, the 48k/serialized-single-consumer/no-affect-dials caveats); **awaiting their cutover decision** (chatterbox :8197 kept up as their rollback). (2) **LFM2.5-2.6b bake-off for brokkr — DONE:** `vllm-lfm25` LIVE on ana-ml2 GPU1 :8021 (LiquidAI/LFM2.5-2.6B, Lfm2ForCausalLM/vLLM 0.24.0, BF16, util 0.09 ~8.7GB into GPU1's unreserved slack — no prod reservation touched; max-len 16384; NO `--reasoning-parser` so content non-empty). LiteLLM alias `lfm2.5-2.6b` (vendor sampling temp0.1 + topk50/reppen1.1 via `extra_body` so drop_params doesn't strip them). Smoke: content_len 636, reasoning_content 0, finish stop. brokkr notified (`01KZP08NB7…`). EVAL-ONLY pending operator production ruling (LFM Open License); NOT in any default/fallback chain. Backups: litellm `config.yaml.bak-pre-lfm25-20260810`; targeted `up -d vllm-lfm25` (shared-.env → never bare up-d). Repo: `stacks/vllm/` + `stacks/litellm/conf/config.yaml`.
- **OPEN FOLLOW-UPS:** chatterbox-fast **deployed flat-build-context vs package-repo divergence** (reconcile so a repo rebuild matches deploy — repo commit `6bc7bf0` has cap+norm_loudness in package layout; deployed is flat cap-only). Standing/parked: CI-flip runner-auth research, gitea `REQUIRE_SIGNIN_VIEW=false`, #363 research-wing ingest (no deadline), zonos-gateway CI-wire.