diff --git a/persistent-memory.md b/persistent-memory.md index 898c419..f6bafab 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -119,7 +119,7 @@ _As of 2026-08-08 — long session; all major arcs LANDED (full detail per arc i - **EVIDENCE HOLD (partial):** WT #394 index-row half LIFTED + swept (kb 8,230→2,876); the **FILE half STILL STANDS** — do NOT delete the on-disk generation dirs (`fiction/rex390-dcc`, `rex392-dcc`, `b59c147c5ce0`). Also un-actioned worldtree-side: the `reconcile --repair` rc=1 git-staging bug (phantom `docs/specs/saga-taxonomy-redesign-plan.md`) — flagged, their code. -- **OPEN LOOPS (2026-08-10):** (1) **dots-tts ratatoskr cutover** — draft the ask to repoint ratatoskr's RP TTS to irv-ml1:8198 `/v1/audio/speech` (Phase-2 tail; dots container LIVE, see Recent decisions). (2) **LFM2.5-2.6b bake-off alias for brokkr-smithy-dev** — arch confirmed (Lfm2ForCausalLM, vLLM 0.24.0 on ana-ml2 supports it, no bump), replied w/ alias `lfm2.5-2.6b` + plan (non-prod, vendor sampling temp0.1/topk50/reppen1.1, reasoning-channel content-non-empty smoke, LFM Open License = eval-only pending operator ruling); **stand-up QUEUED** (thread `01KZNA4MKPDR…`, brokkr said no-rush). +- **OPEN LOOPS (2026-08-10):** (1) **dots-tts ratatoskr cutover** — handoff SENT to ratatoskr-dev (thread `01KZNZF1…`: endpoint :8198, OpenAI `/v1/audio/speech` shape, 4 voices, streaming WAV format, the 48k/serialized-single-consumer/no-affect-dials caveats); **awaiting their cutover decision** (chatterbox :8197 kept up as their rollback). (2) **LFM2.5-2.6b bake-off for brokkr — DONE:** `vllm-lfm25` LIVE on ana-ml2 GPU1 :8021 (LiquidAI/LFM2.5-2.6B, Lfm2ForCausalLM/vLLM 0.24.0, BF16, util 0.09 ~8.7GB into GPU1's unreserved slack — no prod reservation touched; max-len 16384; NO `--reasoning-parser` so content non-empty). LiteLLM alias `lfm2.5-2.6b` (vendor sampling temp0.1 + topk50/reppen1.1 via `extra_body` so drop_params doesn't strip them). Smoke: content_len 636, reasoning_content 0, finish stop. brokkr notified (`01KZP08NB7…`). EVAL-ONLY pending operator production ruling (LFM Open License); NOT in any default/fallback chain. Backups: litellm `config.yaml.bak-pre-lfm25-20260810`; targeted `up -d vllm-lfm25` (shared-.env → never bare up-d). Repo: `stacks/vllm/` + `stacks/litellm/conf/config.yaml`. - **OPEN FOLLOW-UPS:** chatterbox-fast **deployed flat-build-context vs package-repo divergence** (reconcile so a repo rebuild matches deploy — repo commit `6bc7bf0` has cap+norm_loudness in package layout; deployed is flat cap-only). Standing/parked: CI-flip runner-auth research, gitea `REQUIRE_SIGNIN_VIEW=false`, #363 research-wing ingest (no deadline), zonos-gateway CI-wire. diff --git a/stacks/litellm/conf/config.yaml b/stacks/litellm/conf/config.yaml index 871d031..7b1ab2e 100644 --- a/stacks/litellm/conf/config.yaml +++ b/stacks/litellm/conf/config.yaml @@ -426,6 +426,23 @@ model_list: # model names. Removed so unknown models now fail loudly (404). Re-add an # explicit per-model entry if a swappable zoo ever returns. --- + # --- lfm2.5-2.6b -> LiquidAI LFM2.5-2.6B (ana-ml2 GPU1 :8021, vLLM). NON-PROD bake-off + # vs granite-4.1-8b (brokkr R-target 2026-08-10). LFM Open License v1.0 (); served raw (no vLLM +# reasoning-parser) so content is non-empty. util 0.09 (~8.6GB) fits GPU1's free slack +# without touching production reservations; max-len 16384 keeps KV small (bake-off +# doesn't need the 131k ceiling). Vendor sampling lives in the LiteLLM alias. +LFM25_PORT=8021 +LFM25_GPU_ID=1 +LFM25_MODEL=LiquidAI/LFM2.5-2.6B +LFM25_SERVED_NAME=lfm2.5-2.6b +LFM25_MAX_MODEL_LEN=16384 +LFM25_KV_CACHE_DTYPE=auto +LFM25_GPU_MEM_UTIL=0.09 +LFM25_MAX_NUM_SEQS=8 diff --git a/stacks/vllm/compose.yaml b/stacks/vllm/compose.yaml index 3461041..0b39196 100644 --- a/stacks/vllm/compose.yaml +++ b/stacks/vllm/compose.yaml @@ -342,6 +342,73 @@ services: - homepage.description=Qwen2.5-Coder-1.5B FIM code-completion (ana-ml2, Zed edit-predictions) - homepage.href=http://10.250.50.54:${CODER_PORT}/docs + vllm-lfm25: + image: vllm/vllm-openai:${VLLM_VERSION} + container_name: vllm-lfm25 + restart: unless-stopped + ipc: host + ports: + - "${LFM25_PORT}:8000" + volumes: + - /tank/aimodels/huggingface:/hfcache + environment: + - HF_HOME=/hfcache + - HF_HUB_CACHE=/hfcache/hub + - HUGGING_FACE_HUB_TOKEN=${HF_TOKEN:-} + - VLLM_API_KEY=${API_KEY:-} + command: + # LiquidAI/LFM2.5-2.6B (BF16, Lfm2ForCausalLM). NON-PRODUCTION bake-off alias + # vs granite-4.1-8b on structured extraction / classification / tool-call + # formatting (brokkr R-target, 2026-08-10). LFM Open License v1.0 (; deliberately NO --reasoning-parser, so the full + # generation (thinking + answer) lands in `content` non-empty — brokkr's explicit + # requirement (an empty content with the answer stranded in reasoning_content + # reads as a degenerate model). Vendor sampling (temp 0.1 / top_k 50 / rep_pen 1.1) + # is the LiteLLM alias default, not a launch arg. + - ${LFM25_MODEL} + - --served-model-name + - ${LFM25_SERVED_NAME} + - --host + - 0.0.0.0 + - --port + - "8000" + - --gpu-memory-utilization + - ${LFM25_GPU_MEM_UTIL} + - --max-model-len + - ${LFM25_MAX_MODEL_LEN} + - --max-num-seqs + - ${LFM25_MAX_NUM_SEQS} + - --dtype + - auto + - --kv-cache-dtype + - ${LFM25_KV_CACHE_DTYPE} + - --enable-prefix-caching + deploy: + resources: + reservations: + devices: + - driver: nvidia + device_ids: + - "${LFM25_GPU_ID}" + capabilities: + - gpu + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:8000/health"] + interval: 30s + timeout: 10s + retries: 3 + start_period: 300s + networks: + - tnet + labels: + - homepage.group=AI - Inference + - homepage.name=vLLM LFM2.5-2.6B (bake-off) + - homepage.icon=mdi-flask-outline + - homepage.description=LiquidAI LFM2.5-2.6B non-prod bake-off vs granite (ana-ml2) + - homepage.href=http://10.250.50.54:${LFM25_PORT}/docs + networks: tnet: name: traefik-net