vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(vllm): lfm2.5 reasoning-parser (deepseek_r1) — scoreable JSON for brokkr's bake-off

Browse Source 
LFM2.5 is </think>-delimited (opening tag in prompt); deepseek_r1 splits
reasoning into reasoning_content so content is the clean post-</think>
answer. Re-smoke: content valid JSON + reasoning_content populated. License
production-cleared (operator <$10M ruling), still out of routing per the
measurement gate.
This commit is contained in:
vh
2026-08-10 07:27:36 -07:00
parent bb65f36f70
commit d3727dee53
2 changed files with 7 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
persistent-memory.md
+1 -1
View File
@@ -119,7 +119,7 @@ _As of 2026-08-08 — long session; all major arcs LANDED (full detail per arc i
- **EVIDENCE HOLD (partial):** WT #394 index-row half LIFTED + swept (kb 8,230→2,876); the **FILE half STILL STANDS** — do NOT delete the on-disk generation dirs (`fiction/rex390-dcc`, `rex392-dcc`, `b59c147c5ce0`). Also un-actioned worldtree-side: the `reconcile --repair` rc=1 git-staging bug (phantom `docs/specs/saga-taxonomy-redesign-plan.md`) — flagged, their code.
- **OPEN LOOPS (2026-08-10):** (1) **dots-tts ratatoskr cutover — COMPLETE.** Operator ear-confirmed on the live RP surface (:8765); **dots-tts v0.22.2 is ratatoskr's PRIMARY RP voice** (tts.py seam → OpenAI `/v1/audio/speech`, SR→48k, default voice→glados, donut carries; 520 tests green; their one-synth-per-turn lock DEC-5 matches the server's serialized model). ratatoskr RELEASED chatterbox-fast :8197. **chatterbox-fast RECLAIMED (operator go 2026-08-10):** `docker compose stop` on irv-ml1 freed ~7GB on the 3090 (GPU0 16.0→9.1GB); container Exited(0) — `restart: unless-stopped` honors the manual stop so it stays down across reboots; revive with `docker start chatterbox-fast`. dots-tts unaffected. **dots.tts migration FULLY CLOSED.** thread `01KZNZF1…`. (2) **LFM2.5-2.6b bake-off for brokkr — DONE:** `vllm-lfm25` LIVE on ana-ml2 GPU1 :8021 (LiquidAI/LFM2.5-2.6B, Lfm2ForCausalLM/vLLM 0.24.0, BF16, util 0.09 ~8.7GB into GPU1's unreserved slack — no prod reservation touched; max-len 16384; NO `--reasoning-parser` so content non-empty). LiteLLM alias `lfm2.5-2.6b` (vendor sampling temp0.1 + topk50/reppen1.1 via `extra_body` so drop_params doesn't strip them). Smoke: content_len 636, reasoning_content 0, finish stop. brokkr notified (`01KZP08NB7…`). EVAL-ONLY pending operator production ruling (LFM Open License); NOT in any default/fallback chain. Backups: litellm `config.yaml.bak-pre-lfm25-20260810`; targeted `up -d vllm-lfm25` (shared-.env → never bare up-d). Repo: `stacks/vllm/` + `stacks/litellm/conf/config.yaml`.
- **OPEN LOOPS (2026-08-10):** (1) **dots-tts ratatoskr cutover — COMPLETE.** Operator ear-confirmed on the live RP surface (:8765); **dots-tts v0.22.2 is ratatoskr's PRIMARY RP voice** (tts.py seam → OpenAI `/v1/audio/speech`, SR→48k, default voice→glados, donut carries; 520 tests green; their one-synth-per-turn lock DEC-5 matches the server's serialized model). ratatoskr RELEASED chatterbox-fast :8197. **chatterbox-fast RECLAIMED (operator go 2026-08-10):** `docker compose stop` on irv-ml1 freed ~7GB on the 3090 (GPU0 16.0→9.1GB); container Exited(0) — `restart: unless-stopped` honors the manual stop so it stays down across reboots; revive with `docker start chatterbox-fast`. dots-tts unaffected. **dots.tts migration FULLY CLOSED.** thread `01KZNZF1…`. (2) **LFM2.5-2.6b bake-off for brokkr — DONE:** `vllm-lfm25` LIVE on ana-ml2 GPU1 :8021 (LiquidAI/LFM2.5-2.6B, Lfm2ForCausalLM/vLLM 0.24.0, BF16, util 0.09 ~8.7GB into GPU1's unreserved slack — no prod reservation touched; max-len 16384; NO `--reasoning-parser` so content non-empty). LiteLLM alias `lfm2.5-2.6b` (vendor sampling temp0.1 + topk50/reppen1.1 via `extra_body` so drop_params doesn't strip them). **PRODUCTION-CLEARED 2026-08-10** (operator ruled they're <USD 10M → LFM Open License permits commercial use; eval-only hold dropped) — but STILL kept OUT of any default/fallback routing chain per brokkr's *measurement* gate (nothing's benched it; lifts when the bake-off says). **`--reasoning-parser deepseek_r1` ADDED** (LFM2.5 is `</think>`-delimited: opening `<think>` injected by chat template into the prompt, output = `reasoning…</think>answer`; deepseek_r1 splits on `</think>`). Re-smoke: content=valid JSON (51 chars, `json.loads` passes) + reasoning_content=824 chars → both halves of brokkr's gate met (scoreable + non-empty). ⚠ model wraps its answer in a ```json fence — a bare `json.loads(content)` chokes; scorer must strip the fence. brokkr notified + clear to run (`01KZP142…`). Backups: litellm `config.yaml.bak-pre-lfm25-20260810`; targeted `up -d vllm-lfm25` (shared-.env → never bare up-d). Repo: `stacks/vllm/` + `stacks/litellm/conf/config.yaml`.
- **OPEN FOLLOW-UPS:** chatterbox-fast **deployed flat-build-context vs package-repo divergence** (reconcile so a repo rebuild matches deploy — repo commit `6bc7bf0` has cap+norm_loudness in package layout; deployed is flat cap-only). Standing/parked: CI-flip runner-auth research, gitea `REQUIRE_SIGNIN_VIEW=false`, #363 research-wing ingest (no deadline), zonos-gateway CI-wire.