feat(nh3-ml1): LFM2.5-VL-3B (llama.cpp) + VibeVoice-ASR-Streaming-1.5B (audio.cpp) utility seats

For brokkr's dataset foundry (operator-approved 2026-09-26, relayed). - stacks/lfm-vl-seat: llama.cpp server-cuda b11176 (digest-pinned), Q5_K_M + mmproj Q8_0, :8030; gateway alias lfm25-vl-3b (LiteLLM restarted, 36 s). Positive control exact; null control shows it describes a missing image. - stacks/vibevoice-asr-seat: audio.cpp v0.8.2-audio8-perf-hotfix (the GGUF's own runtime, not vibevoice.cpp) on cuda 12.8 runtime + libgomp + libsoxr, sha256-pinned; :8031 direct. LibriSpeech WER 3/69, RTF 0.07-0.14; ~31 s cold first request.
2026-09-26 00:41:01 -07:00
parent 45484a0007
commit f2792183d4
11 changed files with 235 additions and 0 deletions
@@ -0,0 +1,14 @@
+# lfm-vl-seat tunables (nh3-ml1). Copy to `.env` on the server.
+
+# llama.cpp CUDA server, pinned by digest: build b11176 (commit f805c57a2),
+# published 2026-09-25. Ada sm_89 is in the CUDA build's target set.
+LLAMACPP_IMAGE=ghcr.io/ggml-org/llama.cpp@sha256:1f4b9cf58982dd4d7cc497aea31b1a456ca9a3a1f94f527d317d3fdee0d60ab6
+
+VL_PORT=8030
+VL_ALIAS=lfm25-vl-3b
+VL_MODEL_FILE=LFM2.5-VL-3B-Q5_K_M.gguf
+VL_MMPROJ_FILE=mmproj-LFM2.5-VL-3B-Q8_0.gguf
+
+# 2 parallel slots sharing 16k context (8k each): room for a few images plus text.
+VL_CTX=16384
+VL_PARALLEL=2
@@ -0,0 +1,23 @@
+# lfm-vl-seat
+
+**LiquidAI LFM2.5-VL-3B**, a small vision-language model, on **nh3-ml1**, served
+by llama.cpp (`ghcr.io/ggml-org/llama.cpp` server-cuda, build b11176, pinned by
+digest) on `:8030`. A utility seat for brokkr's dataset foundry (image
+understanding). Operator-approved 2026-09-26, relayed by brokkr-smithy-dev.
+
+| | |
+|---|---|
+| direct | `http://10.100.50.80:8030/v1/chat/completions`, model `lfm25-vl-3b` |
+| gateway | `lfm25-vl-3b` (ana-docker:4000, `hosted_vllm/`, `supports_vision`) |
+| files | `LFM2.5-VL-3B-Q5_K_M.gguf` + **`mmproj-LFM2.5-VL-3B-Q8_0.gguf` (required for images)** from `LiquidAI/LFM2.5-VL-3B-GGUF` @ `6f730e9a2c45` |
+| VRAM | ~2.9 GB (2 slots × 8k ctx) |
+
+**Checks, 2026-09-26.**
+- Positive control: a synthetic image (red square, blue circle, the text
+  "NH3 42") was described exactly at T=0, both direct and through the gateway.
+- Speed: prefill ~800 tok/s, decode ~92 tok/s.
+- ⚠ **Null control: with no image attached it confidently describes one anyway**
+  ("a solid black background"). Callers must make sure the image actually went in.
+
+Not wired yet: `LiquidAI/LFM2.5-VL-3B-DSpark-GGUF`, a speculative-decoding
+drafter (brokkr's "optional later").
@@ -0,0 +1,62 @@
+# lfm-vl-seat — LiquidAI LFM2.5-VL-3B (vision-language) on nh3-ml1 (CT 109 on
+# nh3-pve, RTX 2000E Ada 16 GB), llama.cpp server. Utility seat for brokkr's
+# dataset foundry (image understanding), operator-approved 2026-09-26 (relayed
+# by brokkr-smithy-dev).
+#
+#   llama-server  LFM2.5-VL-3B Q5_K_M + mmproj Q8_0 → /v1/chat/completions
+#                 (OpenAI, with image_url content parts)                 :8030
+#
+# ⚠ --mmproj is REQUIRED for image input; without it the model is text-only and
+# says nothing about it.
+# Batch work, not latency-critical: this card decodes ~5x slower than an fv-ml1
+# Blackwell (servers/nh3-ml1/README.md), which is fine for a foundry.
+#
+# Files (not in git): /opt/aimodels/gguf/lfm25-vl-3b/ from
+# LiquidAI/LFM2.5-VL-3B-GGUF @ 6f730e9a2c45.
+name: lfm-vl-seat
+services:
+  llama-server:
+    image: ${LLAMACPP_IMAGE}
+    container_name: lfm-vl
+    restart: unless-stopped
+    ports:
+      - "${VL_PORT}:8080"
+    volumes:
+      - /opt/aimodels/gguf/lfm25-vl-3b:/models:ro
+    command:
+      - -m
+      - /models/${VL_MODEL_FILE}
+      - --mmproj
+      - /models/${VL_MMPROJ_FILE}
+      - --alias
+      - ${VL_ALIAS}
+      - --host
+      - 0.0.0.0
+      - --port
+      - "8080"
+      - -ngl
+      - "999"
+      - -c
+      - "${VL_CTX}"
+      - -np
+      - "${VL_PARALLEL}"
+      - --jinja
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              device_ids: ["0"]
+              capabilities: [gpu]
+    healthcheck:
+      test: ["CMD", "curl", "-fsS", "http://localhost:8080/health"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 120s
+    labels:
+      - homepage.group=AI - Eval & Retrieval
+      - homepage.name=VL — LFM2.5-VL-3B (llama.cpp, nh3-ml1)
+      - homepage.icon=mdi-image-search
+      - homepage.description=Image understanding for the dataset foundry (gateway lfm25-vl-3b)
+      - homepage.href=http://10.100.50.80:${VL_PORT}