vh
f2792183d4
For brokkr's dataset foundry (operator-approved 2026-09-26, relayed). - stacks/lfm-vl-seat: llama.cpp server-cuda b11176 (digest-pinned), Q5_K_M + mmproj Q8_0, :8030; gateway alias lfm25-vl-3b (LiteLLM restarted, 36 s). Positive control exact; null control shows it describes a missing image. - stacks/vibevoice-asr-seat: audio.cpp v0.8.2-audio8-perf-hotfix (the GGUF's own runtime, not vibevoice.cpp) on cuda 12.8 runtime + libgomp + libsoxr, sha256-pinned; :8031 direct. LibriSpeech WER 3/69, RTF 0.07-0.14; ~31 s cold first request.
24 lines
1.2 KiB
Markdown
24 lines
1.2 KiB
Markdown
# lfm-vl-seat
|
||
|
||
**LiquidAI LFM2.5-VL-3B**, a small vision-language model, on **nh3-ml1**, served
|
||
by llama.cpp (`ghcr.io/ggml-org/llama.cpp` server-cuda, build b11176, pinned by
|
||
digest) on `:8030`. A utility seat for brokkr's dataset foundry (image
|
||
understanding). Operator-approved 2026-09-26, relayed by brokkr-smithy-dev.
|
||
|
||
| | |
|
||
|---|---|
|
||
| direct | `http://10.100.50.80:8030/v1/chat/completions`, model `lfm25-vl-3b` |
|
||
| gateway | `lfm25-vl-3b` (ana-docker:4000, `hosted_vllm/`, `supports_vision`) |
|
||
| files | `LFM2.5-VL-3B-Q5_K_M.gguf` + **`mmproj-LFM2.5-VL-3B-Q8_0.gguf` (required for images)** from `LiquidAI/LFM2.5-VL-3B-GGUF` @ `6f730e9a2c45` |
|
||
| VRAM | ~2.9 GB (2 slots × 8k ctx) |
|
||
|
||
**Checks, 2026-09-26.**
|
||
- Positive control: a synthetic image (red square, blue circle, the text
|
||
"NH3 42") was described exactly at T=0, both direct and through the gateway.
|
||
- Speed: prefill ~800 tok/s, decode ~92 tok/s.
|
||
- ⚠ **Null control: with no image attached it confidently describes one anyway**
|
||
("a solid black background"). Callers must make sure the image actually went in.
|
||
|
||
Not wired yet: `LiquidAI/LFM2.5-VL-3B-DSpark-GGUF`, a speculative-decoding
|
||
drafter (brokkr's "optional later").
|