vh
f2792183d4
For brokkr's dataset foundry (operator-approved 2026-09-26, relayed). - stacks/lfm-vl-seat: llama.cpp server-cuda b11176 (digest-pinned), Q5_K_M + mmproj Q8_0, :8030; gateway alias lfm25-vl-3b (LiteLLM restarted, 36 s). Positive control exact; null control shows it describes a missing image. - stacks/vibevoice-asr-seat: audio.cpp v0.8.2-audio8-perf-hotfix (the GGUF's own runtime, not vibevoice.cpp) on cuda 12.8 runtime + libgomp + libsoxr, sha256-pinned; :8031 direct. LibriSpeech WER 3/69, RTF 0.07-0.14; ~31 s cold first request.
15 lines
554 B
Bash
15 lines
554 B
Bash
# lfm-vl-seat tunables (nh3-ml1). Copy to `.env` on the server.
|
|
|
|
# llama.cpp CUDA server, pinned by digest: build b11176 (commit f805c57a2),
|
|
# published 2026-09-25. Ada sm_89 is in the CUDA build's target set.
|
|
LLAMACPP_IMAGE=ghcr.io/ggml-org/llama.cpp@sha256:1f4b9cf58982dd4d7cc497aea31b1a456ca9a3a1f94f527d317d3fdee0d60ab6
|
|
|
|
VL_PORT=8030
|
|
VL_ALIAS=lfm25-vl-3b
|
|
VL_MODEL_FILE=LFM2.5-VL-3B-Q5_K_M.gguf
|
|
VL_MMPROJ_FILE=mmproj-LFM2.5-VL-3B-Q8_0.gguf
|
|
|
|
# 2 parallel slots sharing 16k context (8k each): room for a few images plus text.
|
|
VL_CTX=16384
|
|
VL_PARALLEL=2
|