feat(nh3-ml1): abliterated LFM2.5-VL-3B parallel seat :8032 for brokkr's NSFW-caption A/B (direct only)
This commit is contained in:
@@ -53,8 +53,11 @@ VRAM ~2.7 GB for both, so ~13 GB is free.
|
|||||||
- `stacks/vibevoice-asr-seat`: VibeVoice-ASR-Streaming-1.5B on audio.cpp,
|
- `stacks/vibevoice-asr-seat`: VibeVoice-ASR-Streaming-1.5B on audio.cpp,
|
||||||
`:8031`, direct only.
|
`:8031`, direct only.
|
||||||
|
|
||||||
GPU total with TEI is ~7.5 of 16 GB. Both are batch workloads, which suits this
|
- `stacks/lfm-vl-uncensored-seat`: an abliterated LFM2.5-VL-3B running in
|
||||||
card.
|
parallel for brokkr's NSFW-caption A/B, `:8032`, direct only.
|
||||||
|
|
||||||
|
GPU total with TEI is ~10.2 of 16 GB. All of these are batch workloads, which
|
||||||
|
suits this card.
|
||||||
|
|
||||||
**Tried and removed (2026-09-25):** a copy of the code-completion seat
|
**Tried and removed (2026-09-25):** a copy of the code-completion seat
|
||||||
(Qwen2.5-Coder-1.5B). It gave the same quality but ran about 5× slower than on
|
(Qwen2.5-Coder-1.5B). It gave the same quality but ran about 5× slower than on
|
||||||
|
|||||||
@@ -0,0 +1,9 @@
|
|||||||
|
# lfm-vl-uncensored-seat tunables (nh3-ml1). Copy to `.env` on the server.
|
||||||
|
# Same llama.cpp build as the stock seat (b11176).
|
||||||
|
LLAMACPP_IMAGE=ghcr.io/ggml-org/llama.cpp@sha256:1f4b9cf58982dd4d7cc497aea31b1a456ca9a3a1f94f527d317d3fdee0d60ab6
|
||||||
|
VL_PORT=8032
|
||||||
|
VL_ALIAS=lfm25-vl-3b-uncensored
|
||||||
|
VL_MODEL_FILE=LFM2.5-VL-3B-Uncensored-Q4_K_M.gguf
|
||||||
|
VL_MMPROJ_FILE=mmproj-LFM2.5-VL-3B-Q8_0.gguf
|
||||||
|
VL_CTX=16384
|
||||||
|
VL_PARALLEL=2
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
# lfm-vl-uncensored-seat
|
||||||
|
|
||||||
|
The **abliterated LFM2.5-VL-3B** (`SC117/LFM2.5-VL-3B-Uncensored-GGUF`, Q4_K_M plus
|
||||||
|
its own mmproj Q8_0) on **nh3-ml1** `:8032`, direct only. It runs **in parallel**
|
||||||
|
with the stock seat (`stacks/lfm-vl-seat`, `:8030`) so brokkr can A/B them for
|
||||||
|
NSFW captioning. The stock model sanitized nudity and refused one image. The
|
||||||
|
operator wants an uncensored one (relayed by brokkr-smithy-dev, 2026-09-26).
|
||||||
|
|
||||||
|
- endpoint: `http://10.100.50.80:8032/v1/chat/completions`, model `lfm25-vl-3b-uncensored`
|
||||||
|
- same llama.cpp build (b11176) and flags as the stock seat. The mmproj is SC117's
|
||||||
|
own export, which differs slightly from LiquidAI's (different sha256) and is
|
||||||
|
used as shipped.
|
||||||
|
- positive control 2026-09-26: the synthetic shapes and text image was read
|
||||||
|
exactly, same as stock. Decode ~102 tok/s.
|
||||||
|
- "Uncensored" is the repo's claim (abliterix Trial 65). brokkr's 6-image eval
|
||||||
|
decides whether it engages and stays coherent. Fallbacks, if it fails, are in
|
||||||
|
brokkr's request: the Heretic builds by MuXodious or AX1Y2JP, or an in-house
|
||||||
|
Heretic pass.
|
||||||
|
- no gateway entry until that eval passes.
|
||||||
@@ -0,0 +1,60 @@
|
|||||||
|
# lfm-vl-uncensored-seat — ABLITERATED LFM2.5-VL-3B on nh3-ml1, running PARALLEL to
|
||||||
|
# the stock seat (stacks/lfm-vl-seat) so brokkr can A/B them. Stock tested censored
|
||||||
|
# on NSFW captioning; the operator wants an uncensored one (relayed by
|
||||||
|
# brokkr-smithy-dev, 2026-09-26). Same llama.cpp image and flags as the stock seat.
|
||||||
|
#
|
||||||
|
# llama-server SC117/LFM2.5-VL-3B-Uncensored-GGUF Q4_K_M + its mmproj Q8_0 :8032
|
||||||
|
#
|
||||||
|
# "Uncensored" is the repo's claim (abliterix Trial 65 merged into the official
|
||||||
|
# base). brokkr re-runs its 6-image eval against this seat to check it engages
|
||||||
|
# AND stays coherent. Direct only, no gateway entry, until that eval passes.
|
||||||
|
#
|
||||||
|
# Files (not in git): /opt/aimodels/gguf/lfm25-vl-3b-uncensored/ from
|
||||||
|
# SC117/LFM2.5-VL-3B-Uncensored-GGUF @ 7c1d3fb71e56.
|
||||||
|
name: lfm-vl-uncensored-seat
|
||||||
|
services:
|
||||||
|
llama-server:
|
||||||
|
image: ${LLAMACPP_IMAGE}
|
||||||
|
container_name: lfm-vl-uncensored
|
||||||
|
restart: unless-stopped
|
||||||
|
ports:
|
||||||
|
- "${VL_PORT}:8080"
|
||||||
|
volumes:
|
||||||
|
- /opt/aimodels/gguf/lfm25-vl-3b-uncensored:/models:ro
|
||||||
|
command:
|
||||||
|
- -m
|
||||||
|
- /models/${VL_MODEL_FILE}
|
||||||
|
- --mmproj
|
||||||
|
- /models/${VL_MMPROJ_FILE}
|
||||||
|
- --alias
|
||||||
|
- ${VL_ALIAS}
|
||||||
|
- --host
|
||||||
|
- 0.0.0.0
|
||||||
|
- --port
|
||||||
|
- "8080"
|
||||||
|
- -ngl
|
||||||
|
- "999"
|
||||||
|
- -c
|
||||||
|
- "${VL_CTX}"
|
||||||
|
- -np
|
||||||
|
- "${VL_PARALLEL}"
|
||||||
|
- --jinja
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
reservations:
|
||||||
|
devices:
|
||||||
|
- driver: nvidia
|
||||||
|
device_ids: ["0"]
|
||||||
|
capabilities: [gpu]
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "curl", "-fsS", "http://localhost:8080/health"]
|
||||||
|
interval: 30s
|
||||||
|
timeout: 10s
|
||||||
|
retries: 3
|
||||||
|
start_period: 120s
|
||||||
|
labels:
|
||||||
|
- homepage.group=AI - Eval & Retrieval
|
||||||
|
- homepage.name=VL — LFM2.5-VL-3B Uncensored (llama.cpp, nh3-ml1)
|
||||||
|
- homepage.icon=mdi-image-search
|
||||||
|
- homepage.description=Abliterated LFM2.5-VL-3B for the dataset foundry A/B (direct only)
|
||||||
|
- homepage.href=http://10.100.50.80:${VL_PORT}
|
||||||
Reference in New Issue
Block a user