feat(nh3-ml1): abliterated LFM2.5-VL-3B parallel seat :8032 for brokkr's NSFW-caption A/B (direct only)

2026-09-26 00:57:04 -07:00
parent 50c85e0e8a
commit 9cc3824aec
4 changed files with 93 additions and 2 deletions
@@ -53,8 +53,11 @@ VRAM ~2.7 GB for both, so ~13 GB is free.
 - `stacks/vibevoice-asr-seat`: VibeVoice-ASR-Streaming-1.5B on audio.cpp,
  `:8031`, direct only.

-GPU total with TEI is ~7.5 of 16 GB. Both are batch workloads, which suits this
-card.
+- `stacks/lfm-vl-uncensored-seat`: an abliterated LFM2.5-VL-3B running in
+  parallel for brokkr's NSFW-caption A/B, `:8032`, direct only.
+
+GPU total with TEI is ~10.2 of 16 GB. All of these are batch workloads, which
+suits this card.

 **Tried and removed (2026-09-25):** a copy of the code-completion seat
 (Qwen2.5-Coder-1.5B). It gave the same quality but ran about 5× slower than on
@@ -0,0 +1,9 @@
+# lfm-vl-uncensored-seat tunables (nh3-ml1). Copy to `.env` on the server.
+# Same llama.cpp build as the stock seat (b11176).
+LLAMACPP_IMAGE=ghcr.io/ggml-org/llama.cpp@sha256:1f4b9cf58982dd4d7cc497aea31b1a456ca9a3a1f94f527d317d3fdee0d60ab6
+VL_PORT=8032
+VL_ALIAS=lfm25-vl-3b-uncensored
+VL_MODEL_FILE=LFM2.5-VL-3B-Uncensored-Q4_K_M.gguf
+VL_MMPROJ_FILE=mmproj-LFM2.5-VL-3B-Q8_0.gguf
+VL_CTX=16384
+VL_PARALLEL=2
@@ -0,0 +1,19 @@
+# lfm-vl-uncensored-seat
+
+The **abliterated LFM2.5-VL-3B** (`SC117/LFM2.5-VL-3B-Uncensored-GGUF`, Q4_K_M plus
+its own mmproj Q8_0) on **nh3-ml1** `:8032`, direct only. It runs **in parallel**
+with the stock seat (`stacks/lfm-vl-seat`, `:8030`) so brokkr can A/B them for
+NSFW captioning. The stock model sanitized nudity and refused one image. The
+operator wants an uncensored one (relayed by brokkr-smithy-dev, 2026-09-26).
+
+- endpoint: `http://10.100.50.80:8032/v1/chat/completions`, model `lfm25-vl-3b-uncensored`
+- same llama.cpp build (b11176) and flags as the stock seat. The mmproj is SC117's
+  own export, which differs slightly from LiquidAI's (different sha256) and is
+  used as shipped.
+- positive control 2026-09-26: the synthetic shapes and text image was read
+  exactly, same as stock. Decode ~102 tok/s.
+- "Uncensored" is the repo's claim (abliterix Trial 65). brokkr's 6-image eval
+  decides whether it engages and stays coherent. Fallbacks, if it fails, are in
+  brokkr's request: the Heretic builds by MuXodious or AX1Y2JP, or an in-house
+  Heretic pass.
+- no gateway entry until that eval passes.
@@ -0,0 +1,60 @@
+# lfm-vl-uncensored-seat — ABLITERATED LFM2.5-VL-3B on nh3-ml1, running PARALLEL to
+# the stock seat (stacks/lfm-vl-seat) so brokkr can A/B them. Stock tested censored
+# on NSFW captioning; the operator wants an uncensored one (relayed by
+# brokkr-smithy-dev, 2026-09-26). Same llama.cpp image and flags as the stock seat.
+#
+#   llama-server  SC117/LFM2.5-VL-3B-Uncensored-GGUF Q4_K_M + its mmproj Q8_0  :8032
+#
+# "Uncensored" is the repo's claim (abliterix Trial 65 merged into the official
+# base). brokkr re-runs its 6-image eval against this seat to check it engages
+# AND stays coherent. Direct only, no gateway entry, until that eval passes.
+#
+# Files (not in git): /opt/aimodels/gguf/lfm25-vl-3b-uncensored/ from
+# SC117/LFM2.5-VL-3B-Uncensored-GGUF @ 7c1d3fb71e56.
+name: lfm-vl-uncensored-seat
+services:
+  llama-server:
+    image: ${LLAMACPP_IMAGE}
+    container_name: lfm-vl-uncensored
+    restart: unless-stopped
+    ports:
+      - "${VL_PORT}:8080"
+    volumes:
+      - /opt/aimodels/gguf/lfm25-vl-3b-uncensored:/models:ro
+    command:
+      - -m
+      - /models/${VL_MODEL_FILE}
+      - --mmproj
+      - /models/${VL_MMPROJ_FILE}
+      - --alias
+      - ${VL_ALIAS}
+      - --host
+      - 0.0.0.0
+      - --port
+      - "8080"
+      - -ngl
+      - "999"
+      - -c
+      - "${VL_CTX}"
+      - -np
+      - "${VL_PARALLEL}"
+      - --jinja
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              device_ids: ["0"]
+              capabilities: [gpu]
+    healthcheck:
+      test: ["CMD", "curl", "-fsS", "http://localhost:8080/health"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 120s
+    labels:
+      - homepage.group=AI - Eval & Retrieval
+      - homepage.name=VL — LFM2.5-VL-3B Uncensored (llama.cpp, nh3-ml1)
+      - homepage.icon=mdi-image-search
+      - homepage.description=Abliterated LFM2.5-VL-3B for the dataset foundry A/B (direct only)
+      - homepage.href=http://10.100.50.80:${VL_PORT}