From 3132a16ca0e09df39b6ce0af340acdde3d610681 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Sat, 12 Sep 2026 23:05:29 -0700 Subject: [PATCH] fv-ml1: finish the renumber the cutover missed -- 16 dead dashboard links Every fv-ml1 link on the Homepage dashboard was broken. Measured against the live dashboard API before the fix: 16 entries pointing at the dead 10.250.50.54 and zero at the live 10.251.50.54, covering gen, M.O.G.-SEC, Scriberr, Embed, Rerank, Reward, Coder, Dockge and six dormant seats. The miss was structural, not careless. fv-ml1-rename-sweep.sh works from an allowlist assembled from files that mention the HOST, and a homepage.href label mentions only an IP -- so every stack whose sole stale reference was a label fell outside it. The allowlist now covers those 24 files, and records how to derive the list next time (grep the old address, subtract history) rather than enumerating from memory. History is still untouched, and the exclusions are now written down with the reason each one keeps the old address: recorded benchmark results, whose base_url is part of a measurement's provenance; the one LiteLLM comment preserving a retired hand-test endpoint; and the cutover runbooks, where the old address is the subject matter. Two bugs found while applying it, both fixed here: - deploy-stack.sh rejected any stack name containing a dot, so qwen3.5-122b, qwopus3.5-122b and mistral-medium-3.5 could not be deployed by the script at all. The check exists to stop path traversal, which means rejecting ".." and "/" -- not every dot. Traversal is now rejected explicitly and tested. - stacks/scriberr/.env.example allowed CORS only from the dead IP and from scriberr.ana.internal, which no longer resolves; the box is at the fv site and DNS already carries scriberr.fv.internal. The live .env had both stale origins, i.e. an allowlist with nothing reachable in it. Host side, applied separately: canonical pushed for the 16 stacks whose only difference from the host was this renumber, and an in-place address-only fix for the nine whose host copy has genuinely drifted or has no canonical copy, so that drift survives for a deliberate reconciliation instead of being clobbered. Every compose.yaml on fv-ml1 now reads 10.251.50.54. The labels themselves only take effect at container creation, so the running containers still need recreating. --- scripts/deploy-stack.sh | 11 ++++- scripts/fv-ml1-rename-sweep.sh | 48 +++++++++++++++++++ stacks/char-rp-gguf/README.md | 14 +++--- stacks/char-rp-gguf/compose.yaml | 10 ++-- .../compose.yaml | 6 +-- stacks/dockge/README.md | 8 ++-- stacks/dozzle/.env.example | 8 ++-- stacks/dozzle/README.md | 18 +++---- stacks/fablefusion-charrp-probe/README.md | 12 ++--- stacks/gen-seat/README.md | 6 +-- stacks/gen-seat/compose.yaml | 6 +-- stacks/heretic2-charrp-reasoning/compose.yaml | 6 +-- stacks/litellm/README.md | 6 +-- stacks/mistral-small-4-heretic/compose.yaml | 4 +- stacks/mistral-small-4/compose.yaml | 6 +-- stacks/mog-sec/compose.yaml | 6 +-- stacks/qwen-image-bench/compose.yaml | 6 +-- stacks/qwen3.5-122b/compose.yaml | 6 +-- stacks/qwen36-vl/compose.yaml | 6 +-- stacks/qwopus3.5-122b/compose.yaml | 6 +-- stacks/scriberr/.env.example | 4 +- stacks/scriberr/README.md | 14 +++--- stacks/scriberr/compose.yaml | 10 ++-- stacks/selene/compose.yaml | 6 +-- stacks/sglang/README.md | 10 ++-- stacks/vllm/compose.yaml | 26 +++++----- 26 files changed, 163 insertions(+), 106 deletions(-) diff --git a/scripts/deploy-stack.sh b/scripts/deploy-stack.sh index 3bb8793..7df28e2 100755 --- a/scripts/deploy-stack.sh +++ b/scripts/deploy-stack.sh @@ -90,7 +90,16 @@ done [ -n "$HOST" ] || { echo "usage: $(basename "$0") " >&2; exit 2; } [ -n "$STACK" ] || { echo "usage: $(basename "$0") " >&2; exit 2; } DEST_STACK=${DEST_STACK:-$STACK} -[[ "$DEST_STACK" =~ ^[a-zA-Z0-9][a-zA-Z0-9_-]*$ ]] || { echo "invalid DEPLOY_DEST_STACK" >&2; exit 2; } +# Dots ARE legal — several stacks carry a version in the name (qwen3.5-122b, +# qwopus3.5-122b, mistral-medium-3.5). The previous pattern excluded them, so +# those stacks could not be deployed by this script AT ALL; it surfaced on +# 2026-09-13 as "invalid DEPLOY_DEST_STACK" during the fv-ml1 label repoint. +# The check exists to stop path traversal and shell metacharacters, which means +# it has to reject `..` and `/` — not every dot. +case "$DEST_STACK" in + *..*|*/*) echo "invalid stack name (path traversal): $DEST_STACK" >&2; exit 2 ;; +esac +[[ "$DEST_STACK" =~ ^[a-zA-Z0-9][a-zA-Z0-9_.-]*$ ]] || { echo "invalid stack name: $DEST_STACK" >&2; exit 2; } resolve_target() { # ssh-target file wins when present (may carry user@ or non-default port); diff --git a/scripts/fv-ml1-rename-sweep.sh b/scripts/fv-ml1-rename-sweep.sh index 1dbf876..758d5b9 100755 --- a/scripts/fv-ml1-rename-sweep.sh +++ b/scripts/fv-ml1-rename-sweep.sh @@ -40,10 +40,58 @@ ALLOW=( stacks/erp-seat/compose.yaml stacks/sglang/compose.yaml stacks/heretic2-charrp-reasoning/README.md + # --- ADDED 2026-09-13. The first pass's allowlist missed every stack whose only + # --- stale reference was a `homepage.href` LABEL, and the result was measurable: + # --- 16 live Homepage entries pointing at the dead 10.250.50.54 and ZERO at the + # --- live 10.251.50.54, i.e. every fv-ml1 dashboard link broken. The miss was + # --- structural, not careless -- the original list was assembled from files that + # --- mention the HOST, and a homepage.href mentions only an IP. If this script is + # --- ever reused for another relocation, derive the list from + # --- grep -rl '' stacks/ docs/ services/ + # --- and subtract history, rather than enumerating from memory. + # --- ⚠ A label edit needs `docker compose up -d `, not `restart`: labels are + # --- only applied at container CREATION. Editing the file changes nothing live. + stacks/char-rp-gguf/compose.yaml + stacks/char-rp-gguf/README.md + stacks/darkscarlett-charrp-reasoning/compose.yaml + stacks/dockge/README.md + stacks/dozzle/.env.example + stacks/dozzle/README.md + stacks/fablefusion-charrp-probe/README.md + stacks/gen-seat/compose.yaml + stacks/gen-seat/README.md + stacks/heretic2-charrp-reasoning/compose.yaml + stacks/litellm/README.md + stacks/mistral-small-4/compose.yaml + stacks/mistral-small-4-heretic/compose.yaml + stacks/mog-sec/compose.yaml + stacks/qwen-image-bench/compose.yaml + stacks/qwen3.5-122b/compose.yaml + stacks/qwen36-vl/compose.yaml + stacks/qwopus3.5-122b/compose.yaml + stacks/scriberr/compose.yaml + stacks/scriberr/README.md + stacks/scriberr/.env.example + stacks/selene/compose.yaml + stacks/sglang/README.md + stacks/vllm/compose.yaml ) # --- NEVER touched: history, provenance, the mirror ------------------------- # persistent-memory.md, persistent-memory.d/, archival-memory.md, # **/*.PROVENANCE.txt (on the box, not the repo), graphify-out/, stacks-mirror/ +# +# Also deliberately excluded, verified 2026-09-13 -- each of these still contains +# 10.250.50.54 and each SHOULD: +# services/*/bench/*.json, services/refusal-probe/results-*/raw-*.json +# recorded benchmark results; the base_url IS part of the measurement's +# provenance. Repointing it would claim a run happened somewhere it did not. +# stacks/litellm/conf/config.yaml (one comment) +# preserves a past hand-test endpoint, :8099, which no longer exists anywhere. +# docs/runbooks/fv-ml1-cutover.md, docs/runbooks/fv-onsite-playbook.md +# the old address is the SUBJECT MATTER of those runbooks. +# configs/homepage/ +# a dead April-era duplicate of stacks/homepage/conf/ -- nothing reads it. +# It needs deleting, not repointing, which is a separate decision. echo "=== 1. rename the server dir ===" if [ -d servers/ana-ml2 ]; then run "git mv servers/ana-ml2 servers/fv-ml1"; else say "servers/ana-ml2 already moved"; fi diff --git a/stacks/char-rp-gguf/README.md b/stacks/char-rp-gguf/README.md index 6097748..e85bb9d 100644 --- a/stacks/char-rp-gguf/README.md +++ b/stacks/char-rp-gguf/README.md @@ -1,6 +1,6 @@ # char-rp-gguf -Dedicated GGUF **character-RP seat** on ana-ml2 GPU 0, served via llama.cpp +Dedicated GGUF **character-RP seat** on fv-ml1 GPU 0, served via llama.cpp (`llama-server`) — **never Ollama**. Replaces the broken `ms32-24b-angel` NVFP4 serve (self-quant produced garbage even at greedy). @@ -47,8 +47,8 @@ Canonical copies live here; authoritative copies live on the host under `/opt/docker/compose/char-rp-gguf/`. ```bash -# On ana-ml2 (models already pre-pulled to /tank/aimodels/llm/rp/): -scripts/deploy-stack.sh ana-ml2 char-rp-gguf # from this workspace +# On fv-ml1 (models already pre-pulled to /tank/aimodels/llm/rp/): +scripts/deploy-stack.sh fv-ml1 char-rp-gguf # from this workspace # or, on the host: cd /opt/docker/compose/char-rp-gguf && cp .env.example .env && docker compose up -d ``` @@ -64,8 +64,8 @@ docker run -d --entrypoint python3 -v /tank:/tank vllm/vllm-openai:latest -c \ ## Gateway wiring (ana-docker LiteLLM `:4000`) -`char-rp` → `http://10.250.50.54:8016/v1`, `char-rp-reasoning` → -`http://10.250.50.54:8018/v1`. Sampler defaults (per the model cards): +`char-rp` → `http://10.251.50.54:8016/v1`, `char-rp-reasoning` → +`http://10.251.50.54:8018/v1`. Sampler defaults (per the model cards): - **char-rp (Magidonia / Mistral):** `temperature 1.0, top_p 0.95, min_p 0.03`; no repetition penalty (Magistral is stable without one). Callers may override. @@ -89,7 +89,7 @@ broken — do not re-serve it). ## Verify ```bash -curl -s http://10.250.50.54:8016/v1/models # char-rp up -curl -s http://10.250.50.54:8018/v1/models # char-rp-reasoning up +curl -s http://10.251.50.54:8016/v1/models # char-rp up +curl -s http://10.251.50.54:8018/v1/models # char-rp-reasoning up nvidia-smi --query-gpu=index,memory.free --format=csv # GPU 0 margin ``` diff --git a/stacks/char-rp-gguf/compose.yaml b/stacks/char-rp-gguf/compose.yaml index 7b4d6a4..d01144d 100644 --- a/stacks/char-rp-gguf/compose.yaml +++ b/stacks/char-rp-gguf/compose.yaml @@ -1,4 +1,4 @@ -# char-rp-gguf — dedicated GGUF character-RP seat on ana-ml2 GPU 0, REPLACING the +# char-rp-gguf — dedicated GGUF character-RP seat on fv-ml1 GPU 0, REPLACING the # broken ms32-24b-angel NVFP4 serve (garbage output — bad self-quant W4A4). # # Two co-located llama.cpp (llama-server) instances on GPU 0, served alongside the @@ -91,8 +91,8 @@ services: - homepage.group=AI - Inference - homepage.name=char-rp (Magidonia-24B GGUF) - homepage.icon=mdi-drama-masks - - homepage.description=Dark-romantasy RP prose seat, non-thinking (llama.cpp, ana-ml2 GPU 0) - - homepage.href=http://10.250.50.54:${CHARRP_PORT:-8016} + - homepage.description=Dark-romantasy RP prose seat, non-thinking (llama.cpp, fv-ml1 GPU 0) + - homepage.href=http://10.251.50.54:${CHARRP_PORT:-8016} # ── REASONING seat — NEO-CODE = Heretic2-Thinking (Qwen3.6-27B) managed thinking. gateway char-rp-reasoning. ── llama-charrp-reasoning: @@ -172,8 +172,8 @@ services: - homepage.group=AI - Dormant - homepage.name=char-rp-reasoning (QwQ-32B RpR-v4 GGUF) - homepage.icon=mdi-brain - - homepage.description=Dark-romantasy RP reasoning seat, managed CoT (llama.cpp, ana-ml2 GPU 0) - - homepage.href=http://10.250.50.54:${CHARRP_REASONING_PORT:-8018} + - homepage.description=Dark-romantasy RP reasoning seat, managed CoT (llama.cpp, fv-ml1 GPU 0) + - homepage.href=http://10.251.50.54:${CHARRP_REASONING_PORT:-8018} networks: tnet: diff --git a/stacks/darkscarlett-charrp-reasoning/compose.yaml b/stacks/darkscarlett-charrp-reasoning/compose.yaml index 22b5bff..74b6f2d 100644 --- a/stacks/darkscarlett-charrp-reasoning/compose.yaml +++ b/stacks/darkscarlett-charrp-reasoning/compose.yaml @@ -1,5 +1,5 @@ # darkscarlett-charrp-reasoning — Dark-Scarlett-v1.0-27B NVFP4A16 char-rp-reasoning seat -# (THINKING default) on ana-ml2 GPU1. Replaces the retired heretic2-charrp-reasoning seat. +# (THINKING default) on fv-ml1 GPU1. Replaces the retired heretic2-charrp-reasoning seat. # # Qwen3.6 hybrid GatedDeltaNet linear-attention -> KV-cheap, serves the full native 256K. # Quantized with llm-compressor NVFP4A16 loaded through the Qwen3_5ForConditionalGeneration @@ -67,8 +67,8 @@ services: - homepage.group=AI - Inference - homepage.name=char-rp-reasoning (Dark-Scarlett NVFP4) - homepage.icon=mdi-brain - - homepage.description=Dark-Scarlett-v1.0-27B NVFP4A16 thinking seat, 256K (ana-ml2 GPU1) - - homepage.href=http://10.250.50.54:${DS_PORT:-8018}/docs + - homepage.description=Dark-Scarlett-v1.0-27B NVFP4A16 thinking seat, 256K (fv-ml1 GPU1) + - homepage.href=http://10.251.50.54:${DS_PORT:-8018}/docs networks: tnet: diff --git a/stacks/dockge/README.md b/stacks/dockge/README.md index e7c528e..59bda03 100644 --- a/stacks/dockge/README.md +++ b/stacks/dockge/README.md @@ -40,10 +40,10 @@ DOCKGE_HOST_LABEL=ana-docker DOCKGE_HOST_IP=10.250.50.70 EOF' -# ana-ml2 -ssh -t ana-ml2 'sudo tee /opt/docker/compose/dockge/.env > /dev/null < /dev/null < GPU 0 on 2026-08-28 (operator-directed). GPU 1 carries the five # resident fleet seats (gen 46 GB + embed + coder + rerank + reward = ~69.9 GB of @@ -215,8 +215,8 @@ services: - homepage.group=AI - Inference - homepage.name=M.O.G.-SEC 27B (pen-test) - homepage.icon=mdi-shield-lock - - homepage.description=Uncensored security model, Qwen3.8-27B NVFP4+MTP, 262K — the `mog-sec` seat (ana-ml2 GPU 0) - - homepage.href=http://10.250.50.54:${MOG_PORT:-8019}/docs + - homepage.description=Uncensored security model, Qwen3.8-27B NVFP4+MTP, 262K — the `mog-sec` seat (fv-ml1 GPU 0) + - homepage.href=http://10.251.50.54:${MOG_PORT:-8019}/docs networks: tnet: diff --git a/stacks/qwen-image-bench/compose.yaml b/stacks/qwen-image-bench/compose.yaml index 5b2cef8..27b8e99 100644 --- a/stacks/qwen-image-bench/compose.yaml +++ b/stacks/qwen-image-bench/compose.yaml @@ -1,4 +1,4 @@ -# qwen-image-bench — flukethoughts/Qwen-Image-Bench-NVFP4 on ana-ml2 GPU 1, +# qwen-image-bench — flukethoughts/Qwen-Image-Bench-NVFP4 on fv-ml1 GPU 1, # REPLACING qwen3.6-35b-a3b (operator 2026-06-19). Qwen's text-to-image quality # JUDGE model (vision-language, NVFP4 weights / vision tower bf16). NOT generative — # it scores T2I outputs on 5 dims (overall quality, prompt match, aesthetic, LoRA @@ -75,8 +75,8 @@ services: - homepage.group=AI - Eval & Retrieval - homepage.name=Qwen-Image-Bench (T2I judge, NVFP4) - homepage.icon=mdi-image-check - - homepage.description=Qwen text-to-image quality judge (NVFP4, vision-intact) on ana-ml2 GPU 1 - - homepage.href=http://10.250.50.54:${QIB_PORT:-8014}/docs + - homepage.description=Qwen text-to-image quality judge (NVFP4, vision-intact) on fv-ml1 GPU 1 + - homepage.href=http://10.251.50.54:${QIB_PORT:-8014}/docs networks: tnet: diff --git a/stacks/qwen3.5-122b/compose.yaml b/stacks/qwen3.5-122b/compose.yaml index ba77ebc..f89cafd 100644 --- a/stacks/qwen3.5-122b/compose.yaml +++ b/stacks/qwen3.5-122b/compose.yaml @@ -1,4 +1,4 @@ -# qwen3.5-122b — bjk110/Qwen3.5-122B-A10B-abliterated-NVFP4 on ana-ml2 GPU 0, +# qwen3.5-122b — bjk110/Qwen3.5-122B-A10B-abliterated-NVFP4 on fv-ml1 GPU 0, # REPLACING mistral-small-4 (operator 2026-06-19: down the heretic, serve this as # the new general/`gen` model). Abliterated Qwen3.5 MoE (256 experts, 10B active), # NVFP4 (compressed-tensors), HF format. @@ -76,8 +76,8 @@ services: - homepage.group=AI Systems - homepage.name=Qwen3.5-122B-A10B (abliterated NVFP4) - homepage.icon=mdi-creation - - homepage.description=Abliterated Qwen3.5 122B-A10B NVFP4, the new `gen` model (ana-ml2 GPU 0) - - homepage.href=http://10.250.50.54:${QWEN35_PORT:-8013}/docs + - homepage.description=Abliterated Qwen3.5 122B-A10B NVFP4, the new `gen` model (fv-ml1 GPU 0) + - homepage.href=http://10.251.50.54:${QWEN35_PORT:-8013}/docs networks: tnet: diff --git a/stacks/qwen36-vl/compose.yaml b/stacks/qwen36-vl/compose.yaml index d232465..795b5e6 100644 --- a/stacks/qwen36-vl/compose.yaml +++ b/stacks/qwen36-vl/compose.yaml @@ -1,4 +1,4 @@ -# qwen36-vl — Qwen3.6-35B-A3B vision-language MoE (official NVFP4) on ana-ml2. +# qwen36-vl — Qwen3.6-35B-A3B vision-language MoE (official NVFP4) on fv-ml1. # # Replaces the qwen35-vl stack (Qwen3.5-9B) 2026-06-14. Co-located on GPU 1 with # the granite summarizer + embed/rerank/reward trio. Serves on :8007. @@ -119,8 +119,8 @@ services: - homepage.group=AI Systems - homepage.name=Qwen3.6-35B-A3B VL (NVFP4) - homepage.icon=mdi-image-search - - homepage.description=Qwen3.6-35B-A3B vision-language MoE (NVFP4) via vLLM (ana-ml2) - - homepage.href=http://10.250.50.54:${QWEN_PORT}/docs + - homepage.description=Qwen3.6-35B-A3B vision-language MoE (NVFP4) via vLLM (fv-ml1) + - homepage.href=http://10.251.50.54:${QWEN_PORT}/docs networks: tnet: diff --git a/stacks/qwopus3.5-122b/compose.yaml b/stacks/qwopus3.5-122b/compose.yaml index 973e6c2..67a225e 100644 --- a/stacks/qwopus3.5-122b/compose.yaml +++ b/stacks/qwopus3.5-122b/compose.yaml @@ -1,5 +1,5 @@ # qwopus3.5-122b — OpenYourMind/Qwopus3.5-122B-A10B-Kimi-K2.6-destilled-abliterated-NVFP4 -# on ana-ml2 GPU 0, REPLACING the bjk110 text-only qwen3.5-122b as the `gen` model +# on fv-ml1 GPU 0, REPLACING the bjk110 text-only qwen3.5-122b as the `gen` model # (operator 2026-06-19: "already ablated, already quanted, vision tower intact"). # # Qwen3.5-122B-A10B MoE, Kimi-K2.6-distilled + abliterated, NVFP4 — and crucially @@ -99,8 +99,8 @@ services: - homepage.group=AI - Dormant - homepage.name=Qwopus3.5-122B-A10B (abliterated NVFP4, vision) - homepage.icon=mdi-creation - - homepage.description=Kimi-distilled abliterated Qwen3.5-122B-A10B NVFP4, vision-intact, the `gen` model (ana-ml2 GPU 0) - - homepage.href=http://10.250.50.54:${QWOPUS_PORT:-8013}/docs + - homepage.description=Kimi-distilled abliterated Qwen3.5-122B-A10B NVFP4, vision-intact, the `gen` model (fv-ml1 GPU 0) + - homepage.href=http://10.251.50.54:${QWOPUS_PORT:-8013}/docs networks: tnet: diff --git a/stacks/scriberr/.env.example b/stacks/scriberr/.env.example index cb7e438..43490d1 100644 --- a/stacks/scriberr/.env.example +++ b/stacks/scriberr/.env.example @@ -1,5 +1,5 @@ # Scriberr — copy to .env on the host at /opt/docker/compose/scriberr/.env -# Real .env is gitignored and lives only on ana-ml2. +# Real .env is gitignored and lives only on fv-ml1. # ── Image ──────────────────────────────────────────────────────────────── # Built locally from Dockerfile.cuda.12.9 — see the compose header for why @@ -11,7 +11,7 @@ SCRIBERR_PORT=8080 SCRIBERR_BIND=0.0.0.0 # CORS. Must list every origin the UI is actually reached from, or the # browser blocks the API calls. Comma-separated, no spaces, no trailing /. -SCRIBERR_ALLOWED_ORIGINS=http://10.250.50.54:8080,http://scriberr.ana.internal:8080 +SCRIBERR_ALLOWED_ORIGINS=http://10.251.50.54:8080,http://scriberr.fv.internal:8080 # ── GPU ────────────────────────────────────────────────────────────────── # GPU0 is fully committed to the `gen` seat; GPU1 is the one with headroom. diff --git a/stacks/scriberr/README.md b/stacks/scriberr/README.md index edf19b8..c4ac323 100644 --- a/stacks/scriberr/README.md +++ b/stacks/scriberr/README.md @@ -1,17 +1,17 @@ -# scriberr — self-hosted transcription + diarization (ana-ml2, GPU1) +# scriberr — self-hosted transcription + diarization (fv-ml1, GPU1) Web UI for transcribing audio/video locally. WhisperX (Whisper + pyannote speaker diarization) with NVIDIA Parakeet/Canary also selectable; SQLite for state; optional summarisation and transcript chat against any OpenAI-compatible endpoint. -- **Host:** `ana-ml2` (10.250.50.54) — GPU1 -- **URL:** http://10.250.50.54:8080 +- **Host:** `fv-ml1` (10.251.50.54) — GPU1 +- **URL:** http://10.251.50.54:8080 - **Upstream:** https://github.com/rishikanthc/Scriberr ## The image is built locally, and that is not incidental -ana-ml2's RTX PRO 6000 Blackwell cards are **sm_120**. Upstream's published +fv-ml1's RTX PRO 6000 Blackwell cards are **sm_120**. Upstream's published images do not cover that: | image | built for | usable here | @@ -28,7 +28,7 @@ image** — it will fail on these cards or quietly fall back to CPU. ### Rebuilding ```bash -ssh ana-ml2 +ssh fv-ml1 cd /tank/scriberr/src/Scriberr git pull docker build -f Dockerfile.cuda.12.9 -t scriberr:local-blackwell . @@ -41,7 +41,7 @@ Source checkout lives on `/tank`, not the root pool — see storage below. ```bash # from this workstation -scripts/deploy-stack.sh ana-ml2 scriberr +scripts/deploy-stack.sh fv-ml1 scriberr ``` Then on the host, the usual: @@ -54,7 +54,7 @@ docker compose up -d scriberr # target the service, not the whole stack ## Storage — deliberately on /tank -`/var/lib/docker` on ana-ml2 sits on `zroot` at ~87% used. Whisper, pyannote +`/var/lib/docker` on fv-ml1 sits on `zroot` at ~87% used. Whisper, pyannote and NeMo weights are multi-GB and land in the `whisperx-env` volume, so both mounts are bind-mounted onto `/tank` (4+ TB) instead of named volumes: diff --git a/stacks/scriberr/compose.yaml b/stacks/scriberr/compose.yaml index ac302da..e46809b 100644 --- a/stacks/scriberr/compose.yaml +++ b/stacks/scriberr/compose.yaml @@ -7,7 +7,7 @@ # point it at the LiteLLM gateway rather than a paid API (see README). # # ── IMAGE: BUILT LOCALLY, ON PURPOSE ────────────────────────────────────── -# ana-ml2's RTX PRO 6000 Blackwell cards are **sm_120**. Upstream publishes +# fv-ml1's RTX PRO 6000 Blackwell cards are **sm_120**. Upstream publishes # `scriberr-cuda` (built for sm_61…sm_89 — no sm_120 kernels) and documents a # `scriberr-cuda-blackwell` image that **has never actually been published** # (GHCR returns no tags for it, checked 2026-08-23). The sm_120 path upstream @@ -15,7 +15,7 @@ # Do NOT "simplify" this to the published `scriberr-cuda` image — it will # fail on these cards or silently fall back to CPU. # Rebuild: see README "Rebuilding" — checkout lives at -# /tank/scriberr/src/Scriberr on ana-ml2. +# /tank/scriberr/src/Scriberr on fv-ml1. # # ── GPU PINNING ─────────────────────────────────────────────────────────── # Pinned to **GPU1** via explicit device_ids, per the house convention and @@ -34,7 +34,7 @@ services: ports: - "${SCRIBERR_BIND:-0.0.0.0}:${SCRIBERR_PORT}:8080" volumes: - # Bind mounts rather than named volumes: /var/lib/docker on ana-ml2 + # Bind mounts rather than named volumes: /var/lib/docker on fv-ml1 # lives on zroot with limited headroom, while /tank has terabytes. # Model weights (Whisper, pyannote, NeMo) land in whisperx-env and are # multi-GB — they must not go anywhere near the root pool. @@ -102,8 +102,8 @@ services: - homepage.group=AI - Studios - homepage.name=Scriberr - homepage.icon=mdi-microphone-message - - homepage.description=Audio/video transcription + diarization (ana-ml2, GPU1) - - homepage.href=http://10.250.50.54:${SCRIBERR_PORT} + - homepage.description=Audio/video transcription + diarization (fv-ml1, GPU1) + - homepage.href=http://10.251.50.54:${SCRIBERR_PORT} networks: tnet: diff --git a/stacks/selene/compose.yaml b/stacks/selene/compose.yaml index ad99566..7a27720 100644 --- a/stacks/selene/compose.yaml +++ b/stacks/selene/compose.yaml @@ -1,4 +1,4 @@ -# selene — AtlaAI Selene 1 Mini (Llama 3.1 8B) judge/eval model on ana-ml2 GPU 1. +# selene — AtlaAI Selene 1 Mini (Llama 3.1 8B) judge/eval model on fv-ml1 GPU 1. # # Restores the judge that went offline when llama-swap was downed (it was the # Q6_K GGUF `selene-1-mini-8b` in the llama-swap zoo). Re-served on vLLM at the @@ -77,8 +77,8 @@ services: - homepage.group=AI - Eval & Retrieval - homepage.name=Selene 1 Mini 8B (judge, FP8) - homepage.icon=mdi-gavel - - homepage.description=AtlaAI Selene 1 Mini Llama-3.1-8B judge (FP8) via vLLM (ana-ml2 GPU1) - - homepage.href=http://10.250.50.54:${SELENE_PORT}/docs + - homepage.description=AtlaAI Selene 1 Mini Llama-3.1-8B judge (FP8) via vLLM (fv-ml1 GPU1) + - homepage.href=http://10.251.50.54:${SELENE_PORT}/docs networks: tnet: diff --git a/stacks/sglang/README.md b/stacks/sglang/README.md index b26178c..fc67b15 100644 --- a/stacks/sglang/README.md +++ b/stacks/sglang/README.md @@ -1,4 +1,4 @@ -# sglang — vLLM-vs-SGLang bench on ana-ml2 +# sglang — vLLM-vs-SGLang bench on fv-ml1 Stood up to benchmark **SGLang against vLLM** on the same model + hardware, to see whether SGLang's throughput/latency wins justify it as a serving option @@ -26,18 +26,18 @@ avoid here. ## Run ```bash -# 1. On ana-ml2, after the eval frees a GPU: cp .env.example .env, set +# 1. On fv-ml1, after the eval frees a GPU: cp .env.example .env, set # SGLANG_MODEL / SGLANG_QUANT to match the vLLM config under test, and # SGLANG_GPU_ID to an EXCLUSIVE card. -scripts/deploy-stack.sh ana-ml2 sglang +scripts/deploy-stack.sh fv-ml1 sglang # (or docker compose up -d on the host) # 2. Bench SGLang: -python3 stacks/sglang/bench.py --url http://10.250.50.54:30000/v1 \ +python3 stacks/sglang/bench.py --url http://10.251.50.54:30000/v1 \ --model granite-4.1-8b-nvfp4 --concurrency 1 10 50 100 200 --in-tokens 2048 --out-tokens 256 # 3. Stop SGLang, bring up vLLM on the SAME GPU + model, bench identically: -python3 stacks/sglang/bench.py --url http://10.250.50.54:8006/v1 \ +python3 stacks/sglang/bench.py --url http://10.251.50.54:8006/v1 \ --model granite-4.1-8b-nvfp4 --concurrency 1 10 50 100 200 --in-tokens 2048 --out-tokens 256 # 4. Repeat the sweep at --in-tokens 30000 (the prefill-heavy agent-memory diff --git a/stacks/vllm/compose.yaml b/stacks/vllm/compose.yaml index 06e6f4a..4b35046 100644 --- a/stacks/vllm/compose.yaml +++ b/stacks/vllm/compose.yaml @@ -1,7 +1,7 @@ # vLLM — Qwen3 Embedding + Reranker + Skywork Reward-V2 classifier. # # Originally created to replace the unmaintained Infinity stack (embed + -# rerank); generalized 2026-05-13 to host any vLLM-served model on ana-ml2, +# rerank); generalized 2026-05-13 to host any vLLM-served model on fv-ml1, # starting with the Skywork-Reward-V2-Llama-3.1-8B reward classifier # (AWQ-quantized locally, served from /tank/aimodels/llm/). # @@ -19,13 +19,13 @@ # All tunables live in .env — edit that, not this file. # # Pre-download models to avoid first-run delay: -# scripts/elway ana-ml2 --playbook playbooks/pull-hf-repo.yaml \ +# scripts/elway fv-ml1 --playbook playbooks/pull-hf-repo.yaml \ # --var hf_repo=Qwen/Qwen3-Embedding-0.6B -# scripts/elway ana-ml2 --playbook playbooks/pull-hf-repo.yaml \ +# scripts/elway fv-ml1 --playbook playbooks/pull-hf-repo.yaml \ # --var hf_repo=Qwen/Qwen3-Reranker-0.6B # # Skywork-Reward-V2-Llama-3.1-8B-AWQ is a locally-quantized model — lives at -# /tank/aimodels/llm/Skywork-Reward-V2-Llama-3.1-8B-AWQ on ana-ml2 and is +# /tank/aimodels/llm/Skywork-Reward-V2-Llama-3.1-8B-AWQ on fv-ml1 and is # bind-mounted into the reward service at /local-models. Not from HF Hub. services: @@ -80,8 +80,8 @@ services: - homepage.group=AI - Eval & Retrieval - homepage.name=vLLM Embed (Qwen3) - homepage.icon=mdi-vector-arrange-below - - homepage.description=Qwen3 Embedding via vLLM (ana-ml2) - - homepage.href=http://10.250.50.54:${EMBED_PORT}/docs + - homepage.description=Qwen3 Embedding via vLLM (fv-ml1) + - homepage.href=http://10.251.50.54:${EMBED_PORT}/docs # THE fleet reranker. Backs the LiteLLM `reranker` alias, which is what every # consumer should name — never the model, never a bake-off arm name. @@ -155,8 +155,8 @@ services: - homepage.group=AI - Eval & Retrieval - homepage.name=vLLM Rerank (bge-v2-m3) - homepage.icon=mdi-sort-variant - - homepage.description=BAAI bge-reranker-v2-m3 — the fleet reranker, backs the `reranker` alias (ana-ml2) - - homepage.href=http://10.250.50.54:${RERANK_PORT}/docs + - homepage.description=BAAI bge-reranker-v2-m3 — the fleet reranker, backs the `reranker` alias (fv-ml1) + - homepage.href=http://10.251.50.54:${RERANK_PORT}/docs vllm-reward: image: vllm/vllm-openai:${VLLM_VERSION} @@ -213,8 +213,8 @@ services: - homepage.group=AI - Eval & Retrieval - homepage.name=vLLM Reward (Skywork) - homepage.icon=mdi-scale-balance - - homepage.description=Skywork-Reward-V2 8B classifier via vLLM (ana-ml2) - - homepage.href=http://10.250.50.54:${REWARD_PORT}/docs + - homepage.description=Skywork-Reward-V2 8B classifier via vLLM (fv-ml1) + - homepage.href=http://10.251.50.54:${REWARD_PORT}/docs # vllm-granite (ibm-granite/granite-4.1-8b-fp8, :8004) — RETIRED 2026-08-12, # service block removed 2026-08-20. It was the fleet summarizer until the @@ -296,8 +296,8 @@ services: - homepage.group=AI - Inference - homepage.name=vLLM Qwen2.5-Coder 1.5B (FIM) - homepage.icon=mdi-code-braces - - homepage.description=Qwen2.5-Coder-1.5B FIM code-completion (ana-ml2, Zed edit-predictions) - - homepage.href=http://10.250.50.54:${CODER_PORT}/docs + - homepage.description=Qwen2.5-Coder-1.5B FIM code-completion (fv-ml1, Zed edit-predictions) + - homepage.href=http://10.251.50.54:${CODER_PORT}/docs # vllm-lfm25 (LiquidAI/LFM2.5-2.6B, :8021) — RETIRED PERMANENTLY 2026-08-20 by # operator directive. It was an EVAL-ONLY bake-off seat against granite-4.1-8b @@ -305,7 +305,7 @@ services: # pending. Its comparator is gone (granite retired from the roster 2026-08-15), # it was deliberately never wired into any default/fallback routing chain, and # LiteLLM spend logs showed 0 calls in the 4-day window ending 2026-08-21. - # Freed 8,772 MiB on ana-ml2 GPU1. The `lfm2.5-2.6b` gateway alias was removed + # Freed 8,772 MiB on fv-ml1 GPU1. The `lfm2.5-2.6b` gateway alias was removed # in the same pass so the name 404s cleanly rather than erroring against a dead # backend. Weights remain in the shared HF cache; nothing was deleted from disk.