stacks(char-rp): compose-ify the two NVFP4 RP seats (MeroMero + Dark-Scarlett)

Replaces the ad-hoc docker-run seats with proper compose stacks on ana-ml2, mirrored here: - meromero-charrp: G4-MeroMero-v2-31B NVFP4A16, char-rp prose (non-thinking, multimodal, vision-enabled), GPU0, 256K @ ~2x. util 0.52 (leaves ~4.6GB GPU0 headroom). - darkscarlett-charrp-reasoning: Dark-Scarlett-v1.0-27B NVFP4A16 (Qwen wrapper recipe), char-rp-reasoning thinking seat, GPU1, 256K. MTP deferred (no spec-decode). Both survive reboot now. Supersede the retired char-rp-gguf + heretic2-charrp-reasoning stacks.
2026-08-12 11:15:33 -07:00
parent 398b58a161
commit f08b6cbddf
6 changed files with 201 additions and 0 deletions
@@ -0,0 +1,69 @@
+# meromero-charrp — G4-MeroMero-v2-31B NVFP4A16 char-rp PROSE seat (non-thinking, MULTIMODAL)
+# on ana-ml2 GPU0. Replaces the retired Magidonia GGUF seat (char-rp-gguf / llama-charrp).
+#
+# Gemma-4 dense but with SLIDING-WINDOW attention -> KV-efficient, serves the full native 256K
+# at ~2x concurrency. Vision enabled: preprocessor_config.json was materialized from
+# processor_config.json's image_processor section (Gemma4ImageProcessor); audio is
+# config-declared but weightless (no audio tensors) so it serves image + text.
+#
+# Co-located on GPU0 with vllm-aeon-gen (gen). util 0.52 leaves ~5GB GPU0 headroom
+# (0.55 left only ~1.8GB). Tunables in .env.
+
+name: meromero-charrp
+
+services:
+  vllm-meromero-rp:
+    image: ${MEROMERO_IMAGE:-vllm/vllm-openai:latest}
+    container_name: ${MEROMERO_CONTAINER:-vllm-meromero-rp}
+    restart: unless-stopped
+    ipc: host
+    ports:
+      - "${MEROMERO_PORT:-8016}:8000"
+    volumes:
+      - /tank/aimodels:/tank/aimodels
+    environment:
+      - PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
+      - VLLM_API_KEY=${API_KEY:-}
+    command:
+      - ${MEROMERO_MODEL:-/tank/aimodels/meromero-v2-nvfp4-work/G4-MeroMero-v2-31B-NVFP4A16}
+      - --quantization
+      - compressed-tensors
+      - --served-model-name
+      - char-rp
+      - --max-model-len
+      - "${MEROMERO_MAX_MODEL_LEN:-262144}"
+      - --max-num-seqs
+      - "${MEROMERO_MAX_NUM_SEQS:-32}"
+      - --gpu-memory-utilization
+      - "${MEROMERO_GPU_MEM_UTIL:-0.52}"
+      - --kv-cache-dtype
+      - fp8
+      - --trust-remote-code
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              device_ids:
+                - "${MEROMERO_GPU_ID:-0}"
+              capabilities:
+                - gpu
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 600s
+    networks:
+      - tnet
+    labels:
+      - homepage.group=AI - Inference
+      - homepage.name=char-rp (MeroMero-v2 NVFP4, multimodal)
+      - homepage.icon=mdi-drama-masks
+      - homepage.description=G4-MeroMero-v2-31B NVFP4A16 non-thinking prose seat, vision-enabled, 256K (ana-ml2 GPU0)
+      - homepage.href=http://10.250.50.54:${MEROMERO_PORT:-8016}/docs
+
+networks:
+  tnet:
+    name: traefik-net
+    external: true