vh
7e7130172e
Two related changes shipped together. The stack rename is independent but adding `vllm-reward` to the existing `vllm-qwen3` would have made that name actively misleading. **Rename:** `stacks/vllm-qwen3/ → stacks/vllm/`. Updated all in-repo references (README.md root, servers/ana-ml2/, stacks/llama-swap/, configs/restic/ana-ml2/, docs/runbooks/disaster-recovery.md). Two intentional history mentions retained (servers/ana-ml2 + stacks/vllm README). **Add `vllm-reward` service:** serves Skywork-Reward-V2-Llama-3.1-8B-AWQ on port 8003. The AWQ output is a locally-quantized model (not from HF), so bind-mounts `/tank/aimodels/llm:/local-models:ro` rather than the shared HF cache. Model config.json declares LlamaForSequenceClassification which vLLM's pooling runner picks up automatically — produces a single reward score per input via /classify. **Flag note:** the user's spec listed `--task classify`, but vLLM 0.19.1 deprecated --task in favor of --runner pooling (model architecture in config.json drives the classification head). Compose uses --runner pooling with a comment explaining the substitution. **GPU memory:** no rebalance needed — production had already tuned EMBED/RERANK down from 0.40 to 0.20 each (canonical .env.example now matches reality). Adding REWARD at 0.30 totals 0.70, leaving ~14 GB headroom on the 48 GB Ada. **Server-side:** brought existing vllm-qwen3 down, mv'd /opt/docker/compose/vllm-qwen3 → /opt/docker/compose/vllm, appended REWARD_* lines to existing .env (preserving API_KEY/HF_TOKEN), deployed new compose via scripts/deploy-stack.sh, brought all 3 services up. **Smoke tests:** - /health on 8001/8002/8003 → 200 - /v1/models on 8003 → lists Skywork/Skywork-Reward-V2-Llama-3.1-8B-AWQ with max_model_len 16384 - /classify with a sample conversation → returns LABEL_0 with prob 0.9999 (single-output regression-style reward score, expected shape for a reward model)
87 lines
3.1 KiB
YAML
87 lines
3.1 KiB
YAML
# resticprofile config for ana-ml2.
|
|
#
|
|
# **Bare metal** — not protected by any Proxmox vzdump. This is the only
|
|
# backup for this host's config/state, so it's the highest-stakes repo in
|
|
# the fleet. Writes to the Anaheim rest-server at 10.250.50.70 as user
|
|
# `ana-ml2`. The full REST URL (with HTTP basic auth creds) lives in
|
|
# /etc/restic/restic.env. The client-side encryption passphrase lives in
|
|
# /etc/restic/password.
|
|
#
|
|
# Notable exclusions:
|
|
# - /tank/* is NOT in source paths. That's ~TB of model weights (HF
|
|
# caches, llama.cpp GGUFs, ComfyUI models, etc.) — all regenerable
|
|
# from upstream. Backing them up would blow the repo size budget.
|
|
# - No DB dumps needed. None of the stacks on this host (llama-swap,
|
|
# vllm, comfyui, kokoro, parakeet, vibevoice, beszel-agent,
|
|
# dozzle-agent, dockge) store relational data.
|
|
|
|
version: "1"
|
|
|
|
global:
|
|
priority: low
|
|
ionice: true
|
|
ionice-class: 2
|
|
ionice-level: 7
|
|
min-memory: 100
|
|
|
|
default:
|
|
env-file: /etc/restic/restic.env # RESTIC_REPOSITORY=rest:http://user:pw@…
|
|
env:
|
|
RESTIC_PASSWORD_FILE: /etc/restic/password
|
|
initialize: false # repo created manually by `restic init`
|
|
lock: /var/lock/restic-ana-ml2.lock
|
|
|
|
backup:
|
|
verbose: 1
|
|
run-after:
|
|
- date +%s > /var/lib/restic/last-success
|
|
source:
|
|
- /opt/docker # compose files + host-level configs — small (~100MB)
|
|
- /etc # host config (systemd units, chrony, apparmor, ssh, etc.)
|
|
- /root # root shell history, ssh keys, any ad-hoc scripts
|
|
- /var/lib/docker/volumes # named volumes (small; models live on /tank, not here)
|
|
exclude:
|
|
# Docker internals we never want in a backup
|
|
- /var/lib/docker/volumes/backingFsBlockDev
|
|
- /var/lib/docker/volumes/metadata.db
|
|
# Parakeet's HF model cache — named volume (stacks that bind-mount
|
|
# from /tank/ are already outside the source list; parakeet is the
|
|
# odd one out using a docker volume). Regenerable from Hugging Face.
|
|
- /var/lib/docker/volumes/parakeet_parakeet_cache
|
|
# Ephemeral / regenerable junk
|
|
- /opt/docker/compose/*/logs
|
|
- "**/*.log"
|
|
- "**/*.log.*"
|
|
- "**/*.pid"
|
|
# Root's noise — shell caches, tmp
|
|
- /root/.cache
|
|
- /root/.local/share/Trash
|
|
- /root/.npm
|
|
- /root/.python_history
|
|
tag:
|
|
- host:ana-ml2
|
|
- site:ana
|
|
- fleet:pfi
|
|
schedule: "*-*-* 01:00:00"
|
|
schedule-permission: system
|
|
schedule-log: /var/log/restic-backup.log
|
|
|
|
forget:
|
|
keep-daily: 7
|
|
keep-weekly: 4
|
|
keep-monthly: 12
|
|
keep-yearly: 3
|
|
# NOTE: no `prune: true` — rest-server runs with --append-only.
|
|
# See README.md "Prune ceremony".
|
|
tag:
|
|
- host:ana-ml2
|
|
# Schedule removed 2026-04-21: forget against --append-only rest-server
|
|
# always fails (delete ops blocked). Run manually during the prune
|
|
# ceremony when --append-only is temporarily off.
|
|
|
|
check:
|
|
read-data-subset: 10%
|
|
schedule: "Sun *-*-* 05:00:00"
|
|
schedule-permission: system
|
|
schedule-log: /var/log/restic-check.log
|