docs(ana-ml2): correct GPU spec Ada -> RTX PRO 6000 Blackwell (96GB, cc 12.0)

ana-ml2 was upgraded 2026-06 from dual RTX 6000 Ada (46GB, cc 8.9) to
dual RTX PRO 6000 Blackwell Max-Q (96GB, cc 12.0 / sm_120). Update the
stale hardware facts across the workspace:

- CLAUDE.md servers table row
- servers/ana-ml2/README.md hardware spec (+ refreshed system-details.txt)
- stacks/vllm compose + .env.example FP8/KV comments (Ada cc 8.9 -> Blackwell cc 12.0)
- stacks/llama-swap config VRAM-budget comment (48GB -> 96GB, GPU-0 pin)

Also corrects the adjacent stale 'Phi-4-mini' comment in the granite
service block (the service has been Granite 4.1 8B since 34a43a0).
Doc/comment-only; no runtime change.
This commit is contained in:
vh
2026-06-13 13:36:14 -07:00
parent 0fc9083d16
commit 355a2407a2
6 changed files with 466 additions and 162 deletions
+6 -6
View File
@@ -197,10 +197,10 @@ services:
- homepage.description=Skywork-Reward-V2 8B classifier via vLLM (ana-ml2)
- homepage.href=http://10.250.50.54:${REWARD_PORT}/docs
# Phi-4-mini (FP8) — summarizer + "dreaming" agent. Supersedes the
# llama-swap granite-4-small pin. Generative chat model (OpenAI
# /v1/chat/completions), so NO --runner pooling. FP8 on RTX 6000 Ada
# (cc 8.9): near-lossless, ~1.2x, ~6 GB.
# Granite 4.1 8B (FP8) — production summarizer (replaced phi4-mini
# 2026-06-05, which had superseded the llama-swap granite-4-small pin).
# Generative chat model (OpenAI /v1/chat/completions), so NO --runner
# pooling. FP8 on RTX PRO 6000 Blackwell (cc 12.0): near-lossless, ~1.2x, ~6 GB.
vllm-granite:
image: vllm/vllm-openai:${VLLM_VERSION}
container_name: vllm-granite
@@ -218,7 +218,7 @@ services:
command:
# Production summarizer (replaced phi4-mini 2026-06-05). Default = official
# IBM pre-quantized FP8 (compressed-tensors), loaded directly; FP8 is native
# on the RTX 6000 Ada (cc 8.9). Fallback to vLLM-native dynamic FP8 from
# on the RTX PRO 6000 Blackwell (cc 12.0). Fallback to vLLM-native dynamic FP8 from
# BF16: GRANITE_MODEL=ibm-granite/granite-4.1-8b + GRANITE_QUANT=fp8.
- ${GRANITE_MODEL}
- --served-model-name
@@ -240,7 +240,7 @@ services:
# (they were over-provisioned at 5.9x/2.0x/3.9x concurrency); GPU 1 now has
# ~17 GB free after granite, so graph-capture buffers fit. If the trio
# ever grows back, granite may need --enforce-eager again on this card.
# FP8 KV cache — halves KV memory; near-lossless on Ada (cc 8.9).
# FP8 KV cache — halves KV memory; near-lossless on Blackwell (cc 12.0).
- --kv-cache-dtype
- ${GRANITE_KV_CACHE_DTYPE}
# Prefix caching pinned EXPLICIT (vLLM v1 defaults it on, but pin so a