docs(ana-ml2): correct GPU spec Ada -> RTX PRO 6000 Blackwell (96GB, cc 12.0)

ana-ml2 was upgraded 2026-06 from dual RTX 6000 Ada (46GB, cc 8.9) to
dual RTX PRO 6000 Blackwell Max-Q (96GB, cc 12.0 / sm_120). Update the
stale hardware facts across the workspace:

- CLAUDE.md servers table row
- servers/ana-ml2/README.md hardware spec (+ refreshed system-details.txt)
- stacks/vllm compose + .env.example FP8/KV comments (Ada cc 8.9 -> Blackwell cc 12.0)
- stacks/llama-swap config VRAM-budget comment (48GB -> 96GB, GPU-0 pin)

Also corrects the adjacent stale 'Phi-4-mini' comment in the granite
service block (the service has been Granite 4.1 8B since 34a43a0).
Doc/comment-only; no runtime change.
This commit is contained in:
vh
2026-06-13 13:36:14 -07:00
parent 0fc9083d16
commit 355a2407a2
6 changed files with 466 additions and 162 deletions
+2 -2
View File
@@ -78,7 +78,7 @@ GRANITE_PORT=8004
# pin llama-swap to GPU 0 for clean separation (follow-up).
GRANITE_GPU_ID=1
# Official IBM pre-quantized FP8 (compressed-tensors) — calibrated, ~9.6 GB,
# loaded directly (FP8 native on Ada cc 8.9). Fallback to vLLM-native dynamic FP8
# loaded directly (FP8 native on Blackwell cc 12.0). Fallback to vLLM-native dynamic FP8
# from BF16: GRANITE_MODEL=ibm-granite/granite-4.1-8b + GRANITE_QUANT=fp8.
GRANITE_MODEL=ibm-granite/granite-4.1-8b-fp8
GRANITE_QUANT=compressed-tensors
@@ -90,7 +90,7 @@ GRANITE_SERVED_NAME=granite-4.1-8b
# summarize turn uses ~1k tokens, so the pool holds ~300 concurrently; the "2.33x"
# headline is worst-case (every request maxing 131k). Granite 4.1 supports 131072.
GRANITE_MAX_MODEL_LEN=131072
# FP8 KV cache (native on Ada cc 8.9). At 50K ≈ ~4.2 GB (vs ~8.4 GB at fp16).
# FP8 KV cache (native on Blackwell cc 12.0). At 50K ≈ ~4.2 GB (vs ~8.4 GB at fp16).
GRANITE_KV_CACHE_DTYPE=fp8
# util 0.35 (~33.6 GB) — tuned 2026-06-13 to leave ~3.5 GB free on GPU 1 alongside
# the trio + qwen co-tenants. On this shared card vLLM needs free >= util*total at