memory: snapshot — FV recovered, fv-ml1 seat reorg, gen-large blocked on orca runtime
FV colo recovered 2026-09-13 midday (chassis on PDU, firewall on the Eaton 5P1000, GPU caps 275W/card). All-night fv-ml1 seat reorganization: - flash-next gained MTP k=3 (campaign measured it a win here, +52% at conc=1), inverting vLLM's 4xH100 recipe; KV 14->10 GiB. - gen consolidated onto flash-next (all 8 gen/summarizer/classifier/judge aliases repointed); 27B dense gen seat retired, 38 GB freed on GPU0. - char-rp restored to the in-house MeroMero-v2-31B dense heretic (was serving a leftover-test RedHatAI 26B); char-rp-fast is the deliberate speed tier. - Sentinel-R3 (SFT pentest finetune) served for an A/B vs mog-sec, then dflash k=7 cut over after measuring it beat MTP (2.40 vs 2.18 acceptance, ~121 tok/s warm). gen-large is intentionally DOWN: the orcarouter weight-only NVFP4 build downloaded (170 GB, verified) but no mainline vLLM loads its compressed-tensors qwen4_exp PLE; the third-party backport was vetted and is unfit (old-hardware fork, no Blackwell image). Runtime decision pending -- this is the resume point. Also this session: vh/infra-reference repo, scripts/seat-inventory.py + daily drift alarm, OPNsense API reference vendored, secrets shed from a prior scratchpad. Leaves the fv-to-ana-nat files (another session's) and graphify-out untouched.
This commit is contained in:
@@ -0,0 +1,10 @@
|
||||
SENTINEL_IMAGE=vllm/vllm-openai:nightly-e9d1398d9edfd90fcc1cf783805240e3effec013
|
||||
API_KEY=
|
||||
SENTINEL_GPU_ID=0
|
||||
SENTINEL_PORT=8025
|
||||
SENTINEL_MODEL=/tank/aimodels/sentinel-r3-nvfp4-mixed
|
||||
SENTINEL_GPU_MEM_UTIL=0.40
|
||||
SENTINEL_MAX_MODEL_LEN=163840
|
||||
SENTINEL_KV_CACHE_MEMORY=8589934592
|
||||
SENTINEL_MAX_NUM_BATCHED_TOKENS=4096
|
||||
SENTINEL_DRAFT_MODEL=/tank/aimodels/qwen38-27b-dflash2-drafter
|
||||
Reference in New Issue
Block a user