feat(sudo-io-archive): weekly fleet archive of sudo session recordings (pack, verify, then clear); fv-ml1 HF_HOME on tank and root-pool cleanup
This commit is contained in:
@@ -18,6 +18,14 @@ Primary AI inference host for PFI.
|
||||
- **GPUs:** **4x** NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition (95.6 GB VRAM each = 382 GB total, cc 12.0 / sm_120, GPU 0-3) — upgraded 2026-06 from 2x RTX 6000 Ada (46 GB, cc 8.9). Blackwell adds native FP4 (NVFP4) tensor cores. ⚠ This line read "2x" until 2026-09-19; `nvidia-smi` reports four. Read the host, not the doc.
|
||||
- **PSUs:** four present (PS1-PS4, all `ok`). ⚠ See § Power — on a single circuit that redundancy does not protect against the failure most likely to happen.
|
||||
- **Storage:** ZFS `zroot` (434 GB root) + `tank` pool (raidz2, 8× NVMe, 8.6 TB at `/tank`) — drive inventory below
|
||||
- **Root-pool hygiene (2026-10-04, Prime):** `HF_HOME=/tank/aimodels/huggingface` system-wide (`/etc/environment` +
|
||||
`/etc/profile.d/hf-home.sh`), the same tree the model containers mount as `/hf`; group `llm` (lkraven, llmuser,
|
||||
infra-ops) writes it and every dir carries setgid so new files stay group `llm`. All user HF caches were merged
|
||||
into it (`rsync --ignore-existing`, nothing lost) and cleared, as were the uv and pip caches: zroot went from
|
||||
124 G free to 194 G. Left on purpose: lkraven's `models--Cloudflare--clef-flash` (18 G), in use by the
|
||||
2026-10-04 bench; it is already on tank, so delete it once that bench is done. Docker (179 G) is still on zroot;
|
||||
moving its data-root to tank needs a model-seat outage window (Prime's call). sudo session recordings are
|
||||
archived weekly by `services/sudo-io-archive/`.
|
||||
- **OS:** Debian 13 (trixie), kernel 6.12.x
|
||||
- **Docker:** 29.3.1, runtimes: runc (default), nvidia, io.containerd.runc.v2
|
||||
|
||||
|
||||
Reference in New Issue
Block a user