HF_HOME, TORCH_HOME and NLTK_DATA now point under /app/whisperx-env (the
/tank bind mount). Before, ~6 GB of models sat in the container layer and
every recreate threw them away. The cache was copied across (98/2/78 files,
apparent sizes identical).
HF_TOKEN is now a placeholder. Measured: with an invalid token, cached gated
pyannote models still load (huggingface_hub falls back to the cache on the 401),
and public models still download (HTTP 200). Scriberr only checks that the token
is non-empty. Re-verified on GPU 3: pyannote community-1 and WhisperX+3.1
diarization both exit 0 from the /tank cache with no downloads.
Scriberr: pyannote's diarization models are gated, and Scriberr had no Hugging
Face token, so every diarized job died at the diarize step (WhisperX after a full
CPU transcription). HF_TOKEN now passes through compose; the value is only in the
host .env (vault fv-ml1/hf-token). The separate pyannote env had torchcodec 0.16
(needs torch >=2.11/CUDA 13) beside torch 2.8, so pyannote 4 had no
AudioDecoder. Pinned to 0.7 by hand on the live env; patch 0003 makes it
durable at the next rebuild. WhisperX's adapter defaults to cpu/small/float32,
now documented. Both paths verified on GPU 3 with a real multi-speaker clip.
pbs-nh3: the backups-mirror prune job now keeps the primary's history
(last 3, daily 14, weekly 8, monthly 12, yearly 5). Dry-run: keep 738,
remove 3,194 across 32 groups; first run at the job's 00:00.
Layout: GPU 0 = cyberprev (KV 16 -> 15 GiB, 1.29x), erp-seat (moved from
GPU 1, KV 8.49 -> 5.5 GiB, 1.32x), voices (ctx 8,192 -> 40,960, its native
maximum, KV 6 GiB, 1.07x), coder (moved from GPU 1, ctx 16,384 -> 32,768,
the card maximum, 2.35x), parakeet-nemo. GPU 1 = gen-small (moved from
GPU 0, 5.5 GiB, 1.76x), meromero-rp (22 GiB, 1.32x), intern-decision.
Sizing rule: KV >= 1.25x max context rounded up to 0.5 GiB, checked against
7 days of LiteLLM traffic and the engines' own KV-usage logs. Each card keeps
spare memory for its bursty tenant (parakeet's cap on GPU 0, intern-decision's
32k call on GPU 1). YaRN extensions are not enabled: they do not fit, and the
Qwen cards warn static YaRN degrades short prompts.
gen-small gets --override-generation-config max_new_tokens 32768. It applies
only to requests that omit max_tokens (vLLM get_max_tokens) and answers 9
runaway ~260k-token replies in 7 days; the longest normal reply was 6,357.
Applied with playbooks/fv-ml1-rebalance-20261005.yaml (boot-check guard and
health/KV verification per seat); all nine seats healthy, gateway smoke
tests pass.
meromero-rp sized its KV from --gpu-memory-utilization, and vLLM's profile
counts co-booting neighbours as its own non-torch memory. Three boots gave
20.37 / 26.74 / 29.04 GiB of KV, and the 2026-10-05 Docker-move restart cut
GPU 1 free memory to 3,981 MiB, below intern-decision's 32k-token budget.
Coder read negative KV on every co-boot. Both are now pinned, so util only
gates the boot check (meromero util 0.52 -> 0.47, just above its footprint).
GPU 0 plateaus at ~60 MiB free 1-2 days after a boot (Beszel), so
gen-small's KV drops to 5.5 GiB (461,215 tokens, 1.76x) and parakeet-nemo
stays on GPU 0 with room for its 3,840 MiB cap. GPU 3 stays clear.
Applied with playbooks/fv-ml1-kv-pin-20261005.yaml: GPU 1 free 3,981 ->
11,801 MiB, GPU 0 495 -> 5,039 MiB right after boot; all nine seats healthy.
Front door: request bodies capped at 1 MiB, headers within 5 s, body within 15 s, no response deadlines (SSE stays open). Validated live: 413 over the cap, stalled body cut at 15 s, signed ping and an SSE stream unaffected. svos-front-inventory also serves the previous release's hashed assets credential-free from that release's static dir, hash-checked. Backup check: reads the logs of vzdump tasks PVE records as OK and reports ERROR lines; an fs-freeze/thaw error is an application-consistency failure, new verdict DEGRADED (exit 4), wired into the alert wrapper. nh3-pve-2 added to the node and namespace lists (its job results were never read). The svos drill task is acknowledged by UPID; BACKUP_IGNORE_ACKS=1 counts it (positive control: exit 4).
svos-dev: the service worker fetches its hash-pinned INVENTORY with credentials omitted. Exactly those files plus /sw.js are now served before basic_auth (GET/HEAD, exact files). The list is generated from the release's sw.js by svos-front-inventory, which hash-checks each file and validates before reload. Gated static routes follow the web/README hosting contract: missing assets 404 instead of index.html, other document requests get index.html, everything else 404, unversioned files no-cache. acceptance-matrix.sh: 35/35 live; 16 FAIL against the previous config (positive control). git added to the nh3-svos playbook (kernel's remote check). Live backup drill: quiesced backup ~1-2 s outage; a refused freeze does NOT abort vzdump (unfrozen backup, job reported successful); both images restore-mounted read-only and verified.
Prime's ruling: basic auth saved in the browser, mesh-only access. basic_auth argon2id with the account hash in a root:caddy file outside git (plaintext vaulted). After auth, Tailscale-User-Login is set from the authenticated user and Authorization is deleted, since the kernel's human path refuses requests that carry it. Removed svos-whois-auth (every mesh node is login pfi). Verified from a mesh peer against a temporary header echo: 401 without/with wrong credentials, 200 with exactly one Tailscale-User-Login: Vuong and no forged or Authorization headers upstream.
vh