memory: snapshot — Flash-Next seat on one card, and the FV outage that followed

Durable capture so tomorrow's session does not have to reconstruct either half.

Built and verified before the power failed: Qwen3.8-Flash-Next serving on a single
RTX PRO 6000 with its 51B n-gram table pinned in host RAM and read over CUDA UVA --
74.36 GiB weights resident, 14.00 GiB KV for 560,654 tokens at the full 262,144
context, 67 GiB host RSS -- plus a gen-large gateway alias verified end to end.

The five findings worth carrying: the offload is #54371 (UVA, merged) which supersedes
the paused worker-based #53899 and designs out its entire bug family;
text_config.ple_embedding_dtype is the load-or-fail discriminator for any community
build; --kv-cache-memory makes vLLM SKIP memory profiling and ignore
gpu-memory-utilization, which inverts the usual pin-bytes advice and let a 16 GiB pin
nearly OOM with no visible failure; MTP is off pending measurement here rather than
written off, because the recipe's number is cross-harness and tested k=3 only while the
head is one layer run autoregressively; and a container once reported (healthy) with no
published port at all, because the healthcheck runs inside the boundary it was trusted
to validate.

Then the outage. Records it as will-not-self-recover, so no session wastes effort
polling a dead site, and carries the three things that change the visit: bypass the UPS
rather than using its surge-only bank (both banks share one 12 A inlet -- the surge
bank bypasses the inverter, not the current rating), recover power.log before anything
else because it is the only load measurement that exists anywhere, and bring seats up
one at a time because ten restart:unless-stopped containers loading at once is the
largest transient the box can make into whatever just failed.

Also records what is still half-done: the stale homepage labels on the 10 containers
that died before they could be recreated, which the staged bring-up fixes as a side
effect, and the eight drifted stacks plus three untracked host-only stacks that were
deliberately left for a deliberate reconciliation.
This commit is contained in:
2026-09-13 00:14:46 -07:00
parent d79f10457a
commit 312725ddfb
3 changed files with 203 additions and 3 deletions
+53 -1
View File
@@ -1,6 +1,9 @@
# FV site dark — 2026-09-13 ~06:56Z
**Status: UNRESOLVED, needs hands at Fountain Valley.** Written while the site is
**Status: UNRESOLVED, WILL NOT SELF-RECOVER, needs hands at Fountain Valley.**
Operator 2026-09-13: an overload-tripped UPS does not clear itself and a dead one
needs replacing, so there is no recovery to wait for and no point polling the site.
Visit planned for 2026-09-14. Do not leave watchers running against FV addresses. Written while the site is
down so recovery does not have to be reconstructed from memory.
## What is down
@@ -138,9 +141,58 @@ much longer recovery.
4. **Do not restart the MTP campaign.** It is the prime suspect.
5. Watch power while seats come up: `nvidia-smi --query-gpu=index,power.draw --format=csv`.
### ⭐ The staged bring-up also finishes the homepage-label fix, for free
The 2026-09-13 renumber (commit `3132a16`) repaired 25 compose files on this box but
the 10 RUNNING containers were never recreated, so their labels still carried the dead
10.250.50.54. Those containers are gone with the power loss.
⚠ A PLAIN POWER-ON DOES NOT FIX THEM. `restart: unless-stopped` restarts the existing
container with its existing labels; labels only attach at container CREATION. But the
staged `docker compose up -d <svc>` sequence above **is** a recreate, and the compose
files on disk are already corrected — so bringing seats up that way applies the new
labels as a side effect and the dashboard comes back correct. Bring them up with
`compose up -d`, not by letting Docker restore the old containers.
Afterwards, confirm with:
curl -s http://10.0.50.45:5100/api/services | \
python3 -c 'import json,sys;d=json.load(sys.stdin);print([s["href"] for g in d for s in (g.get("services") or []) if "10.250.50.54" in (s.get("href") or "")])'
Expect `[]`. Before the outage that query returned 16 entries.
**Do not reboot the OPNsense firewall** (standing operator directive; its reboot API
403s anyway).
## ⚠ PROPOSED, NOT RATIFIED — split the power so the management plane survives
**This is a recommendation awaiting the operator's call, not settled intent.** Written
down so tomorrow's rebuild can adopt or reject it deliberately rather than restoring
the arrangement that just failed by default.
UPS <- OPNsense firewall + fv-ml1 BMC only (tens of watts, long runtime)
PDU / wall <- GPU chassis (no UPS in series)
Two reasons:
1. **It fixes the OOB gap this outage exposed.** The BMC's only route to the fleet is
through the firewall, so a power event at the GPU box takes out the management plane
with it -- which is precisely why this incident needs a drive rather than a console
session. Separate the two and a repeat leaves a live firewall, a live BMC, and
remote eyes on a dark chassis.
2. **A 1500 VA unit was never going to hold this box.** It has four cards, not the two
every record claimed until 2026-09-12.
**Do NOT use a UPS's surge-only outlets to get around its rating.** Both outlet banks
sit downstream of the same input cord, inlet and internal breaker; for a 120 V 1500 VA
unit that is a single NEMA 5-15P rated **12 A at maximum load**, total across all
outlets. The surge bank bypasses the inverter, not the current rating. Overloading the
inverter trips or kills the unit; overloading the cord is a thermal problem in an
unattended rack. Bypass the UPS entirely instead.
If the GPU box is ever to go on battery, it is a 3000 VA / 2700 W-class unit and
probably a 20 A circuit -- but size it from `power.log`, not from a spec sheet.
## Afterwards
-**The OOB design gap this exposes.** The cutover chose OPNsense-as-subnet-router
@@ -0,0 +1,130 @@
# `[2026-09-13]` Flash-Next seat stood up on fv-ml1 — then the FV site went dark under two-card load
Two things happened in one session and they are causally linked: a 176B model was put
on one GPU with its embedding table in host RAM, and the attempt to benchmark it on a
second GPU took Fountain Valley off the air.
## What was built and is real
**`stacks/flash-next-seat/` — Qwen3.8-Flash-Next (abliterated) on fv-ml1 GPU 2, `:8022`.**
The first seat on the fleet whose weights do not fit its card. 176B total = a 125B main
model + a **51B n-gram (PLE) lookup table** that lives in pinned host RAM and is read
by the GPU over **CUDA UVA**. Measured, not predicted:
weights resident 74.36 GiB of 95.6 (load 97-242 s depending on page cache)
KV cache 14.00 GiB pinned in bytes -> 560,654 tokens
context 262,144 (native max), 2.14x concurrency
engine host RSS 67 GiB (47.7 GiB table + runtime)
throughput (n=1!) 75.5 / 212.3 / 387.8 tok/s at concurrency 1 / 4 / 8
Checkpoint `dealignai/Qwen3.8-Flash-Next-ABLITERATED-NVFP4` @ `be794b99`, 126.0 GiB at
`/tank/aimodels/qwen38-flash-next-abliterated-nvfp4`. Composition measured from the
manifest: **63.3 GiB NVFP4 W4A4 routed experts + 47.7 GiB FP8 PLE table + 14.9 GiB
BF16** — so it is NOT an all-4-bit quant, and the single largest item on disk is the
FP8 table, 38% of the download.
**`gen-large` on the LiteLLM gateway**, one alias, verified registered for both the
infra-ops admin key and the shared all-agents key. Sampling is the checkpoint's own
declared set (temp 1.0 / top_p 0.95 / top_k 20); presence_penalty, min_p and
repetition_penalty deliberately unset because the checkpoint declares no canonical
value. Commits `f964a47`, `7e62a07`.
## ⭐⭐ The five things worth carrying forward
1. **The offload is upstream and merged — via a DIFFERENT PR than the park predicted.**
vLLM **#54371** "UVA PLE-offload and Engram tensor parallelism" merged
2026-09-09T14:32Z: pinned host memory + direct UVA row reads, no worker process.
It **supersedes #53899** (worker-based, now explicitly paused), and that matters
because #53899's entire bug family — the TP=1 startup deadlock #53960, the
`pidfd_getfd`/yama ptrace gate, the shared-CUDA-event race, silently one-step-stale
PLE outputs under CUDA graphs — all came from the separate worker the UVA path does
not have. Flag: `--engram-config '{"cpu_offload": true}'`. **In `v0.29.1rc0` and
main, NOT in `v0.29.0`** (cut ~6 h before the merge), and not in any `nightly-<sha>`
dated 2026-09-09 or earlier — the nightly builds at ~06:16 UTC.
2. **`text_config.ple_embedding_dtype` is the load-or-fail discriminator.** vLLM picks
the PLE table's format from that field FIRST. A checkpoint with an FP8 table and no
such declaration resolves through ModelOpt's `*.ple.*` exclude to the *unquantized*
method and dies on the scale tensor it does ship. Check it before trying any other
community build.
3. **⚠⚠ `--kv-cache-memory` makes vLLM SKIP MEMORY PROFILING and ignore
`--gpu-memory-utilization`.** This inverts the usual "pin bytes for
reproducibility" advice. 16.00 GiB was tried and a 155,497-token prefill drove GPU 2
to 97,074 of 97,887 MiB with the allocator logging *"OOM on device 0 while trying to
allocate 488636416 bytes (free: 422117376)"* — 466 MiB wanted, 403 free. The request
still completed, so **nothing failed visibly**; that is one step before the shape
that crashed mog-sec twice. 14.00 GiB re-probes clean. Do **not** take vLLM's own
*"17.46 GiB to fully utilize gpu memory"* line: it is computed from a profile taken
at `max-num-batched-tokens` depth and sits 3.5 GiB above what a 150K request
survives. `max-num-batched-tokens` is what bounds the activation peak — doubling
`max_model_len` left the profiled peak unchanged at 1.65 GiB.
4. **⚠ MTP is OFF but NOT written off, and the operator was right to push back.** The
seat initially defaulted MTP off citing vLLM's recipe (worse at every concurrency on
4xH100: 8-36% less throughput, 32-173% more latency, ~36% acceptance). **That was a
cross-harness comparison** — 4xH100/TP=4/Hopper vs 1xBlackwell/TP=1 — and our own
rule says those are invalid, not merely noisy. The real gap: the recipe tested
**k=3 only**, and `mtp_num_hidden_layers` is **1**, so the draft head is a single
module run autoregressively for k>1. If throughput falls monotonically in k while
acceptance rises, **k=1 may win and nobody has published that cell.** Campaign built
at `services/flash-next-mtp-bench/` with repeats, a boot-to-boot noise floor, and
acceptance-greater-than-zero as a positive control. It completed exactly **one**
`off_A` rep before the power failed.
5. **⚠ "Healthy" and "reachable" are on opposite sides of the container boundary.** The
seat once ran `Up 4 minutes (healthy)` with **`PORTS=[]`** — its port bind had lost a
race with a dying container, and the healthcheck passed throughout because it runs
*inside* the container against `localhost:8000`. Verify `docker port` / the
docker-proxy listener, not the healthcheck.
## ⚠⚠ And then the site went dark — see `docs/runbooks/fv-site-dark-20260913.md`
Operator authorised running the MTP campaign on idle GPU 3 in parallel with live
testing on GPU 2, with the two-card power risk explicitly raised and accepted.
06:51:08 campaign starts on GPU 3; idle draw GPU0 3.80 / GPU1 3.88 / GPU2 14.27 / GPU3 6.75 W
06:54:39 off_A healthy; rep 1 clean
06:56:04 off_A rep 2 starts <-- last line ever written
06:58:40 EVERY FV address unreachable, BMC included; all other sites 0% loss
**Leading hypothesis (operator): the 1500 VA Eaton UPS overloaded and died.** It fits
better than a breaker trip because a UPS's output rating sits far below the circuit's,
making it the first protective device to give — which explains why the site let go at
**two** cards loaded rather than four, and why the ~25 W firewall died with it.
-**Will not self-recover.** An overload-tripped UPS needs a human; a dead one needs
replacing. Do not poll FV addresses waiting for it.
-**Do NOT use a UPS's surge-only outlets to exceed its rating.** Both banks are
downstream of one input cord/inlet/breaker — for a 120 V 1500 VA unit, a single
NEMA 5-15P rated **12 A at maximum load**, total. The surge bank bypasses the
inverter, not the current rating. Bypass the UPS entirely instead.
-**Recover `/tank/aimodels/flash-next-mtp-bench/power.log` before anything else.**
It sampled all four cards every 10 s to the cut and lives on `/tank`, not in a
container. **No load figure exists anywhere else** — only idle. Nothing should put a
wattage into a UPS purchase until it has been read.
- **Blast radius: 19 of 30 gateway aliases**, probed not inferred. ⚠ **There is no
local fallback** — every free local model was on fv-ml1; irv-ml1 runs no chat seat at
all (TTS/ComfyUI/arbo/clipper on two partly-occupied Ampere cards). The only non-fv
chat backends are paid. Any paid coverage must go under a **new opt-in alias name**,
never a silent repoint of `summarizer`/`gen`/`classifier`.
-**The OOB design gap.** OPNsense-as-subnet-router protects box-down/gateway-up and
does nothing for a site-wide loss, because the BMC's only route out is through that
same gateway. Proposed-not-ratified fix in the runbook: firewall + BMC on the UPS,
GPU chassis on raw PDU.
## Also landed: the renumber the FV cutover missed
Every fv-ml1 link on the Homepage dashboard was dead — measured **16 entries pointing
at 10.250.50.54 and zero at 10.251.50.54**. The sweep script's allowlist was built from
files that mention the HOST, and a `homepage.href` mentions only an IP, so every
label-only stack fell outside it by construction. Fixed across 24 files plus the host
copies; allowlist extended with how to derive it next time. Commit `3132a16`.
Two bugs fell out: `deploy-stack.sh` rejected any stack name containing a dot (so
`qwen3.5-122b`, `qwopus3.5-122b`, `mistral-medium-3.5` could not be deployed **at
all**), and scriberr's CORS allowlist contained only the dead IP and the dead
`scriberr.ana.internal` — an allowlist with nothing reachable in it.
**Still incomplete:** the 10 running containers were never recreated, so the labels
are still stale on the host — and a plain power-on will NOT fix them, because labels
attach only at creation. The staged `docker compose up -d <svc>` recovery sequence
applies them as a side effect. Eight stacks were deliberately NOT pushed from canonical
because their host copies have genuine drift; three of those are untracked host-only
stacks that should be brought into `stacks/`.
+20 -2
View File
@@ -1,6 +1,6 @@
# Persistent memory — eshpfi-management
_Last updated: 2026-09-12 ~19:30 PT (FV cutover EXECUTED — fv-ml1 live at Fountain Valley, renamed/renumbered/serving; BMC online after a VLAN-250 find; four GPUs not two)_
_Last updated: 2026-09-13 ~00:10 PT (FV SITE DARK — UPS suspected overloaded/dead under two-card load; 19 gateway aliases down; visit 2026-09-14. Flash-Next seat + gen-large built and verified before the outage.)_
> **Always check for `/tmp/infra-ops-handoff.md`** — if it exists and its
> `Written:` stamp is under an hour old, read it (it carries the in-flight
@@ -112,7 +112,19 @@ no longer deployed sidecars here. See Recent decisions.)
## Current state / in-flight
_As of 2026-09-12 ~19:30 PT._
### FV colo — LIVE. Cutover done, one gap open.
### ⚠⚠ FV colo — DARK since 2026-09-13 06:56Z. Site visit 2026-09-14.
Every Fountain Valley address is unreachable, **including the BMC** — the OOB path
goes through the same OPNsense gateway, which is also dark. All other sites healthy.
Leading cause: the 1500 VA Eaton UPS overloaded and died under two-card GPU load.
**It will not self-recover; do not poll FV addresses.** 19 of 30 gateway aliases are
down with no local fallback (every free local model was on fv-ml1; irv-ml1 runs no
chat seat). On recovery: recover `power.log` first, mask Docker before the network,
then bring seats up one at a time with `compose up -d``gen` first, `flash-next`
last, and do NOT restart the MTP campaign.
`docs/runbooks/fv-site-dark-20260913.md`
### FV colo — the pre-outage state (cutover done, one gap open)
- **fv-ml1** (ex ana-ml2) is racked at Fountain Valley, renamed, on `10.251.50.54`, mesh
node `100.64.0.7`; **vb-gateway** OPNsense on `10.251.50.1` / `100.64.0.8`; **BMC** on
`10.251.250.50`. Public `fv.phasefinal.com``172.83.89.66`. `tank` intact, all vLLM
@@ -169,6 +181,12 @@ _As of 2026-09-12 ~19:30 PT._
## Recent decisions
- `[2026-09-13]` ⚠⚠⚠ **FV SITE DARK — every Fountain Valley address including the BMC went unreachable ~2.5 min into a two-card load test; all other sites healthy. Operator's leading hypothesis: the 1500 VA Eaton UPS overloaded and DIED.** It fits better than a breaker trip because a UPS's output rating sits far below the circuit's, making it the first protective device to give — which explains why the site let go at **two** cards loaded rather than four, and why the ~25 W OPNsense box died with it. ⚠ **Will not self-recover** (tripped needs a human, dead needs replacing) — do NOT poll FV. ⚠ **Do NOT use surge-only outlets to exceed a UPS rating**: both banks share one NEMA 5-15P inlet rated 12 A total; the surge bank bypasses the inverter, not the current limit. ⭐ **Recover `/tank/aimodels/flash-next-mtp-bench/power.log` FIRST** — all four cards every 10 s to the cut, on `/tank` not in a container, and the ONLY load measurement that exists. ⚠ **19 of 30 gateway aliases dark and NO local fallback** — every free local model was on fv-ml1, irv-ml1 runs no chat seat at all; the only non-fv chat backends are paid, and any coverage must be a NEW opt-in alias, never a silent repoint. ⭐ **OOB design gap**: OPNsense-as-subnet-router covers box-down/gateway-up and nothing for a site-wide loss, since the BMC's only route out is that gateway. ⚠ Recovery hazard: ten `restart: unless-stopped` vLLM containers will all load at once on power-up — mask Docker first, then `compose up -d` seat by seat (which also finishes the stale-homepage-label fix, since labels attach only at creation). → `docs/runbooks/fv-site-dark-20260913.md`, `persistent-memory.d/2026-09-13-flash-next-seat-and-fv-outage.md`
- `[2026-09-13]` ⭐⭐ **Qwen3.8-Flash-Next serving on ONE card with its 51B n-gram table in host RAM — the first seat whose weights do not fit its GPU.** `stacks/flash-next-seat/`, fv-ml1 GPU 2 `:8022`, plus a `gen-large` LiteLLM alias. Measured: 74.36 GiB weights resident, 14.00 GiB KV = 560,654 tokens at the full 262,144 context, 67 GiB host RSS, 75.5/212.3/387.8 tok/s at conc 1/4/8 (⚠ n=1). ⭐ The offload is vLLM **#54371 (UVA, merged 2026-09-09)** which **supersedes the paused #53899** — it has no worker process, so #53899's whole bug family (TP=1 deadlock #53960, `pidfd_getfd`/ptrace gate, stale-output-under-graphs) is designed out; in `v0.29.1rc0`, **not** `v0.29.0`. ⚠ **`text_config.ple_embedding_dtype` is the load-or-fail discriminator** for any community build. ⚠⚠ **`--kv-cache-memory` makes vLLM SKIP MEMORY PROFILING and ignore `--gpu-memory-utilization`** — 16 GiB nearly OOM'd on a 155K prefill with no visible failure; 14 GiB is the measured-safe value and vLLM's own "17.46 GiB to fully utilize" is 3.5 GiB too high. ⚠ MTP is off **pending measurement here, not written off** — the recipe's number is cross-harness and tested k=3 only, while the head is ONE layer run autoregressively, so k=1 is unpublished and may win (`services/flash-next-mtp-bench/`, one `off_A` rep banked before the outage). ⚠ A container once ran `(healthy)` with `PORTS=[]` — verify `docker port`, not the healthcheck. → `persistent-memory.d/2026-09-13-flash-next-seat-and-fv-outage.md`
- `[2026-09-13]` **Finished the ana-ml2→fv-ml1 renumber the cutover missed: 16 live Homepage entries pointed at the dead 10.250.50.54 and zero at the live IP.** The sweep allowlist was built from files that mention the HOST and a `homepage.href` mentions only an IP, so every label-only stack fell outside it by construction; 24 files fixed plus host copies, allowlist extended with how to derive it next time. Two bugs fell out: `deploy-stack.sh` **rejected any stack name containing a dot** (so `qwen3.5-122b`/`qwopus3.5-122b`/`mistral-medium-3.5` could not be deployed at all), and scriberr's CORS allowlist held only the dead IP and the dead `scriberr.ana.internal`. ⚠ Incomplete: the 10 running containers were never recreated and a plain power-on will NOT apply labels — the staged `compose up -d` recovery does. Eight stacks deliberately not pushed (real host drift); three of those are untracked host-only stacks. Commits `3132a16`, `969a1b6`, `d79f104`.
- `[2026-09-13]` **FV→ANA fixed, Beszel18/18 up:** scoped OPNsense hybrid NAT for fv-ml1→ANA; prior NAT/filter rules preserved and rollback-guarded. → `persistent-memory.d/2026-09-13-fv-to-ana-nat.md`
- `[2026-09-12]` ⭐⭐⭐ **FV CUTOVER EXECUTED — fv-ml1 live at Fountain Valley, renamed, renumbered to 10.251/16, serving inference; BMC online after finding it was tagging 802.1q VLAN 250 into an untagged port; and the box has FOUR RTX PRO 6000 (391 GB VRAM), not the two every doc claimed.** Also: OPNsense write APIs need an `X-CSRFToken` header scraped from a `<script>` block — session cookies alone 403, which is why reboots appeared to work while the operator was power-cycling by hand. → `persistent-memory.d/2026-09-12-fv-cutover-executed.md`
- `[2026-09-12]` **esh-vm-db Restic fixed:** stale April PG dump caused by TCP auth + masked errors; peer-auth/fail-closed hook and bounded retries installed. Snapshot bc5eeaff and repository check verified. → `persistent-memory.d/2026-09-12-esh-vm-db-restic-repair.md`