Prime removed the second instance after the v0.1.3 bench. esh-ml1 handles a face
in ~48 ms, sits in the house next to the cameras, and holds the verified
backup. fv-ml1's gallery was empty (0 identities). The container, gallery volume,
image, compose dir (with its .env), backup dir and build sources are removed from
fv-ml1. GPU_ID / CARD_SUFFIX stay in the compose for any future second host.
Prime asked for augaman on fv-ml1's utility card, beside vllm-coder. Mirror
augaman-dev's f77164f compose, which parameterises the GPU reservation (GPU_ID,
default 0) and the Homepage card name (CARD_SUFFIX). esh-ml1's resolved config is
unchanged: same config hash, no recreate.
On fv-ml1: augaman:0.1.2 built on-box from the tag, GPU_ID=1, healthy on CUDA
at 1264 MiB, and pytest -m gpu tests/vision passes 3/3 on the Blackwell. It has
its own gallery and no gallery backup, so it is fixtures-only. The host's raw
restic copy of /var/lib/docker/volumes is not a consistent SQLite backup.
docs/pfi/augaman-speed-bench/ holds the harness (augaman-dev's recipe plus a
no-face control frame and a face-count check on every response), the raw rows
and the summary. Server-side, one face:
- esh-ml1 GPU 144 ms
- fv-ml1 GPU 75 ms
- fv-ml1 CPU on 6 cores 152 ms
- esh-ml1 CPU 888 ms
It agrees with augaman-dev's independent esh-ml1 measurement once each
harness's floor is subtracted. This is the before for v0.1.3's detector fix.
Operator confirmed 2026-09-19 that fv-ml1 and the R420 running OPNsense are the
only loads on a dedicated 20 A circuit.
The governing number is 1920 W, not 2400: a GPU inference host running for hours
is a continuous load, so NEC's 80% rule applies. Worst case lands at ~1625 W
with the current caps -- about 85% of budget.
Measured via the BMC rather than assumed: 390 W instantaneous, 461 W max over a
2423 s sample, with GPUs at idle, giving a ~313 W non-GPU baseline.
Compare the GPU caps against the 300 W stock TGP, NOT the 325 W firmware
ceiling. The operator corrected this: 275 W across four cards saves 100 W, not
the 200 W you get by measuring against a number nobody would ever run at. Stock
300 W would put the circuit near 90%, which is not illegal but leaves nothing
for a heavier R420, PSU efficiency, or a warm day. Keep the caps.
The coupling matters more than the trip. OPNsense IS the Fountain Valley edge
and shares the breaker with the thing most likely to trip it, so an overload
takes the router with it and removes the remote path needed to diagnose or
power-cycle anything. fv-ml1's four PSUs do not help -- PSU redundancy protects
against a PSU dying, not against the circuit going away.
Three things are explicitly NOT measured and the file says so: fv-ml1 under real
4-GPU load, whether the BMC reports AC input or DC output, and the R420's actual
draw. Treat 1625 W as a floor.
Also corrects the hardware section, which claimed 2x GPUs. nvidia-smi reports
four.
Placed on GPU 3 first because it was the empty card. That was the wrong read:
the seat is ~800 MiB, under 1% of a 96 GB card, so the question was never "where
does it fit" but "whose headroom is cheapest to spend".
vLLM sizes its KV cache as a fraction of TOTAL VRAM, not free VRAM. A resident
tenant on an otherwise-clean card therefore does not cost its own megabytes — it
costs the profiling margin of whatever full-size seat lands there later, and
flash-next needs 93 GiB of 96. A 96 GB card at 2 MiB can still take that; the
same card at 922 MiB is one where the next big seat needs its utilization
hand-trimmed, which this repo's flash-next history shows is both thin and silent
when it goes wrong.
Committed utilization per card is the number that governs, not free bytes:
GPU 0 0.40 + 0.48 = 0.88 ~13 GB spare <- moved here
GPU 1 0.52+0.24+0.10+0.055+0.03+0.03 = 0.975 ~4.3 GB
GPU 2 0.96 ~1.8 GB
GPU 3 - kept empty as reserve
GPU 3 is back to 2 MiB / 97,247 MiB free and is now documented as a deliberate
reserve rather than a spare.
Post-move n=5 on the same clip: 0.68 / 0.54 / 0.54 / 0.52 / 0.53 s, median 0.54 s
against 0.50 s on GPU 3. The spreads overlap at this sample size and no difference
is claimed; the GPU 3 figure was taken on an idle card and is now noted as a best
case, since the seat shares GPU 0 with the hot serving path. Silence control and
the gateway round-trip both re-verified after the move.
Also records both Parakeet endpoints (FV v3 on Blackwell, IRV v2 on a 3090) and
the four confounds that make them not an A/B pair, sent to tts-dev for the bench.
Retargets the existing sherpa-onnx stack from irv-ml1 to fv-ml1's utility card
and puts it behind the gateway. GPU 3 was the only card with room: 0/1/2 carry
the vLLM seats at 84-95.5 GB of 96.
Changes:
- compose: pin GPU via `device_ids: ["3"]` (the dead on-host stub used
`count: all`, which would have handed a 0.6B ASR seat all four cards);
join traefik-net; port 8300; homepage href to the live FV address.
- .env.example: default to the v3 int8 model (25 European languages, 464 MiB)
rather than English-only v2; models to /tank/parakeet/models.
- app.py: warm the recognizer at startup before uvicorn accepts traffic.
The warmup is not an optimisation. ONNX Runtime's CUDA EP compiles and autotunes
lazily on the FIRST DECODE, and on sm_120 that measured 45.7s cold (reproduced at
45.1s on a second container) against ~0.50s warm. A 45s first request is
indistinguishable from a hang and LiteLLM's default timeout abandons it long
before it returns. Decoding 1s of silence at load moves the cost inside the
healthcheck's 300s start_period; first real request after restart is now 0.65s.
Verification, because "provider=cuda" in the log is only an echo of the env var:
ORT falls back to CPU silently and still returns correct text, so the service
being up and the transcript being right establishes nothing. The discriminator is
a process on GPU 3 (922 MiB), confirmed. Controls both directions — a known TTS
sentence transcribes near-exactly (positive), 3s of digital silence returns
empty (null). Warm throughput 0.50s median on an 8.52s clip, n=5, spread
0.47-0.65s, single-stream, one clip: a smoke measurement with its harness
stated, not a benchmark.
Gateway aliases `ext-stt` (engine-neutral, mirrors ext-tts) and `whisper-1`
(OpenAI-compatible drop-in) registered via POST /model/new, i.e. LiteLLM's
Postgres store where the ext-tts family already lives — no gateway restart, and
config.yaml is consequently not a complete picture of what the gateway serves.
Both verified end to end.
The aliases use a raw IP deliberately: ana-docker resolves no .internal names at
all (resolv.conf points at 1.1.1.1), and LiteLLM only reaches irv-ml1 through a
hand-pinned extra_hosts entry. A second hosts entry would mean recreating the
container and bouncing the gateway for every consumer.
Also records the svos_miranda plugin validation pass and its structural findings,
and notes that the irv-ml1 parakeet is still running — there are two now, and
retiring the old one is the operator's call.
The box is physically at Fountain Valley, renamed, renumbered onto 10.251/16,
and serving inference again. This lands the repo half of that.
Host: hostname ana-ml2 -> fv-ml1, pinned to 10.251.50.54 by a dnsmasq
reservation so the address the runbook, DNS and LiteLLM all assume is the
address it actually has. Its headscale node is renamed too.
The sweep ran from scripts/fv-ml1-rename-sweep.sh, whose allowlist is the
reason this diff touches current-state files and not the record. Dated
persistent-memory entries, archival-memory and incident notes still say
ana-ml2 in 31 and 62 places respectively, because that is what the box was
when those things happened. Rewriting them would make the history lie.
LiteLLM was the load-bearing piece and needed more than the api_base sed the
runbook describes. Twenty api_base entries repointed, but a grep-and-verify
pass also caught a LIVE pass_through_endpoints target for the scalar-judge
reward route still on the old address -- an api_base-only substitution would
have left it dead. Four prose references describing current state were
repointed as well; one historical note recording where a hand-test was run
is deliberately left pointing at 10.250.50.54.
Two facts in the server tables were wrong and are corrected here. The site is
Fountain Valley, not Anaheim. And the box has FOUR RTX PRO 6000 Blackwell
Max-Q, not two -- verified by nvidia-smi -L and independently by PCI
enumeration of four GB202GL devices. That is 391 GB of VRAM rather than 196,
which changes what fits on it.
DNS: fv-ml1, fv-ml1-bmc and fv-gw added under the fv site via the piggyback
approach, scriberr re-homed, and the ana-ml2 records removed. Applied to all
three resolvers. The BMC record carries a warning that its 802.1q VLAN tag
must stay disabled -- it shipped tagging VLAN 250 into an untagged port,
which made it invisible to every network-side diagnostic and is the reason
it appeared dead through several cable changes.
Verified end to end: summarizer and sec both answer through the Anaheim
gateway across the mesh to FV seats on different ports.
vh