vh
317868dc7e
Prime asked for augaman on fv-ml1's utility card, beside vllm-coder. Mirror augaman-dev's f77164f compose, which parameterises the GPU reservation (GPU_ID, default 0) and the Homepage card name (CARD_SUFFIX). esh-ml1's resolved config is unchanged: same config hash, no recreate. On fv-ml1: augaman:0.1.2 built on-box from the tag, GPU_ID=1, healthy on CUDA at 1264 MiB, and pytest -m gpu tests/vision passes 3/3 on the Blackwell. It has its own gallery and no gallery backup, so it is fixtures-only. The host's raw restic copy of /var/lib/docker/volumes is not a consistent SQLite backup. docs/pfi/augaman-speed-bench/ holds the harness (augaman-dev's recipe plus a no-face control frame and a face-count check on every response), the raw rows and the summary. Server-side, one face: - esh-ml1 GPU 144 ms - fv-ml1 GPU 75 ms - fv-ml1 CPU on 6 cores 152 ms - esh-ml1 CPU 888 ms It agrees with augaman-dev's independent esh-ml1 measurement once each harness's floor is subtracted. This is the before for v0.1.3's detector fix.
augaman speed bench: CPU vs esh-ml1 GPU vs fv-ml1 GPU
Asked by Prime on 2026-09-27. This is the v0.1.2 "before" baseline. v0.1.3 gives each detector canvas its own ONNX session, removing the ~90 ms CUDA shape-switching cost that comfy-dev found. The same harness is to be re-run on v0.1.3 for the "after".
Harness (it is part of the number)
- Client:
bench.pyon nh3-dev, sequential. Each request uses a fresh connection (no keep-alive) and is timed around the whole POST. - Protocol: 3 runs. Per (target, frame, run) there are 3 warm-up requests, then 20 timed ones. Runs are interleaved across targets and frames, so drift spreads over every condition. Time: 0021–0026 PT, 2026-09-27.
- Server-side column: the change in
augaman_pipeline_seconds{op="probe"}sum/count over the 20 timed requests. It is decode + detect + embed with no network or HTTP, and it is the device comparison. It is a mean, not a p50. - Floor: a
/healthround trip, same client. It came out 28–34 ms, higher than augaman-dev's 16 ms because of the fresh connection per request. - Frames (
make_frames.py): (a) 1920x1080 grey canvas with one face; (b)obama_biden_2015.jpgas-is, 960x1001, two faces; (c) the same 1080p grey canvas with no face, a detector-only control. Every response was checked for the expected face count, and 0 of 900 were wrong. - Images:
augaman:0.1.2, built from the same tag on each host (separate builds, same source). - Targets:
| target | device | notes |
|---|---|---|
| esh-ml1 GPU | RTX 2000E Ada 16 GB (50 W) | the production instance; shares the card with TEI + vllm-reward |
| fv-ml1 GPU1 | RTX PRO 6000 Blackwell Max-Q | the new second instance; shares GPU 1 with coder, erp and meromero seats |
| esh-ml1 CPU6 | the LXC's 6 host threads | throwaway AUGAMAN_DEVICE=cpu container. pve moved the LXC's cpuset between two reads, so it ran unpinned, with an affinity of 6 |
| fv-ml1 CPU6 | EPYC 9254, cpuset 0-5 | throwaway container; physical cores on socket 0 |
| fv-ml1 CPU24 | EPYC 9254, cpuset 0-23 | throwaway container; all physical cores of socket 0 |
GPU utilisation read 0% on fv-ml1 GPU 1 and 0–9% on esh-ml1 across the samples (0020–0021, 3 samples each), so the co-resident seats were idle.
Results (ms): median of the 3 run p50s, [min–max of the run p50s]
| target | (a) 1 face, e2e | (b) 2 faces, e2e | (c) no face, e2e | server (a) | server (b) | server (c) |
|---|---|---|---|---|---|---|
| esh-ml1 GPU | 196.5 [195.0–196.6] | 195.1 [194.0–196.1] | 181.0 [177.6–182.9] | 143.8 | 138.7 | 136.2 |
| fv-ml1 GPU1 | 126.6 [121.8–128.3] | 122.7 [122.4–127.9] | 109.2 [108.7–111.0] | 75.0 | 73.9 | 69.5 |
| esh-ml1 CPU6 | 954.6 [792.8–1023.3] | 1325.6 [1306.9–1327.6] | 205.1 [202.1–228.0] | 887.6 | 1253.4 | 159.4 |
| fv-ml1 CPU6 | 203.6 [203.5–204.5] | 222.5 [216.0–236.4] | 111.6 [110.6–112.7] | 152.4 | 179.4 | 72.3 |
| fv-ml1 CPU24 | 208.1 [204.6–221.6] | 223.7 [219.8–237.1] | 101.1 [100.5–113.4] | 155.9 | 167.3 | 61.9 |
The /health floor was 27.6–33.7 ms (median of run p50s) on every target. The raw
rows are in rows-2026-09-27.json, and summarize.py reproduces this table.
What it says
- fv-ml1's Blackwell is ~2× esh-ml1's Ada on v0.1.2: 75 vs 144 ms server-side for one face. End to end from nh3-dev it is 127 vs 197 ms.
- On v0.1.2, a GPU buys little over a good CPU. 6 EPYC cores do one face in 152 ms, as fast as esh-ml1's GPU (144 ms). With no face, fv-ml1's CPU (62–72 ms) matches fv-ml1's GPU (70 ms). The GPU pipeline is dominated by something other than GPU compute, and the cards read 0–9% utilisation while it ran. That is consistent with the detector's shape-switching cost that v0.1.3 removes.
- esh-ml1's CPU is not a viable home: 0.9 s for one face and 1.3 s for two. Its embedder costs ~730 ms per face, against ~80 ms on 6 EPYC cores. That gap is much larger than the detector gap (159 vs 72 ms), and its runs were the noisiest (793–1023 ms), so treat the esh CPU number as indicative. It was also sharing its 6 threads with TEI.
- More EPYC threads do not help one frame: CPU24 is roughly equal to CPU6 on faces.
Controls and sensitivity
- Repeats: 3 runs × 20 per cell. The A-vs-A spread is the min–max above.
- Noise floor: the GPU cells spread by ≤6 ms across runs, and the fv CPU cells by ≤20 ms. The esh-ml1 CPU (a) cell spread by 230 ms. Differences smaller than those spreads are not findings.
- Positive control (instrument agreement): augaman-dev measured esh-ml1 (a) independently at 186 ms with a 16 ms floor. This harness gets 196.5 ms with a 28 ms floor. Minus each floor, 170 vs 169 ms: they agree.
- Positive control (detects work): frame (c) is fastest on every target, and the gap from (a) to (c) is the embedding cost. It is visible on every device.
- Negative control: the face count was checked on all 900 responses, and none were wrong.