feat(augaman): v0.1.3 on esh-ml1 and fv-ml1; after-bench: GPU ~3x faster, CPU mode regressed

Both hosts are rebuilt from tag v0.1.3 (one ONNX session per detector canvas) and
redeployed. pytest -m gpu tests/vision passes 3/3 on each card.

Server-side for one face, same harness as the v0.1.2 baseline:
- esh-ml1 GPU 144 -> 48 ms
- fv-ml1 GPU 75 -> 27 ms
End to end from nh3-dev: 101.6 and 73.8 ms.

CPU mode got slower on every CPU target: fv-ml1 cpuset 0-5 went 152 -> 205 ms
with a face, and the no-face frame roughly doubled. That is well outside the
run-to-run spread. The suspected cause (not measured) is per-session ORT
thread pools spinning. Reported to augaman-dev. Neither deployment uses CPU
mode.

On esh-ml1 the dependency layer missed the build cache and the rootfs touched
90% until the v0.1.2 image was removed. fv-ml1's build hit the cache, and the
exported requirements are identical, so the stack README now says to check
disk before building on esh-ml1.
This commit is contained in:
vh
2026-09-27 01:13:01 -07:00
parent 317868dc7e
commit ef64a69e30
4 changed files with 589 additions and 6 deletions
+1 -1
View File
@@ -6,7 +6,7 @@
# compose.yaml. Do not add an env_file.
# Built locally on esh-ml1 from the release tag (see README "Building").
IMAGE=augaman:0.1.2
IMAGE=augaman:0.1.3
PORT=8040
HOST_IP=10.0.50.80
+9 -2
View File
@@ -60,7 +60,8 @@ cd /opt/docker/src/augaman-vX.Y.Z && docker build -t augaman:X.Y.Z .
The build fetches the two pinned models and SHA-256-checks them; a mismatch fails
the build. v0.1.0 built in under 2 minutes cold; v0.1.1 was the first version
deployed (2026-09-26), then v0.1.2 (2026-09-27).
deployed (2026-09-26), then v0.1.2 and v0.1.3 (2026-09-27; v0.1.3 gives each
detector canvas its own session, about +190 MiB VRAM on esh-ml1).
⚠ **Disk:** a build that has to install the third-party packages takes ~8–11 GB
transiently (image ~5 GB plus the ~3 GB uv download cache). Beszel alerts at 85%.
@@ -69,7 +70,13 @@ From v0.1.2 the packages install from a layer keyed on `uv export
--no-emit-project`, so a bump that changes no dependency reuses that layer. Keep
the layer cache (`docker builder prune --filter type=exec.cachemount` drops only
the download cache); a full `docker builder prune` forces the next build to
re-install everything.
re-install everything. ⚠ **The cache did not survive on esh-ml1 for v0.1.3** (the
deps layer re-ran for 1m43s, and the rootfs touched 90% until the old image was
removed). On fv-ml1 the same build was CACHED (49 s), and the exported requirements
were byte-identical across the two versions, so the layering itself works. The
esh-ml1 miss is unexplained; suspect the cache pruning done there after v0.1.2.
**On esh-ml1, check `df` before a build, and have ≥15 GB free or remove the old
image first.**
## Startup is fail-closed on CUDA