vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(soong-lab-ci): sync web/ frontend on deploy (was serving stale web)

Browse Source 
soong-dev found the studio serving a stale web/ (52015 vs 55025 bytes — missing the
01-Role section, favicon, thinking-status): the deploy rsynced backend/ but never web/,
so SOONG_LAB_WEB_DIR stayed pinned to the initial manual copy while the backend updated.
Deploy now rsyncs BOTH backend/->studio AND web/->SOONG_LAB_WEB_DIR (read from the env)
on every green run. Verified: served frontend now 55025 bytes, current.
This commit is contained in:
Vuong Hoang
2026-07-13 15:09:09 -07:00
parent cc0e3af87d
commit e0f1dbfae6
2 changed files with 30 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/runbooks/soong-lab-autodeploy.md
+7 -1
View File
@@ -12,7 +12,7 @@ push→main → gitea webhook (POST, HMAC) → soong-webhook listener :9010 on c
→ ~/soong-lab-deploy.sh:
git clone (read-only deploy key, internal SSH :222)
uv sync ; uv run pytest ── RED → abort, studio UNTOUCHED, status=red
rsync backend/ → studio dir ; uv sync --no-dev ; systemctl restart
rsync backend/ → studio dir + web/ → SOONG_LAB_WEB_DIR ; uv sync --no-dev ; restart
→ status=green, studio healthy
```
@@ -43,6 +43,12 @@ ssh corviduo-dev 'bash ~/soong-lab-deploy.sh'
## Notes / gotchas
- **Frontend (`web/`) sync IS part of the deploy**: the studio serves `web/` from
`SOONG_LAB_WEB_DIR` (`/home/infra-ops/soong-lab/web`), *separate* from the backend
`WorkingDirectory`. The deploy rsyncs BOTH `backend/`→studio and `web/`→`SOONG_LAB_WEB_DIR`.
(Added 2026-07-13 after soong-dev caught the served frontend silently rotting — the backend
was updating while `web/` stayed pinned to the initial manual copy; a bounce alone re-serves
the same stale file.)
- **Green-gated by construction**: `pytest || fail` runs BEFORE any studio touch,
so a red suite aborts with the studio still on the old version. Validated
2026-07-13 (a mid-deploy rsync failure left the studio untouched/active).
services/soong-lab-ci/soong-lab-deploy.sh
+23 -7
View File
@@ -1,18 +1,21 @@
#!/usr/bin/env bash
# soong-lab CI/CD deploy: clone -> uv sync + pytest -> deploy studio ONLY on green.
# Running studio is NEVER touched on a red test run (safe-by-construction).
# soong-lab CI/CD deploy: clone -> uv sync + pytest -> deploy studio (backend + web)
# ONLY on green. Running studio is NEVER touched on a red test run.
set -uo pipefail
LOG=/home/infra-ops/soong-lab-deploy.log
exec >>"$LOG" 2>&1
echo; echo "========== deploy run $(date -u) =========="
SRC=/tmp/soong-lab-ci
STUDIO=/home/infra-ops/soong-lab/backend
ENVF=/home/infra-ops/soong-lab/soong-lab.env
WEBDIR=$(grep -E '^SOONG_LAB_WEB_DIR=' "$ENVF" 2>/dev/null | cut -d= -f2- | tr -d '"')
KEY=/home/infra-ops/.ssh/soong-deploy_ed25519
UV=/home/infra-ops/.local/bin/uv
STATUS=/home/infra-ops/.config/soong/last-deploy.json
export GIT_SSH_COMMAND="ssh -i $KEY -o IdentitiesOnly=yes -p 222"
SHA="?"
fail(){ echo "DEPLOY ABORTED [$2]: $1"; printf "{\"result\":\"red\",\"stage\":\"%s\",\"detail\":\"%s\",\"sha\":\"%s\",\"at\":\"%s\"}\n" "$2" "$1" "$SHA" "$(date -u +%FT%TZ)" > "$STATUS"; exit 1; }
fail(){ echo "DEPLOY ABORTED [$2]: $1"; printf '{"result":"red","stage":"%s","detail":"%s","sha":"%s","at":"%s"}\n' "$2" "$1" "$SHA" "$(date -u +%FT%TZ)" > "$STATUS"; exit 1; }
rm -rf "$SRC"
git clone --depth 1 ssh://git@10.250.50.70:222/vh/soong-lab.git "$SRC" || fail "clone failed" clone
SHA=$(git -C "$SRC" rev-parse --short HEAD); echo "cloned $SHA"
@@ -20,12 +23,25 @@ cd "$SRC/backend"
"$UV" sync || fail "uv sync (test env) failed" sync
echo "=== test suite ==="
"$UV" run pytest -q || fail "TESTS RED — studio left untouched" tests
echo "=== tests GREEN — deploying to studio ==="
rsync -a --delete --exclude .venv/ --exclude "*.env" --exclude "*.db" --exclude "*.sqlite*" --exclude "data/" --exclude "state/" --exclude "logs/" "$SRC/backend/" "$STUDIO/" || fail "rsync to studio failed" rsync
echo "=== tests GREEN — deploying backend + web ==="
# backend code -> studio WorkingDirectory
rsync -a --delete --exclude .venv/ --exclude '*.env' --exclude '*.db' --exclude '*.sqlite*' \
--exclude 'data/' --exclude 'state/' --exclude 'logs/' "$SRC/backend/" "$STUDIO/" \
|| fail "rsync backend failed" rsync
cd "$STUDIO"
"$UV" sync --no-dev || fail "studio uv sync failed" studio_sync
# frontend: the SERVED web/ (SOONG_LAB_WEB_DIR) must track the repo web/ too, else the
# frontend silently rots while the backend updates (soong-dev caught this 2026-07-13).
if [ -n "$WEBDIR" ] && [ -d "$SRC/web" ]; then
rsync -a --delete "$SRC/web/" "$WEBDIR/" || fail "rsync web failed" web
echo "web/ synced -> $WEBDIR"
else
echo "WARN: SOONG_LAB_WEB_DIR unset or repo web/ missing — skipping web sync (WEBDIR='$WEBDIR')"
fi
sudo systemctl restart soong-lab-studio.service || fail "systemctl restart failed" restart
sleep 3
systemctl is-active --quiet soong-lab-studio.service || fail "studio not active post-restart" health
echo "DEPLOYED $SHA — studio healthy"
printf "{\"result\":\"green\",\"sha\":\"%s\",\"at\":\"%s\"}\n" "$SHA" "$(date -u +%FT%TZ)" > "$STATUS"
echo "DEPLOYED $SHA — studio healthy (backend + web)"
printf '{"result":"green","sha":"%s","at":"%s"}\n' "$SHA" "$(date -u +%FT%TZ)" > "$STATUS"