feat(soong-lab-ci): green-gated push-to-deploy CI/CD for the soong-lab studio

Vuong-directed. gitea webhook (push→main) → HMAC listener on corviduo-dev:9010 → clone (read-only deploy key) → uv sync + pytest → redeploy soong-lab-studio.service ONLY on green (running studio untouched on red). Validated end-to-end 2026-07-13. Canonical copies of the deploy script + listener + unit; runbook in docs/runbooks.
2026-07-13 14:16:26 -07:00
parent 85792f4b55
commit fb556586e3
4 changed files with 129 additions and 0 deletions
@@ -0,0 +1,58 @@
+# soong-lab push-to-deploy (gitea webhook → corviduo-dev, test-gated)
+
+Green-gated CI/CD for the soong-lab studio: **push to `main` → run the test
+suite → redeploy the studio ONLY if tests pass** (running studio is never
+touched on a red run). Built 2026-07-13 (Vuong-directed). Adapts the
+[ytvc-autodeploy](./ytvc-autodeploy.md) webhook pattern.
+
+## Flow
+
+```
+push→main → gitea webhook (POST, HMAC) → soong-webhook listener :9010 on corviduo-dev
+  → ~/soong-lab-deploy.sh:
+       git clone (read-only deploy key, internal SSH :222)
+       uv sync ; uv run pytest        ── RED → abort, studio UNTOUCHED, status=red
+       rsync backend/ → studio dir ; uv sync --no-dev ; systemctl restart
+       → status=green, studio healthy
+```
+
+## Components (all on corviduo-dev, user `infra-ops`)
+
+- `~/soong-lab-deploy.sh` — clone → test → deploy-on-green. Logs to
+  `~/soong-lab-deploy.log`; writes `~/.config/soong/last-deploy.json`
+  (`{result: green|red, stage, sha, at}`).
+- `~/soong-webhook.py` — HTTP listener on `:9010`. HMAC-SHA256 (`X-Gitea-Signature`)
+  vs `~/.config/soong/webhook-secret` (mode 600); fires the deploy only on
+  `ref == refs/heads/main`. `GET /` returns `ok | last: <status>`.
+- `soong-webhook.service` (system unit, enabled) — runs the listener.
+- Read-only deploy key `~/.ssh/soong-deploy_ed25519` → gitea repo key id 5 on
+  `vh/soong-lab` (read_only). Clone via `ssh://git@10.250.50.70:222/vh/soong-lab.git`.
+- Studio unit `soong-lab-studio.service` (WD `/home/infra-ops/soong-lab/backend`);
+  restart needs infra-ops NOPASSWD sudo (present).
+- Gitea webhook: repo `vh/soong-lab` hook id 3 → `http://10.250.50.152:9010/`,
+  JSON, Push events, the shared secret.
+
+## Verify / operate
+
+```bash
+ssh corviduo-dev 'systemctl is-active soong-webhook.service; curl -s localhost:9010/'
+ssh corviduo-dev 'tail -30 ~/soong-lab-deploy.log'          # deploy history
+# manual deploy (same as the webhook does):
+ssh corviduo-dev 'bash ~/soong-lab-deploy.sh'
+```
+
+## Notes / gotchas
+
+- **Green-gated by construction**: `pytest || fail` runs BEFORE any studio touch,
+  so a red suite aborts with the studio still on the old version. Validated
+  2026-07-13 (a mid-deploy rsync failure left the studio untouched/active).
+- **rsync is required** on corviduo-dev (`apt install rsync` — installed 2026-07-13;
+  it wasn't present initially).
+- **bifrost dep** resolves from the internal Gitea PyPI via `~/.netrc` (already
+  present on corviduo-dev); no extra auth in the deploy script.
+- **SSRF**: gitea reached corviduo-dev `10.250.50.152` fine (test-delivery 204) —
+  no `ALLOWED_HOST_LIST` relax needed (unlike the ytvc/WG case).
+- **No althing on corviduo-dev** → red-run notify is log/status-file based
+  (`last-deploy.json` + `GET :9010`). A gitea commit-status or althing relay
+  could be added if push-notify on red is wanted.
+- Test suite: `uv run pytest` in `backend/` (242 tests as of v0.3.6).