From b19581558633dcf9ad1e17754ec4413437fbcb3d Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Tue, 14 Jul 2026 08:29:28 -0700 Subject: [PATCH] =?UTF-8?q?docs(soong-lab-ci):=20correct=20webhook=20runbo?= =?UTF-8?q?ok=20=E2=80=94=20root=20cause=20was=20ufw=20firewall=20(not=20S?= =?UTF-8?q?SRF)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The auto-deploy silently never worked: corviduo-dev's ufw is default-deny and port 9010 was never allowed, so gitea's webhook deliveries timed out (DROP). v0.3.6 was a manual deploy; v0.3.7-v0.3.13 never auto-deployed. The setup-time 'test-delivery 204' was gitea queuing, not the listener receiving. Fixed by 'ufw allow from 10.0.0.0/8' (operator-directed). Confirmed end-to-end. --- docs/runbooks/soong-lab-autodeploy.md | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/docs/runbooks/soong-lab-autodeploy.md b/docs/runbooks/soong-lab-autodeploy.md index 803e2ab..0d2db16 100644 --- a/docs/runbooks/soong-lab-autodeploy.md +++ b/docs/runbooks/soong-lab-autodeploy.md @@ -56,8 +56,22 @@ ssh corviduo-dev 'bash ~/soong-lab-deploy.sh' it wasn't present initially). - **bifrost dep** resolves from the internal Gitea PyPI via `~/.netrc` (already present on corviduo-dev); no extra auth in the deploy script. -- **SSRF**: gitea reached corviduo-dev `10.250.50.152` fine (test-delivery 204) — - no `ALLOWED_HOST_LIST` relax needed (unlike the ytvc/WG case). +- **⚠️ ufw firewall — the auto-deploy silently never worked until 2026-07-14.** + The webhook LISTENER binds `0.0.0.0:9010` fine, but corviduo-dev's ufw is + `default-deny` (only 22 + 8080 were allowed), so gitea's deliveries from + ana-docker **silently timed out (DROP)** — every push landed as a no-op and + the studio drifted (v0.3.6 was a manual deploy; v0.3.7–v0.3.13 never + auto-deployed). The setup-time "test-delivery 204" was a RED HERRING: 204 is + gitea *queuing* the delivery, NOT the listener receiving it — it never proved + reachability. **Diagnosis signal**: `ssh ana-docker 'curl -m8 http://10.250.50.152:9010/'` + → HTTP 000 timeout while `ssh corviduo-dev 'curl localhost:9010'` → 200 = a + firewall/bind gap, not a listener bug. **Fix**: `sudo ufw allow from 10.0.0.0/8` + on corviduo (operator-directed 2026-07-14 — "that footgun happens a lot", so + accept the whole internal fleet rather than per-port). Confirmed end-to-end: + gitea→:9010 = 200, a signed `refs/heads/main` push → listener 202 → green + deploy; gitea hook secret force-synced to the listener's. The listener's HMAC + secret is the auth layer, so 10/8 exposure is fine. (NOT an `ALLOWED_HOST_LIST` + / SSRF issue — gitea's egress allowlist was never the blocker here.) - **Red-run push-notify** via an **althing relay on nh3-dev** (`soong-ci-relay.timer`, 2-min poll of corviduo's `last-deploy.json` → pings **soong-dev** via althing on a NEW red run; green runs stay silent = fire-and-forget). corviduo itself has no