vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(soong-lab-ci): correct webhook runbook — root cause was ufw firewall (not SSRF)

Browse Source 
The auto-deploy silently never worked: corviduo-dev's ufw is default-deny and
port 9010 was never allowed, so gitea's webhook deliveries timed out (DROP).
v0.3.6 was a manual deploy; v0.3.7-v0.3.13 never auto-deployed. The setup-time
'test-delivery 204' was gitea queuing, not the listener receiving. Fixed by
'ufw allow from 10.0.0.0/8' (operator-directed). Confirmed end-to-end.
This commit is contained in:
vh
2026-07-14 08:29:28 -07:00
parent f960a73a79
commit b195815586
1 changed files with 16 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/runbooks/soong-lab-autodeploy.md
+16 -2
View File
@@ -56,8 +56,22 @@ ssh corviduo-dev 'bash ~/soong-lab-deploy.sh'
it wasn't present initially).
- **bifrost dep** resolves from the internal Gitea PyPI via `~/.netrc` (already
present on corviduo-dev); no extra auth in the deploy script.
- **SSRF**: gitea reached corviduo-dev `10.250.50.152` fine (test-delivery 204) —
no `ALLOWED_HOST_LIST` relax needed (unlike the ytvc/WG case).
- **⚠️ ufw firewall — the auto-deploy silently never worked until 2026-07-14.**
The webhook LISTENER binds `0.0.0.0:9010` fine, but corviduo-dev's ufw is
`default-deny` (only 22 + 8080 were allowed), so gitea's deliveries from
ana-docker **silently timed out (DROP)** — every push landed as a no-op and
the studio drifted (v0.3.6 was a manual deploy; v0.3.7–v0.3.13 never
auto-deployed). The setup-time "test-delivery 204" was a RED HERRING: 204 is
gitea *queuing* the delivery, NOT the listener receiving it — it never proved
reachability. **Diagnosis signal**: `ssh ana-docker 'curl -m8 http://10.250.50.152:9010/'`
→ HTTP 000 timeout while `ssh corviduo-dev 'curl localhost:9010'` → 200 = a
firewall/bind gap, not a listener bug. **Fix**: `sudo ufw allow from 10.0.0.0/8`
on corviduo (operator-directed 2026-07-14 — "that footgun happens a lot", so
accept the whole internal fleet rather than per-port). Confirmed end-to-end:
gitea→:9010 = 200, a signed `refs/heads/main` push → listener 202 → green
deploy; gitea hook secret force-synced to the listener's. The listener's HMAC
secret is the auth layer, so 10/8 exposure is fine. (NOT an `ALLOWED_HOST_LIST`
/ SSRF issue — gitea's egress allowlist was never the blocker here.)
- **Red-run push-notify** via an **althing relay on nh3-dev** (`soong-ci-relay.timer`,
2-min poll of corviduo's `last-deploy.json` → pings **soong-dev** via althing on a
NEW red run; green runs stay silent = fire-and-forget). corviduo itself has no