From f27ee47facf43f76db242737d998c608d8abc20f Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Tue, 2 Jun 2026 22:37:08 -0700 Subject: [PATCH] feat(nh3-egress-proxy): durable SOCKS5 egress on nh3-dev for datacenter-IP-gated colo services MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit YouTube (and a growing set of services) hard-flag datacenter IPs, bot-gating even public content regardless of cookies/PO-tokens. Origin case: yt-voice-clipper on irv-ml1 (Irvine colo) — every yt-dlp fetch returned LOGIN_REQUIRED. Confirmed pure IP reputation: the same public video fetches cleanly (no cookies) once routed through nh3-dev's residential egress (70.230.226.88). - scripts/setup-nh3-egress-proxy.sh: idempotent dante (SOCKS5) install + config. Internal-only ACL (10.100.0.0/16), bound to the WG interface, systemd-managed. - docs/runbooks/nh3-egress-proxy.md: purpose, usage, security model, caveats. Reusable fleet egress, not yt-voice-clipper-specific. --- docs/runbooks/nh3-egress-proxy.md | 75 +++++++++++++++++++++++++++ scripts/setup-nh3-egress-proxy.sh | 85 +++++++++++++++++++++++++++++++ 2 files changed, 160 insertions(+) create mode 100644 docs/runbooks/nh3-egress-proxy.md create mode 100755 scripts/setup-nh3-egress-proxy.sh diff --git a/docs/runbooks/nh3-egress-proxy.md b/docs/runbooks/nh3-egress-proxy.md new file mode 100644 index 0000000..557b003 --- /dev/null +++ b/docs/runbooks/nh3-egress-proxy.md @@ -0,0 +1,75 @@ +# NH3 egress proxy (nh3-dev) + +A durable, **internal-only** SOCKS5 egress proxy on `nh3-dev` +(`10.100.10.50:1080`) for colo/fleet services that get gated on their +**datacenter IP**. Traffic proxied through it exits via NH3's +**residential** line (egress ~`70.230.226.88`), which is not on the +datacenter blocklists that flag the Anaheim/Irvine colo ranges. + +## Why it exists + +YouTube (and a growing set of services) hard-flag datacenter IPs — +returning `LOGIN_REQUIRED` / "Sign in to confirm you're not a bot" even +for public content, regardless of cookies or PO-tokens. The origin case +was **yt-voice-clipper** on `irv-ml1` (Irvine colo): every yt-dlp fetch +was bot-gated. Confirmed it was purely IP reputation — the *same public +video fetched cleanly with no cookies* once routed through nh3-dev's +residential egress. This is **not** yt-voice-clipper-specific; it's a +reusable egress for any gated colo service. + +## What it is + +- **Daemon:** `dante-server` (danted), SOCKS5. +- **Listener:** `10.100.10.50:1080`, bound to the WG/LAN interface + (`ens18`) only. +- **Egress:** nh3-dev's default route (`ens18` → residential uplink). +- **ACL:** client connections restricted to `10.100.0.0/16` (the + WG/LAN). **NOT an open proxy** — an open proxy on a residential IP + gets abused within hours and the IP flagged, defeating the purpose. +- **Managed by:** [`scripts/setup-nh3-egress-proxy.sh`](../../scripts/setup-nh3-egress-proxy.sh) + (idempotent; writes `/etc/danted.conf`, enables the systemd service). + +## How to use it + +From any host in the WG/LAN (`10.100.0.0/16`), point the client at: + +``` +socks5h://10.100.10.50:1080 +``` + +(the `h` = resolve DNS through the proxy). Examples: + +```bash +curl --socks5-hostname 10.100.10.50:1080 https://api.ipify.org # -> 70.230.226.88 +yt-dlp --proxy socks5h://10.100.10.50:1080 -f bestaudio -g +``` + +**Scope the proxy to only the gated calls.** Don't blanket-proxy a +whole service through it — large/un-gated transfers (model downloads, +bulk data) should go direct so they don't burn the residential line's +bandwidth. For yt-voice-clipper this means proxying the YouTube yt-dlp +calls only, not the whisperx/pyannote/HF model fetches. + +## Caveats + +- **Residential bandwidth is shared.** Every service pointed here egresses + via one home broadband line. Watch aggregate volume / any data cap. +- **SPOF for whatever depends on it.** If nh3-dev or the proxy is down, + every dependent service loses its un-gated egress. It's a convenience + layer, not HA. +- **Reboot durability:** the systemd unit is enabled, but it must come up + after `ens18` has `10.100.10.50` (dante fails to bind otherwise). If it + doesn't start after a reboot, `systemctl restart danted` once the + interface is up. + +## Re-provision / change + +Edit `scripts/setup-nh3-egress-proxy.sh` (the config is baked into it as +a heredoc — single source of truth), then re-run: + +```bash +sudo bash scripts/setup-nh3-egress-proxy.sh +``` + +Verify: `systemctl is-active danted` and +`curl --socks5-hostname 10.100.10.50:1080 https://api.ipify.org`. diff --git a/scripts/setup-nh3-egress-proxy.sh b/scripts/setup-nh3-egress-proxy.sh new file mode 100755 index 0000000..a5c9cb3 --- /dev/null +++ b/scripts/setup-nh3-egress-proxy.sh @@ -0,0 +1,85 @@ +#!/usr/bin/env bash +# NH3 egress proxy — durable SOCKS5 (dante) on nh3-dev (10.100.10.50:1080). +# +# Purpose: a reusable, INTERNAL-ONLY SOCKS5 egress for colo/fleet services that +# get gated on their datacenter IP (e.g. YouTube's bot-gate on irv-ml1). Traffic +# proxied here exits via nh3-dev's residential line (egress ~70.230.226.88), which +# is not on the datacenter blocklists. +# +# Security: client ACL restricts connections to the WG/LAN (10.100.0.0/16) and the +# listener binds only the internal interface. This is NOT an open proxy — an open +# proxy on a residential IP would be abused within hours and get the IP flagged, +# defeating the purpose. Do not widen `client pass` to 0.0.0.0/0. +# +# Idempotent; safe to re-run. Run with sudo: sudo bash scripts/setup-nh3-egress-proxy.sh +set -euo pipefail + +CONF=/etc/danted.conf +BIND_IP=10.100.10.50 +PORT=1080 +EXT_IF=ens18 +ALLOW_NET=10.100.0.0/16 + +echo "==> [1/5] install dante-server" +if dpkg -s dante-server >/dev/null 2>&1; then + echo " dante-server already installed" +else + export DEBIAN_FRONTEND=noninteractive + apt-get update -qq + apt-get install -y -qq dante-server +fi + +echo "==> [2/5] write $CONF" +cat > "$CONF" < [3/5] free port $PORT (stop the ssh -D stopgap if present)" +if pkill -f "ssh -fN -D $BIND_IP:$PORT" 2>/dev/null; then echo " stopgap killed"; else echo " no stopgap running"; fi +sleep 1 + +echo "==> [4/5] enable + (re)start danted" +systemctl unmask danted >/dev/null 2>&1 || true +systemctl enable danted >/dev/null 2>&1 || true +systemctl restart danted +sleep 2 + +echo "==> [5/5] verify" +if systemctl is-active --quiet danted && ss -ltn | grep -q "$BIND_IP:$PORT "; then + echo "OK: danted active and listening on $BIND_IP:$PORT" +else + echo "FAIL: danted not healthy — check: journalctl -u danted -n 30 --no-pager" + systemctl is-active danted || true + exit 1 +fi