feat(nh3-egress-proxy): durable SOCKS5 egress on nh3-dev for datacenter-IP-gated colo services
YouTube (and a growing set of services) hard-flag datacenter IPs, bot-gating even public content regardless of cookies/PO-tokens. Origin case: yt-voice-clipper on irv-ml1 (Irvine colo) — every yt-dlp fetch returned LOGIN_REQUIRED. Confirmed pure IP reputation: the same public video fetches cleanly (no cookies) once routed through nh3-dev's residential egress (70.230.226.88). - scripts/setup-nh3-egress-proxy.sh: idempotent dante (SOCKS5) install + config. Internal-only ACL (10.100.0.0/16), bound to the WG interface, systemd-managed. - docs/runbooks/nh3-egress-proxy.md: purpose, usage, security model, caveats. Reusable fleet egress, not yt-voice-clipper-specific.
This commit is contained in:
@@ -0,0 +1,75 @@
|
||||
# NH3 egress proxy (nh3-dev)
|
||||
|
||||
A durable, **internal-only** SOCKS5 egress proxy on `nh3-dev`
|
||||
(`10.100.10.50:1080`) for colo/fleet services that get gated on their
|
||||
**datacenter IP**. Traffic proxied through it exits via NH3's
|
||||
**residential** line (egress ~`70.230.226.88`), which is not on the
|
||||
datacenter blocklists that flag the Anaheim/Irvine colo ranges.
|
||||
|
||||
## Why it exists
|
||||
|
||||
YouTube (and a growing set of services) hard-flag datacenter IPs —
|
||||
returning `LOGIN_REQUIRED` / "Sign in to confirm you're not a bot" even
|
||||
for public content, regardless of cookies or PO-tokens. The origin case
|
||||
was **yt-voice-clipper** on `irv-ml1` (Irvine colo): every yt-dlp fetch
|
||||
was bot-gated. Confirmed it was purely IP reputation — the *same public
|
||||
video fetched cleanly with no cookies* once routed through nh3-dev's
|
||||
residential egress. This is **not** yt-voice-clipper-specific; it's a
|
||||
reusable egress for any gated colo service.
|
||||
|
||||
## What it is
|
||||
|
||||
- **Daemon:** `dante-server` (danted), SOCKS5.
|
||||
- **Listener:** `10.100.10.50:1080`, bound to the WG/LAN interface
|
||||
(`ens18`) only.
|
||||
- **Egress:** nh3-dev's default route (`ens18` → residential uplink).
|
||||
- **ACL:** client connections restricted to `10.100.0.0/16` (the
|
||||
WG/LAN). **NOT an open proxy** — an open proxy on a residential IP
|
||||
gets abused within hours and the IP flagged, defeating the purpose.
|
||||
- **Managed by:** [`scripts/setup-nh3-egress-proxy.sh`](../../scripts/setup-nh3-egress-proxy.sh)
|
||||
(idempotent; writes `/etc/danted.conf`, enables the systemd service).
|
||||
|
||||
## How to use it
|
||||
|
||||
From any host in the WG/LAN (`10.100.0.0/16`), point the client at:
|
||||
|
||||
```
|
||||
socks5h://10.100.10.50:1080
|
||||
```
|
||||
|
||||
(the `h` = resolve DNS through the proxy). Examples:
|
||||
|
||||
```bash
|
||||
curl --socks5-hostname 10.100.10.50:1080 https://api.ipify.org # -> 70.230.226.88
|
||||
yt-dlp --proxy socks5h://10.100.10.50:1080 -f bestaudio -g <url>
|
||||
```
|
||||
|
||||
**Scope the proxy to only the gated calls.** Don't blanket-proxy a
|
||||
whole service through it — large/un-gated transfers (model downloads,
|
||||
bulk data) should go direct so they don't burn the residential line's
|
||||
bandwidth. For yt-voice-clipper this means proxying the YouTube yt-dlp
|
||||
calls only, not the whisperx/pyannote/HF model fetches.
|
||||
|
||||
## Caveats
|
||||
|
||||
- **Residential bandwidth is shared.** Every service pointed here egresses
|
||||
via one home broadband line. Watch aggregate volume / any data cap.
|
||||
- **SPOF for whatever depends on it.** If nh3-dev or the proxy is down,
|
||||
every dependent service loses its un-gated egress. It's a convenience
|
||||
layer, not HA.
|
||||
- **Reboot durability:** the systemd unit is enabled, but it must come up
|
||||
after `ens18` has `10.100.10.50` (dante fails to bind otherwise). If it
|
||||
doesn't start after a reboot, `systemctl restart danted` once the
|
||||
interface is up.
|
||||
|
||||
## Re-provision / change
|
||||
|
||||
Edit `scripts/setup-nh3-egress-proxy.sh` (the config is baked into it as
|
||||
a heredoc — single source of truth), then re-run:
|
||||
|
||||
```bash
|
||||
sudo bash scripts/setup-nh3-egress-proxy.sh
|
||||
```
|
||||
|
||||
Verify: `systemctl is-active danted` and
|
||||
`curl --socks5-hostname 10.100.10.50:1080 https://api.ipify.org`.
|
||||
Executable
+85
@@ -0,0 +1,85 @@
|
||||
#!/usr/bin/env bash
|
||||
# NH3 egress proxy — durable SOCKS5 (dante) on nh3-dev (10.100.10.50:1080).
|
||||
#
|
||||
# Purpose: a reusable, INTERNAL-ONLY SOCKS5 egress for colo/fleet services that
|
||||
# get gated on their datacenter IP (e.g. YouTube's bot-gate on irv-ml1). Traffic
|
||||
# proxied here exits via nh3-dev's residential line (egress ~70.230.226.88), which
|
||||
# is not on the datacenter blocklists.
|
||||
#
|
||||
# Security: client ACL restricts connections to the WG/LAN (10.100.0.0/16) and the
|
||||
# listener binds only the internal interface. This is NOT an open proxy — an open
|
||||
# proxy on a residential IP would be abused within hours and get the IP flagged,
|
||||
# defeating the purpose. Do not widen `client pass` to 0.0.0.0/0.
|
||||
#
|
||||
# Idempotent; safe to re-run. Run with sudo: sudo bash scripts/setup-nh3-egress-proxy.sh
|
||||
set -euo pipefail
|
||||
|
||||
CONF=/etc/danted.conf
|
||||
BIND_IP=10.100.10.50
|
||||
PORT=1080
|
||||
EXT_IF=ens18
|
||||
ALLOW_NET=10.100.0.0/16
|
||||
|
||||
echo "==> [1/5] install dante-server"
|
||||
if dpkg -s dante-server >/dev/null 2>&1; then
|
||||
echo " dante-server already installed"
|
||||
else
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
apt-get update -qq
|
||||
apt-get install -y -qq dante-server
|
||||
fi
|
||||
|
||||
echo "==> [2/5] write $CONF"
|
||||
cat > "$CONF" <<EOF
|
||||
# NH3 egress proxy (PFI infra) — SOCKS5, INTERNAL-ONLY.
|
||||
# Reusable egress for colo services gated on their datacenter IP; exits via the
|
||||
# nh3-dev residential line. Managed by scripts/setup-nh3-egress-proxy.sh in the
|
||||
# eshpfi-management repo — edit there, not here.
|
||||
logoutput: syslog
|
||||
internal: $BIND_IP port = $PORT
|
||||
external: $EXT_IF
|
||||
|
||||
socksmethod: none
|
||||
clientmethod: none
|
||||
user.privileged: root
|
||||
user.unprivileged: nobody
|
||||
|
||||
# Client ACL — only the WG/LAN may connect. NOT an open proxy.
|
||||
client pass {
|
||||
from: $ALLOW_NET to: 0.0.0.0/0
|
||||
log: error
|
||||
}
|
||||
client block {
|
||||
from: 0.0.0.0/0 to: 0.0.0.0/0
|
||||
log: connect error
|
||||
}
|
||||
|
||||
# SOCKS rules — internal clients may egress anywhere.
|
||||
socks pass {
|
||||
from: $ALLOW_NET to: 0.0.0.0/0
|
||||
protocol: tcp udp
|
||||
log: error
|
||||
}
|
||||
socks block {
|
||||
from: 0.0.0.0/0 to: 0.0.0.0/0
|
||||
}
|
||||
EOF
|
||||
|
||||
echo "==> [3/5] free port $PORT (stop the ssh -D stopgap if present)"
|
||||
if pkill -f "ssh -fN -D $BIND_IP:$PORT" 2>/dev/null; then echo " stopgap killed"; else echo " no stopgap running"; fi
|
||||
sleep 1
|
||||
|
||||
echo "==> [4/5] enable + (re)start danted"
|
||||
systemctl unmask danted >/dev/null 2>&1 || true
|
||||
systemctl enable danted >/dev/null 2>&1 || true
|
||||
systemctl restart danted
|
||||
sleep 2
|
||||
|
||||
echo "==> [5/5] verify"
|
||||
if systemctl is-active --quiet danted && ss -ltn | grep -q "$BIND_IP:$PORT "; then
|
||||
echo "OK: danted active and listening on $BIND_IP:$PORT"
|
||||
else
|
||||
echo "FAIL: danted not healthy — check: journalctl -u danted -n 30 --no-pager"
|
||||
systemctl is-active danted || true
|
||||
exit 1
|
||||
fi
|
||||
Reference in New Issue
Block a user