esh-pfi-infrastructure/stacks/zed-fim-proxy/README.md

zed-fim-proxy

A keyless /v1/completions front door on ana-docker for Zed's editor edit-prediction (inline FIM completion) feature, which cannot send an Authorization header. Runs on a separate port from LiteLLM and forwards to it with an injected, model-scoped key.

• Host: ana-docker 10.250.50.70, port 4141 (network_mode: host).
• Backs: the coder-fast model (Qwen2.5-Coder-1.5B FIM seat, stacks/vllm vllm-coder on ana-ml2:8020) via LiteLLM :4000.
• Zed config (edit_predictions.open_ai_compatible_api): api_url: http://10.250.50.70:4141/v1, model: coder-fast, prompt_format: qwen, max_output_tokens: <n>. (The proxy also accepts http://10.250.50.70:4141 — it matches both /v1/completions and /completions.)

Security model (stdlib proxy in conf/proxy.py)

Four guards + a scoped key — a keyless route that injects a working key is only safe if it can't be pivoted:

1. POST + path /v1/completions (or /completions) only. GET /ping is an anonymous liveness ({"service":"ok"}). /v1/chat/completions is rejected.
2. Model allowlist — request body model must equal ZED_ALLOWED_MODEL (coder-fast); anything else → 403.
3. Injected scoped key — a LiteLLM virtual key scoped to coder-fast ONLY (POST /key/generate {"models":["coder-fast"]}). Even if guards 1–2 were bypassed, the key reaches nothing else (verified: 403 on gen). This is the real blast-radius bound.
4. Source-IP allowlist (ZED_ALLOWED_IPS) — intentionally left OFF (operator direction 2026-07-27): Zed roams the operator's WireGuard 10.0.0.0/8, so a single-IP pin would break it. Do NOT tighten. The keyless route is bounded by guards 1–3 (coder-fast-only, /v1/completions-only, scoped key) and is internal-network only. (The proxy matches exact IPs; a 10.0.0.0/8 CIDR would need CIDR support — deliberately not added.)

Deploy

# conf/proxy.py -> /opt/docker/conf/zed-fim-proxy/proxy.py
# compose.yaml  -> /opt/docker/compose/zed-fim-proxy/compose.yaml
# .env (from .env.example, with ZED_SCOPED_KEY filled) -> same dir, mode 600
cd /opt/docker/compose/zed-fim-proxy && docker compose up -d
# verify keyless:
curl -s http://localhost:4141/v1/completions -H 'Content-Type: application/json' \
  -d '{"model":"coder-fast","prompt":"def add(a,b):\n    return","max_tokens":16,"temperature":0.2}'

Stdlib-only proxy (no pip) in a bare python:3.12-slim container — no build.