49 lines
2.5 KiB
Markdown
49 lines
2.5 KiB
Markdown
# zed-fim-proxy
|
||
|
||
A **keyless `/v1/completions` front door** on ana-docker for Zed's editor
|
||
edit-prediction (inline FIM completion) feature, which **cannot send an
|
||
`Authorization` header**. Runs on a separate port from LiteLLM and forwards to it
|
||
with an injected, model-scoped key.
|
||
|
||
- **Host:** ana-docker `10.250.50.70`, port **4141** (`network_mode: host`).
|
||
- **Backs:** the `coder-fast` model (Qwen2.5-Coder-1.5B FIM seat, `stacks/vllm`
|
||
`vllm-coder` on ana-ml2:8020) via LiteLLM `:4000`.
|
||
- **Zed config** (`edit_predictions.open_ai_compatible_api`): `api_url:
|
||
http://10.250.50.70:4141/v1`, `model: coder-fast`, `prompt_format: qwen`,
|
||
`max_output_tokens: <n>`. (The proxy also accepts `http://10.250.50.70:4141` —
|
||
it matches both `/v1/completions` and `/completions`.)
|
||
|
||
## Security model (stdlib proxy in `conf/proxy.py`)
|
||
|
||
Four guards + a scoped key — a keyless route that injects a working key is only
|
||
safe if it can't be pivoted:
|
||
|
||
1. **POST + path** `/v1/completions` (or `/completions`) only. `GET /ping` is an
|
||
anonymous liveness (`{"service":"ok"}`). `/v1/chat/completions` is rejected.
|
||
2. **Model allowlist** — request body `model` must equal `ZED_ALLOWED_MODEL`
|
||
(`coder-fast`); anything else → 403.
|
||
3. **Injected scoped key** — a LiteLLM virtual key scoped to `coder-fast` ONLY
|
||
(`POST /key/generate {"models":["coder-fast"]}`). Even if guards 1–2 were
|
||
bypassed, the key reaches nothing else (verified: 403 on `gen`). **This is the
|
||
real blast-radius bound.**
|
||
4. **Source-IP allowlist** (`ZED_ALLOWED_IPS`) — **intentionally left OFF**
|
||
(operator direction 2026-07-27): Zed roams the operator's WireGuard
|
||
`10.0.0.0/8`, so a single-IP pin would break it. **Do NOT tighten.** The
|
||
keyless route is bounded by guards 1–3 (coder-fast-only, `/v1/completions`-only,
|
||
scoped key) and is internal-network only. (The proxy matches exact IPs; a
|
||
`10.0.0.0/8` CIDR would need CIDR support — deliberately not added.)
|
||
|
||
## Deploy
|
||
|
||
```
|
||
# conf/proxy.py -> /opt/docker/conf/zed-fim-proxy/proxy.py
|
||
# compose.yaml -> /opt/docker/compose/zed-fim-proxy/compose.yaml
|
||
# .env (from .env.example, with ZED_SCOPED_KEY filled) -> same dir, mode 600
|
||
cd /opt/docker/compose/zed-fim-proxy && docker compose up -d
|
||
# verify keyless:
|
||
curl -s http://localhost:4141/v1/completions -H 'Content-Type: application/json' \
|
||
-d '{"model":"coder-fast","prompt":"def add(a,b):\n return","max_tokens":16,"temperature":0.2}'
|
||
```
|
||
|
||
Stdlib-only proxy (no pip) in a bare `python:3.12-slim` container — no build.
|