Files

T

vh a300cdcd26 feat: Zed edit-predictions keyless FIM route (Qwen2.5-Coder-1.5B / coder-fast)

Deep-research-picked Qwen2.5-Coder-1.5B (BASE, Apache-2.0, native FIM) as a
low-latency inline-completion seat:
- stacks/vllm: vllm-coder service (ana-ml2 GPU1 :8020) + granite shrunk
  (util 0.27->0.13, max-len 131072->16384, seqs 1024->256; granite phasing out)
  to free GPU1 room.
- stacks/litellm: coder-fast alias -> :8020 (mode: completion, /v1/completions).
- stacks/zed-fim-proxy (NEW): keyless /v1/completions front door on ana-docker
  :4141 for Zed (which can't send an auth header) — POST + path + model
  allowlist, injects a coder-fast-scoped virtual key -> LiteLLM :4000. Anon
  /ping liveness. Verified keyless FIM end-to-end.

Zed api_url = http://10.250.50.70:4141/v1, model coder-fast, prompt_format qwen.
Source-IP allowlist off pending the Mac's observed source IP.

2026-07-27 22:55:21 -07:00

2.4 KiB

Raw Blame History

zed-fim-proxy

A keyless /v1/completions front door on ana-docker for Zed's editor edit-prediction (inline FIM completion) feature, which cannot send an Authorization header. Runs on a separate port from LiteLLM and forwards to it with an injected, model-scoped key.

Host: ana-docker 10.250.50.70, port 4141 (network_mode: host).
Backs: the coder-fast model (Qwen2.5-Coder-1.5B FIM seat, stacks/vllm vllm-coder on ana-ml2:8020) via LiteLLM :4000.
Zed config (edit_predictions.open_ai_compatible_api): api_url: http://10.250.50.70:4141/v1, model: coder-fast, prompt_format: qwen, max_output_tokens: <n>. (The proxy also accepts http://10.250.50.70:4141 — it matches both /v1/completions and /completions.)

Security model (stdlib proxy in `conf/proxy.py`)

Four guards + a scoped key — a keyless route that injects a working key is only safe if it can't be pivoted:

POST + path /v1/completions (or /completions) only. GET /ping is an anonymous liveness ({"service":"ok"}). /v1/chat/completions is rejected.
Model allowlist — request body model must equal ZED_ALLOWED_MODEL (coder-fast); anything else → 403.
Injected scoped key — a LiteLLM virtual key scoped to coder-fast ONLY (POST /key/generate {"models":["coder-fast"]}). Even if guards 1–2 were bypassed, the key reaches nothing else (verified: 403 on gen). This is the real blast-radius bound.
Best-effort source-IP allowlist (ZED_ALLOWED_IPS) — only enforceable if the proxy sees the real client IP (hence network_mode: host; docker port-publish would NAT it away). A site-to-site NAT may still mask the Mac's 10.0.10.83 — verify against docker logs zed-fim-proxy and tighten. Internal network only; no public exposure.

Deploy

# conf/proxy.py -> /opt/docker/conf/zed-fim-proxy/proxy.py
# compose.yaml  -> /opt/docker/compose/zed-fim-proxy/compose.yaml
# .env (from .env.example, with ZED_SCOPED_KEY filled) -> same dir, mode 600
cd /opt/docker/compose/zed-fim-proxy && docker compose up -d
# verify keyless:
curl -s http://localhost:4141/v1/completions -H 'Content-Type: application/json' \
  -d '{"model":"coder-fast","prompt":"def add(a,b):\n    return","max_tokens":16,"temperature":0.2}'

Stdlib-only proxy (no pip) in a bare python:3.12-slim container — no build.

2.4 KiB Raw Blame History Unescape Escape

zed-fim-proxy

Security model (stdlib proxy in conf/proxy.py)

Deploy

2.4 KiB

Raw Blame History

Security model (stdlib proxy in `conf/proxy.py`)