feat: Zed edit-predictions keyless FIM route (Qwen2.5-Coder-1.5B / coder-fast)

Deep-research-picked Qwen2.5-Coder-1.5B (BASE, Apache-2.0, native FIM) as a
low-latency inline-completion seat:
- stacks/vllm: vllm-coder service (ana-ml2 GPU1 :8020) + granite shrunk
  (util 0.27->0.13, max-len 131072->16384, seqs 1024->256; granite phasing out)
  to free GPU1 room.
- stacks/litellm: coder-fast alias -> :8020 (mode: completion, /v1/completions).
- stacks/zed-fim-proxy (NEW): keyless /v1/completions front door on ana-docker
  :4141 for Zed (which can't send an auth header) — POST + path + model
  allowlist, injects a coder-fast-scoped virtual key -> LiteLLM :4000. Anon
  /ping liveness. Verified keyless FIM end-to-end.

Zed api_url = http://10.250.50.70:4141/v1, model coder-fast, prompt_format qwen.
Source-IP allowlist off pending the Mac's observed source IP.
This commit is contained in:
vh
2026-07-27 22:55:21 -07:00
parent 8822a0bb81
commit a300cdcd26
8 changed files with 315 additions and 6 deletions
+21
View File
@@ -0,0 +1,21 @@
# zed-fim-proxy tunables — copy to `.env` on ana-docker (the real .env holds the
# scoped key and is server-only / gitignored). See conf/proxy.py + README.md.
# Port the keyless route listens on (host network).
ZED_PORT=4141
# LiteLLM gateway to forward to (localhost:4000 via network_mode: host).
ZED_UPSTREAM=http://localhost:4000
# The ONLY model this route will forward (proxy rejects any other "model").
ZED_ALLOWED_MODEL=coder-fast
# Best-effort source-IP allowlist (comma-separated). Empty = allow all — fine on the
# internal network, but TIGHTEN to the Mac's observed source IP once it connects
# (watch `docker logs zed-fim-proxy` for the real source). Only meaningful if the
# proxy sees the real client IP (network_mode: host); a site-to-site NAT may mask it.
ZED_ALLOWED_IPS=
# A LiteLLM virtual key SCOPED TO ZED_ALLOWED_MODEL ONLY (the real blast-radius
# bound). Mint: POST /key/generate {"models":["coder-fast"]}. NEVER commit the value.
ZED_SCOPED_KEY=