feat: Zed edit-predictions keyless FIM route (Qwen2.5-Coder-1.5B / coder-fast)
Deep-research-picked Qwen2.5-Coder-1.5B (BASE, Apache-2.0, native FIM) as a low-latency inline-completion seat: - stacks/vllm: vllm-coder service (ana-ml2 GPU1 :8020) + granite shrunk (util 0.27->0.13, max-len 131072->16384, seqs 1024->256; granite phasing out) to free GPU1 room. - stacks/litellm: coder-fast alias -> :8020 (mode: completion, /v1/completions). - stacks/zed-fim-proxy (NEW): keyless /v1/completions front door on ana-docker :4141 for Zed (which can't send an auth header) — POST + path + model allowlist, injects a coder-fast-scoped virtual key -> LiteLLM :4000. Anon /ping liveness. Verified keyless FIM end-to-end. Zed api_url = http://10.250.50.70:4141/v1, model coder-fast, prompt_format qwen. Source-IP allowlist off pending the Mac's observed source IP.
This commit is contained in:
@@ -0,0 +1,21 @@
|
||||
# zed-fim-proxy tunables — copy to `.env` on ana-docker (the real .env holds the
|
||||
# scoped key and is server-only / gitignored). See conf/proxy.py + README.md.
|
||||
|
||||
# Port the keyless route listens on (host network).
|
||||
ZED_PORT=4141
|
||||
|
||||
# LiteLLM gateway to forward to (localhost:4000 via network_mode: host).
|
||||
ZED_UPSTREAM=http://localhost:4000
|
||||
|
||||
# The ONLY model this route will forward (proxy rejects any other "model").
|
||||
ZED_ALLOWED_MODEL=coder-fast
|
||||
|
||||
# Best-effort source-IP allowlist (comma-separated). Empty = allow all — fine on the
|
||||
# internal network, but TIGHTEN to the Mac's observed source IP once it connects
|
||||
# (watch `docker logs zed-fim-proxy` for the real source). Only meaningful if the
|
||||
# proxy sees the real client IP (network_mode: host); a site-to-site NAT may mask it.
|
||||
ZED_ALLOWED_IPS=
|
||||
|
||||
# A LiteLLM virtual key SCOPED TO ZED_ALLOWED_MODEL ONLY (the real blast-radius
|
||||
# bound). Mint: POST /key/generate {"models":["coder-fast"]}. NEVER commit the value.
|
||||
ZED_SCOPED_KEY=
|
||||
Reference in New Issue
Block a user