feat: Zed edit-predictions keyless FIM route (Qwen2.5-Coder-1.5B / coder-fast)

Deep-research-picked Qwen2.5-Coder-1.5B (BASE, Apache-2.0, native FIM) as a
low-latency inline-completion seat:
- stacks/vllm: vllm-coder service (ana-ml2 GPU1 :8020) + granite shrunk
  (util 0.27->0.13, max-len 131072->16384, seqs 1024->256; granite phasing out)
  to free GPU1 room.
- stacks/litellm: coder-fast alias -> :8020 (mode: completion, /v1/completions).
- stacks/zed-fim-proxy (NEW): keyless /v1/completions front door on ana-docker
  :4141 for Zed (which can't send an auth header) — POST + path + model
  allowlist, injects a coder-fast-scoped virtual key -> LiteLLM :4000. Anon
  /ping liveness. Verified keyless FIM end-to-end.

Zed api_url = http://10.250.50.70:4141/v1, model coder-fast, prompt_format qwen.
Source-IP allowlist off pending the Mac's observed source IP.
This commit is contained in:
vh
2026-07-27 22:55:21 -07:00
parent 8822a0bb81
commit a300cdcd26
8 changed files with 315 additions and 6 deletions
+18
View File
@@ -257,6 +257,24 @@ model_list:
model_info:
mode: rerank
# --- coder-fast → Qwen2.5-Coder-1.5B (BASE), FIM code-completion seat (ana-ml2
# GPU1 :8020, vLLM; deep-research pick 2026-07-27). For Zed editor inline
# edit-predictions via the LEGACY /v1/completions endpoint with Qwen FIM
# markers (<|fim_prefix|>/<|fim_suffix|>/<|fim_middle|>). BASE not -Instruct
# (FIM is a pretraining objective; base completions are cleaner). Apache-2.0.
# mode: completion — this is text-completion, not chat. Reached KEYLESS from
# Vuong's Mac via the zed-fim-proxy (separate port on ana-docker) which injects
# a coder-fast-scoped virtual key; the proxy's model-allowlist + the scoped key
# bound the blast radius. Runner-up was Qwen2.5-Coder-3B (higher HumanEval-FIM,
# non-commercial Qwen-Research license). ---
- model_name: coder-fast
litellm_params:
model: hosted_vllm/qwen2.5-coder-1.5b
api_base: http://10.250.50.54:8020/v1
api_key: os.environ/VLLM_API_KEY
model_info:
mode: completion
# --- z.ai GLM (cloud API) — fronted for unified logging across local
# + cloud inference. Explicit entries, so they win over the "*"
# wildcard below (no collision with llama-swap's glm4.7-flash etc.