feat: Zed edit-predictions keyless FIM route (Qwen2.5-Coder-1.5B / coder-fast)
Deep-research-picked Qwen2.5-Coder-1.5B (BASE, Apache-2.0, native FIM) as a low-latency inline-completion seat: - stacks/vllm: vllm-coder service (ana-ml2 GPU1 :8020) + granite shrunk (util 0.27->0.13, max-len 131072->16384, seqs 1024->256; granite phasing out) to free GPU1 room. - stacks/litellm: coder-fast alias -> :8020 (mode: completion, /v1/completions). - stacks/zed-fim-proxy (NEW): keyless /v1/completions front door on ana-docker :4141 for Zed (which can't send an auth header) — POST + path + model allowlist, injects a coder-fast-scoped virtual key -> LiteLLM :4000. Anon /ping liveness. Verified keyless FIM end-to-end. Zed api_url = http://10.250.50.70:4141/v1, model coder-fast, prompt_format qwen. Source-IP allowlist off pending the Mac's observed source IP.
This commit is contained in:
@@ -257,6 +257,24 @@ model_list:
|
||||
model_info:
|
||||
mode: rerank
|
||||
|
||||
# --- coder-fast → Qwen2.5-Coder-1.5B (BASE), FIM code-completion seat (ana-ml2
|
||||
# GPU1 :8020, vLLM; deep-research pick 2026-07-27). For Zed editor inline
|
||||
# edit-predictions via the LEGACY /v1/completions endpoint with Qwen FIM
|
||||
# markers (<|fim_prefix|>/<|fim_suffix|>/<|fim_middle|>). BASE not -Instruct
|
||||
# (FIM is a pretraining objective; base completions are cleaner). Apache-2.0.
|
||||
# mode: completion — this is text-completion, not chat. Reached KEYLESS from
|
||||
# Vuong's Mac via the zed-fim-proxy (separate port on ana-docker) which injects
|
||||
# a coder-fast-scoped virtual key; the proxy's model-allowlist + the scoped key
|
||||
# bound the blast radius. Runner-up was Qwen2.5-Coder-3B (higher HumanEval-FIM,
|
||||
# non-commercial Qwen-Research license). ---
|
||||
- model_name: coder-fast
|
||||
litellm_params:
|
||||
model: hosted_vllm/qwen2.5-coder-1.5b
|
||||
api_base: http://10.250.50.54:8020/v1
|
||||
api_key: os.environ/VLLM_API_KEY
|
||||
model_info:
|
||||
mode: completion
|
||||
|
||||
# --- z.ai GLM (cloud API) — fronted for unified logging across local
|
||||
# + cloud inference. Explicit entries, so they win over the "*"
|
||||
# wildcard below (no collision with llama-swap's glm4.7-flash etc.
|
||||
|
||||
Reference in New Issue
Block a user