feat: Zed edit-predictions keyless FIM route (Qwen2.5-Coder-1.5B / coder-fast)

Deep-research-picked Qwen2.5-Coder-1.5B (BASE, Apache-2.0, native FIM) as a
low-latency inline-completion seat:
- stacks/vllm: vllm-coder service (ana-ml2 GPU1 :8020) + granite shrunk
  (util 0.27->0.13, max-len 131072->16384, seqs 1024->256; granite phasing out)
  to free GPU1 room.
- stacks/litellm: coder-fast alias -> :8020 (mode: completion, /v1/completions).
- stacks/zed-fim-proxy (NEW): keyless /v1/completions front door on ana-docker
  :4141 for Zed (which can't send an auth header) — POST + path + model
  allowlist, injects a coder-fast-scoped virtual key -> LiteLLM :4000. Anon
  /ping liveness. Verified keyless FIM end-to-end.

Zed api_url = http://10.250.50.70:4141/v1, model coder-fast, prompt_format qwen.
Source-IP allowlist off pending the Mac's observed source IP.
This commit is contained in:
vh
2026-07-27 22:55:21 -07:00
parent 8822a0bb81
commit a300cdcd26
8 changed files with 315 additions and 6 deletions
+64
View File
@@ -278,6 +278,70 @@ services:
- homepage.description=Granite 4.1 8B FP8 via vLLM (ana-ml2)
- homepage.href=http://10.250.50.54:${GRANITE_PORT}/docs
vllm-coder:
image: vllm/vllm-openai:${VLLM_VERSION}
container_name: vllm-coder
restart: unless-stopped
ipc: host
ports:
- "${CODER_PORT}:8000"
volumes:
- /tank/aimodels/huggingface:/hfcache
environment:
- HF_HOME=/hfcache
- HF_HUB_CACHE=/hfcache/hub
- HUGGING_FACE_HUB_TOKEN=${HF_TOKEN:-}
- VLLM_API_KEY=${API_KEY:-}
command:
# Qwen2.5-Coder-1.5B (BASE) — FIM code-completion seat for Zed edit-predictions
# (deep-research pick 2026-07-27). Native fill-in-the-middle: <|fim_prefix|> /
# <|fim_suffix|> / <|fim_middle|> (IDs 151659/151660/151661); Zed sends the
# FIM-formatted prompt to /v1/completions and vLLM passes it through (the FIM
# special tokens live in the tokenizer). BASE not -Instruct (FIM is a
# pretraining objective; base completions are cleaner). Apache-2.0. Runner-up =
# Qwen2.5-Coder-3B (higher HumanEval-FIM but non-commercial Qwen-Research license).
- ${CODER_MODEL}
- --served-model-name
- ${CODER_SERVED_NAME}
- --host
- 0.0.0.0
- --port
- "8000"
- --gpu-memory-utilization
- ${CODER_GPU_MEM_UTIL}
- --max-model-len
- ${CODER_MAX_MODEL_LEN}
- --max-num-seqs
- ${CODER_MAX_NUM_SEQS}
- --dtype
- auto
- --kv-cache-dtype
- ${CODER_KV_CACHE_DTYPE}
- --enable-prefix-caching
deploy:
resources:
reservations:
devices:
- driver: nvidia
device_ids:
- "${CODER_GPU_ID}"
capabilities:
- gpu
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 300s
networks:
- tnet
labels:
- homepage.group=AI - Inference
- homepage.name=vLLM Qwen2.5-Coder 1.5B (FIM)
- homepage.icon=mdi-code-braces
- homepage.description=Qwen2.5-Coder-1.5B FIM code-completion (ana-ml2, Zed edit-predictions)
- homepage.href=http://10.250.50.54:${CODER_PORT}/docs
networks:
tnet:
name: traefik-net