vh
a300cdcd26
Deep-research-picked Qwen2.5-Coder-1.5B (BASE, Apache-2.0, native FIM) as a low-latency inline-completion seat: - stacks/vllm: vllm-coder service (ana-ml2 GPU1 :8020) + granite shrunk (util 0.27->0.13, max-len 131072->16384, seqs 1024->256; granite phasing out) to free GPU1 room. - stacks/litellm: coder-fast alias -> :8020 (mode: completion, /v1/completions). - stacks/zed-fim-proxy (NEW): keyless /v1/completions front door on ana-docker :4141 for Zed (which can't send an auth header) — POST + path + model allowlist, injects a coder-fast-scoped virtual key -> LiteLLM :4000. Anon /ping liveness. Verified keyless FIM end-to-end. Zed api_url = http://10.250.50.70:4141/v1, model coder-fast, prompt_format qwen. Source-IP allowlist off pending the Mac's observed source IP.
37 lines
1.5 KiB
YAML
37 lines
1.5 KiB
YAML
name: zed-fim-proxy
|
|
|
|
# Keyless /v1/completions front door for Zed edit-predictions (coder-fast only).
|
|
# See conf/proxy.py for the guard model. Runs network_mode: host so it (a) sees the
|
|
# real client source IP for the best-effort allowlist (docker port-publish would NAT
|
|
# it away) and (b) reaches the LiteLLM gateway on localhost:4000. Stdlib-only proxy
|
|
# in a bare python image — no build, no pip.
|
|
|
|
services:
|
|
zed-fim-proxy:
|
|
image: python:3.12-slim
|
|
container_name: zed-fim-proxy
|
|
restart: unless-stopped
|
|
network_mode: host
|
|
command: ["python", "/app/proxy.py"]
|
|
volumes:
|
|
- /opt/docker/conf/zed-fim-proxy/proxy.py:/app/proxy.py:ro
|
|
environment:
|
|
- ZED_PORT=${ZED_PORT:-4141}
|
|
- ZED_UPSTREAM=${ZED_UPSTREAM:-http://localhost:4000}
|
|
- ZED_ALLOWED_MODEL=${ZED_ALLOWED_MODEL:-coder-fast}
|
|
# best-effort source-IP allowlist (comma-separated); empty = allow all.
|
|
- ZED_ALLOWED_IPS=${ZED_ALLOWED_IPS:-}
|
|
- ZED_SCOPED_KEY=${ZED_SCOPED_KEY}
|
|
healthcheck:
|
|
test: ["CMD", "python", "-c", "import urllib.request,os; urllib.request.urlopen('http://localhost:%s/ping' % os.environ.get('ZED_PORT','4141'))"]
|
|
interval: 30s
|
|
timeout: 5s
|
|
retries: 3
|
|
start_period: 10s
|
|
labels:
|
|
- homepage.group=AI - Gateways & Chat
|
|
- homepage.name=Zed FIM Proxy
|
|
- homepage.icon=mdi-code-braces-box
|
|
- homepage.description=Keyless /v1/completions for Zed edit-predictions (coder-fast, ana-docker)
|
|
- homepage.href=http://10.250.50.70:4141/ping
|