stacks/voxtral: add entrypoint: vllm serve — image has no default ENTRYPOINT/CMD

Second voxtral attempt got past the image pull (v0.18.0 published,
~3 min download) but container init failed:
  unable to start container process: error during container init:
  exec: "--model=mistralai/Voxtral-4B-TTS-2603": stat ...: no such file

vllm/vllm-omni:v0.18.0 has Entrypoint=null AND Cmd=null — there's no
default executable. The compose's `command:` array becomes the full
exec invocation, with --model=... interpreted as the binary name.

Standard vLLM serving CLI is `vllm serve <model> [flags]`. The
binary's at /usr/local/bin/vllm. Set entrypoint: ["vllm", "serve"]
and pass the model as a positional arg.

While we're here: HF cache was empty too (Voxtral 4B BF16 ~8 GB
download on first start) — vLLM auto-downloads from HF on model
load, so no separate pre-pull step needed.
This commit is contained in:
vh
2026-04-27 23:59:05 -07:00
parent 131d746c92
commit 5a1b715f3d
+9 -5
View File
@@ -33,12 +33,16 @@ services:
volumes:
- ${VOXTRAL_CACHE_DIR}:/root/.cache/huggingface
- ${VOXTRAL_VOICES_DIR}:/voices:ro
# vLLM-Omni's serve command — model + dtype + port pinned.
# vllm/vllm-omni image has no default ENTRYPOINT or CMD — the
# container init expected --model=... as argv[0]. Set entrypoint
# to `vllm serve` (the standard CLI) and pass model as positional
# + tuning flags via command.
entrypoint: ["vllm", "serve"]
command:
- --model=${VOXTRAL_MODEL:-mistralai/Voxtral-4B-TTS-2603}
- --port=8000
- --dtype=bfloat16
- --gpu-memory-utilization=${VOXTRAL_GPU_UTIL:-0.85}
- "${VOXTRAL_MODEL:-mistralai/Voxtral-4B-TTS-2603}"
- "--port=8000"
- "--dtype=bfloat16"
- "--gpu-memory-utilization=${VOXTRAL_GPU_UTIL:-0.85}"
healthcheck:
# vLLM-Omni exposes /health for liveness + /v1/models for readiness.
# /health 200 means the server's listening; /v1/models 200 means