stacks/voxtral: add entrypoint: vllm serve — image has no default ENTRYPOINT/CMD
Second voxtral attempt got past the image pull (v0.18.0 published, ~3 min download) but container init failed: unable to start container process: error during container init: exec: "--model=mistralai/Voxtral-4B-TTS-2603": stat ...: no such file vllm/vllm-omni:v0.18.0 has Entrypoint=null AND Cmd=null — there's no default executable. The compose's `command:` array becomes the full exec invocation, with --model=... interpreted as the binary name. Standard vLLM serving CLI is `vllm serve <model> [flags]`. The binary's at /usr/local/bin/vllm. Set entrypoint: ["vllm", "serve"] and pass the model as a positional arg. While we're here: HF cache was empty too (Voxtral 4B BF16 ~8 GB download on first start) — vLLM auto-downloads from HF on model load, so no separate pre-pull step needed.
This commit is contained in:
@@ -33,12 +33,16 @@ services:
|
||||
volumes:
|
||||
- ${VOXTRAL_CACHE_DIR}:/root/.cache/huggingface
|
||||
- ${VOXTRAL_VOICES_DIR}:/voices:ro
|
||||
# vLLM-Omni's serve command — model + dtype + port pinned.
|
||||
# vllm/vllm-omni image has no default ENTRYPOINT or CMD — the
|
||||
# container init expected --model=... as argv[0]. Set entrypoint
|
||||
# to `vllm serve` (the standard CLI) and pass model as positional
|
||||
# + tuning flags via command.
|
||||
entrypoint: ["vllm", "serve"]
|
||||
command:
|
||||
- --model=${VOXTRAL_MODEL:-mistralai/Voxtral-4B-TTS-2603}
|
||||
- --port=8000
|
||||
- --dtype=bfloat16
|
||||
- --gpu-memory-utilization=${VOXTRAL_GPU_UTIL:-0.85}
|
||||
- "${VOXTRAL_MODEL:-mistralai/Voxtral-4B-TTS-2603}"
|
||||
- "--port=8000"
|
||||
- "--dtype=bfloat16"
|
||||
- "--gpu-memory-utilization=${VOXTRAL_GPU_UTIL:-0.85}"
|
||||
healthcheck:
|
||||
# vLLM-Omni exposes /health for liveness + /v1/models for readiness.
|
||||
# /health 200 means the server's listening; /v1/models 200 means
|
||||
|
||||
Reference in New Issue
Block a user