stacks/voxtral: add entrypoint: vllm serve — image has no default ENTRYPOINT/CMD
Second voxtral attempt got past the image pull (v0.18.0 published, ~3 min download) but container init failed: unable to start container process: error during container init: exec: "--model=mistralai/Voxtral-4B-TTS-2603": stat ...: no such file vllm/vllm-omni:v0.18.0 has Entrypoint=null AND Cmd=null — there's no default executable. The compose's `command:` array becomes the full exec invocation, with --model=... interpreted as the binary name. Standard vLLM serving CLI is `vllm serve <model> [flags]`. The binary's at /usr/local/bin/vllm. Set entrypoint: ["vllm", "serve"] and pass the model as a positional arg. While we're here: HF cache was empty too (Voxtral 4B BF16 ~8 GB download on first start) — vLLM auto-downloads from HF on model load, so no separate pre-pull step needed.
This commit is contained in:
@@ -33,12 +33,16 @@ services:
|
|||||||
volumes:
|
volumes:
|
||||||
- ${VOXTRAL_CACHE_DIR}:/root/.cache/huggingface
|
- ${VOXTRAL_CACHE_DIR}:/root/.cache/huggingface
|
||||||
- ${VOXTRAL_VOICES_DIR}:/voices:ro
|
- ${VOXTRAL_VOICES_DIR}:/voices:ro
|
||||||
# vLLM-Omni's serve command — model + dtype + port pinned.
|
# vllm/vllm-omni image has no default ENTRYPOINT or CMD — the
|
||||||
|
# container init expected --model=... as argv[0]. Set entrypoint
|
||||||
|
# to `vllm serve` (the standard CLI) and pass model as positional
|
||||||
|
# + tuning flags via command.
|
||||||
|
entrypoint: ["vllm", "serve"]
|
||||||
command:
|
command:
|
||||||
- --model=${VOXTRAL_MODEL:-mistralai/Voxtral-4B-TTS-2603}
|
- "${VOXTRAL_MODEL:-mistralai/Voxtral-4B-TTS-2603}"
|
||||||
- --port=8000
|
- "--port=8000"
|
||||||
- --dtype=bfloat16
|
- "--dtype=bfloat16"
|
||||||
- --gpu-memory-utilization=${VOXTRAL_GPU_UTIL:-0.85}
|
- "--gpu-memory-utilization=${VOXTRAL_GPU_UTIL:-0.85}"
|
||||||
healthcheck:
|
healthcheck:
|
||||||
# vLLM-Omni exposes /health for liveness + /v1/models for readiness.
|
# vLLM-Omni exposes /health for liveness + /v1/models for readiness.
|
||||||
# /health 200 means the server's listening; /v1/models 200 means
|
# /health 200 means the server's listening; /v1/models 200 means
|
||||||
|
|||||||
Reference in New Issue
Block a user