The §5 open question was whether LoRA-on-NVFP4 hot-swap still silently no-ops
as it did on vLLM 0.24.0 (#47639), with merged weights as the fallback if it
did. Retested on vllm/vllm-openai:latest against the NVFP4A16 base plus the
live run's checkpoint adapter.
It does not no-op. It refuses to start:
AttributeError: To support LoRA for MoE model,
'get_expert_mapping' must be implemented
And the reason is bigger than the quant. The check is in
vllm/lora/utils.py::process_packed_modules_mapping and branches on whether the
model is MoE — quantization is not in the condition. gemma4.py, gemma4_mm.py,
gemma4_mtp.py and gemma4_unified.py contain zero occurrences of
get_expert_mapping, while deepseek_v2, glm4_moe and ernie45_moe do implement
it. So vLLM cannot serve a LoRA on Gemma-4 at all, BF16 or quantized. Merging
is not a workaround for a quantization limitation; it is the only path for this
architecture.
This holds even though the adapter never touches experts —
validate_adapter_parameters forbids per-expert params, so all 205 targets are
attention and dense MLP. The refusal is about the model being MoE, not about
what the adapter targets.
Worth recording that the current behaviour is an improvement: a loud refusal
beats the 0.24.0 silent no-op, which would ship a base model wearing the tune's
name and pass every check that does not compare against base.
vh
8de5f7a73c