Commit Graph
3 Commits
Author SHA1 Message Date
vh a5b626b3d5 fix(qwen3.5-122b): enable tool-calling (--enable-auto-tool-choice --tool-call-parser qwen3_xml)
gen/gen-reasoning tool-calling 400'd (operator + brokkr's capability battery both caught it):
the bjk110 serve command shipped --reasoning-parser qwen3 but no tool flags. Qwen3.5 emits XML
tool calls (<tool_call><function=NAME><parameter=K>V</parameter></function></tool_call>), NOT
Hermes JSON — so `hermes` mis-parsed to raw text; `qwen3_xml` is the correct parser. Reasoning +
tools coexist (gen-reasoning keeps its thinking split). Verified live: a get_weather request
returns tool_calls=[get_weather {"city":"Paris"}].
2026-06-19 01:56:56 -07:00
vh 3ba0e544db tune(qwen3.5-122b): gpu-mem-util 0.90->0.95, max-num-seqs 4->8 (KV 260K->446K tokens, 3.4x concurrency @131K, no OOM) 2026-06-19 01:21:28 -07:00
vh 89c83c4271 feat(qwen3.5-122b): replace mistral-small-4 as gen (abliterated NVFP4, text-only)
bjk110/Qwen3.5-122B-A10B-abliterated-NVFP4 on ana-ml2 GPU 0 (heretic downed):
- stacks/qwen3.5-122b/ — vLLM serve via the repo's text-only patch (Qwen3.5 MoE is a
  multimodal arch but this checkpoint is text-only weights), --reasoning-parser qwen3,
  GPU 0 pin, :8013; entrypoint+patch mounted from the model dir.
- serve-qwen3.5-122b.yaml — displace heretic + serve + verify.
- litellm: REMOVED dead mistral-small-4 / -reasoning; added qwen3.5-122-a10b[-reasoning]
  + aliases qwen-large[-reasoning] + repointed gen[-reasoning] -> qwen (thinking split via
  chat_template_kwargs.enable_thinking + --reasoning-parser qwen3).

Verified live: qwen healthy on :8013; gen / qwen-large / qwen3.5-122-a10b route, and
gen-reasoning returns reasoning_content; mistral-small-4 removed.
NOTE: Worldtree character backend (was bound to mistral-small-4) is dark until repointed
(operator-acknowledged).
2026-06-19 00:49:04 -07:00