llama-swap: add Qwen3.6 35B-A3B stock + swap abliterated, 128K ctx

- New: qwen3.6-35-a3b using -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q6_K_XL
  (~29 GB weights, thinking on, KB Qwen sampler defaults).
- Swap: qwen3.6-35-a3b-abliterated from IIEleven11 Heretic Q8_0 to
  mradermacher/Qwen3.6-35B-A3B-abliterated-i1-GGUF:i1-Q6_K (already in
  the /hfcache on ana-ml2; ~27 GB).
- Context: bumped both Qwen 3.6 entries 32768 -> 131072.
This commit is contained in:
vh
2026-04-20 22:14:26 -07:00
parent 0be8de8ab0
commit 8d6921cb75
+31 -5
View File
@@ -20,6 +20,10 @@
# 2026-04-17: Added Qwen3.6-35B-A3B Abliterated Heretic Q8_0 via -hf syntax.
# Requires HF_HOME=/hfcache in compose (see docker-compose.yml).
# New convention: use -hf repo[:quant] instead of --model /path.
# 2026-04-20: Added stock unsloth Qwen3.6-35B-A3B at Q6_K_XL via -hf syntax.
# Swapped abliterated entry from IIEleven11 Heretic Q8_0 to
# mradermacher abliterated i1-Q6_K (already cached).
# Bumped both Qwen 3.6 entries ctx-size 32768 → 131072.
# ============================================================================
# Default 1200 seconds (20 min) to wait for model to be available to load.
@@ -113,18 +117,40 @@ models:
# Qwen 3.6 — uses -hf syntax, reads from HF_HOME=/hfcache (host pre-download)
# --------------------------------------------------------------------------
"qwen3.6-35-a3b-abliterated":
name: "Qwen 3.6 35B-A3B Abliterated Heretic Q8_0"
description: "Qwen3.6 MoE, 3B active. Abliterated/heretic variant of BF16 quantized to Q8_0. ~38GB."
"qwen3.6-35-a3b":
name: "Qwen 3.6 35B-A3B UD-Q6_K_XL"
description: "Qwen3.6 MoE, 3B active of 35B. Stock unsloth Q6_K_XL (~29GB). Thinking on by default."
ttl: 600
cmd: |
/app/llama-server
--context-shift
--jinja
-hf IIEleven11/Qwen3.6-35B-A3B-Abliterated-Heretic-BF16-Q8_0-GGUF
-hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q6_K_XL
--port ${PORT}
--n-gpu-layers 999
--ctx-size 32768
--ctx-size 131072
--flash-attn on
--temp 1.0
--top-p 0.95
--top-k 20
--min-p 0.00
--presence-penalty 1.5
--repeat-penalty 1.0
--reasoning on
--reasoning-format deepseek
"qwen3.6-35-a3b-abliterated":
name: "Qwen 3.6 35B-A3B Abliterated i1-Q6_K"
description: "Qwen3.6 MoE, 3B active of 35B. mradermacher abliterated imatrix Q6_K (~27GB)."
ttl: 600
cmd: |
/app/llama-server
--context-shift
--jinja
-hf mradermacher/Qwen3.6-35B-A3B-abliterated-i1-GGUF:i1-Q6_K
--port ${PORT}
--n-gpu-layers 999
--ctx-size 131072
--flash-attn on
--temp 1.0
--top-p 0.95