llama-swap: drop qwen3.6-35-a3b from pinned group — was blocking other model loads under memory pressure

Symptom: qwen3.6-35-a3b refused to deload when other models needed
the VRAM, even with the model itself at ttl: 0. The pinning came from
the `pinned` group's `persistent: true` flag, which exempts members
from eviction by the scheduler regardless of memory pressure. The
model's ttl: 0 only governs idle-timeout, NOT scheduler eviction —
those are separate concerns.

Removed qwen3.6-35-a3b from the group's members. Kept ttl: 0 on the
model itself: still no idle-unload, but the scheduler CAN now evict
it when another non-coexistent model is requested. qwen3.5-9b stays
pinned (~6 GB at Q4 — cheap to hold).

Updated the inline comment + the group-header docstring to reflect
the new semantics so future-Claude doesn't undo this.
This commit is contained in:
2026-04-27 18:05:49 -07:00
parent 0baf1e8e2d
commit 894e92275d
+8 -4
View File
@@ -129,7 +129,8 @@ models:
"qwen3.6-35-a3b":
name: "Qwen 3.6 35B-A3B UD-Q6_K_XL"
description: "Qwen3.6 MoE, 3B active of 35B. Stock unsloth Q6_K_XL (~29GB). Thinking on by default."
ttl: 0 # pinned — member of the `pinned` group, never unloads
ttl: 0 # never idle-unload, but evictable under memory pressure
# (out of `pinned` group as of 2026-04-27 — was OOM'ing other loads).
cmd: |
/app/llama-server
--context-shift
@@ -546,12 +547,15 @@ groups:
# Pinned general-purpose chat models. Coexist in VRAM, never unload.
# Members also have ttl: 0 individually so idle-timeout can't drop them.
# VRAM budget check: qwen3.5-9b (~6 GB at Q4) + qwen3.6-35-a3b (~29 GB
# at Q6_K_XL) + 128K KV cache for each ≈ 50-60 GB. Size for your GPU.
# Currently just qwen3.5-9b — small enough (~6 GB at Q4 + KV) to be
# cheap to keep resident. qwen3.6-35-a3b was previously here but its
# ~29 GB at Q6_K_XL made other model loads OOM under memory pressure
# (persistent: true exempts members from eviction). It now sits with
# ttl: 0 outside any group — never idle-unloads, but the scheduler
# CAN evict it when another non-coexistent model is requested.
"pinned":
swap: false
exclusive: false
persistent: true
members:
- "qwen3.5-9b"
- "qwen3.6-35-a3b"