llama-swap: drop qwen3.6-35-a3b from pinned group — was blocking other model loads under memory pressure
Symptom: qwen3.6-35-a3b refused to deload when other models needed the VRAM, even with the model itself at ttl: 0. The pinning came from the `pinned` group's `persistent: true` flag, which exempts members from eviction by the scheduler regardless of memory pressure. The model's ttl: 0 only governs idle-timeout, NOT scheduler eviction — those are separate concerns. Removed qwen3.6-35-a3b from the group's members. Kept ttl: 0 on the model itself: still no idle-unload, but the scheduler CAN now evict it when another non-coexistent model is requested. qwen3.5-9b stays pinned (~6 GB at Q4 — cheap to hold). Updated the inline comment + the group-header docstring to reflect the new semantics so future-Claude doesn't undo this.
This commit is contained in:
@@ -129,7 +129,8 @@ models:
|
||||
"qwen3.6-35-a3b":
|
||||
name: "Qwen 3.6 35B-A3B UD-Q6_K_XL"
|
||||
description: "Qwen3.6 MoE, 3B active of 35B. Stock unsloth Q6_K_XL (~29GB). Thinking on by default."
|
||||
ttl: 0 # pinned — member of the `pinned` group, never unloads
|
||||
ttl: 0 # never idle-unload, but evictable under memory pressure
|
||||
# (out of `pinned` group as of 2026-04-27 — was OOM'ing other loads).
|
||||
cmd: |
|
||||
/app/llama-server
|
||||
--context-shift
|
||||
@@ -546,12 +547,15 @@ groups:
|
||||
|
||||
# Pinned general-purpose chat models. Coexist in VRAM, never unload.
|
||||
# Members also have ttl: 0 individually so idle-timeout can't drop them.
|
||||
# VRAM budget check: qwen3.5-9b (~6 GB at Q4) + qwen3.6-35-a3b (~29 GB
|
||||
# at Q6_K_XL) + 128K KV cache for each ≈ 50-60 GB. Size for your GPU.
|
||||
# Currently just qwen3.5-9b — small enough (~6 GB at Q4 + KV) to be
|
||||
# cheap to keep resident. qwen3.6-35-a3b was previously here but its
|
||||
# ~29 GB at Q6_K_XL made other model loads OOM under memory pressure
|
||||
# (persistent: true exempts members from eviction). It now sits with
|
||||
# ttl: 0 outside any group — never idle-unloads, but the scheduler
|
||||
# CAN evict it when another non-coexistent model is requested.
|
||||
"pinned":
|
||||
swap: false
|
||||
exclusive: false
|
||||
persistent: true
|
||||
members:
|
||||
- "qwen3.5-9b"
|
||||
- "qwen3.6-35-a3b"
|
||||
|
||||
Reference in New Issue
Block a user