vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

docs/pfi/docker-stack: replace static model table with pointer to live config

Browse Source 
Same anti-pattern as the deleted model-list.md — a hardcoded snapshot
of mutable config that's guaranteed to drift. Replaced the 15-row
table with a one-liner pointing at stacks/llama-swap/conf/config.yaml
(the canonical source) and the live /v1/models HTTP endpoint. Found
via /tend-docs.
This commit is contained in:
vh
2026-04-29 17:03:13 -07:00
parent 65e1ca4478
commit 5a2770f0f7
1 changed files with 6 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/pfi/docker-stack.md
+6 -19
View File
@@ -126,25 +126,12 @@ Dockge expects compose files under `/opt/docker/compose/` on the Docker host. Th
| `/opt/docker/conf/llama-swap/config.yaml` | `/app/config.yaml` | llama-swap configuration (models, groups, params) |
| `/tank/aimodels/llm` | `/models` | LLM model files (GGUF format) |
**Configured Models** (from `config.yaml`):
| Model ID | Display Name | Quantization | Context Size | TTL (s) | Notes |
|---|---|---|---|---|---|
| `qwen3-4b` | Qwen3-4B-Instruct-2507-Q6_K | Q6_K | default | 0 (persistent) | Small general-purpose model |
| `glm4.5-air` | GLM-4.5-Air Q4_K_M | Q4_K_M | 40,000 | 600 | Flash attention enabled |
| `skyfall-r1-31b-q6k` | Skyfall 31B v4 | Q6_K_L | 40,000 | 600 | Flash attention, full GPU offload |
| `GLM-Steam-106B-QK4M-A12B` | GLM-Steam 106B A12B | Q4_K_M | 40,000 | 600 | 2-shard model, MoE with 12B active |
| `kimik2-q2kxl` | Kimi K2 Instruct | UD-Q2_K_XL | default | 600 | 8-shard model, only 2 GPU layers (CPU-heavy) |
| `qwen3-coder-30b-iq4-nl` | Qwen3 Coder 30B A3B | IQ4_NL | 40,000 | 0 (persistent) | MoE 3B active, coding-optimized |
| `unsloth-granite-4-small` | Granite 4.0 Small | Q4_K_M | 120,000 | 0 (persistent) | IBM Granite, deterministic (temp=0) |
| `qwen3.5-35-a3b` | Qwen 3.5 35B A3B | UD-Q4_K_XL | 32,768 | 0 (persistent) | MoE, thinking mode, temp=1.0 |
| `qwen3.5-35-a3b-code` | Qwen 3.5 35B A3B Code | UD-Q4_K_XL | 32,768 | 0 (persistent) | Same model, code-tuned params (temp=0.6) |
| `gemma4-26b-a4b` | Gemma 4 26B A4B | UD-Q4_K_XL | 32,768 | 600 | MoE 4B active, thinking enabled, supports images |
| `gemma4-31b-dense` | Gemma 4 31B Dense | UD-Q4_K_XL | 32,768 | 600 | Full dense model, thinking enabled, supports images |
| `embeddinggemma-300M` | Embedding Gemma 300M | Q8_0 | 2,048 | 0 (persistent) | Embedding model, cls pooling |
| `qwen3-embedding-0.6B` | Qwen3 Embedding 0.6B | Q8_0 | 32,768 | 0 (persistent) | Embedding model, mean pooling |
| `jina-reranker-v3-0.6B` | Jina Reranker v3 | Q8_0 | 32,768 | 0 (persistent) | Reranking model |
| `bge-reranker-v2-m3-0.6B` | BGE Reranker v2 m3 | Q8_0 | 32,768 | 0 (persistent) | Reranking model |
**Configured Models:** the live list (with quants, context, TTLs,
group memberships) is in
[`stacks/llama-swap/conf/config.yaml`](../../stacks/llama-swap/conf/config.yaml).
Don't mirror the table here — it drifted last time and immediately
went stale. Live HTTP listing:
`curl http://10.250.50.54:9292/v1/models`.
**Model Groups**: