esh-pfi-infrastructure

vh/esh-pfi-infrastructure

Fork 0

Commit Graph

Author	SHA1	Message	Date
vh	58f2a22966	llama-swap: pin qwen3.5-9b + qwen3.6-35-a3b as a coexistence group Adds a new `pinned` group with swap: false (models coexist in VRAM), exclusive: false (group shares with other groups), persistent: true (never unload). Each member also gets ttl: 0 so the per-model idle-timeout can't drop them either — belt + suspenders. Pair is currently qwen3.5-9b (~6 GB Q4) + qwen3.6-35-a3b (~29 GB Q6). Plus the 128K KV caches, roughly 50-60 GB VRAM resident. Appropriate for an A6000/H100-class card; verify fit after deploy. Committed as a canonical change; push + restart still needed on ana-ml2.	2026-04-22 09:05:19 -07:00
vh	8d6921cb75	llama-swap: add Qwen3.6 35B-A3B stock + swap abliterated, 128K ctx - New: qwen3.6-35-a3b using -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q6_K_XL (~29 GB weights, thinking on, KB Qwen sampler defaults). - Swap: qwen3.6-35-a3b-abliterated from IIEleven11 Heretic Q8_0 to mradermacher/Qwen3.6-35B-A3B-abliterated-i1-GGUF:i1-Q6_K (already in the /hfcache on ana-ml2; ~27 GB). - Context: bumped both Qwen 3.6 entries 32768 -> 131072.	2026-04-20 22:14:26 -07:00
vh	e376d0aec9	Initial commit: PFI fleet inventory, stacks, tooling, and backup pipeline Captures the full workspace state built up to this point: - CLAUDE.md + README.md describing conventions and the four-host fleet (ana-ml2, ana-docker, nh3-docker, esh-docker-vm). - Per-host notes under servers/<host>/ with ssh-target fallback files and latest system-details snapshots (two in-compose credential leaks scrubbed; the upstream compose files still need to move those to .env). - scripts/: server_inspect.sh (read-only remote diagnostic), refresh-server-info.sh (dir-driven discovery + snapshot capture with validation warnings), add-host.sh, sync-stacks.sh (pull compose/conf trees), deploy-stack.sh (push with per-file diff + prompt). - stacks/: canonical compose for backrest, beszel, dozzle, llama-swap, rest-server-ana, rest-server-nh3, vllm-qwen3, plus the retired infinity reference. All use the .env-driven + traefik-net + homepage label pattern. - configs/restic/ana-docker/: first resticprofile config + pre-backup hook (Synapse pg_dump, Seafile mysqldump, Vaultwarden SQLite); templates for the other three hosts to come. - docs/pfi/: general infrastructure reference carried over. - .gitignore excludes .env, stacks-mirror/, and assorted secret/state filenames to prevent re-leaks on later commits.	2026-04-20 14:29:48 -07:00

Author

SHA1

Message

Date

58f2a22966

llama-swap: pin qwen3.5-9b + qwen3.6-35-a3b as a coexistence group

Adds a new `pinned` group with swap: false (models coexist in VRAM),
exclusive: false (group shares with other groups), persistent: true
(never unload). Each member also gets ttl: 0 so the per-model
idle-timeout can't drop them either — belt + suspenders.

Pair is currently qwen3.5-9b (~6 GB Q4) + qwen3.6-35-a3b (~29 GB Q6).
Plus the 128K KV caches, roughly 50-60 GB VRAM resident. Appropriate
for an A6000/H100-class card; verify fit after deploy.

Committed as a canonical change; push + restart still needed on
ana-ml2.

2026-04-22 09:05:19 -07:00

8d6921cb75

llama-swap: add Qwen3.6 35B-A3B stock + swap abliterated, 128K ctx

- New: qwen3.6-35-a3b using -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q6_K_XL
  (~29 GB weights, thinking on, KB Qwen sampler defaults).
- Swap: qwen3.6-35-a3b-abliterated from IIEleven11 Heretic Q8_0 to
  mradermacher/Qwen3.6-35B-A3B-abliterated-i1-GGUF:i1-Q6_K (already in
  the /hfcache on ana-ml2; ~27 GB).
- Context: bumped both Qwen 3.6 entries 32768 -> 131072.

2026-04-20 22:14:26 -07:00

e376d0aec9

Initial commit: PFI fleet inventory, stacks, tooling, and backup pipeline

Captures the full workspace state built up to this point:

  - CLAUDE.md + README.md describing conventions and the four-host fleet
    (ana-ml2, ana-docker, nh3-docker, esh-docker-vm).
  - Per-host notes under servers/<host>/ with ssh-target fallback files
    and latest system-details snapshots (two in-compose credential leaks
    scrubbed; the upstream compose files still need to move those to .env).
  - scripts/: server_inspect.sh (read-only remote diagnostic),
    refresh-server-info.sh (dir-driven discovery + snapshot capture with
    validation warnings), add-host.sh, sync-stacks.sh (pull
    compose/conf trees), deploy-stack.sh (push with per-file diff + prompt).
  - stacks/: canonical compose for backrest, beszel, dozzle, llama-swap,
    rest-server-ana, rest-server-nh3, vllm-qwen3, plus the retired
    infinity reference. All use the .env-driven + traefik-net + homepage
    label pattern.
  - configs/restic/ana-docker/: first resticprofile config + pre-backup
    hook (Synapse pg_dump, Seafile mysqldump, Vaultwarden SQLite); templates
    for the other three hosts to come.
  - docs/pfi/: general infrastructure reference carried over.
  - .gitignore excludes .env, stacks-mirror/, and assorted secret/state
    filenames to prevent re-leaks on later commits.

2026-04-20 14:29:48 -07:00

3 Commits