Initial commit: PFI fleet inventory, stacks, tooling, and backup pipeline

Captures the full workspace state built up to this point: - CLAUDE.md + README.md describing conventions and the four-host fleet (ana-ml2, ana-docker, nh3-docker, esh-docker-vm). - Per-host notes under servers/<host>/ with ssh-target fallback files and latest system-details snapshots (two in-compose credential leaks scrubbed; the upstream compose files still need to move those to .env). - scripts/: server_inspect.sh (read-only remote diagnostic), refresh-server-info.sh (dir-driven discovery + snapshot capture with validation warnings), add-host.sh, sync-stacks.sh (pull compose/conf trees), deploy-stack.sh (push with per-file diff + prompt). - stacks/: canonical compose for backrest, beszel, dozzle, llama-swap, rest-server-ana, rest-server-nh3, vllm-qwen3, plus the retired infinity reference. All use the .env-driven + traefik-net + homepage label pattern. - configs/restic/ana-docker/: first resticprofile config + pre-backup hook (Synapse pg_dump, Seafile mysqldump, Vaultwarden SQLite); templates for the other three hosts to come. - docs/pfi/: general infrastructure reference carried over. - .gitignore excludes .env, stacks-mirror/, and assorted secret/state filenames to prevent re-leaks on later commits.
2026-04-20 14:29:48 -07:00
commit e376d0aec9
55 changed files with 9101 additions and 0 deletions
@@ -0,0 +1,146 @@
+# ChromaDB Setup Documentation
+
+**Project**: Infrastructure-PFI  
+**Target Server**: PFI-ANA-Docker (VM 102)  
+**IP Address**: 10.250.50.x (VLAN 50)  
+**Status**: Ready for deployment  
+
+## Overview
+
+ChromaDB is an embedded vector database optimized for AI/ML applications. This deployment provides:
+
+- Persistent vector storage on `/tank/chromadb/`
+- REST API on port 8000 (internal + Traefik-routed)
+- Token-based authentication
+- Automated backup and health monitoring
+
+## Architecture
+
+```
+┌──────────────────────────────────────────────────────────┐
+│               PFI-ANA-Docker (VM 102)                    │
+│                                                          │
+│  ┌──────────────┐    ┌──────────────┐    ┌───────────┐  │
+│  │   Traefik    │───▶│   ChromaDB   │    │  Dockge   │  │
+│  │   Reverse    │    │  (Port 8000) │    │  Manager  │  │
+│  │    Proxy     │    │              │    │           │  │
+│  └──────────────┘    └──────┬───────┘    └───────────┘  │
+│                             │                            │
+│                    ┌────────▼────────┐                   │
+│                    │  /tank/chromadb │                   │
+│                    │  (bind mount)   │                   │
+│                    └─────────────────┘                   │
+└──────────────────────────────────────────────────────────┘
+```
+
+## File Locations (on VM 102)
+
+| Host Path | Purpose |
+|---|---|
+| `/opt/docker/conf/chromadb/` | Compose file, auth token, config |
+| `/opt/docker/conf/chromadb/docker-compose.yml` | Main compose file |
+| `/opt/docker/conf/chromadb/auth_token` | Token for API authentication |
+| `/tank/chromadb/` | Persistent vector data (bind mount) |
+| `/opt/docker/backups/chromadb/` | Backup archives |
+
+## Authentication
+
+This deployment uses **ChromaDB's native token auth**:
+
+- A random 64-hex-char token is generated during setup (`openssl rand -hex 32`)
+- The token is stored at `/opt/docker/conf/chromadb/auth_token` (mode 600)
+- Clients must supply the token via `Settings`:
+
+```python
+import chromadb
+from chromadb.config import Settings
+
+client = chromadb.HttpClient(
+    host="10.250.50.x",  # or chromadb.pfi.local via Traefik
+    port=8000,
+    settings=Settings(
+        chroma_client_auth_provider="chromadb.auth.token.TokenAuthClientProvider",
+        chroma_client_auth_credentials="YOUR_TOKEN_HERE",
+    ),
+)
+print(client.heartbeat())
+```
+
+## Deployment Steps
+
+### 1. Copy compose file to VM 102
+
+```bash
+scp configs/pfi-ana/docker/compose-examples/chromadb/docker-compose.yml \
+    root@10.250.50.x:/opt/docker/conf/chromadb/docker-compose.yml
+```
+
+### 2. Run the setup script (on VM 102)
+
+```bash
+# Copy scripts to VM 102
+scp scripts/setup-chromadb.sh root@10.250.50.x:/opt/docker/conf/chromadb/
+ssh root@10.250.50.x
+
+# Run setup
+cd /opt/docker/conf/chromadb
+chmod +x setup-chromadb.sh
+./setup-chromadb.sh
+```
+
+### 3. Verify
+
+```bash
+curl http://localhost:8000/api/v1/health
+```
+
+### 4. (Optional) Run the demo
+
+```bash
+# Copy demo files
+scp -r configs/pfi-ana/docker/compose-examples/chromadb/ root@10.250.50.x:/tmp/chromadb-demo/
+
+# On VM 102, edit CHROMA_TOKEN in docker-compose.demo.yml
+cd /tmp/chromadb-demo
+# Set CHROMA_TOKEN in docker-compose.demo.yml to match auth_token
+docker compose -f docker-compose.demo.yml up
+```
+
+## Monitoring & Maintenance
+
+| Task | Command |
+|---|---|
+| Health check | `./scripts/health-check-chromadb.sh` |
+| View logs | `docker logs -f chromadb` |
+| Backup | `./scripts/backup-chromadb.sh` |
+| Restart | `docker compose restart` |
+| Stop | `docker compose down` |
+
+### Cron (daily backup at 2 AM)
+
+```cron
+0 2 * * * /opt/docker/conf/chromadb/backup-chromadb.sh >> /var/log/chromadb-backup.log 2>&1
+```
+
+## Networking
+
+| Aspect | Value |
+|---|---|
+| Docker network | `traefik-net` (aliased as `tnet`) |
+| Internal port | 8000 |
+| Traefik host rule | `chromadb.pfi.local` |
+| Traefik entrypoint | `websecure` (HTTPS) |
+| TLS | Enabled via Traefik |
+
+## Project Files
+
+| File | Purpose |
+|---|---|
+| `configs/pfi-ana/docker/compose-examples/chromadb/docker-compose.yml` | Production compose (for Dockge) |
+| `configs/pfi-ana/docker/compose-examples/chromadb/docker-compose.demo.yml` | Demo client |
+| `configs/pfi-ana/docker/compose-examples/chromadb/Dockerfile.demo` | Demo image |
+| `configs/pfi-ana/docker/compose-examples/chromadb/scripts/demo.py` | Demo test script |
+| `scripts/setup-chromadb.sh` | Deployment script (run on VM 102) |
+| `scripts/backup-chromadb.sh` | Backup script (run on VM 102) |
+| `scripts/health-check-chromadb.sh` | Health monitoring (run on VM 102) |
+| `scripts/quickstart-chromadb.sh` | Convenience wrapper for setup |
@@ -0,0 +1,333 @@
+# PFI-ANA Docker Stack
+
+## Overview
+
+PFI-ANA (the Colo) runs Docker services managed through **Dockge**, a compose-aware Docker management UI. All services that require inbound HTTP/HTTPS routing join a shared external Docker network called `traefik-net`, allowing **Traefik** to act as a reverse proxy and handle TLS termination and routing.
+
+## Conventions
+
+### Network
+
+| Network | Docker Name | Purpose |
+|---|---|---|
+| Traefik network | `traefik-net` | Shared external network. Services join as `tnet` so Traefik can discover them. |
+
+Every compose file that needs to be reachable through Traefik **must** include:
+
+```yaml
+networks:
+  tnet:
+    name: traefik-net
+    external: true
+```
+
+And the service must list `tnet` under its `networks` key.
+
+### Storage Paths (Host)
+
+| Host Path | Purpose |
+|---|---|
+| `/opt/docker/compose/<service>/` | Per-service compose files (managed by Dockge) |
+| `/opt/docker/conf/<service>/` | Per-service configuration files (bind-mounted into containers) |
+| `/tank/` | Large / persistent data storage (e.g., AI models, generated images, voice data) |
+
+### GPU Support
+
+Services requiring GPU access use the **NVIDIA Container Toolkit**:
+
+```yaml
+runtime: nvidia
+```
+
+or the more explicit device reservation:
+
+```yaml
+deploy:
+  resources:
+    reservations:
+      devices:
+        - driver: nvidia
+          count: all
+          capabilities: [gpu]
+```
+
+### Homepage Dashboard Labels
+
+Several services include Docker labels for **Homepage** (a dashboard UI). The convention is:
+
+```yaml
+labels:
+  - homepage.group=<Group Name>
+  - homepage.name=<Display Name>
+  - homepage.icon=<icon identifier>
+  - homepage.description=<brief description>
+  - homepage.href=http://<host>:<port>
+```
+
+All services reference the VM 102 host IP `10.250.50.70` (PFI-ANA_DOCKER).
+
+### Compose File Location
+
+Dockge expects compose files under `/opt/docker/compose/` on the Docker host. The live compose files are version-controlled in this project under `configs/pfi-ana/docker/compose/`. Configuration files that containers bind-mount live under `configs/pfi-ana/docker/conf/`.
+
+---
+
+## Live Services Index
+
+| Service | Host Port | Container Port | GPU | Homepage Group | Status | Compose File | Config File |
+|---|---|---|---|---|---|---|---|
+| **Dockge** | 5001 | 5001 | No | PFI-ANA | ✅ Live | `compose/dockge/compose.yaml` | — |
+| **llama-swap** | 9292 | 8080 | Yes (CUDA) | — | ✅ Live | `compose/llama-swap/compose.yaml` | `conf/llama-swap/config.yaml` |
+| **ComfyUI** | 8188 | 8188 | Yes (all caps) | AI Systems | ✅ Live | `compose/comfyui/compose.yaml` | — |
+| **VibeVoice** | 8745 | 8745 | Yes (gpu) | AI Systems | ✅ Live | `compose/vibevoice/compose.yaml` | — |
+| **Parakeet STT** | 8300 | 8000 | Yes (gpu) | AI Systems | ✅ Live | `compose/parakeet/compose.yaml` | — |
+| **ChromaDB** | 8000 | 8000 | No | AI Systems | ✅ Live | `compose/chromadb/compose.yaml` | `/opt/docker/conf/chromadb/auth_token` |
+
+> All paths relative to `configs/pfi-ana/docker/`.
+
+---
+
+## Service Details
+
+### 1. Dockge — Docker Compose Management UI
+
+- **Image**: `louislam/dockge:latest`
+- **Port**: 5001 → 5001
+- **Restart policy**: `unless-stopped`
+- **Homepage group**: PFI-ANA
+- **Compose file**: `compose/dockge/compose.yaml`
+
+**Volumes**:
+| Host / Volume | Container | Purpose |
+|---|---|---|
+| `/var/run/docker.sock` | `/var/run/docker.sock` | Docker socket for managing containers |
+| `dockge_data` (named volume) | `/app/data` | Dockge application data |
+| `/opt/docker/compose` | `/opt/docker/compose` | Compose stack directory |
+
+**Environment**:
+- `DOCKGE_STACKS_DIR=/opt/docker/compose` — tells Dockge where to find/manage compose stacks
+
+**Notes**: Dockge is the management interface for all other compose stacks on this host. It has full Docker daemon access via the socket mount.
+
+---
+
+### 2. llama-swap — Multi-Model LLM Gateway
+
+- **Image**: `ghcr.io/mostlygeek/llama-swap:cuda`
+- **Port**: 9292 → 8080
+- **Runtime**: `nvidia` (CUDA)
+- **Compose file**: `compose/llama-swap/compose.yaml`
+- **Config file**: `conf/llama-swap/config.yaml`
+- **Interactive**: `stdin_open: true`, `tty: true` (required by llama-swap)
+
+**Volumes**:
+| Host Path | Container Path | Purpose |
+|---|---|---|
+| `/opt/docker/conf/llama-swap/config.yaml` | `/app/config.yaml` | llama-swap configuration (models, groups, params) |
+| `/tank/aimodels/llm` | `/models` | LLM model files (GGUF format) |
+
+**Configured Models** (from `config.yaml`):
+
+| Model ID | Display Name | Quantization | Context Size | TTL (s) | Notes |
+|---|---|---|---|---|---|
+| `qwen3-4b` | Qwen3-4B-Instruct-2507-Q6_K | Q6_K | default | 0 (persistent) | Small general-purpose model |
+| `glm4.5-air` | GLM-4.5-Air Q4_K_M | Q4_K_M | 40,000 | 600 | Flash attention enabled |
+| `skyfall-r1-31b-q6k` | Skyfall 31B v4 | Q6_K_L | 40,000 | 600 | Flash attention, full GPU offload |
+| `GLM-Steam-106B-QK4M-A12B` | GLM-Steam 106B A12B | Q4_K_M | 40,000 | 600 | 2-shard model, MoE with 12B active |
+| `kimik2-q2kxl` | Kimi K2 Instruct | UD-Q2_K_XL | default | 600 | 8-shard model, only 2 GPU layers (CPU-heavy) |
+| `qwen3-coder-30b-iq4-nl` | Qwen3 Coder 30B A3B | IQ4_NL | 40,000 | 0 (persistent) | MoE 3B active, coding-optimized |
+| `unsloth-granite-4-small` | Granite 4.0 Small | Q4_K_M | 120,000 | 0 (persistent) | IBM Granite, deterministic (temp=0) |
+| `qwen3.5-35-a3b` | Qwen 3.5 35B A3B | UD-Q4_K_XL | 32,768 | 0 (persistent) | MoE, thinking mode, temp=1.0 |
+| `qwen3.5-35-a3b-code` | Qwen 3.5 35B A3B Code | UD-Q4_K_XL | 32,768 | 0 (persistent) | Same model, code-tuned params (temp=0.6) |
+| `gemma4-26b-a4b` | Gemma 4 26B A4B | UD-Q4_K_XL | 32,768 | 600 | MoE 4B active, thinking enabled, supports images |
+| `gemma4-31b-dense` | Gemma 4 31B Dense | UD-Q4_K_XL | 32,768 | 600 | Full dense model, thinking enabled, supports images |
+| `embeddinggemma-300M` | Embedding Gemma 300M | Q8_0 | 2,048 | 0 (persistent) | Embedding model, cls pooling |
+| `qwen3-embedding-0.6B` | Qwen3 Embedding 0.6B | Q8_0 | 32,768 | 0 (persistent) | Embedding model, mean pooling |
+| `jina-reranker-v3-0.6B` | Jina Reranker v3 | Q8_0 | 32,768 | 0 (persistent) | Reranking model |
+| `bge-reranker-v2-m3-0.6B` | BGE Reranker v2 m3 | Q8_0 | 32,768 | 0 (persistent) | Reranking model |
+
+**Model Groups**:
+
+| Group | Swap | Exclusive | Persistent | Members |
+|---|---|---|---|---|
+| `high-reasoning` | false | false | — | qwen3.5-35-a3b, qwen3.5-35-a3b-code, gemma4-31b-dense |
+| `utility` | false | false | ✅ | embeddinggemma-300M, bge-reranker-v2-m3-0.6B |
+
+**Global Settings**:
+- `healthCheckTimeout`: 1200 seconds (20 minutes) — long timeout for large models
+- `logLevel`: info
+- `metricsMaxInMemory`: 1000
+
+---
+
+### 3. ComfyUI — Image Generation UI
+
+- **Image**: `mmartial/comfyui-nvidia-docker:ubuntu24_cuda13.0-latest`
+- **Port**: 8188 → 8188
+- **Runtime**: `nvidia` with full device reservation (gpu, compute, utility capabilities)
+- **Restart policy**: `unless-stopped`
+- **Homepage group**: AI Systems
+- **Compose file**: `compose/comfyui/compose.yaml`
+
+**Volumes**:
+| Host Path | Container Path | Purpose |
+|---|---|---|
+| `/tank/comfy/run` | `/comfy/mnt` | ComfyUI workspace / output directory |
+| `/tank/aimodels/img/comfy` | `/basedir` | Image models and ComfyUI base directory |
+
+**Environment**:
+| Variable | Value | Purpose |
+|---|---|---|
+| `WANTED_UID` | 1001 | Run as user ID 1001 |
+| `WANTED_GID` | 1002 | Run as group ID 1002 |
+| `BASE_DIRECTORY` | /basedir | ComfyUI base directory path |
+| `SECURITY_LEVEL` | weak | Relaxed security (private network) |
+| `NVIDIA_VISIBLE_DEVICES` | all | Expose all GPUs |
+| `NVIDIA_DRIVER_CAPABILITIES` | all | Enable all GPU capabilities |
+
+**Notes**: Runs with user-mapped permissions (UID 1001 / GID 1002). The `basedir` points to the image model storage on `/tank`.
+
+---
+
+### 4. VibeVoice — Voice/Audio AI Service
+
+- **Image**: `eworkerinc/vibevoice:latest`
+- **Container name**: `vibevoice`
+- **Port**: 8745 → 8745
+- **GPU**: Yes (all devices, gpu capability)
+- **Restart policy**: `unless-stopped`
+- **Homepage group**: AI Systems
+- **Compose file**: `compose/vibevoice/compose.yaml`
+
+**Volumes**:
+| Host Path | Container Path | Purpose |
+|---|---|---|
+| `/tank/vibevoice/hf` | `/root/.cache/huggingface` | HuggingFace model cache |
+| `/tank/vibevoice/voices` | `/app/voices` | Voice data / presets |
+| `/tank/vibevoice/state` | `/var/lib/eworker` | Application state persistence |
+
+**Environment**:
+| Variable | Value | Purpose |
+|---|---|---|
+| `ENABLE_1_5B` | true | Enable 1.5B parameter voice model |
+| `ENABLE_LARGE` | true | Enable large voice model |
+| `AUTH_REQUIRED` | true | Require authentication |
+| `CORS_ENABLED` | true | Enable CORS headers |
+| `ALLOWED_ORIGINS` | * | Allow all origins (development/private network) |
+
+---
+
+### 5. Parakeet STT — Speech-to-Text Service
+
+- **Image**: `parakeet-stt` (locally built)
+- **Port**: 8300 → 8000
+- **GPU**: Yes (all devices, gpu capability)
+- **Restart policy**: `unless-stopped`
+- **Homepage group**: AI Systems
+- **Compose file**: `compose/parakeet/compose.yaml`
+- **Env file**: `.env` (not tracked in project — likely contains API keys or model config)
+
+**Volumes**:
+| Volume | Container Path | Purpose |
+|---|---|---|
+| `parakeet_cache` (named volume) | `/root/.cache` | Model download cache |
+
+**Notes**: Uses a locally-built image (no registry prefix). The `.env` file is referenced but not stored in the project — it likely contains environment-specific configuration on the Docker host.
+
+---
+
+### 6. ChromaDB — Vector Database
+
+- **Image**: `chromadb/chroma:latest`
+- **Container name**: `chromadb`
+- **Port**: 8000 → 8000
+- **GPU**: No
+- **Restart policy**: `unless-stopped`
+- **Homepage group**: AI Systems
+- **Compose file**: `compose/chromadb/compose.yaml`
+
+**Volumes**:
+| Host Path | Container Path | Mode | Purpose |
+|---|---|---|---|
+| `/opt/docker/conf/chromadb` | `/conf` | read-only | Config directory (contains auth_token) |
+| `/tank/chromadb` | `/data` | read-write | Persistent vector data |
+
+**Environment**:
+| Variable | Value | Purpose |
+|---|---|---|
+| `CHROMA_SERVER_AUTHN_CREDENTIALS_FILE` | `/conf/auth_token` | Path to auth token file inside container |
+| `CHROMA_SERVER_AUTHN_PROVIDER` | `chromadb.server.auth.token.TokenAuthenticationServerProvider` | Enable token-based authentication |
+| `IS_PERSISTENT` | TRUE | Enable persistent storage |
+| `PERSIST_DIRECTORY` | `/data` | Where vector data is stored inside container |
+| `ANONYMIZED_TELEMETRY` | FALSE | Disable telemetry |
+
+**Health Check**:
+| Setting | Value |
+|---|---|
+| Test | `curl -f http://localhost:8000/api/v1/health` |
+| Interval | 30s |
+| Timeout | 10s |
+| Retries | 3 |
+| Start period | 40s |
+
+**Authentication**: Token-based. The auth token is stored at `/opt/docker/conf/chromadb/auth_token` on the host (mode 600), generated with `openssl rand -hex 32`. Clients must supply this token to access the API.
+
+**Notes**: CPU-only service (no GPU). The compose-examples directory contains a reference compose file with Traefik labels and a demo app — see [chromadb-setup.md](chromadb-setup.md) for full deployment instructions and the demo.
+
+---
+
+## Infrastructure Summary
+
+```
+┌──────────────────────────────────────────────────────────────┐
+│                    PFI-ANA (10.250.50.70)               │
+│                   Docker Host (PFI-ANA_DOCKER)          │
+│                                                         │
+│  ┌──────────┐    Manages all compose stacks              │
+│  │  Dockge  │◄─── /opt/docker/compose/*                  │
+│  │  :5001   │    /var/run/docker.sock                    │
+│  └──────────┘                                           │
+│                                                         │
+│  ┌──────────────────┐   GPU via passthrough          │
+│  │  llama-swap  :9292 (CUDA, multi-model gateway)   │   │
+│  │  ├── 15 models (chat, code, embedding, reranker) │   │
+│  │  ├── 2 groups (high-reasoning, utility)          │   │
+│  │  └── Models from /tank/aimodels/llm              │   │
+│  └──────────────────┘   GPU via passthrough          │
+│                                                         │
+│  ┌──────────┐  ┌──────────┐  ┌──────────┐  GPU via      │
+│  │  ComfyUI     │  │  VibeVoice   │  │  Parakeet    │  passthrough  │
+│  │  :8188       │  │  :8745       │  │  :8300       │              │
+│  │  (GPU, img)  │  │  (GPU, voice)│  │  (GPU, STT)  │              │
+│  └──────────┘  └──────────┘  └──────────┘              │
+│                                                         │
+│  ┌──────────┐                                       │
+│  │  ChromaDB    │  Token auth, persistent vectors        │
+│  │  :8000       │  /tank/chromadb (data)                 │
+│  │  (CPU only)  │  /opt/docker/conf/chromadb (config)    │
+│  └──────────┘                                       │
+│                                                         │
+│   ── All services on traefik-net (external) ──         │
+│   ── /tank/* = persistent large data storage ──        │
+│   ── /opt/docker/* = config + compose files ──        │
+└──────────────────────────────────────────────────────────────┘
+```
+
+## Port Allocation
+
+| Port | Service | Protocol |
+|---|---|---|
+| 5001 | Dockge | HTTP |
+| 8000 | ChromaDB | HTTP |
+| 8188 | ComfyUI | HTTP |
+| 8300 | Parakeet STT | HTTP (→ container 8000) |
+| 8745 | VibeVoice | HTTP |
+| 9292 | llama-swap | HTTP (→ container 8080) |
+
+## Example Configurations
+
+Reference/example compose files (not live) are stored in `configs/pfi-ana/docker/compose-examples/`:
+- `compose-examples/llama-swap/docker-compose.yml` — earlier llama-swap reference
+- `compose-examples/chromadb/` — ChromaDB with Traefik labels, auth, health checks, and demo app
+
+Full ChromaDB setup instructions: [chromadb-setup.md](chromadb-setup.md)
@@ -0,0 +1,81 @@
+# PFI-ANA Model Inventory
+# Generated from: find /models -maxdepth 3 -name "*.gguf" | sort
+# Synchronized with llama-swap config.yaml on 2025-07-18
+#
+# Status Key:
+#   [ACTIVE]   = configured in llama-swap
+#   [PENDING]  = on disk, not yet in config (or still downloading)
+#   [REMOVED]  = was in old config, not on disk
+#
+# ============================================================================
+# KB Optimization Pass (2025-07-18):
+#   - Qwen 3.5: added presence_penalty, thinking mode enable_thinking
+#   - Qwen3-Coder-Next: temp 0.3→1.0, top-p 0.85→0.95, top-k 20→40, min-p 0→0.01
+#   - Gemma 4: added repeat_penalty 1.0, thinking mode enable_thinking
+#   - Nemotron: already applied in prior pass (--special, min-p 0.01, seed 3407)
+# ============================================================================
+
+[ACTIVE] ./ggml-org_Qwen3-Reranker-0.6B-Q8_0-GGUF/qwen3-reranker-0.6b-q8_0.gguf
+
+[ACTIVE] ./unsloth_Qwen3.5-122B-A10B-GGUF/UD-Q4_K_XL/Qwen3.5-122B-A10B-UD-Q4_K_XL-00001-of-00003.gguf
+[ACTIVE] ./unsloth_Qwen3.5-122B-A10B-GGUF/UD-Q4_K_XL/Qwen3.5-122B-A10B-UD-Q4_K_XL-00002-of-00003.gguf
+[ACTIVE] ./unsloth_Qwen3.5-122B-A10B-GGUF/UD-Q4_K_XL/Qwen3.5-122B-A10B-UD-Q4_K_XL-00003-of-00003.gguf
+
+[ACTIVE] ./ibm-granite_granite-4.0-micro-GGUF/granite-4.0-micro-Q4_K_M.gguf
+
+[ACTIVE] ./unsloth_GLM-4.7-Flash-GGUF/GLM-4.7-Flash-UD-Q4_K_XL.gguf
+
+[ACTIVE] ./jinaai_jina-reranker-v3-GGUF/jina-reranker-v3-Q8_0.gguf
+
+[ACTIVE] ./unsloth_NVIDIA-Nemotron-3-Super-120B-A12B-GGUF/UD-Q4_K_XL/NVIDIA-Nemotron-3-Super-120B-A12B-UD-Q4_K_XL-00001-of-00003.gguf
+[ACTIVE] ./unsloth_NVIDIA-Nemotron-3-Super-120B-A12B-GGUF/UD-Q4_K_XL/NVIDIA-Nemotron-3-Super-120B-A12B-UD-Q4_K_XL-00002-of-00003.gguf
+[ACTIVE] ./unsloth_NVIDIA-Nemotron-3-Super-120B-A12B-GGUF/UD-Q4_K_XL/NVIDIA-Nemotron-3-Super-120B-A12B-UD-Q4_K_XL-00003-of-00003.gguf
+
+[ACTIVE] ./bartowski_TheDrummer_Skyfall-31B-v4-GGUF/TheDrummer_Skyfall-31B-v4-Q6_K_L.gguf
+
+[ACTIVE] ./RP/BeaverAI_Skyfall-R1-31B-v4a-GGUF/Skyfall-R1-31B-v4a-Q6_K.gguf
+
+[ACTIVE] ./RP/bartowski_TheDrummer_GLM-Steam-106B-A12B-v1-GGUF/TheDrummer_GLM-Steam-106B-A12B-v1-Q4_K_M-00001-of-00002.gguf
+[ACTIVE] ./RP/bartowski_TheDrummer_GLM-Steam-106B-A12B-v1-GGUF/TheDrummer_GLM-Steam-106B-A12B-v1-Q4_K_M-00002-of-00002.gguf
+
+[ACTIVE] ./unsloth_Qwen3.5-9B-GGUF/Qwen3.5-9B-UD-Q4_K_XL.gguf
+
+[ACTIVE] ./unsloth_granite-4.0-h-small-GGUF/granite-4.0-h-small-Q4_K_M.gguf
+
+[ACTIVE] ./unsloth_Kimi-K2-Instruct-0905-GGUF/UD-Q2_K_XL/Kimi-K2-Instruct-0905-UD-Q2_K_XL-00001-of-00008.gguf
+[ACTIVE] ./unsloth_Kimi-K2-Instruct-0905-GGUF/UD-Q2_K_XL/Kimi-K2-Instruct-0905-UD-Q2_K_XL-00002-of-00008.gguf
+[ACTIVE] ./unsloth_Kimi-K2-Instruct-0905-GGUF/UD-Q2_K_XL/Kimi-K2-Instruct-0905-UD-Q2_K_XL-00003-of-00008.gguf
+[ACTIVE] ./unsloth_Kimi-K2-Instruct-0905-GGUF/UD-Q2_K_XL/Kimi-K2-Instruct-0905-UD-Q2_K_XL-00004-of-00008.gguf
+[ACTIVE] ./unsloth_Kimi-K2-Instruct-0905-GGUF/UD-Q2_K_XL/Kimi-K2-Instruct-0905-UD-Q2_K_XL-00005-of-00008.gguf
+[ACTIVE] ./unsloth_Kimi-K2-Instruct-0905-GGUF/UD-Q2_K_XL/Kimi-K2-Instruct-0905-UD-Q2_K_XL-00006-of-00008.gguf
+[ACTIVE] ./unsloth_Kimi-K2-Instruct-0905-GGUF/UD-Q2_K_XL/Kimi-K2-Instruct-0905-UD-Q2_K_XL-00007-of-00008.gguf
+[ACTIVE] ./unsloth_Kimi-K2-Instruct-0905-GGUF/UD-Q2_K_XL/Kimi-K2-Instruct-0905-UD-Q2_K_XL-00008-of-00008.gguf
+
+[PENDING] ./unsloth_Qwen3-Coder-Next-GGUF/Qwen3-Coder-Next-UD-Q4_K_XL.gguf
+  NOTE: Download may be incomplete (has .incomplete files). Config entry ready.
+
+[ACTIVE] ./unsloth_Nemotron-3-Nano-30B-A3B-GGUF/Nemotron-3-Nano-30B-A3B-UD-Q4_K_XL.gguf
+
+[ACTIVE] ./unsloth_gemma-4-26B-A4B-it-GGUF/gemma-4-26B-A4B-it-UD-Q4_K_XL.gguf
+
+[ACTIVE] ./unsloth_Qwen3.5-35B-A3B-GGUF/Qwen3.5-35B-A3B-UD-Q4_K_XL.gguf
+
+[ACTIVE] ./Qwen_Qwen3-Embedding-0.6B-GGUF/Qwen3-Embedding-0.6B-Q8_0.gguf
+
+[ACTIVE] ./unsloth_gemma-4-31B-it-GGUF/gemma-4-31B-it-UD-Q4_K_XL.gguf
+
+[ACTIVE] ./ggml-org_embeddinggemma-300M-GGUF/embeddinggemma-300M-Q8_0.gguf
+
+# ============================================================================
+# MODELS REMOVED FROM CONFIG (not on disk or superseded):
+# ============================================================================
+# [REMOVED] qwen3-4b              — Qwen3-4B-Instruct-2507-Q6_K (not on disk)
+# [REMOVED] glm4.5-air            — GLM-4.5-Air-Q4_K_M (not on disk, superseded by glm4.7-flash)
+# [REMOVED] qwen3-coder-30b       — Qwen3-Coder-30B-A3B (not on disk, superseded by qwen3-coder-next)
+# [REMOVED] bge-reranker-v2-m3    — Replaced by qwen3-reranker-0.6B
+# [FIXED]   glm-steam-106b        — Path fixed: was missing RP/ prefix
+# ============================================================================
+# STILL DOWNLOADING (not yet configured):
+# ============================================================================
+# [PENDING] Hermes-4-14B          — Multiple quant downloads in progress (.incomplete files)
+# ============================================================================
@@ -0,0 +1,260 @@
+# PFI-ANA Proxmox VM Inventory
+
+**Hypervisor**: Proxmox VE at `10.250.250.31:8006`
+**Storage Pool**: `ospool` (CEPH/zfs — all VM disks reside here)
+**Network Bridge**: `vmbr0` with VLAN tag `50` on all VMs
+**QEMU Version**: 7.2.0 (primary), VM 106 on 8.1.5
+
+## Virtual Machines
+
+### VM 100 — PFI-ANA-TRUENAS
+
+| Property | Value |
+|---|---|
+| **VMID** | 100 |
+| **OS Type** | Linux (l26) |
+| **CPU** | 2 sockets × 2 cores = 4 vCPU (host passthrough) |
+| **Memory** | 8,196 MB |
+| **Disk** | `scsi0`: 80G on ospool |
+| **CDROM** | `ide2`: TrueNAS-SCALE-22.12.1.iso |
+| **Network** | `net0`: virtio, MAC `9A:90:79:7A:86:87`, vmbr0, VLAN 50 |
+| **Boot** | scsi0 → ide2 → net0 |
+| **Startup** | Order 2, delay 120s |
+| **Onboot** | No |
+
+**Purpose**: TrueNAS SCALE storage appliance. Provides NAS/NFS/iSCSI to the colo environment.
+
+---
+
+### VM 101 — PFI-ANA-DC
+
+| Property | Value |
+|---|---|
+| **VMID** | 101 |
+| **OS Type** | Windows 11 |
+| **BIOS** | OVMF (UEFI) with TPM 2.0 |
+| **Machine** | pc-q35-7.2 |
+| **CPU** | 2 sockets × 6 cores = 12 vCPU (host passthrough) |
+| **Memory** | 24,576 MB |
+| **Disk** | `scsi0`: 240G on ospool |
+| **EFI Disk** | `efidisk0`: 1M on ospool |
+| **TPM** | `tpmstate0`: 4M, v2.0 on ospool |
+| **CDROM** | `scsi1`: virtio-win-0.1.229.iso (VirtIO drivers) |
+| **Network** | `net0`: e1000, MAC `CE:C8:D7:FE:32:40`, vmbr0, VLAN 50 |
+| **Boot** | scsi0 → net0 → ide0 → scsi1 |
+| **Startup** | Order 5, delay 120s |
+| **Onboot** | Yes |
+
+**Purpose**: Windows Domain Controller for the Anaheim environment. UEFI with TPM 2.0 suggests Active Directory / Group Policy services.
+
+---
+
+### VM 102 — PFI-ANA-Docker
+
+| Property | Value |
+|---|---|
+| **VMID** | 102 |
+| **OS Type** | Linux (l26) |
+| **CPU** | 2 sockets × 4 cores = 8 vCPU (x86-64-v2-AES) |
+| **Memory** | 16,384 MB |
+| **Disk** | `scsi0`: 250G on ospool |
+| **Network** | `net0`: virtio, MAC `BA:AF:E7:E9:79:23`, vmbr0, VLAN 50 |
+| **Boot** | scsi0 → net0 → scsi1 |
+| **Startup** | Order 4 |
+| **Onboot** | Yes |
+
+**Purpose**: Primary Docker host for the colo. Runs Dockge for compose management and Traefik for reverse proxy. All Docker services documented in [docker-stack.md](docker-stack.md) run here.
+
+---
+
+### VM 103 — PFI-SlaveBot
+
+| Property | Value |
+|---|---|
+| **VMID** | 103 |
+| **OS Type** | Windows 10 |
+| **Machine** | pc-i440fx-7.2 |
+| **CPU** | 2 sockets × 4 cores = 8 vCPU (host passthrough) |
+| **Memory** | 8,192 MB |
+| **Disk** | `ide0`: 256G on ospool |
+| **Network** | `net0`: e1000, MAC `CE:F0:49:C9:03:70`, vmbr0, VLAN 50 |
+| **Boot** | ide0 → net0 → scsi0 |
+| **Startup** | Not configured |
+| **Onboot** | Yes |
+
+**Purpose**: Windows 10 workstation/bot. Likely a task automation or RDP-accessible machine.
+
+---
+
+### VM 104 — PFI-Mongo
+
+| Property | Value |
+|---|---|
+| **VMID** | 104 |
+| **OS Type** | Linux (l26) |
+| **CPU** | 2 sockets × 4 cores = 8 vCPU (host passthrough) |
+| **Memory** | 8,196 MB |
+| **Disk** | `scsi0`: 256G on ospool |
+| **Network** | `net0`: virtio, MAC `32:57:90:B2:66:61`, vmbr0, VLAN 50 |
+| **Boot** | scsi0 → ide2 → net0 |
+| **Startup** | Order 3, delay 60s |
+| **Onboot** | Yes |
+
+**Purpose**: MongoDB server. Config file contains a connection string reference: `mongodb://10.250.50.81:27017/`.
+
+---
+
+### VM 105 — PFI-Postgres
+
+| Property | Value |
+|---|---|
+| **VMID** | 105 |
+| **OS Type** | Linux (l26) |
+| **CPU** | 4 sockets × 4 cores = 16 vCPU |
+| **Memory** | 8,196 MB |
+| **Disk** | `scsi0`: 80G on ospool |
+| **CDROM** | `ide2`: debian-11.6.0-amd64-netinst.iso |
+| **Network** | `net0`: virtio, MAC `C2:1F:CC:71:66:D0`, vmbr0, VLAN 50 |
+| **Serial** | `serial0`: socket (IPMI/serial console) |
+| **Boot** | scsi0 → ide2 → net0 |
+| **Startup** | Order 3, delay 60s |
+| **Onboot** | Yes |
+
+**Purpose**: PostgreSQL database server running Debian 11.
+
+---
+
+### VM 106 — PFI-Tailscale
+
+| Property | Value |
+|---|---|
+| **VMID** | 106 |
+| **OS Type** | Linux (l26) |
+| **CPU** | 2 sockets × 4 cores = 8 vCPU (x86-64-v2-AES) |
+| **Memory** | 2,048 MB |
+| **Disk** | `scsi0`: 256G on ospool |
+| **CDROM** | `ide2`: debian-12.2.0-amd64-netinst.iso |
+| **Network** | `net0`: virtio, MAC `BC:24:11:D7:E9:52`, vmbr0, VLAN 50 |
+| **Boot** | scsi0 → ide2 → net0 |
+| **Startup** | Not configured |
+| **Onboot** | Yes |
+
+**Purpose**: Tailscale VPN node for mesh connectivity. Provides the VPN tunnel endpoints that link the three PFI sites together. Running Debian 12 (newer than most other VMs). Lightweight at 2G RAM.
+
+---
+
+### VM 107 — PFI-Pteradactyl
+
+| Property | Value |
+|---|---|
+| **VMID** | 107 |
+| **OS Type** | Linux (l26) |
+| **CPU** | 2 sockets × 4 cores = 8 vCPU (host passthrough) |
+| **Memory** | 8,192 MB |
+| **Disk** | `scsi0`: 256G on ospool |
+| **CDROM** | `ide2`: debian-11.6.0-amd64-netinst.iso |
+| **Network** | `net0`: virtio, MAC `CA:44:37:8A:BF:E0`, vmbr0, VLAN 50 |
+| **Boot** | scsi0 → ide2 → net0 |
+| **Startup** | Not configured |
+| **Onboot** | Yes |
+
+**Purpose**: Pterodactyl game server panel. Manages game server instances.
+
+---
+
+### VM 108 — PFI-ANA--DEV
+
+| Property | Value |
+|---|---|
+| **VMID** | 108 |
+| **OS Type** | Linux (l26) |
+| **CPU** | 2 sockets × 4 cores = 8 vCPU |
+| **Memory** | 8,192 MB |
+| **Disk** | `scsi0`: 120G on ospool |
+| **CDROM** | `ide2`: debian-11.6.0-amd64-netinst.iso |
+| **Network** | `net0`: virtio, MAC `F2:EF:82:2C:AF:90`, vmbr0, VLAN 50 |
+| **Boot** | scsi0 → ide2 → net0 |
+| **Startup** | Order 10, delay 60s |
+| **Onboot** | No |
+
+**Purpose**: Development environment. Not set to auto-boot, starts after core infrastructure (order 10).
+
+---
+
+### VM 110 — PFI-ANA-Webhost
+
+| Property | Value |
+|---|---|
+| **VMID** | 110 |
+| **OS Type** | Linux (l26) |
+| **CPU** | 4 sockets × 4 cores = 16 vCPU |
+| **Memory** | 4,096 MB (balloon: 1024 MB minimum) |
+| **Disk** | `scsi0`: 250G on ospool |
+| **CDROM** | `ide2`: debian-11.6.0-amd64-netinst.iso |
+| **Network** | `net0`: virtio, MAC `E6:F9:3A:C9:61:2A`, vmbr0, VLAN 50 |
+| **Boot** | scsi0 → ide2 → net0 |
+| **Startup** | Order 30, up 120s, down 120s |
+| **Onboot** | Yes |
+
+**Purpose**: Web hosting server. Highest startup order (30) — starts last. Also has the longest graceful shutdown timeout (120s). Memory ballooning enabled for dynamic allocation.
+
+---
+
+### VM 111 — pfi-tacticalrmm
+
+| Property | Value |
+|---|---|
+| **VMID** | 111 |
+| **OS Type** | Linux (l26) |
+| **CPU** | 4 sockets × 4 cores = 16 vCPU |
+| **Memory** | 8,192 MB |
+| **Disk** | `scsi0`: 256G on ospool |
+| **Network** | `net0`: virtio, MAC `BA:FA:65:F6:46:25`, vmbr0, VLAN 50 |
+| **Boot** | scsi0 → ide2 → net0 |
+| **Startup** | Order 20 |
+| **Onboot** | Yes |
+
+**Purpose**: Tactical RMM (Remote Monitoring and Management) server. Provides IT management, remote access, and monitoring capabilities.
+
+---
+
+## Startup Order Summary
+
+VMs are brought up in the following order on host boot:
+
+| Order | VMID | Name | Delay |
+|---|---|---|---|
+| 2 | 100 | PFI-ANA-TRUENAS | 120s |
+| 3 | 104 | PFI-Mongo | 60s |
+| 3 | 105 | PFI-Postgres | 60s |
+| 4 | 102 | PFI-ANA-Docker | — |
+| 5 | 101 | PFI-ANA-DC | 120s |
+| 10 | 108 | PFI-ANA--DEV | 60s |
+| 20 | 111 | pfi-tacticalrmm | — |
+| 30 | 110 | PFI-ANA-Webhost | 120s |
+
+VMs without a startup order (103, 106, 107) will start based on their `onboot` setting but without a specific sequencing delay.
+
+## Resource Summary
+
+| VMID | Name | vCPU | RAM (MB) | Disk | OS |
+|---|---|---|---|---|---|
+| 100 | PFI-ANA-TRUENAS | 4 | 8,196 | 80G | TrueNAS SCALE |
+| 101 | PFI-ANA-DC | 12 | 24,576 | 240G | Windows 11 |
+| 102 | PFI-ANA-Docker | 8 | 16,384 | 250G | Linux |
+| 103 | PFI-SlaveBot | 8 | 8,192 | 256G | Windows 10 |
+| 104 | PFI-Mongo | 8 | 8,196 | 256G | Linux |
+| 105 | PFI-Postgres | 16 | 8,196 | 80G | Debian 11 |
+| 106 | PFI-Tailscale | 8 | 2,048 | 256G | Debian 12 |
+| 107 | PFI-Pteradactyl | 8 | 8,192 | 256G | Debian 11 |
+| 108 | PFI-ANA--DEV | 8 | 8,192 | 120G | Debian 11 |
+| 110 | PFI-ANA-Webhost | 16 | 4,096 | 250G | Debian 11 |
+| 111 | pfi-tacticalrmm | 16 | 8,192 | 256G | Linux |
+| | **Totals** | **112** | **105,348** | **2,300G** | |
+
+## Network Notes
+
+- All VMs are on **VLAN 50** via `vmbr0`.
+- All VMs have **firewall enabled** on the network interface.
+- Linux VMs use `virtio` network adapters; Windows VMs use `e1000`.
+- The MongoDB connection string embedded in VM 104's config references IP `10.250.50.81`, suggesting VLAN 50 maps to the `10.250.50.0/24` subnet within the `10.250.0.0/16` range.
@@ -0,0 +1,554 @@
+# Recommended Model Inference Settings — Reference Document
+
+> **Source:** AIPA Knowledge Base — compiled from 5 KB reference documents.
+> **Last Updated:** 2025-07-14
+> **Purpose:** Canonical reference for llama-server / llama.cpp inference parameters across all model families with KB-documented settings.
+
+---
+
+## Table of Contents
+
+1. [NVIDIA Nemotron 3 Super (120B-A12B)](#1-nvidia-nemotron-3-super-120b-a12b)
+2. [NVIDIA Nemotron 3 Nano (4B / 30B-A3B)](#2-nvidia-nemotron-3-nano-4b--30b-a3b)
+3. [Qwen 3.5 Family (0.8B – 397B-A17B)](#3-qwen-35-family-08b--397b-a17b)
+4. [Qwen3-Coder-Next (80B MoE)](#4-qwen3-coder-next-80b-moe)
+5. [Google Gemma 4 Family (E2B – 31B)](#5-google-gemma-4-family-e2b--31b)
+6. [Quick Reference Cards](#6-quick-reference-cards)
+7. [Critical Warnings by Model](#7-critical-warnings-by-model)
+8. [Models Without KB Settings](#8-models-without-kb-settings)
+
+---
+
+## 1. NVIDIA Nemotron 3 Super (120B-A12B)
+
+### Model Overview
+
+| Property | Value |
+|---|---|
+| Architecture | MoE — 120B total, **12B active parameters** |
+| Max Context | **1,048,576** (1M tokens) |
+| Recommended Starting Context | **16K or 32K** — increase gradually |
+| Reasoning Tokens | `<think)>` (ID 12), `</think)>` (ID 13) |
+| Positional Embeddings | **NoPE** — YaRN NOT needed |
+| Best For | Multi-agent AI, high-efficiency reasoning, coding, math |
+| Performance Tier | ~GPT-5.2 / Claude Opus 4.5 level |
+
+### Inference Parameters
+
+| Parameter | General Chat / Instruction | Tool Calling |
+|---|---|---|
+| `temperature` | **1.0** | **0.6** |
+| `top_p` | **1.0** | **0.95** |
+| `min_p` | **0.01** | **0.01** |
+
+### Additional Settings
+
+| Setting | Value | Notes |
+|---|---|---|
+| `--seed` | **3407** | Reproducibility |
+| `--prio` | **2** or **3** | Priority scheduling |
+| `--special` | Required | To see reasoning tokens |
+| `--verbose-prompt` | Required | To see prepended `<think)>` tokens |
+| `max_new_tokens` | 32,768 – 262,144 | Up to 1M |
+
+### Quantization & Memory
+
+| Precision | Memory Required |
+|---|---|
+| UD-Q2_K_XL (2-bit) | ~32–36 GB |
+| **UD-Q4_K_XL (4-bit)** | **~64–72 GB** |
+| 8-bit | ~128 GB |
+| BF16 | ~240 GB |
+
+### Example Command
+
+```bash
+./llama.cpp/llama-server \
+    --model Nemotron-3-Super-UD-Q4_K_XL.gguf \
+    --ctx-size 16384 \
+    --temp 1.0 --top-p 1.0 --min-p 0.01 \
+    --seed 3407 --special \
+    --flash-attn on \
+    --port 8001
+```
+
+---
+
+## 2. NVIDIA Nemotron 3 Nano (4B / 30B-A3B)
+
+### Model Variants
+
+| Variant | Architecture | Context | Active Params | Best Fit |
+|---|---|---|---|---|
+| **Nano-4B** | Dense | 128K | 4B | Lightweight coding, math, agentic tasks |
+| **Nano-30B-A3B** | MoE | 128K | 3B active | Best performance/size on 24GB devices |
+
+### Inference Parameters (Both Variants)
+
+| Parameter | General Chat / Instruction | Tool Calling |
+|---|---|---|
+| `temperature` | **1.0** | **0.6** |
+| `top_p` | **1.0** | **0.95** |
+| `min_p` | **0.01** | **0.01** |
+
+### Additional Settings
+
+| Setting | Value | Notes |
+|---|---|---|
+| `--seed` | **3407** | Reproducibility |
+| `--special` | Not required | Standard chat template for Nano variants |
+
+### Hardware & Quantization
+
+#### Nano-4B
+
+| Precision | Memory |
+|---|---|
+| Q8_0 (8-bit, recommended) | ~3 GB |
+| 4-bit | ~5 GB |
+
+#### Nano-30B-A3B
+
+| Precision | Memory |
+|---|---|
+| **UD-Q4_K_XL (4-bit, recommended)** | **~24 GB** |
+| 8-bit | ~36 GB |
+
+### Example Commands
+
+```bash
+# Nano-4B (8-bit)
+./llama.cpp/llama-server \
+    -hf unsloth/Nemotron-3-Nano-4B-GGUF:Q8_0 \
+    --ctx-size 16384 \
+    --temp 1.0 --top-p 1.0 --min-p 0.01 \
+    --seed 3407 --flash-attn on --port 8001
+
+# Nano-30B-A3B (4-bit)
+./llama.cpp/llama-server \
+    -hf unsloth/Nemotron-3-Nano-30B-A3B-GGUF:UD-Q4_K_XL \
+    --ctx-size 16384 \
+    --temp 1.0 --top-p 1.0 --min-p 0.01 \
+    --seed 3407 --flash-attn on --port 8001
+```
+
+---
+
+## 3. Qwen 3.5 Family (0.8B – 397B-A17B)
+
+### Model Variants
+
+| Variant | Architecture | Context | Languages | Best Fit |
+|---|---|---|---|---|
+| **0.8B** | Dense | 256K | 201 | Smallest edge inference |
+| **2B** | Dense | 256K | 201 | Small device inference |
+| **4B** | Dense | 256K | 201 | Lightweight local use |
+| **9B** | Dense | 256K | 201 | Capable small model |
+| **27B** | Dense | 256K | 201 | Slightly more accurate than 35B-A3B; fits 18GB |
+| **35B-A3B** | MoE (3B active) | 256K | 201 | Best speed/quality tradeoff; fits 22GB |
+| **122B-A10B** | MoE (10B active) | 256K | 201 | High quality; needs ~70GB (4-bit) |
+| **397B-A17B** | MoE (17B active) | 256K (extendable to 1M via YaRN) | 201 | Top-tier performance |
+
+### 27B vs 35B-A3B Decision
+
+- **27B** — Choose for slightly more accurate results when you can't fit a larger model.
+- **35B-A3B** — Choose for much faster inference. MoE with only 3B active parameters.
+
+### Hardware Requirements
+
+| Variant | 3-bit | 4-bit | 6-bit | 8-bit | BF16 |
+|---|---|---|---|---|---|
+| **0.8B / 2B** | 3 GB | 3.5 GB | 5 GB | 7.5 GB | 9 GB |
+| **4B** | 4.5 GB | 5.5 GB | 7 GB | 10 GB | 14 GB |
+| **9B** | 5.5 GB | 6.5 GB | 9 GB | 13 GB | 19 GB |
+| **27B** | 14 GB | 17 GB | 24 GB | 30 GB | 54 GB |
+| **35B-A3B** | 17 GB | 22 GB | 30 GB | 38 GB | 70 GB |
+| **122B-A10B** | 60 GB | 70 GB | 106 GB | 132 GB | 245 GB |
+| **397B-A17B** | 180 GB | 214 GB | 340 GB | 512 GB | 810 GB |
+
+### Inference Parameters
+
+#### Thinking Mode
+
+| Parameter | General Tasks | Precise Coding (e.g. WebDev) |
+|---|---|---|
+| `temperature` | **1.0** | **0.6** |
+| `top_p` | **0.95** | **0.95** |
+| `top_k` | **20** | **20** |
+| `min_p` | **0.0** | **0.0** |
+| `presence_penalty` | **1.5** | **0.0** |
+| `repetition_penalty` | **1.0** (disabled) | **1.0** (disabled) |
+
+#### Non-Thinking (Instruct) Mode
+
+| Parameter | General Tasks | Reasoning Tasks |
+|---|---|---|
+| `temperature` | **0.7** | **1.0** |
+| `top_p` | **0.8** | **0.95** |
+| `top_k` | **20** | **20** |
+| `min_p` | **0.0** | **0.0** |
+| `presence_penalty` | **1.5** | **1.5** |
+| `repetition_penalty` | **1.0** (disabled) | **1.0** (disabled) |
+
+### Thinking Mode Control
+
+Enable thinking:
+```bash
+--chat-template-kwargs '{"enable_thinking":true}'
+```
+
+Disable thinking:
+```bash
+--chat-template-kwargs '{"enable_thinking":false}'
+```
+
+#### Default Thinking Behavior by Variant
+
+| Variant | Thinking Default |
+|---|---|
+| **0.8B, 2B, 4B, 9B** (Small) | **Disabled** — must explicitly enable |
+| **27B, 35B-A3B, 122B-A10B, 397B-A17B** | **Enabled** — must explicitly disable if unwanted |
+
+### Context Settings
+
+| Setting | Value |
+|---|---|
+| Max context window | **262,144** (256K) |
+| Context extension | Up to **1M** via YaRN |
+| Recommended starting context | **16,384** (16K) for responsiveness |
+| Adequate output length | **32,768** tokens |
+
+### Quantization Notes
+
+- All GGUFs use **Unsloth Dynamic 2.0** quantization — important layers upcasted to 8 or 16-bit even in 4-bit.
+- Recommended starting point: **Dynamic 4-bit** (`UD-Q4_K_XL`).
+- Minimum recommended: **Dynamic 2-bit** (`UD-Q2_K_XL`).
+
+### Example Commands
+
+```bash
+# 35B-A3B — Thinking Mode (General)
+./llama.cpp/llama-server \
+    -hf unsloth/Qwen3.5-35B-A3B-GGUF:UD-Q4_K_XL \
+    --ctx-size 16384 \
+    --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.00 \
+    --chat-template-kwargs '{"enable_thinking":true}' \
+    --flash-attn on --port 8001
+
+# 9B — Thinking Enabled (small models default to disabled)
+./llama.cpp/llama-server \
+    -hf unsloth/Qwen3.5-9B-GGUF:UD-Q4_K_XL \
+    --ctx-size 16384 \
+    --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.00 \
+    --chat-template-kwargs '{"enable_thinking":true}' \
+    --flash-attn on --port 8001
+```
+
+---
+
+## 4. Qwen3-Coder-Next (80B MoE)
+
+### Model Overview
+
+| Property | Value |
+|---|---|
+| Architecture | MoE — 80B total, **3B active parameters** |
+| Max Context | **262,144** (256K) |
+| Recommended Context | **32,768** for less memory use |
+| Thinking Mode | **Non-reasoning only** — no `<think)>` blocks |
+| Best For | Fast agentic coding, long-horizon reasoning, complex tool use |
+| Performance Tier | Comparable to models with 10–20× more active parameters |
+
+### Inference Parameters
+
+| Parameter | Value | Notes |
+|---|---|---|
+| `temperature` | **1.0** | |
+| `top_p` | **0.95** | |
+| `top_k` | **40** | Note: higher than Qwen3.5 general |
+| `min_p` | **0.01** | llama.cpp default is 0.05 — override to 0.01 |
+| `repetition_penalty` | **1.0** (disabled) | Only increase if you see looping |
+
+### Quick Reference One-Liner
+
+```
+temperature=1.0, top_p=0.95, top_k=40, min_p=0.01, repetition_penalty=1.0
+```
+
+### Hardware Requirements
+
+| Precision | Memory Required |
+|---|---|
+| 3-bit (UD-IQ3_XXS) | ~34 GB |
+| **4-bit (UD-Q4_K_XL)** | **~46 GB** |
+| 8-bit | ~85 GB |
+| BF16 | ~160 GB |
+
+### Key Differences from Qwen3.5 General Models
+
+- **No thinking mode** — this is a non-reasoning model; ultra-quick code responses
+- **Higher `top_k`** (40 vs. 20) — broader sampling for creative code generation
+- `enable_thinking` flag is not applicable
+
+### Example Commands
+
+```bash
+# llama-server deployment
+./llama.cpp/llama-server \
+    --model Qwen3-Coder-Next-UD-Q4_K_XL.gguf \
+    --alias "unsloth/Qwen3-Coder-Next" \
+    --seed 3407 \
+    --temp 1.0 --top-p 0.95 --min-p 0.01 --top-k 40 \
+    --ctx-size 32768 \
+    --flash-attn on --port 8001
+```
+
+### vLLM FP8 Dynamic (GPU Premium)
+
+```bash
+CUDA_VISIBLE_DEVICES='0,1,2,3' vllm serve unsloth/Qwen3-Coder-Next-FP8-Dynamic \
+    --served-model-name unsloth/Qwen3-Coder-Next \
+    --tensor-parallel-size 4 \
+    --tool-call-parser qwen3_coder \
+    --enable-auto-tool-choice \
+    --dtype bfloat16 --seed 3407 \
+    --max-model-len 200000 \
+    --gpu-memory-utilization 0.93 \
+    --port 8001
+```
+
+---
+
+## 5. Google Gemma 4 Family (E2B – 31B)
+
+### Model Variants
+
+| Variant | Architecture | Context | Modalities | Best Fit |
+|---|---|---|---|---|
+| **E2B** | Dense + PLE | 128K | Text, Image, Audio | Phone / edge, ASR, speech translation |
+| **E4B** | Dense + PLE | 128K | Text, Image, Audio | Laptops, fast local multimodal |
+| **26B-A4B** | MoE (4B active) | 256K | Text, Image | Best speed/quality tradeoff |
+| **31B** | Dense | 256K | Text, Image | Strongest performance |
+
+### Inference Parameters (All Variants)
+
+These are Google's default Gemma 4 parameters:
+
+| Parameter | Value |
+|---|---|
+| `temperature` | **1.0** |
+| `top_p` | **0.95** |
+| `top_k` | **64** |
+| `repetition_penalty` | **1.0** (disabled — only increase if looping) |
+| End-of-sentence token | `<turn|>` |
+
+### Context Length
+
+- **E2B / E4B**: max **128K**
+- **26B-A4B / 31B**: max **256K**
+- **Practical tip**: Start with **32K** for responsiveness, then increase as needed.
+
+### Hardware Requirements
+
+| Variant | 4-bit | 8-bit | BF16/FP16 |
+|---|---|---|---|
+| **E2B** | 4 GB | 5–8 GB | 10 GB |
+| **E4B** | 5.5–6 GB | 9–12 GB | 16 GB |
+| **26B-A4B** | 16–18 GB | 28–30 GB | 52 GB |
+| **31B** | 17–20 GB | 34–38 GB | 62 GB |
+
+### Quantization Recommendations
+
+- **E2B / E4B** (small): prefer **Q8_0** (8-bit) for quality.
+- **26B-A4B / 31B** (large): prefer **UD-Q4_K_XL** (Dynamic 4-bit) as starting point.
+
+### 26B-A4B vs 31B Decision
+
+- **26B-A4B** — Choose when RAM is limited. MoE with 4B active params = faster. Slight quality tradeoff.
+- **31B** — Choose when you have ≥20 GB (4-bit) and want maximum quality. Slower inference.
+
+### Thinking Mode
+
+**Enable thinking** — add to system prompt:
+```
+<|think|>
+You are a careful coding assistant. Explain your answer clearly.
+```
+
+Model outputs:
+```
+<|channel>thought
+[internal reasoning]
+<channel|>
+[final answer]
+```
+
+**Disable thinking** via llama-server:
+```bash
+--chat-template-kwargs '{"enable_thinking":false}'
+```
+
+> **Multi-turn rule:** Only keep the final visible answer in chat history. Do NOT feed prior thought blocks back.
+
+### Multimodal Settings
+
+- **Images/Audio** go **before text** in prompts.
+- **Video**: pass frames first, then instruction.
+- **Audio** is only on E2B and E4B. Max audio: 30s. Max video: 60s (1 fps).
+
+#### Visual Token Budgets
+
+| Budget | Use Case |
+|---|---|
+| 70 / 140 | Classification, captioning, fast video |
+| 280 / 560 | General multimodal chat, charts, UI |
+| 1120 | OCR, document parsing, handwriting |
+
+### Example Commands
+
+```bash
+# 26B-A4B (Dynamic 4-bit)
+./llama.cpp/llama-server \
+    -hf unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q4_K_XL \
+    --temp 1.0 --top-p 0.95 --top-k 64 \
+    --ctx-size 32768 --flash-attn on --port 8001
+
+# 31B (Dynamic 4-bit) with thinking
+./llama.cpp/llama-server \
+    -hf unsloth/gemma-4-31B-it-GGUF:UD-Q4_K_XL \
+    --temp 1.0 --top-p 0.95 --top-k 64 \
+    --ctx-size 32768 \
+    --chat-template-kwargs '{"enable_thinking":true}' \
+    --flash-attn on --port 8001
+```
+
+---
+
+## 6. Quick Reference Cards
+
+### Temperature by Model & Use Case
+
+| Model Family | General / Chat | Coding / Precise | Tool Calling | Non-Thinking |
+|---|---|---|---|---|
+| **Nemotron 3 Super** | 1.0 | — | 0.6 | — |
+| **Nemotron 3 Nano** | 1.0 | — | 0.6 | — |
+| **Qwen 3.5** (thinking) | 1.0 | 0.6 | — | — |
+| **Qwen 3.5** (non-thinking) | — | — | — | 0.7 (general) / 1.0 (reasoning) |
+| **Qwen3-Coder-Next** | 1.0 | 1.0 | — | — |
+| **Gemma 4** | 1.0 | 1.0 | — | — |
+
+### Top-P by Model & Use Case
+
+| Model Family | General / Chat | Coding / Precise | Tool Calling | Non-Thinking |
+|---|---|---|---|---|
+| **Nemotron 3 Super** | 1.0 | — | 0.95 | — |
+| **Nemotron 3 Nano** | 1.0 | — | 0.95 | — |
+| **Qwen 3.5** (thinking) | 0.95 | 0.95 | — | — |
+| **Qwen 3.5** (non-thinking) | — | — | — | 0.8 (general) / 0.95 (reasoning) |
+| **Qwen3-Coder-Next** | 0.95 | 0.95 | — | — |
+| **Gemma 4** | 0.95 | 0.95 | — | — |
+
+### Top-K by Model
+
+| Model Family | Top-K |
+|---|---|
+| **Nemotron 3** | Default (not specified) |
+| **Qwen 3.5** | **20** |
+| **Qwen3-Coder-Next** | **40** |
+| **Gemma 4** | **64** |
+
+### Min-P by Model
+
+| Model Family | Min-P |
+|---|---|
+| **Nemotron 3 (all)** | **0.01** |
+| **Qwen 3.5** | **0.0** |
+| **Qwen3-Coder-Next** | **0.01** |
+| **Gemma 4** | Default (not specified) |
+
+### Presence Penalty by Model
+
+| Model Family | General | Coding | Reasoning |
+|---|---|---|---|
+| **Nemotron 3** | Default | Default | Default |
+| **Qwen 3.5** (thinking) | **1.5** | **0.0** | **1.5** |
+| **Qwen 3.5** (non-thinking) | **1.5** | — | **1.5** |
+| **Qwen3-Coder-Next** | Default | Default | Default |
+| **Gemma 4** | Default | Default | Default |
+
+### Repetition Penalty by Model
+
+| Model Family | Value | Notes |
+|---|---|---|
+| **Nemotron 3** | Default | — |
+| **Qwen 3.5** | **1.0** (disabled) | — |
+| **Qwen3-Coder-Next** | **1.0** (disabled) | Only increase if looping |
+| **Gemma 4** | **1.0** (disabled) | Only increase if looping |
+
+### Seed Values
+
+| Model Family | Recommended Seed |
+|---|---|
+| **Nemotron 3 (all)** | **3407** |
+| **Qwen 3.5** | 3407 (optional) |
+| **Qwen3-Coder-Next** | **3407** |
+| **Gemma 4** | Default |
+
+---
+
+## 7. Critical Warnings by Model
+
+### Nemotron 3 Super
+- ⚠️ Do NOT attempt 1M context on first run — increase gradually from 16K/32K.
+- ⚠️ Setting context to 1M may trigger **CUDA OOM and crash**.
+- ⚠️ Router-layer fine-tuning **disabled by default** in Unsloth for MoE models.
+
+### Qwen 3.5
+- ⚠️ **No Qwen3.5 GGUF works in Ollama** due to separate mmproj vision files. Use llama.cpp-compatible backends.
+- ⚠️ `presence_penalty` above 0.0 may cause **slight performance decrease**.
+- ⚠️ If getting gibberish, check context length or add `--cache-type-k bf16 --cache-type-v bf16`.
+- ⚠️ Redownload older GGUFs — all updated with improved quantization and tool-calling template fixes.
+
+### Qwen3-Coder-Next
+- ⚠️ **Update llama.cpp** — a previous bug in `vectorized key_gdiff` caused looping/output issues.
+- ⚠️ **No Ollama support** — use llama.cpp-compatible backends.
+- ⚠️ If context length too low, may see `exceeds the available context size` errors.
+- ⚠️ Tool-calling improved after llama.cpp parsing fixes (Feb 19 update) — use recent version.
+
+### Gemma 4
+- ⚠️ **Do NOT use CUDA 13.2 runtime** for any GGUF — causes poor outputs.
+- ⚠️ Use `llama-server` (not `llama-cli`) for thinking control — more reliable.
+- ⚠️ Multi-turn: **only keep the final visible answer** in chat history. Do NOT feed prior thought blocks back.
+
+---
+
+## 8. Models Without KB Settings
+
+The following model families are deployed in the Infrastructure-PFI environment but **do not have KB-documented inference parameters**. Settings for these models use general best practices or vendor defaults:
+
+| Model | Notes |
+|---|---|
+| **DeepSeek R1 0528** | No KB doc — use general MoE defaults |
+| **Mistral Small 3.1** | No KB doc — use vendor defaults |
+| **GLM-4.7-Flash** | No KB doc — use vendor defaults |
+| **GLM Steam 106B-A12B** | No KB doc — use general MoE defaults |
+| **Granite 4.0 Micro** | No KB doc — use IBM defaults |
+| **Kimi K2** | No KB doc — use general MoE defaults |
+| **Skyfall R1 31B v4a** | No KB doc — use general defaults |
+| **Hermes 4 14B** | No KB doc (download pending) — use vendor defaults |
+
+---
+
+## KB Source Documents
+
+| Document | Path in KB |
+|---|---|
+| Nemotron 3 Super Running Parameters | `reference/nemotron-3-super-running-parameters.md` |
+| Nemotron 3 Nano Running Parameters | `reference/nemotron-3-nano-running-parameters.md` |
+| Qwen 3.5 Running Parameters | `reference/qwen3.5-running-parameters.md` |
+| Qwen3-Coder-Next Running Parameters | `reference/qwen3-coder-next-running-parameters.md` |
+| Gemma 4 Running Parameters | `reference/gemma-4-running-parameters.md` |
+
+---
+
+*Document generated by Linus (Systems Architect) from AIPA Knowledge Base content curated by Atlas.*
@@ -0,0 +1,261 @@
+---
+id: vm-102-matrix-appservice
+title: "VM 102 — Matrix Appservice Configuration"
+summary: "AIPA Matrix Application Service bridge setup for VM 102. Covers appservice registration, agent virtual users, room routing, env vars, operational notes, and troubleshooting."
+tags: ["infrastructure", "pfi", "pfi-ana", "matrix", "docker", "vm-102", "appservice", "bridge", "aipa", "deployment", "integration"]
+keywords: ["matrix.pfi.local", "appservice", "as_token", "hs_token", "aipa-bridge", "matrix_bridge.py", "8009", "send_notification", "virtual-users", "room-routing", "element"]
+links:
+  - "[VM 102 — Matrix Synapse Deployment](vm-102-matrix-synapse.md)"
+  - "[Matrix Bridge — Application Service Integration](../../../KB/projects/aipa/matrix-bridge.md)"
+  - "[Docker Stack Conventions](docker-stack.md)"
+created: 2026-04-11T00:00:00+00:00
+modified: 2026-04-11T00:00:00+00:00
+path: docs/pfi-ana/vm-102-matrix-appservice.md
+---
+
+# VM 102 — Matrix Appservice Configuration
+
+## Overview
+
+The AIPA Matrix bridge registers with Synapse as a Matrix Application Service. This means:
+- Synapse **pushes** all relevant events to the bridge (no polling)
+- The bridge manages **virtual users** for each agent — no real accounts needed
+- The bridge authenticates to Synapse with a shared `as_token`
+- Synapse authenticates its pushes to the bridge with a shared `hs_token`
+
+> Prerequisite: [VM 102 — Matrix Synapse Deployment](vm-102-matrix-synapse.md) must be complete.
+
+---
+
+## Application Service Model
+
+```
+Synapse ──── push events ────► AIPA Bridge (port 8009)
+                                    │
+                              routes to agent
+                                    │
+                              posts response ◄──── Synapse ◄──── Element ◄──── User
+```
+
+### Agent Virtual Users
+
+| Agent    | Matrix ID                  | Display Name |
+|----------|----------------------------|--------------|
+| Atlas    | `@atlas:matrix.pfi.local`  | Atlas        |
+| Linus    | `@linus:matrix.pfi.local`  | Linus        |
+| Hermione | `@hermione:matrix.pfi.local` | Hermione   |
+
+These are **virtual** — managed entirely by the bridge. Do not register them as real Synapse accounts.
+
+---
+
+## Step 1 — Generate Tokens
+
+Two random tokens are needed:
+
+```bash
+python3 -c "import secrets; print(secrets.token_hex(32))"  # as_token (bridge → Synapse)
+python3 -c "import secrets; print(secrets.token_hex(32))"  # hs_token (Synapse → bridge)
+```
+
+---
+
+## Step 2 — Create Appservice Registration File
+
+File: `/opt/docker/data/synapse/aipa_appservice.yaml`
+
+```yaml
+id: aipa-bridge
+url: http://<BRIDGE_HOST_IP>:8009    # IP/hostname the bridge is reachable from Synapse container
+as_token: "<AS_TOKEN_FROM_STEP_1>"
+hs_token: "<HS_TOKEN_FROM_STEP_1>"
+sender_localpart: aipa-bot           # @aipa-bot:matrix.pfi.local (unused fallback sender)
+namespaces:
+  users:
+    - exclusive: true
+      regex: "@(atlas|linus|hermione):.*"
+  rooms: []
+  aliases: []
+rate_limited: false
+```
+
+- **`exclusive: true`** — only the bridge can act as those users; no one can register `@atlas` as a real account.
+- **`url`** — must be reachable from inside the Synapse container. If the bridge runs on the VM host, use the host IP or Docker gateway IP (e.g., `172.17.0.1`).
+
+### Restart Synapse to Load
+
+```bash
+cd /opt/docker/compose/synapse && docker compose restart synapse
+```
+
+---
+
+## Step 3 — Configure AIPA Environment
+
+### env.sh additions
+
+```bash
+# ── Matrix bridge ──────────────────────────────────────────────────────
+export MATRIX_HOMESERVER_URL="http://localhost:8008"
+export MATRIX_SERVER_NAME="matrix.pfi.local"
+export MATRIX_AS_TOKEN="<as_token from Step 1>"
+export MATRIX_HS_TOKEN="<hs_token from Step 1>"
+# Optional: room to send agent notifications when no channel is specified
+# export MATRIX_DEFAULT_ROOM="!roomid:matrix.pfi.local"
+```
+
+### providers.yaml matrix section
+
+```yaml
+matrix:
+  homeserver_url: "${MATRIX_HOMESERVER_URL}"
+  server_name: "${MATRIX_SERVER_NAME}"
+  as_token: "${MATRIX_AS_TOKEN}"
+  hs_token: "${MATRIX_HS_TOKEN}"
+  bridge_host: "0.0.0.0"
+  bridge_port: 8009
+  default_notification_room: "${MATRIX_DEFAULT_ROOM}"
+  agents:
+    atlas:
+      display_name: "Atlas"
+      avatar_url: ""
+    linus:
+      display_name: "Linus"
+      avatar_url: ""
+    hermione:
+      display_name: "Hermione"
+      avatar_url: ""
+```
+
+---
+
+## Step 4 — Start the Bridge
+
+```bash
+source .venv/bin/activate && source env.sh
+python -m core.matrix_bridge
+```
+
+Options:
+
+```
+python -m core.matrix_bridge --host 0.0.0.0 --port 8009
+python -m core.matrix_bridge --no-profile-setup   # skip display name / avatar init
+```
+
+On first start, the bridge:
+1. Registers display names for each agent user
+2. Starts the appservice HTTP server on port 8009
+3. Listens for Matrix transactions from Synapse
+4. Accepts invitations to rooms
+5. Begins routing messages
+
+---
+
+## Step 5 — First Use
+
+1. Open Element at `http://10.250.50.70:8080`
+2. Log in with your admin account (server: `matrix.pfi.local`)
+3. Open a DM with `@atlas:matrix.pfi.local`
+4. Send a message — the bridge routes it to Atlas and posts the response
+
+To start a room with a specific agent, invite them:
+- New room → Invite `@linus:matrix.pfi.local` → Linus joins automatically
+- All messages in that room go to Linus
+
+---
+
+## Room Routing
+
+1. User invites an agent user to a room (or opens a DM)
+2. Bridge receives the `m.room.member` invite event, agent auto-joins
+3. Room is permanently mapped to that agent in `sessions/matrix_rooms.json`
+4. All subsequent messages in that room are routed to the mapped agent
+5. If multiple agent users are in the same room, the first invite wins for session purposes;
+   subsequent agents each get their own session (multi-agent collaboration rooms)
+
+---
+
+## Operational Notes
+
+### Room Session Persistence
+
+The bridge persists the room→agent→session mapping to `sessions/matrix_rooms.json`
+under `AIPA_ROOT`. If the bridge restarts, existing rooms continue their sessions.
+
+To reset a room's conversation history:
+- Ask the agent `/reset`, or
+- Delete the entry from `matrix_rooms.json` and restart the bridge.
+
+### Formatting
+
+The bridge converts markdown in agent responses to Matrix HTML
+(`format: org.matrix.custom.html`). Code blocks, bold, and inline code are
+rendered correctly in Element.
+
+### Typing Indicators
+
+The bridge sends a typing indicator (`m.typing`) while the agent is processing,
+so users see the animated dots while waiting.
+
+### Proactive Notifications
+
+Agents can push messages to Matrix rooms via `send_notification`:
+
+```
+send_notification(
+    content="KB rebuild complete — 1,247 chunks indexed.",
+    channel="!roomid:matrix.pfi.local"
+)
+```
+
+- If `channel` is set, it must be a Matrix room ID (`!roomid:server`)
+- If `channel` is omitted, the notification goes to `MATRIX_DEFAULT_ROOM` if configured
+
+### Agent Users Always Show as Offline
+
+Expected behavior — virtual users don't have presence. This is normal for appservice users.
+
+---
+
+## Troubleshooting
+
+| Symptom | Likely Cause | Fix |
+|---------|-------------|-----|
+| Bridge starts but Synapse doesn't push events | Appservice URL wrong in registration YAML | Verify `url:` is reachable from the Synapse container; check `docker inspect synapse` network |
+| Agent joins room but doesn't respond | `hs_token` mismatch | Verify token in `aipa_appservice.yaml` matches `MATRIX_HS_TOKEN` env var |
+| 401 errors from Synapse | `as_token` mismatch | Verify token in `aipa_appservice.yaml` matches `MATRIX_AS_TOKEN` env var |
+| Agent user shows as offline always | Expected — virtual users don't have presence | Normal for appservice users; no fix needed |
+| Messages loop (agent replies to itself) | Bridge not filtering its own messages | Check `SENDER_LOCALPART` filter in bridge; bridge ignores all managed agent users as senders |
+| Element can't connect to homeserver | Wrong `base_url` in element-config.json | Must be the IP/hostname Element's browser can reach, not the Docker container name |
+
+---
+
+## Dependencies
+
+| Dependency | Version | Purpose |
+|------------|---------|---------|
+| `markdown` | any | Markdown→HTML rendering for Matrix responses |
+| `fastapi` | any | Appservice HTTP server |
+| `httpx` | any | Client-Server API calls to Synapse |
+
+Install: `pip install markdown` (or `pip install -r requirements.txt`)
+
+---
+
+## Security Notes
+
+- The `as_token` and `hs_token` are secrets equivalent to admin credentials. Store
+  them in `env.sh` (gitignored) and never commit them.
+- The bridge runs with `MATRIX_HS_TOKEN` to authenticate Synapse's push requests. Any
+  request without this token in the `Authorization` header is rejected (403).
+- Restrict port 8009 to internal access only (firewall or bind to `127.0.0.1` if bridge and Synapse are on the same host).
+
+---
+
+## Sources
+
+- Source deployment guide: `projects/matrix/matrix-deployment.md` (2026-04-11)
+- AIPA bridge code: `core/matrix_bridge.py`
+- AIPA bridge KB entry: `KB/projects/aipa/matrix-bridge.md`
+- Matrix Synapse appservice docs: https://element-hq.github.io/synapse/latest/application_services.html
@@ -0,0 +1,340 @@
+---
+id: vm-102-matrix-synapse
+title: "VM 102 — Matrix Synapse Deployment"
+summary: "Docker deployment of Synapse homeserver with PostgreSQL and Element Web on VM 102 (PFI-ANA-Docker). Covers compose file, configuration, storage paths, and admin setup."
+tags: ["infrastructure", "pfi", "pfi-ana", "matrix", "docker", "vm-102", "synapse", "self-hosted", "deployment"]
+keywords: ["matrix.pfi.local", "synapse", "element-web", "10.250.50.70", "VM-102", "PFI-ANA-Docker", "postgres:16", "appservice", "AIPA"]
+links:
+  - "[Matrix Protocol Reference](../Infrastructure-PFI-Project-Index.md)"
+  - "[VM-102 Proxmox Config](../../../configs/pfi-ana/proxmox/vm-102.conf)"
+  - "[Docker Stack Conventions](docker-stack.md)"
+created: 2026-04-11T00:00:00+00:00
+modified: 2026-04-11T00:00:00+00:00
+path: docs/pfi-ana/vm-102-matrix-synapse.md
+---
+
+# VM 102 — Matrix Synapse Deployment
+
+## Overview
+
+Matrix (Synapse) is deployed on VM 102 (PFI-ANA-Docker, `10.250.50.70`) as the primary
+human-to-agent communication channel for AIPA. The stack consists of:
+
+- **Synapse** — Matrix homeserver (event routing, auth, persistence)
+- **PostgreSQL 16** — Synapse database backend
+- **Element Web** — Web client for users
+
+Federation is disabled (internal-only deployment). Registration is disabled (admin-created accounts only).
+
+---
+
+## Architecture
+
+```
+User (Element client)
+       │  m.room.message events
+       ▼
+┌──────────────────────┐
+│  Synapse             │  Matrix homeserver — event routing, auth, persistence
+│  (homeserver)        │  Port 8008 (Client-Server API)
+└──────────┬───────────┘
+           │  Appservice push  PUT /_matrix/app/v1/transactions/{txnId}
+           ▼
+┌──────────────────────────────────────────┐
+│  AIPA Matrix Bridge  (core/matrix_bridge.py)  │
+│  Port 8009                                │
+└──────────────────────────────────────────┘
+```
+
+> Full appservice bridge details: [VM 102 — Matrix Appservice Configuration](vm-102-matrix-appservice.md)
+
+---
+
+## Components
+
+| Component       | Image                          | Port  | Purpose                           |
+|-----------------|--------------------------------|-------|-----------------------------------|
+| `synapse`       | `matrixdotorg/synapse:latest`  | 8008  | Matrix homeserver (Client-Server API) |
+| `synapse-db`    | `postgres:16`                  | —     | Synapse database                  |
+| `element-web`   | `vectorim/element-web:latest`  | 8080  | Web client for users              |
+
+---
+
+## Storage Paths
+
+| Host Path                          | Purpose                                 |
+|------------------------------------|-----------------------------------------|
+| `/opt/docker/compose/synapse/`     | Compose file (managed by Dockge)        |
+| `/opt/docker/conf/synapse/`        | Configuration files                     |
+| `/opt/docker/data/synapse/`        | Synapse data (`homeserver.yaml`, signing keys, appservice reg) |
+| `/opt/docker/conf/synapse/element-config.json` | Element Web client config    |
+
+> Follows the VM 102 convention: compose in `/opt/docker/compose/<service>/`, config in `/opt/docker/conf/<service>/`.
+
+### Volumes
+
+| Volume / Path                  | Container Path                | Purpose                     |
+|--------------------------------|-------------------------------|-----------------------------|
+| `/opt/docker/data/synapse`     | `/data`                       | Synapse config, signing keys, media store |
+| `synapse-db-data` (named vol)  | `/var/lib/postgresql/data`    | PostgreSQL persistent data   |
+
+---
+
+## Step 1 — Create Directory Structure
+
+```bash
+mkdir -p /opt/docker/data/synapse
+mkdir -p /opt/docker/conf/synapse
+mkdir -p /opt/docker/compose/synapse
+```
+
+---
+
+## Step 2 — Generate Synapse Config
+
+```bash
+docker run --rm \
+  -v /opt/docker/data/synapse:/data \
+  -e SYNAPSE_SERVER_NAME=matrix.pfi.local \
+  -e SYNAPSE_REPORT_STATS=no \
+  matrixdotorg/synapse:latest generate
+```
+
+This writes `/opt/docker/data/synapse/homeserver.yaml` and
+`/opt/docker/data/synapse/matrix.pfi.local.signing.key`.
+
+**Do not modify `server_name` after generation — it is permanent.**
+
+---
+
+## Step 3 — Edit homeserver.yaml
+
+Open `/opt/docker/data/synapse/homeserver.yaml` and apply:
+
+```yaml
+# Use PostgreSQL instead of SQLite (required for production)
+database:
+  name: psycopg2
+  args:
+    user: synapse
+    password: synapse_db_password   # match POSTGRES_PASSWORD in compose
+    database: synapse
+    host: synapse-db
+    cp_min: 5
+    cp_max: 10
+
+# Disable open registration — accounts are created by admin only
+enable_registration: false
+
+# Disable federation (internal deployment only)
+federation_domain_whitelist: []
+
+# Allow the application service to be registered (add AFTER generating the AS file)
+app_service_config_files:
+  - /data/aipa_appservice.yaml
+```
+
+---
+
+## Step 4 — Docker Compose
+
+File: `/opt/docker/compose/synapse/docker-compose.yml`
+
+```yaml
+---
+# =============================================================================
+# Synapse Matrix Homeserver — AIPA internal deployment on VM 102
+# =============================================================================
+#
+# Conventions:
+#   - Config:  /opt/docker/conf/synapse/
+#   - Data:    /opt/docker/data/synapse/  (bind mount) + synapse-db-data (named vol)
+#   - Compose: /opt/docker/compose/synapse/
+#   - Network: synapse-net (dedicated, not on traefik-net)
+#
+# Notes:
+#   - Federation disabled (internal only)
+#   - Port 8448 (federation) commented out
+#   - Resource limits set for VM 102 (8 vCPU, 16 GB RAM)
+
+services:
+  synapse-db:
+    image: postgres:16
+    container_name: synapse-db
+    restart: unless-stopped
+    environment:
+      POSTGRES_USER: synapse
+      POSTGRES_PASSWORD: synapse_db_password
+      POSTGRES_DB: synapse
+      POSTGRES_INITDB_ARGS: "--encoding=UTF-8 --lc-collate=C --lc-ctype=C"
+    volumes:
+      - synapse-db-data:/var/lib/postgresql/data
+    networks:
+      - synapse-net
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U synapse"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+
+  synapse:
+    image: matrixdotorg/synapse:latest
+    container_name: synapse
+    restart: unless-stopped
+    depends_on:
+      synapse-db:
+        condition: service_healthy
+    ports:
+      - "8008:8008"   # Client-Server API (HTTP)
+      # - "8448:8448" # Server-Server API (federation) — disabled for internal use
+    volumes:
+      - /opt/docker/data/synapse:/data
+    networks:
+      - synapse-net
+    deploy:
+      resources:
+        limits:
+          memory: 1G
+          cpus: "2.0"
+        reservations:
+          memory: 256M
+    healthcheck:
+      test: ["CMD-SHELL", "curl -fsS http://localhost:8008/health || exit 1"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 30s
+
+  element-web:
+    image: vectorim/element-web:latest
+    container_name: element-web
+    restart: unless-stopped
+    volumes:
+      - /opt/docker/conf/synapse/element-config.json:/app/config.json:ro
+    ports:
+      - "8080:80"
+    networks:
+      - synapse-net
+
+volumes:
+  synapse-db-data:
+
+networks:
+  synapse-net:
+    name: synapse-net
+```
+
+### Start
+
+```bash
+cd /opt/docker/compose/synapse
+docker compose up -d
+```
+
+### Verify
+
+```bash
+docker compose ps
+curl http://localhost:8008/health
+```
+
+---
+
+## Step 5 — Element Web Configuration
+
+File: `/opt/docker/conf/synapse/element-config.json`
+
+```json
+{
+    "default_server_config": {
+        "m.homeserver": {
+            "base_url": "http://10.250.50.70:8008",
+            "server_name": "matrix.pfi.local"
+        }
+    },
+    "brand": "AIPA",
+    "default_theme": "dark",
+    "disable_guests": true,
+    "disable_login_language_selector": true
+}
+```
+
+---
+
+## Step 6 — Create Admin User
+
+```bash
+docker exec -it synapse register_new_matrix_user \
+  -u admin \
+  -p 'yourpassword' \
+  -a \
+  http://localhost:8008
+```
+
+> The `-a` flag makes the user an admin. Omit for regular users.
+
+**Agent users** (`@atlas`, `@linus`, `@hermione`) are **virtual** — managed by the
+appservice. Do **not** register them as real accounts.
+
+---
+
+## Port Allocation
+
+| Port | Service       | Purpose                    | Protocol |
+|------|---------------|----------------------------|----------|
+| 8008 | Synapse       | Client-Server API          | HTTP     |
+| 8080 | Element Web   | Web client                 | HTTP     |
+| 8009 | AIPA Bridge   | Appservice endpoint        | HTTP     |
+
+> See [docker-stack.md](docker-stack.md) for the full VM 102 port allocation table.
+
+---
+
+## Network
+
+This deployment uses a **dedicated `synapse-net` network** (not `traefik-net`), because:
+
+- Synapse is accessed directly by IP (no public domain routing needed)
+- The AIPA bridge connects to Synapse at `http://localhost:8008` from the host
+- Element Web connects at the VM IP:8008 from the browser
+
+If TLS/reverse proxy is added later, join `traefik-net` and add Traefik labels.
+
+---
+
+## Security Notes
+
+- Synapse is exposed on port 8008 (HTTP). For any externally accessible deployment,
+  put it behind a TLS-terminating reverse proxy (Traefik/nginx) and restrict 8009
+  to internal access only.
+- Registration is disabled (`enable_registration: false`) — accounts created by admin only.
+- Federation is disabled (`federation_domain_whitelist: []`) — internal use only.
+- The `as_token` and `hs_token` in the appservice registration are secrets equivalent to
+  admin credentials. Store in `env.sh` (gitignored), never commit.
+
+---
+
+## Troubleshooting
+
+| Symptom | Likely Cause | Fix |
+|---------|-------------|-----|
+| Bridge starts but Synapse doesn't push events | Appservice URL wrong in registration YAML | Verify `url:` is reachable from the Synapse container; check `docker inspect synapse` network |
+| 401 errors from Synapse | `as_token` mismatch | Verify token in `aipa_appservice.yaml` matches `MATRIX_AS_TOKEN` env var |
+| Element can't connect to homeserver | Wrong `base_url` in element-config.json | Must be the IP/hostname Element's browser can reach, not the Docker container name |
+| Synapse won't start | Database connection failure | Verify `synapse-db` is healthy first; check password matches in `homeserver.yaml` and compose |
+
+---
+
+## Next Steps
+
+After Synapse is running and healthy, configure the AIPA appservice bridge:
+→ [VM 102 — Matrix Appservice Configuration](vm-102-matrix-appservice.md)
+
+---
+
+## Sources
+
+- Source deployment guide: `projects/matrix/matrix-deployment.md` (2026-04-11)
+- VM 102 Proxmox config: `configs/pfi-ana/proxmox/vm-102.conf`
+- Docker Stack conventions: `docs/pfi-ana/docker-stack.md`
+- Matrix Protocol Reference: `infrastructure/matrix-docker-deployment.md` (KB)