vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit: PFI fleet inventory, stacks, tooling, and backup pipeline

Browse Source 
Captures the full workspace state built up to this point:

  - CLAUDE.md + README.md describing conventions and the four-host fleet
    (ana-ml2, ana-docker, nh3-docker, esh-docker-vm).
  - Per-host notes under servers/<host>/ with ssh-target fallback files
    and latest system-details snapshots (two in-compose credential leaks
    scrubbed; the upstream compose files still need to move those to .env).
  - scripts/: server_inspect.sh (read-only remote diagnostic),
    refresh-server-info.sh (dir-driven discovery + snapshot capture with
    validation warnings), add-host.sh, sync-stacks.sh (pull
    compose/conf trees), deploy-stack.sh (push with per-file diff + prompt).
  - stacks/: canonical compose for backrest, beszel, dozzle, llama-swap,
    rest-server-ana, rest-server-nh3, vllm-qwen3, plus the retired
    infinity reference. All use the .env-driven + traefik-net + homepage
    label pattern.
  - configs/restic/ana-docker/: first resticprofile config + pre-backup
    hook (Synapse pg_dump, Seafile mysqldump, Vaultwarden SQLite); templates
    for the other three hosts to come.
  - docs/pfi/: general infrastructure reference carried over.
  - .gitignore excludes .env, stacks-mirror/, and assorted secret/state
    filenames to prevent re-leaks on later commits.
This commit is contained in:
vh
2026-04-20 14:29:48 -07:00
commit e376d0aec9
55 changed files with 9101 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/pfi/vm-102-matrix-synapse.md
+340
View File
@@ -0,0 +1,340 @@
---
id: vm-102-matrix-synapse
title: "VM 102 — Matrix Synapse Deployment"
summary: "Docker deployment of Synapse homeserver with PostgreSQL and Element Web on VM 102 (PFI-ANA-Docker). Covers compose file, configuration, storage paths, and admin setup."
tags: ["infrastructure", "pfi", "pfi-ana", "matrix", "docker", "vm-102", "synapse", "self-hosted", "deployment"]
keywords: ["matrix.pfi.local", "synapse", "element-web", "10.250.50.70", "VM-102", "PFI-ANA-Docker", "postgres:16", "appservice", "AIPA"]
links:
- "[Matrix Protocol Reference](../Infrastructure-PFI-Project-Index.md)"
- "[VM-102 Proxmox Config](../../../configs/pfi-ana/proxmox/vm-102.conf)"
- "[Docker Stack Conventions](docker-stack.md)"
created: 2026-04-11T00:00:00+00:00
modified: 2026-04-11T00:00:00+00:00
path: docs/pfi-ana/vm-102-matrix-synapse.md
---
# VM 102 — Matrix Synapse Deployment
## Overview
Matrix (Synapse) is deployed on VM 102 (PFI-ANA-Docker, `10.250.50.70`) as the primary
human-to-agent communication channel for AIPA. The stack consists of:
- **Synapse** — Matrix homeserver (event routing, auth, persistence)
- **PostgreSQL 16** — Synapse database backend
- **Element Web** — Web client for users
Federation is disabled (internal-only deployment). Registration is disabled (admin-created accounts only).
---
## Architecture
```
User (Element client)
│ m.room.message events
▼
┌──────────────────────┐
│ Synapse │ Matrix homeserver — event routing, auth, persistence
│ (homeserver) │ Port 8008 (Client-Server API)
└──────────┬───────────┘
│ Appservice push PUT /_matrix/app/v1/transactions/{txnId}
▼
┌──────────────────────────────────────────┐
│ AIPA Matrix Bridge (core/matrix_bridge.py) │
│ Port 8009 │
└──────────────────────────────────────────┘
```
> Full appservice bridge details: [VM 102 — Matrix Appservice Configuration](vm-102-matrix-appservice.md)
---
## Components
| Component | Image | Port | Purpose |
|-----------------|--------------------------------|-------|-----------------------------------|
| `synapse` | `matrixdotorg/synapse:latest` | 8008 | Matrix homeserver (Client-Server API) |
| `synapse-db` | `postgres:16` | — | Synapse database |
| `element-web` | `vectorim/element-web:latest` | 8080 | Web client for users |
---
## Storage Paths
| Host Path | Purpose |
|------------------------------------|-----------------------------------------|
| `/opt/docker/compose/synapse/` | Compose file (managed by Dockge) |
| `/opt/docker/conf/synapse/` | Configuration files |
| `/opt/docker/data/synapse/` | Synapse data (`homeserver.yaml`, signing keys, appservice reg) |
| `/opt/docker/conf/synapse/element-config.json` | Element Web client config |
> Follows the VM 102 convention: compose in `/opt/docker/compose/<service>/`, config in `/opt/docker/conf/<service>/`.
### Volumes
| Volume / Path | Container Path | Purpose |
|--------------------------------|-------------------------------|-----------------------------|
| `/opt/docker/data/synapse` | `/data` | Synapse config, signing keys, media store |
| `synapse-db-data` (named vol) | `/var/lib/postgresql/data` | PostgreSQL persistent data |
---
## Step 1 — Create Directory Structure
```bash
mkdir -p /opt/docker/data/synapse
mkdir -p /opt/docker/conf/synapse
mkdir -p /opt/docker/compose/synapse
```
---
## Step 2 — Generate Synapse Config
```bash
docker run --rm \
-v /opt/docker/data/synapse:/data \
-e SYNAPSE_SERVER_NAME=matrix.pfi.local \
-e SYNAPSE_REPORT_STATS=no \
matrixdotorg/synapse:latest generate
```
This writes `/opt/docker/data/synapse/homeserver.yaml` and
`/opt/docker/data/synapse/matrix.pfi.local.signing.key`.
**Do not modify `server_name` after generation — it is permanent.**
---
## Step 3 — Edit homeserver.yaml
Open `/opt/docker/data/synapse/homeserver.yaml` and apply:
```yaml
# Use PostgreSQL instead of SQLite (required for production)
database:
name: psycopg2
args:
user: synapse
password: synapse_db_password # match POSTGRES_PASSWORD in compose
database: synapse
host: synapse-db
cp_min: 5
cp_max: 10
# Disable open registration — accounts are created by admin only
enable_registration: false
# Disable federation (internal deployment only)
federation_domain_whitelist: []
# Allow the application service to be registered (add AFTER generating the AS file)
app_service_config_files:
- /data/aipa_appservice.yaml
```
---
## Step 4 — Docker Compose
File: `/opt/docker/compose/synapse/docker-compose.yml`
```yaml
---
# =============================================================================
# Synapse Matrix Homeserver — AIPA internal deployment on VM 102
# =============================================================================
#
# Conventions:
# - Config: /opt/docker/conf/synapse/
# - Data: /opt/docker/data/synapse/ (bind mount) + synapse-db-data (named vol)
# - Compose: /opt/docker/compose/synapse/
# - Network: synapse-net (dedicated, not on traefik-net)
#
# Notes:
# - Federation disabled (internal only)
# - Port 8448 (federation) commented out
# - Resource limits set for VM 102 (8 vCPU, 16 GB RAM)
services:
synapse-db:
image: postgres:16
container_name: synapse-db
restart: unless-stopped
environment:
POSTGRES_USER: synapse
POSTGRES_PASSWORD: synapse_db_password
POSTGRES_DB: synapse
POSTGRES_INITDB_ARGS: "--encoding=UTF-8 --lc-collate=C --lc-ctype=C"
volumes:
- synapse-db-data:/var/lib/postgresql/data
networks:
- synapse-net
healthcheck:
test: ["CMD-SHELL", "pg_isready -U synapse"]
interval: 10s
timeout: 5s
retries: 5
synapse:
image: matrixdotorg/synapse:latest
container_name: synapse
restart: unless-stopped
depends_on:
synapse-db:
condition: service_healthy
ports:
- "8008:8008" # Client-Server API (HTTP)
# - "8448:8448" # Server-Server API (federation) — disabled for internal use
volumes:
- /opt/docker/data/synapse:/data
networks:
- synapse-net
deploy:
resources:
limits:
memory: 1G
cpus: "2.0"
reservations:
memory: 256M
healthcheck:
test: ["CMD-SHELL", "curl -fsS http://localhost:8008/health || exit 1"]
interval: 30s
timeout: 10s
retries: 3
start_period: 30s
element-web:
image: vectorim/element-web:latest
container_name: element-web
restart: unless-stopped
volumes:
- /opt/docker/conf/synapse/element-config.json:/app/config.json:ro
ports:
- "8080:80"
networks:
- synapse-net
volumes:
synapse-db-data:
networks:
synapse-net:
name: synapse-net
```
### Start
```bash
cd /opt/docker/compose/synapse
docker compose up -d
```
### Verify
```bash
docker compose ps
curl http://localhost:8008/health
```
---
## Step 5 — Element Web Configuration
File: `/opt/docker/conf/synapse/element-config.json`
```json
{
"default_server_config": {
"m.homeserver": {
"base_url": "http://10.250.50.70:8008",
"server_name": "matrix.pfi.local"
}
},
"brand": "AIPA",
"default_theme": "dark",
"disable_guests": true,
"disable_login_language_selector": true
}
```
---
## Step 6 — Create Admin User
```bash
docker exec -it synapse register_new_matrix_user \
-u admin \
-p 'yourpassword' \
-a \
http://localhost:8008
```
> The `-a` flag makes the user an admin. Omit for regular users.
**Agent users** (`@atlas`, `@linus`, `@hermione`) are **virtual** — managed by the
appservice. Do **not** register them as real accounts.
---
## Port Allocation
| Port | Service | Purpose | Protocol |
|------|---------------|----------------------------|----------|
| 8008 | Synapse | Client-Server API | HTTP |
| 8080 | Element Web | Web client | HTTP |
| 8009 | AIPA Bridge | Appservice endpoint | HTTP |
> See [docker-stack.md](docker-stack.md) for the full VM 102 port allocation table.
---
## Network
This deployment uses a **dedicated `synapse-net` network** (not `traefik-net`), because:
- Synapse is accessed directly by IP (no public domain routing needed)
- The AIPA bridge connects to Synapse at `http://localhost:8008` from the host
- Element Web connects at the VM IP:8008 from the browser
If TLS/reverse proxy is added later, join `traefik-net` and add Traefik labels.
---
## Security Notes
- Synapse is exposed on port 8008 (HTTP). For any externally accessible deployment,
put it behind a TLS-terminating reverse proxy (Traefik/nginx) and restrict 8009
to internal access only.
- Registration is disabled (`enable_registration: false`) — accounts created by admin only.
- Federation is disabled (`federation_domain_whitelist: []`) — internal use only.
- The `as_token` and `hs_token` in the appservice registration are secrets equivalent to
admin credentials. Store in `env.sh` (gitignored), never commit.
---
## Troubleshooting
| Symptom | Likely Cause | Fix |
|---------|-------------|-----|
| Bridge starts but Synapse doesn't push events | Appservice URL wrong in registration YAML | Verify `url:` is reachable from the Synapse container; check `docker inspect synapse` network |
| 401 errors from Synapse | `as_token` mismatch | Verify token in `aipa_appservice.yaml` matches `MATRIX_AS_TOKEN` env var |
| Element can't connect to homeserver | Wrong `base_url` in element-config.json | Must be the IP/hostname Element's browser can reach, not the Docker container name |
| Synapse won't start | Database connection failure | Verify `synapse-db` is healthy first; check password matches in `homeserver.yaml` and compose |
---
## Next Steps
After Synapse is running and healthy, configure the AIPA appservice bridge:
→ [VM 102 — Matrix Appservice Configuration](vm-102-matrix-appservice.md)
---
## Sources
- Source deployment guide: `projects/matrix/matrix-deployment.md` (2026-04-11)
- VM 102 Proxmox config: `configs/pfi-ana/proxmox/vm-102.conf`
- Docker Stack conventions: `docs/pfi-ana/docker-stack.md`
- Matrix Protocol Reference: `infrastructure/matrix-docker-deployment.md` (KB)