esh-pfi-infrastructure/docs/pfi/vm-102-matrix-appservice.md

# VM 102 — Matrix Appservice Configuration

## Overview

The AIPA Matrix bridge registers with Synapse as a Matrix Application Service. This means:
- Synapse **pushes** all relevant events to the bridge (no polling)
- The bridge manages **virtual users** for each agent — no real accounts needed
- The bridge authenticates to Synapse with a shared `as_token`
- Synapse authenticates its pushes to the bridge with a shared `hs_token`

> Prerequisite: [VM 102 — Matrix Synapse Deployment](vm-102-matrix-synapse.md) must be complete.

---

## Application Service Model

```
Synapse ──── push events ────► AIPA Bridge (port 8009)
                                    │
                              routes to agent
                                    │
                              posts response ◄──── Synapse ◄──── Element ◄──── User
```

### Agent Virtual Users

| Agent    | Matrix ID                  | Display Name |
|----------|----------------------------|--------------|
| Atlas    | `@atlas:matrix.pfi.local`  | Atlas        |
| Linus    | `@linus:matrix.pfi.local`  | Linus        |
| Hermione | `@hermione:matrix.pfi.local` | Hermione   |

These are **virtual** — managed entirely by the bridge. Do not register them as real Synapse accounts.

---

## Step 1 — Generate Tokens

Two random tokens are needed:

```bash
python3 -c "import secrets; print(secrets.token_hex(32))"  # as_token (bridge → Synapse)
python3 -c "import secrets; print(secrets.token_hex(32))"  # hs_token (Synapse → bridge)
```

---

## Step 2 — Create Appservice Registration File

File: `/opt/docker/data/synapse/aipa_appservice.yaml`

```yaml
id: aipa-bridge
url: http://<BRIDGE_HOST_IP>:8009    # IP/hostname the bridge is reachable from Synapse container
as_token: "<AS_TOKEN_FROM_STEP_1>"
hs_token: "<HS_TOKEN_FROM_STEP_1>"
sender_localpart: aipa-bot           # @aipa-bot:matrix.pfi.local (unused fallback sender)
namespaces:
  users:
    - exclusive: true
      regex: "@(atlas|linus|hermione):.*"
  rooms: []
  aliases: []
rate_limited: false
```

- **`exclusive: true`** — only the bridge can act as those users; no one can register `@atlas` as a real account.
- **`url`** — must be reachable from inside the Synapse container. If the bridge runs on the VM host, use the host IP or Docker gateway IP (e.g., `172.17.0.1`).

### Restart Synapse to Load

```bash
cd /opt/docker/compose/synapse && docker compose restart synapse
```

---

## Step 3 — Configure AIPA Environment

### env.sh additions

```bash
# ── Matrix bridge ──────────────────────────────────────────────────────
export MATRIX_HOMESERVER_URL="http://localhost:8008"
export MATRIX_SERVER_NAME="matrix.pfi.local"
export MATRIX_AS_TOKEN="<as_token from Step 1>"
export MATRIX_HS_TOKEN="<hs_token from Step 1>"
# Optional: room to send agent notifications when no channel is specified
# export MATRIX_DEFAULT_ROOM="!roomid:matrix.pfi.local"
```

### providers.yaml matrix section

```yaml
matrix:
  homeserver_url: "${MATRIX_HOMESERVER_URL}"
  server_name: "${MATRIX_SERVER_NAME}"
  as_token: "${MATRIX_AS_TOKEN}"
  hs_token: "${MATRIX_HS_TOKEN}"
  bridge_host: "0.0.0.0"
  bridge_port: 8009
  default_notification_room: "${MATRIX_DEFAULT_ROOM}"
  agents:
    atlas:
      display_name: "Atlas"
      avatar_url: ""
    linus:
      display_name: "Linus"
      avatar_url: ""
    hermione:
      display_name: "Hermione"
      avatar_url: ""
```

---

## Step 4 — Start the Bridge

```bash
source .venv/bin/activate && source env.sh
python -m core.matrix_bridge
```

Options:

```
python -m core.matrix_bridge --host 0.0.0.0 --port 8009
python -m core.matrix_bridge --no-profile-setup   # skip display name / avatar init
```

On first start, the bridge:
1. Registers display names for each agent user
2. Starts the appservice HTTP server on port 8009
3. Listens for Matrix transactions from Synapse
4. Accepts invitations to rooms
5. Begins routing messages

---

## Step 5 — First Use

1. Open Element at `http://10.250.50.70:8080`
2. Log in with your admin account (server: `matrix.pfi.local`)
3. Open a DM with `@atlas:matrix.pfi.local`
4. Send a message — the bridge routes it to Atlas and posts the response

To start a room with a specific agent, invite them:
- New room → Invite `@linus:matrix.pfi.local` → Linus joins automatically
- All messages in that room go to Linus

---

## Room Routing

1. User invites an agent user to a room (or opens a DM)
2. Bridge receives the `m.room.member` invite event, agent auto-joins
3. Room is permanently mapped to that agent in `sessions/matrix_rooms.json`
4. All subsequent messages in that room are routed to the mapped agent
5. If multiple agent users are in the same room, the first invite wins for session purposes;
   subsequent agents each get their own session (multi-agent collaboration rooms)

---

## Operational Notes

### Room Session Persistence

The bridge persists the room→agent→session mapping to `sessions/matrix_rooms.json`
under `AIPA_ROOT`. If the bridge restarts, existing rooms continue their sessions.

To reset a room's conversation history:
- Ask the agent `/reset`, or
- Delete the entry from `matrix_rooms.json` and restart the bridge.

### Formatting

The bridge converts markdown in agent responses to Matrix HTML
(`format: org.matrix.custom.html`). Code blocks, bold, and inline code are
rendered correctly in Element.

### Typing Indicators

The bridge sends a typing indicator (`m.typing`) while the agent is processing,
so users see the animated dots while waiting.

### Proactive Notifications

Agents can push messages to Matrix rooms via `send_notification`:

```
send_notification(
    content="KB rebuild complete — 1,247 chunks indexed.",
    channel="!roomid:matrix.pfi.local"
)
```

- If `channel` is set, it must be a Matrix room ID (`!roomid:server`)
- If `channel` is omitted, the notification goes to `MATRIX_DEFAULT_ROOM` if configured

### Agent Users Always Show as Offline

Expected behavior — virtual users don't have presence. This is normal for appservice users.

---

## Troubleshooting

| Symptom | Likely Cause | Fix |
|---------|-------------|-----|
| Bridge starts but Synapse doesn't push events | Appservice URL wrong in registration YAML | Verify `url:` is reachable from the Synapse container; check `docker inspect synapse` network |
| Agent joins room but doesn't respond | `hs_token` mismatch | Verify token in `aipa_appservice.yaml` matches `MATRIX_HS_TOKEN` env var |
| 401 errors from Synapse | `as_token` mismatch | Verify token in `aipa_appservice.yaml` matches `MATRIX_AS_TOKEN` env var |
| Agent user shows as offline always | Expected — virtual users don't have presence | Normal for appservice users; no fix needed |
| Messages loop (agent replies to itself) | Bridge not filtering its own messages | Check `SENDER_LOCALPART` filter in bridge; bridge ignores all managed agent users as senders |
| Element can't connect to homeserver | Wrong `base_url` in element-config.json | Must be the IP/hostname Element's browser can reach, not the Docker container name |

---

## Dependencies

| Dependency | Version | Purpose |
|------------|---------|---------|
| `markdown` | any | Markdown→HTML rendering for Matrix responses |
| `fastapi` | any | Appservice HTTP server |
| `httpx` | any | Client-Server API calls to Synapse |

Install: `pip install markdown` (or `pip install -r requirements.txt`)

---

## Security Notes

- The `as_token` and `hs_token` are secrets equivalent to admin credentials. Store
  them in `env.sh` (gitignored) and never commit them.
- The bridge runs with `MATRIX_HS_TOKEN` to authenticate Synapse's push requests. Any
  request without this token in the `Authorization` header is rejected (403).
- Restrict port 8009 to internal access only (firewall or bind to `127.0.0.1` if bridge and Synapse are on the same host).

---

## Sources

- Source deployment guide: `projects/matrix/matrix-deployment.md` (2026-04-11)
- AIPA bridge code: `core/matrix_bridge.py`
- AIPA bridge KB entry: `KB/projects/aipa/matrix-bridge.md`
- Matrix Synapse appservice docs: https://element-hq.github.io/synapse/latest/application_services.html