esh-pfi-infrastructure/docs/pfi/vm-102-matrix-appservice.md

VM 102 — Matrix Appservice Configuration

Overview

The AIPA Matrix bridge registers with Synapse as a Matrix Application Service. This means:

• Synapse pushes all relevant events to the bridge (no polling)
• The bridge manages virtual users for each agent — no real accounts needed
• The bridge authenticates to Synapse with a shared as_token
• Synapse authenticates its pushes to the bridge with a shared hs_token

Prerequisite: VM 102 — Matrix Synapse Deployment must be complete.

---

Application Service Model

Synapse ──── push events ────► AIPA Bridge (port 8009)
                                    │
                              routes to agent
                                    │
                              posts response ◄──── Synapse ◄──── Element ◄──── User

Agent Virtual Users

Agent	Matrix ID	Display Name
Atlas	@atlas:matrix.pfi.local	Atlas
Linus	@linus:matrix.pfi.local	Linus
Hermione	@hermione:matrix.pfi.local	Hermione

These are virtual — managed entirely by the bridge. Do not register them as real Synapse accounts.

---

Step 1 — Generate Tokens

Two random tokens are needed:

python3 -c "import secrets; print(secrets.token_hex(32))"  # as_token (bridge → Synapse)
python3 -c "import secrets; print(secrets.token_hex(32))"  # hs_token (Synapse → bridge)

---

Step 2 — Create Appservice Registration File

File: /opt/docker/data/synapse/aipa_appservice.yaml

id: aipa-bridge
url: http://<BRIDGE_HOST_IP>:8009    # IP/hostname the bridge is reachable from Synapse container
as_token: "<AS_TOKEN_FROM_STEP_1>"
hs_token: "<HS_TOKEN_FROM_STEP_1>"
sender_localpart: aipa-bot           # @aipa-bot:matrix.pfi.local (unused fallback sender)
namespaces:
  users:
    - exclusive: true
      regex: "@(atlas|linus|hermione):.*"
  rooms: []
  aliases: []
rate_limited: false

• exclusive: true — only the bridge can act as those users; no one can register @atlas as a real account.
• url — must be reachable from inside the Synapse container. If the bridge runs on the VM host, use the host IP or Docker gateway IP (e.g., 172.17.0.1).

Restart Synapse to Load

cd /opt/docker/compose/synapse && docker compose restart synapse

---

Step 3 — Configure AIPA Environment

env.sh additions

# ── Matrix bridge ──────────────────────────────────────────────────────
export MATRIX_HOMESERVER_URL="http://localhost:8008"
export MATRIX_SERVER_NAME="matrix.pfi.local"
export MATRIX_AS_TOKEN="<as_token from Step 1>"
export MATRIX_HS_TOKEN="<hs_token from Step 1>"
# Optional: room to send agent notifications when no channel is specified
# export MATRIX_DEFAULT_ROOM="!roomid:matrix.pfi.local"

providers.yaml matrix section

matrix:
  homeserver_url: "${MATRIX_HOMESERVER_URL}"
  server_name: "${MATRIX_SERVER_NAME}"
  as_token: "${MATRIX_AS_TOKEN}"
  hs_token: "${MATRIX_HS_TOKEN}"
  bridge_host: "0.0.0.0"
  bridge_port: 8009
  default_notification_room: "${MATRIX_DEFAULT_ROOM}"
  agents:
    atlas:
      display_name: "Atlas"
      avatar_url: ""
    linus:
      display_name: "Linus"
      avatar_url: ""
    hermione:
      display_name: "Hermione"
      avatar_url: ""

---

Step 4 — Start the Bridge

source .venv/bin/activate && source env.sh
python -m core.matrix_bridge

Options:

python -m core.matrix_bridge --host 0.0.0.0 --port 8009
python -m core.matrix_bridge --no-profile-setup   # skip display name / avatar init

On first start, the bridge:

1. Registers display names for each agent user
2. Starts the appservice HTTP server on port 8009
3. Listens for Matrix transactions from Synapse
4. Accepts invitations to rooms
5. Begins routing messages

---

Step 5 — First Use

1. Open Element at http://10.250.50.70:8080
2. Log in with your admin account (server: matrix.pfi.local)
3. Open a DM with @atlas:matrix.pfi.local
4. Send a message — the bridge routes it to Atlas and posts the response

To start a room with a specific agent, invite them:

• New room → Invite @linus:matrix.pfi.local → Linus joins automatically
• All messages in that room go to Linus

---

Room Routing

1. User invites an agent user to a room (or opens a DM)
2. Bridge receives the m.room.member invite event, agent auto-joins
3. Room is permanently mapped to that agent in sessions/matrix_rooms.json
4. All subsequent messages in that room are routed to the mapped agent
5. If multiple agent users are in the same room, the first invite wins for session purposes; subsequent agents each get their own session (multi-agent collaboration rooms)

---

Operational Notes

Room Session Persistence

The bridge persists the room→agent→session mapping to sessions/matrix_rooms.json under AIPA_ROOT. If the bridge restarts, existing rooms continue their sessions.

To reset a room's conversation history:

• Ask the agent /reset, or
• Delete the entry from matrix_rooms.json and restart the bridge.

Formatting

The bridge converts markdown in agent responses to Matrix HTML (format: org.matrix.custom.html). Code blocks, bold, and inline code are rendered correctly in Element.

Typing Indicators

The bridge sends a typing indicator (m.typing) while the agent is processing, so users see the animated dots while waiting.

Proactive Notifications

Agents can push messages to Matrix rooms via send_notification:

send_notification(
    content="KB rebuild complete — 1,247 chunks indexed.",
    channel="!roomid:matrix.pfi.local"
)

• If channel is set, it must be a Matrix room ID (!roomid:server)
• If channel is omitted, the notification goes to MATRIX_DEFAULT_ROOM if configured

Agent Users Always Show as Offline

Expected behavior — virtual users don't have presence. This is normal for appservice users.

---

Troubleshooting

Symptom	Likely Cause	Fix
Bridge starts but Synapse doesn't push events	Appservice URL wrong in registration YAML	Verify url: is reachable from the Synapse container; check docker inspect synapse network
Agent joins room but doesn't respond	hs_token mismatch	Verify token in aipa_appservice.yaml matches MATRIX_HS_TOKEN env var
401 errors from Synapse	as_token mismatch	Verify token in aipa_appservice.yaml matches MATRIX_AS_TOKEN env var
Agent user shows as offline always	Expected — virtual users don't have presence	Normal for appservice users; no fix needed
Messages loop (agent replies to itself)	Bridge not filtering its own messages	Check SENDER_LOCALPART filter in bridge; bridge ignores all managed agent users as senders
Element can't connect to homeserver	Wrong base_url in element-config.json	Must be the IP/hostname Element's browser can reach, not the Docker container name

---

Dependencies

Dependency	Version	Purpose
markdown	any	Markdown→HTML rendering for Matrix responses
fastapi	any	Appservice HTTP server
httpx	any	Client-Server API calls to Synapse

Install: pip install markdown (or pip install -r requirements.txt)

---

Security Notes

• The as_token and hs_token are secrets equivalent to admin credentials. Store them in env.sh (gitignored) and never commit them.
• The bridge runs with MATRIX_HS_TOKEN to authenticate Synapse's push requests. Any request without this token in the Authorization header is rejected (403).
• Restrict port 8009 to internal access only (firewall or bind to 127.0.0.1 if bridge and Synapse are on the same host).

---

Sources

• Source deployment guide: projects/matrix/matrix-deployment.md (2026-04-11)
• AIPA bridge code: core/matrix_bridge.py
• AIPA bridge KB entry: KB/projects/aipa/matrix-bridge.md
• Matrix Synapse appservice docs: https://element-hq.github.io/synapse/latest/application_services.html