From 931bac8f68327f4ef961b5979a6e2fd85cbaa97e Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Tue, 1 Sep 2026 11:16:07 -0700 Subject: [PATCH] docs(matrix): current state, upgrade procedure, alias convention, push findings Synapse v1.120.0 -> v1.159.0 and Element-web v1.11.80 -> v1.12.27 (2026-09-01). The existing build steps date from the AIPA era and are now marked as provenance rather than as instructions. Records what only existed in a session transcript: - Schema migrations are one-way; rollback is restore-from-dump. Pre-upgrade pg_dump procedure, with a pg_restore --list verification step. - Why the appservice user namespace is now exclusive: false. exclusive governs who ELSE may act, not what the appservice may do, so on a closed single-admin server it locked out all other account creation to prevent squatting that cannot occur. Includes the two things not to do: narrow the regex (orphans 13 accounts) or rename the id (Synapse keys ownership on it). - The shared-secret registration HMAC takes no trailing null after notadmin. - Room alias convention #-, operator-ratified, with its cost accepted deliberately and its rationale stated as room-identity-carries-tier rather than push-payload-carries-room-name. - Push reality: the pusher is event_id_only, so the notification is assembled on-device by Element X's service extension. Records the resulting server-invisible failure mode when the phone cannot reach the homeserver. - QR sign-in requires Matrix Authentication Service and why it is deferred. Ops ownership recorded: worldtree-dev writes the bridge, infra-ops operates this instance. --- docs/pfi/vm-102-matrix-synapse.md | 164 ++++++++++++++++++++++++++++++ 1 file changed, 164 insertions(+) diff --git a/docs/pfi/vm-102-matrix-synapse.md b/docs/pfi/vm-102-matrix-synapse.md index 9529d6a..00c9ec6 100644 --- a/docs/pfi/vm-102-matrix-synapse.md +++ b/docs/pfi/vm-102-matrix-synapse.md @@ -11,6 +11,170 @@ human-to-agent communication channel for AIPA. The stack consists of: Federation is disabled (internal-only deployment). Registration is disabled (admin-created accounts only). +> ⚠ **The build steps below date from the AIPA era and describe how this was +> originally stood up.** They are kept for provenance. For what is actually +> running, read *Current state* immediately below — the versions, the appservice +> owner, and the bridge have all moved since. + +--- + +## Current state — 2026-09-01 + +| | | +|---|---| +| Synapse | **v1.159.0** (upgraded from v1.120.0, 2026-09-01 — 39 releases, 21 months) | +| Element Web | **v1.12.27** (from v1.11.80) | +| PostgreSQL | **16-alpine** — deliberately NOT upgraded alongside Synapse | +| Appservice | registration `id: aipa-bridge`, but the service behind it is **`wt-matrix-bridge`**, the Worldtree PERSONAL instance on corviduo-dev `10.250.50.152:8010`. The `aipa` name is legacy. | +| Accounts | 15 — 13 appservice-owned, plus `@vhoang` (admin) and `@miranda` | +| Encryption | **zero encrypted rooms**, server-wide | +| Federation | `federation_domain_whitelist: []` — present but EMPTY, so federation is effectively OFF. The federation endpoint still answers, which is misleading. | + +**Ops ownership:** worldtree-dev writes the bridge code; **infra-ops operates this +instance** and has full authority to change it (operator, 2026-09-01). Do not route +provisioning or config requests to worldtree-dev. + +### Upgrade procedure (learned 2026-09-01) + +**Synapse schema migrations are one-way.** v1.120 cannot start against a v1.159 +database. Rollback is restore-from-dump, not revert-the-tag. Therefore: + +```bash +D=/opt/docker/backups/synapse-preupgrade-$(date -u +%Y%m%dT%H%M%SZ) +docker exec synapse-db pg_dump -U synapse -d synapse --format=custom --compress=6 > $D/synapse.dump +docker exec -i synapse-db pg_restore --list < $D/synapse.dump | wc -l # verify it reads back +``` + +Snapshot `compose.yaml`, `homeserver.yaml`, `aipa_appservice.yaml` and +`element-config.json` alongside it. Then bump tags and `up -d`; watch for +`Schema now up to date` in the logs. + +A benign startup warning — `Failed to listen on 0.0.0.0 ... Address already in use` +— is normal Linux dual-stack behaviour after it binds `[::]:8008`. Not a fault. + +### Appservice namespace — why `exclusive` is false + +The `aipa-bridge` registration claims a user namespace of +`@[a-z][a-z0-9_-]*:matrix.phasefinal.com` — effectively every localpart on the +server. It was `exclusive: true` until 2026-09-01. + +**`exclusive` governs who ELSE may act, not what the appservice may do.** On a +homeserver with registration disabled, one admin, and no competing actor, it bought +anti-squatting protection against a threat that cannot occur, while locking out +every other means of account creation — admin shared-secret registration returned +`M_EXCLUSIVE` for any conventional name, with no explanation of why. + +Now `exclusive: false`, regex unchanged. The bridge is unaffected: it creates and +puppets exactly as before. The narrow **aliases** namespace (`#aipa-debug-*`) is +left exclusive — it is specific and costs nothing. + +⚠ **Do not narrow the users regex to a prefix.** All 13 appservice accounts fall +inside it; narrowing would orphan them. ⚠ **Do not rename the `id`** — Synapse keys +account ownership on `aipa-bridge` in the `users` table. The FILE may be renamed. + +### Shared-secret registration — the HMAC has no trailing null + +`POST /_synapse/admin/v1/register`. The MAC is +`HMAC-SHA1(secret, nonce \0 user \0 password \0 "notadmin")` — the null **separates**, +it does not **terminate**. A trailing `\x00` after `notadmin` yields `HMAC incorrect`. +Run it inside the container against `localhost:8008`; port 8008 is not published to +the host. + +--- + +## Conventions + +### User IDs + +`@:matrix.phasefinal.com` — one account per agent or persona. + +### Room aliases — ratified by the operator 2026-09-01 + +``` +#-:matrix.phasefinal.com +``` + +Proposed by ledger-dev, ratified by the operator. Mirrors the user-ID convention +rather than introducing a second, independent axis. + +- **Ownership is legible from the alias alone** — `#miranda-summons` is + unambiguously Miranda's without opening the room. +- **Namespaced on the AGENT, not a product**, so it survives a consuming project's + rename. +- **`#aipa-debug-*` is avoided by construction** — no agent is named `aipa-debug`, + and that range is the bridge's exclusive alias namespace. +- **Load-bearing rationale, not tidiness:** purpose-as-tier is what makes a summons + triageable **so the room IDENTITY carries the tier**. (Stated carefully: it is + *not* that the push payload carries the room name — see the pusher note below.) + +**Known cost, accepted deliberately:** a genuinely shared room with no single owner +has no natural slot. Preferred over pre-building a namespace for a room that may +never exist; name that case deliberately if it arrives. + +**Pre-existing rooms are not renamed.** "The High Seat" +(`!NiVVoMsyoHCBRPrrrn`) predates the convention, has no canonical alias, and works. +Renaming a live proven channel to satisfy a scheme it did not exist under is the +churn the convention exists to prevent. New rooms follow it. + +--- + +## Push notifications — what actually reaches a phone + +Established 2026-09-01 while proving Ledger's summons channel. + +The registered pusher (`@vhoang`, Element X iOS) uses +**`"format": "event_id_only"`** via `https://matrix.org/_matrix/push/v1/notify`. +That payload carries `event_id`, `room_id` and counts — **no room name, no sender, +no content.** + +It still produces a useful notification because `mutable-content: 1` means Element X +runs a **Notification Service Extension**: iOS wakes it with the near-empty payload, +and the extension then **fetches the event from the homeserver and renders the +notification on the device.** + +Three consequences: + +1. **The tier-in-room-identity scheme works, but via the client fetch** — not + because the push carried the name. `m.room.name` must still be set at creation. +2. **`push: include_content: false` is irrelevant for clients with an NSE.** The + extension reads the event directly. It *does* bite clients without one. +3. **Server-invisible failure mode:** if the phone cannot reach + `matrix.phasefinal.com` at wake time, the fetch fails and iOS shows the fallback + — the bare word "Notification". **Synapse records `last_success` and sees a + delivered push.** The degradation happens entirely on the device. + +Because the payload carries nothing meaningful, **self-hosted sygnal was considered +and rejected** — it would remove matrix.org from a path that already carries no +content, room name or sender. + +⚠ Sygnal is a **relay** to FCM/APNs, not a replacement for them. Self-hosting it +removes matrix.org and nothing else. "Notifications must not leave the fleet" is not +satisfiable with mobile push at any price. + +--- + +## Sign in with QR code — requires Matrix Authentication Service + +Asked 2026-09-01; **not enabled.** From Synapse 1.159.0's own config validator: + +```python +if (self.msc4108_enabled or self.msc4108_delegation_endpoint is not None) \ + and not auth_delegated: + raise ConfigError("MSC4108 requires matrix_authentication_service to be enabled") +``` + +QR sign-in is **MSC4108**, an OIDC flow that hard-requires **MAS**. Synapse refuses +to start with the flag set and no MAS. + +**MSC4388** (`msc4388_mode: off|open|authenticated`) enables independently, but it is +only the **rendezvous channel** — the transport, not a login flow. + +Deferred because MAS is a service, a database, and a migration of every account off +built-in auth — and the v1.139.0 upgrade note warns that `/register` from **old +application service implementations may break under MAS**, which is precisely the +bridge owning 13 of the 15 accounts. Revisit when MAS becomes mandatory rather than +optional, in its own change window, with the bridge regression-tested either side. + --- ## Architecture