vh
b6340519bc
Session captured for a context reset. Six new detail entries. THE OPEN ITEM: Anaheim's IPsec tunnel delivers ~550 Mbit/s aggregate against a circuit measured at 2,153 Mbit/s. Not WireGuard (it is IPsec on ana-gw), not CPU (idle), not crypto exhaustion (NPU-offloaded), not the fibre. Both tunnels negotiate aes256-sha1; AES-GCM proposed. Operator signalled authorization; execution pending, untracked by operator choice. Also recorded: selene retired after losing a head-to-head on its own job with chat-judge moved to gen and the model name left to 404; the 7-alias collision on the gen seat that makes cross-alias corroboration an echo; hrafn adopted and its CI found to have been reporting green while deploying nothing for its whole life; all three Worldtree instances de-armed from a 69-day-stale :latest and the Matrix homeserver re-plumbed to personal; every secret-bearing .env on ana-docker tightened to 0600; the pfi org closing the repo-creation half of the credential-migration directive; booth kept-board deletion and link pruning. Two entries under Tried and abandoned: the CI checkout assertion that broke the pipeline twice and was removed, and my proposal to alias a retired model name at a different model, which the operator correctly overruled. Index 271 -> 282 lines, under the 300 cap, so no archival fired.
71 lines
3.9 KiB
Markdown
71 lines
3.9 KiB
Markdown
# [2026-08-23] Worldtree b187 shipped; all three instances de-armed from a 69-day-stale `:latest`; Matrix homeserver re-plumbed
|
|
|
|
## b187 pre-stage (#405 phases 1+2)
|
|
|
|
The matrix bridge stopped embedding the engine and became an HTTP client of the
|
|
Conversation API, so `WORLDTREE_API_URL` became **boot-blocking** — absent from the
|
|
container env, the bridge exits by design. Demo's compose never passed it; the next
|
|
recreate would have crash-looped. Pre-staged on demo and personal (additive, backed
|
|
up, verified with `docker compose config`, nothing restarted).
|
|
|
|
**Key decision, and I got its scope wrong first.** I argued demo should stay keyless
|
|
(no homeserver -> no rooms -> no turns -> no 401s). Right about turns, **wrong about
|
|
scope**: the engine preflight authenticates at boot regardless of homeserver, so demo
|
|
booted permanently degraded. Corrected — key `341c1488` minted under worldtree-dev's
|
|
recorded authorization, vaulted, wired, three-hop hash-verified.
|
|
|
|
## The 69-day-stale `:latest` landmine
|
|
|
|
All three instances pinned `WORLDTREE_IMAGE=.../worldtree:latest` in `.env` while
|
|
running SHA-tagged images built that day. Local `:latest` = `b19afd71d7cc`, built
|
|
**2026-06-14**. So ANY `docker compose up` — anyone's, for any reason — silently
|
|
downgraded that service by 69 days. Same footgun as the 2026-06-15 outage.
|
|
|
|
Re-pinned all three to their running SHAs (Worldtree #410), verified by rendering
|
|
compose config rather than reading `.env`, containers untouched. Playbook at
|
|
`playbooks/repin-worldtree-image.yaml`.
|
|
|
|
**`worldtree-pinned` was the worst case:** the instance whose entire purpose is being
|
|
frozen was running a **dangling image with no repo tags**, kept alive only by the
|
|
running container. One `docker rm` from garbage collection. Tagged
|
|
`:446e5807bf43` first, then pinned.
|
|
|
|
The guard I wrote had two bugs the pinned case exposed: it compared the container's
|
|
`.Config.Image` **string** (only the tag it was CREATED from — pinned was created
|
|
from `:latest` back when that meant 446e5807), and it reported CHANGED
|
|
unconditionally. Now compares **image IDs** and skips when already correct.
|
|
|
|
## Matrix homeserver ownership
|
|
|
|
Operator ruled: **personal owns the Matrix bridge.** The appservice tokens were never
|
|
missing — both sat at length 64 in the vaulted dev `env.sh` while both deployed
|
|
instances had them at length **zero**. Someone wired four of six Matrix vars and
|
|
stopped. Wired them into personal, three-hop verified.
|
|
|
|
**The trap worth remembering:** Synapse's registration pointed at
|
|
`http://10.100.10.50:8009` — nh3-dev, a dead epoch, with transaction 2801 queued at
|
|
512s backoff. The natural fix (swap the IP) gives `10.250.50.152:8009` which is
|
|
**DEMO's** bridge, and Synapse can reach both — it would have connected, delivered,
|
|
and looked correct while routing the operator's live rooms to the demo instance.
|
|
**Personal's bridge is :8010.** `docker port` is ground truth.
|
|
|
|
Corrected the URL, restarted Synapse (healthy in 32s after 3.5 months up), verified
|
|
`GET /_matrix/app/v1/ping -> 200` from inside the Synapse container. worldtree-dev's
|
|
smoke passed first try: room created, mimir accepted the invite, a real engine turn
|
|
ran, mimir replied in persona voice. #408 closed.
|
|
|
|
## Open on worldtree-dev's side
|
|
|
|
- **#411** — personal's bridge logs `Debug sink init failed: Permission denied:
|
|
/app/sessions/debug_rooms.json`. It creates two debug rooms but cannot persist
|
|
their IDs, so **every restart mints a fresh pair on the live homeserver**. Room
|
|
litter that compounds silently. Needs a which-container-writes-what check on the
|
|
sessions volume before anyone chowns it.
|
|
- Bridge/engine agent-roster drift: 6 of the bridge's 9 configured agents are not
|
|
listed by the engine on either instance.
|
|
- Historical Domari pairwise verdicts from the selene era are coin-flip-grade
|
|
(see [[2026-08-23-selene-retired-alias-collision]]); worldtree-dev banked that so
|
|
no future arc leans on them without re-judging.
|
|
|
|
Commits `064181a`, `bb19a96`.
|