Bifrost-binding for the chat client: self-drive + log affect/memory ops #17

Closed
opened 2026-06-15 23:08:16 -07:00 by vh · 1 comment
Owner

Context

Ratatoskr now wears two hats: the conversation-API canary client (TUI/web/CLI that runs turns against Worldtree and observes the SSE flow) and, since 2026-06-14, a Bifrost Tier-3 provider — the durable affect store (shipped v0.17.2, live-proven) and memory store (shipped v0.17.3, #195-parity green).

But the canary client can't drive its own provider. create_session sends only {agent_id, end_user_id} (src/ratatoskr/sessions.py:196) — it never sets the session's Bifrost endpoint_url. Supplying Bifrost endpoints from the client was an explicit negative clause in the original design-brief (§6: "Bifrost-binding consumer support — not a Ratatoskr concern"). That decision predates the provider identity — so ratatoskr now owns both ends of the Bifrost round-trip but can't connect them itself. The affect + memory smokes both had to be driven externally (infra-ops/worldtree-dev crafting the binding).

This issue closes that gap: §6 is stale against the provider identity; reverse it.

Goal

Make ratatoskr a complete, self-contained chat client that drives AND observes the full Bifrost round-trip against its own provider — for troubleshooting latent bugs across the affect/memory planes. Two parts:

  1. Bind — the client sets the session-creation bifrost field so a turn dispatches affect/memory to our provider (:8390 affect, :8391 memory).
  2. Observe + log — surface every affect/memory operation (handshake / upsert_many / search / emit / get / delete) in the debug surface + logs, correlated with the turn that triggered it. Because ratatoskr owns both ends, it can show exactly which memory writes/recalls a given turn produced — the lens for latent-bug hunting.

Feasibility anchor

The conversation API already supports client-set binding: POST /sessions takes an optional bifrost field (issue #160 "MCP-in-reverse"); Worldtree runs the handshake synchronously before 201 (502 on failure, no session created). So the binding is settable from the client in principle — no Worldtree change needed to bind.

Open questions (resolve before/during contract)

The authoritative recipe is in flight via infra-ops ↔ worldtree-dev (althing thread 01KV7D82MJYB…, the memory-smoke wiring) — it doubles as this feature's feasibility answer. Loop the result in here.

  1. Plane coverage — does the session-creation bifrost field bind affect/memory dispatch, or only #160 tools? affect/memory are separate planes (#204 / #294).
  2. Memory trigger — memory-call fires from Tier-3 promotion, not per-turn appraisal (per infra-ops). Personal defaults to memory.agent_self_enabled: false; binding the endpoint may not be enough — may need a memory-enabled agent or session-level promotion-on-bind.
  3. Transport — spec requires endpoint_url be HTTPS; our provider is HTTP. Dev-mode relaxation?
  4. Auth#160 registered consumers sign RS256; our provider uses HS256 (the heimdall key). Which for the bound planes?
  5. Observation channel — how does the chat client see the provider-side ops? The provider is a separate process. Options: a provider-side op feed → UI; Worldtree SSE events (affect_update exists; memory equivalent?); admin Bifrost inspection (GET /admin/sessions/{id}/bifrost). Self-loop correlation is the most powerful.

Dependencies / risk

  • worldtree-dev Tier-3 promotion + binding recipe (in flight).
  • Possible Worldtree-side change if affect/memory don't ride the #160 field, or if HTTP/HS256 dev-binding isn't supported.

Out of scope / supersedes

  • Supersedes design-brief §6 "no Bifrost-binding consumer support" — this issue is the deliberate reversal, justified by the provider identity (2026-06-14).
  • Production hardening of the bound-provider path (TLS, RS256 issuance, etc.) — dev-loop first.

Pointers

  • Client session creation: src/ratatoskr/sessions.py (create_session)
  • Provider (the other end): src/ratatoskr/provider/{affect_store,memory_store,serve,serve_memory}.py
  • Bifrost binding in the pinned spec: docs/conversation-api-spec.md §"Optional Bifrost binding (issue #160)" (~L977), GET /admin/sessions/{id}/bifrost (~L685)
  • Smoke-wiring thread (recipe source): infra-ops althing 01KV7D82MJYB…
## Context Ratatoskr now wears two hats: the conversation-API **canary client** (TUI/web/CLI that runs turns against Worldtree and observes the SSE flow) and, since 2026-06-14, a **Bifrost Tier-3 provider** — the durable affect store (shipped v0.17.2, live-proven) and memory store (shipped v0.17.3, #195-parity green). But the canary client can't drive its own provider. `create_session` sends only `{agent_id, end_user_id}` (`src/ratatoskr/sessions.py:196`) — it never sets the session's Bifrost `endpoint_url`. Supplying Bifrost endpoints from the client was an explicit negative clause in the original design-brief (§6: "Bifrost-binding consumer support — not a Ratatoskr concern"). That decision **predates the provider identity** — so ratatoskr now owns both ends of the Bifrost round-trip but can't connect them itself. The affect + memory smokes both had to be driven externally (infra-ops/worldtree-dev crafting the binding). This issue closes that gap: **§6 is stale against the provider identity; reverse it.** ## Goal Make ratatoskr a complete, self-contained chat client that drives AND observes the full Bifrost round-trip against its own provider — for troubleshooting latent bugs across the affect/memory planes. Two parts: 1. **Bind** — the client sets the session-creation `bifrost` field so a turn dispatches affect/memory to our provider (`:8390` affect, `:8391` memory). 2. **Observe + log** — surface every affect/memory operation (`handshake` / `upsert_many` / `search` / `emit` / `get` / `delete`) in the debug surface + logs, **correlated with the turn that triggered it**. Because ratatoskr owns both ends, it can show exactly which memory writes/recalls a given turn produced — the lens for latent-bug hunting. ## Feasibility anchor The conversation API already supports client-set binding: `POST /sessions` takes an optional `bifrost` field (issue #160 "MCP-in-reverse"); Worldtree runs the handshake synchronously before 201 (502 on failure, no session created). So the binding is settable from the client in principle — no Worldtree change needed to *bind*. ## Open questions (resolve before/during contract) The authoritative recipe is in flight via infra-ops ↔ worldtree-dev (althing thread `01KV7D82MJYB…`, the memory-smoke wiring) — it doubles as this feature's feasibility answer. Loop the result in here. 1. **Plane coverage** — does the session-creation `bifrost` field bind affect/memory *dispatch*, or only #160 *tools*? affect/memory are separate planes (#204 / #294). 2. **Memory trigger** — memory-call fires from **Tier-3 promotion**, not per-turn appraisal (per infra-ops). Personal defaults to `memory.agent_self_enabled: false`; binding the endpoint may not be enough — may need a memory-enabled agent or session-level promotion-on-bind. 3. **Transport** — spec requires `endpoint_url` be HTTPS; our provider is HTTP. Dev-mode relaxation? 4. **Auth** — #160 registered consumers sign RS256; our provider uses HS256 (the heimdall key). Which for the bound planes? 5. **Observation channel** — how does the chat client see the provider-side ops? The provider is a separate process. Options: a provider-side op feed → UI; Worldtree SSE events (`affect_update` exists; memory equivalent?); admin Bifrost inspection (`GET /admin/sessions/{id}/bifrost`). Self-loop correlation is the most powerful. ## Dependencies / risk - worldtree-dev Tier-3 promotion + binding recipe (in flight). - Possible Worldtree-side change if affect/memory don't ride the #160 field, or if HTTP/HS256 dev-binding isn't supported. ## Out of scope / supersedes - **Supersedes design-brief §6** "no Bifrost-binding consumer support" — this issue is the deliberate reversal, justified by the provider identity (2026-06-14). - Production hardening of the bound-provider path (TLS, RS256 issuance, etc.) — dev-loop first. ## Pointers - Client session creation: `src/ratatoskr/sessions.py` (`create_session`) - Provider (the other end): `src/ratatoskr/provider/{affect_store,memory_store,serve,serve_memory}.py` - Bifrost binding in the pinned spec: `docs/conversation-api-spec.md` §"Optional Bifrost binding (issue #160)" (~L977), `GET /admin/sessions/{id}/bifrost` (~L685) - Smoke-wiring thread (recipe source): infra-ops althing `01KV7D82MJYB…`
vh added the tuienhancementobservability labels 2026-06-15 23:08:16 -07:00
Author
Owner

Closing — shipped end-to-end and now empirically proven from a real Worldtree-driven turn.

Shipped: self-drive Bifrost binding + dispatch-layer op-feed of affect/memory ops, slices 1–3c (v0.17.8v0.17.13) plus the op-feed handshake field-name fix (v0.17.17, d60b77d). The composite both-plane extension (one session binds BOTH planes) landed under #18 (v0.18.0).

Final live proof (2026-06-20, personal Worldtree v0.36.0 → composite provider :8392): one bound session (b83a66b6) dispatched the full both-plane lifecycle through one endpoint, every op correlated to the bound session_id by the op-feed:

  • handshakecaps_granted=[memory, affect]
  • affect.fetch + memory.search (reads)
  • affect.emitstored:true — PAD persisted to affect_snapshots
  • memory.upsert_manyupserted:1 — promotion chunk persisted to memory_chunks

Both writes verified directly in our SQLite stores, not just dispatch-ok. This is the thesis realized: ratatoskr, owning both ends of the round-trip, sees exactly which memory/affect ops a turn produced — from the provider side, correlated to the turn that triggered them.

Closing — shipped end-to-end and now empirically proven from a real Worldtree-driven turn. **Shipped:** self-drive Bifrost binding + dispatch-layer op-feed of affect/memory ops, slices 1–3c (`v0.17.8`–`v0.17.13`) plus the op-feed handshake field-name fix (`v0.17.17`, `d60b77d`). The composite both-plane extension (one session binds BOTH planes) landed under #18 (`v0.18.0`). **Final live proof (2026-06-20, personal Worldtree v0.36.0 → composite provider `:8392`):** one bound session (`b83a66b6`) dispatched the full both-plane lifecycle through one endpoint, every op correlated to the bound `session_id` by the op-feed: - `handshake` → `caps_granted=[memory, affect]` - `affect.fetch` + `memory.search` (reads) - `affect.emit` → `stored:true` — PAD persisted to `affect_snapshots` - `memory.upsert_many` → `upserted:1` — promotion chunk persisted to `memory_chunks` Both writes verified directly in our SQLite stores, not just dispatch-ok. This is the thesis realized: ratatoskr, owning both ends of the round-trip, sees exactly which memory/affect ops a turn produced — from the provider side, correlated to the turn that triggered them.
vh closed this issue 2026-06-20 12:56:51 -07:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: vh/ratatoskr#17