ratatoskr/docs/bifrost-self-test.md

Bifrost round-trip self-test

How to drive and observe a full Tier-3 Bifrost round-trip against ratatoskr's own provider — bind a Worldtree session to our affect/memory store, fire a turn, and read exactly what Worldtree dispatched to us, correlated with the turn that triggered it.

This is the manual form of issue #17 (self-drive + correlated-log affect/memory ops). Until #17 ships that capability inside the TUI/web/CLI, this runbook is the reproducible loop — and it is the substrate worldtree-dev's #296 (salience-algorithm research) and #297 (recall scope/assembly research) diagnose against.

First proven by hand 2026-06-16 while isolating #295's recall leg.

The load-bearing tripwire: which key signs the bind

A bound session-create authenticates as the Bifrost consumer, not the canary client. Worldtree signs the Bifrost handshake JWT with the session-create bearer token (HS256 shared-secret model); our provider verifies it against RATATOSKR_HEIMDALL_KEY.

So the bearer on POST /sessions must be the consumer Heimdall key (RATATOSKR_HEIMDALL_KEY, in ~/.config/ratatoskr/provider.env) — not WORLDTREE_API_KEY (the mimir/foundational TUI key in env.sh). They are two different keys for two identities of ratatoskr.

Bearer used on POST /sessions	Handshake at our provider
WORLDTREE_API_KEY (mimir/TUI key)	401 bifrost.auth_rejected → 502 to client
RATATOSKR_HEIMDALL_KEY (consumer key)	200 OK → session bound

ratatoskr is two identities: the conversation-API canary client (WORLDTREE_API_KEY) and the Bifrost consumer/provider (RATATOSKR_HEIMDALL_KEY). Self-driving a bound session crosses into the consumer identity, so it uses the consumer key. #17's Bind half has to carry this distinction.

Prereqs

• Provider(s) running on this box (nh3-dev, 10.100.10.50):
  • memory → ratatoskr-memory-provider on :8391
  • affect → ratatoskr-provider on :8390
  • Each is a dev background shell, env-sourced from provider.env. The memory provider's stdout carries the inbound observe log ([memory-provider] lines added in memory_store.py).
• The provider endpoint is reachable + allowlisted from Worldtree (10.250.50.152): http://10.100.10.50:8391. The allowlist (BIFROST_CLIENT_ALLOWED_HOSTS) is Worldtree-side, infra-ops-owned — if a bind 502s with a route/allowlist error, that entry is the thing to check.
• A memory-enabled Tier-3 agent defined on the instance. ratatoskr:smoke (scope end_user:smoke-user) is the standing fixture; it is hidden from GET /agents (Tier-3 agents are not in the public roster) but resolves on session-create.

Steps

cd ~/development/ratatoskr
set -a && . ~/.config/ratatoskr/provider.env && set +a   # RATATOSKR_HEIMDALL_KEY etc.
URL=http://10.250.50.152:8081                              # personal Worldtree
HK="$RATATOSKR_HEIMDALL_KEY"                               # the CONSUMER key — the tripwire

1 — Bind a fresh (cold) session to our provider. bifrost.endpoint_url points at the plane you want (:8391 memory, :8390 affect); caps are negotiated by the handshake, not declared here (BifrostBindingRequest is {endpoint_url, scope} only, additionalProperties:false). A 201 means the handshake verified.

curl -sS -X POST "$URL/sessions" -H "Authorization: Bearer $HK" \
  -H "Content-Type: application/json" \
  -d '{"agent_id":"ratatoskr:smoke","end_user_id":"smoke-user",
       "bifrost":{"endpoint_url":"http://10.100.10.50:8391","scope":null}}'
# -> 201 {"session_id":"...", "kind":"consumer_defined", ...}

2 — Snapshot the fixture (to detect any promotion the turn writes):

sqlite3 -readonly memory.db \
  "SELECT chunk_id, json_extract(record_json,'\$.verbatim.text') FROM memory_chunks;"

3 — Fire ONE turn into the bound session, reusing ratatoskr's own SSE client (handles composite ids + no-read-timeout). Bearer = the consumer key:

RATATOSKR_HEIMDALL_KEY="$HK" uv run python - <<'PY'
import asyncio, os, httpx
from ratatoskr.sse_client import stream_turn, Text, Done, Error, Cancelled
SESSION="<session_id from step 1>"
URL="http://10.250.50.152:8081"; KEY=os.environ["RATATOSKR_HEIMDALL_KEY"]
async def main():
    async with httpx.AsyncClient(base_url=URL,
        headers={"Authorization":f"Bearer {KEY}","User-Agent":"ratatoskr-selftest"},
        timeout=httpx.Timeout(connect=10.0,read=None,write=10.0,pool=10.0)) as c:
        async for ev in stream_turn(c, SESSION, "What kind of chocolate do I like?"):
            if isinstance(ev, Done):  print("ANSWER:", ev.response); return
            if isinstance(ev, Error): print("ERROR:", ev.error_code, ev.message); return
asyncio.run(asyncio.wait_for(main(), 150))
PY

4 — Read the inbound pair from the provider's stdout (the observe brick). For a background-shell provider, that is the task output file; tail it:

[memory-provider] memory-call search REQUEST: scope_all={...} scope_any=[...] top_k=... vec_dim=1024
[memory-provider] memory-call search RESPONSE: N hit(s) [{'chunk_id':..., 'score':..., 'scope':...}]

5 — Re-snapshot the fixture (step 2's query). A new row = the turn was promoted (a salience-algorithm event; relevant to #296). Audit, don't blindly delete — promoted failure-surfaces may be wanted corpus.

Reading the result

The search REQUEST scope_all/scope_any vs the search RESPONSE hit count is the whole diagnosis surface:

• 0 hits → the composed v0.6 filter matched no stored chunk. scope_all axes are AND-matched — an extra axis the chunks don't carry (e.g. agent_self) zeroes the result even when end_user matches. scope_any is the OR/union escape hatch: a subset-scoped chunk recalls if its scope ⊇ any one element. So 0 hits with a populated store now means Worldtree sent an over-specified scope_all instead of a scope_any union — a scope-build question (Worldtree-side, post-v0.6).
• Hit present but the model says "no memory" → we returned it; Worldtree dropped it downstream of search → recall-assembly / injection (Worldtree-side).

Either way our store + search are provable from this surface; the recall efficacy must be judged at the model's answer in a cold (history-free) session, never from a wire 200 (a search returns 200 whether or not its hits are injected).

Worked example (2026-06-16, #295 → #297)

Cold turn "What kind of chocolate do I like?" against the smoke-user fixture:

REQUEST:  scope_filter={'end_user': 'smoke-user', 'agent_self': 'ratatoskr:smoke'} top_k=128
RESPONSE: 0 hit(s)
ANSWER:   "I don't have access to your past preferences..."

(That capture is the pre-v0.6 wire — a single AND-only scope_filter.)

Root cause: the recall filter carried agent_self but the stored chunks are {end_user: smoke-user} only → the agent_self axis excluded all of them. Branch (a), scope asymmetry — fed to #297.

Resolution (bifrost 0.8.0 / wire v0.6, #11): the single scope_filter is split into scope_all (AND) + scope_any (OR/union). Worldtree can now send the visible scopes as a scope_any union (e.g. [{end_user: smoke-user}, {end_user: smoke-user, agent_self: ...}]), so the subset-scoped chunk recalls via the matching OR member. Our store implements this at parity with the v0.6 reference; Worldtree adopted the v0.6 wire and now emits scope_any on the recall path (#297 client-side union recall

• #298/#299 bifrost-v0.6 adoption, v0.35.16), so cold cross-session recall is closed end-to-end — pending a live re-smoke against a personal instance running v0.35.16.

Notes / foot-guns

• HTTP, not HTTPS. The spec requires endpoint_url be HTTPS; dev is relaxed via the Worldtree-side allowlist. Don't "fix" our provider to HTTPS to make a bind work — check the allowlist entry first.
• Cold means cold. Reuse of a session with history can satisfy a "recall" from plain conversation history. Always bind a fresh session for a recall probe.
• Stray sessions created by probes are harmless empty rows on the dev instance; no cleanup required.
• This loop is the thing #17 productizes into the chat surfaces; when #17 lands, the bind+observe steps move inside the TUI/web/CLI and this runbook becomes the underlying contract check.