docs(#17): self-drive+observe contract, bifrost self-test runbook + snapshot

- docs/contracts/issues/17.contract.md — issue-scoped v2.1 contract for #17
  (Bifrost-binding the chat client). v1 scope = single-plane bind +
  dispatch-layer op-feed (composite endpoint + turn-pane UI parked). Design
  consulted via /heid, paraphrase-gated via /heid-contract-review panel; two
  internal inconsistencies fixed (OpEvent turn_id reservation made literal;
  session_id-for-all-verbs correction). Validates OK, prd drift-clean.
- docs/bifrost-self-test.md — reusable runbook for driving + observing the
  full Bifrost round-trip against our own provider (the manual form of #17;
  pins the consumer-key-as-bearer tripwire).
- persistent-memory.md — snapshot: observe brick shipped, self-drive proven,
  #295 root-caused (upstream, scope-axis asymmetry) -> #296/#297, agent_self
  -> canonical decided.

persistent-memory.md

@@ -1,6 +1,6 @@
 # Persistent memory — ratatoskr
 
-_Last updated: 2026-06-15_
+_Last updated: 2026-06-16_
 
 This file captures durable intent and supporting evidence (goals, decisions,
 foot-gun warnings, in-flight state) across context resets. Read it at session
@@ -41,7 +41,7 @@ model output is untrusted); upstream API key stays server-side (INV-003).
 
 ## Current state / in-flight
 
-_As of 2026-06-15:_
+_As of 2026-06-16:_
 
 **Ratatoskr now has a SECOND identity: the v1 Bifrost Tier-3 consumer** — the
 durable persistence provider Worldtree writes Tier-3 agent affect/persona +
@@ -56,21 +56,43 @@ background shell (`ratatoskr-provider`, `0.0.0.0:8390`, env-sourced from
 Worldtree **v0.35.2** (`10.250.50.152`): handshake 200 + `affect.emit` 200 →
 durable row persisted, opacity held.
 
-**MEMORY plane: PROVIDER LIVE-PROVEN** (v0.17.3, commit `cd12951`; provider
-running on `0.0.0.0:8391` as a dev bg shell). Store + dev shell shipped (7
-contract blocks, 26+4 tests, #195 parity, heid-reviewed zero-drift) AND the
-live round-trip ran against personal **v0.35.3**: handshake + `upsert_many`
-(persist) + `search` (recall-dispatch) all green; `memory.db` holds 3 durable
-chunks (2 facts + 1 promoted question, scope `end_user:smoke-user`, 1024-d
-vec-indexed). **Proven: persist + dispatch + our search-correctness** — direct
-probe (embed the recall query via gateway `qwen3-embedding`, search our live
-store) recalls the right fact at cosine 0.60, correctly ranked. **NOT proven,
-and upstream (not ours): end-to-end recall-INJECTION** — Worldtree's
-recall-assembly doesn't surface our hits into the prompt (cold-session probe:
-search dispatched 200 but the model said "no memory"). worldtree-dev looped in
-(thread `01KV7JH8…`; #295 held open). Provider stays up with the live fixture.
-So NO minor bump / "all-green" announce until worldtree-dev closes the injection
-+ question-promotion gaps.
+**MEMORY plane: PROVIDER LIVE-PROVEN + recall-miss ROOT-CAUSED (upstream).**
+Store + dev shell shipped (v0.17.3, `cd12951`; running on `0.0.0.0:8391`).
+`memory.db` holds 3 durable chunks (choc-fact `498ed752` sal 0.9, name `8241e569`
+sal 0.8, promoted-question `c863bb6b` sal 0.9 — all scope `{end_user:smoke-user}`).
+**The #295 cold-recall miss is now ROOT-CAUSED and it's UPSTREAM, not ours**
+(2026-06-16 debug-assist with worldtree-dev): a self-driven bound cold-recall
+probe captured the inbound pair via our new observe log — Worldtree's recall sends
+`scope_filter={end_user:smoke-user, agent_self:ratatoskr:smoke}` (TWO axes) but our
+chunks carry `{end_user}` ONLY; our AND `_scope_matches` (byte-faithful to bifrost
+reference `reference_server/memory.py:398`) drops everything on the unmatched
+`agent_self` axis → 0 hits → the model says "no memory". So **our store + search
+are SOUND**; the fix is Worldtree-side. F2 (question-promotion) → research issue
+**#296** (keyword-regex salience suspected fundamentally flawed; `c863bb6b` is a
+corpus fixture, KEEP PINNED). F1 (recall-miss) → research issue **#297**
+(Worldtree-local fix = per-visible-scope single-axis search unioned client-side;
+HELD until the agent_self lattice question lands — see Recent decisions).
+
+**OBSERVE BRICK SHIPPED** (`memory_store.py`, uncommitted at snapshot time):
+structured `[memory-provider]` request/response logging on the memory-call path —
+the first concrete brick of #17's observe half, and the lens that caught #295's
+root cause. Live-verified.
+
+**SELF-DRIVE PROVEN BY HAND** (2026-06-16): ratatoskr's own client drove a
+Bifrost-bound cold-recall end-to-end (bind → handshake 200 → turn → captured the
+recall pair). Load-bearing finding: a bound session-create must use the CONSUMER
+Heimdall key (`RATATOSKR_HEIMDALL_KEY`) as the bearer, NOT the canary
+`WORLDTREE_API_KEY` — Worldtree signs the Bifrost handshake JWT with the
+session-create bearer (canary → 401; consumer → 200). Runbook:
+`docs/bifrost-self-test.md`. This is #17's substrate, proven before the contract.
+
+**ISSUE #17 (self-drive + observe) — CONTRACT WRITTEN + HEID-REVIEWED + FIXED,
+TDD NEXT.** `docs/contracts/issues/17.contract.md` (validates OK, drift-clean).
+v1 scope operator-locked: single-plane bind (composite endpoint PARKED) +
+dispatch-layer op-feed with session-level correlation (turn-pane UI PARKED).
+`/heid-contract-review` panel caught + fixed two real internal inconsistencies
+(OpEvent `turn_id` reservation made literal; the `session_id`-for-all-verbs
+correction). NEXT: TDD tracer-bullet (bind affect plane → turn → assert OpEvent).
 
 **Sindra:** a REGISTERED Tier-3 agent (`ratatoskr:sindra`, was model
 `artemis-31b-v1i`) — registration is REQUIRED to use a Tier-3 character (a
@@ -86,10 +108,10 @@ registration.
 `~/.config/ratatoskr/provider.env` (mode 600, nh3-dev) — `consumer="ratatoskr"`,
 HS256 = the API-key STRING utf-8-encoded; rotate via infra-ops.
 
-**Pushed:** `main` is in sync with `origin` (pushed through `e57b054`); tags
-`v0.17.3` + a straggler `v0.8.2` pushed. Only `graphify-out/GRAPH_REPORT.md`
-runs dirty (auto-regenerated by the commit hook — generated artifact, not
-chased).
+**Committed (2026-06-16, NOT yet pushed):** the observe brick (logger, patch
+bump), `docs/bifrost-self-test.md`, the #17 contract, + this snapshot. Push is
+the operator's call. `graphify-out/GRAPH_REPORT.md` still runs dirty
+(auto-regenerated artifact, not chased).
 
 **Still standing from before:** Worldtree spec pin v0.29.0 (`562001a`) for the
 conversation-API/TUI surface (untouched by the Bifrost work). Codex-first pilot
@@ -151,6 +173,12 @@ decision. Captures rationale that won't be obvious from code alone.
 - `[2026-06-15]` **Providers run as dev-box BACKGROUND SHELLS, not infra-ops/systemd** (operator call — it's a dev box). `ratatoskr-provider` (affect) + a future `ratatoskr-memory-provider` as background processes; no productionization track.
 - `[2026-06-15]` **Affect plane shipped (v0.17.2) + LIVE-PROVEN end-to-end against real Worldtree v0.35.2.** Personal handshake 200 + `affect.emit` 200 from `10.250.50.152` → durable row persisted (opacity held). HS256 key = the consumer's Heimdall API-key STRING utf-8-encoded (NOT base64/raw — the tripwire); cross-subnet route + `BIFROST_CLIENT_ALLOWED_HOSTS` allowlist all held (infra-ops-owned). worldtree-dev confirmed ADR-0009 holding as designed.
 
+- `[2026-06-16]` **#295 cold-recall miss root-caused — UPSTREAM, branch (a) scope-axis asymmetry.** A self-driven bound cold-recall probe (our own client, consumer-key bearer) captured the inbound pair via the new observe log: Worldtree's recall filter carries `{end_user, agent_self}`; our chunks are `{end_user}`-only; AND-matching drops everything on `agent_self` → 0 hits. Our store + search are SOUND; the fix is Worldtree-side. F2 (question-promotion) → **#296** research; F1 (recall-miss) → **#297** research (worldtree-dev's Worldtree-local per-scope-union fix, HELD pending the lattice question).
+- `[2026-06-16]` **agent_self → make it CANONICAL (operator decided A).** The cross-repo "is agent_self a valid bifrost scope axis?" question: bifrost's reference lattice is `{end_user, group, tenant}` only (agent_self → `invalid_filter` 400); Worldtree emits agent_self (`bifrost_memory_store.py:479` #248 agent-self primitive). Operator chose canonical-not-re-expressed; worldtree-dev filed the lattice-addition with bifrost-dev (thread `01KV7PXF…`). **Implication: our store's permissive axis-acceptance becomes CORRECT once bifrost adds agent_self — so do NOT add axis-validation; our missing `_validate_scope_filter` is HELD, not a bug to fix.** #297 union build held until the axis lands.
+- `[2026-06-16]` **Self-drive auth identity: bound session-create uses the CONSUMER Heimdall key as bearer, NOT `WORLDTREE_API_KEY`.** Worldtree signs the Bifrost handshake JWT with the session-create bearer (canary key → handshake 401; consumer key → 200). Two keys, two identities. Proven by hand; documented in `docs/bifrost-self-test.md`; load-bearing for #17's Bind half.
+- `[2026-06-16]` **Issue #17 v1 scope locked (operator 1A/2A): single-plane bind + dispatch-layer op-feed.** `BifrostBindingRequest` is one `endpoint_url` (one plane per session); composite-both-planes endpoint PARKED. Observe = structured op-feed instrumented at the DISPATCH layer (bifrost passes ctx to upsert_many but NOT search/get/delete — `memory.py:244`), session-level correlation; turn-correlated pane UI PARKED (needs turn_id, TBD). Direct in-session TDD (live-smoke load-bearing). Contract `docs/contracts/issues/17.contract.md` written, `/heid`-design-consulted + `/heid-contract-review`-panel'd + fixed (validates OK). NEXT: TDD.
+- `[2026-06-16]` **Provider stores confirmed byte-faithful to bifrost's AND reference** (`reference_server/memory.py:398` `_matches_scope` = `all(...)`, identical to ours). OR-union was considered + rejected (ecosystem-wide change); flagged the silent-zero foot-gun to bifrost-dev (docs-only landed, bifrost stays 0.6.4). Do NOT flip `_scope_matches` to OR.
+
 _For per-issue TDD implementation notes, Volva findings, and contract amendments, see the git log — every per-issue commit carries a structured message capturing the trail._
 
 ## Tried and abandoned
@@ -186,3 +214,6 @@ defense against re-attempting the same cul-de-sac.
 - `[2026-06-15]` **Diagnostic: our recall-search is SOUND — the cross-session recall gap is UPSTREAM, not the store — and it caught an upstream bug.** Embedded the recall query via gateway `qwen3-embedding` + searched our live store directly → the dark-chocolate fact recalls at cosine 0.60, correctly ranked above the unrelated name fact (0.16). So the cold-session recall failure is Worldtree's recall-assembly/injection (hits not reaching the prompt), NOT our search. ALSO found a latent UPSTREAM bug: a recall QUESTION got promoted as a durable chunk and ranks **#1 (0.70 > the fact's 0.60)**, polluting recall. Relayed to worldtree-dev (thread `01KV7JH8…`). **This is exactly #17's thesis — ratatoskr-as-provider caught an upstream bug invisible from the chat side.**
 - `[2026-06-15]` **"Wire 200 ≠ recall works" — prove recall efficacy at the model's answer in a COLD (history-free) session, not on the wire.** A `search`/memory-call returns 200 whether or not its results are injected into the prompt, and same-session "recall" can be plain session history. infra-ops' cold cross-session probe caught my premature "all-green" (search dispatched 200, model had no memory). Don't call cross-session recall proven from a clean wire.
 - `[2026-06-15]` **Issue #17 filed — Bifrost-binding for the chat client (self-drive + correlated-log affect/memory ops).** REVERSES design-brief §6's "no Bifrost-binding consumer support" — that negative clause predates ratatoskr's provider identity (2026-06-14), so the canary now owns both ends but its client can't drive its own provider (`create_session` sends only `{agent_id, end_user_id}`; no Bifrost `endpoint_url`). Today's smoke proved the substrate (bind→dispatch→persist); only the observe/log channel design (open question #5) remains. The recall-injection caveat is upstream and doesn't block #17. NEXT on #17: `/heid` consult on the now-grounded framing → contract → TDD. (tracked: Gitea #17, labels enhancement/observability/tui)
+- `[2026-06-16]` **`scripts/contract_drift_check.py` defaults `GITEA_REPO` to "Worldtree"** (line 74), so a bare run in ratatoskr false-positives DRIFT by hashing Worldtree's same-numbered issue. Always `export GITEA_REPO=ratatoskr GITEA_OWNER=vh` (env.sh leaves the GITEA vars commented out) before running the drift-checker here.
+- `[2026-06-16]` **My #295 coupling hypothesis (the promoted question crowds out the fact at small top_k) was REFUTED** — worldtree-dev's recall over-fetches `top_k=128` (`injector.py:203`/`_store_helpers.py:101`), so the question can't crowd the fact out at search level. Reasonable cross-frontier hypothesis, correctly framed as a hypothesis not a conclusion; the real cause was the scope-axis asymmetry. Lesson: offer provider-side hypotheses, let the upstream owner check them against their code.
+- `[2026-06-16]` **Contract drifted from its own design in two spots, caught only by `/heid-contract-review` (not same-author paraphrase):** the `OpEvent` dataclass omitted the `turn_id` that INV-005 promised; the `session_id` comment said "None for search/get/delete" contradicting the dispatch-layer design (the JWT carries session_id for all verbs at dispatch). Cross-model paraphrase is load-bearing for catching an author's own contract-vs-intent drift.