Compare commits

...

7 Commits

Author SHA1 Message Date
vh e5ec63967e feat(web): memory viewer + design iteration-3 + markdown pass-2 (v0.20.7)
Web-UI iteration-3 — three queued items landed together.

(A) Design iteration-3 into static/index.html:
- sparkline grid background (<pattern id=sparkGrid> + bg rect behind each
  relation-row spark polyline);
- PAD strips → per-turn Δ bars: replace the vertical polyline strip
  (stripPoints/proj3 removed) with padDeltas→deltaStrip, a 12-cell HTML
  column of diverging bars (newest at bottom, magnitude→width, age→opacity);
- mood orbit → dimetric open box (viewBox 124×140, az35/el25,
  D-right/A-left-back/P-up): ghost A×P wall + D×A floor +
  orbitProj/orbitShadowY/orbitWallPt/orbitAxisPt, JS-driven animated replay
  (orbitFrame per rAF via a singleton startOrbitAnim reading live ORBIT_HIST;
  reduced-motion → static final-state).

(B) Memory viewer — a non-bifrost debug read on OUR own store, mirroring the
    #18-D2 affect read:
- provider: RatatoskrMemoryStore.list_chunks + count_chunks + shared
  add_memory_read_route (GET /memory/chunks?agent_id=&end_user_id=), wired
  into build_memory_provider_app + the combined :8392 provider. end_user
  strict, agent_id lenient (an {end_user}-only chunk stays visible);
  {chunks,count,total}, empty match = 200 (not 404);
- web: _memory_chunks_endpoint (GET /api/memory/chunks) supplies end_user_id
  server-side, forwards the browser agent_id, proxies to memory_read_url;
  create_app gains memory_read_url, entrypoint reads RATATOSKR_MEMORY_READ_URL;
- pane: loadMemory/renderMemory/setMemHead — a live-polling MEMORY console
  pane (content·scope·origin·revision per chunk; count/total distinguish
  empty-store from scope-mismatch), polled on open + post-turn.

(C) Markdown pass-2 in markdownSafe: GFM pipe tables (mdTable, alignment
    colons), indentation-nested lists (child list inside the open <li>),
    ordered-list start=N numbering, streaming robustness (partial fence →
    code block; header-without-delimiter → paragraph until it streams in).
    esc-first → INV-004 held.

Contract web_debug_surface.contract.md amended in-commit (create_app
memory_read_url; dimetric-orbit + Δ-bar renderConsole POST-002; memory-viewer
+ markdown-pass-2 function contracts). 631 tests green; Playwright-verified
all render paths (dark+light).
2026-07-07 02:11:29 -07:00
vh 24644ab90e memory: snapshot — queue web-UI iteration-3 (design re-import) + memory viewer + markdown pass-2 for fresh context 2026-07-07 01:28:04 -07:00
vh c988f273b1 feat(web): markdown RP semantic coloring + paragraph reflow (v0.20.6)
Markdown rework, pass 1 (operator-directed):

- "quoted" dialogue → speech (bright --md-speech), *asterisk* → action/narration
  (muted-italic --md-action). Two tunable CSS vars that cascade through --fg-* so
  they auto-adapt to the light theme; plain text stays default narration.
- speech-wrap runs BEFORE the em/link passes so a generated class="…"/href="…"
  quote can't be mis-read as dialogue; straight + smart quotes; apostrophes are
  safe; unbalanced/half-streamed quotes stay uncolored until they close.
- fix the ugliest existing bug: single newlines were hard <br>s → now CommonMark
  soft-breaks (space); a hard break needs 2+ trailing spaces or a trailing \.
- INV-004 escape-first preserved.

Verified: pytest tests/test_web_* (84) + a 10-case Playwright unit-check of
markdownSafe (speech / action / attribute-quote trap / apostrophes / mixed /
reflow / hard-break / html-escaping / unbalanced / paragraphs) + a visual render.
2026-07-07 01:03:45 -07:00
vh 7704959f48 feat(web): tooltips on the affect-derived cells (v0.20.5)
Native title hints on all 6 affect-derived cells (samples / updated / baseline
P·A / drift Δv / volatility) — each explains what it measures and its Tier-1 vs
Tier-3 availability. Completes the mouseover-hint pass (PAD faders + relation
rows already had them).

Verified: pytest tests/test_web_* (84) + node Playwright (all 6 cells titled).
2026-07-07 00:37:36 -07:00
vh fd6bed2d11 chore(design): vendor the transparent squirrel brand mark
The bg-removed source (ImageMagick corner floodfill) the inlined web favicon +
brand marks derive from. 1024², alpha-transparent; downscale for any reuse.
2026-07-07 00:37:36 -07:00
vh 459e7fa602 feat(web): SVG sparklines + 3D isometric mood cube (v0.20.4)
Import the updated Claude Design prototype's SVG sparkline system + 3D graph,
replacing the unicode-char sparklines:

- per-PAD-fader vertical SVG strips (stripPoints, 26x132 beside each bar — time
  down Y newest-at-bottom, value on X, gradient-faded, dot at newest); also
  answers the earlier "next to each meter" ask
- relation-row horizontal SVG sparklines (sparkPointsH, 56x13, auto-scaled,
  gradient + end dot) — fixed-width, so the old unicode overflow onto the n
  column can't recur
- mood-orbit reworked from a 2D P×A scatter into a 3D ISOMETRIC P×A×D cube
  (proj3: P right-down / A left-down / D up, 2:1 iso, scale 26, reverse-derived
  from the design's now-point + verified) with the trajectory, a pulsing
  now-marker, and a drop line + floor-shadow ellipse for depth
- gradients in one hidden <defs> svg; removed the orphaned sparkline()/_SPARK

Contract amended. Verified: pytest tests/test_web_* (84) + node Playwright
(injected 24-sample history — 3 PAD strips + 4 relation sparklines + the 3D
cube trajectory/drop/floor all render; gradients resolve).
2026-07-07 00:29:18 -07:00
vh cc6e85cd9b feat(web): brand mark + favicon → the aurora squirrel, replacing the ᛯ rune (v0.20.3)
- remove the black background from the operator's squirrel art (ImageMagick corner
  flood-fill, fuzz 20% — keeps interior linework + glow, drops connected background),
  downscale 1024->80px + quantize, inline as one SQUIRREL data-URI const
- wire it to the favicon <link> href + both brand marks (rail brand-row + setup h1);
  .brand-glyph (font rune) -> .brand-mark (img, drop-shadow glow + breathe)

Also banks worldtree-dev's exact 3-gate we-framing rule (b24) in persistent-memory —
canonDirective held as-is until it deploys on personal (would otherwise drift from the
currently-deployed renderer).

Verified: pytest tests/test_web_* (84) + node Playwright (favicon + both brand marks
decode from the data-URI; no leftover rune).
2026-07-06 23:24:06 -07:00
13 changed files with 851 additions and 98 deletions
+24 -5
View File
@@ -82,11 +82,12 @@ functions:
flexibility: "prescriptive"
- name: "create_app (amendment)"
signature: "create_app(client_factory, *, end_user_id=None, bifrost_consumer_key=None, bifrost_visible_host=None, affect_read_url=None, admin_key=None) -> Starlette"
description: "New optional admin_key param stored at app.state.admin_key; entrypoint passes RATATOSKR_ADMIN_API_KEY. Three new routes registered."
signature: "create_app(client_factory, *, end_user_id=None, bifrost_consumer_key=None, bifrost_visible_host=None, affect_read_url=None, memory_read_url=None, admin_key=None) -> Starlette"
description: "New optional admin_key param stored at app.state.admin_key; entrypoint passes RATATOSKR_ADMIN_API_KEY. v0.20.7 adds memory_read_url (app.state.memory_read_url, from RATATOSKR_MEMORY_READ_URL) for the memory viewer. Four new routes registered across the arc."
postconditions:
- "POST-001: app.state.admin_key = admin_key (default None -> the two admin routes fail-visible per their PRE-001)."
- "POST-002: routes /api/sessions/{session_id}/tools, /api/sessions/{session_id}/bifrost, /api/admin/events added; existing routes unchanged."
- "POST-003 (v0.20.7): app.state.memory_read_url = memory_read_url; route /api/memory/chunks added (the memory-viewer proxy)."
flexibility: "closed"
- name: "reasoning indicator (index.html: showThinkingNote / hideThinkingNote)"
@@ -125,11 +126,11 @@ functions:
flexibility: "open"
- name: "renderConsole + trend (v0.20.0 — unified persona/affect console; supersedes renderAffectPane/renderPersonaPane/setPersonaStrip)"
signature: "renderConsole(snap) ; setFader(axis,v) ; setFaderTrend(axis) ; renderOrbit() ; renderDominant(snap) ; renderDerived(snap) ; renderRelations(snap) ; renderDirective(snap) ; pushAffectHistory(snap) ; sparkline(vals) ; trendDelta(vals)"
signature: "renderConsole(snap) ; setFader(axis,v) ; setFaderTrend(axis) ; renderOrbit() ; renderDominant(snap) ; renderDerived(snap) ; renderRelations(snap) ; renderDirective(snap) ; pushAffectHistory(snap) ; sparkPointsH(vals,w,h,endX) ; padDeltas(vals) ; deltaStrip(deltas) ; orbitFrame(H,head,ts) ; orbitProj/orbitShadowY/orbitWallPt/orbitAxisPt ; startOrbitAnim() ; trendDelta(vals)"
description: "ONE render path for BOTH the Tier-1 persona_state snapshot and the Tier-3 affect snapshot (renderConsole), feeding the right affect console: dominant/canonical-mood centerpiece, bipolar PAD faders (each with a turn-to-turn Δ + sparkline), a P×A mood orbit from PAD history, an affect-derived grid, relations metric rows, and the canonical directive. Replaces the v0.19.x split of renderPersonaPane (Tier-1 pane) + renderAffectPane (Tier-3 pane) + setPersonaStrip (top-bar strip, removed — PAD now lives in the console faders)."
postconditions:
- "POST-001: reads snap.relations (relation_edge/1: target_entity + trust_ability/benevolence/integrity + warmth as {value,confidence,evidence_count} + agency + relation_context) — the CURRENT Worldtree emit shape; falls back to the legacy flat snap.valence for an older emitter. Tier-1 fields (baseline_pad, mood_drift, dominant_emotion, emotions_active) render WHEN PRESENT, '—' when absent (Tier-3 lacks them)."
- "POST-002: each PAD fader + relation metric shows current value + Δ-vs-previous (▲/▼) + a unicode sparkline auto-scaled to its OWN observed range (flat ▄/— when sub-0.01 stable — no noise amplification), drawn from AFFECT_HIST (rolling, HIST_CAP=24, session-lived). setFaderTrend fills the per-meter Δ+spark slots; renderOrbit plots the last N (P,A) samples as a scaled trail with a pulsing current marker."
- "POST-002: SVG sparklines + affect visuals (v0.20.4, adapted from the design prototype; v0.20.7 = design iteration-3). Each relation metric shows a HORIZONTAL SVG sparkline (`sparkPointsH`, 56×13, auto-scaled to its OWN range, sparkFade gradient + end dot), now BACKED by a subtle grid (`<pattern id=sparkGrid>` + a bg `<rect>` behind the polyline). Each PAD fader shows current value + Δ-vs-previous (▲/▼) + a per-turn Δ STRIP: v0.20.7 REPLACES the vertical polyline strip (removed `stripPoints`) with a column of 12 diverging HTML bars (`padDeltas``deltaStrip`, newest at bottom, each bar offset L/R of a center line by that turn's Δ, magnitude→width, age→opacity, zero-Δ→faint center dot). renderOrbit is now a DIMETRIC OPEN BOX (azimuth 35° / elevation 25°, D→right / A→left-back / P→up; removed the isometric `proj3` for `orbitProj/orbitShadowY/orbitWallPt/orbitAxisPt`) — a ghost A×P wall (carrying the P readout) + a D×A floor, JS-DRIVEN animated replay (`orbitFrame` rebuilt per rAF frame by a singleton `startOrbitAnim` loop reading live `ORBIT_HIST`; no SMIL/CSS-keyframes; reduced-motion → static final-state render). All drawn from AFFECT_HIST (rolling, HIST_CAP=24, session-lived); coords are computed numerics (no upstream strings → INV-004 trivially held). Gradients/patterns live in one hidden `<defs>` svg in the console."
- "POST-003: pushAffectHistory dedupes by emitted_at||last_updated_at so the ~4x/turn post-turn PAD poll contributes ONE sample/turn; history is CLIENT-side only (lost on reload — durable cross-session history via a provider-side snapshot log is a deferred follow-up, NOT built here)."
- "POST-004: INV-001 honesty — no fabricated Tier-1 fields. The dominant-emotion centerpiece shows a real OCC dominant_emotion (Tier-1) OR the CANONICAL mood word from canonMood(pad) (Tier-3, dimmed) OR '—'; NEVER a synthesized emotion. The affect-derived grid drops non-emitted metrics (intensity/decay-τ) and shows only real/client-derived cells (baseline/drift real for Tier-1, client-derived samples/volatility). INV-004 — every dynamic value passes through esc(); numerics go through toFixed, never innerHTML-raw."
flexibility: "open"
@@ -144,8 +145,26 @@ functions:
- "POST-004: HONEST-PARTIAL provenance (affect-egress-reference §3). The mood descriptor + relationship directive are EXACT (tagged 'exact'); the mood DIRECTIVE is a CANDIDATE pair (tagged 'candidate') — the OCC emotion directive for the delivered dominant_emotion type AND the PAD-band fallback — because affect.emit is type-only (no intensity) so the salience gate (≥0.2) can't be evaluated; BOTH are shown with the 'injected if intensity ≥ salience' caveat, never asserting which fires. When dominant_emotion is absent the fallback alone is EXACT. The panel is labeled reconstructed + hidden-from-consumers + dev-only (the reference-impl's sanctioned understand/reconstruct use, NOT end-user display per the reference's caveat). WATCH: a pending Worldtree render_d2_canonical change conditionally drops the trailing 'avoid premature we-framing' clause under a 3-gate combo — canonDirective holds as-is until worldtree-dev pings with the exact conditional + a canon bump."
flexibility: "open"
- name: "memory viewer (v0.20.7 — provider debug read → web proxy → console pane)"
signature: "server: _memory_chunks_endpoint(request) [GET /api/memory/chunks] ; provider: add_memory_read_route(app, store) [GET /memory/chunks] + RatatoskrMemoryStore.list_chunks(*, agent_id, end_user_id) + .count_chunks() ; index.html: loadMemory(agentId) ; renderMemory(data) ; setMemHead(count, total)"
description: "Durable memory chunks Worldtree promoted into OUR store, surfaced as a live-polling MEMORY console pane (content·scope·origin·revision per chunk). Mirrors the #18-D2 affect read pattern: a NON-bifrost debug read on OUR own store (bifrost's memory protocol has no list-all verb) → a web proxy supplying end_user_id server-side → the pane. Polled on session open + the post-turn window (promotion is async, like affect.emit)."
postconditions:
- "POST-001 (provider read): GET /memory/chunks?agent_id=&end_user_id= returns {chunks:[{chunk_id,content,scope,origin,revision}], count, total}. end_user_id REQUIRED (400 missing_end_user_id) — the partition boundary. Filter: end_user STRICT (scope.end_user==end_user_id), agent_id LENIENT (excluded only if the chunk CARRIES an agent_self axis that differs — so an {end_user}-only chunk, the real WT promotion shape, is not hidden). An empty match is a 200 empty list (0-chunks is a visible answer, never a 404). `total` = unfiltered store-wide count (distinguishes empty-store from scope-mismatch). content = best-effort text field / distillate summary / compact JSON-minus-embedding — a DEBUG read; bifrost verbs stay index/conduit-faithful."
- "POST-002 (web proxy): GET /api/memory/chunks supplies end_user_id from app.state.end_user_id (NEVER the browser), forwards the browser-named agent_id, proxies to app.state.memory_read_url (the combined :8392 provider serves both read routes). 400 memory_not_configured when unset; 502 memory_provider_unreachable on network error; status passthrough otherwise. Mirrors _affect_state_endpoint (#18 D2 INV-002)."
- "POST-003 (pane): renderMemory shows count(matched)/total(store-wide) in the head + one .mem-chunk per chunk (scope axes + origin + revision + content, ALL esc()'d — INV-004). Empty states are honest + diagnostic: total 0 → 'no memory chunks yet — promotion needs a bound memory/combined session + ~6 turns (or idle); if 0/0 the bind wasn't memory-granted or closed pre-promotion'; total>0 → 'scope mismatch, not an empty store'."
flexibility: "open"
- name: "markdownSafe pass-2 (v0.20.6 RP coloring + v0.20.7 tables / nested lists / streaming)"
signature: "markdownSafe(raw) ; mdTable(lines, i) ; mdInline(s)"
description: "The escape-first whitelist Markdown renderer, extended pass-2: GFM pipe tables, indentation-nested lists, ordered-list start numbering, and streaming-partial robustness. Pass-1 (RP speech/action coloring + CommonMark paragraph reflow) shipped v0.20.6."
postconditions:
- "POST-001: GFM pipe tables (`mdTable`) — a pipe row + an alignment/delimiter row (`|---|:--:|`) → <table class=md-table> with per-column text-align from the delimiter colons; body rows parsed until a non-pipe line."
- "POST-002: indentation-nested lists — leading-space depth builds a stack of <ul>/<ol> with each child list INSIDE the open parent <li> (valid nested HTML); same-level items are siblings; ul↔ol switches close+reopen. Ordered lists honor the first item's number (<ol start=N> when != 1)."
- "POST-003: streaming robustness — an unterminated code fence renders as a partial code block; a table header without its delimiter yet falls through to a paragraph (becomes a table once the delimiter streams in); parsing never throws on a partial. INV-004 held — esc() runs FIRST on the whole input, so table cells / list items / code all carry escaped content."
flexibility: "open"
invariants:
- "INV-004 (untrusted-render): ALL model / tool / admin / agent-supplied text is escaped before entering the DOM (esc via textContent, or esc(JSON.stringify)). No new render path introduces an innerHTML sink for upstream content. This is the highest-value review target — the new JS render paths are NOT unit-tested."
- "INV-004 (untrusted-render): ALL model / tool / admin / agent-supplied text is escaped before entering the DOM (esc via textContent, or esc(JSON.stringify)). No new render path introduces an innerHTML sink for upstream content. This is the highest-value review target — the new JS render paths are NOT unit-tested. v0.20.7: the memory pane (chunk content/scope/origin), the delta-strip bars, and markdownSafe table cells / list items all pass through esc() (esc runs FIRST on the whole markdown input)."
- "INV-ADMIN-KEY: the admin key exists ONLY at app.state.admin_key (from RATATOSKR_ADMIN_API_KEY). It is never serialized into any response, never sent to the browser, never logged. The browser receives only the session-filtered RESULT of admin-scoped reads."
- "INV-FILTER: AdminEvents filtering happens SERVER-side (_admin_event_matches_web) — the browser never receives the cross-session admin firehose; only active-session events + non-heartbeat system.* cross the wire."
- "INV-FAIL-VISIBLE: both admin routes return 400 admin_key_not_configured when the key is absent — never a silent empty pane, never an upstream call with an empty bearer."
Binary file not shown.

After

Width:  |  Height:  |  Size: 521 KiB

+17 -3
View File
@@ -1,6 +1,6 @@
# Persistent memory — ratatoskr
_Last updated: 2026-07-06_
_Last updated: 2026-07-07_
This file captures durable intent and supporting evidence (goals, decisions,
foot-gun warnings, in-flight state) across context resets. Read it at session
@@ -39,7 +39,15 @@ upstream API key stays server-side (INV-003).
## Current state / in-flight
_As of 2026-07-06 (session cont.):_
_As of 2026-07-07 (session cont.):_
**✅ SHIPPED — web-UI iteration-3, all three queued items (`v0.20.7`, patch, agent-discretion; 631 tests green; :8392 combined provider + :8765 web both restarted on the new code).**
**(A) Design prototype iteration-3 adapted into `index.html`** (re-pulled `Ratatoskr Console.dc.html`, project `bc0b65d1-…`): (1) sparkline **grid bg**`<pattern id=sparkGrid>` in the hidden defs + a bg `<rect fill=url(#sparkGrid)>` behind every relation-row spark polyline; (2) **PAD strips → per-turn Δ bars** — REMOVED the vertical polyline strip (`stripPoints`/`proj3` gone) for `padDeltas``deltaStrip`: a 12-cell HTML column (newest at bottom) of diverging bars offset L/R of a center line by that turn's Δ (magnitude→width, age→opacity, zero→faint center dot); head legend now "Δ/turn · last 12 · newest ↓"; (3) **mood orbit → DIMETRIC OPEN BOX** (viewBox 124×140, az35/el25, D-right/A-left-back/P-up) — ghost A×P wall (P readout) + D×A floor, `orbitProj/orbitShadowY/orbitWallPt/orbitAxisPt` projections, **JS-driven animated replay** (`orbitFrame` rebuilt per rAF by a singleton `startOrbitAnim` reading live `ORBIT_HIST`; reduced-motion → static final-state; no SMIL). Playwright-verified (dimetric frame + 17 dyn children + 21 Δ-bars + 5 grid rects; dark+light screenshots).
**(B) Memory viewer SHIPPED + the 0/0 mystery ROOT-CAUSED.** New non-bifrost debug read `GET /memory/chunks?agent_id=&end_user_id=` on the combined `:8392` provider (`RatatoskrMemoryStore.list_chunks` + `count_chunks` + shared `add_memory_read_route`, wired into `build_memory_provider_app` + `combined.py`; **end_user STRICT, agent_id LENIENT** so `{end_user}`-only WT chunks aren't hidden; returns `{chunks,count,total}`, empty=200 not 404) → web proxy `GET /api/memory/chunks` (server-supplied end_user, new `RATATOSKR_MEMORY_READ_URL` env, default `:8391`, set to `:8392` in env.sh) → a live-polling MEMORY console pane (`loadMemory`/`renderMemory`/`setMemHead`, polled on open + post-turn). **ROOT CAUSE of the operator's 0/0** (settled via a bound 6-turn probe + op-feed): the Bifrost memory plane **binds and grants fine** (handshake `caps_requested:[affect,memory]``caps_granted:[memory,affect]`), but **sindra dispatches ZERO memory ops** (not even a recall search) — her reset-clean agent definition has **no `memory:{}` block**, so Worldtree never runs the memory pipeline for her. NOT a bind-grant failure, NOT promotion-timing. **PROVEN end-to-end** with a throwaway memory-enabled `ratatoskr:memprobe` (defined with `memory:{}`): 6 bound turns → 6 `memory.search` recalls + **4 `memory.upsert_many` → 4 real chunks in `memory.db`** → the pane renders all 4 (content·scope·origin·revision) live. **⚠ LEFTOVER debug state to optionally clean:** `ratatoskr:memprobe` agent on personal WT + its 4 test chunks (scope `{end_user:ratatoskr-tui}`) now sit in the live `memory.db` — harmless (they make the pane show real data) but they're test cruft; `scripts/reset-sindra-stores.sh` or a `DELETE /agents/ratatoskr:memprobe` clears them. **To fix sindra's memory for real: DELETE+redefine her WITH `memory:{}`** (memory is immutable post-define) — operator's call on her agent.
**(C) Markdown pass-2 SHIPPED** — `markdownSafe` extended: GFM pipe tables (`mdTable`, alignment colons), indentation-nested lists (stack of `<ul>`/`<ol>`, child list inside the open `<li>` = valid nested HTML), ordered-list `start=N` numbering, and streaming robustness (unterminated fence → partial code block; header-without-delimiter → paragraph until the delimiter streams in; never throws). esc-first → INV-004 held. Playwright-verified all cases.
**SHIPPED — web UI redesign via Claude Design (`v0.20.0`, MINOR, operator-approved).** The Claude Design prototype **`Ratatoskr Console.dc.html`** (project `bc0b65d1-a33e-422a-8bc1-3635c9112775`) was pulled via `DesignSync get_file` (design scopes already granted this session — no `/design-login` needed) and adapted into `src/ratatoskr/web/static/index.html`: translated OUT of the `.dc.html` dialect (`<x-dc>`/`<sc-if>`/`<sc-for>`/`{{}}`/`DCLogic`/external `_ds/` CSS — none runnable) into single-file/no-CDN/vanilla, with ALL real `/api/*` fetch + SSE wired into its DOM (endpoint set + SSE vocab unchanged from the prior SPA — ported verbatim, only DOM hooks re-targeted). New shape = a **3-column command-console**: left engine-ticker rail (DEBUG+ADMIN+tool/turn-lifecycle MERGED into one timeline via `tickerAdd` + a tools-armed chip list + a FULL-detail Bifrost rail pane) · center conversation (per-turn INLINE chain-of-thought, replacing the Think pane) · right RESIZABLE affect console (dominant/canonical-mood centerpiece + bipolar PAD faders EACH with a turn-to-turn Δ+sparkline + a P×A mood orbit + relations metric rows + canonical directive). ADDED (round 2, operator-requested): a **light/dark theme toggle** (dark default; FULL token override — surfaces+fg+borders+accent-as-text, since the designer's light theme only did surfaces → would've been light-on-light) + a **full-detail Bifrost pane** (endpoint/connected/consumer/caps/tools) + fixed the **engine-ticker spine** (was a container-anchored `::before` that scrolled out of view on auto-scroll → re-anchored to a content-height `.ticker-inner` wrapper) + **per-fader PAD turn-to-turn Δ+sparkline** (fills the room beside each meter, from the deduped-per-turn AFFECT_HIST) + an **INLINED data-URI favicon** (operator's `/home/lkraven/rata.png` — chibi aurora squirrel — downscaled 1024→64px via PIL, ~8.6KB base64, kills the /favicon.ico 404). ALL server routes UNCHANGED (**84 web tests green**). Verified BOTH lenses: `pytest tests/test_web_*` (84) + node Playwright drove the real UI end-to-end against personal :8081 (session open → Sindra seeded greeting → live turn SSE → affect console + relations + bifrost detail; theme toggle + PAD deltas + ticker spine + no-favicon-404 all confirmed, dark+light screenshots). `:8765` restarted on the new code. Contract `web_debug_surface.contract.md` amended in-commit (v0.20.0 presenter renames: `renderBifrostState``renderBifrost`, `renderAffectPane``renderConsole`, `setPersonaStrip` removed; INV-001/INV-004 held). **HONEST-SHAPE call (INV-001, agent-discretion within settled policy):** the dominant-emotion centerpiece shows a real OCC emotion (Tier-1) OR the CANONICAL mood word (Tier-3 e.g. Sindra→"positive and energized", dimmed) OR "—", NEVER a fabricated emotion; the affect-derived grid drops non-emitted intensity/decay-τ, shows only real/client-derived cells. **OPEN (operator's call):** the per-fader PAD Δ placement is a sensible default — operator offered to have the designer spec the exact treatment (hooks are in place to swap it). **`v0.20.1` patch (operator-reported UI):** fixed the relations sparkline overflowing onto the `n` (evidence-count) column — the sparkline grows one char/sample (HIST_CAP=24) and overflowed its fixed grid cell, covering `n`; now capped (relations last-8, faders last-7) + `overflow:hidden` clip; verified via Playwright injecting a 24-sample history (sparkline→n bounding-box overlap = 0). ADDED native `title` mouseover hints on all 3 PAD faders + every relationship metric row (meaning + range; static METRIC_HINTS, esc()'d). Added `state.lastSnap` (console can re-render without a refetch). Playwright-verified.
@@ -168,7 +176,13 @@ decision. Captures rationale that won't be obvious from code alone.
- `[2026-07-06]` **Web UI now RENDERS the seeded first-message (`v0.19.9`) — operator-reported "i don't see Sindra's greeting on the web ui".** Diagnosis: the auto-seed WORKED (greeting was in the ledger at seq-0), but the web SPA never fetched a session's EXISTING history — NO `/api/sessions/{id}/messages` route (GET /messages was originally deferred out-of-scope; sessions used to start empty so it never mattered) and `startSession()` went straight from create → persona/tools/admin hydration, so the transcript only filled from the live turn stream + user echoes. Fix: (1) NEW web proxy route `GET /api/sessions/{id}/messages``get_session_messages` (mirrors the tools/bifrost proxies; status-preserving `session_messages_unavailable` envelope); (2) SPA `loadTranscript(sessionId)` — fetches the route on open, renders assistant items as `.response .md-body` (markdownSafe, same escape-first path as appendResponse) + user items as `.prompt-echo` (textContent), called in `startSession` after the workspace opens; best-effort (swallows failures). Contract `web_debug_surface.contract.md` amended (server endpoint + loadTranscript entries). TDD (2 web route tests, suite 617 green) + **Playwright DOM check PROVED the render** (drove the real UI: pick sindra → open → her greeting bubble appears — the JS-render lens unit tests can't reach; [[feedback_debug_surface_uses_canonical_surface_only]] cousin lesson). Web restarted on the fix. **FOOT-GUN (self-inflicted): `pkill -f "ratatoskr-web --host"` SELF-MATCHES the bash command running it → exit 144, killed its own restart mid-flight — kill the web by PID, never `pkill -f` on a pattern your own command contains.** **FOOT-GUN: uvicorn hangs on SIGTERM with an open admin-events SSE → needed SIGKILL.** **Playwright: python module absent from the venv; use node + `executablePath=/opt/ms-playwright/chromium-1223/chrome-linux64/chrome` — the shared browser is build 1223, npm-latest playwright wants 1228 (version-mismatch), so pin executablePath instead of letting playwright resolve.**
- `[2026-07-06]` **Web UI: pivot from incremental CSS polish to a designed prototype (Claude Design) that I wire into.** Operator saw an Australis polish pass ("looks fine, but we're attacking it differently") and chose the prototype route — a designer builds the visual shell, I wire real data/SSE into its DOM. Authored the full design brief `docs/design/ratatoskr-web-design-brief.md` (complete information inventory of every pane/datum/state + Australis direction + single-file/no-CDN/vanilla wire-ability constraints). **Tracking surface:** the brief file + Claude Design project `bc0b65d1-a33e-422a-8bc1-3635c9112775` (file `Ratatoskr Console.dc.html`). Import mechanism = the `DesignSync` MCP; blocked on `/design-login` (claude.ai design scopes) — see Current state for the post-auth wiring plan.
- `[2026-07-07]` **Affect-egress reference delivered by worldtree-dev + a we-framing render DRIFT-WATCH banked.** worldtree-dev shipped `docs/affect-egress-consumer-reference.md` (`74d2408`, their origin/main) — the authoritative DELIVERED-on-wire vs HIDDEN (system-prompt-only) map for our affect surface. Confirms the v0.20.x console consumes it correctly: DELIVERED via affect.emit = pad + relations[RelationEdge] + dominant_emotion (**type-only, b23**; the #204 `affect_update` SSE is SUPPRESSED for Tier-3, so its richer `emotions_active` is Tier-1-only — we don't get it, and correctly poll our own affect store post-turn instead). HIDDEN render OUTPUTS are deterministically reconstructable from the canon; we reconstruct only the two FULLY-reconstructable (mood descriptor via `canonMood`, relationship directive via `canonDirective`) and SKIP the PARTIAL mood-directive (dominant_emotion is type-only/no-intensity → can't tell if the OCC directive fires at salience≥0.2 vs the PAD-band fallback) — honest per INV-001. Vendoring the ref doc as a `tolerate_drift` pin is SURFACED to Vuong (worldtree-dev will co-sign + honor a ping-on-change handshake, same as the d2-canon). **DRIFT-WATCH (banked; do NOT act yet):** a `render_d2_canonical` change is UNDER DESIGN (NOT shipped) — conditionally DROP the fixed trailing "avoid premature we-framing" clause under a 3-gate combo (**unsafe-capable agent AND deep warmth AND expressive tie**). Our `canonDirective()` ALWAYS appends that clause → it WILL drift for exactly that cell when this lands. worldtree-dev will ping ratatoskr-dev FIRST with the exact conditional + a canon bump; hold `canonDirective` as-is until then. [reference-impl: byte-exact reconstruction means a dep render change is our drift — the exact reason we vendor+pin the d2 canon]
- `[2026-07-07]` **Affect-egress reference delivered by worldtree-dev + a we-framing render DRIFT-WATCH banked.** worldtree-dev shipped `docs/affect-egress-consumer-reference.md` (`74d2408`, their origin/main) — the authoritative DELIVERED-on-wire vs HIDDEN (system-prompt-only) map for our affect surface. Confirms the v0.20.x console consumes it correctly: DELIVERED via affect.emit = pad + relations[RelationEdge] + dominant_emotion (**type-only, b23**; the #204 `affect_update` SSE is SUPPRESSED for Tier-3, so its richer `emotions_active` is Tier-1-only — we don't get it, and correctly poll our own affect store post-turn instead). HIDDEN render OUTPUTS are deterministically reconstructable from the canon; we reconstruct only the two FULLY-reconstructable (mood descriptor via `canonMood`, relationship directive via `canonDirective`) and SKIP the PARTIAL mood-directive (dominant_emotion is type-only/no-intensity → can't tell if the OCC directive fires at salience≥0.2 vs the PAD-band fallback) — honest per INV-001. Vendoring the ref doc as a `tolerate_drift` pin is SURFACED to Vuong (worldtree-dev will co-sign + honor a ping-on-change handshake, same as the d2-canon). **WE-FRAMING DRIFT-WATCH → STOOD DOWN (worldtree-dev 2026-07-07, `01KWXMQPHN…`).** The b24 3-gate we-framing conditional (drop "; avoid premature we-framing" under unsafe_capable+deep-warmth+expressive) was **REVERTED** — Vuong reframed it. So **`canonDirective` STAYS UNCONDITIONAL** (always appends the clause), which is CORRECT: it matches the currently-deployed renderer (b23) AND my pinned `affect-egress-consumer-reference.md` §2d (the doc reverted too — nothing changed on the wire or in my pin; NO re-vendor needed). The HOLD was right → ZERO rework. **NEW forward-watch (replaces this one):** the we-framing gate was a SYMPTOM — the render bakes enterprise safety-guards into the directive strings, so even `full`-tier characters get DEFANGED emotions (a hostile villain told to "keep a firm emotional boundary"). Fix = a **use-case-segregated persona render** (assistant / companion / RP-gaming), now a fresh **brokkr deep-research epic**. When it lands + is implemented, the render behavior for character/RP agents (→ our `canonDirective` + `canonEmotionDirective` reconstruction + the vendored d2 canons) will change MATERIALLY — worldtree-dev re-engages ratatoskr-dev then with the new reconstruction spec + a coordinated re-vendor. Until then: NO action, reconstruction stays as-is. [my unsafe_capable/mood_tier heuristic — character→full, agent→safe — was confirmed correct + is banked for whenever tier-gating returns]
- `[2026-07-07]` **Web-UI iteration-3 SHIPPED (`v0.20.7`, patch) — all three queued items + the memory 0/0 root-caused.** (A) design iteration-3 (sparkline grid-bg, PAD Δ-bar strips replacing the polyline strips, dimetric-open-box mood orbit w/ JS replay) adapted into index.html; (B) memory viewer (provider `GET /memory/chunks` debug read on `:8392` → web `/api/memory/chunks` proxy → MEMORY console pane) + the mystery SETTLED: bind grants memory fine (`caps_granted:[memory,affect]`) but sindra emits ZERO memory ops → her reset-clean def lacks `memory:{}`; PROVEN via throwaway `ratatoskr:memprobe` (memory-enabled) → 4 real chunks landed + rendered in the pane; (C) markdown pass-2 (tables/nested-lists/ordered-start/streaming). 631 tests green; `:8392`+`:8765` restarted on new code. New env `RATATOSKR_MEMORY_READ_URL=:8392`. Contract `web_debug_surface.contract.md` amended in-commit. **Leftover:** memprobe agent + its 4 test chunks live in `memory.db` (optional cleanup). See Current state § ✅ SHIPPED for full detail. [supersedes the QUEUED entry below]
- `[2026-07-07]` **Web-UI next-work QUEUED for a fresh-context session (operator-directed `/snapshot` handoff) — full specs in Current state § ⏭ QUEUED.** Three deferred-but-scoped items: **(A)** Claude Design prototype ITERATION-3 re-import (sparkline bg grid `<pattern>`, PAD strips → per-turn-Δ-bar HTML columns, mood orbit → DIMETRIC open-box az35/el25 D-right/A-left-back/P-up + JS-animated replay `orbitDynamics`); **(B)** memory viewer console pane (provider `GET /memory/chunks` read → web proxy → polling pane, mirror #18-D2; bundle a 6-turn bound round-trip proof to settle the 0/0-memory mystery); **(C)** markdown pass-2 (tables / nested lists / streaming). **Tracking surface:** the Claude Design projectId `bc0b65d1-a33e-422a-8bc1-3635c9112775` (durable — re-pull for exact coords) + this snapshot capture; operator-directed. Design scopes already granted (no `/design-login`).
- `[2026-07-07]` **Markdown pass-1 SHIPPED (`v0.20.6`) — RP semantic coloring + paragraph reflow (operator-directed markdown rework, step ①).** The transcript renderer (`markdownSafe`/`mdInline`) now colors the two roleplay registers: `"quoted"` dialogue → SPEECH (bright `--md-speech`=fg-0), `*asterisk*` → ACTION/narration (muted-italic `--md-action`=fg-3, on `em.md-action`); both tunable via 2 CSS vars that cascade through `--fg-*` so they auto-adapt to the light theme. Plain text stays default narration. **KEY ORDERING:** speech-wrap runs BEFORE the em/link passes so a generated `class="md-action"` / `href="…"` quote can't be mis-read as dialogue (adversarially verified). Straight + smart quotes; apostrophes don't trigger; unbalanced/half-streamed quotes stay uncolored until they close. ALSO fixed the ugliest existing bug: single newlines were hard-`<br>`s → now CommonMark soft-breaks (space); a hard break needs 2+ trailing spaces or a trailing `\`. INV-004 escape-first preserved (html inside a quote escaped). Verified: 10-case Playwright unit-check of `markdownSafe` (speech/action/attr-trap/apostrophes/mixed/reflow/hard-break/escaping/unbalanced/paragraphs) all green + a visual render. Patch bump. **Markdown rework remaining: pass-2 (tables / nested lists / ordered-list numbering / streaming robustness); + the MEMORY VIEWER (step ②, console pane recommended) still queued.**
- `[2026-07-07]` **Affect-derived tooltips (`v0.20.5`) + transparent squirrel vendored + v0.20.0.5 PUSHED to origin (operator-authorized).** Native `title` hints on all 6 affect-derived cells (samples/updated/baseline P·A/drift Δv/volatility — each with meaning + Tier-1-vs-Tier-3 availability). Committed the transparent full-res brand mark at `docs/design/ratatoskr-mark.png` (1024², alpha; the bg-removed source the inlined favicon derives from — reproducible via the documented ImageMagick corner floodfill). The web-UI redesign arc (v0.20.0 Claude Design console → .1 sparkline-overflow+tooltips → .2 context-injection → .3 squirrel brand/favicon → .4 SVG sparklines+3D cube → .5 derived tooltips) is now all on `origin/main`.
- `[2026-07-07]` **SVG sparklines + 3D mood cube imported from the updated Claude Design prototype (`v0.20.4`) — operator: "the svg sparklines and the new 3d graph".** Re-pulled `Ratatoskr Console.dc.html` via `DesignSync get_file` (designer iterated the same project, +6KB). Adapted 3 SVG systems out of the `.dc.html` into vanilla, replacing the unicode-char sparklines: **(1)** per-PAD-fader VERTICAL strips (`stripPoints`, 26×132 SVG beside each bar — time down Y newest-at-bottom, value on X ±11, `stripFade{P,A,D}` gradient, dot at newest; this also nails the earlier "next to each meter" ask); **(2)** relation-row HORIZONTAL sparklines (`sparkPointsH`, 56×13, auto-scaled, `sparkFade` gradient + end dot — also kills the old unicode-overflow "n behind graph" for good since it's a fixed-width SVG); **(3)** the mood-orbit reworked from a 2D P×A scatter into a **3D ISOMETRIC P×A×D cube** (`proj3`: P right-down/A left-down/D up, 2:1 iso, center 62,66, scale 26 — REVERSE-DERIVED from the design's placeholder now-point + verified: `x=62+26P26A, y=66+13P+13A26D`), with the trajectory + pulsing now-marker + a drop line to the D=1 floor + a floor-shadow ellipse for depth. Gradients in one hidden `<defs>` svg. Removed the orphaned `sparkline()`/`_SPARK`. Contract amended. Playwright-verified (injected 24-sample history: 3 strips + 4 relation sparklines + the 3D cube trail/drop/floor all render; gradients resolve). Patch bump.
- `[2026-07-07]` **Brand mark + favicon → the aurora squirrel (`v0.20.3`), replacing the `ᛯ` rune.** Operator supplied `/home/lkraven/rata.png` (chibi cyan-green aurora squirrel + acorn). Removed the black background via ImageMagick corner flood-fill (`-fuzz 20% -floodfill` from all 4 corners — keeps the squirrel's interior black linework/eyes (not edge-connected) + the glow, drops only the connected background), downscaled 1024→80px + quantized-64-colors (~12KB base64), inlined as ONE `SQUIRREL` data-URI const in the JS wiring the favicon `<link id="favicon">` href + both `.brand-mark` imgs (rail brand-row + setup-card h1). `.brand-glyph` (font-rune) CSS replaced by `.brand-mark` (img, drop-shadow glow + breathe). Playwright-verified (both marks + favicon decode, naturalWidth>0; no brand-glyph left). Source PNG stays at `/home/lkraven/rata.png` (not committed — data-URI is self-contained + reproducible via the documented floodfill). Patch bump.
- `[2026-07-07]` **Context-injection view SHIPPED (`v0.20.2`) — the console now reconstructs the FULL hidden affect block Worldtree injects into the agent's system prompt (operator: "use that canon in the interface, see as much context injection as possible").** No new canon vendored — the strings were ALREADY in the pinned `d2-mood-render-canon-v1.json`; extended `build_persona_canon.py` to emit `mood_directive {occ_directives(15), pad_band_fallback, salience 0.2, pad_band_cutoff 0.3, full_only[love,anger,disgust,shame]}` into `persona_render_canon.json` (regen via Worldtree venv). New JS `canonPadFallback(pad)` + `canonEmotionDirective(type)` — BYTE-EXACT mirrors of Worldtree `core/persona/renderer._pad_band_fallback` + `derive_directive`; `renderDirective` expanded into a "CONTEXT INJECTION · reconstructed · hidden from consumers" panel showing mood descriptor [exact] + mood directive [candidate] + relationship directive [exact]. **HONEST-PARTIAL (affect-egress-ref §3):** affect.emit is type-only (no intensity) → can't evaluate the salience gate (≥0.2) → show BOTH candidates (OCC emotion directive + PAD-band fallback) with the "injected if intensity ≥ 0.2" caveat, never assert which fires; when dominant_emotion absent the fallback alone is exact. Panel labeled dev-only per the reference's "not-for-end-user-display" caveat (ratatoskr = the sanctioned reconstruct-platform-behavior use). Vendored + pinned `affect-egress-consumer-reference.md` (tolerate_drift, worldtree-dev co-signs + pings on change; drift 6/6 green). Contract amended. Playwright-verified (sindra: dominant_emotion=joy → joy OCC directive candidate + PAD-band fallback both render, exact/candidate tags color-coded). Patch bump (single-commit feature, no downstream coordination; minor-defensible but tie-breaks to patch). **OPEN — SURFACED to Vuong:** take worldtree-dev's standing offer to add emotion INTENSITY to affect.emit → resolves the OCC-directive-vs-fallback EXACTLY (drops the candidate ambiguity). [reference-impl privileged view: ratatoskr shows what WT hides from regular consumers]
- `[2026-07-06]` **Claude Design console SHIPPED (`v0.20.0` MINOR, operator-approved) — see Current state for the full record.** Pulled via `DesignSync get_file` (scopes already granted), adapted `.dc.html`→vanilla single-file, wired all `/api/*`+SSE into the new 3-column console DOM, then a round-2 fixup (light theme, full Bifrost pane, ticker-spine fix, per-fader PAD Δ, inlined favicon). 84 web tests + node-Playwright-vs-personal-:8081 both green; contract amended in-commit; INV-001 honest-shape held (canonical mood word for Tier-3, no fabricated emotion). **Foot-guns reconfirmed:** the `.dc.html` dialect is NOT runnable (translate, don't paste); a scroll-container-anchored `::before` timeline spine scrolls out of view on auto-scroll (anchor it to a content-height inner wrapper instead); a favicon 404 shows as a browser `console.error` even when handled (don't count it as a JS-test failure). **Foot-gun (favicon):** operator PNGs are full-res (1024² / 805KB) — downscale to ≤64px before inlining as a data URI.
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "ratatoskr"
version = "0.20.2"
version = "0.20.7"
description = "Worldtree Conversation API debug TUI — multi-pane observability dashboard"
readme = "README.md"
requires-python = ">=3.12"
+5 -3
View File
@@ -17,7 +17,7 @@ from bifrost.consumer import ConsumerRegistration, build_combined_app
from bifrost.reference_server import JwtVerifier
from ratatoskr.provider.affect_store import RatatoskrAffectStore, add_affect_read_route
from ratatoskr.provider.memory_store import RatatoskrMemoryStore
from ratatoskr.provider.memory_store import RatatoskrMemoryStore, add_memory_read_route
def build_combined_provider_app(
@@ -27,8 +27,9 @@ def build_combined_provider_app(
consumer_id: str = "ratatoskr",
):
"""Compose `build_combined_app` over BOTH stores + mount the shared affect read
route. Returns a Starlette app exposing POST /bifrost/handshake +
/bifrost/memory-call + /bifrost/affect-call + GET /affect/state/{agent_id}.
route AND the memory-viewer debug read route. Returns a Starlette app exposing POST
/bifrost/handshake + /bifrost/memory-call + /bifrost/affect-call + GET
/affect/state/{agent_id} + GET /memory/chunks.
Both stores are REQUIRED (INV-009): bifrost's build_combined_app raises if either
is None. The affect cap depends on the affect store advertising affect_supported +
@@ -45,4 +46,5 @@ def build_combined_provider_app(
# ValueError on None) and mounts handshake + memory-call + affect-call (no tool-call).
app = build_combined_app(memory_store, affect_store, verifier, registration)
add_affect_read_route(app, affect_store) # INV-011: the SAME read route, same db
add_memory_read_route(app, memory_store) # DEBUG read: GET /memory/chunks (memory viewer)
return app
+103 -1
View File
@@ -30,6 +30,8 @@ from bifrost.memory import (
StoreCapabilities,
)
from bifrost.reference_server import JwtVerifier
from starlette.requests import Request
from starlette.responses import JSONResponse
_SHORT_RETRY_TTL_SECONDS = 300
_DURABLE_JOB_TTL_SECONDS = 24 * 60 * 60
@@ -130,6 +132,29 @@ def _validate_injection(record: dict) -> None:
raise InvalidArguments("injection_source only valid for injected_context origin")
def _chunk_content_preview(record: dict) -> str:
"""Best-effort human-readable content for the DEBUG memory viewer only. Prefers an
explicit text field, then the distillate summary, and last-resorts to a compact JSON
of the record MINUS the (large, non-human) embedding — never a fabricated blank, so
the viewer shows whatever IS there. Read-only; the store's normal bifrost verbs stay
conduit/index-faithful (this is a separate debug read, not an interpretation of the
chunk on the recall path)."""
for key in ("content", "text", "body", "summary"):
value = record.get(key)
if isinstance(value, str) and value:
return value
distillate = record.get("distillate")
if isinstance(distillate, str) and distillate:
return distillate
if isinstance(distillate, dict):
for key in ("summary", "text", "content"):
value = distillate.get(key)
if isinstance(value, str) and value:
return value
trimmed = {k: v for k, v in record.items() if k not in ("embedding", "vector")}
return json.dumps(trimmed, separators=(",", ":"), default=str)
class RatatoskrMemoryStore:
"""The MemoryDataStore-shaped store handed to bifrost's build_memory_app."""
@@ -325,6 +350,54 @@ class RatatoskrMemoryStore:
self._conn.execute("DELETE FROM memory_vec WHERE chunk_id = ?", (chunk_id,))
return {"deleted": deleted}
def count_chunks(self) -> int:
"""DEBUG read seam: total stored chunk rows (unfiltered). Lets the memory
viewer distinguish 'store is empty' (total 0 — no upsert ever landed) from
'scope mismatch' (total > 0 but 0 matched the queried partition)."""
return int(self._conn.execute("SELECT COUNT(*) FROM memory_chunks").fetchone()[0])
def list_chunks(
self, *, agent_id: str | None = None, end_user_id: str | None = None
) -> list[dict]:
"""DEBUG read (non-bifrost): list stored chunks as a content·scope·origin view
for the web memory pane, filtered by the `end_user` (strict) and `agent_self`
(lenient) scope axes. bifrost's memory protocol has NO list-all verb, so this is
OUR read on OUR store (per the debug-surface-uses-canonical-surface principle:
we read only our own store, never a dep's private). Returns [] when nothing
matches — an empty list is a valid, visible answer (the 0-chunks state).
- `end_user_id`: strict — a chunk passes only if `scope.end_user == end_user_id`
(the partition boundary; the route requires it, the web proxy supplies it).
- `agent_id`: lenient — a chunk is excluded only if it CARRIES an `agent_self`
axis that differs; chunks written without one are not hidden (so a chunk
scoped `{end_user}`-only stays visible for diagnosis).
"""
rows = self._conn.execute(
"SELECT chunk_id, record_json, revision, scope_json, origin FROM memory_chunks"
).fetchall()
out: list[dict] = []
for chunk_id, record_json, revision, scope_json, origin in rows:
scope = json.loads(scope_json) if scope_json else {}
if not isinstance(scope, dict):
scope = {}
if end_user_id is not None and scope.get("end_user") != end_user_id:
continue
if agent_id is not None:
chunk_agent = scope.get("agent_self")
if chunk_agent is not None and chunk_agent != agent_id:
continue
record = json.loads(record_json)
out.append(
{
"chunk_id": chunk_id,
"content": _chunk_content_preview(record),
"scope": scope,
"origin": origin,
"revision": revision,
}
)
return out
def open_memory_store(db_path: str, *, embedding_dim: int) -> RatatoskrMemoryStore:
"""Open the SQLite+sqlite-vec memory store, creating schema + the vec index on first use."""
@@ -360,6 +433,31 @@ def open_memory_store(db_path: str, *, embedding_dim: int) -> RatatoskrMemorySto
return RatatoskrMemoryStore(conn, embedding_dim)
def add_memory_read_route(app, store: RatatoskrMemoryStore) -> None:
"""Mount the non-bifrost DEBUG read route GET /memory/chunks?agent_id=&end_user_id=
on `app`, reading store.list_chunks. SHARED by build_memory_provider_app and the
combined provider (mirrors the affect D2 add_affect_read_route). add_route (NOT Mount)
keeps /bifrost/* top-level so the op-feed path check still matches them and passes
this route through untouched. No JWT (internal-LAN trust model).
end_user_id is REQUIRED (400 missing_end_user_id) — the partition boundary, supplied
server-side by the web proxy, never named by the browser. agent_id is an optional
lenient filter. An empty match is a 200 with an empty list (the 0-chunks state is a
visible answer, not a 404).
"""
async def _memory_chunks_route(request: Request) -> JSONResponse:
end_user_id = request.query_params.get("end_user_id")
if not end_user_id: # never scan against a None/empty partition
return JSONResponse({"error_code": "missing_end_user_id"}, status_code=400)
agent_id = request.query_params.get("agent_id") or None
chunks = store.list_chunks(agent_id=agent_id, end_user_id=end_user_id)
return JSONResponse(
{"chunks": chunks, "count": len(chunks), "total": store.count_chunks()}
)
app.add_route("/memory/chunks", _memory_chunks_route, methods=["GET"])
def build_memory_provider_app(
store: RatatoskrMemoryStore,
heimdall_key: bytes,
@@ -369,6 +467,8 @@ def build_memory_provider_app(
Returns a Starlette ASGI app exposing POST /bifrost/handshake and
POST /bifrost/memory-call. The library owns the wire; this is the thin glue.
Additionally mounts the non-bifrost GET /memory/chunks DEBUG read route (the
memory-viewer pane's read seam), the memory-plane analogue of the affect D2 route.
"""
if not isinstance(store.describe_store(), dict): # PRE-001 / INV-008
raise ValueError("store must advertise capabilities via describe_store()")
@@ -376,4 +476,6 @@ def build_memory_provider_app(
raise ValueError("heimdall_key must be non-empty bytes")
verifier = JwtVerifier(algorithm="HS256", key_bytes=heimdall_key)
registration = ConsumerRegistration(consumer_id=consumer_id)
return build_memory_app(store=store, verifier=verifier, registration=registration)
app = build_memory_app(store=store, verifier=verifier, registration=registration)
add_memory_read_route(app, store) # DEBUG read: GET /memory/chunks (memory viewer)
return app
+8
View File
@@ -68,6 +68,13 @@ def main(argv: list[str] | None = None) -> int:
affect_read_url = os.environ.get(
"RATATOSKR_AFFECT_READ_URL", "http://127.0.0.1:8390"
)
# Memory viewer: the provider's memory DEBUG-read base URL (server→provider hop on
# the same dev box) so the MEMORY pane can render the chunks Worldtree persisted into
# OUR store. The combined :8392 provider serves both read routes; default to the
# standalone memory provider port, analogous to the affect default.
memory_read_url = os.environ.get(
"RATATOSKR_MEMORY_READ_URL", "http://127.0.0.1:8391"
)
# Admin observability panes (BifrostState + AdminEvents): the readonly-admin
# key stays SERVER-SIDE — the server proxies admin-scoped reads; the browser
# never receives the key, only the session-filtered result.
@@ -112,6 +119,7 @@ def main(argv: list[str] | None = None) -> int:
bifrost_consumer_key=bifrost_consumer_key,
bifrost_visible_host=bifrost_visible_host,
affect_read_url=affect_read_url,
memory_read_url=memory_read_url,
admin_key=admin_key,
)
+31
View File
@@ -430,6 +430,31 @@ async def _affect_state_endpoint(request: Request) -> JSONResponse:
return JSONResponse(r.json(), status_code=r.status_code)
async def _memory_chunks_endpoint(request: Request) -> JSONResponse:
"""GET /api/memory/chunks?agent_id=… → proxy the provider memory DEBUG read route.
Supplies end_user_id SERVER-SIDE (never the browser); proxies to the configured
memory-read URL, forwarding the browser-named agent_id as a filter. The
memory-plane analogue of the #18-D2 affect proxy — a live-polling view of what
Worldtree has persisted into OUR store (content·scope·origin per chunk)."""
memory_read_url = request.app.state.memory_read_url
end_user_id = request.app.state.end_user_id
if not (memory_read_url and end_user_id): # PRE-001: fail-visible, never silent
return JSONResponse({"error_code": "memory_not_configured"}, status_code=400)
params = {"end_user_id": end_user_id}
agent_id = request.query_params.get("agent_id")
if agent_id:
params["agent_id"] = agent_id
url = f"{memory_read_url}/memory/chunks"
try:
async with httpx.AsyncClient() as client:
r = await client.get(url, params=params)
except httpx.RequestError:
return JSONResponse(
{"error_code": "memory_provider_unreachable"}, status_code=502
)
return JSONResponse(r.json(), status_code=r.status_code)
async def _session_tools_endpoint(request: Request) -> JSONResponse:
"""GET /api/sessions/{session_id}/tools → owner-scoped tool inventory (spec #183).
@@ -551,6 +576,7 @@ def create_app(
bifrost_consumer_key: str | None = None,
bifrost_visible_host: str | None = None,
affect_read_url: str | None = None,
memory_read_url: str | None = None,
admin_key: str | None = None,
) -> Starlette:
"""Construct the Starlette app — wire routes + state per FN create_app.
@@ -612,6 +638,7 @@ def create_app(
Route("/api/sessions", _create_session_endpoint, methods=["POST"]),
Route("/api/agents/{agent_id}/persona_state", _persona_state_endpoint),
Route("/api/affect/{agent_id}", _affect_state_endpoint),
Route("/api/memory/chunks", _memory_chunks_endpoint),
Route("/api/sessions/{session_id}/tools", _session_tools_endpoint),
Route("/api/sessions/{session_id}/messages", _session_messages_endpoint),
Route("/api/sessions/{session_id}/bifrost", _session_bifrost_endpoint),
@@ -631,6 +658,10 @@ def create_app(
# Issue #18 (Deliverable 2): the provider affect-read base URL (server→provider hop,
# same dev box) — distinct from the WT-visible host used for binding.
app.state.affect_read_url = affect_read_url
# Memory viewer: the provider memory-read base URL (server→provider hop, same dev
# box) — the combined :8392 provider serves BOTH read routes, so in practice this
# points at the same host as affect_read_url; kept as its own config for isolation.
app.state.memory_read_url = memory_read_url
# Admin observability panes (BifrostState + AdminEvents): the admin key is
# SERVER-HELD (RATATOSKR_ADMIN_API_KEY) and never reaches the browser — the
# server proxies admin-scoped reads and forwards only the session-filtered result.
File diff suppressed because one or more lines are too long
+1
View File
@@ -112,6 +112,7 @@ def test_builds_both_planes_and_read_route():
assert "/bifrost/memory-call" in paths
assert "/bifrost/affect-call" in paths
assert "/affect/state/{agent_id}" in paths
assert "/memory/chunks" in paths # memory-viewer debug read, shared helper
def test_handshake_grants_both_caps():
+122
View File
@@ -510,3 +510,125 @@ async def test_parity_expected_revisions_vs_reference_through_dispatch():
assert await dispatch_memory_call(stale, wctx, ref) == await dispatch_memory_call(
stale, wctx, mine
)
# --- memory viewer DEBUG read route (GET /memory/chunks) ---------------------
# Non-bifrost debug read on OUR store: list_chunks + add_memory_read_route + the
# GET /memory/chunks route. Mirrors the affect D2 read-route tests.
from starlette.testclient import TestClient # noqa: E402
from ratatoskr.provider.memory_store import ( # noqa: E402
add_memory_read_route,
build_memory_provider_app as _build_mem_app, # noqa: F401 (re-import for clarity)
)
async def _seed_chunk(store, cid, *, scope, content=None, origin="worldtree"):
extra = {}
if content is not None:
extra["content"] = content
await store.upsert_many(
[_chunk(cid, embedding=_vec(1.0), scope=scope, origin=origin, **extra)],
idempotency_key="seed-" + cid,
ctx=_ctx(),
)
async def test_list_chunks_filters_strict_end_user_lenient_agent():
store = open_memory_store(":memory:", embedding_dim=EMBEDDING_DIM)
await _seed_chunk(store, "c1", scope={"end_user": "vuong", "agent_self": "ratatoskr:sindra"})
await _seed_chunk(store, "c2", scope={"end_user": "vuong"}) # no agent_self → lenient keep
await _seed_chunk(store, "c3", scope={"end_user": "other", "agent_self": "ratatoskr:sindra"})
await _seed_chunk(store, "c4", scope={"end_user": "vuong", "agent_self": "ratatoskr:other"})
got = store.list_chunks(agent_id="ratatoskr:sindra", end_user_id="vuong")
ids = sorted(c["chunk_id"] for c in got)
assert ids == ["c1", "c2"] # c3 wrong end_user, c4 different agent_self
# content·scope·origin·revision surfaced
c1 = next(c for c in got if c["chunk_id"] == "c1")
assert c1["content"] == "content-c1"
assert c1["scope"] == {"end_user": "vuong", "agent_self": "ratatoskr:sindra"}
assert c1["origin"] == "worldtree"
assert c1["revision"] == 1
async def test_list_chunks_no_agent_filter_returns_all_for_end_user():
store = open_memory_store(":memory:", embedding_dim=EMBEDDING_DIM)
await _seed_chunk(store, "c1", scope={"end_user": "vuong", "agent_self": "a"})
await _seed_chunk(store, "c2", scope={"end_user": "vuong", "agent_self": "b"})
await _seed_chunk(store, "c3", scope={"end_user": "nope"})
got = store.list_chunks(end_user_id="vuong")
assert sorted(c["chunk_id"] for c in got) == ["c1", "c2"]
def test_count_chunks_reports_total_unfiltered():
store = open_memory_store(":memory:", embedding_dim=EMBEDDING_DIM)
assert store.count_chunks() == 0
def _seed_row(store, cid, *, scope, content="x", origin="worldtree", revision=1):
"""Sync seed for the route tests (TestClient is sync): insert a chunk row directly.
The read route only reads memory_chunks, so the vec row is unnecessary here."""
import json as _j
rec = {"id": cid, "content": content, "scope": scope, "origin": origin}
store._conn.execute(
"INSERT INTO memory_chunks (chunk_id, record_json, revision, scope_json, origin) "
"VALUES (?, ?, ?, ?, ?)",
(cid, _j.dumps(rec), revision, _j.dumps(scope), origin),
)
store._conn.commit()
def test_memory_chunks_route_returns_matched_and_total():
store = open_memory_store(":memory:", embedding_dim=EMBEDDING_DIM)
_seed_row(store, "c1", scope={"end_user": "vuong", "agent_self": "ratatoskr:sindra"})
_seed_row(store, "c2", scope={"end_user": "other"})
app = build_memory_provider_app(store, heimdall_key=b"k")
client = TestClient(app)
r = client.get("/memory/chunks", params={"agent_id": "ratatoskr:sindra", "end_user_id": "vuong"})
assert r.status_code == 200
body = r.json()
assert body["count"] == 1
assert body["total"] == 2 # store has 2 chunks; only 1 matched the partition
assert body["chunks"][0]["chunk_id"] == "c1"
def test_memory_chunks_route_empty_match_is_200_empty_list():
# The 0-chunks state is a VISIBLE answer (not a 404): count 0, total shows the store.
store = open_memory_store(":memory:", embedding_dim=EMBEDDING_DIM)
app = build_memory_provider_app(store, heimdall_key=b"k")
r = TestClient(app).get("/memory/chunks", params={"end_user_id": "vuong"})
assert r.status_code == 200
body = r.json()
assert body == {"chunks": [], "count": 0, "total": 0}
def test_memory_chunks_route_missing_end_user_id_returns_400():
store = open_memory_store(":memory:", embedding_dim=EMBEDDING_DIM)
app = build_memory_provider_app(store, heimdall_key=b"k")
r = TestClient(app).get("/memory/chunks") # no end_user_id
assert r.status_code == 400
assert r.json()["error_code"] == "missing_end_user_id"
def test_build_memory_app_keeps_bifrost_routes_top_level():
# POST-002 parity with affect D2: add_memory_read_route uses add_route (not Mount),
# so /bifrost/* stay top-level and the op-feed path check still matches them.
store = open_memory_store(":memory:", embedding_dim=EMBEDDING_DIM)
app = build_memory_provider_app(store, heimdall_key=b"k")
paths = {getattr(r, "path", None) for r in app.routes}
assert "/bifrost/handshake" in paths
assert "/bifrost/memory-call" in paths
assert "/memory/chunks" in paths
def test_add_memory_read_route_is_shared_helper_on_bare_app():
# The helper mounts the route on any app (used by both build_memory_provider_app and
# the combined provider) — mirror of add_affect_read_route's shared-helper shape.
from starlette.applications import Starlette
store = open_memory_store(":memory:", embedding_dim=EMBEDDING_DIM)
app = Starlette()
add_memory_read_route(app, store)
r = TestClient(app).get("/memory/chunks", params={"end_user_id": "u"})
assert r.status_code == 200
assert r.json()["total"] == 0
+93
View File
@@ -681,6 +681,7 @@ class TestCreateAppShape:
"/", "/version", "/api/agents", "/api/sessions",
"/api/agents/{agent_id}/persona_state",
"/api/affect/{agent_id}",
"/api/memory/chunks",
# v0.19.2 debug-surface parity (create_app POST-002)
"/api/sessions/{session_id}/tools",
"/api/sessions/{session_id}/bifrost",
@@ -1267,3 +1268,95 @@ class TestAdminEventsEndpoint:
assert "event: stream_error" in body
assert "SseConnectFailed" in body
assert body.count("event: stream_error") == 1 # exactly one, then ends
class TestMemoryChunksEndpoint:
"""memory_chunks_endpoint FN — memory viewer: web proxy to the provider debug read.
Mirrors TestAffectStateEndpoint (the #18-D2 affect proxy shape)."""
@respx.mock
def test_happy_proxies_and_supplies_server_end_user_id(self) -> None:
"""tracer: GET /api/memory/chunks → proxies to the configured provider read URL,
supplying end_user_id SERVER-SIDE and forwarding the browser-named agent_id."""
from ratatoskr.web.server import create_app
payload = {
"chunks": [
{"chunk_id": "c1", "content": "the user's cat is Mochi",
"scope": {"end_user": "vuong", "agent_self": "ratatoskr:sindra"},
"origin": "worldtree", "revision": 1}
],
"count": 1,
"total": 1,
}
route = respx.get(url__regex=r"http://prov:8391/memory/chunks.*").mock(
return_value=httpx.Response(200, json=payload)
)
app = create_app(
_mock_client_factory(),
end_user_id="vuong",
memory_read_url="http://prov:8391",
)
resp = TestClient(app).get("/api/memory/chunks?agent_id=ratatoskr:sindra")
assert resp.status_code == 200
assert resp.json() == payload
assert route.calls.last.request.url.params["end_user_id"] == "vuong"
assert route.calls.last.request.url.params["agent_id"] == "ratatoskr:sindra"
@respx.mock
def test_browser_supplied_end_user_id_is_ignored(self) -> None:
"""The server's configured partition is used; a browser end_user_id is ignored."""
from ratatoskr.web.server import create_app
route = respx.get(url__regex=r"http://prov:8391/memory/chunks.*").mock(
return_value=httpx.Response(200, json={"chunks": [], "count": 0, "total": 0})
)
app = create_app(
_mock_client_factory(), end_user_id="vuong", memory_read_url="http://prov:8391"
)
TestClient(app).get("/api/memory/chunks?end_user_id=attacker&agent_id=a")
assert route.calls.last.request.url.params["end_user_id"] == "vuong"
def test_unconfigured_returns_400(self) -> None:
"""PRE-001: no memory_read_url → 400 memory_not_configured (no silent attempt)."""
from ratatoskr.web.server import create_app
app = create_app(_mock_client_factory(), end_user_id="vuong") # no memory_read_url
resp = TestClient(app).get("/api/memory/chunks?agent_id=a")
assert resp.status_code == 400
assert resp.json()["error_code"] == "memory_not_configured"
def test_no_end_user_configured_returns_400(self) -> None:
from ratatoskr.web.server import create_app
app = create_app(_mock_client_factory(), memory_read_url="http://prov:8391")
resp = TestClient(app).get("/api/memory/chunks?agent_id=a")
assert resp.status_code == 400
assert resp.json()["error_code"] == "memory_not_configured"
@respx.mock
def test_provider_unreachable_returns_502(self) -> None:
from ratatoskr.web.server import create_app
respx.get(url__regex=r"http://prov:8391/memory/chunks.*").mock(
side_effect=httpx.ConnectError("refused")
)
app = create_app(
_mock_client_factory(), end_user_id="vuong", memory_read_url="http://prov:8391"
)
resp = TestClient(app).get("/api/memory/chunks?agent_id=a")
assert resp.status_code == 502
assert resp.json()["error_code"] == "memory_provider_unreachable"
@respx.mock
def test_provider_400_passes_through(self) -> None:
from ratatoskr.web.server import create_app
respx.get(url__regex=r"http://prov:8391/memory/chunks.*").mock(
return_value=httpx.Response(400, json={"error_code": "missing_end_user_id"})
)
app = create_app(
_mock_client_factory(), end_user_id="vuong", memory_read_url="http://prov:8391"
)
resp = TestClient(app).get("/api/memory/chunks?agent_id=a")
assert resp.status_code == 400
Generated
+1 -1
View File
@@ -1052,7 +1052,7 @@ wheels = [
[[package]]
name = "ratatoskr"
version = "0.20.2"
version = "0.20.7"
source = { editable = "." }
dependencies = [
{ name = "httpx" },