# Worldtree Conversation API — Client Interface Specification **Version:** 1.0 **Date:** 2026-04-15 **Status:** Stable (pre-auth) **Contract:** `docs/contracts/conversation_api.contract.md` This document specifies the external interface for clients integrating with the Worldtree Conversation API. It is the reference for building web apps, TUI clients, bridges, or any system that needs to converse with Worldtree agents. ## Base URL ``` http://{host}:{port} ``` Default: `http://127.0.0.1:8080` Start the server: ```bash python -m core.conversation_api python -m core.conversation_api --host 0.0.0.0 --port 8080 ``` ## Authentication API key authentication via the `Authorization` header: ``` Authorization: Bearer sk-your-api-key-here ``` **Configuration** (`config/defaults.yaml`): ```yaml conversation_api: api_keys: - key: "${WORLDTREE_API_KEY_1}" user_id: "admin" label: "Admin key" cors_origins: - "http://localhost:3000" ``` **Dev mode:** When `api_keys` is empty or omitted, auth is disabled — all endpoints are open and CORS allows all origins. This preserves the zero-config development experience. **Session ownership:** Each API key maps to a `user_id`. Sessions are scoped to the user who created them. Users can only see and access their own sessions. Access denials return `404` (not `403`) to avoid leaking session existence. **Errors:** | Status | Condition | |--------|-----------| | `401` | Missing `Authorization` header (auth enabled) | | `401` | Invalid API key | | `404` | Session exists but belongs to another user | --- ## Authorization model — agent invocation When you call `POST /sessions` against an agent, the authorization check that fires depends on **which kind of agent** you target. There are two distinct scope namespaces — the spelling differs by one character (`agent` vs `agents`) and the granting mechanism differs entirely. Confusing the two is a common source of bug reports. ### Tier 1 — foundational agents (no `:` in agent_id) Agents bundled with Worldtree: `mimir`, `lofn`, `soong`, `forseti`, `domari`, `vili`, `actor`, `saga`, `bragi`, `leif`, `troi`, `cara`, `glados`, and any future Asgardian. The agent_id is a simple slug like `mimir` — no colon. > **About tiers:** Your `tier` is set on the `users` table row your API key resolves to, assigned at key-mint time (see `POST /admin/keys`). Tiers are `anonymous` (dev-mode unauthenticated), `user` (default for newly-issued keys), `free`/`pro` (subscription-shaped, not actively differentiated), and `admin`. The tier you have is visible via `GET /me`'s `tier` field. Tier-derived scopes come from `config/policies.yaml > tiers..scopes` — there is no per-key scope override. **Authorization rule (singular `agent`):** ```yaml - id: agent-call-baseline-allow principal: tiers: ["anonymous", "user", "free", "pro", "admin"] action: "agent.call:*" resource: "*" effect: allow ``` This baseline rule lives at `config/policies.yaml`. Every authenticated tier — including the `user` tier that newly-issued keys default to — already passes this check for every Tier 1 agent. **There is no per-agent scope you can add to "grant" Tier 1 access; it's covered by tier.** If you get a 422 calling a Tier 1 agent (e.g., `lofn` rejecting with `end_user_id_required`), that's a **request-body validation**, not a scope denial. Check the `error_code` in the response detail — `END_USER_ID_REQUIRED` means the agent requires an `end_user_id` field in the request body; `AUTH_SCOPE_DENIED` (403) would be the actual scope problem. They're not interchangeable. ### Tier 3 — consumer-defined agents (`:` in agent_id) Agents created at runtime via `POST /agents/define`. The agent_id is `:`, e.g., `acme:support-bot`. The `:` in the path is the trigger that switches the auth model. **Authorization is DB-backed per-resource, NOT policy-driven (plural `agents`):** ``` scope action checked: agents.call:: ^^^^^^ PLURAL — different namespace from Tier 1 ``` There is **no blanket allow rule** for `agents.call:*` in policy. The grant comes from the live `consumer_agents` table: - A non-soft-deleted row in `consumer_agents` owned by `ctx.user_id` IS the grant. - Cascade soft-delete and owner-initiated `DELETE` revoke it. - Missing row → policy defaults to deny (403 `auth_scope_denied`). To "add the scope" for a Tier 3 agent, you don't amend any config or call an admin endpoint — you `POST /agents/define` to register it under your `user_id`. Owning the row IS the grant. You cannot call another user's Tier 3 agent; ownership is checked at session-create (`row.user_id == ctx.user_id`). ### Common pitfalls - **Singular vs plural.** Tier 1 uses `agent.call:*` (singular `agent`). Tier 3 uses `agents.call::` (plural `agents`). One character difference, two completely different mechanisms. There is no Tier 1 scope named `agent.call:mimir` or `agents.call:mimir` — Tier 1 is granted by baseline rule, not per-agent name. - **No scope-mutation API.** `POST /admin/keys` accepts `{user_id, label, tier}` only. There is no per-key scope override mechanism in the storage schema. To change a user's effective scopes, change their `tier`, not their key. Per-resource Tier 3 grants flow through `POST /agents/define` (and its DELETE counterpart), not through admin endpoints. - **422 vs 403.** A 422 is body-validation (e.g., `end_user_id_required`); a 403 is auth-policy denial (`auth_scope_denied`). Different fix paths. Read the `error_code` in `detail`. ### Quick decision table for consumers | Target | Auth requirement | |---|---| | Tier 1 agent (e.g., `mimir`) | Authenticated tier ≥ `user`. No additional body requirements | | Tier 1 agent `lofn` (the default welcoming intermediary) | Authenticated tier ≥ `user` + `end_user_id` field required in request body. 422 `END_USER_ID_REQUIRED` if absent | | Tier 3 agent (any agent_id containing `:`) | `end_user_id` field required in body. AND the row must be owner-matched: `POST /agents/define` first to create a row under your `user_id`, then session-create works against your existing key. Cross-user Tier 3 invocation is rejected with 403 | > **Programmatic discovery of `end_user_id` requirements:** as of v0.22.x there is no field on `GET /agents` indicating which agents require `end_user_id` — the spec line above (lofn + Tier 3) is the authoritative list, and 422 `END_USER_ID_REQUIRED` is the fallback signal at request time. Adding a discoverable `requires_end_user_id` field on `AgentInfoResponse` is on the table as a small future capability; ping if you want to drive it. --- ## GET /me Returns the authenticated principal's identity and key metadata. Lets a client verify its key on boot without triggering agent-config-loading side effects. ``` GET /me Authorization: Bearer (or absent in dev mode) ``` **Auth:** API key via `Authorization: Bearer `. In dev mode (no `api_keys` configured), no header required — returns anonymous shape. **Rate-limit:** Naturally exempt. The per-user budget check is a per-handler call inside `send_message`; GET /me contains no such call and does not consume per-user request budget. **No audit emission.** GET /me does not write audit entries — read-only inspect-self call, same posture as GET /agents. ### Response shapes **Authenticated (real key):** ```json { "user_id": "alice", "scopes": ["agents.converse", "conversations.read", "conversations.write"], "tier": "user", "display_name": "alice phone", "user_created_at": "2026-04-15T10:23:00+00:00", "key_id": "a1b2c3d4", "key_label": "alice phone", "key_created_at": "2026-04-15T10:23:00+00:00" } ``` **Authenticated, key in rotation grace:** ```json { "user_id": "alice", "scopes": ["..."], "tier": "user", "display_name": "alice phone", "user_created_at": "2026-04-15T10:23:00+00:00", "key_id": "a1b2c3d4", "key_label": "alice phone", "key_created_at": "2026-04-15T10:23:00+00:00", "superseded_at": "2026-05-04T17:00:00+00:00", "superseded_by_key_id": "e5f6g7h8", "grace_seconds": 300 } ``` **Anonymous (dev mode, no Authorization header):** ```json { "user_id": "anonymous", "scopes": ["conversations.read", "conversations.write"], "tier": "anonymous" } ``` **Authenticated but Heimdall lookup failed (degraded):** ```json { "user_id": "alice", "scopes": ["..."], "tier": "unknown" } ``` ### Response fields | Field | Type | When present | |-------|------|-------------| | `user_id` | string | Always | | `scopes` | list[string] | Always; alphabetically sorted | | `tier` | string | Always (`"unknown"` on Heimdall failure) | | `display_name` | string | When user record has a non-null display_name | | `user_created_at` | ISO 8601 | When Heimdall user record found | | `key_id` | string | When an active key was resolved | | `key_label` | string | When an active key was resolved and has a label | | `key_created_at` | ISO 8601 | When an active key was resolved | | `superseded_at` | ISO 8601 | When the resolved key is in rotation grace | | `superseded_by_key_id` | string | When the resolved key is in rotation grace | | `grace_seconds` | integer | When the resolved key is in rotation grace | Optional fields are **omitted** (not `null`) when not applicable. ### Key resolution rule (best-effort identification) 1. Fetch all keys for the authenticated `user_id` via `get_api_keys_for_user`. 2. Filter to **active** keys: `disabled_at IS NULL` AND (`superseded_at IS NULL` OR `superseded_at + grace_seconds > now()`). 3. Sort by `last_used_at DESC NULLS LAST`; take the first. 4. If no active key: omit all `key_*` fields; user-derived fields still present. 5. If the resolved key has `superseded_at` set: include `superseded_at`, `superseded_by_key_id`, `grace_seconds`. Heimdall stamps `last_used_at` synchronously on every successful auth. The first row is therefore overwhelmingly likely to be the key that authenticated this request. Documented as **best-effort** — the invariant contracts the resolution rule, not absolute identity. ### Failure mode When the Heimdall user-store lookup raises (DB locked, transient I/O): - The response is a degraded `{user_id, scopes, tier: "unknown"}`. - The error is logged via `logger.exception`. - The status code is still **200** (non-5xx). ### Status codes | Status | Condition | |--------|-----------| | `200` | Success (all response shapes) | | `401` | Missing/invalid Bearer token when auth is enabled | --- ## Admin: API Key Management Runtime key provisioning, listing, and revocation. All endpoints require a `tier=admin` API key. > **Security**: The cleartext key value appears exactly once — in the `POST /admin/keys` response body. It is never re-derivable. Configure your reverse proxy or log pipeline to **redact the response body** of `POST /admin/keys` (e.g., Nginx `proxy_hide_header` or a log scrubber). The `key_suffix` field in list responses is a leak-detection fingerprint — the last 8 chars of the original cleartext, persisted at issuance time. ### Bootstrap: first admin key Set `WORLDTREE_BOOTSTRAP_ADMIN_KEY` before starting the service for the first time: ```bash export WORLDTREE_BOOTSTRAP_ADMIN_KEY="wt_live_$(openssl rand -hex 16)" python -m core.conversation_api ``` The format must be `wt_live_<32 hex chars>`. The service provisions a `bootstrap_admin` user at `tier=admin` with this key on first start. Subsequent restarts are a no-op — the key is not rotated even if the env var changes. To rotate: use `DELETE /admin/keys/{key_id}` on the old key, then `POST /admin/keys` to issue a new one. If `WORLDTREE_BOOTSTRAP_ADMIN_KEY` is unset and no admin user exists, the service starts but logs a CRITICAL warning and admin endpoints return 403 for everyone. ### POST /admin/keys Issue a new API key for a user. **Request:** ```json {"user_id": "alice", "label": "alice phone"} ``` If `user_id` does not exist, a user is auto-created at `tier=user`. **Response (200):** ```json { "key_id": "a1b2c3d4", "key": "wt_live_<32 hex chars>", "user_id": "alice", "label": "alice phone", "key_suffix": "", "created_at": "2026-05-01T10:00:00+00:00", "last_used_at": null, "disabled_at": null, "expires_at": null } ``` The `key` field appears **only here, never again**. Save it immediately. ### GET /admin/keys List all API keys (active and revoked), sorted by `created_at` descending. Never includes the cleartext `key`. **Response (200):** JSON array. Each element has the same shape as the POST response, minus `key`, plus three additional fields: | Field | Type | Description | |-------|------|-------------| | `superseded_at` | ISO 8601 or `null` | When this key was rotated (null if not rotated) | | `superseded_by_key_id` | string or `null` | `key_id` of the replacement key | | `effectively_disabled` | bool | `true` if `disabled_at` is set OR if grace has expired (derived, read-only) | `key_suffix` may be `null` for keys provisioned before schema v2. `effectively_disabled` is computed at response time and does **not** mutate `disabled_at` or any other stored field — use it to see post-grace state without triggering a key lookup. ### DELETE /admin/keys/{key_id} Revoke an API key. Idempotent — revoking an already-revoked key returns 200 with `disabled_at` unchanged. | Status | Condition | |--------|-----------| | `200` | Revoked (or already revoked — idempotent) | | `403` | Caller lacks `admin.keys.revoke` scope | | `404` | `key_id` never existed | **Response (200):** same shape as GET list items, with `disabled_at` now populated. ### POST /admin/keys/{key_id}/rotate Atomically replace one key with another for the same user. The old key stays valid for a configurable **grace window** during which both keys authenticate — giving multi-instance clients time to swap without seeing 401s. After the grace window, the old key is lazy-disabled on its next auth attempt. **Request body (JSON, optional):** | Field | Type | Description | |-------|------|-------------| | `grace_seconds` | int or omit | Grace window in seconds. Range: `[0, 86400]` (inclusive). Omit to use the server default (300 s = 5 min). | `grace_seconds=0` is **compromise mode**: the old key fails on the very next auth attempt. `grace_seconds=86400` gives a 24-hour window for fleet-wide distribution. **Response (200):** same fields as `POST /admin/keys`, plus: | Field | Type | Description | |-------|------|-------------| | `supersedes` | string | `key_id` of the old key that was rotated | | `supersedes_until` | ISO 8601 | When the old key's grace window expires | The `key` field appears **only here, never again**. Configure reverse-proxy log redaction for this endpoint alongside `POST /admin/keys`. **In-flight streams are NOT force-disconnected** — even with `grace_seconds=0`. Auth is enforced at request boundary only. Existing SSE/WebSocket connections using the rotated key continue until they naturally end. **Concurrency:** two simultaneous rotate calls for the same `key_id` are serialized at the database level. The first write wins (200); the second gets 409. | Status | Condition | |--------|-----------| | `200` | Rotation succeeded; new cleartext key in body | | `403` | Caller lacks `admin.keys.write` scope | | `404` | `key_id` never existed | | `409` | Key is already superseded (concurrent or repeat rotate) | | `410` | Key has been revoked (`disabled_at` is set); cannot rotate | | `422` | `grace_seconds` out of `[0, 86400]` range or body validation failed | ### Status codes | Status | Endpoint | Condition | |--------|----------|-----------| | `200` | POST | Key issued (cleartext in body) | | `200` | GET | List returned | | `200` | DELETE | Revoked or already-revoked (idempotent) | | `200` | POST rotate | Rotation succeeded (new cleartext in body) | | `403` | any | Caller lacks required admin scope | | `404` | DELETE | `key_id` never existed | | `404` | POST rotate | `key_id` never existed | | `409` | POST rotate | Key already superseded | | `410` | POST rotate | Key is revoked | | `422` | POST | Body validation failed | | `422` | POST rotate | `grace_seconds` out of range or body validation failed | ### Trust boundary Any `tier=admin` principal can issue, rotate, revoke, and list keys. Granular per-admin scoping is out of scope for v0. --- ## Health & Readiness Two unauthenticated endpoints for k8s probes and load-balancer health checks. Neither carries credentials; both bypass session ownership. ### GET /healthz Liveness probe. Returns `200` unconditionally if the process is responsive. **Response:** `200 OK` ```json {"status": "ok"} ``` Hit very frequently in k8s (default ~10s per pod). Body is intentionally minimal — if the request reached the handler, the process is alive. Don't extend it with deeper checks; that defeats its purpose as a fast liveness probe. ### GET /readyz Readiness probe. Returns `200` only when the service has finished `startup()` and is in a serviceable state; `503` otherwise. **Response (200, ready):** ```json { "status": "ok", "version": "1.0.0", "git_sha": "abc1234" } ``` | Field | Source | |-------|--------| | `version` | `WORLDTREE_VERSION` env var, `"unknown"` if unset | | `git_sha` | `WORLDTREE_GIT_SHA` env var, `"unknown"` if unset | Both env vars are read once at process start; a deploy that bumps either requires a process restart (which is what k8s does on rollout). **Response (503, not ready):** ```json { "detail": { "status": "not_ready", "reason": "startup_incomplete" } } ``` | `reason` value | Meaning | |---|---| | `startup_incomplete` | `ConversationService.startup()` hasn't finished, or `shutdown()` has begun | | `store_unavailable` | SQLite conversation store is not initialised | | `no_agents` | No agents successfully loaded | Probes from inside the cluster typically don't issue OPTIONS preflight, so CORS configuration doesn't affect them. If `/readyz` is exposed to untrusted networks, restrict at the Ingress layer — version + git_sha are deployment metadata, not secrets, but unconditional public access still warrants Ingress-level scoping in hostile environments. --- ## Pending Tasks Surfaces in-flight work that is invisible to the per-turn SSE stream — primarily inter-agent Bus-v2 calls that may run for minutes beneath or between turns. Two endpoints: a per-session view and a cross-session per-user view. Pull-only in v0; SSE push is deferred. **Pending-task visibility is independent of any agent capability** — every agent's Bus-v2 calls are visible to its session's pending list as long as the originating `submit()` carried a `session_id` contextvar. ### Endpoints ``` GET /sessions/{session_id}/pending — per-session active tasks GET /pending — cross-session per-user active tasks ``` Both require `Authorization: Bearer ` and the `pending.read` scope (both user-tier and admin-tier have it by default). Both are structurally exempt from per-user GCRA rate-limiting (INV-068) — see Rate-Limit Exemption below. ### Query parameters (both endpoints) | Parameter | Type | Default | Description | |---|---|---|---| | `limit` | integer | 50 | Max items per page. Range: 1–200. Out-of-range → 422. | | `cursor` | string | — | Opaque pagination cursor from a previous response's `next_cursor`. | ### Response shape ```json { "items": [ , ... ], "next_cursor": " | null" } ``` `next_cursor` is `null` on the last page. Clients MUST treat cursor values as opaque — the encoding is internal and subject to change. ### PendingTask envelope (INV-070 — stable shape) All 8 fields are always present; none are ever omitted. ```json { "task_id": "", "kind": "bus_call", "target_agent_id": "", "started_at": "", "eta_seconds": null, "status": "pending", "session_id": "", "turn_id": "" } ``` | Field | Notes | |---|---| | `task_id` | Unique handle id (UUID hex). | | `kind` | `"bus_call"` in v0. Future values are RESERVED (see below). | | `target_agent_id` | Agent receiving the bus call. | | `started_at` | ISO 8601 UTC timestamp of when `submit()` was called. | | `eta_seconds` | Always `null` in v0 — Bus-v2 has no timer estimates yet. | | `status` | Always `"pending"` — only active (non-terminal) tasks appear. | | `session_id` | The conversation session that originated the call; `null` for programmatic/scheduled submits without session context. | | `turn_id` | The turn that triggered the call; `null` when submitted without turn context. | ### `kind` enum (INV-071 — additive) `"bus_call"` is the only emitted value in v0. The following values are **RESERVED** — future versions may emit them; clients MUST tolerate unknown kinds (treat as opaque, not as errors): | `kind` | Reserved for | |---|---| | `"huginn_job"` | Huginn pipeline DAG jobs | | `"muninn_ingestion"` | Muninn document ingestion jobs | | `"scheduled_task"` | Scheduled/cron agent submissions | ### Scope ``` pending.read ``` User-tier keys have this scope for their own pending tasks. Admin-tier keys have it for all users. Anonymous (dev mode) has `*` which includes `pending.read`. ### Rate-Limit Exemption (INV-068) Both endpoints are structurally exempt from GCRA per-user rate-limiting — the handlers never call `check_rate_limits`. Rationale: a 5s polling cadence across N active sessions would burn the user's request budget at N/5 req/s, pre-empting all message-send capacity. Structural exemption is the only viable design for a polling-based pending UI. ### In-Memory-Only Persistence (INV-067) The pending list reflects in-memory Bus-v2 state and **resets on process restart**. Active tasks themselves are also gone after restart (agents that submitted them must re-submit). Gateway code building long-lived pending UIs **MUST treat empty-after-restart as a normal path**, not as an error. ### Cross-User Isolation (INV-069) Cross-user pending references return 404 on the per-session endpoint (no body that distinguishes missing-vs-cross-user — single-error-path leak prevention per INV-009). The per-user endpoint (`GET /pending`) automatically scopes results to `ctx.user_id` at the bus-filter layer; cross-user data simply does not appear. ### Tasks With Null Session Context Tasks submitted without a `session_id` contextvar (programmatic or scheduled submits) are **excluded** from per-session pending lists. They appear in `GET /pending` for the calling user when `originator_user_id == ctx.user_id`, otherwise excluded. ### Recommended Polling Pseudocode ```javascript async function pollPending(sessionId, intervalMs = 5000) { let cursor = null; while (true) { const url = cursor ? `/sessions/${sessionId}/pending?cursor=${cursor}` : `/sessions/${sessionId}/pending`; const res = await fetch(url, { headers: { Authorization: `Bearer ${apiKey}` } }); if (!res.ok) break; // process restart or session deleted const { items, next_cursor } = await res.json(); if (items.length > 0) render(items); cursor = next_cursor; await sleep(cursor ? 0 : intervalMs); // drain pages, then wait } } ``` --- ## Admin Event Stream A server-sent event stream that broadcasts all Conversation API lifecycle events to admin-tier consumers (gateway, audit tooling). Open once at startup; use `Last-Event-ID` to reconnect without losing events. ### GET /admin/events ``` GET /admin/events Authorization: Bearer Last-Event-ID: (optional, on reconnect) ``` **Auth:** Requires `admin.events.read` scope (admin tier). Returns 403 `auth_scope_denied` if the scope is absent, 401 if the bearer token is missing or invalid. **Rate-limit exempt:** This endpoint never triggers the GCRA rate limiter (structural exemption, same as `/healthz` and the admin key endpoints). **Audit:** Each connection is audited as `conversation_api:admin:events:connect` with the actor's `user_id`. Per-event emission is NOT audited. **Response:** `200 OK` — chunked SSE stream. Each SSE event carries: - `id:` — the event's integer id (monotonic per process, resets on restart) - `data:` — JSON-encoded envelope (see shape below) ### Envelope shape Every event uses a stable, additive envelope (INV-046): ```json { "id": 42, "type": "session.created", "timestamp": "2026-05-06T10:00:00.000Z", "data": { ... } } ``` Fields: `id` (int), `type` (dotted-namespace string), `timestamp` (ISO 8601 with Z suffix), `data` (type-specific object). Field additions are allowed; reordering or renaming are breaking changes. ### v0 event vocabulary | type | data fields | |------|-------------| | `session.created` | `session_id`, `agent_id`, `user_id` (nullable) | | `session.updated` | `session_id`, `changed_fields: list[str]` | | `session.archived` | `session_id` | | `session.unarchived` | `session_id` | | `session.deleted` | `session_id` | | `turn.started` | `session_id`, `turn_id`, `agent_id`, `user_id` (nullable) | | `turn.completed` | `session_id`, `turn_id`, `duration_ms`, `phase: "succeeded"` | | `turn.failed` | `session_id`, `turn_id`, `duration_ms`, `phase: "failed"`, `error: str` | | `turn.cancelled` | `session_id`, `turn_id`, `duration_ms`, `phase: "cancelled"`, `reason: "user_cancel"` | | `turn.stalled` | `session_id`, `turn_id`, `duration_ms`, `phase: "stalled"`, `reason: "stall"` | | `key.issued` | `key_id`, `target_user_id`, `actor_user_id` | | `key.revoked` | `key_id`, `target_user_id`, `actor_user_id` | | `key.rotated` | `old_key_id`, `new_key_id`, `target_user_id`, `actor_user_id` | | `system.startup` | `version`, `git_sha`, `first_event_id` | | `system.heartbeat` | _(empty object)_ | | `system.shutdown` | _(empty object)_ | | `system.events_dropped` | `count`, `gap_first_id`, `gap_last_id` | | `system.replay_gap` | `requested_id`, `available_from_id` | **Events fire AFTER the canonical SQL commit** (INV-048): a consumer who receives `session.created` can immediately query `GET /sessions/{id}` and find the row. **PII discipline** (INV-049): events carry IDs and small metadata only — never message content, tool-call payloads, cleartext API keys, partial assistant output, or persona affect state. ### In-memory ring buffer The bus holds the last 1000 events in memory (INV-050). No persistence; buffer resets on process restart. Per-consumer queues are bounded at 1000 items (INV-051). ### Last-Event-ID resume semantics On reconnect, pass the last-seen event id as a plain decimal integer: ``` GET /admin/events Last-Event-ID: 42 ``` Server behaviour: | Condition | Response | |-----------|----------| | `id` within buffer (`buffer_floor ≤ id < buffer_top`) | Replay events with `id > Last-Event-ID`, then resume live | | `id == buffer_top` | No replay; resume live silently | | `id < buffer_floor` (events evicted from buffer) | Emit `system.replay_gap`, then resume live | | `id > buffer_top` (e.g. id from a prior process run) | Emit `system.replay_gap`, then resume live | | Header absent | No replay; start from next live event | `system.replay_gap` data: `{requested_id, available_from_id}` — tells the client where the buffer starts so it can decide whether to re-request from that point. A malformed `Last-Event-ID` (non-integer, negative) returns 400 `malformed_request`. ### Queue overflow and system.events_dropped Each subscriber has a bounded `asyncio.Queue(maxsize=1000)`. If a consumer is too slow and the queue fills: 1. The oldest item is popped from the subscriber's queue. 2. A `system.events_dropped` event is enqueued in its place: `{count, gap_first_id, gap_last_id}`. 3. The incoming event is dropped (not enqueued). Drop notifications are best-effort — a persistently-slow consumer can lose them too. ### Heartbeat `system.heartbeat` is emitted to each subscriber's queue every 30 seconds when no other events have arrived. This keeps long-idle connections alive through NATs and proxies. Heartbeat events have `id=0` and are not in the ring buffer (not replayable). ### Example JS client ```js let lastId = localStorage.getItem("lastEventId") ?? undefined; function connect() { const headers = { Authorization: `Bearer ${ADMIN_KEY}` }; if (lastId) headers["Last-Event-ID"] = lastId; const source = new EventSource("/admin/events"); // headers via fetch+ReadableStream // (EventSource doesn't support custom headers; use fetch with stream reader in practice) source.onmessage = (e) => { const event = JSON.parse(e.data); lastId = String(event.id); if (lastId !== "0") localStorage.setItem("lastEventId", lastId); switch (event.type) { case "session.created": handleSessionCreated(event.data); break; case "turn.completed": handleTurnCompleted(event.data); break; case "system.replay_gap": console.warn("Gap detected; missed events from", event.data.requested_id, "to", event.data.available_from_id); break; case "system.events_dropped": console.warn("Slow consumer: dropped", event.data.count, "events"); break; } }; source.onerror = () => { source.close(); setTimeout(connect, 3000); // reconnect with Last-Event-ID set }; } connect(); ``` --- ## Admin inspection endpoints Two read-only admin-tier endpoints for diagnosing Bifrost-bound sessions (issue #176). Both require the `admin.sessions.read` scope and emit a single audit-store entry per successful call. They never mutate session state, re-handshake, or invoke a tool. ### GET /admin/sessions/{session_id}/bifrost ``` GET /admin/sessions/{session_id}/bifrost Authorization: Bearer ``` **Auth:** Requires `admin.sessions.read` scope (admin tier). Returns 403 `auth_scope_denied` if the scope is absent, 401 if the bearer token is missing or invalid. **Audit:** Each success is audited as `conversation_api:admin:session:read.bifrost` with `extra.target_session_id` set. Failure paths (403/404) are not audited at the resource layer; the auth layer audits auth failures. **Response (200):** ```json { "endpoint_url": "https://bifrost.example/mcp", "consumer_id": "alice", "connected": true, "capabilities_granted": ["tools:call", "tools:read"], "tools": [ {"name": "bifrost.alice.echo", "description": "echo tool"} ] } ``` - `tools` is sourced from `session.bifrost_tools` (the post-tier-filter / post-namespace list the agent actually receives at turn-fire time), NOT from the raw handshake list. Tools that the tier filter excluded never appear here. - `connected` reflects the live `BifrostClient._is_connected` flag, which may lag a silently-dropped TCP socket by several seconds. Graceful-on-stale: when the connection has died but the live client is still registered, the response returns `connected: false` with the cached tool list intact rather than re-handshaking or returning 503. **Errors:** | Status | `error_code` | When | |---|---|---| | 401 | `auth_missing` / `auth_invalid` | Missing or invalid bearer token | | 403 | `auth_scope_denied` | Caller lacks `admin.sessions.read` scope | | 404 | `session_not_found` | `session_id` does not exist | | 404 | `session_not_bifrost_bound` | Session exists but has no live `_bifrost_clients` entry (covers both never-bound sessions and restored sessions whose live client is gone post-restart) | ### GET /admin/sessions/{session_id}/tools ``` GET /admin/sessions/{session_id}/tools Authorization: Bearer ``` **Auth:** Same as `/bifrost`. **Audit:** Each success is audited as `conversation_api:admin:session:read.tools`. **Response (200):** ```json { "agent_id": "mimir", "builtin_tools": [ {"name": "search_kb", "description": "..."} ], "bifrost_tools": [ {"name": "bifrost.alice.echo"} ] } ``` - `builtin_tools` is the agent's native tool schemas at session-create time. - `bifrost_tools` mirrors the actual turn-time merge predicate (`session.bifrost_tools` non-empty AND a live `_bifrost_clients` entry exists); otherwise `[]`. Sessions whose bifrost tool list was persisted but whose live client has been evicted (post-restart restore) report `bifrost_tools: []`, faithfully reflecting what the agent will see at the next turn. - Per-message override tools (issue #166) are turn-scoped and never appear in this response. **Errors:** same shape as `/bifrost`, minus `session_not_bifrost_bound` — `/tools` works on any existing session. --- ## Reconnect & Resume The SSE transport supports mid-turn reconnect without re-running the agent. Each event carries a monotonic id; a reconnecting client provides `Last-Event-ID` and the server replays buffered events before resuming live streaming. ### SSE id format Every SSE event has an `id:` field in the wire format: ``` id: {turn_id}:{seq} ``` - `turn_id` — integer from SQLite `turns.id` (same as today; used for the cancel endpoint path parameter) - `seq` — per-turn monotonic integer starting at `1`, incremented for every yielded event. Resets to `1` on each new turn. Example: ``` id: 42:3 data: {"type": "text", "content": "..."} ``` Clients parse `turn_id` from the prefix for cancellation. The combination `{turn_id}:{seq}` is unique within a session. ### Reconnect flow Set `Last-Event-ID` on a re-`POST /sessions/{session_id}/messages` request to resume an in-flight turn: ``` POST /sessions/{session_id}/messages Last-Event-ID: 42:3 Content-Type: application/json {"content": "..."} ``` The server: 1. Validates the header format and that `turn_id` matches the resume target. 2. Replays buffered events with `seq > Last-Event-ID.seq`. 3. Drains the live queue until the turn's terminal event (`done`/`cancelled`/`error`). The agent's tools and LLM call run exactly once regardless of how many disconnects/reconnects occur. ### Status codes for resume requests | Status | Condition | Body | |---|---|---| | 200 | Resume succeeded; events follow as SSE | (SSE stream) | | 400 | `Last-Event-ID` malformed or `turn_id` mismatch | `{"error": "invalid_last_event_id"}` | | 404 | Session not found / different user / turn belongs to a different session | `{"detail": "Session ... not found"}` | | 410 | Turn finished and cleaned up | `{"error": "turn_finished", "turn_id": N}` | | 412 | `Last-Event-ID` older than replay buffer | `{"error": "buffer_expired", "buffered_from_seq": X, "turn_id": N}` | ### Replay buffer The server keeps the last **10 events** per active turn in an in-memory replay buffer. Buffer retention is by event count, tied to the turn's lifetime — there is no time-based expiry. The buffer does **not** survive process restart. A reconnect after a server restart returns HTTP 410; the client should start a fresh turn. ### Client reconnect guidance Recommended client-side reconnect timeout: 30 seconds. After a drop, re-POST with `Last-Event-ID` set to the last `id:` value received. On 410, start a new turn. On 412, start a new turn (buffer gap too large to replay). --- ## Rate Limiting Three enforcement layers protect the upstream LLM provider. All apply only to `POST /sessions/{session_id}/messages`; probe endpoints (`/healthz`, `/readyz`) and admin endpoints (`/admin/keys/*`) are exempt. ### Scopes | Scope | Key | Mechanism | |---|---|---| | Per-user request rate | `conv:req:{user_id}` | GCRA, N req/min per authenticated user | | Per-user token rate | `conv:tok:{user_id}` | GCRA, N tokens/hour per user (post-charged) | | Per-provider concurrency | in-memory Semaphore | cap on simultaneous in-flight LLM calls per provider | Anonymous (unauthenticated) requests share buckets `conv:req:anonymous` and `conv:tok:anonymous`. ### Configuration (`config/defaults.yaml`) ```yaml conversation_api: rate_limits: per_user_req_per_min: 60 # 60 turn submissions per user per minute per_user_tokens_per_hour: 100000 # 100k total tokens per user per hour per_provider_concurrency: default: 10 # cap for any provider not listed anthropic: 20 openai_compat: 15 anonymous: req_per_min: 30 tokens_per_hour: 20000 ``` Omit the entire `rate_limits` block to disable all enforcement (compatible with current dev behavior). ### 429 Response When any limit is exceeded: ```json { "detail": "rate_limited", "scope": "per_user_req_per_min", "retry_after_s": 12.4 } ``` HTTP headers: - `Retry-After: 13` — integer seconds, ceiling of `retry_after_s` (RFC 7231) `scope` is one of: `per_user_req_per_min`, `per_user_tokens_per_hour`, `per_provider_concurrency`. For `per_provider_concurrency`, `retry_after_s` is `0` — the right action is to retry immediately (slot may free at any moment). ### Successful response headers (X-RateLimit-*) Every successful `POST /sessions/{session_id}/messages` response includes: ``` X-RateLimit-Limit-User-Req: 60 X-RateLimit-Remaining-User-Req: 47 X-RateLimit-Limit-User-Tokens: 100000 X-RateLimit-Remaining-User-Tokens: 87234 ``` Provider concurrency is not surfaced — it is not a clock-based bucket and a "remaining slots" value is stale immediately. ### Token-rate post-charge The per-user token rate is **post-charged**: tokens are debited AFTER the turn completes. A single long turn can over-spend the bucket by any amount. The next turn's pre-check will see the debt and return 429. Recovery requires natural refill; there is no special-casing. Operators wanting hard per-turn token ceilings should pair this with the cost-cap feature (#114). Cancel mid-turn does **not** refund tokens (if the LLM call completed before cancel took effect). Predictable and matches the cost-cap principle. --- ## Endpoints ### GET /agents List all available agents with their metadata. **Response:** `200 OK` — JSON array of agent objects. Fields are **omitted when null or empty** (no null values in the response). **Minimum shape** (agent has only the three required fields): ```json [ { "agent_id": "minimal", "name": "Minimal Agent", "description": "Just a sketch." } ] ``` **Full shape** (agent has all populatable fields): ```json [ { "agent_id": "mimir", "name": "Mimir", "description": "Keeper of the Well of Knowledge...", "version": "0.2.0", "capabilities": ["knowledge_base", "semantic_search", "version_history"], "supported_models": ["default", "heavy", "summarizer", "fast"], "persona_traits": { "ocean": { "openness": 0.7, "conscientiousness": 0.9, "extraversion": 0.1, "agreeableness": 0.5, "neuroticism": 0.3 }, "vibe": "contemplative" }, "ui_hints": { "icon": "well", "color_hint": "#5b8aa3" } } ] ``` **Fields:** | Field | Type | Always present | Source | |-------|------|----------------|--------| | `agent_id` | string | yes | `agent.id` in config | | `name` | string | yes | `agent.name` in config | | `description` | string | yes | `agent.description` in config | | `version` | string | no | `agent.version` in config | | `capabilities` | list[str] | no | `agent.capabilities` in config; free-form strings | | `supported_models` | list[str] | no | `llm_profiles` keys (profile names, not resolved model IDs) | | `persona_traits` | object | no | Only when `persona.enabled: true`; contains `ocean` (5 floats) and optionally `vibe` | | `ui_hints` | object | no | From `agent.ui_hints` block; contains `icon` and/or `color_hint` | **Omit-when-null and omit-when-empty rules:** - A field is omitted entirely when its source data is null, absent, or empty. - `capabilities: []` and absent `capabilities` both result in the field being omitted. - `supported_models: []` likewise. - `persona_traits` is omitted entirely when `persona.enabled` is not `true`. - `ui_hints` is omitted entirely when neither `icon` nor `color_hint` is configured. - Within `ui_hints`, subfields are independently omitted when unset. - `vibe` (configured under `agent.ui_hints.vibe`) is returned under `persona_traits`, not `ui_hints`. It is absent from the response if `persona.enabled` is not `true`. **Recommended `capabilities` vocabulary** (non-enforced): `knowledge_base`, `semantic_search`, `kb_search`, `version_history`, `web_research`, `code_generation`, `image_generation`, `voice_io`, `chat`, `mechanics_analysis` --- ### POST /sessions Create a new conversation session with an agent. **Request:** ```json { "agent_id": "mimir" } ``` **Optional Bifrost binding** (issue #160): include a `bifrost` field to bind consumer-side tools via the MCP-in-reverse protocol. Worldtree runs the handshake synchronously before returning 201 — on failure a 502 is returned and no session is created. ```json { "agent_id": "mimir", "bifrost": { "endpoint_url": "https://consumer.example.com/mcp", "scope": "vor-frame-locking" } } ``` **Bifrost field validation:** - `endpoint_url`: required, must be an HTTPS URL. - `scope`: optional, ≤ 256 chars, opaque string passed through to the JWT payload unchanged. - Bifrost binding is **incompatible with ephemeral (Saga) sessions** — returns 422 `ephemeral_does_not_accept_bifrost`. - Requires the `bifrost:invoke` scope (included in the `user` tier by default). **Response:** `201 Created` ```json { "session_id": "550e8400-e29b-41d4-a716-446655440000", "agent_id": "mimir", "message_count": 0, "created_at": "2026-04-15T12:00:00+00:00", "last_active": "2026-04-15T12:00:00+00:00", "metadata": {} } ``` **Errors:** | Status | Condition | |--------|-----------| | `404` | Unknown agent_id | | `422` | Missing or invalid request body | | `502` | Bifrost handshake failed; `error_code: "bifrost_handshake_failed"`, `detail.bifrost_error` carries the spec-level code | --- ## Tool Surface (Bifrost-bound sessions, issue #160) When a session was created with a `bifrost` binding, the agent's tool list for every turn in that session includes the consumer's tools alongside platform tools. Consumer tools are namespaced as `bifrost..` to prevent collision. **Reentrancy cap**: at most 25 Bifrost tool invocations per agent turn. The 26th invocation returns an error without contacting the consumer (`bifrost.reentrancy_cap_exceeded`). Counter resets per turn. **Tier filtering**: tools tagged with `bifrost_required_tier` are filtered at session-create time. Tools requiring a higher tier than the session creator's tier are absent from the tool list for the session's lifetime. Mid-session tier changes do NOT re-evaluate the filter. **Audit log**: `turn.completed` events for Bifrost-using turns include a `bifrost_calls` array with per-call metadata: `name`, `duration_ms`, `success`, `layer1_error`, `layer2_error`, `redacted`. Arguments and results are redacted by default unless the consumer's tool schema sets `bifrost_log_arguments: true`. **RS256 consumer registration (Bifrost v0.2, issue #165):** By default, the platform uses HS256 JWT signing (shared-secret model — the consumer's Bearer token is also the JWT signing key). For consumers that require asymmetric JWT verification (stricter security posture, no shared secret), operators can register an RS256 keypair: ```bash python scripts/heimdall_generate_bifrost_keypair.py --user-id ``` This generates an RSA-2048 private key, stores it in the Heimdall DB under the consumer's user record (`bifrost_jwt_algorithm = 'RS256'`), and prints the corresponding public PEM to stdout. The operator delivers the public PEM out-of-band to the consumer's MCP server operator, who configures it for JWT verification. Once registered, all session-creates for that consumer use RS256 JWT signing automatically. The handshake envelope includes `auth.jwt_alg: "RS256"` and requests the `"rs256-auth"` capability. If the consumer's server does not grant `"rs256-auth"`, the handshake fails with `bifrost.auth_misconfigured` (no silent fallback to HS256). The private key is never logged, never included in API responses, and the `User` record's `repr()` redacts it. To rotate: re-run the script (prompts for confirmation when stdin is a TTY; requires `--force` in non-interactive mode). --- ### GET /sessions List sessions (scoped to authenticated user). Archived sessions are **hidden by default**. Results are cursor-paginated newest-first. **Query parameters:** | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `include_archived` | bool | `false` | When `true`, includes archived sessions in the response | | `limit` | int | `50` | Page size. Range `[1, 200]`. Values outside range return 422. | | `cursor` | string | *(absent)* | Opaque pagination cursor from a previous response's `next_cursor`. Absent = first page. | **Response:** `200 OK` ```json { "items": [ { "session_id": "550e8400-...", "agent_id": "mimir", "created_at": "2026-04-15T12:00:00+00:00", "last_active": "2026-04-15T12:05:00+00:00", "metadata": {"model": "glm5-turbo"}, "name": "Research session", "archived": false, "tags": ["work", "urgent"] } ], "next_cursor": "v1.eyJjIjoiMjAyNi0wNC0xNVQxMjowMDowMCswMDowMCIsImkiOiI1NTBlODQwMC4uLiJ9" } ``` `next_cursor` is `null` on the last page. `message_count` is not included in list items. **Note:** This is a backwards-incompatible shape change. To retrieve archived sessions, pass `?include_archived=true`. **Errors:** | Status | `error_code` | Condition | |--------|-------------|-----------| | 422 | `validation_failed` | `?limit` out of range | | 422 | `cursor_invalid` | `?cursor` is malformed or wrong version | --- ## Session Mutation ### PATCH /sessions/{session_id} Mutate one or more fields on an existing session. All fields are optional; omitted fields are not changed. An empty body `{}` is a valid no-op. Returns the full updated session info on success. **Request:** ```json { "name": "My research chat", "archived": true, "tags": ["work", "urgent"], "metadata": {"client": "web", "old_key": null} } ``` **Field semantics:** | Field | Type | Semantics | |-------|------|-----------| | `name` | `string \| null` | Human-readable label. `null` clears it. Max 200 chars. | | `archived` | `boolean` | Hides from default `GET /sessions` list. `null` rejected (use `false`). | | `tags` | `array[string]` | **Replace semantics** — the new list IS the new set. `null` rejected. | | `metadata` | `object` | **Merge with null-deletion** — keys set to `null` are deleted; other keys add/replace. `null` rejected. | **Tag validation:** each tag is stripped of whitespace, then rejected if empty, contains non-ASCII-printable chars (outside `[\x20-\x7E]`), exceeds 50 chars, or if the final deduped list exceeds 20 tags. **Metadata:** keys starting with `_` are reserved for internal use and rejected with 422. **Metadata size cap:** 16 KiB serialized JSON after merge. Enforced post-merge to defend against incremental growth. **Response:** `200 OK` — full updated session info shape ```json { "session_id": "550e8400-...", "agent_id": "mimir", "message_count": 3, "created_at": "2026-04-15T12:00:00+00:00", "last_active": "2026-04-15T12:10:00+00:00", "metadata": {"client": "web"}, "name": "My research chat", "archived": true, "tags": ["work", "urgent"] } ``` **Errors:** | Status | Condition | |--------|-----------| | `404` | Session not found or caller doesn't own it | | `422` | Unknown field, name too long, tag invalid, metadata size cap exceeded, `_`-prefixed metadata key, `null` for `archived`/`tags`/`metadata` | **Notes:** - Archived sessions are still accessible via direct `GET /sessions/{id}` and can still receive turns via `POST /sessions/{id}/messages`. Archive only affects the list endpoint. - Ownership uses the same `_check_session_access` pattern as other session endpoints — owning the session is sufficient; no additional scopes required. --- ### GET /sessions/{session_id} Get session info (no messages). **Response:** `200 OK` — same shape as items in `GET /sessions` **Errors:** `404` if session not found. --- ### DELETE /sessions/{session_id} End and remove a session. **Response:** `204 No Content` **Errors:** `404` if session not found. --- ### GET /sessions/{session_id}/messages Get paginated message history for a session (chronological, oldest first). **Query parameters:** | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `limit` | int | `50` | Page size. Range `[1, 200]`. | | `cursor` | string | *(absent)* | Opaque cursor from a previous response's `next_cursor`. | **Response:** `200 OK` ```json { "session_id": "550e8400-...", "items": [ {"role": "user", "content": "What notes do we have about OAuth?", "seq": 0}, {"role": "assistant", "content": "I found 3 notes related to OAuth...", "seq": 1} ], "next_cursor": null } ``` `session_id` is always present in the body (for client-side response indexing). `message_count` is not included. `next_cursor` is `null` on the last page. **Invariant:** Messages are user/assistant pairs in chronological order. Tool call intermediates are not persisted. **Errors:** | Status | `error_code` | Condition | |--------|-------------|-----------| | 404 | `session_not_found` | Session not found or not owned by caller | | 422 | `validation_failed` | `?limit` out of range | | 422 | `cursor_invalid` | `?cursor` is malformed or wrong version | **Note:** This endpoint is distinct from SSE/Last-Event-ID resume (`POST /sessions/{id}/messages`). Pagination is for browsing message history; SSE resume is for in-flight turn streams. --- ### POST /sessions/{session_id}/messages Send a message and receive the agent's response as a Server-Sent Events (SSE) stream. **Request:** ```json { "content": "Search for notes about authentication" } ``` **Response:** `200 OK` with `Content-Type: text/event-stream` The stream emits JSON-encoded events. Each SSE `data:` line contains a JSON object with a `type` field. **Errors:** `404` if session not found (returned before stream starts). --- ## Pagination List endpoints (`GET /sessions`, `GET /sessions/{id}/messages`) use **cursor-based forward-only pagination**. Cursors are opaque to clients — do not parse or construct them. ### Cursor envelope ``` v1. ``` The `v1.` prefix is a version handshake. The body is a base64url-encoded JSON object (RFC 4648 §5, no `=` padding). The JSON keys are server-internal and subject to change without notice. ### Query parameters | Parameter | Default | Max | Description | |-----------|---------|-----|-------------| | `?limit=N` | `50` | `200` | Page size. Values outside `[1, 200]` yield 422. | | `?cursor=` | *(absent)* | — | Opaque token from `next_cursor`. Absent = first page. | ### Response shape **GET /sessions:** ```json { "items": [ /* session info objects */ ], "next_cursor": "" | null } ``` **GET /sessions/{id}/messages:** ```json { "session_id": "", "items": [ /* message objects with role, content, seq */ ], "next_cursor": "" | null } ``` `next_cursor` is **always present** on paginated responses — `null` signals the last page, a string signals more pages available. ### Semantics - **Forward-only, exclusive anchor:** the cursor encodes the last item of a page; the next page starts _after_ that item (anchor row not re-emitted). - **Sessions** sort `created_at DESC, session_id DESC` (newest first). - **Messages** sort `seq ASC` (chronological). - **Filter-agnostic:** the cursor encodes only sort-key fields, not filter state. `?include_archived` and other filters apply fresh on every request. - **Stale anchor:** if the anchor row is deleted between requests, the server returns the next available items silently — no error. - **Forged cursors:** safe; `user_id` scoping is always enforced server-side regardless of cursor content. ### Error code | `error_code` | HTTP | Condition | |-------------|------|-----------| | `cursor_invalid` | 422 | Cursor is malformed, wrong version, or has missing/wrong-typed fields | ### Forward iteration (client pseudocode) ```python cursor = None while True: resp = GET /sessions?limit=50&cursor={cursor} for item in resp["items"]: process(item) cursor = resp["next_cursor"] if cursor is None: break ``` **Cross-endpoint cursor reuse:** `GET /sessions` cursors encode `(created_at, session_id)` pairs; `GET /sessions/{id}/messages` cursors encode `seq` integers. They are not interchangeable. Issue #122 (cross-session search) reuses the same `v1.` envelope with different JSON body keys. --- ## Search `GET /search` performs cross-session full-text search over message content. The backend is **SQLite FTS5** in v0. Client code MUST treat the response as backend-agnostic — the wire shape will remain stable if the backend migrates to Postgres FTS or an external search engine. ### Endpoint ``` GET /search ``` **Required scope:** `search.read` **Rate-limit:** applied (same per-user GCRA bucket as `POST /sessions/{id}/messages` — not exempt). **No audit entry written** for search calls (read-only, high-volume; differs from upload mutations which are audited). ### Query parameters | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `?q=` | string | required | FTS5 query (1–200 chars). Supports plain words, phrase, boolean, prefix. | | `?limit=N` | int | `50` | Page size. Range `[1, 200]`. | | `?cursor=` | string | absent | Opaque cursor from `next_cursor` (first page = absent). | | `?session_id=` | string | absent | Restrict results to one session. | | `?after=` | string | absent | Only messages with `created_at >= after` (excludes NULL-timestamp legacy rows). | | `?before=` | string | absent | Only messages with `created_at <= before` (excludes NULL-timestamp legacy rows). | | `?role=user\|assistant` | string | absent | Restrict results to one message role. | | `?include_archived=false` | bool | `true` | Set to `false` to hide hits from archived sessions. Default **includes** archived (differs from `GET /sessions` where archived are excluded by default — divergence is search-consistency-by-design). | ### FTS5 query syntax The `?q=` value is passed directly to SQLite FTS5 MATCH. Supported syntax: | Syntax | Example | Matches | |--------|---------|---------| | Plain word | `auth` | messages containing the word "auth" | | Phrase | `"foo bar"` | exact two-word phrase | | Boolean AND | `foo AND bar` | both words present | | Boolean OR | `foo OR bar` | either word present | | Boolean NOT | `foo NOT bar` | "foo" without "bar" | | Prefix | `auth*` | "auth", "authenticate", "authority", … | Implicit AND applies when no operator is specified between words (`foo bar` = `foo AND bar`). On FTS5 syntax errors (e.g. unbalanced quotes) the server returns `422 search_query_invalid` with the FTS error message in `detail.message`. ### BM25 ranking Results are sorted by **BM25 score ASC, then message_id DESC**. BM25 scores are negative floats (SQLite convention — lower value = better match, typically in the range −1 to −20). The `score` field in the response carries the raw BM25 value for client-side debugging; clients MUST NOT rely on the numeric range remaining stable across backend changes. ### Response shape ```json { "items": [, ...], "next_cursor": "" | null } ``` `next_cursor` follows the same cursor envelope as other paginated endpoints: `null` = last page, string = more pages available. ### SearchHit envelope (8 fields, always present) ```json { "session_id": "", "message_id": , "agent_id": "", "role": "user" | "assistant", "content": "", "snippet": "", "score": , "created_at": "" | null } ``` - **`snippet`** — FTS5-generated excerpt with `` / `` markers around matched tokens and `…` for truncation (16 tokens of context). Strip or render markers as appropriate for your UI. - **`created_at`** — `null` for legacy rows that predate the FTS migration. Date filters (`?after=`, `?before=`) automatically exclude NULL-timestamp rows. - **`agent_id`** — agent that owns the session (joined from `sessions` at query time). ### Legacy `created_at` caveat The `messages.created_at` column was added by the FTS migration. Messages inserted before the migration have `created_at = null`. These messages ARE returned by unfiltered searches but are excluded from `?after=` / `?before=` date-windowed queries. Clients that need to filter by date can only rely on the timestamp for messages created after the migration was applied. ### Error codes | `error_code` | HTTP | Condition | |-------------|------|-----------| | `search_query_invalid` | 422 | FTS5 syntax error in `?q=` (e.g. unbalanced quote) | | `cursor_invalid` | 422 | Malformed, wrong-version, or wrong-schema cursor | | `validation_failed` | 422 | `?q=` empty or >200 chars; invalid date format for `?after=`/`?before=` | | `auth_scope_denied` | 403 | Caller lacks `search.read` scope | | `rate_limited` | 429 | Per-user request budget exceeded | ### Example: curl ```bash # Basic search curl -H "Authorization: Bearer $KEY" \ "https://api.example.com/search?q=authentication" # Phrase + session filter + date window curl -H "Authorization: Bearer $KEY" \ "https://api.example.com/search?q=%22foo+bar%22&session_id=s123&after=2026-01-01T00:00:00Z" # Prefix search excluding archived sessions curl -H "Authorization: Bearer $KEY" \ "https://api.example.com/search?q=auth*&include_archived=false" ``` ### Example: JavaScript pagination loop ```javascript async function* searchAll(key, query, opts = {}) { let cursor = null; do { const params = new URLSearchParams({ q: query, limit: 50, ...opts }); if (cursor) params.set('cursor', cursor); const resp = await fetch(`/search?${params}`, { headers: { Authorization: `Bearer ${key}` } }); if (!resp.ok) throw new Error(`search failed: ${resp.status}`); const data = await resp.json(); yield* data.items; cursor = data.next_cursor; } while (cursor !== null); } // Usage for await (const hit of searchAll(apiKey, 'authentication')) { console.log(hit.session_id, hit.snippet); } ``` --- ## Tool-Call Persistence Optional per-session persistence of tool-call metadata. By default tool calls live only on the live SSE stream and are dropped after the turn (INV-002 default). Sessions opting in via `record_tool_intermediates: true` get a queryable history of every tool call the agent made — **metadata only**, no arguments, no results (INV-082 PII discipline). ### Opting in ```bash curl -X POST $BASE/sessions \ -H "Authorization: Bearer $KEY" \ -d '{"agent_id": "mimir", "record_tool_intermediates": true}' ``` The flag is **sticky** for the session's lifetime (INV-081). It cannot be flipped on or off mid-session via PATCH; rotate to a new session if you need to change recording disposition. Default is `false`. ### `GET /sessions/{session_id}/tool-events` Returns paginated tool-call metadata for a session. **Query params:** - `limit` — page size, default 50, max 200 - `cursor` — opaque pagination token from a prior response's `next_cursor` **Response:** ```json { "session_id": "sess_…", "items": [ { "tool_call_id": "tc_abc123", "session_id": "sess_…", "turn_id": 17, "tool_name": "search_kb", "started_at": "2026-05-07T03:00:00Z", "ended_at": "2026-05-07T03:00:00.042Z", "status": "completed", "error_type": null, "error_msg": null, "duration_ms": 42.3 } ], "next_cursor": "eyJ2I…" } ``` **`status` values:** - `started` — write hit but `tool_result` hasn't fired yet (transient; you usually won't see this in a list call) - `completed` — tool returned successfully - `failed` — tool raised; `error_type` and `error_msg` (truncated to ≤200 chars) populated - `cancelled` — the turn was cancelled before the tool finished. `ended_at` is **NULL** in this case (INV-086) — the truth is that we don't know when the tool would have ended ### Auth - Scope required: `tool_events.read` (granted to all tiers) - Per-session ownership enforced at the handler — non-owners get 404 (never leak existence) - Rate-limit applied (NOT exempt — same as `/messages` listing) ### Errors - `401` — missing bearer - `403 auth_scope_denied` — token lacks `tool_events.read` - `404 session_not_found` — unknown session OR cross-user (no leak) - `422 cursor_invalid` — malformed pagination cursor - `429 rate_limited` — per-user request budget exhausted ### Sessions opted out Sessions created without the flag (the default) ALWAYS return `{items: []}` — no tool events are written for them. The endpoint is safe to call against any session; it just returns empty. ### What is NOT stored By design (INV-082), the `tool_events` table never contains: - Tool **arguments** (the LLM's call payload) - Tool **results** (whatever the tool returned) - Cleartext API keys, persona affect state, LLM tokens This mirrors the admin event stream's PII discipline (INV-049). If you need to capture call arguments or results for compliance, capture them at the gateway side from the live SSE stream. ### Retention v0 has no retention policy — `tool_events` rows live forever until manually cleaned. Configurable retention is deferred to issue #133 (audit-log retention). Operators with high-traffic deployments should plan accordingly. ### Cancelled-mid-flight semantics If a turn is cancelled while a tool call is in flight (the tool's `started` row was written but `tool_result` hasn't fired yet), the row gets: - `status='cancelled'` - `ended_at=NULL` (per INV-086 — the truth) - `duration_ms=` elapsed wall time from start to cancel (debugging context) This captures the audit-trail truth without forcing a misleading 'completed'. ### Admin event When a session is created with `record_tool_intermediates=true`, the admin event stream (`GET /admin/events`) emits one `session.tool_recording_enabled` event with `{session_id, agent_id, user_id}`. Per-tool-call writes do NOT emit admin events (INV-087) — they would dwarf the stream. ### Example client (JS) ```js const resp = await fetch(`${base}/sessions/${sid}/tool-events?limit=20`, { headers: { Authorization: `Bearer ${key}` } }); const { items, next_cursor } = await resp.json(); for (const tc of items) { console.log(`${tc.tool_name} (${tc.status}, ${tc.duration_ms}ms)`); } // follow next_cursor for additional pages ``` --- ## Ephemeral Templates (issue #161) Ephemeral templates are a second tier of agent, distinct from foundational persistent agents (Mimir, Soong, etc.). They have no persona, no memory, no tools, and no motivational context. The consumer supplies the system prompt and (optionally) the model at session-create time; that config is frozen for the session's lifetime. **Saga** is the first ephemeral template — Norse goddess of history and chronicle, a blank-slate actor that becomes whatever the consumer's system prompt instills. ### Discovering available templates ``` GET /capabilities Authorization: Bearer ``` ```json { "ephemeral_templates": { "saga": { "allowed_models": ["glm5-turbo", "glm4.7", "glm4.5-air", "granite-structured", "qwen3.6-35-a3b"], "default_model": "glm5-turbo", "system_prompt_max_bytes": 32768 } } } ``` `GET /capabilities` does not require `instantiate:saga` scope — any authenticated caller can read what's available before deciding to instantiate. ### Creating an ephemeral session ```json POST /sessions { "agent_id": "saga", "config": { "system_prompt": "You are a careful, skeptical frame-clarifier...", "model": "glm5-turbo" } } ``` **Validation order** (each step returns `422` with a stable `error_code` on failure): | `error_code` | Trigger | |---|---| | `ephemeral_requires_config` | `agent_id` is an ephemeral template but `config` is absent | | `foundational_does_not_accept_config` | `agent_id` is a foundational agent but `config` is present | | `system_prompt_required` | `config.system_prompt` missing or null | | `system_prompt_empty` | `config.system_prompt` is whitespace-only | | `system_prompt_too_large` | `config.system_prompt` > 32768 bytes UTF-8 | | `model_not_allowed` | `config.model` present but not in `saga_allowed_models` | **`config.model` resolution:** When `config.model` is omitted (or `null`), the server resolves it to `saga.default_model` from `config/defaults.yaml`. The resolved value is always populated in the session snapshot; `model` is never left absent or null in the stored config. **Response:** Same 201 shape as foundational sessions, with two new fields: ```json { "session_id": "...", "agent_id": "saga", "kind": "ephemeral", "config": { "system_prompt": "You are a careful, skeptical frame-clarifier...", "model": "glm5-turbo" }, "message_count": 0, "created_at": "...", ... } ``` **`kind` field:** `"ephemeral"` for Saga sessions, `"foundational"` for all other sessions. Present on both `GET /sessions` list items and `GET /sessions/{id}`. ### Sending messages to an ephemeral session ``` POST /sessions/{id}/messages ``` SSE, cancel, `persist_partial`, rate limits, and error shapes are bit-identical to foundational sessions. The only differences are pre-turn: - **System prompt:** `session.config.system_prompt` verbatim — not extended with registry text, persona, or motivational context. - **Provider:** resolved by `session.config.model`. - **Tools:** empty `[]` — no tools loaded, no tool schemas declared. ### Scope Creating a Saga session requires the `instantiate:saga` scope. This scope is bundled in the `user` tier. Tier `admin` inherits it via the wildcard. ### What Saga does NOT do - No persona injection (`PersonaRegistry.inject_context` not called) - No post-turn appraisal (`PersonaRegistry.update_after_turn` not called) - No valence writes - No memory writes - No tool calls --- ## Transient Characters Public primitive for downstream consumers (Skaldsong, RPG/game engines, dialog tools) that need to spin up dozens of distinct OCEAN-driven personas per session. Worldtree owns no durable state — the consumer ships character JSON, gets a `character_id`, and runs sessions against it. Persists in-memory only; process restart drops everything. The character displaces the **persona + model** layer of the bound session. The `agent_id` (typically `actor`) still resolves system prompt, tools, and the LLM provider unless the character carries a `model` override. ### `POST /characters` Create a transient character. Requires `character.write` scope. ```json { "character": { "schema_version": "1", "name": "Hamlet", "ocean": { "openness": 0.5, "conscientiousness": 0.6, "extraversion": -0.3, "agreeableness": 0.2, "neuroticism": 0.6 }, "description": "A melancholy prince.", "narrative": "He speaks to himself when alone, weighs every choice three ways…", "voice_profile_block": "Formal English; iambic pentameter under pressure; weights every word.", "model": "fast" }, "state": null } ``` Response 201: ```json { "character_id": "char_abc123…", "ttl_expires_at": "2026-05-07T07:00:00Z" } ``` `state` is optional — when supplied (a `CharacterStateSchema` JSON), the new character starts at the supplied PAD vector with the supplied active emotions. Used for mid-conversation rehydration after a process restart. ### `GET /characters/{character_id}/state` Export the live runtime state. Requires `character.read` scope. Refreshes the character's TTL. Response: ```json { "schema_version": "1", "pad": [0.4, 0.1, -0.2], "emotions_active": [ { "type": "joy", "intensity": 0.5, "fired_at": "2026-05-07T03:00:00Z", "decayed": false } ], "mood_drift": null, "goal_signal_history": null } ``` ### `DELETE /characters/{character_id}` Remove the character. Requires `character.write` scope. Sessions bound to this character are detached (next turn returns 410 `character_not_found`); other session messages are unaffected. ### `GET /models/available-for-characters` Return the model profiles the actor (and the `model` field on `CharacterSchema`) can reference. Requires `character.read` scope. ```json { "items": [ { "name": "fast", "description": "…", "thinking": false } ] } ``` ### `POST /sessions` extension Existing endpoint accepts an optional `character_id`: ```json { "agent_id": "actor", "character_id": "char_abc123…" } ``` When supplied, the session binds the character. The character's persona (OCEAN, mood, voice profile, narrative) displaces the agent-level persona for every turn within the session. The session's `SessionInfo` response includes the `character_id` field. ### Lifecycle - **TTL** — sliding, refreshed on every binding-session turn and on `GET /characters/{id}/state`. Default 4h stale; max 24h at create time (override with `ttl_seconds`). Sweep runs every 60s. - **In-flight refusal** — sweep refuses to evict a character whose binding sessions have any in-flight turn; resumes next cycle. - **Quota** — default 100 characters per user (configurable via `conversation_api.transient_characters.max_characters_per_user`). 429 `quota_exceeded` over cap. - **Process restart** — drops all transient characters. Sessions whose character is gone return 410 `character_not_found` on the next turn. - **DELETE detaches, does not cascade** — binding sessions live on; their next turn returns 410 (not the session itself). ### Errors - `404 character_not_found` — unknown or cross-user character (no leak per INV-009) - `410 character_not_found` — character was deleted/expired (returned at the session-bind path on next turn) - `422 ttl_too_large` — `ttl_seconds` over the configured ceiling - `422 state_schema_outdated` — schema_version mismatch; detail includes `accepted_versions` - `422 model_not_available_for_characters` — `model` field references a profile not in the allowlist - `422 validation_failed` — OCEAN out of `[-1, 1]`, voice_profile_block too long, etc. - `429 quota_exceeded` — per-user character cap hit - `403 auth_scope_denied` — missing `character.write` or `character.read` ### PII discipline - Admin events (`character.created` / `character.deleted` / `character.expired`) carry only `{character_id, user_id}`. Never name, description, narrative, OCEAN values, or voice profile content. - The `narrative` field's free-form prose IS rendered into the system prompt (this is the point) but is NOT echoed in admin events or audit payloads. ### Example client (JS) ```js // Create a character const char = await fetch(`${BASE}/characters`, { method: "POST", headers: { "Content-Type": "application/json", Authorization: `Bearer ${KEY}` }, body: JSON.stringify({ character: { schema_version: "1", name: "Hamlet", ocean: { openness: 0.5, conscientiousness: 0.6, extraversion: -0.3, agreeableness: 0.2, neuroticism: 0.6 }, narrative: "A melancholy prince…", voice_profile_block: "Formal English; iambic pentameter under pressure.", }, }), }).then(r => r.json()); // Bind to a session and run a turn const sess = await fetch(`${BASE}/sessions`, { method: "POST", headers: { "Content-Type": "application/json", Authorization: `Bearer ${KEY}` }, body: JSON.stringify({ agent_id: "actor", character_id: char.character_id }), }).then(r => r.json()); // (POST /sessions/{sess.session_id}/messages as usual) // Export state for consumer-side persistence const state = await fetch(`${BASE}/characters/${char.character_id}/state`, { headers: { Authorization: `Bearer ${KEY}` }, }).then(r => r.json()); // Clean up when the scene ends await fetch(`${BASE}/characters/${char.character_id}`, { method: "DELETE", headers: { Authorization: `Bearer ${KEY}` }, }); ``` --- ## Voice Harness The Voice Harness is a set of SSE-side extensions that give low-latency voice consumers (TTS gateways, voice-tier clients) the surface they need to run a two-voice TTS pipeline against Worldtree turns. Three capabilities compose the Voice Harness: 1. **`text_boundary` SSE events** — speakable breakpoint markers emitted between `text` events (R-8, issue #129) 2. **Implicit tool-call narration** — convention for agent prompts that controls whether a narration `text` event precedes `tool_start` (R-9, issue #130) 3. **`voice.classifier_markers` per-agent config** — opt-in system-prompt addendum that instructs the LLM to wrap output in `` / `` tags for two-voice TTS demultiplexing (R-23, issue #131) Issues #130 and #131 ship via this implementation. ### `text_boundary` SSE event A new additive event type emitted by `stream_turn` immediately after each `text` event at speakable breakpoints. Clients that ignore unknown event types see today's behaviour bit-identically (INV-098). **Payload shape:** ```json { "type": "text_boundary", "kind": "sentence", "char_offset": 42, "ts": "2026-05-07T16:14:26Z" } ``` | Field | Type | Description | |---|---|---| | `type` | string | Always `"text_boundary"` | | `kind` | `"sentence" \| "clause" \| "forced"` | Boundary classification | | `char_offset` | integer | Exclusive end position in the cumulative text stream for this turn (sum of all prior `text.content` lengths) | | `ts` | ISO-8601 with `Z` | Wall time of emission | **`kind` values:** - `"sentence"` — `.!?` followed by whitespace + capital-letter-or-EOS. Mid-word abbreviations (`Mr. Smith`) are suppressed via lookbehind. - `"clause"` — clause punctuation (`,;:—`) past 40-token threshold since last boundary; also used for code-fence (` ``` `) hard boundaries and paragraph breaks (`\n\n`). - `"forced"` — hard cap after 200 tokens with no prior boundary (pathological- prose escape hatch). **Ordering and cancellation:** - Each `text_boundary` is yielded IMMEDIATELY AFTER its parent `text` event in the stream — INV-099. - `text_boundary` events are checkpoints, not commitments. Cancellation between two boundary events discards unbuffered text; no synthesised final boundary is emitted on cancel — INV-100. - Empty boundaries are suppressed (no `text_boundary` when there is no text since the last boundary). - `text_boundary` events are ALWAYS emitted regardless of agent voice config. **Limitation (v1):** Boundary detection assumes English / Western European punctuation. Languages without `.!?` sentence conventions (Chinese `。`, Japanese, Arabic) need different heuristics — deferred to a future follow-up. **Example client (TTS chunking):** ```javascript const eventSource = new EventSource(`/sessions/${sid}/stream`); let buffer = ""; let lastBoundaryOffset = 0; eventSource.addEventListener("message", (e) => { const ev = JSON.parse(e.data); if (ev.type === "text") { buffer += ev.content; } else if (ev.type === "text_boundary") { // Chunk ready: buffer[lastBoundaryOffset..ev.char_offset] const chunk = buffer.slice(lastBoundaryOffset, ev.char_offset); tts.speak(stripMarkdown(chunk)); lastBoundaryOffset = ev.char_offset; } else if (ev.type === "done") { // Flush any remaining buffered text const remainder = buffer.slice(lastBoundaryOffset); if (remainder.trim()) tts.speak(stripMarkdown(remainder)); } }); ``` ### Implicit tool-call narration Agents control tool-call narration through the **presence or absence of a `text` event immediately before `tool_start`** (INV-102). - **Narration on:** the agent emits a `text` event (`"Let me search the KB…"`) before `tool_start`. The boundary detector processes that text normally. - **Narration off:** the agent emits no `text` event before `tool_start`. The gateway sees `tool_start` with no preceding text and suppresses TTS for that tool call. No new field on `tool_start`. The mechanism is purely prompt-engineering: agents learn when to narrate via their system prompt instruction. Worldtree owns what is spoken (SEA C-4 commitment) without adding wire surface. **Example narration-on agent prompt addition:** > Before each tool call, emit a one-line narration of what you are doing — > "Let me search the KB…" or "One moment, checking the upload…" ### `voice.classifier_markers` per-agent config Optional per-agent flag in `agents//config.yaml`: ```yaml voice: classifier_markers: true # default: false ``` When `true`, `PersonaRegistry.inject_context` auto-appends the canonical instruction block from `core/persona/voice/classifier_prompt.md` to the agent's system prompt. The block instructs the LLM to wrap output in: - `...` — structural/filler/acknowledgement lines (neutral TTS voice) - `...` — substantive/emotive lines (character TTS voice) Markers appear inline in `text` event content; the gateway parses them client-side and routes chunks to the appropriate TTS voice. Worldtree adds no wire-surface for this — gateway owns the demultiplexing (INV-104). **Constraints:** - Default `false` — agents without the flag behave identically to today. - Static for the agent's lifetime — cannot be flipped mid-session (INV-103). - The prompt block contains no per-session content (INV-104 PII discipline). - No admin events emitted for classifier-marker config (INV-101). --- ## SSE Event Types All events are JSON objects in the SSE `data:` field. ### worker_phase Worker phase transition — the agent has entered a new processing phase. ```json {"type": "worker_phase", "phase": "BuildingPrompt", "turn_id": 42} ``` The `phase` field is one of the five closed-set values: - `BuildingPrompt` — agent is loading persona, resolving tools, assembling the LLM prompt. - `CallingLLM` — agent is in an LLM call (may interleave with `thinking`/`text` content events). - `ProcessingTools` — agent is dispatching one or more tool calls between LLM round-trips. - `Streaming` — agent is receiving text deltas from the LLM. - `Finishing` — LLM streaming is complete; agent is running post-turn work (appraisal, audit) before the terminal event. Phase events emit on ENTRY to each phase. The terminal event (`done` / `error` / `cancelled`) marks the exit from the last in-flight phase. Cancel and error paths skip `Finishing` and emit the terminal event directly after the most recent in-flight phase. Tool-using turns cycle through `CallingLLM → ProcessingTools → CallingLLM → ...` once per round-trip (not per individual tool call within a round-trip). Clients that don't need phase events can filter on `event["type"] != "worker_phase"` client-side. Existing SSE consumers that switch on `event["type"]` ignore this event type without code changes. ### affect_update Persona-state observability event (issue #204). Fires twice per turn for agents with `persona.enabled: true` on non-ephemeral sessions; suppressed entirely for persona-disabled agents (e.g. `domari`, `muninn`), Tier 3 consumer-defined agents (Phase 2.0), and ephemeral sessions. **Start-of-turn — `status: "current"`:** Emitted immediately at the start of each qualifying turn, before any `worker_phase` event. Carries the agent's current persona snapshot reflecting all prior turns' completed appraisals. ```json { "type": "affect_update", "status": "current", "turn_id": 42, "snapshot": { "agent_id": "mimir", "pad": {"pleasure": 0.52, "arousal": 0.47, "dominance": 0.50}, "dominant_emotion": "curiosity", "emotions_active": [ {"type": "curiosity", "intensity": 0.6, "decay_remaining_s": 202.7} ], "baseline_pad": {"pleasure": 0.50, "arousal": 0.40, "dominance": 0.50}, "mood_drift": {"valence_delta": 0.02, "arousal_delta": 0.07}, "last_updated_at": "2026-05-25T22:30:18+00:00" } } ``` **End-of-turn — `status: "scheduled"`:** Emitted after the post-turn appraisal task has been scheduled (per #177 Phase A's fire-and-forget discipline) and before `done`. Lightweight notification — no PAD numbers, since the appraisal is still running asynchronously. The result lands in the NEXT turn's `status: "current"` snapshot. ```json {"type": "affect_update", "status": "scheduled", "turn_id": 42} ``` `scheduled` is skipped on turn failure/cancel paths (the appraisal was never reached); `current` still fires unconditionally for qualifying turns. Bootstrap reads available via `GET /agents/{agent_id}/persona_state` (same `snapshot` shape, requires `persona.read` scope). ### awaiting_llm_first_token Periodic heartbeat event (issue #201) emitted at a configurable interval during the gap between `worker_phase: phase="BuildingPrompt"` and `worker_phase: phase="CallingLLM"`. Solves the legitimate-slow first-token visibility gap: consumer TUIs can render a "thinking for Ns…" timer rather than a frozen line during heavy-CoT prompt warmup. ```json { "type": "awaiting_llm_first_token", "turn_id": 42, "elapsed_ms_since_building_prompt": 5012.3 } ``` `elapsed_ms_since_building_prompt` is the server-authoritative wall-clock milliseconds since `BuildingPrompt` was emitted. Independent of network latency or clock skew. Heartbeats stop the moment the engine produces its first event (the `CallingLLM` marker). They do NOT re-fire during tool-roundtrip `CallingLLM` re-entries — the heartbeat is scoped to the FIRST `BuildingPrompt → CallingLLM` gap only. **Configuration:** `conversation_api.awaiting_llm_first_token_heartbeat_s` (default `5.0`). Per-agent override via `agent.conversation.awaiting_llm_first_token_heartbeat_s`. Value `0.0` disables emission entirely. Cancellation paths (stall watchdog, user-cancel) also stop the heartbeat — no `awaiting_llm_first_token` event appears after the terminal `cancelled` event. ### thinking Incremental reasoning/thinking content (from thinking-enabled models). ```json {"type": "thinking", "content": "Let me consider the authentication options..."} ``` Thinking events appear before text events. They may be hidden from the user or shown in a collapsible section. ### text Incremental response text from the agent. ```json {"type": "text", "content": "I found "} ``` Text arrives in chunks. Concatenate all `text` event `content` fields to build the full response. The complete text is also available in the `done` event's `response` field. ### tool_start The agent is about to execute a tool. ```json { "type": "tool_start", "name": "search_library", "arguments": {"query": "authentication", "wings": ["kb"]} } ``` ### tool_result A tool call has completed. ```json { "type": "tool_result", "name": "search_library", "result": {"results": [...], "count": 3}, "duration_ms": 150 } ``` ### done The turn has completed. This is always the final event on success. The `phase` field is always `"succeeded"`. ```json { "type": "done", "phase": "succeeded", "response": "I found 3 notes related to authentication...", "model": "glm5-turbo", "duration_ms": 2450, "usage": { "prompt_tokens": 1280, "completion_tokens": 184, "total_tokens": 1464, "cached_input_tokens": 1024 } } The `usage` object reports token counts for the turn: - `prompt_tokens` — total input tokens sent to the model (including any cached portion) - `completion_tokens` — output tokens generated by the model - `total_tokens` — server-computed `prompt_tokens + completion_tokens` (does NOT include `cached_input_tokens`, which is a subset of `prompt_tokens`) - `cached_input_tokens` — input tokens served from prompt cache (relevant for cost calculations; cached tokens are typically priced at ~10% of full) When the underlying provider doesn't report token counts, all four fields are `0` (predictable schema; never omitted). ``` | Field | Type | Description | |-------|------|-------------| | `response` | string | Complete response text (same as concatenated `text` events) | | `model` | string | Model ID that generated the response | | `duration_ms` | int | Total turn duration in milliseconds | ### error An error occurred during the turn. This is always the final event on failure. The `phase` field is always `"failed"`. ```json { "type": "error", "phase": "failed", "message": "Provider connection timeout" } ``` ### cancelled The turn was cancelled server-side via `POST /sessions/{id}/turns/{turn_id}/cancel`. Always the final event on cancellation. The `phase` field is always present: `"cancelled"` for user-initiated cancel, `"stalled"` when the stall watchdog fired. ```json { "type": "cancelled", "phase": "cancelled", "turn_id": 42, "reason": "user_cancel", "partial_message_id": null } ``` **Invariant:** Every stream terminates with exactly one `done`, `error`, or `cancelled` event. Clients should listen for any of the three to know the stream is complete. **Error recovery:** If an error occurs mid-turn, the user message that triggered the turn is automatically rolled back from the session history. The session remains usable for subsequent messages. **Cancellation:** Cancellation is NOT an error — the user message stays in history. The cancel takes effect at the next agent_turn event boundary (between LLM streaming chunks or between tool calls), so any tool currently executing runs to completion before cancel applies. Partial assistant text is dropped by default; pass `?persist_partial=true` on the cancel call to persist it as an assistant message with `partial: true` metadata. **Turn IDs:** Every SSE event carries a composite `{turn_id}:{seq}` in the SSE `id:` wire field, and the integer `turn_id` as a JSON field in the event body. Clients read `turn_id` from the first event of a turn and pass it as the path parameter when cancelling. The `id:` field is used as `Last-Event-ID` for reconnect (see [Reconnect & Resume](#reconnect--resume)). --- ### POST /sessions/{session_id}/turns/{turn_id}/cancel Cancel an in-flight turn server-side. **Query parameters:** - `persist_partial` (boolean, default `false`) — if true, partial assistant text emitted before cancellation is persisted as an assistant message with `partial: true` metadata. Default behaviour drops partial output. **Response (200):** ```json { "turn_id": 42, "cancelled": true, "reason": null, "partial_message_id": null } ``` **Idempotent responses (200):** - Cancel after the turn has already finished: ```json {"turn_id": 42, "cancelled": false, "reason": "already_complete", "partial_message_id": null} ``` - A second cancel while the first is still propagating: ```json {"turn_id": 42, "cancelled": false, "reason": "already_cancelling", "partial_message_id": null} ``` **Errors:** - `404` — unknown session, unknown turn_id, or turn doesn't belong to the calling user (sessions are scoped per `INV-007`; ownership mismatches return 404 not 403 to avoid leaking turn existence). **Latency:** ≤100ms p99. Cancel is administrative — sets a flag, returns. Stream shutdown happens asynchronously after the response is sent. **Cooperative model:** cancel applies between agent_turn event boundaries. If a tool call is hanging (slow shell, wedged remote LLM), cancel won't fire until the tool returns. The provider's HTTP call may also continue server-side and bill us for in-flight tokens — cancellation stops scheduling NEW work, not work already in flight. --- ## Event Ordering A typical turn produces events in this order: ``` thinking* → tool_start → tool_result → thinking* → text+ → done ``` - `thinking` events are optional (only from thinking-capable models) - Tool events may repeat (agent can use multiple tools per turn) - `text` events appear after all tool calls complete - `done` is always last Minimal turn (no tools, no thinking): ``` text+ → done ``` --- ## Session Lifecycle ``` POST /sessions → session created (empty) POST .../messages → turn 1 (SSE stream) POST .../messages → turn 2 (SSE stream) GET .../messages → full history DELETE /sessions/{id} → session removed ``` Sessions persist across server restarts (SQLite-backed). On restart, sessions are lazy-loaded from the store on first access. Session metadata (accessible via `GET /sessions/{id}`) includes the model used in the most recent turn. --- ## Client Implementation Guide ### Minimal Python Client ```python import httpx import json BASE = "http://127.0.0.1:8080" HEADERS = {"Authorization": "Bearer sk-your-api-key"} # omit in dev mode # List agents agents = httpx.get(f"{BASE}/agents", headers=HEADERS).json() # Create session session = httpx.post(f"{BASE}/sessions", json={"agent_id": "mimir"}, headers=HEADERS).json() sid = session["session_id"] # Send message and consume SSE stream with httpx.stream("POST", f"{BASE}/sessions/{sid}/messages", json={"content": "Hello!"}, headers=HEADERS) as resp: for line in resp.iter_lines(): if line.startswith("data: "): event = json.loads(line[6:]) if event["type"] == "text": print(event["content"], end="", flush=True) elif event["type"] == "done": print() # newline after response break elif event["type"] == "error": print(f"Error: {event['message']}") break ``` ### Minimal JavaScript Client ```javascript const BASE = "http://127.0.0.1:8080"; // Create session const session = await fetch(`${BASE}/sessions`, { method: "POST", headers: {"Content-Type": "application/json"}, body: JSON.stringify({agent_id: "mimir"}), }).then(r => r.json()); // Send message via SSE const resp = await fetch(`${BASE}/sessions/${session.session_id}/messages`, { method: "POST", headers: {"Content-Type": "application/json"}, body: JSON.stringify({content: "Hello!"}), }); const reader = resp.body.getReader(); const decoder = new TextDecoder(); let buffer = ""; while (true) { const {done, value} = await reader.read(); if (done) break; buffer += decoder.decode(value, {stream: true}); // Parse SSE lines const lines = buffer.split("\n"); buffer = lines.pop(); // keep incomplete line for (const line of lines) { if (line.startsWith("data: ")) { const event = JSON.parse(line.slice(6)); switch (event.type) { case "text": process.stdout.write(event.content); break; case "thinking": /* show in UI or ignore */ break; case "tool_start": console.log(`[tool] ${event.name}...`); break; case "done": console.log(`\n[done in ${event.duration_ms}ms]`); break; case "error": console.error(event.message); break; } } } } ``` --- ## Uploads The Conversation API provides a server-side upload primitive: clients upload files first, receive an opaque `upload_id`, then reference those ids in subsequent message sends. This decouples storage from messaging and enables per-user quota, TTL-based expiry, and MIME validation. ### Endpoints | Method | Path | Scope required | Description | |--------|------|---------------|-------------| | `POST` | `/uploads` | `uploads.write` | Upload a file (multipart/form-data) | | `GET` | `/uploads` | `uploads.read` | List user's uploads (paginated) | | `GET` | `/uploads/{upload_id}` | `uploads.read` | Fetch upload metadata (not file bytes) | | `DELETE` | `/uploads/{upload_id}` | `uploads.write` | Revoke an upload (idempotent soft-delete) | ### POST /uploads Upload a file. The request must be `multipart/form-data` with a `file` field. **Response (200):** ```json { "upload_id": "upl_a1b2c3d4e5f60718293a4b5c6d7e8f90", "mime_type": "image/png", "size": 4096, "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "original_filename": "diagram.png", "created_at": "2026-05-06T10:00:00+00:00", "expires_at": "2026-05-07T10:00:00+00:00", "user_id": "alice" } ``` All 8 fields are always present in every response from `POST /uploads` and `GET /uploads/{id}`. `original_filename` may be `null` when the source filename sanitised empty; all others always have a value. **MIME validation:** The server detects MIME type from the file's magic bytes — the `Content-Type` header is recorded but not trusted as the gate. If the magic-byte signature doesn't match the configured allowlist (default: `image/png`, `image/jpeg`, `image/gif`, `image/webp`, `application/pdf`, `text/plain`, `text/markdown`, `text/csv`), the upload is rejected with 415. A Content-Type disagreement logs a WARNING but does not fail the upload — magic-byte authority wins. Text MIME types (`text/plain`, `text/markdown`, `text/csv`) are detected heuristically: no NUL bytes in the first 4096 bytes AND valid UTF-8. **Error responses:** | Status | `error_code` | Cause | |--------|-------------|-------| | 413 | `upload_too_large` | File exceeds `max_upload_bytes` (default 25 MiB) | | 413 | `quota_exceeded` | Per-user quota exceeded; `limit_type` is `total_bytes` or `max_count` | | 415 | `mime_type_disallowed` | Magic-byte signature not in allowlist; `detected` and `allowed` fields included | | 401 | `auth_missing` | No Authorization header (auth mode) | | 403 | `auth_scope_denied` | Missing `uploads.write` scope | **Example curl:** ```bash curl -X POST https://api.example.com/uploads \ -H "Authorization: Bearer $TOKEN" \ -F "file=@diagram.png;type=image/png" ``` ### GET /uploads Paginated listing of the authenticated user's uploads. Sorted by `created_at` DESC. **Query parameters:** `limit` (1–200, default 50), `cursor` (opaque pagination cursor). **Response:** ```json { "items": [{ "upload_id": "...", "mime_type": "...", ... }], "next_cursor": "v1.eyJjIjoiMjAyNi0wNS0wNlQxMDowMDowMCIsImkiOiJ1cGxfLi4uIn0" } ``` Cursor is `null` on the last page. Pass `cursor=` to fetch the next page. **Error responses:** `422 cursor_invalid` for malformed cursors. ### GET /uploads/{upload_id} Returns metadata for a single upload. Returns the same 8-field envelope as `POST /uploads`. Returns `404` for missing or cross-user uploads (no existence leak per INV-009). Returns `410 upload_expired` when the upload row exists but has passed its TTL. ### DELETE /uploads/{upload_id} Soft-delete an upload. Idempotent — safe to call multiple times. **First call response (200):** ```json {"upload_id": "upl_...", "deleted": true, "deleted_at": "2026-05-06T12:00:00+00:00"} ``` **Subsequent calls (200, already deleted):** ```json {"upload_id": "upl_...", "deleted": false, "reason": "already_deleted", "deleted_at": "2026-05-06T12:00:00+00:00"} ``` Returns `404` for missing or cross-user uploads. ### TTL and expiry Uploads expire 24 hours after creation (configurable via `conversation_api.uploads.ttl_hours`). A background TTL sweep runs every 5 minutes (configurable via `sweep_interval_seconds`) and hard-deletes expired rows + files. Between sweeps, the access endpoints perform lazy expiry checks and return `410 upload_expired`. Soft-deleted rows (via `DELETE`) persist in the database until their `expires_at` passes, at which point the sweep hard-deletes them (file was already removed on the `DELETE` call). ### Per-user quota Default limits (configurable in `conversation_api.uploads`): - `max_upload_bytes`: 25 MiB per upload - `max_total_bytes_per_user`: 100 MiB total live bytes - `max_count_per_user`: 50 live uploads - `max_uploads_per_message`: 10 uploads per message Quota is enforced **incrementally during the stream** — the server reads in 64 KiB chunks and aborts mid-stream on quota exceeded, deleting the temporary file. ### Auth scopes | Scope | Operations | Default grant | |-------|-----------|--------------| | `uploads.write` | `POST /uploads`, `DELETE /uploads/{id}` | User tier (own uploads), admin tier (all users) | | `uploads.read` | `GET /uploads`, `GET /uploads/{id}` | User tier (own uploads), admin tier (all users) | In dev mode (no `api_keys` configured), `anonymous` gets both scopes. ### Attaching uploads to messages The `POST /sessions/{session_id}/messages` endpoint accepts an additive `upload_ids` field: ```json {"content": "Analyze this diagram.", "upload_ids": ["upl_a1b2c3..."]} ``` **Validation (order matters):** 1. Pydantic validates `upload_ids`: non-empty list, ≤10 entries, each matches `^upl_[a-f0-9]{32}$` 2. Each upload existence/ownership/expiry check — first failure short-circuits with `410 upload_expired` or `404` 3. Agent capability gate — if `upload_ids` non-empty and `agent.capabilities` does not include `accepts_uploads` → `422 agent_lacks_upload_support` ### Per-call model override (issue #157) The `POST /sessions/{session_id}/messages` endpoint also accepts an additive `model` field that overrides the character / agent default for this turn only: ```json {"content": "Reformat this YAML.", "model": "granite-structured"} ``` **Semantics:** - Override is per-call only. Stored `CharacterSchema.model` is NOT mutated. - Validated against the same `available_for_characters` allowlist that gates `CharacterSchema.model` at create time (#153 INV-091). - Override displaces the character's bound model when both are set (per-call wins). - Override is REJECTED on ephemeral (Saga) sessions — their config is frozen at session-create per INV-161-2. **Validation:** 1. Pydantic validates `model`: optional string, non-empty after stripping whitespace. 2. If the session is ephemeral (`ephemeral_config != None`): `422 validation_failed` ("per-call 'model' override is not permitted on ephemeral sessions"). 3. If `model` is not in the `available_for_characters` allowlist: `422 model_not_available_for_characters`. **Audit:** the `turn.started` admin event carries the effective `model` plus a `model_override_applied: bool` flag so cross-system audit trails can see what model actually ran on a per-turn basis. **Driving use case** (per Skaldsong via althing): pipelines that cycle between models on every call (e.g. `granite-structured` for structured passes, `qwen3.6-35-a3b` for creative passes — ~40 calls per pipeline run) are operationally simpler with per-call override than with a pool of model-keyed transient characters. ### Agent capability: `accepts_uploads` Agents declare upload support in their `config.yaml`: ```yaml agent: capabilities: - accepts_uploads ``` Agents without this capability reject messages that include `upload_ids`. No agent in the current tree has `accepts_uploads: true` — the infrastructure is ready but no agent has been wired to consume uploads yet. ### Dispatch channels When a message with `upload_ids` reaches an agent that has `accepts_uploads`, the `agent_turn` engine populates two channels before the first LLM call: 1. **Tool channel** (`ctx.tool_context['uploads']`): always populated with the list of `Upload` objects, regardless of MIME type. Agent tools import `from core.conversation_api.uploads import Upload` to type-hint. 2. **LLM channel** (content blocks): only populated when the agent's active LLM profile has `supports_vision: true` **and** the uploads include `image/*` MIMEs (`image/png`, `image/jpeg`, `image/gif`, `image/webp`). Non-image MIMEs (PDF, text, csv) are never sent as LLM content blocks — they go through the tool channel only. LLM profile `supports_vision` is set per profile in agent config or `config/providers.yaml`. Example: ```yaml llm_profiles: default: provider: anthropic model: claude-3-5-sonnet-latest supports_vision: true ``` **Startup INFO log:** When an agent has `accepts_uploads: true` but its primary LLM profile has `supports_vision: false`, the server logs: ``` INFO agent mimir accepts uploads but its primary LLM is text-only — uploads will route to tools only ``` ### Example JavaScript client (upload-then-reference) ```javascript // 1. Upload the file const form = new FormData(); form.append('file', fileBlob, 'diagram.png'); const upResp = await fetch('/uploads', { method: 'POST', headers: { Authorization: `Bearer ${token}` }, body: form, }); const { upload_id } = await upResp.json(); // 2. Reference the upload in a message const msgResp = await fetch(`/sessions/${sessionId}/messages`, { method: 'POST', headers: { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json', }, body: JSON.stringify({ content: 'Analyze this diagram.', upload_ids: [upload_id] }), }); // Stream SSE response... ``` ### Capability vocabulary The `agent.capabilities` list accepts these well-known strings: | Capability | Description | |-----------|-------------| | `accepts_uploads` | Agent can receive file uploads via `upload_ids` on messages (issue #118) | | `multimodal` | Agent produces or consumes images/audio (legacy; prefer `accepts_uploads` for file input) | | `image_generation` | Agent can generate images | | `voice_io` | Agent supports voice input/output | --- ## Error Codes Every error response (REST 4xx/5xx and SSE `error` events) carries a stable `error_code` field so gateway clients can dispatch on a fixed identifier rather than pattern-matching human-readable messages. ### ErrorCode enum | `error_code` | Semantic | Typical HTTP status | Common cause | |---|---|---|---| | `auth_missing` | No `Authorization` header when auth is enabled | 401 | Client missing Bearer token | | `auth_invalid` | Token present but not recognized | 401 | Bad, expired, or rotated key | | `auth_user_disabled` | Key's user account is disabled | 401 | Admin disabled the account | | `auth_key_disabled` | Key has been revoked | 401 | Admin revoked via DELETE /admin/keys | | `auth_key_superseded` | Key is past its rotation grace window | 401 | Old key after grace period | | `auth_scope_denied` | Key lacks a required permission scope | 403 | User key accessing admin endpoint | | `session_not_found` | Session not found or belongs to another user | 404 | Stale ref or cross-user access | | `agent_not_available` | Agent not loaded | 404 | Bad `agent_id` on session create | | `turn_not_found` | Turn not found or belongs to a different session | 404 | Bad turn_id on cancel | | `turn_finished` | Turn completed and cleaned up | 410 | Reconnect after process restart | | `key_not_found` | API key record not found | 404 | Bad `key_id` on admin endpoints | | `key_revoked` | Key is revoked; operation not permitted | 410 | Rotate on a revoked key | | `key_already_superseded` | Key already superseded | 409 | Concurrent or repeat rotate | | `last_event_id_invalid` | `Last-Event-ID` header malformed or turn_id mismatch | 400 | Bad resume header | | `buffer_expired` | Resume seq older than replay buffer's earliest entry | 412 | Too many events since disconnect | | `rate_limited` | Request denied by rate limiting | 429 | Burst or token budget exceeded | | `validation_failed` | Request body or query params failed validation | 422 | Missing required field or wrong type | | `content_too_long` | Request content exceeds the configured size limit | 422 | Message body too large | | `not_ready` | Service has not finished startup | 503 | Readiness probe during boot | | `method_not_allowed` | HTTP method not supported for this path | 405 | Framework-raised | | `malformed_request` | Request body could not be parsed | 400 | Non-JSON body when JSON expected | | `llm_output_truncated` | Model output truncated (`finish_reason='length'` / max_tokens hit) | SSE error event | Provider stopped generating before reaching a natural end | | `context_overflow` | Request exceeds model context window | SSE error event | Prompt + max_tokens > context_window for the chosen model | | `model_unavailable` | Provider unreachable, returning 5xx, or model not loaded | SSE error event | Provider outage, connection refused, model not deployed | | `provider_timeout` | Provider timed out before completing the response | SSE error event | TTFT exceeded, stream stall watchdog, or read timeout | | `llm_output_invalid` | Provider returned malformed JSON, no content, or violated content-type expectations | SSE error event | Provider response unparseable or rejected request shape | | `internal_error` | Unhandled server-side error or unclassified provider failure | 500 / SSE error event | Bug or unexpected runtime failure not matched by classifier | | `upload_too_large` | Upload exceeds `max_upload_bytes` | 413 | File too big | | `quota_exceeded` | Per-user quota exceeded; `limit_type` field indicates which bound | 413 | Storage quota | | `mime_type_disallowed` | Magic-byte signature not in allowlist | 415 | Disallowed file type | | `upload_expired` | Upload exists but TTL has passed | 410 | Lazy expiry gate | | `agent_lacks_upload_support` | Agent `capabilities` missing `accepts_uploads` | 422 | Agent not wired for uploads | The enum is **additive** — new codes may be added as new error sites are identified; existing codes are never repurposed. ### REST error envelope ```json { "detail": { "error_code": "session_not_found", "message": "Session 'abc-123' not found" } } ``` Structured fields from prior issues are preserved alongside `error_code` and `message`: ```json { "detail": { "error_code": "rate_limited", "scope": "per_user_req_per_min", "retry_after_s": 12.4, "message": "Rate limit exceeded for per_user_req_per_min" } } ``` ### SSE error event SSE error events carry `error_code` at the **top level** (not inside a `detail` wrapper): ```json { "type": "error", "error_code": "session_not_found", "message": "Session 'abc-123' not found" } ``` ### Custom exception handler status-code mapping When FastAPI raises an `HTTPException` with a plain string `detail`, the handler maps known status codes to designated error codes before falling back to `internal_error`: | `exc.status_code` | `error_code` | |---|---| | `405` | `method_not_allowed` | | `400` | `malformed_request` | | (other) | `internal_error` + server-side warning logged | --- ## Versioning This is v1.0 of the interface spec. The SSE event format (field names, types, and semantics) is the public contract. Changes that add new event types or new optional fields to existing events are backward-compatible. Changes that rename fields, remove fields, or change types require a version bump. --- ## Appendix: `agent.ui_hints` config block The `agent.ui_hints` block is an optional section in `agents/{name}/config.yaml` that provides cosmetic UI metadata for gateway rendering. All subfields are optional. ```yaml agent: id: "mimir" name: "Mimir" description: "..." version: "0.2.0" capabilities: [knowledge_base, semantic_search] ui_hints: icon: "well" # optional, free-form string color_hint: "#5b8aa3" # optional, #RRGGBB hex (exactly 6 hex digits) vibe: "contemplative" # optional, ≤30 chars, non-empty; returned under persona_traits in the API ``` **Subfields:** | Field | Format | Notes | |-------|--------|-------| | `icon` | free-form string | Suggested: `book`, `globe`, `code`, `mic`, `chat`, `well`, `search`, `brain` | | `color_hint` | `#RRGGBB` hex | Exactly 6 hex digits after `#`; 3-char shorthand (`#fff`) is NOT accepted | | `vibe` | string, ≤30 chars | One-word affective feel descriptor; returned under `persona_traits.vibe` in the API response, NOT under `ui_hints` | **Important notes:** - `agent.ui_hints` does NOT carry `capabilities`. Capabilities live at `agent.capabilities`. Do not add a duplicate slot. - `vibe`, though configured under `agent.ui_hints`, is returned under `persona_traits` in the API response. It is absent from the response if `persona.enabled` is not `true`. - Validation fires at startup. Invalid `color_hint` (bad format) or `vibe` (empty or >30 chars) logs a warning and omits the offending field. The agent continues to load normally. - The schema slot ships empty; agents adopt `ui_hints` opportunistically via config edits. ## Per-Message Bifrost Endpoint Override (issue #166) Bifrost v0.2 supports **per-message endpoint overrides**: a single message can target a different Bifrost MCP server than the one bound to the session (or target a Bifrost server when none is session-bound). This enables stateless one-off consumer-MCP calls — Skaldsong's YAML-linting case is the primary driver. ### Request payload extension `POST /sessions/{session_id}/messages` gains an optional `bifrost` field: ```json { "content": "Lint this YAML: ...", "bifrost": { "endpoint_url": "https://skaldsong.example.com/mcp", "consumer_id": "skaldsong-heimdall-user-id", "scope": null } } ``` | Field | Type | Required | Description | |-------|------|----------|-------------| | `endpoint_url` | string | yes | HTTPS URL of the override target's MCP server. `http://localhost` and `http://127.0.0.1` are also accepted for local testing. | | `consumer_id` | string | yes | Heimdall user_id whose registered Bifrost algorithm + key to use for the ad-hoc handshake. Must be non-empty. | | `scope` | string \| null | no | Optional opaque scope string passed to the JWT payload. | When `bifrost` is absent or null, the message uses only the session-bound Bifrost tools (if any) and platform tools. Zero overhead for messages without the field. ### Capability requirement The override target's handshake response MUST include `'per-message-endpoint-override'` in `capabilities_granted`. If absent, the message is rejected with 502 and the turn is NOT processed. There is no fallback. ### Ad-hoc session lifecycle 1. Worldtree mints a 60-second JWT for the override session (`session_id = f'override-{message_id}'`). 2. Synchronous handshake runs before the first SSE event is emitted. 3. Override tools are merged into the message's effective tool list, namespaced as `bifrost..`. 4. Session-bound Bifrost tools (if any) coexist under their own namespace (`bifrost..`). 5. At message end (success, error, or cancellation), the ad-hoc client's `disconnect()` is called. No state survives. ### Reentrancy cap The override client has a fresh 25-call reentrancy budget, independent of the session-bound client's counter. Exhausting one counter does not affect the other. ### Error responses | Condition | HTTP | `error_code` | `bifrost_error` | |-----------|------|-------------|----------------| | `endpoint_url` is not HTTPS | 422 | `validation_failed` | — | | Ephemeral (Saga) session | 422 | `validation_failed` | — | | Missing `bifrost:invoke` scope | 403 | `auth_scope_denied` | — | | `consumer_id` not in Heimdall or not Bifrost-registered | 502 | `bifrost_consumer_not_found` | — | | Handshake failed (network, auth, etc.) | 502 | `bifrost_handshake_failed` | spec error code | | `per-message-endpoint-override` not granted | 502 | `bifrost_handshake_failed` | `bifrost.capability_unavailable` | ### Telemetry The `turn.started` event always carries `bifrost_override_applied: bool` (True/False, never null): ```json { "type": "turn.started", "session_id": "...", "turn_id": 42, "agent_id": "mimir", "model": "glm5-turbo", "model_override_applied": false, "bifrost_override_applied": true } ``` ## Tier 3 — Consumer-defined agents (Phase 2.0, issue #181) Tier 3 agents are consumer-owned, Worldtree-hosted agents whose identity lives at `:`. They share the persistent session infrastructure with Tier 1 / Tier 2 but layer-specific machinery (persona, motivational, memory, valence) is reserved for later phases — Phase 2.0 ships baseline addressing + ownership + lifecycle only. ### Endpoints | Method | Path | Purpose | | -------- | --------------------------------- | ------------------------------------------ | | `POST` | `/agents/define` | Create a Tier 3 agent. | | `DELETE` | `/agents/:` | Owner-initiated hard-delete. | | `PATCH` | `/agents/:` | Mutate `system_prompt` and/or `model`. | | `POST` | `/sessions` | Tier 3 routing when `agent_id` has a `:`. | | `GET` | `/sessions//tools` | Owner-scoped session-tools introspection (#183, Phase 2.0.1). | #### `POST /agents/define` ```json { "agent_name": "wizard", "system_prompt": "You are a guided-elicitation wizard...", "model": "glm5-turbo", "persona": null, // schema-reserved; non-null → 422 layer_deferred "motivational": null, "valence": null, "memory": null } ``` Response (`201 Created`): ```json { "agent_id": "alice:wizard", "user_id": "alice", "agent_name": "wizard", "system_prompt": "...", "model": "glm5-turbo", "created_at": "2026-05-19T12:00:00+00:00", "updated_at": "2026-05-19T12:00:00+00:00" } ``` Caller must: - hold `agents.define` scope (default for `user` tier); - have a slug-safe `user_id` matching `[a-z][a-z0-9-]{2,63}` (Phase 2.0 gate — non-slug user_ids receive 403 `tier3_user_id_unsupported`); - be authenticated via a real bearer key (the key's hash becomes `owner_key_hash` for quota and cascade tracking). `agent_name` is a strict slug `[a-z][a-z0-9-]{2,63}` and immutable after definition. #### `DELETE /agents/:` — `204 No Content` Owner-initiated hard-delete. Bypasses the 24h grace (distinct from the key-revocation cascade which uses soft-delete). Cancels every active session bound to this agent and revokes the owner's per-resource `agents.call::` scope grant. #### `PATCH /agents/:` Phase 2.0 minimal: only `system_prompt` and/or `model` may be patched. Any other key (including the immutable `agent_name`, `user_id`, or layer fields — even `null`) returns 422 `field_not_mutable` BEFORE the DB lookup. Active sessions continue using their cached `AgentContext`; the new values take effect at the next session-create. #### `POST /sessions` — Tier 3 routing When `agent_id` contains a `:`, the handler routes through the `consumer_agents` table. Required body: ```json { "agent_id": "alice:wizard", "end_user_id": "bob", "bifrost": { "endpoint_url": "https://...", "scope": "..." } // optional } ``` `end_user_id` is required at session-create from Phase 2.0 so consumer integration code doesn't change at the Phase 2.3 valence boundary (where cross-namespace isolation will be enforced). The session record carries the SESSION's `owner_key_hash` — the key that authenticated the session-create call. Revoking that key invalidates this session immediately; the agent's defining key (potentially different) is unaffected by this session's lifecycle. ### Quota Hard cap: **50 Tier 3 agents per Heimdall key**. Quota is per-(active) key — a user holding multiple keys gets independent 50-agent budgets. Exceeded at define → `429 agent_quota_exceeded` with `Retry-After: 0`. ### Key-revocation cascade When an API key is revoked (`DELETE /admin/keys/{key_id}`), every Tier 3 agent whose `owner_key_hash` equals the revoked key's hash is soft-deleted in the same SQL transaction. The 24h grace window starts immediately. Active sessions whose `owner_key_hash` matches the revoked key are invalidated (next message → 401 `auth_revoked`). Other keys belonging to the same user are not touched. An `agents.cascade_delete` audit event is emitted per affected agent with the revocation-initiator as `actor`. A background sweeper hard-deletes rows whose `deleted_at` is older than 24h and revokes their orphaned per-resource scope grants. ### `GET /sessions//tools` — owner-scoped tool introspection (#183, Phase 2.0.1) Owners of a Tier 3 (or Tier 1) session can read the merged tool list the LLM saw at turn-fire time without needing admin scope. Mirrors the existing `GET /admin/sessions//tools` handler's response shape and merge predicate; auth gate is `ctx.user_id == session.user_id` (existence-hiding 404 on cross-owner access). Response (`200 OK`): ```json { "agent_id": "alice:wizard", "builtin_tools": [], "bifrost_tools": [ {"name": "bifrost.alice.get_missing_fields", "description": "...", "parameters": {...}}, {"name": "bifrost.alice.set_field", ...} ] } ``` Notes: - `bifrost_tools` field mirrors the per-turn merge predicate at `service.py:2138` — entries appear only when both `session.bifrost_tools` is populated AND a live `BifrostClient` is registered. After a process restart, restored sessions show `bifrost_tools: []` until the client re-handshakes (which matches what the LLM would see). - Cross-owner access returns `404 session_not_found` (NOT 403) to avoid leaking session existence across users. - Revoked sessions (Tier 3 cascade or owner DELETE) return `401 auth_revoked` rather than 404. - No audit event is emitted — owners querying their own sessions is routine self-service, not admin oversight. The admin variant at `/admin/sessions//tools` keeps its `admin.sessions.read` gate and audit-event emission for cross-user operator debug. A per-turn INFO log line — `session= turn=NA tool_schemas= (builtin= bifrost= override=)` — emits at the merge point in `service.py` for log-grep diagnostics when the introspection endpoint isn't reachable. ### Error codes (Phase 2.0) | Code | Status | Surface | | ----------------------------- | ------ | ------------------------------------------------------------- | | `agent_name_invalid` | 422 | `agent_name` violates `[a-z][a-z0-9-]{2,63}`. | | `system_prompt_too_large` | 422 | `system_prompt` > 32 KiB. | | `model_not_available` | 422 | `model` not in `providers.yaml`. | | `layer_deferred` | 422 | One of `persona` / `motivational` / `valence` / `memory` set. | | `field_not_mutable` | 422 | PATCH carries an immutable key (any value, even `null`). | | `end_user_id_required` | 422 | Tier 3 session-create without a non-empty `end_user_id`. | | `tier3_user_id_unsupported` | 403 | Caller's `ctx.user_id` not slug-safe. | | `auth_scope_denied` | 403 | Missing `agents.define` or wrong owner. | | `agent_not_found` | 404 | No active row at `:`. | | `agent_name_taken` | 409 | Active-rows partial unique index collision. | | `agent_quota_exceeded` | 429 | 50 agents per key cap hit. `Retry-After: 0`. | | `auth_revoked` | 401 | Session's `owner_key_hash` was revoked since last call. | ## Default agent (Lofn) (issue #182) Lofn — Worldtree's welcoming-intermediary Asgardian — is the **default-resolved agent** for `POST /sessions` when `agent_id` is absent, null, or empty. ### Request shape `agent_id` is now optional. The following three forms all resolve to a Lofn session: ```json { "end_user_id": "alice" } { "agent_id": null, "end_user_id": "alice" } { "agent_id": "", "end_user_id": "alice" } ``` Explicit `"agent_id": "lofn"` works identically. Explicit non-Lofn agent_ids (e.g., `"mimir"`) are never silently rerouted. ### `end_user_id` is required Lofn sessions require `end_user_id` as a non-empty string (mirrors the Tier 3 requirement). Missing / null / empty returns: ```http HTTP/1.1 422 Unprocessable Entity Content-Type: application/json { "detail": { "error_code": "end_user_id_required", "message": "end_user_id is required (non-empty string) for Lofn sessions" } } ``` ### What Lofn does She is a thin LLM agent — no tools, no specialist dispatch, no server-side handoff. When the user's intent fits a specialist's lane (Mimir for archived-conversation search, Forseti for procedure, Domari for judgment, etc.), Lofn will say so in plain prose: *"Mimir would have better visibility on archived conversation history than I do."* That suggestion has no structured signal underneath — no marker, no parser, no audit event, no session-switch. The user (or, in multi-agent contexts, the named specialist via the existing agent bus) acts on the suggestion organically. The act of saying it IS the handoff. ### Continuity Lofn maintains conversational continuity through Worldtree's existing valence engine + memory engine (per ADR-0003). She does not have her own preference store, so "remember I prefer X" is an organic memory-engine write, not an explicit API call. ### Matrix bridge The bridge provisions `@lofn:` as a virtual Matrix user. Lofn responds to DMs the same way other Asgardians do; her per-agent rendering knobs live at `agents/lofn/config.yaml → matrix.rendering`.