feat(provider): person-prime scan verb + sortable_chunk_fields cap (WT #349)
Implement the memory-store `scan` verb — a query-LESS, LIVE-only, globally
ordered top-N-by-recency read — and advertise `sortable_chunk_fields=
[{updated_at}]` at the Bifrost handshake. Advertising the cap is what lights
up Worldtree's #349 person-prime turn-1 durable-fact injection (Branch-A
`"updated_at" in caps.sort_fields_supported`); the fix is ZERO Worldtree
change — the running provider announcing the cap is the trigger.
scan is:
- LIVE-only server-side (INV-009): superseded/tombstoned excluded — a dead
fact can never inject; person-prime's `lifecycle_state=live` does not ride
the scan wire, so server-side is authoritative.
- Globally ordered before pagination (INV-010): the full scope-filtered live
set is ordered by (sort.field, direction) globally; missing value LAST,
chunk_id tiebreak. Backed by an expression index on
json_extract(record_json,'$.updated_at') to stay in the 500ms budget.
- Cursor = offset into the global order; emits a next cursor only when a
further match exists (no empty trailing page — matches the reference).
Sort is dispatch-gated: an unadvertised sort.field raises InvalidArguments,
never a silent unsorted fallback.
Contract amended: un-defers scan, adds the FN spec + INV-009/INV-010 +
sortable_chunk_fields to INV-006. TDD 7/7 green (scan_recency tracer,
live_only, scope_isolation, unadvertised_sort, person_prime_record_shape,
cursor pagination, parity_vs_reference vs InMemoryMemoryStore #195). Full
suite 638 green.
This commit is contained in:
@@ -116,9 +116,13 @@ interpreted.
|
||||
to the reference `_matches_scope`. (`scope_any` is the union-visibility primitive that
|
||||
resolves the #295/#297 silent-zero — a subset-scoped chunk now recalls via an OR member.)
|
||||
- **INV-006** [hard]: **Capabilities match implementation** (advertise-⇒-implement).
|
||||
`describe_store` advertises ONLY what v1 implements: `relational_edges_supported=False`,
|
||||
`describe_store` advertises ONLY what is implemented: `relational_edges_supported=False`,
|
||||
`atomic_supersede_supported=False`, `transaction_supported=False`,
|
||||
`optimistic_locking_supported=True`, `filterable_metadata_fields=[]`.
|
||||
`optimistic_locking_supported=True`, `filterable_metadata_fields=[]`,
|
||||
**`sortable_chunk_fields=[{"name": "updated_at"}]`** (the ONLY globally-sortable field;
|
||||
gates `scan`'s sort at the bifrost dispatch `_validate_scan_sort` AND Worldtree's #349
|
||||
person-prime Branch-A `"updated_at" in caps.sort_fields_supported` — advertising it is
|
||||
what lights up turn-1 durable-fact injection).
|
||||
(`transaction_supported` is the bifrost **wire-level** multi-op transaction
|
||||
capability — NOT our internal SQLite transactions, which we use for atomic
|
||||
batches.) The client gates the gated verbs off these.
|
||||
@@ -127,6 +131,20 @@ interpreted.
|
||||
`InvalidArguments` (mirrors the reference).
|
||||
- **INV-008** [hard]: The store is REQUIRED (`build_memory_app(store=None)` raises);
|
||||
identity/scope/actor come from `ctx`, never call args.
|
||||
- **INV-009** [hard]: **`scan` is LIVE-only.** `scan` returns ONLY live chunks —
|
||||
superseded / tombstoned / any non-live governance state is EXCLUDED server-side. This
|
||||
is load-bearing because Worldtree's person-prime requests `lifecycle_state="live"` but
|
||||
that filter does NOT ride the scan wire today and the client does not re-check it
|
||||
(worldtree-dev flagged the adapter gap); server-side live-only is authoritative, so a
|
||||
dead fact can never inject. The additive `lifecycle_state` scan arg, when present, is
|
||||
honored but never relied upon.
|
||||
- **INV-010** [hard]: **`scan` is globally ordered before pagination.** The FULL
|
||||
scope-filtered live set is ordered by `(sort.field, direction)` GLOBALLY before the
|
||||
`limit` page is taken — never page-local. Missing sort value sorts LAST; ties broken by
|
||||
`chunk_id` (stable). A single `limit`-page returns the N globally-newest (for
|
||||
`updated_at desc`), matching bifrost's cross-pagination conformance negative. The sort
|
||||
field is indexed (`json_extract(record_json, '$.updated_at')`) so the read stays within
|
||||
person-prime's 500 ms fail-open budget.
|
||||
|
||||
## Concurrency
|
||||
|
||||
@@ -163,7 +181,7 @@ negotiation, routes). **This contract** owns the store (the basic verbs + SQLite
|
||||
|
||||
## Out of scope (deferred — do NOT flag as drift)
|
||||
|
||||
- **Gated/maintenance verbs:** `upsert_edges`/`get_edges_for`, `scan`, `mark_invalid`/`mark_superseded`, `patch_many`, `atomic_supersede`, lease/checkpoint. Absent + advertised-unsupported.
|
||||
- **Gated/maintenance verbs:** `upsert_edges`/`get_edges_for`, `mark_invalid`/`mark_superseded`, `patch_many`, `atomic_supersede`, lease/checkpoint. Absent + advertised-unsupported. (`scan` is NO LONGER deferred — it is implemented + advertised via `sortable_chunk_fields` to light up Worldtree's #349 person-prime turn-1 durable-fact injection; see the `scan` FN spec + INV-009/INV-010.)
|
||||
- **metadata_filter beyond scope:** advertise `filterable_metadata_fields=[]`; a non-empty `metadata_filter` is unsupported in v1 (rejected — see search PRE).
|
||||
- **The combined two-plane server** (guide §7) — separate memory + affect apps in v1.
|
||||
- **Deployment** — dev-box background shell (`ratatoskr-memory-provider`), no systemd/infra.
|
||||
@@ -270,6 +288,29 @@ TESTS:
|
||||
delete_absent [boundary]: unknown id → {"deleted":0}
|
||||
```
|
||||
|
||||
```contract
|
||||
FN scan(self, *, scope_all: dict | None = None, scope_any: list | None = None, cursor: str | None = None, limit: int, sort: dict | None = None, lifecycle_state=None) -> dict
|
||||
BRIEF: Query-LESS paginated LIVE-chunk scan, globally ordered by an advertised sort field (updated_at) — the #349 person-prime turn-1 durable-fact injection primitive (no query vector, unlike search). Returns {records, cursor}.
|
||||
PRE: [PRE-001 hard] limit is a positive int -- else InvalidArguments
|
||||
PRE: [PRE-002 hard] scope_all/scope_any shape + lattice-validated via _validate_scope (identical to search PRE-003) -- else InvalidArguments / InvalidFilter
|
||||
PRE: [PRE-003 hard] sort, when present, is {field, direction}: field ∈ the advertised sortable_chunk_fields names ("updated_at"), direction ∈ {asc,desc}. The bifrost dispatch layer (_validate_scan_sort) is the enforcement gate; an unadvertised/malformed sort → InvalidArguments — NEVER a silent unsorted fallback
|
||||
POST: [POST-001 return_value] {records: [<verbatim chunk wire records, same shape as a search hit's chunk>], cursor: <opaque next-page str | None>}; ≤ limit records; each record carries updated_at + agent_id + subject{type,id} + worldtree_scope (the fields person-prime's client _scan_filter_matches keys on — a record missing any is silently dropped client-side) -- assert
|
||||
POST: [POST-002 return_value] LIVE-only — returns ONLY live chunks; superseded/tombstoned excluded server-side (INV-009)
|
||||
POST: [POST-003 return_value] GLOBAL-order — the FULL scope-filtered live set is ordered by (sort.field, direction) GLOBALLY before the limit page; missing value LAST; chunk_id tiebreak (INV-010)
|
||||
STEPS:
|
||||
1. [setup] validate limit (>0) + scope (as search); sort ← the dispatch-validated {field,direction}
|
||||
2. [sequential, flexibility=indicative] SELECT scope-filtered LIVE chunks ordered by the indexed sort field (json_extract(record_json,'$.updated_at')) in `direction`, missing-last, chunk_id tiebreak, GLOBALLY; apply cursor offset; take limit
|
||||
3. [cleanup] RETURN {records: verbatim chunks, cursor: next-page-or-None}
|
||||
TESTS:
|
||||
scan_recency [happy,tracer]: upsert 4 live chunks w/ distinct updated_at; scan(scope_all={end_user}, limit=3, sort={field:updated_at,direction:desc}) → the 3 newest, newest-first
|
||||
global_before_paginate [scenario]: 5 chunks, limit=2 → page-1 = the 2 globally-newest; the cursor page continues the GLOBAL order, not a page-local re-sort (INV-010; bifrost cross-pagination conformance)
|
||||
live_only [adversarial]: a superseded/tombstoned chunk is NEVER returned even if it is the newest (INV-009)
|
||||
scope_isolation [adversarial]: scope_all one end_user → never returns another partition's chunk (INV-005 applies to scan)
|
||||
unadvertised_sort [adversarial]: sort.field ∉ sortable_chunk_fields → InvalidArguments at dispatch (never silent unsorted)
|
||||
person_prime_record_shape [scenario]: each record carries agent_id + subject{type,id} + worldtree_scope + updated_at + verbatim/distillate — the _scan_filter_matches keys (else the client silently drops it)
|
||||
parity_vs_reference [scenario]: identical scan envelopes vs InMemoryMemoryStore → same ordered chunk_ids/shape (#195)
|
||||
```
|
||||
|
||||
```contract
|
||||
FN build_memory_provider_app(store: RatatoskrMemoryStore, heimdall_key: bytes, consumer_id: str = "ratatoskr") -> Starlette
|
||||
BRIEF: Wire JwtVerifier + registration; hand the store to bifrost's build_memory_app.
|
||||
|
||||
Reference in New Issue
Block a user