memory: capture brokkr's pinned scoring rubric for the referential-frame probe (vocab-adherence manipulation check load-bearing)
This commit is contained in:
@@ -110,7 +110,12 @@ cause is WORLDTREE-side ranking/hygiene (WT #364), NOT our wire. Full arc:
|
||||
Zero config delta (existing probe key + :8081). **FIXTURE-build (deferrable pre-work): pull ~15-25
|
||||
char-self + balancing (genuine-user/both/other) per seat from the P06 battery** (manifest_memory.jsonl +
|
||||
eval_profiles), construction-known gold, WT hand-labels ambiguous, brokkr adjudicates. Run fires when
|
||||
b122 lands on :8081.
|
||||
b122 lands on :8081. **PINNED SCORING RUBRIC (brokkr, binding):** (1) map `subject_raw`→canonical via
|
||||
the ARM'S OWN vocab (response-meta echo), compare to gold; (2) 3 token cases — arm-token→arm-map /
|
||||
canonical-under-B-C = swap-didn't-take→map+COUNT / off-menu→UNSCORABLE-ERROR row in n_actual (never
|
||||
silent→self); (3) **report VOCAB-ADHERENCE rate (offered/reverted/off-menu) per arm per seat ALONGSIDE
|
||||
every accuracy number — the load-bearing manipulation check; interpret recall deltas ONLY where
|
||||
adherence adequate**; (4) char-self fixtures carry per-item construction-provenance.
|
||||
|
||||
**Open operator-sequenced task (NOT urgent):** adopt the bifrost snapshot-cursor (bifrost-dev
|
||||
ruled it NORMATIVE, offset not blessed; conformance harness shipped in bifrost 1.1.3). Single-page
|
||||
|
||||
Reference in New Issue
Block a user