memory: snapshot — run 3 corpus built and held on a megamix containment defect

Run 2 is finished, gated FAIL, and serving on the gateway at operator request.
Run 3's corpus was built to brokkr's first recipe and held before any GPU spend:
creative-writing-multiturn is a DECLARED MEGAMIX containing bluemoon, PIPPA,
LimaRP and stheno, and the remix promoted creative-writing AND bluemoon -- the
two roots that overlap, at median jaccard 0.873. Containment, not overlap.

Dedup direction reversed so the primary source survives rather than the copy
inside the bag: bluemoon 67 -> 126 conversations and 38.6% of loss signal, the
largest contributor. Wholly-human share up, megamix share down, total context
unchanged at 12.49M so the operator's settled mix arithmetic survived.

Two structural findings recorded because they outlive this recipe: F1 'excise
PIPPA' removes the ROOT and not the MATERIAL (F2's 250-word floor does that
work, since PIPPA turns cannot exceed 123 words wherever they live), and LimaRP
and stheno remain unchecked against any other root.

Also records the correction I published wrong twice: run 2 was never unstable.
All 46 flags were too_short, the collapse guards fired zero times, and it is the
left tail of a length distribution -- not new to run 2 either, so it is a
property of the recipe and a further base swap will not fix it.

persistent-memory.d/2026-08-26-run3-corpus-and-the-megamix-containment.md

@@ -0,0 +1,116 @@
+# `[2026-08-26]` Run 3 corpus built, then held — and the megamix containment defect it exposed
+
+Run 3's corpus was built to brokkr's `SFT-RECIPE-run3.md` (`f6c0589`), a blocker
+was found before any GPU spend, and **brokkr is now redoing the recipe**. Nothing
+launched. The build artifacts survive and are re-parameterisable.
+
+## ⚠ THE FINDING: `creative-writing-multiturn` is a MEGAMIX containing our other roots
+
+Its own Hoard card says so — `hoard/datasets/creative-writing-multiturn/dataset.yaml`,
+`archive_provenance_note`:
+
+> *"Dampfinchen/Creative_Writing_Multiturn — 2026 curated ~4.6k high-turn
+> creative-writing/RP **megamix (bluemoon, LimaRP/PIPPA, stheno, gutenberg-dpo
+> chosen, VellumK2 subset)**"*
+
+**It is not a sibling root. It is a bag holding several of our other roots.**
+
+Found because my corpus build would not reproduce brokkr's numbers: he computed
+on the raw roots, I computed on run 1's deduped survivor list. bluemoon came out
+67 against his 126. Run 1's dedup record explains it:
+
+    DROPPED                     ->  KEPT                          n   median jaccard
+    bluemoon                    ->  creative-writing-multiturn   59       1.000
+    creative-writing-multiturn  ->  pippa                        30       1.000
+    c2-logs-32k-llama3          ->  creative-writing-multiturn   23       0.945
+
+brokkr then measured it directly against the raw corpora, and it is worse than
+the dedup threshold showed — **the MEDIAN bluemoon conversation is 87% similar to
+something in creative-writing** (≥0.99: 28.3%, ≥0.90: 49.6%, max 0.999). That is
+containment, not overlap.
+
+**Why it mattered here specifically:** the remix elevates creative-writing to
+43.2% and bluemoon to 31.4% of loss signal — *the two roots that overlap*.
+Building on raw roots would have double-counted bluemoon inside creative-writing's
+share, in a run whose entire justification is the composition change.
+
+## Three consequences, two of them structural
+
+**1. "bluemoon is the only human-authored root" was overstated.** It is the only
+*wholly* human-authored root; creative-writing is *partly bluemoon*. brokkr
+restated rather than withdrew.
+
+**2. ⚠ F1 does not do what the recipe says.** "Excise PIPPA" removes the ROOT, not
+the MATERIAL — PIPPA persists inside the megamix, which the remix promotes to
+largest contributor. **F2's 250-word floor is what actually removes PIPPA
+material**, because PIPPA turns cannot exceed 123 words wherever they live. The
+outcome was right; the stated mechanism was wrong. **Had the operator chosen a
+floor below 123, F1 would have been nearly cosmetic.**
+
+**3. ⚠ The dedup kept the wrong side of the pair, 59 times.** When a primary
+source and a megamix containing it collide, the survivor should be the primary
+source, not the copy inside the bag — the megamix version may have been
+reprocessed by its curator. We kept the derivative and discarded the origin.
+
+**Reversed for run 3** (brokkr's call, ~3 minutes): keep bluemoon, drop the
+megamix copy. Result is better than either prior version —
+
+    root                     run 2    recipe    DEDUPED+REVERSED
+    bluemoon                  6.9%     31.4%              38.6%
+    creative-writing         18.0%     43.2%              30.4%
+    c2-logs                  40.2%     25.4%              31.0%
+    pippa                    34.9%        0%                 0%
+
+**The wholly-human share went UP and the megamix share went DOWN** — strictly the
+composition the recipe argued for, now honestly counted. Total context unchanged
+at 12.49M, so the operator's settled mix arithmetic survived untouched.
+
+## ⚠ The catalog-level problem, unsolved
+
+**A megamix root silently contains other catalog roots and nothing in the mix
+arithmetic knows.** Any share computed over roots including creative-writing
+double-counts bluemoon, PIPPA, LimaRP and stheno by an unknown amount. Run 2's
+mix had this defect and nobody noticed, because run 2 did not promote the
+overlapping roots.
+
+LimaRP and stheno are **still unchecked against anything.** Wants a
+`contains_datasets:` field on the Hoard card plus a preflight assertion that no
+recipe includes both a megamix and a root it contains. brokkr is raising it as
+its own item; not a run-3 blocker.
+
+## What is staged on ana-ml2
+
+    /tank/erp-tune/recipe-r3/recipe-erp-seat-sft-r3.json
+    /tank/erp-tune/recipe-r3/dialogue-survivors.jsonl      610 conversations
+    /tank/erp-tune/recipe-r3/lowquality-mask-v1.jsonl    5,867 mask rows
+    /tank/erp-tune/build_r3.py       the filter build; floors are constants at the top
+    /tank/erp-tune/pippa_clip.py     length-distribution measurement
+    /tank/erp-tune/clip_share.py     turn-share vs word-share
+
+⚠ **`/mnt/smithy` is READ-ONLY from ana-ml2**, so recipe artifacts live under
+`/tank/erp-tune/recipe-r3/` rather than beside the r1 recipe. The harness takes
+absolute paths, so this is fine — but do not expect to write to the r1 recipe dir.
+
+**F2/F3/F4 are implemented as a LOSS MASK reusing the impersonation-mask format**,
+so `load_impersonation_mask` reads it and `_impersonation_flags` sha-asserts every
+masked turn with no new harness code. F4 is masked rather than stripped:
+stripping mutates content, which invalidates the shas, and the read-only mount
+forecloses a mutated derived root anyway. 28 turns of 6,209 (0.45%).
+
+Re-parameterising for a new recipe is one command and ~90 seconds; the dedup
+reversal and F4-as-mask handling are baked in and survive it.
+
+## Decisions I took
+
+**No conversation admission threshold** (brokkr delegated it, then endorsed).
+81% of surviving bot turns are context-only, but masked context costs the
+transformer body and **not** the LM head — the harness drops `IGNORE_INDEX`
+before the 262k-vocab head, so the waste is bounded well below what the 81%
+suggests. A 20% context trim buys ~35 minutes against a 3-hour run, and it would
+be a fifth filter moving composition inside a run whose purpose is testing the
+filters alone. **Buying 35 minutes by confounding a 3-hour experiment is a bad
+trade.** brokkr: "your reason 1 is decisive."
+
+**Left `erp-tune-v2` UP** on the operator's discretion grant. The card is not
+needed until a recipe is settled, and it is his evaluation surface plus a live
+gateway route. Take it down when run 3 is actually ready, not before.