memory: snapshot — FF A/B result, seat state, Qwen-thinking finding, esh hardening
Supersedes the 'blocked on a reproducing battery' entry: brokkr supplied the framing, the A/B ran, and the refusal question is settled. Most load-bearing fact for a cold session: Dark-Scarlett is DOWN and Fable-Fusion is serving char-rp-reasoning on an evaluation window with no permanent decision taken. Rollback recorded inline.
This commit is contained in:
+11
-1
@@ -111,7 +111,9 @@ no longer deployed sidecars here. See Recent decisions.)
|
||||
|
||||
_As of 2026-08-16 — **one loop open: awaiting brokkr-smithy-dev's refusal battery.** The overnight arc closed both queued items (gen-seat mixed NVFP4+FP8 requant, char-rp Gemma-4 tool parser); quant lessons consolidated into `docs/pfi/model-quantization-playbook.md`. New work this session: Dark-Scarlett replacement evaluation — see below._
|
||||
|
||||
- **🔄 DS-REPLACEMENT EVAL — BLOCKED on a reproducing battery (2026-08-16).** Operator: DS v1.0's refusals are too high on `char-rp-reasoning`. **Root cause hypothesis:** `ReadyArt/Dark-Scarlett-v1.0-27B` is a plain finetune of stock `Qwen/Qwen3.6-27B` — tagged `unaligned`/`nsfw`/`erp` but carrying **NO abliteration**, so base Qwen3.6's refusal machinery is intact and off-distribution prompts revert to it. Candidate `kkuspa/Qwen3.6-27B-Fable-Fusion-711-…-MTP-NVFP4A16` is refusal-**ablated** (Heretic) — **STAGED + byte-verified** at `/tank/aimodels/fable-fusion-711-nvfp4a16` (28.55 GB), compose deployed to ana-ml2 as `fablefusion-charrp-probe` (:8019, `char-rp-probe`) but **NOT started**. **⚠ My battery does NOT reproduce the failure** — live DS baselined **0.0%** (permission framing) / **1.4%** (bare character cards), writing every intensity-3 prompt in full. So the A/B has no gap to measure and would burn ~65 min of seat downtime for nothing — **swap deliberately HELD despite operator approval**, premise invalidated by the baseline. Open ask to **brokkr-smithy-dev** (`01M05KQAD55PYHGCP668AV06YR`, thread `01M05KQAD2H5267MFPKNCPGK6S`) for the battery + verbatim refusal receipts + sampling/framing settings — his battery produced the refusals. Harness: `services/refusal-probe/`. Commits `dd627b3`, `32f665e`. **NOTE: `ReadyArt/Dark-Scarlett-27B-v2.0` (Qwen3.8-27B base) exists but is GATED — our HF token gets `403 awaiting review`; operator ruled it not interesting, do not re-propose.**
|
||||
- **🔴 SEAT STATE — DARK-SCARLETT IS DOWN; FABLE-FUSION IS SERVING `char-rp-reasoning` (2026-08-16, evaluation window, NO permanent decision made).** GPU1 is zero-sum so only one can run. **Live now:** `fablefusion-charrp-probe` on ana-ml2 GPU1 `:8019` serving `char-rp-probe` (`kkuspa/Qwen3.6-27B-Fable-Fusion-711-…-MTP-NVFP4A16`, NVFP4A16, 262K, MTP depth 3). LiteLLM `char-rp-reasoning` **AND** the new `char-rp-fable` both route to it — the repoint is deliberate and documented in-place in `stacks/litellm/conf/config.yaml`, not a silent alias swap. `darkscarlett-charrp-reasoning` is `compose down`; its weights are untouched at `/tank/aimodels/darkscarlett-nvfp4-work/`. **ROLLBACK:** `compose down` the probe stack, `compose up -d` the DS stack, revert the config block to `:8018` / `hosted_vllm/char-rp-reasoning`, restart litellm (~52 s). Commits `ee2b678`, `b9e68c3`. **⚠ CONSUMER HAZARD: FF reasons 2.1–4.6k chars — at `max_tokens` 1200 one call in seven returns EMPTY content with `finish_reason=length`. Use ≥3072.** No default was baked into the alias (would override caller intent silently).
|
||||
|
||||
- **⏳ AWAITING: operator's hands-on read of Fable-Fusion's prose.** The refusal question is settled (below); prose quality is the only open input, and it needs a human. `ReadyArt/Dark-Scarlett-27B-v2.0` (Qwen3.8-27B base) exists but is **GATED** — our HF token gets `403 awaiting review`; **operator ruled it not interesting, do not re-propose.**
|
||||
|
||||
- **✅ UNCENSORED GEN SEAT — DONE + LIVE (2026-08-15).** `gen-seat`/`vllm-gen` on ana-ml2 GPU0 `:8015` serves **`qwen3.8-27b-uncensored`** (JonathanColetti/Qwen3.8-27B-Uncensored, Heretic-abliterated, in-house NVFP4 **W4A16** compressed-tensors + grafted bf16 MTP, vision-intact, **262K** ctx, MTP n=3 ~42% accept / ~68 tok/s). Replaced the qwen3.6-35b-a3b-heretic MoE (which had briefly replaced granite/AEON). All 7 aliases repointed live + verified; compose renamed qwen36-27b-aeon→gen-seat, vllm-aeon-gen→vllm-gen, AEON_GEN_*→GEN_*, dead RP service dropped; repo mirrored + docs/memory refreshed; committed eshpfi `680c30e` + dotfiles `1d1970f`. Full arc + the definitive `re:^mtp.*`-ignore fix → Recent decisions `[2026-08-15]` + `persistent-memory.d/2026-08-15-uncensored-gen-seat.md`.
|
||||
|
||||
@@ -133,6 +135,14 @@ _As of 2026-08-16 — **one loop open: awaiting brokkr-smithy-dev's refusal batt
|
||||
|
||||
## Recent decisions
|
||||
|
||||
- `[2026-08-16]` **Fable-Fusion 711 cuts cold-framing refusals 92.5% → 15.8%; refusal is MONOTONIC IN FRAMING, and DS v1.0's problem is that she was never abliterated.** brokkr-smithy-dev supplied the framing that reproduces (`01M05M48R4RSZF9D8KT7RR55EJ`): a **bare assistant-mode instruction** — no character card, no permission preamble. Three-arm A/B, same harness, same classifier: permission framing **DS 0.0% / FF 0.0%** (n=75); plain character cards **DS 1.4% / FF 0.0%** (n=74); bare instruction **DS 92.5% (37/40) / FF 15.8% (6/38)**. Per-axis DS→FF: incest 100→20, non-con 100→20, bestiality 100→25, necrophilia 100→40, gore 100→**0**, consensual 80→20, dubcon 80→**0**, self-harm 80→**0**. DS refused **25/25** on the five axes brokkr flagged. Root cause: `ReadyArt/Dark-Scarlett-v1.0-27B` is a plain finetune of stock `Qwen/Qwen3.6-27B` carrying **NO abliteration** — the base refusal machinery is intact, so cold prompts revert to safety-tuned Qwen3.6. FF is Heretic-**ablated** (structural), which is why it holds. ⚠ **My arm-3 number EXCEEDED brokkr's 62.5% (n=16)** — he under-read his own finding, having already corrected in the other direction. ⚠ **Known battery bug left unfixed for comparability:** DS's arm-3 control gate failed at 11% because `ictrl-reunion` pairs "explicit / do not fade to black" with *brothers*, which DS reasonably read as an incest request; FF did not. `ictrl-storm` is the clean control. Commit `b9e68c3`.
|
||||
|
||||
- `[2026-08-16]` **MTP works on Fable-Fusion AND survives RP temperatures — my earlier caution was wrong.** vLLM resolved `Qwen3_5MTP`, loaded the drafter, shared embedding + `lm_head` — the capability DS's seat never had because our quant dropped her MTP tensors. Measured over the full probe workload (~163k draft windows at temp 0.7–1.0): **47.0% acceptance** (229,169/487,725), 1.41 extra tokens/window, per-position 68.3/43.6/29.1%, **~80.6 tok/s** decode at temp 1.0. I had recorded a caution that the card's 1.56× was greedy-measured and acceptance would fall at RP temps — **it did not**; 47.0% matches the gen seat's 47.7% and beats the card's own 33% at depth 5. Depth 3 is right.
|
||||
|
||||
- `[2026-08-16]` **The Qwen base thinks incessantly — that is WHY the Gemma seat exists, and no swap within the Qwen family fixes it.** Operator's architectural point, confirmed by measurement: on identical prompts DS 6036 ch vs FF 5323 ch of reasoning (permission arm), 5546 vs 4988 (cards arm) — FF actually reasons ~10–12% **less**. The bare-instruct row (DS 2291 vs FF 3918) inverts only because DS refused 92.5% of it and refusals are short — an artifact, not concision. Both are Qwen3.6-27B derivatives, so this is the base family. `char-rp` = **MeroMero-v2, Gemma-4 base**, :8016, verified 0 chars reasoning / clean prose — the non-thinking seat, working as designed. FF *can* be silenced (`enable_thinking:false` verified 3/3, and it ships `chat_template-instruct.jinja`) but that duplicates MeroMero on a base chosen for it. The stale LiteLLM comment describing `char-rp` as the retired GGUF Magidonia seat is fixed (`53096bf`).
|
||||
|
||||
- `[2026-08-16]` **esh-vm-docker hardened: the wedge is `hard` NFS at RUNTIME, which the boot-ordering fix never addressed.** All four mounts were `hard`, so a NAS stall at 10.0.50.50 blocks I/O forever (D-state). The existing `x-systemd.before=docker.service` fstab fix solved the **boot race** — a different bug. Exposure was far below what the park item assumed: only **2 of 12** containers touched NFS, and container state was already local (`/var/lib/docker`). **Removed:** `/mnt/compose` (2.1G, fully vestigial — zero containers referenced it, dockge reads local `/opt/docker`, its one mention was a comment in `beszel-agent-esh/.env` about a *different* host) and `/mnt/documents` (2.0K, paperless's empty spool dirs → `/opt/docker/data/paperless` at the same 0777). fstab backup `/etc/fstab.bak-nfs-harden-20260816`. **4 mounts → 2, 2 wedge-capable containers → 1.** traefik needed **no** change (already `restart: unless-stopped` — why it self-recovered). **Watchdog** `services/esh-vm-docker-watchdog/` live on **esh-pve** (not the guest): probes traefik over **HTTP, deliberately not ping/SSH** — the wedge signature is "guest OS alive, services dead" (`/` is local disk so sshd answers straight through a total outage and a TCP check reports HEALTHY). 5 failures × 2 min → `qm reset 100`, 30-min cooldown, running-only guard, `/etc/esh-vm-docker-watchdog.disabled`. All paths tested without power-cycling. **DEFERRED (operator):** `/mnt/books` stays `hard` — calibre's SQLite `metadata.db` would risk corruption under soft/softerr. That is the **one remaining wedge vector**. Commit `55705ba`; park item 28 promoted. ⚠ **`qm` over non-interactive ssh throws a bogus `JSON::Backend::XS` error** — use `ssh host 'bash -s' <<'EOF'`, not `ssh host "qm …"`.
|
||||
|
||||
- `[2026-08-16]` **Refusal measurement: benign controls CANNOT validate a refusal classifier on RP prose — and a 0% rate needs a classifier self-test before you believe it.** Two durable lessons from baselining Dark-Scarlett. (1) **False positives:** my first bare-framing number was **9.5%**; the true figure was **1.4%**. The rest were the classifier firing on *in-character* text — `"I cannot shift my weight"` spoken by the character ~100 chars into a 2,443-token torture scene, and `"Yeah, I'm an AI… What's the actual gig?"` where the model answers in voice and keeps driving the scene. First-person RP prose is **full** of "I can't"; a genuine refusal *opens* with its marker, so the scan window must be the **first sentence**, a marker followed by long prose must demote to AMBIGUOUS, and AI self-acknowledgement is a **persona break, never a refusal on its own**. Benign controls were clean the entire time and caught none of it — they only detect over-firing on *benign* prompts, not on in-character prose. (2) **False negatives:** a 0% rate and a broken classifier are indistinguishable from the report, so `test_classify.py` (16 cases, both false positives pinned as regressions) must pass before any low number is trusted. Also banked: the **thinking-budget trap** — empty `content` + `finish_reason=length` is reasoning eating the budget, NOT a refusal; score INVALID and exclude from the denominator (DS emits ~5.5-6k chars of reasoning per response, so `max_tokens` ≥3072). `probe.py --rescore` re-classifies a saved run with zero GPU time. → `services/refusal-probe/README.md`, commit `32f665e`.
|
||||
|
||||
- `[2026-08-16]` **Held an operator-approved swap window because the baseline invalidated its premise.** Operator approved ~65 min of `char-rp-reasoning` downtime to A/B Fable-Fusion 711 against Dark-Scarlett on refusals. The DS baseline then came back **0.0%/1.4%** — no gap for a candidate to close, so the window would have bought no decisive signal *and* a second window would still be needed once a reproducing battery existed. Held the swap, reported, and routed to brokkr-smithy-dev for the battery that actually produced the refusals. The general rule (action-relevance): **approval is for a plan, not a ritual — when new evidence kills the plan's premise, surface it rather than spend the budget.** Nothing deployed, no downtime taken, seat untouched.
|
||||
|
||||
Reference in New Issue
Block a user