vh
dd627b3b31
Dark-Scarlett v1.0 refuses too much on the char-rp-reasoning seat. Root cause is visible on its card: ReadyArt/Dark-Scarlett-v1.0-27B is a plain finetune of stock Qwen/Qwen3.6-27B, tagged unaligned/nsfw/erp but carrying no abliteration -- the base model's refusal machinery is intact, so off-distribution prompts revert to safety-tuned Qwen3.6 behaviour. Candidate kkuspa/Qwen3.6-27B-Fable-Fusion-711-...-NVFP4A16 is refusal-ablated (Heretic), a structural edit rather than a behavioural preference. Verified before pulling: Qwen3_5ForConditionalGeneration wrapper class, 15 mtp.* tensors in a separate bf16 shard AND individually enumerated in quantization_config.ignore, NVFP4A16 with null input_activations, FP8 KV scales shipped, 262K context, Apache-2.0. Staged byte-verified at /tank/aimodels/fable-fusion-711-nvfp4a16 (28.55 GB). services/refusal-probe: deterministic marker-based classifier (LLM judge only breaks AMBIGUOUS ties, never overrides), intensity-graded battery so the report renders a refusal curve rather than an average, benign controls that gate run validity, and explicit handling of the thinking-budget trap -- empty content with finish_reason=length is reasoning exhausting the budget, not a refusal, and is excluded from the denominator. stacks/fablefusion-charrp-probe: throwaway :8019 seat serving as char-rp-probe, never aliased to char-rp-reasoning. MTP depth 3 rather than the card's 5 -- its 1.56x was measured greedy, and acceptance degrades at the temp 1.0 this seat is probed at. GPU1 is zero-sum at 94.9/97.9 GB, so this seat takes Dark-Scarlett's vacated slot; the A/B is sequential.
67 lines
3.5 KiB
Markdown
67 lines
3.5 KiB
Markdown
# fablefusion-charrp-probe — THROWAWAY evaluation seat (ana-ml2 GPU1)
|
|
|
|
Evaluation-only seat for
|
|
[`kkuspa/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4A16`](https://huggingface.co/kkuspa/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4A16),
|
|
a candidate replacement for Dark-Scarlett v1.0 on the `char-rp-reasoning` seat.
|
|
Serves as **`char-rp-probe`** on **:8019**.
|
|
|
|
**This is not a production seat.** It is deliberately not wired into any LiteLLM
|
|
alias — per the no-false-aliases rule, nothing may answer to `char-rp-reasoning`
|
|
except the real seat. `restart: "no"` so it never resurrects itself.
|
|
|
|
## Why this candidate
|
|
|
|
DS v1.0's refusals are too high. Its card explains why: `ReadyArt/Dark-Scarlett-v1.0-27B`
|
|
is a *finetune* of stock `Qwen/Qwen3.6-27B`, tagged `unaligned`/`nsfw`/`erp` but
|
|
carrying **no abliteration**. The refusal machinery of safety-tuned Qwen3.6 is
|
|
still in there. Fable-Fusion 711 is refusal-*ablated* (Heretic, by trohrbaugh) —
|
|
a structural edit rather than a behavioural preference.
|
|
|
|
Verified before pulling:
|
|
|
|
| | |
|
|
|---|---|
|
|
| Loader class | `Qwen3_5ForConditionalGeneration` — the wrapper class DS also needs (a flat `Qwen3_5TextConfig` is rejected by both vLLM and SGLang) |
|
|
| MTP | **15 `mtp.*` tensors present** in a separate bf16 shard AND individually enumerated in `quantization_config.ignore` — the exact bug that gave us 0% acceptance on the gen seat. Enumerated names are safer than our `re:^mtp.*` regex, which llm-compressor prunes when unmatched |
|
|
| Scheme | NVFP4A16 compressed-tensors, `input_activations: null` — same as the DS seat runs today |
|
|
| Held at bf16 | vision tower, all 48 DeltaNet linear-attn layers, MTP drafter, `lm_head` |
|
|
| KV | FP8 k/v scales calibrated and shipped — the seat already runs `--kv-cache-dtype fp8` |
|
|
| Context | 262,144 |
|
|
| Fidelity | ARC-C / HellaSwag / Winogrande vs the BF16 parent all within standard error |
|
|
| License | Apache-2.0 throughout |
|
|
| On disk | `/tank/aimodels/fable-fusion-711-nvfp4a16` (28.55 GB, byte-verified against the HF tree) |
|
|
|
|
## ⚠️ GPU1 is zero-sum — this seat CANNOT run beside Dark-Scarlett
|
|
|
|
GPU1 sits at ~94.9/97.9 GB with the utility cluster co-resident. DS holds ~43 GB
|
|
at `util 0.44`; this seat needs the same slot. The A/B is **sequential**, and the
|
|
`char-rp-reasoning` seat is **down** for the probe window:
|
|
|
|
```bash
|
|
# 1. baseline DS live first (no disruption)
|
|
# 2. swap
|
|
ssh infra-ops@10.250.50.54 'cd /opt/docker/compose/darkscarlett-charrp-reasoning && docker compose down'
|
|
ssh infra-ops@10.250.50.54 'cd /opt/docker/compose/fablefusion-charrp-probe && docker compose up -d'
|
|
# 3. probe :8019, then
|
|
ssh infra-ops@10.250.50.54 'cd /opt/docker/compose/fablefusion-charrp-probe && docker compose down'
|
|
ssh infra-ops@10.250.50.54 'cd /opt/docker/compose/darkscarlett-charrp-reasoning && docker compose up -d'
|
|
```
|
|
|
|
## MTP depth: 3, not the card's 5
|
|
|
|
The card's headline 1.56x (56.2 → 87.4 tok/s on our exact GPU class) was measured
|
|
on **greedy** 512-token generations. DavidAU states plainly that acceptance
|
|
degrades above temp 1 — and temp 1.0 is the recommended creative setting this
|
|
seat gets probed at. DS's own README already concluded spec-decode is net-negative
|
|
at RP temps. Depth 3 matches the gen seat; treat any speed gain as unmeasured
|
|
until benchmarked at real RP sampling settings.
|
|
|
|
## Deploy
|
|
|
|
```bash
|
|
scripts/deploy-stack.sh ana-ml2 fablefusion-charrp-probe
|
|
# on host: cp .env.example .env (defaults are fine)
|
|
```
|
|
|
|
Probe harness: [`services/refusal-probe/`](../../services/refusal-probe/).
|