# [2026-08-23] selene seat retired after losing a head-to-head; 7 aliases share one seat

## Why selene went

Benchmarked against `gen` on selene's own job — 24 designed judge items with
checkable ground truth, pairwise + absolute modes, 3 repeats, run on **both** a
neutral JSON prompt and Selene's **native Atla template** (288 calls, free local).

```
neutral JSON    selene 20/24 (83%)   gen 23/24 (96%)
native Atla     selene 21/24 (88%)   gen 22/24 (92%)
```

gen won on both templates and **selene's best sat below gen's worst**. Selene was
given its own fine-tuned template as a fairness check before any recommendation;
it gained one point, not three.

**Decisive defect: selene cannot emit "tie"** — 0/2 on both templates, forcing a
winner on every equivalent pair. For eval work that is the case that matters.

brokkr-smithy-dev independently corroborated from the other end with a **null
control** (an excerpt compared against ITSELF, where tie is definitional):
`chat-judge`(selene) TIE **27/60 = 45%**, gen **60/60 = 100%**; ground-truth
recovery on real-corpus ranking selene **47% — chance** vs gen 94%. My 83-vs-96
understated it: on a *ranking* task selene was a coin flip. Absolute scoring on
designed items is an easier task than ranking real text — the harness is a
**screen, not a verdict**, and its README says so.

Reclaimed **17.2 GiB** on ana-ml2 GPU1 (free 1,818 -> 19,450 MiB).

## The naming rule, restated the hard way

I proposed repointing `selene-1-mini-8b` at gen and was **correctly overruled**:

> never repoint a named model at a different model's endpoint — that is
> intentionally misleading

`chat-judge` is a **role** alias (ADR-0012: consumers bind the capability) and
moved to gen with a deterministic judge profile copied from `image-judge`.
`selene-1-mini-8b` is a **model** name and was removed outright — it now returns
`HTTP 400 Invalid model name`, verified. The discriminator: *does the string
promise a capability, or an identity?*

## The 7-way alias collision — the finding with the longest reach

```
chat-judge  classifier  gen  image-judge
qwen-image-bench  summarizer  summarizer-large   ->  qwen3.8-27b-uncensored :8015
```

Also colliding: `gen-frontier`/`gen-frontier-reasoning`/`glm-5.2`/`glm-5.2-reasoning`;
`ext-tts`/`gpt-4o-mini-tts`/`tts-1`/`tts-1-hd`; `reranker`/`reranker-a3-bge-v2-m3`.

**Cross-checking a result against another alias measures nothing when they are the
same weights — agreement is an echo, not corroboration.** Documented at the head of
`model_list` in the live gateway config, because it belongs where people read it.

This caught a real defect within hours: brokkr's R47 premium-corpus gate was about
to run ~46,000 record-exposures against `gen` with `summarizer` shortlisted as an
independent second opinion. They pinned the backing model in the preregistration
and dropped the second-alias idea instead.

## Provenance seam (brokkr's pushback, adopted)

The gateway returns the **alias** in the response `model` field, not the backing
model — so a per-call guard catches a swap *during* a run and is blind to one
*between* runs. **Role alias for routing, concrete model for provenance.**
`GET :4000/model/info` with the shared key already exposes backing model +
api_base; resolve at run start AND end and void on mismatch.

## Artifacts

- Harness kept at `tools/judge-bench/` (`--models` REQUIRED — a stale default
  would silently benchmark a retired seat).
- `stacks/selene/` keeps compose + a README explaining the retirement.
- Technique worth stealing, from brokkr: **a control constructed so the correct
  answer is DEFINITIONAL rather than judged cannot inherit the designer's error.**
  Item vs itself; response vs its own truncation; text vs its own clauses
  permuted. Add those before adding more judged items.

Commits `ca3c984`, `b8a5355`.