# gen-seat — the fleet `gen` seat (ana-ml2 GPU0, :8015)

Serves **`qwen3.8-27b-uncensored`** (JonathanColetti/Qwen3.8-27B-Uncensored,
Heretic-abliterated Qwen3.8-27B, vision-intact, 262K context) plus the
`-thinking` served-name for the reasoning split.

Backs **7 LiteLLM aliases**: `gen`, `gen-reasoning`, `summarizer`,
`summarizer-large`, `classifier`, `image-judge`, `qwen-image-bench`. Treat any
change here as fleet-wide.

- **Quant:** in-house **mixed-precision** — NVFP4 W4A4 for layers 0-55 MLPs,
  FP8 W8A8 for attention / `linear_attn` / `lm_head` / layers 56-63 MLPs, FP8 KV.
  ~94.5 tok/s decode at bs=1, MTP n=3 @ ~48% acceptance.
- **Speculative decoding:** `qwen3_5_mtp`, `num_speculative_tokens=3` (measured
  optimum — see the quant service README).
- **Tunables:** `.env` on the host. ⚠ it is mode 0600 / lkraven-owned, so every
  `docker compose` call against this stack needs `sudo` — without it compose
  cannot read `.env`, fails with `permission denied`, and leaves the old
  container running while appearing to have succeeded.

## Pipeline, acceptance gate, and rollback

All of it — the recipe, the `re:^mtp.*` foot-gun, the benchmark harness, the GPU0
budget interaction with `meromero-charrp`, and the rollback command — lives in
[`services/gen-seat-mixed-quant/README.md`](../../services/gen-seat-mixed-quant/README.md).

## Deploy

```bash
scripts/deploy-stack.sh ana-ml2 gen-seat        # diffs vs live, prompts y/N
# on host:
ssh infra-ops@10.250.50.54
cd /opt/docker/compose/gen-seat && sudo docker compose up -d vllm-gen
```