# Direct inference seats

Most seats are vLLM behind the LiteLLM gateway — prefer the gateway
(`docs/fleettools/litellm.md`) unless you need something it does not expose.

## ⚠ Query the live inventory, never a written list

```sh
# every fleet service, curated, with hrefs and descriptions
curl -s http://10.0.50.45:5100/api/services | python3 -m json.tool
# the LLM roster
curl -s -H "Authorization: Bearer $KEY" http://10.250.50.70:4000/v1/models
```

Seats move between hosts and GPUs often enough that any transcribed table here would
mislead within a month.

## Asset Engine — RETIRED 2026-10-17

The control plane at `http://10.250.50.70:8200` is gone (Prime). Its last job was 2026-08-02, and every
catalog endpoint pointed at irv-ml1's dead wg0 address after the 09-06 mesh cutover. `docker compose
down` on ana-docker; the DB, image, repo (`vh/asset-engine`) and catalog (`docs/asset-engine/`) are kept, so
`docker compose up -d` in `/opt/docker/compose/asset-engine` brings it back. Start/stop a seat by hand
(`docker compose up -d` / `stop` in its compose dir on the GPU host).

## GPU hosts

| host | GPUs | notes |
|---|---|---|
| fv-ml1 `10.251.50.54` | 4× RTX PRO 6000 Blackwell Max-Q, 96 GB ea (391 GB) | primary; native FP8/NVFP4 |
| ana-ml2 `10.250.50.54` | see live inventory | |
| irv-ml1 `irv-ml1.nh3.internal` | RTX 3090 + RTX A6000 (72 GB) | **Ampere sm_86: no native FP8/NVFP4** |

Every seat is FastAPI — **`GET /openapi.json` on any of them is the live contract.**
Do not transcribe endpoint shapes; fetch them.

## Traps

- **Idle VRAM is reserved, not waste.** Single-user dev fleet. Never propose raising
  `gpu-memory-utilization` to fill free VRAM.
- **vLLM temp 0 is not deterministic under concurrency** — ~3.12pp replicate spread at
  `jobs=8`. Any eval delta under ~3.1pp is instrument floor, not signal. Only
  `--jobs 1` shrinks it.
- **Never alias a model under a stale name.** Serve under the TRUE served-name only;
  let the old name 404 and notify consumers. No silent substitution.
- **Ollama is banned fleet-wide.** Never stand one up; tear down any found. Use
  llama-swap or vLLM.
- **`docker compose up -d` recreates the WHOLE stack** when a shared `.env` changes.
  Target one service: `up -d <service>`.