# Direct inference seats + the Asset Engine control plane

Most seats are vLLM behind the LiteLLM gateway — prefer the gateway
(`docs/fleettools/litellm.md`) unless you need something it does not expose.

## ⚠ Query the live inventory, never a written list

```sh
# every fleet service, curated, with hrefs and descriptions
curl -s http://10.0.50.45:5100/api/services | python3 -m json.tool

# the inference control plane: what exists, what is up, GPU state
curl -s http://10.250.50.70:8200/api/v1/services
curl -s http://10.250.50.70:8200/api/v1/gpus
```

Seats move between hosts and GPUs often enough that any transcribed table here would
mislead within a month.

## Asset Engine — `http://10.250.50.70:8200`

Control plane over the PFI inference fleet. Can enumerate services, read GPU state,
**start and stop seats**, submit jobs and fetch assets.

```
GET  /api/v1/services              GET  /api/v1/services/{id}/state
POST /api/v1/services/{id}/start   POST /api/v1/services/{id}/stop
GET  /api/v1/gpus                  POST /api/v1/jobs
GET  /api/v1/assets/{id}/blob      GET  /api/v1/presets
```

⚠ **Liveness is keyed on the compose PROJECT name.** A seat reads OFFLINE unless the
compose project name equals the catalog's `lifecycle.stack`. Pin `name: <stack>` in
the compose file.

## GPU hosts

| host | GPUs | notes |
|---|---|---|
| fv-ml1 `10.251.50.54` | 4× RTX PRO 6000 Blackwell Max-Q, 96 GB ea (391 GB) | primary; native FP8/NVFP4 |
| ana-ml2 `10.250.50.54` | see live inventory | |
| irv-ml1 `irv-ml1.nh3.internal` | RTX 3090 + RTX A6000 (72 GB) | **Ampere sm_86: no native FP8/NVFP4** |

Every seat is FastAPI — **`GET /openapi.json` on any of them is the live contract.**
Do not transcribe endpoint shapes; fetch them.

## Traps

- **Idle VRAM is reserved, not waste.** Single-user dev fleet. Never propose raising
  `gpu-memory-utilization` to fill free VRAM.
- **vLLM temp 0 is not deterministic under concurrency** — ~3.12pp replicate spread at
  `jobs=8`. Any eval delta under ~3.1pp is instrument floor, not signal. Only
  `--jobs 1` shrinks it.
- **Never alias a model under a stale name.** Serve under the TRUE served-name only;
  let the old name 404 and notify consumers. No silent substitution.
- **Ollama is banned fleet-wide.** Never stand one up; tear down any found. Use
  llama-swap or vLLM.
- **`docker compose up -d` recreates the WHOLE stack** when a shared `.env` changes.
  Target one service: `up -d <service>`.