docs: add FleetTools — an agent-family-agnostic index of fleet capability

Every agent on nh3-dev — Claude, Codex, Grok, Aider — needs the same
answers: what runs here, how do I call it, what will bite me. Until now
that lived in ~/.claude/CLAUDE.md, which only Claude sessions load, and
it was interleaved with operator preferences that other families have
no use for.

Two-tier by design, matching the persistent-memory split: FLEETTOOLS.md
is a 135-line index an agent reads whole, and each entry links to a
detail file it opens only when it actually needs that tool. Reading the
index costs about a fifth of reading the tree.

Detail paths are absolute so they resolve from any working directory,
since a non-Claude agent will cat the path rather than follow a
markdown link. ~/FLEETTOOLS.md symlinks to the index for discovery.

Rule zero is that live inventories get queried, not transcribed:
Homepage /api/services, asset-engine /api/v1/services, LiteLLM
/v1/models, and every FastAPI seat's /openapi.json. A copied service
table would be stale within a month and this repo already has a
standing rule against second copies that drift.

Contents verified against the running fleet rather than copied from
existing docs: binaries resolved on PATH, seven endpoints probed live,
the LiteLLM roster counted at 40 models where the old note said ~30.
No credentials are included; the vault and its CLI are pointed at
instead.
This commit is contained in:
vh
2026-09-18 07:35:28 -07:00
parent d6a9d70b9e
commit 53c3e8000e
15 changed files with 737 additions and 0 deletions
+59
View File
@@ -0,0 +1,59 @@
# Direct inference seats + the Asset Engine control plane
Most seats are vLLM behind the LiteLLM gateway — prefer the gateway
(`docs/fleettools/litellm.md`) unless you need something it does not expose.
## ⚠ Query the live inventory, never a written list
```sh
# every fleet service, curated, with hrefs and descriptions
curl -s http://10.0.50.45:5100/api/services | python3 -m json.tool
# the inference control plane: what exists, what is up, GPU state
curl -s http://10.250.50.70:8200/api/v1/services
curl -s http://10.250.50.70:8200/api/v1/gpus
```
Seats move between hosts and GPUs often enough that any transcribed table here would
mislead within a month.
## Asset Engine — `http://10.250.50.70:8200`
Control plane over the PFI inference fleet. Can enumerate services, read GPU state,
**start and stop seats**, submit jobs and fetch assets.
```
GET /api/v1/services GET /api/v1/services/{id}/state
POST /api/v1/services/{id}/start POST /api/v1/services/{id}/stop
GET /api/v1/gpus POST /api/v1/jobs
GET /api/v1/assets/{id}/blob GET /api/v1/presets
```
⚠ **Liveness is keyed on the compose PROJECT name.** A seat reads OFFLINE unless the
compose project name equals the catalog's `lifecycle.stack`. Pin `name: <stack>` in
the compose file.
## GPU hosts
| host | GPUs | notes |
|---|---|---|
| fv-ml1 `10.251.50.54` | 4× RTX PRO 6000 Blackwell Max-Q, 96 GB ea (391 GB) | primary; native FP8/NVFP4 |
| ana-ml2 `10.250.50.54` | see live inventory | |
| irv-ml1 `irv-ml1.nh3.internal` | RTX 3090 + RTX A6000 (72 GB) | **Ampere sm_86: no native FP8/NVFP4** |
Every seat is FastAPI — **`GET /openapi.json` on any of them is the live contract.**
Do not transcribe endpoint shapes; fetch them.
## Traps
- **Idle VRAM is reserved, not waste.** Single-user dev fleet. Never propose raising
`gpu-memory-utilization` to fill free VRAM.
- **vLLM temp 0 is not deterministic under concurrency** — ~3.12pp replicate spread at
`jobs=8`. Any eval delta under ~3.1pp is instrument floor, not signal. Only
`--jobs 1` shrinks it.
- **Never alias a model under a stale name.** Serve under the TRUE served-name only;
let the old name 404 and notify consumers. No silent substitution.
- **Ollama is banned fleet-wide.** Never stand one up; tear down any found. Use
llama-swap or vLLM.
- **`docker compose up -d` recreates the WHOLE stack** when a shared `.env` changes.
Target one service: `up -d <service>`.