chore: retire asset-engine (Prime 2026-10-17); kuma gains delete-by-name; FleetTools/bootstrap/catalog marked
This commit is contained in:
@@ -1,5 +1,10 @@
|
||||
# services.yaml — contract for downstream consumers
|
||||
|
||||
> **RETIRED 2026-10-17 (Prime).** asset-engine, this catalog's only consumer, is `compose down` on ana-docker.
|
||||
> The catalog is NOT maintained: every irv-ml1 endpoint in `services.yaml` still points at `10.100.79.3`, the wg0
|
||||
> lifeline that died at the 09-06 mesh cutover, and several of those ports now belong to other services. Kept as
|
||||
> the record and for a bring-back, which would need a full per-entry refresh first.
|
||||
|
||||
`services.yaml` in this directory is the **canonical, versioned**
|
||||
description of the inference services this fleet exposes. It is a
|
||||
first-class artifact: external tools (the asset-engine UI is the
|
||||
|
||||
@@ -17,7 +17,6 @@ their own:
|
||||
|
||||
```sh
|
||||
curl -s http://10.0.50.45:5100/api/services # every fleet service, curated
|
||||
curl -s http://10.250.50.70:8200/api/v1/services # inference seats + GPU state
|
||||
curl -s http://10.250.50.70:4000/v1/models # the LLM roster (needs a key)
|
||||
```
|
||||
|
||||
|
||||
@@ -14,12 +14,11 @@ directory; that defeats the point.
|
||||
|
||||
## Rule zero: query live inventories, never a written list
|
||||
|
||||
Three endpoints answer "what exists right now" better than any document, and they stay
|
||||
Two endpoints answer "what exists right now" better than any document, and they stay
|
||||
correct on their own. **Reach for these before asking anyone:**
|
||||
|
||||
```sh
|
||||
curl -s http://10.0.50.45:5100/api/services # every fleet service, curated
|
||||
curl -s http://10.250.50.70:8200/api/v1/services # inference seats + GPU state
|
||||
curl -s -H "Authorization: Bearer $KEY" \
|
||||
http://10.250.50.70:4000/v1/models # the LLM roster (~40 models)
|
||||
```
|
||||
@@ -70,13 +69,13 @@ live contract for its API. Fetch it rather than trusting a transcription.
|
||||
⚠ *The shared key CAN spend real vendor credits.*
|
||||
*Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/litellm.md`
|
||||
|
||||
- **Direct inference seats + Asset Engine** — the control plane at
|
||||
`http://10.250.50.70:8200` enumerates seats, reads GPU state, and starts/stops them.
|
||||
- **Direct inference seats** — most sit behind the gateway; the GPU hosts and their traps.
|
||||
(The Asset Engine control plane at `:8200` was RETIRED 2026-10-17.)
|
||||
*When:* the gateway does not expose what you need, or a seat is down.
|
||||
*Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/inference.md`
|
||||
|
||||
- **Speech (TTS/ASR)** — Breeze-TTS-2 is the fleet primary; Chatterbox-fast is the
|
||||
low-latency chat front end; Speaches and Parakeet do ASR.
|
||||
low-latency chat front end; Parakeet does ASR (`whisper-1` / `ext-stt` on the gateway).
|
||||
*When:* you need audio out of text, or text out of audio.
|
||||
*Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/speech.md`
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Direct inference seats + the Asset Engine control plane
|
||||
# Direct inference seats
|
||||
|
||||
Most seats are vLLM behind the LiteLLM gateway — prefer the gateway
|
||||
(`docs/fleettools/litellm.md`) unless you need something it does not expose.
|
||||
@@ -8,30 +8,20 @@ Most seats are vLLM behind the LiteLLM gateway — prefer the gateway
|
||||
```sh
|
||||
# every fleet service, curated, with hrefs and descriptions
|
||||
curl -s http://10.0.50.45:5100/api/services | python3 -m json.tool
|
||||
|
||||
# the inference control plane: what exists, what is up, GPU state
|
||||
curl -s http://10.250.50.70:8200/api/v1/services
|
||||
curl -s http://10.250.50.70:8200/api/v1/gpus
|
||||
# the LLM roster
|
||||
curl -s -H "Authorization: Bearer $KEY" http://10.250.50.70:4000/v1/models
|
||||
```
|
||||
|
||||
Seats move between hosts and GPUs often enough that any transcribed table here would
|
||||
mislead within a month.
|
||||
|
||||
## Asset Engine — `http://10.250.50.70:8200`
|
||||
## Asset Engine — RETIRED 2026-10-17
|
||||
|
||||
Control plane over the PFI inference fleet. Can enumerate services, read GPU state,
|
||||
**start and stop seats**, submit jobs and fetch assets.
|
||||
|
||||
```
|
||||
GET /api/v1/services GET /api/v1/services/{id}/state
|
||||
POST /api/v1/services/{id}/start POST /api/v1/services/{id}/stop
|
||||
GET /api/v1/gpus POST /api/v1/jobs
|
||||
GET /api/v1/assets/{id}/blob GET /api/v1/presets
|
||||
```
|
||||
|
||||
⚠ **Liveness is keyed on the compose PROJECT name.** A seat reads OFFLINE unless the
|
||||
compose project name equals the catalog's `lifecycle.stack`. Pin `name: <stack>` in
|
||||
the compose file.
|
||||
The control plane at `http://10.250.50.70:8200` is gone (Prime). Its last job was 2026-08-02, and every
|
||||
catalog endpoint pointed at irv-ml1's dead wg0 address after the 09-06 mesh cutover. `docker compose
|
||||
down` on ana-docker; the DB, image, repo (`vh/asset-engine`) and catalog (`docs/asset-engine/`) are kept, so
|
||||
`docker compose up -d` in `/opt/docker/compose/asset-engine` brings it back. Start/stop a seat by hand
|
||||
(`docker compose up -d` / `stop` in its compose dir on the GPU host).
|
||||
|
||||
## GPU hosts
|
||||
|
||||
|
||||
@@ -33,7 +33,7 @@ the router cooldown** — list voices first.
|
||||
|
||||
Retired, verified 2026-10-17: **Speaches** (irv-ml1 :8204 is now Breeze-TTS-2), the sherpa-onnx `parakeet` container
|
||||
on fv-ml1 (stopped; NeMo replaced it), and the asset-engine `parakeet` entry at `10.100.79.3:8765` (the dead wg0
|
||||
lifeline address). The first two's leftover containers were removed 2026-10-17 (images and compose dirs kept), so
|
||||
lifeline address; asset-engine itself was retired 2026-10-17). The first two's leftover containers were removed 2026-10-17 (images and compose dirs kept), so
|
||||
Homepage no longer lists them.
|
||||
|
||||
## Traps
|
||||
|
||||
@@ -30,7 +30,7 @@ Sister repos (separate gitea repos, deployed by playbooks here):
|
||||
| `vh/task-board` | MCP + web dashboard for assistant task state (port 7878) — **[2026-09-24] MOTHBALLED** by Prime (superseded by the High Seat + ledger); container removed on ana-docker, data/image/compose kept (`e6da607`) | push-to-main → CI deploys (2026-04-29) |
|
||||
| `vh/vor` | Inquisitor UI sidecar (port 7879) | push-to-main → CI deploys (2026-04-29) |
|
||||
| `vh/nevermore` | Twice-daily LLM-curated briefing (port 8181, replaces news-digest) | push-to-main → CI deploys (2026-04-30) |
|
||||
| `vh/asset-engine` | Internal control plane over inference services (port 8200, LAN-direct) | push-to-main → CI deploys (2026-05-12) |
|
||||
| `vh/asset-engine` | Internal control plane over inference services (port 8200, LAN-direct) — **[2026-10-17] RETIRED** by Prime: `compose down` on ana-docker (DB in `/opt/docker/conf/asset-engine/db` incl. its unmerged WAL, image `asset-engine:local`, repo all kept; bring-back = `docker compose up -d`). Last job 2026-08-02, and every catalog endpoint had been dead since the 09-06 mesh cutover. ⚠ CI still deploys on a push to main, so a push would resurrect it | push-to-main → CI deploys (2026-05-12) |
|
||||
| `vh/althing` | Lean trusted inter-agent message bus — **v3.0.0 "the post office" as of 2026-08-28 (U9b flag day, one-way, no rollback)**: ONE container on nh3-dev at `http://10.100.50.40:8390` is the only stateful component; `althing-po-herald` one per box; `althing-listen` one per session; `postbox` is the client. **Every v2 command was DELETED, not deprecated** — `althing-cli`→`postbox`, `althing-wake-listener`→`althing-listen`, `althing-light-monitor`/`althing-receiver` gone. Sessions need BOTH `ALTHING_POST_OFFICE` and `ALTHING_HANDLE`; there is no default address. ⚠ An unreachable post office is an OUTAGE, never an empty inbox. → `persistent-memory.d/2026-08-28-althing-v3-cutover.md` | per-box install (NOT CI-deploy); **nh3-dev** = container host + repo; **nh3-extdev** = system WHEEL at `/opt/uv-tools`, needs its own wheel install (`playbooks/nh3-extdev-althing-v3.yaml`) |
|
||||
| `vh/mead-hall` | Bifrost tool-provider sidecar (port 5173 on dev VM 10.100.10.50) | push-to-main → CI deploys (2026-05-16) |
|
||||
| `vh/skaldsong` | Wizard + reader surface (port 8300, ana-docker, registry-pull pattern) | push-to-main → CI deploys (2026-05-19) |
|
||||
@@ -185,6 +185,7 @@ Every vLLM seat on a shared card (GPU 0, GPU 1) has byte-pinned KV (`--kv-cache-
|
||||
|
||||
## Recent decisions
|
||||
|
||||
- `[2026-10-17]` **asset-engine RETIRED (Prime)**, along with the stale speech entries: the Homepage Speaches (irv-ml1) and sherpa Parakeet (fv-ml1) ghosts were removed via `docker rm` of their dead containers (images and compose dirs kept); the Uptime Kuma monitor was deleted (`scripts/kuma delete`, new); asset-engine is `compose down` (data kept). Evidence: last job 08-02; all 21 catalog endpoints on the dead 10.100.79.3. FleetTools, the agent bootstrap and global CLAUDE.md (dotfiles f909dcf, unpushed) no longer list `:8200`.
|
||||
- `[2026-10-16]` **albok unit 6: the first encrypted wing `personal/sealed` provisioned on nh3-albok** (Prime's goal "land all of u6", via albok-dev): a 20 GiB LUKS2 file + ext4, key ONLY in vault `albok/wing-key-personal-sealed`, gid 1513, mountpoint `/srv/albok/sealed/personal.sealed` (immutable while unmounted), the parent bound rslave. The lock/unlock lifecycle was tested with the container running. It is not in albok.yaml until albok-dev's release. Runbook: `stacks/albok-service/README.md` § Sealed. After an nh3-albok reboot the wing is LOCKED until someone unlocks it.
|
||||
- `[2026-10-13]` **pewpewstudio LIVE on nh3-docker :8185** (Prime via pewpew-dev): api + Caddy front; the Host allowlist must sit in a `route` (foreign Host 400); DB, assets and shop config on named volumes. → `stacks/pewpewstudio/README.md`
|
||||
- `[2026-10-13]` **infra-ops on esh-nas (CT 103) + per-host satellite keys (Prime)** — bootstrapped via pct from esh-pve-nas; satellite-1 authorized on esh-nas only (`playbooks/files/infra-ops-satellite-keys`); its login test + name pending.
|
||||
|
||||
@@ -421,6 +421,25 @@ def cmd_add(args):
|
||||
k.__exit__()
|
||||
|
||||
|
||||
def cmd_delete(args):
|
||||
"""Remove ONE monitor by its exact name. `seed` never prunes (a missing row in
|
||||
monitors.yaml is not a delete), so retiring a service needs this as well as the
|
||||
yaml edit. Exits 1 if the name is not on the board."""
|
||||
k = connected_and_logged_in()
|
||||
try:
|
||||
mons = k.monitors_by_name()
|
||||
if args.name not in mons:
|
||||
print(f"no monitor named {args.name!r}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
k.delete(mons[args.name]["id"])
|
||||
if args.name in k.monitors_by_name():
|
||||
print(f"delete of {args.name!r} did not take", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
print(f"deleted {args.name}")
|
||||
finally:
|
||||
k.__exit__()
|
||||
|
||||
|
||||
def main():
|
||||
p = argparse.ArgumentParser(prog="kuma", description=__doc__.split("\n")[0])
|
||||
sub = p.add_subparsers(dest="cmd", required=True)
|
||||
@@ -432,6 +451,7 @@ def main():
|
||||
a.add_argument("--type", default="http", choices=["http", "ping", "port"])
|
||||
a.add_argument("--port", type=int)
|
||||
a.set_defaults(fn=cmd_add)
|
||||
d = sub.add_parser("delete"); d.add_argument("name"); d.set_defaults(fn=cmd_delete)
|
||||
sub.add_parser("dedupe").set_defaults(fn=cmd_dedupe)
|
||||
sub.add_parser("channels").set_defaults(fn=cmd_channels)
|
||||
args = p.parse_args()
|
||||
|
||||
@@ -89,8 +89,7 @@ monitors:
|
||||
url: http://10.250.50.70:4000/health/liveliness
|
||||
description: liveliness endpoint, so a model outage does not read as a gateway outage
|
||||
|
||||
- name: Asset Engine
|
||||
url: http://10.250.50.70:8200/api/v1/services
|
||||
# Asset Engine: RETIRED 2026-10-17 (Prime), monitor deleted with `scripts/kuma delete`.
|
||||
|
||||
# ---- fleet embed/rerank: the one EXCEPTION to "seats are OUT" ----
|
||||
# Since 2026-09-25 these back the gateway's `qwen3-embedding` and `reranker`
|
||||
|
||||
Reference in New Issue
Block a user