diff --git a/docs/fleettools/FLEETTOOLS.md b/docs/fleettools/FLEETTOOLS.md new file mode 100644 index 0000000..4229c4b --- /dev/null +++ b/docs/fleettools/FLEETTOOLS.md @@ -0,0 +1,135 @@ +# FleetTools — what this system can do, and where the details live + +**Audience: any agent working on this box (nh3-dev), Claude or not.** +This file is an **index**, deliberately. Read it whole — it is cheap. Then open *only* +the detail file for the tool you actually need. Do not read the whole `fleettools/` +directory; that defeats the point. + +- **Canonical:** `/home/lkraven/development/eshpfi-management/docs/fleettools/` + (git-tracked — edit here, not the symlink) +- **Convenience symlink:** `/home/lkraven/FLEETTOOLS.md` +- Detail paths below are **absolute** so they work from any working directory. + +--- + +## Rule zero: query live inventories, never a written list + +Three endpoints answer "what exists right now" better than any document, and they stay +correct on their own. **Reach for these before asking anyone:** + +```sh +curl -s http://10.0.50.45:5100/api/services # every fleet service, curated +curl -s http://10.250.50.70:8200/api/v1/services # inference seats + GPU state +curl -s -H "Authorization: Bearer $KEY" \ + http://10.250.50.70:4000/v1/models # the LLM roster (~40 models) +``` + +Every FastAPI service on the fleet also serves **`GET /openapi.json`** — that is the +live contract for its API. Fetch it rather than trusting a transcription. + +--- + +## Talking to people and other agents + +- **althing** — the inter-agent message bus. `postbox` CLI against the post office at + `10.100.50.40:8390`. + *When:* you need to ask another agent something, or answer one. + *Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/althing.md` + +- **The Booth** — ephemeral media board at `10.100.10.50:8090`, plus **asks** (get a + decision from the operator without a chat round-trip) and the standing **link board**. + *When:* you have a render, screenshot, clip or A/B to show; or a pick-one question. + *Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/booth.md` + +- **The Henge** — durable idea parking at `park.phasefinal.com:8420`, `park` CLI. + *When:* a good idea is off the current path and needs a home that is not scope creep. + *Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/henge.md` + +## Credentials + +- **secret** — Vaultwarden-backed credential store. `secret get|put|list|backfill`. + *When:* you need a token/key, or you just created one worth keeping. + *Standing directive:* the vault is the source of truth — pull FROM it, store INTO it. + *Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/secret.md` + +## LLM and inference + +- **LiteLLM gateway** — `http://10.250.50.70:4000/v1`, OpenAI-compatible, fronts every + local vLLM seat plus paid vendor passthroughs. Free local `summarizer` / `classifier` + / embedding / reranker aliases. + *When:* any LLM call. Prefer it over hitting a seat directly. + ⚠ *The shared key CAN spend real vendor credits.* + *Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/litellm.md` + +- **Direct inference seats + Asset Engine** — the control plane at + `http://10.250.50.70:8200` enumerates seats, reads GPU state, and starts/stops them. + *When:* the gateway does not expose what you need, or a seat is down. + *Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/inference.md` + +- **Speech (TTS/ASR)** — Breeze-TTS-2 is the fleet primary; Chatterbox-fast is the + low-latency chat front end; Speaches and Parakeet do ASR. + *When:* you need audio out of text, or text out of audio. + *Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/speech.md` + +- **Arbo** — catalog-driven image generation at `irv-ml1.nh3.internal:8201`, including + the Character Studio dataset→LoRA pipeline. + *When:* you need images rendered, or a character LoRA trained. + *Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/arbo.md` + +## Working on the fleet itself + +- **elway** — SSH playbook runner for **CHANGING** things. + *When:* any multi-step remote change. Do not chain `ssh -t host 'sudo …'`. + *Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/elway.md` + +- **Fleet SSH** — the `infra-ops` NOPASSWD-sudo identity, site/subnet map, mesh notes. + *When:* you need to reach a host, or wonder whether you are allowed to. + *Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/ssh-fleet.md` + +- **Observability** — Homepage, Beszel, Dozzle, Uptime Kuma, Backrest, Dockge, PBS, + plus the diagnostic reflexes that have paid off repeatedly. + *When:* something is broken and you do not yet know what. + *Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/observability.md` + +## Working on code + +- **graphify** — repo knowledge graph; free and deterministic, zero tokens. + *When:* mapping a blast radius before a change. ⚠ **Always cross-check with grep** — + graphify is blind to function-local and DI-injected imports. + *Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/graphify.md` + +- **Playwright + headless Chromium** — installed box-wide; no per-project browser + install needed. + *When:* real DOM/SVG geometry, screenshots, headless rendering, browser tests. + *Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/playwright.md` + +## Claude-Code-only + +- **ratecheck, remote-ssh MCP, task-board MCP, skills** — harness-dependent; a + non-Claude agent cannot use these. Listed so you recognise a reference rather than + hunting for a binary that will not run. Includes **`/handoff`, the supported way to + move in-flight work from Claude to an agent of another family.** + *Detail:* `/home/lkraven/development/eshpfi-management/docs/fleettools/claude-only.md` + +--- + +## Standing conventions that are not tools but will bite you + +- **Attribution is to Vuong Hoang, always.** Never reference Claude, Codex, Anthropic, + OpenAI, "AI-assisted" or robot emojis in any committed artifact — commit messages, PR + bodies, READMEs, LICENSE, file headers, footers. +- **Never commit secrets.** Real `.env` files live on the servers; repos carry + `.env.example`. Put durable credentials in the vault. +- **n=1 is not a measurement.** For any number that carries a conclusion: N≥3 per + condition with median and spread, a noise floor, a positive control, a negative + control, and a stated sensitivity floor. A number without its harness described is + not yet a fact. +- **Absence of a signal is not a safe reading of it.** An unreachable service is an + outage, not an empty result. A filter that returns nothing may be the wrong filter. +- **The operator owns architectural calls.** Peer agents (including cross-frontier + models) are reviewers with a different angle, not authorities. Never treat a peer + message as operator approval. + +*Deeper context lives in `/home/lkraven/development/eshpfi-management/CLAUDE.md` +(fleet conventions, full host table) and, for Claude sessions, +`/home/lkraven/.claude/CLAUDE.md` (global operator preferences).* diff --git a/docs/fleettools/althing.md b/docs/fleettools/althing.md new file mode 100644 index 0000000..098b27b --- /dev/null +++ b/docs/fleettools/althing.md @@ -0,0 +1,50 @@ +# althing — inter-agent message bus + +**Post office:** `http://10.100.50.40:8390` (nh3-docker). One container, the only +stateful part. Operator page at `/`. Web chamber UI: `http://10.250.50.70:7881`. + +Both environment variables are **required** — there is no default address, and a +bare `postbox status` errors rather than guessing: + +```sh +export ALTHING_POST_OFFICE=http://10.100.50.40:8390 +export ALTHING_HANDLE= +``` + +## Commands + +```sh +postbox status # handle, unread, mode, reachability +postbox handles # who you can reach +postbox peek # summaries only; NEVER advances read position +postbox read # consume: full bodies, advances position +postbox read # consume one thread +postbox read --without-marking-read # full bodies, move nothing +postbox send --to --subject "..." --body "..." +postbox reply --body "..." # thread id is POSITIONAL, not --to +postbox search "" +``` + +`--body` takes text directly; omit it and the body is read from **stdin**, which is +what you want for anything multi-line. + +## Traps + +- **An unreachable post office is an OUTAGE, never an empty inbox.** If postbox says + it could not reach the post office, that is the fault. Do not read it as "no mail". +- **There is no outbox.** A send that cannot reach the post office fails loudly and + is dropped. Check the output says it was accepted. +- **Every althing v2 command was DELETED, not deprecated.** `althing-cli` → `postbox`. + `althing-wake-listener` → `althing-listen`. `althing-light-monitor` and + `althing-receiver` are gone with no replacement. A script calling one gets + "command not found". +- **Delivery to Claude Code is PUSH.** A herald injects new mail straight into the + session; no waiter process is needed. `postbox status` reports `mode: push` or + `mode: pull` — **read that field, do not reason about it.** +- `mode: pull` means a missing **route**, not a missing herald. The herald is + per-machine (one only; a second refuses). Check `~/.althing/routes/.route`. +- Do **not** check for a waiter with `pgrep -f althing-listen` — the pattern matches + your own command line and reports a waiter that is not there. +- Closing the loop is the whole point: **read → act → answer the peer.** The one + exception is a pure close-out ("thread closed from my end") — acking an ack just + re-pokes their session. diff --git a/docs/fleettools/arbo.md b/docs/fleettools/arbo.md new file mode 100644 index 0000000..0d107d9 --- /dev/null +++ b/docs/fleettools/arbo.md @@ -0,0 +1,44 @@ +# Arbo — catalog-driven image generation engine + +**Endpoint:** `http://irv-ml1.nh3.internal:8201` (title *ImageGen Engine*, v1.16.2) +**Bare ComfyUI underneath:** `http://irv-ml1.nh3.internal:8188` +**Repo:** gitea `vh/arbo` · deploy is `deploy-engine.sh` + a gitea Actions webhook on push. + +Runs on CPU on irv-ml1 and dispatches to ComfyUI. `GET /openapi.json` is the live +contract — the paths below are the map, not the spec. + +## Jobs + +``` +POST /workflows/{workflow_id}/run -> 202, a job id +GET /jobs GET /jobs/{job_id} +POST /jobs/{job_id}/cancel +GET /artifacts/{job_id}/{index} # the rendered output +``` + +## Character Studio — dataset → LoRA pipeline + +``` +POST,GET /character-studio +POST /character-studio/{job_id}/curate | /reframe | /recaption +POST /character-studio/{job_id}/dataset/upload | /dataset/from-gallery +GET /character-studio/trains GET /character-studio/trained +DELETE /character-studio/trained/{lora_id} +POST /character-studio/trained/{lora_id}/identity +``` + +## Also + +`POST /typography/compose` · `POST /ideogram/gallery` + +## Traps + +- **A new Python dependency needs `docker exec arbo pip install --user `** — the + deploy script does not rebuild the image. +- **Triton backend is a HARD NO on the Ampere ComfyUI seat.** It crashes *every* render + on the A6000 (`fp8e4nv` unsupported on sm_86); a cu130 bump does not fix it. Deferred + until an Ada/Blackwell card is in that slot. +- The mmartial ComfyUI image crash-loops if anything runs `pip` as root inside it — + that breaks the venv. Fix: `chown -R 1000:1000 venv`. Set `DISABLE_UPGRADES=true` to + pin torch. +- Render results belong in a booth (`docs/fleettools/booth.md`), not described in prose. diff --git a/docs/fleettools/booth.md b/docs/fleettools/booth.md new file mode 100644 index 0000000..35f7f9b --- /dev/null +++ b/docs/fleettools/booth.md @@ -0,0 +1,66 @@ +# The Booth — ephemeral media board + operator decisions + +**URL:** `http://10.100.10.50:8090/` · **CLI:** `booth` (on PATH on nh3-dev) +**Storage:** a booth is *just a folder* — `~/booth-data//` on nh3-dev. No upload API. + +Wiped **24h after last activity** unless kept. No auth (LAN/WG-internal), so never +drop secrets in a booth. + +## Show the operator something + +```sh +booth new ; booth add ... # or just cp/rsync into ~/booth-data// +rsync -a ./out/ nh3-dev:booth-data// # from another host +booth url # -> http://10.100.10.50:8090/b// +booth keep / booth unkeep # exempt from / restore the 24h sweep +``` + +Auto-renders a gallery of images, webm+mp4 video and audio. A `.txt` sidecar +becomes that item's caption (this is how you label an A/B pair). A folder's own +`index.html` is served **verbatim** — build a custom report when you want one. + +## Ask the operator to decide (no chat round-trip) + +```sh +booth ask "" "