From 25a7d05f51c1005ab50472277d614a9f1d02d83a Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Mon, 14 Sep 2026 12:37:47 -0700 Subject: [PATCH] feat(nh3-dev): repoint Hermes at gen-large on the LiteLLM gateway MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Operator ruled the repoint; Miranda moves off the paid z.ai Coding Plan onto free local compute. model.default gen-large, provider custom, base_url http://10.250.50.70:4000/v1. Verified by a real turn rather than by config: hermes status reports gen-large / Custom endpoint and a completion through /v1/chat/completions returns 660 tokens. The openrouter/nous credit warnings cleared with it. Records the landmine found on the way: CUSTOM_API_KEY and HERMES_CUSTOM_API_KEY are inert for bare provider: custom — they bind only a named custom_providers entry through its key_env. Without model.api_key the request ships the placeholder no-key-required and LiteLLM 401s inside the response body while hermes status still reports a healthy gen-large / Custom endpoint, so status alone cannot verify this change. Also notes that nh3-dev/hermes/api-server-key must not be rotated until SVOS splits its HS256 signing key off the shared value. --- servers/nh3-dev/README.md | 35 ++++++++++++++++++++++++++--------- 1 file changed, 26 insertions(+), 9 deletions(-) diff --git a/servers/nh3-dev/README.md b/servers/nh3-dev/README.md index 18d4b76..d2a059c 100644 --- a/servers/nh3-dev/README.md +++ b/servers/nh3-dev/README.md @@ -36,15 +36,32 @@ local Bash already executes here — no SSH-to-self needed for non-privileged wo is what SVOS's `_hermes_roster` derives its required-config line from — narrowing does not blind it. Becomes `[svos_miranda]` once SVOS's plugin lands in `$HERMES_HOME/plugins/`. -- ⚠ **Hermes model resolution is NOT the free local gateway.** `model.default` - says `anthropic/claude-opus-4.6` and `model.base_url` says openrouter, but - `provider: auto` plus a lone `zai` credential in `auth.json` resolves Miranda - to **GLM-5.3 over the paid z.ai Coding Plan** - (`https://api.z.ai/api/coding/paas/v4`). Nothing under `~/.hermes` references - the LiteLLM gateway at all. Raised 2026-09-14 by svos-dev, who reports the - operator believes Miranda runs on `gen-large` (free local). `gen-large` does - exist on LiteLLM. **Unresolved — do not repoint without the operator**, the - premise is his to confirm. +- **Hermes model backend → `gen-large` on the fleet LiteLLM gateway** (free local + compute), set 2026-09-14 per operator ruling. Until then `model.default` said + `anthropic/claude-opus-4.6` with `model.base_url` at openrouter, but + `provider: auto` plus a lone `zai` credential in `auth.json` silently resolved + Miranda to **GLM-5.3 on the paid z.ai Coding Plan** — three settings that had + to be read together before the real answer fell out. Now + `default: gen-large` / `provider: custom` / `base_url: + http://10.250.50.70:4000/v1`, verified by a real turn (`hermes status` → + `gen-large` / `Custom endpoint`, plus a 660-token completion through + `/v1/chat/completions`). The openrouter/nous credit warnings cleared with it. + ⚠ **`CUSTOM_API_KEY` / `HERMES_CUSTOM_API_KEY` are INERT for bare + `provider: "custom"`** — they only bind a *named* `custom_providers:` entry via + its `key_env`. Set `model.api_key` in `config.yaml` instead; its only env + fallback is the legacy name `OPENROUTER_API_KEY`. Get this wrong and the + request ships the placeholder `no-key-required`, LiteLLM 401s **inside the + response body**, and `hermes status` still reports a perfectly healthy + `gen-large` / `Custom endpoint` — so status alone cannot verify this change, + only a real completion can. The `zai` credential is still in `auth.json`, + present and unused; `provider: custom` is explicit so it is not a candidate, + and `hermes fallback list` is empty, so there is no degraded-mode route that + quietly re-bills z.ai. Miranda fails rather than fails over if LiteLLM is down. +- ⚠ **Do not rotate `nh3-dev/hermes/api-server-key` yet.** SVOS currently reuses + that value as the HS256 signing key on its Bifrost wall, so a rotation would + 401 every Miranda tool call. svos-dev is splitting theirs off (operator- + approved 2026-09-14) and will confirm when it lands; rotation is safe after + that, not before. - **bloom_music dev** — `~/development/bloom_music`; its `web/` test harness uses Playwright headless Chromium for OSMD browser-geometry assertions. - **The Booth** — ephemeral media drop board (`:8090`, `booth.service`), from