claude/tools: granite-4.1-8b always-available summarizer/classifier + shared gateway key

LiteLLM Global-tools entry: frame granite-4.1-8b as the standing
always-available local summarizer/classifier endpoint (FP8, ana-ml2 GPU1,
131k ctx); add qwen3.5-9b-fp8 (vision) to the gateway listing; document the
shared all-agents virtual key (alias all-agents-local) scoped to the free
local models only (granite/vision/embed/rerank; paid GLM walled off).
Internal-only gateway, rotatable.
This commit is contained in:
Your Name
2026-06-13 12:13:23 -07:00
parent afb8552d37
commit 1045fdf9ad
+40
View File
@@ -236,6 +236,46 @@ note says otherwise. On other boxes, check first.)
browser-based tests or scraping. Add a browser / bump via
`ssh infra-ops@10.100.10.50 'sudo env PLAYWRIGHT_BROWSERS_PATH=/opt/ms-playwright npx -y playwright install <browser>'`.
- **Graphify** — open-source knowledge-graph skill for AI coding assistants
(`uv tool install graphifyy`; CLI `graphify`, MCP `graphify-mcp`). Turns a
repo into a queryable graph. The **free, deterministic path** is
`graphify update <path>`: tree-sitter AST extraction + Leiden clustering,
**zero LLM / zero tokens**, ~25s for ~900 files; writes
`graphify-out/{graph.json,GRAPH_REPORT.md}`. Highest-value free output is
the **God Nodes** list (core-abstraction surfacing) plus `graphify explain
<symbol>` / `query` / `affected` / `path` traversals over `EXTRACTED` call
edges. The LLM layer (community **labeling** = low-caliber, and **semantic
extraction** of docs/INFERRED edges = higher-caliber + noisier) is optional
and points at any backend via `~/.graphify/providers.json` — pair labeling
with the cheap LiteLLM/Granite endpoint below; leave deep semantic
extraction off unless needed (its INFERRED/AMBIGUOUS edges fight the
explicit-over-implicit floor). Complements a hand-authored `docs/CODEBASE.md`
(curated geography), doesn't replace it. Pilot-validated on Worldtree
2026-06-10.
- **LiteLLM gateway (Granite 4.1 + friends)** — OpenAI-compatible gateway at
`http://10.250.50.70:4000/v1` (Logs UI `:4000/ui`) fronting vLLM services.
Essentially-**free local compute** for low-caliber, high-volume,
parallelizable LLM work. The **always-available summarizer / classifier
endpoint** is `granite-4.1-8b` — the go-to for summarization, naming/
labeling, classification, and triage (fast, parallelizes well on vLLM):
served FP8 on ana-ml2 GPU 1, production-stable, 131k context (rebalanced
2026-06-13). Treat it as a standing dependency you can reach for any time.
Also behind the gateway: `qwen3.5-9b-fp8` (vision / multimodal — image +
text chat, GPU 1), `qwen3-embedding`, `qwen3-reranker`, and `glm-5.1` /
`glm-4.7` (via z.ai passthrough). The gateway 401s without a **virtual API
key**. **Shared all-agents key** — internal-only gateway, scoped to the *free
local* models only (granite + qwen-vision + embed/rerank, NOT the paid GLM):
`sk-eA_XOdcs6nIkyYXonohtEQ` (alias `all-agents-local`). Use it directly for
the always-available local endpoints — no per-project provisioning needed.
Example: `curl http://10.250.50.70:4000/v1/chat/completions -H "Authorization:
Bearer sk-eA_XOdcs6nIkyYXonohtEQ" -d '{"model":"granite-4.1-8b","messages":[…]}'`.
For the **paid** GLM passthroughs or broader scope, still request a
project-scoped key from infra-ops via althing (the shared key deliberately
can't spend z.ai cost). Rotatable via infra-ops if it leaks. Reach for
this before spending premium API tokens on tasks a small fast local model
handles fine (the LLM-last-resort floor still applies — deterministic first).
## DevOps delegation
For all DevOps work — deployments, infrastructure provisioning,