claude/tools: granite-4.1-8b always-available summarizer/classifier + shared gateway key
LiteLLM Global-tools entry: frame granite-4.1-8b as the standing always-available local summarizer/classifier endpoint (FP8, ana-ml2 GPU1, 131k ctx); add qwen3.5-9b-fp8 (vision) to the gateway listing; document the shared all-agents virtual key (alias all-agents-local) scoped to the free local models only (granite/vision/embed/rerank; paid GLM walled off). Internal-only gateway, rotatable.
This commit is contained in:
@@ -236,6 +236,46 @@ note says otherwise. On other boxes, check first.)
|
||||
browser-based tests or scraping. Add a browser / bump via
|
||||
`ssh infra-ops@10.100.10.50 'sudo env PLAYWRIGHT_BROWSERS_PATH=/opt/ms-playwright npx -y playwright install <browser>'`.
|
||||
|
||||
- **Graphify** — open-source knowledge-graph skill for AI coding assistants
|
||||
(`uv tool install graphifyy`; CLI `graphify`, MCP `graphify-mcp`). Turns a
|
||||
repo into a queryable graph. The **free, deterministic path** is
|
||||
`graphify update <path>`: tree-sitter AST extraction + Leiden clustering,
|
||||
**zero LLM / zero tokens**, ~25s for ~900 files; writes
|
||||
`graphify-out/{graph.json,GRAPH_REPORT.md}`. Highest-value free output is
|
||||
the **God Nodes** list (core-abstraction surfacing) plus `graphify explain
|
||||
<symbol>` / `query` / `affected` / `path` traversals over `EXTRACTED` call
|
||||
edges. The LLM layer (community **labeling** = low-caliber, and **semantic
|
||||
extraction** of docs/INFERRED edges = higher-caliber + noisier) is optional
|
||||
and points at any backend via `~/.graphify/providers.json` — pair labeling
|
||||
with the cheap LiteLLM/Granite endpoint below; leave deep semantic
|
||||
extraction off unless needed (its INFERRED/AMBIGUOUS edges fight the
|
||||
explicit-over-implicit floor). Complements a hand-authored `docs/CODEBASE.md`
|
||||
(curated geography), doesn't replace it. Pilot-validated on Worldtree
|
||||
2026-06-10.
|
||||
|
||||
- **LiteLLM gateway (Granite 4.1 + friends)** — OpenAI-compatible gateway at
|
||||
`http://10.250.50.70:4000/v1` (Logs UI `:4000/ui`) fronting vLLM services.
|
||||
Essentially-**free local compute** for low-caliber, high-volume,
|
||||
parallelizable LLM work. The **always-available summarizer / classifier
|
||||
endpoint** is `granite-4.1-8b` — the go-to for summarization, naming/
|
||||
labeling, classification, and triage (fast, parallelizes well on vLLM):
|
||||
served FP8 on ana-ml2 GPU 1, production-stable, 131k context (rebalanced
|
||||
2026-06-13). Treat it as a standing dependency you can reach for any time.
|
||||
Also behind the gateway: `qwen3.5-9b-fp8` (vision / multimodal — image +
|
||||
text chat, GPU 1), `qwen3-embedding`, `qwen3-reranker`, and `glm-5.1` /
|
||||
`glm-4.7` (via z.ai passthrough). The gateway 401s without a **virtual API
|
||||
key**. **Shared all-agents key** — internal-only gateway, scoped to the *free
|
||||
local* models only (granite + qwen-vision + embed/rerank, NOT the paid GLM):
|
||||
`sk-eA_XOdcs6nIkyYXonohtEQ` (alias `all-agents-local`). Use it directly for
|
||||
the always-available local endpoints — no per-project provisioning needed.
|
||||
Example: `curl http://10.250.50.70:4000/v1/chat/completions -H "Authorization:
|
||||
Bearer sk-eA_XOdcs6nIkyYXonohtEQ" -d '{"model":"granite-4.1-8b","messages":[…]}'`.
|
||||
For the **paid** GLM passthroughs or broader scope, still request a
|
||||
project-scoped key from infra-ops via althing (the shared key deliberately
|
||||
can't spend z.ai cost). Rotatable via infra-ops if it leaks. Reach for
|
||||
this before spending premium API tokens on tasks a small fast local model
|
||||
handles fine (the LLM-last-resort floor still applies — deterministic first).
|
||||
|
||||
## DevOps delegation
|
||||
|
||||
For all DevOps work — deployments, infrastructure provisioning,
|
||||
|
||||
Reference in New Issue
Block a user