Files

T

vh 91bda3c480 fv-ml1: complete the cutover — rename, renumber, DNS, and the LiteLLM repoint

The box is physically at Fountain Valley, renamed, renumbered onto 10.251/16,
and serving inference again. This lands the repo half of that.

Host: hostname ana-ml2 -> fv-ml1, pinned to 10.251.50.54 by a dnsmasq
reservation so the address the runbook, DNS and LiteLLM all assume is the
address it actually has. Its headscale node is renamed too.

The sweep ran from scripts/fv-ml1-rename-sweep.sh, whose allowlist is the
reason this diff touches current-state files and not the record. Dated
persistent-memory entries, archival-memory and incident notes still say
ana-ml2 in 31 and 62 places respectively, because that is what the box was
when those things happened. Rewriting them would make the history lie.

LiteLLM was the load-bearing piece and needed more than the api_base sed the
runbook describes. Twenty api_base entries repointed, but a grep-and-verify
pass also caught a LIVE pass_through_endpoints target for the scalar-judge
reward route still on the old address -- an api_base-only substitution would
have left it dead. Four prose references describing current state were
repointed as well; one historical note recording where a hand-test was run
is deliberately left pointing at 10.250.50.54.

Two facts in the server tables were wrong and are corrected here. The site is
Fountain Valley, not Anaheim. And the box has FOUR RTX PRO 6000 Blackwell
Max-Q, not two -- verified by nvidia-smi -L and independently by PCI
enumeration of four GB202GL devices. That is 391 GB of VRAM rather than 196,
which changes what fits on it.

DNS: fv-ml1, fv-ml1-bmc and fv-gw added under the fv site via the piggyback
approach, scriberr re-homed, and the ana-ml2 records removed. Applied to all
three resolvers. The BMC record carries a warning that its 802.1q VLAN tag
must stay disabled -- it shipped tagging VLAN 250 into an untagged port,
which made it invisible to every network-side diagnostic and is the reason
it appeared dead through several cable changes.

Verified end to end: summarizer and sec both answer through the Anaheim
gateway across the mesh to FV seats on different ports.

2026-09-12 22:00:50 -07:00

.env.example

Swap the MeroMero A4B onto the erp-seat seat as char-rp-fast, retire the Pfish-6 alias

2026-09-10 11:34:05 -07:00

compose.yaml

fv-ml1: complete the cutover — rename, renumber, DNS, and the LiteLLM repoint

2026-09-12 22:00:50 -07:00

README.md

feat(erp-seat): run 7 quantized to NVFP4A16 and serving as the trial seat on ana-ml2

2026-09-09 15:30:15 -07:00

README.md

erp-seat — ERP-tune seat on ana-ml2 (GPU1, `:8021`)

Serves the latest gated ERP LoRA merge as an NVFP4A16 (weight-only) compressed-tensors checkpoint so the GX10 is free to train the next run.

Current occupant: run 7 — erp-tune-v7-nvfp4a16 = merged-run07 (jenerallee78 ARA-abliterated Gemma-4-26B-A4B-it, index 33c59654…, + R47 SFT r7 = r6 plus the opening-split slot and its companion loss mask), quantized by services/erp-seat-quant/ on 2026-09-09. 49 GiB bf16 → 16 GiB NVFP4A16. Runbook docs/runbooks/gx10-run-07.md.

Previous: run 6 (erp-tune-v6-nvfp4a16, 2026-09-08). Its artifact is still on /tank/aimodels/ and the pre-swap host env is at /tmp/erp-seat-env.v6.bak on ana-ml2, so a rollback is an .env flip plus docker compose up -d.

True name only. --served-model-name erp-tune-v7-nvfp4a16. Gateway aliases (trial) are set in LiteLLM on the operator's word, never here (no silent substitution — the bf16 arm on the GX10 and this NVFP4 arm are different artifacts).
Recipe = stacks/gemma4-charrp (same arch + format, proven on this box): gemma4 tool and reasoning parsers, enable_thinking pinned false, the model's own stock template (ae53464b…, the one it trained through). Without the reasoning parser the post-tool turn leaks <|channel> markers; without the kwargs pin all prose lands in reasoning_content.
tool_choice: "none" trap (measured 2026-09-08, fixed with --exclude-tools-when-tool-choice-none). Without the flag vLLM still renders the tools into the prompt, the model emits a tool call anyway, and because parsing is off for none the reply is content: null, tool_calls: null — an empty turn, 3/3 reproductions. With the flag the tools are dropped from the prompt and the model answers in prose (3/3). The rest of the matrix (auto / required / named / parallel / nested schema / empty tools: [] / streaming / tool-result round trip) was green before and after. stacks/gemma4-charrp has the same exposure and does NOT carry the flag yet.
Forced tool_choice (named / required) is prompt-driven on EVERY Gemma-4 seat, not grammar-enforced — by vLLM design. vllm/tool_parsers/gemma4_engine_tool_parser.py sets supports_required_and_named = False and its adjust_request deliberately skips the structured-output JSON for required/named so the model can emit its native <|tool_call>call:… syntax. A tune that weakened that syntax (this ERP tune) therefore honours forced calls only sometimes. Measured 2026-09-08, 3 conversations × 3 turns, real system prompt: v0.26.0 1/9; nightly 311b3513 (v0.27.2rc1, the gen seat's image) 6/9 and the tool-result round trip stays clean 3/3 — so the seat runs the nightly. tts-dev measured 0/18 on v0.26.0 with gen 18/18 as the positive control, and response_format: json_schema (guided decoding) 18/18 on this seat — that is the deterministic path for a forced call; tool_choice: auto works normally. A parser plugin that re-enables guided JSON would also need JSON extraction in the engine-parser path; not attempted.
GPU1 is shared — check real usage (nvidia-smi --query-compute-apps=pid,used_memory) before raising ERP_GPU_MEM_UTIL; the flag sizes KV, not CUDA context.
Rollback / next run: point ERP_MODEL + ERP_SERVED_NAME at the next quant dir, keep the previous on disk. Deploy with scripts/deploy-stack.sh ana-ml2 erp-seat.

README.md Unescape Escape

erp-seat — ERP-tune seat on ana-ml2 (GPU1, :8021)

README.md

erp-seat — ERP-tune seat on ana-ml2 (GPU1, `:8021`)