vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(intern-decision): h2o-lightning-4b is the production decision seat; fix forward, no INC fallback

Browse Source 
Prime (18:05, via albok-dev): go to h2o and fix forward, do not fall back to
INC. albok-dev removed the chapter margin first (the two published nonfiction
titles declared visibility: public). switch-to-h2o.sh ran 18:05:13, h2o healthy
18:07:40; /health at the pin, /decide/shared 200, 422 over the cap, inline image
answers, image URL refused by the shim and by vLLM's media-domain backstop.
rollback-to-inc.sh stays on the host as an artifact only.
This commit is contained in:
vh
2026-10-12 18:08:31 -07:00
parent c991d939f9
commit 5af8e2bc17
3 changed files with 80 additions and 78 deletions
Download Patch File Download Diff File Expand all files Collapse all files
stacks/intern-decision/README.md
+20 -20
View File
@@ -1,25 +1,25 @@
# intern-decision — the decision seat (fv-ml1 GPU 1, :8033)
> **LIVE: Intern-Decision-4B, since 2026-10-12 1748 PT** (`compose.yaml` = `compose.inc.yaml`). Nemi's judge
> runs at concurrency 1 against it (INC is single-threaded, INV-2).
> **h2o-lightning-4b is STAGED** (`compose.h2o.yaml`, `switch-to-h2o.sh`). It has been rolled back twice, each time on
> albok-dev's word against a rollback line named before the switch.
> **LIVE: h2o-lightning-4b, since 2026-10-12 1807 PT: the production seat.** Prime, 1805, via albok-dev: "okay, go
> to h2o, and fix forward. do not fall back to inc." `compose.yaml` = `compose.h2o.yaml`. Nemi's judge runs the v2
> note question at concurrency 4.
>
> ⚠ **No fallback to INC.** `rollback-to-inc.sh` and `compose.inc.yaml` stay on the host as an artifact, but problems
> on this seat are fixed forward (judge wording, thresholds, `visibility` declarations), not by switching back. Using
> INC again would need Prime's explicit word.
>
> **What happened on 2026-10-12:**
> 1. **h2o 1311–1403, judge v1, rolled back.** Adult flags went from INC's 88 to 242 over 10,841 identical texts.
> 2. **The GPU 3 investigation, 1428–1619.** The v1 question asked two things at once, and the floor was ruled out.
> Prime ruled "text itself", and albok-nemi 0.1.18 shipped the v2 question.
> 3. **h2o 1704–1747, judge v2, rolled back.** Adult on notes held at 0 as validated, but two margins moved past
> the named line:
> - library/nonfiction adult 0 → 79: the Schell game-design chapter at 0.531 on h2o's book profile (INC 0.438),
> with its 77 concepts inheriting;
> - fleet/memory personal 6 → 18 on an unchanged personal question.
>
> albok-dev calls both fixable offline: a per-profile threshold or a `visibility: public` declaration on published
> titles, and a measured look at how h2o reads the personal question. They bring those to Prime before any third
> attempt. "The model is not the problem, the two margins are."
> 1. **h2o 1311–1403 (judge v1), rolled back.** Adult flags went from 88 to 242. The v1 question asked two things at
> once.
> 2. **The GPU 3 investigation.** The floor was ruled out, and the v2 question was shipped (albok-nemi 0.1.18).
> 3. **h2o 1704–1747 (v2), rolled back.** Two margins moved:
> - nonfiction adult 0 → 79, from one textbook chapter at 0.531;
> - memory personal 6 → 18.
> 4. **h2o for good from 1807.** albok-dev fixed the chapter margin first, by declaring the two published nonfiction
> titles `visibility: public`, so the judge is no longer consulted for them. The personal-question reading is
> being re-fitted forward on the live seat.
### The staged h2o seat (`compose.h2o.yaml`; live 1311–1403 and 1704–1747 on 2026-10-12)
### The h2o seat (`compose.h2o.yaml`; production since 2026-10-12 1807)
| | |
|---|---|
@@ -101,7 +101,7 @@ Repeat that check on this seat's loopback port (`:18299`) on the next switch. IN
### Switching between the two
The 1704 switch: `switch-to-h2o.sh` ran from 170223 (rc 0) and h2o was healthy at 170454. The 1705 checks: `/health` at the
The final switch: `switch-to-h2o.sh` ran from 180513, h2o healthy at 180740, with the same checks all passing. The 1704 switch: `switch-to-h2o.sh` ran from 170223 (rc 0) and h2o was healthy at 170454. The 1705 checks: `/health` at the
pin, with floor 0.801; `/decide/shared` 200; a 422 over the cap; inline red → red at 0.984; an https image URL refused
by the shim; the vLLM backstop refusing example.com (400); GPU 1 resting at 14,430 MiB.
@@ -115,7 +115,7 @@ Both run ON fv-ml1 and share one `.env`.
`intern-decision_triton-cache` volume survive the `down`. Used 2026-10-12, 140328 → healthy 140356, and again
174746 → healthy 174811.
After either switch, make `stacks/intern-decision/compose.yaml` match the host again (it is INC since 1748).
After either switch, make `stacks/intern-decision/compose.yaml` match the host again (it is h2o since 1807).
⚠ **Tell albok-dev on either switch: Nemi's judge concurrency depends on the seat.** INC scores on one inference
thread (INV-2, ~16 calls/s at most), so the judge runs with concurrency 1 against it (albok's `nemi.yaml`, since
@@ -124,7 +124,7 @@ h2o, vLLM batches, so it goes back to 4.
---
# The live seat: Intern-Decision-4B (2026-09-30 → 10-12 1308, 1403–1702, and again since 1748)
# History: the Intern-Decision-4B seat (2026-09-30 → 10-12 1308, 1403–1702, 1748–1805; retired, no fallback per Prime)
> **LIVE from 2026-09-30 0941 PT to 2026-10-12 1308 PT, and again since 2026-10-12 1403 PT** on fv-ml1 GPU 1, with cap 9.0 GiB and `MAX_TOKENS` 7,168.
> Acceptance on the live service is below. `semif` is stopped and kept as the rollback.