fix(intern-decision): roll the decision seat back to Intern-Decision-4B; h2o staged with inline-images-only
albok-dev withdrew the judge sign-off on fleet-scale evidence and brokkr concurred: over 10,841 texts judged by both models, h2o flagged 242 adult at 0.5 against INC's 88, and no threshold reproduces INC's set. rollback-to-inc.sh ran 14:03:28, INC healthy 14:03:56, /decide/shared smoke 200 x3. compose.yaml is INC again; the h2o compose is kept as compose.h2o.yaml with switch-to-h2o.sh (GPU 1 free-memory gate, falls back to INC). It now carries Prime's inline-images-only setting: LIGHTNING_IMAGE_URLS=off on the front (verified live: an https image URL is refused before any fetch, inline images still answer) and vLLM --allowed-media-domains media.invalid as a backstop. rollback-to-inc.sh no longer leaves timestamped compose copies behind.
This commit is contained in:
@@ -1,11 +1,21 @@
|
||||
# intern-decision — the decision seat (fv-ml1 GPU 1, :8033)
|
||||
|
||||
> **Since 2026-10-12 1311 PT the seat serves h2o-lightning-4b**, not Intern-Decision-4B. Prime's order came via
|
||||
> brokkr-smithy-dev, gated on albok-dev re-validating Nemi's sensitivity judge (signed off at threshold 0.5 through
|
||||
> `/decide/shared`). INC went down at 130818 and h2o was healthy at 131127. The address, port, token and stack name
|
||||
> are unchanged on purpose (brokkr rule 8), so callers change nothing.
|
||||
> Everything from **"History: the Intern-Decision-4B seat"** down describes the previous seat. It is kept because
|
||||
> the rollback goes back to it.
|
||||
> **LIVE: Intern-Decision-4B again, since 2026-10-12 1403 PT** (`compose.yaml`, image `intern-decision-serve:0.1.3`).
|
||||
> Everything from **"The live seat: Intern-Decision-4B"** down describes it.
|
||||
>
|
||||
> **h2o-lightning-4b is STAGED, not live** (`compose.h2o.yaml`, `switch-to-h2o.sh`). It served 1311–1403 on 2026-10-12
|
||||
> and was rolled back on albok-dev's word, with brokkr concurring. Prime's plan had made the swap conditional on the
|
||||
> judge's sign-off, and albok-dev withdrew it on fleet-scale evidence:
|
||||
> - **adult:** over 10,841 texts judged by both models (same text, prompt and policy), h2o flagged 242 at 0.5 against
|
||||
> INC's 88. That is all 88 of INC's plus 154 more, most of them at 0.80–0.90 where INC put them at 0.30–0.47. No
|
||||
> threshold reproduces INC's set (0.70 still flags 166 and loses 6 of INC's).
|
||||
> - **personal:** INC flagged 5 and h2o 11, with only 2 in common.
|
||||
>
|
||||
> h2o comes back only after albok re-fits the judge for it (prompt and threshold against a labelled set large
|
||||
> enough to resolve a 2× change, as a 4.4 contract revision, on an eval seat). The h2o sections below describe the
|
||||
> staged seat, and its numbers are the 1311 cutover's.
|
||||
|
||||
### The staged h2o seat (`compose.h2o.yaml`)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
@@ -33,7 +43,7 @@
|
||||
returns 503 when the shim is not healthy.
|
||||
- `/v1/systemone` and `/api/alpha/decisions` pass through after admission, and `/v1/models` passes through as is.
|
||||
|
||||
### Verification on the live seat (2026-10-12, 1311–1313 PT, on fv-ml1 loopback)
|
||||
### Verification at the 1311 cutover (2026-10-12, 1311–1318 PT, on fv-ml1 loopback)
|
||||
|
||||
| check | result | expected |
|
||||
|---|---|---|
|
||||
@@ -67,19 +77,42 @@ GPU 1 holds meromero-rp (~44.0 GB) and gen-small (~35.0 GB). Read the free memor
|
||||
`scripts/oneoff/fv-ml1-gpucheck.sh 1 0.1506 intern-decision-vllm`, and stop if it fails. The vLLM container takes about
|
||||
3 minutes to reach healthy (graph capture). The front waits on it, and the healthcheck has a 300 s start period.
|
||||
|
||||
### Rollback to INC
|
||||
### Images: inline only (Prime 2026-10-12, via brokkr)
|
||||
|
||||
On fv-ml1: `sudo bash /opt/docker/compose/intern-decision/rollback-to-inc.sh`. It runs `docker compose down`, keeps
|
||||
the h2o compose as `compose.yaml.h2o-<timestamp>`, copies `compose.inc.yaml` over `compose.yaml`, and starts INC
|
||||
(image `intern-decision-serve:0.1.3`, still on the host; its `intern-decision_triton-cache` volume survives the
|
||||
`down`). It waits up to 180 s for `/health`. INC's peak is 15,220 MiB, below what h2o frees. To return to h2o: put
|
||||
the saved h2o compose back as `compose.yaml` and `up -d`.
|
||||
The h2o seat must never fetch an image URL. `compose.h2o.yaml` sets `LIGHTNING_IMAGE_URLS=off` on the front, and the
|
||||
vendor shim then refuses every image URL with 422 "image URLs are not accepted here". Inline `data:` URIs and base64
|
||||
keep working. vLLM also runs with `--allowed-media-domains media.invalid` as a backstop. That is a reserved name that
|
||||
never resolves, and vLLM 0.30.0 checks every http(s) media URL against it before fetching, while `data:` URLs bypass
|
||||
the check.
|
||||
|
||||
The shim switch was verified live at 1402, on the h2o seat with the front recreated alone:
|
||||
- the same `https://media.invalid/red.png` request went from "could not be fetched" (a fetch was attempted) to "not
|
||||
accepted here" (refused before any fetch);
|
||||
- inline red still answered red at 0.984;
|
||||
- before the change, inline red/blue/green answered at 0.98+ and no-image came out near-uniform.
|
||||
|
||||
The vLLM backstop has **not** been exercised: the seat was rolled back before vLLM was recreated. Verify it on the next
|
||||
switch, by posting an https image URL straight to vLLM's loopback port and expecting a refusal that names the allowed
|
||||
domains. INC needs neither setting, because its served checkpoint has no vision tower.
|
||||
|
||||
### Switching between the two
|
||||
|
||||
Both run ON fv-ml1 and share one `.env`.
|
||||
- **To h2o:** `sudo bash /opt/docker/compose/intern-decision/switch-to-h2o.sh`. It stops INC and checks that GPU 1 has
|
||||
16,900 MiB free (h2o's measured budget). If not, it restarts INC and exits 2. Otherwise it copies
|
||||
`compose.h2o.yaml` over `compose.yaml` and waits up to 300 s for `/health`. Only do this once albok has re-fitted
|
||||
the judge.
|
||||
- **To INC:** `sudo bash /opt/docker/compose/intern-decision/rollback-to-inc.sh`. It stops h2o, copies `compose.inc.yaml`
|
||||
over `compose.yaml` and waits up to 180 s. Image `intern-decision-serve:0.1.3` and its
|
||||
`intern-decision_triton-cache` volume survive the `down`. Used 2026-10-12, 140328 → healthy 140356.
|
||||
|
||||
After either switch, make `stacks/intern-decision/compose.yaml` match the host again (it is INC today).
|
||||
|
||||
---
|
||||
|
||||
# History: the Intern-Decision-4B seat (2026-09-30 → 2026-10-12)
|
||||
# The live seat: Intern-Decision-4B (2026-09-30 → 2026-10-12 1308, and again since 1403)
|
||||
|
||||
> **Was LIVE from 2026-09-30 0941 PT to 2026-10-12 1308 PT** on fv-ml1 GPU 1, with cap 9.0 GiB and `MAX_TOKENS` 7,168.
|
||||
> **LIVE from 2026-09-30 0941 PT to 2026-10-12 1308 PT, and again since 2026-10-12 1403 PT** on fv-ml1 GPU 1, with cap 9.0 GiB and `MAX_TOKENS` 7,168.
|
||||
> Acceptance on the live service is below. `semif` is stopped and kept as the rollback.
|
||||
|
||||
**Intern-Decision-4B typed decisions** on **fv-ml1 GPU 1**, the utility card beside `vllm-coder`,
|
||||
|
||||
Reference in New Issue
Block a user