fix(intern-decision): second rollback to Intern-Decision-4B; h2o staged until the judge's two margins are fixed

The v2 walk on h2o (17:05-17:45) held notes at 0 adult but moved two margins past albok-dev's pre-named rollback line: library/nonfiction adult 0 -> 79 (one textbook chapter at 0.531 on the book profile, INC 0.438, with 77 concepts inheriting) and fleet/memory personal 6 -> 18. rollback-to-inc.sh ran 17:47:46, INC healthy 17:48:11. compose.yaml is INC again.
2026-10-12 17:48:43 -07:00
parent 2d85ee5a25
commit d2fb840bfa
3 changed files with 78 additions and 76 deletions
@@ -1,22 +1,25 @@
 # intern-decision — the decision seat (fv-ml1 GPU 1, :8033)

-> **LIVE: h2o-lightning-4b since 2026-10-12 1704 PT** (Prime 1640, via albok-dev). `compose.yaml` = `compose.h2o.yaml`
-> (`switch-to-h2o.sh` copies one over the other). Nemi's judge runs the **re-fitted v2 note question**
-> (albok-nemi 0.1.18, contract 4.4 rev 1.15) at concurrency 4.
+> **LIVE: Intern-Decision-4B, since 2026-10-12 1748 PT** (`compose.yaml` = `compose.inc.yaml`). Nemi's judge
+> runs at concurrency 1 against it (INC is single-threaded, INV-2).
+> **h2o-lightning-4b is STAGED** (`compose.h2o.yaml`, `switch-to-h2o.sh`). It has been rolled back twice, each time on
+> albok-dev's word against a rollback line named before the switch.
 >
 > **What happened on 2026-10-12:**
-> 1. **1311–1403, h2o's first stint, rolled back.** Under the judge's v1 question, h2o flagged 242 texts adult
->    against INC's 88, over 10,841 identical texts.
-> 2. **1428–1619, the investigation on GPU 3.** The v1 question asked two things at once: INC answered it
->    inconsistently and h2o answered it as written. The shim's floor was ruled out, with identical verdicts on and
->    off. Prime ruled "text itself".
-> 3. **Under v2 the two seats agree:** work notes INC 0/22, h2o 1/22 at 0.52; nulls 0/18 on both. fleet/memory adult
->    went 86 → 0 on INC.
-> 4. **Rollback:** `rollback-to-inc.sh`, and set the judge's concurrency back to 1 (see "Switching").
+> 1. **h2o 1311–1403, judge v1, rolled back.** Adult flags went from INC's 88 to 242 over 10,841 identical texts.
+> 2. **The GPU 3 investigation, 1428–1619.** The v1 question asked two things at once, and the floor was ruled out.
+>    Prime ruled "text itself", and albok-nemi 0.1.18 shipped the v2 question.
+> 3. **h2o 1704–1747, judge v2, rolled back.** Adult on notes held at 0 as validated, but two margins moved past
+>    the named line:
+>    - library/nonfiction adult 0 → 79: the Schell game-design chapter at 0.531 on h2o's book profile (INC 0.438),
+>      with its 77 concepts inheriting;
+>    - fleet/memory personal 6 → 18 on an unchanged personal question.
 >
-> The section headed "The live seat: Intern-Decision-4B" further down describes INC, the rollback target.
+> albok-dev calls both fixable offline: a per-profile threshold or a `visibility: public` declaration on published
+> titles, and a measured look at how h2o reads the personal question. They bring those to Prime before any third
+> attempt. "The model is not the problem, the two margins are."

-### The h2o seat (`compose.h2o.yaml`, live since 1704)
+### The staged h2o seat (`compose.h2o.yaml`; live 1311–1403 and 1704–1747 on 2026-10-12)

 | | |
 |---|---|
@@ -109,9 +112,10 @@ Both run ON fv-ml1 and share one `.env`.
  the judge.
 - **To INC:** `sudo bash /opt/docker/compose/intern-decision/rollback-to-inc.sh`. It stops h2o, copies `compose.inc.yaml`
  over `compose.yaml` and waits up to 180 s. Image `intern-decision-serve:0.1.3` and its
-  `intern-decision_triton-cache` volume survive the `down`. Used 2026-10-12, 140328 → healthy 140356.
+  `intern-decision_triton-cache` volume survive the `down`. Used 2026-10-12, 140328 → healthy 140356, and again
+  174746 → healthy 174811.

-After either switch, make `stacks/intern-decision/compose.yaml` match the host again (it is h2o since 1704).
+After either switch, make `stacks/intern-decision/compose.yaml` match the host again (it is INC since 1748).

 ⚠ **Tell albok-dev on either switch: Nemi's judge concurrency depends on the seat.** INC scores on one inference
 thread (INV-2, ~16 calls/s at most), so the judge runs with concurrency 1 against it (albok's `nemi.yaml`, since
@@ -120,7 +124,7 @@ h2o, vLLM batches, so it goes back to 4.

 ---

-# The live seat: Intern-Decision-4B (2026-09-30 → 2026-10-12 1308, and 1403–1702; now the rollback target)
+# The live seat: Intern-Decision-4B (2026-09-30 → 10-12 1308, 1403–1702, and again since 1748)

 > **LIVE from 2026-09-30 0941 PT to 2026-10-12 1308 PT, and again since 2026-10-12 1403 PT** on fv-ml1 GPU 1, with cap 9.0 GiB and `MAX_TOKENS` 7,168.
 > Acceptance on the live service is below. `semif` is stopped and kept as the rollback.
@@ -1,33 +1,46 @@
-# STAGED, NOT LIVE (since the 2026-10-12 1403 rollback): the h2o-lightning-4b decision seat, which switch-to-h2o.sh
-# copies over compose.yaml. It served 1311-1403 on 2026-10-12 (Prime via brokkr) and was rolled back when Nemi's judge
-# flagged 242 texts adult against INC's 88 at fleet scale. Address intern-decision.fv.internal:8033 (brokkr rule 8).
-# h2o-lightning-4b @ b43f759b. Stock vLLM 0.30.0 + the vendor's UNMODIFIED shim + services/h2o-decision-front (copied to
-# conf/front): 32,000-token REFUSE-not-trim admission, the seat's bearer token (the same INTERN_DECISION_API_TOKEN),
-# /decide + /decide/shared translated onto /v1/systemone, /health model {name, revision}.
-# Rollback to Intern-Decision-4B: compose.inc.yaml (the previous compose, same .env): see the stack README.
-# GPU 1 is shared (meromero-rp, gen-small): KV is pinned in BYTES (4.48 GiB = 135,070 tokens, the benched size);
-# util only gates the boot check. Gate on scripts/oneoff/fv-ml1-gpucheck.sh before any (re)create.
-# Images: INLINE ONLY (Prime 2026-10-12). LIGHTNING_IMAGE_URLS=off makes the shim refuse every image URL; vLLM's
-# --allowed-media-domains media.invalid (a reserved name that never resolves) is the backstop if the shim ever changes.
+# intern-decision: Intern-Decision-4B (internlm, Apache-2.0) behind intern-decision-serve, on
+# fv-ml1 GPU 1 (the utility card, beside vllm-coder and the erp/meromero seats; scriberr moved to
+# GPU 3 on 2026-09-30 1322, Prime, to free this card's headroom for 32k-token calls).
+# Replaces semif (Prime, 2026-09-30: "replace semif with intern-decision now").
+#
+# One forward pass per call, scored by the checkpoint's OWN inference.py (sha256-pinned); the
+# service keeps semif-serve's HTTP surface (/decide, /decide/shared, /health). Service code +
+# contract: services/intern-decision-serve/ (intern-decision-serve.contract.md). Image built on
+# fv-ml1 from that dir.
+#
+# ⚠ VRAM_CAP_GIB is a HARD cap on torch's allocator (per-process memory fraction), set so the
+#   container's WHOLE nvidia-smi footprint, CUDA context included, fits GPU 1's free memory beside
+#   the static vLLM seats (infra-ops budget, 2026-09-30); MAX_TOKENS keeps every accepted call under the cap. A request that needs more
+#   gets 503 out_of_memory and the service stays up. See the README before changing it.
+#
+# .env (tunables): IMAGE, PORT, GPU_ID, VRAM_CAP_GIB, MAX_TOKENS, HOST_IP, INTERN_DECISION_API_TOKEN
+#   (vault intern-decision/api-token, >= 32 chars).
+
 name: intern-decision
+
 services:
-  vllm:
-    image: vllm/vllm-openai:v0.30.0
-    container_name: intern-decision-vllm
+  intern-decision:
+    image: ${IMAGE:?set IMAGE}
+    container_name: intern-decision
    restart: unless-stopped
    ports:
-      - "127.0.0.1:18299:8000"
+      - "${PORT:-8033}:8000"
    environment:
-      VLLM_USE_FLASHINFER_SAMPLER: "0"
+      INTERN_DECISION_API_TOKEN: ${INTERN_DECISION_API_TOKEN:?set INTERN_DECISION_API_TOKEN}
+      INTERN_DECISION_DEVICE: cuda
+      INTERN_DECISION_VRAM_CAP_GIB: ${VRAM_CAP_GIB:?set VRAM_CAP_GIB}
+      # Coupled to VRAM_CAP_GIB: the largest call measured to fit under the cap (README "VRAM").
+      INTERN_DECISION_MAX_TOKENS: ${MAX_TOKENS:?set MAX_TOKENS}
+      INTERN_DECISION_MAX_DECISIONS: ${MAX_DECISIONS:-64}
+      # POSTs in progress (queued + scoring) before new ones get 429 busy.
+      INTERN_DECISION_MAX_QUEUE: ${MAX_QUEUE:-32}
    volumes:
-      - /tank/aimodels/h2o-lightning-4b-b43f759b:/model:ro
-      - vllm-cache:/root/.cache/vllm
-    command: ["--model", "/model", "--served-model-name", "h2oai/h2o-lightning-4b",
-              "--max-model-len", "40960", "--max-num-seqs", "16",
-              "--gpu-memory-utilization", "0.1506", "--kv-cache-memory", "4810363371",
-              "--limit-mm-per-prompt", "{\"image\": 4, \"video\": 0}",
-              "--mm-processor-kwargs", "{\"max_pixels\": 1605632}",
-              "--allowed-media-domains", "media.invalid"]
+      # Pinned weights AND the checkpoint's inference.py, read offline. Never downloads.
+      - /tank/aimodels/huggingface:/hf:ro
+      # Triton/fla autotune cache (README "Cold-shape latency"): the ~6.5 s first-call-per-bucket
+      # autotune results survive RECREATES (deploys, image upgrades, .env edits), not just restarts.
+      # The mount point exists in the image owned by 10001, so the fresh volume is intern-writable.
+      - triton-cache:/tmp/triton-cache
    deploy:
      resources:
        reservations:
@@ -35,41 +48,27 @@ services:
            - driver: nvidia
              device_ids: ["${GPU_ID:-1}"]
              capabilities: [gpu]
-  front:
-    image: python:3.12-slim
-    container_name: intern-decision
-    restart: unless-stopped
-    network_mode: host
-    depends_on: [vllm]
-    environment:
-      FRONT_TOKEN: ${INTERN_DECISION_API_TOKEN:?set INTERN_DECISION_API_TOKEN}
-      SHIM_VLLM: http://127.0.0.1:18299
-      SHIM_PORT: "18291"
-      SHIM_NOUL_FLOOR: "0.801"
-      LIGHTNING_IMAGE_URLS: "off"   # Prime 2026-10-12: inline images only, the seat never fetches an image URL
-      FRONT_SHIM: http://127.0.0.1:18291
-      FRONT_VLLM: http://127.0.0.1:18299
-      FRONT_BIND: 0.0.0.0
-      FRONT_PORT: ${PORT:-8033}
-      FRONT_CAP: "32000"
-      FRONT_SHIM_FILE: /model/h2o_lightning_shim.py
-      FRONT_SERVE_CONFIG: /model/serve_config.json
-      FRONT_MODEL_REVISION: b43f759b6385d21182f87c3829931ec3d4dfa98a
-    volumes:
-      - /tank/aimodels/h2o-lightning-4b-b43f759b:/model:ro
-      - /opt/docker/conf/intern-decision/front:/front:ro
-    command: ["bash", "/front/run.sh"]
    healthcheck:
-      test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8033/health', timeout=5)"]
+      test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=5)"]
      interval: 30s
      timeout: 10s
      retries: 3
+      # Startup loads ~9 GB of weights and scores the warm-up three times before it serves.
      start_period: 300s
+    networks:
+      - tnet
    labels:
      - homepage.group=AI - Eval & Retrieval
-      - homepage.name=Decision seat (h2o-lightning-4b)
+      - homepage.name=Intern-Decision — typed decisions
      - homepage.icon=mdi-scale-balance
-      - homepage.description=Typed decisions, 32k refusing (h2o-lightning-4b on vLLM, fv-ml1 GPU1; /decide/shared kept)
+      - homepage.description=Typed decisions from one forward pass (Intern-Decision-4B, fv-ml1 GPU1)
      - homepage.href=http://${HOST_IP:-10.251.50.54}:${PORT:-8033}/health
+
+networks:
+  tnet:
+    name: traefik-net
+    external: true
+
 volumes:
-  vllm-cache:
+  # -> intern-decision_triton-cache on the host
+  triton-cache: