vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(intern-decision): live on fv-ml1 GPU 1; semif marked REPLACED

Browse Source 
intern-decision deployed 0941 PT (cap 9.0 GiB, MAX_TOKENS 7168). Live acceptance: positive
control 240/259 and Wyrd 79/84, bit-identical to the bench (0/560 rows, Δp 0); negative control
10/122/14; largest accepted requests 200 with no 503; per-process 8,812 MiB at rest and 9,866 peak;
GPU 1 Free 15,442 before and 6,581 after (lowest 5,569 under load). Latency from nh3-dev:
21 criteria 114 ms, 16 over ~3,900 tokens 238 ms. semif README banner now REPLACED with the
rollback; fv-ml1 GPU 1 note updated.
This commit is contained in:
vh
2026-09-30 09:48:30 -07:00
parent 750675e391
commit 1cf763a7b1
10 changed files with 3957 additions and 58 deletions
Download Patch File Download Diff File Expand all files Collapse all files
stacks/semif/README.md
+20 -9
View File
@@ -1,14 +1,25 @@
# semif
> ⚠ **OFFLINE since 2026-09-30 0135 PT (Prime: "take semif offline for now").** It was
> stopped (`docker compose stop`, NOT removed) to give scriberr its GPU 1 headroom back:
> scriberr's Parakeet path cuts audio into 5-minute slices and needs over 6 GB, and with
> SemIf resident it had ~6.7 GB and hit CUDA OOM on a 35-minute file. Stopping it moved
> GPU 1 from 91,052 to 81,806 MiB used. The image, weights, config and token are all kept;
> `unless-stopped` keeps it down across a reboot. **Do not restart it without Prime's word.**
> To bring it back: `cd /opt/docker/compose/semif && docker compose start`, but first
> make sure scriberr has its room (the durable fix, shortening scriberr's slice length, is
> deferred by Prime to later). Everything below describes the service as it was deployed.
> ⚠ **REPLACED by intern-decision (Prime, 2026-09-30).** Prime ruled at ~0510 PT: "replace semif
> with intern-decision now". The replacement is `stacks/intern-decision` (Intern-Decision-4B, live since 0941 PT,
> `http://10.251.50.54:8033`, `intern-decision.fv.internal`). It keeps this service's HTTP surface
> (`/decide`, `/decide/shared`, `/health`), so callers only change the URL and the token
> (`secret get intern-decision/api-token`). Its README lists every deliberate difference.
>
> The `semif` container has been **stopped, not removed**, since 0135 PT that day, when it was
> taken offline to give scriberr its GPU 1 headroom back. Its image (`semif-serve:0.1.4`),
> weights, config and token are all kept, and `unless-stopped` keeps it down across a reboot.
> Port 8032 and `semif.fv.internal` stay reserved for it.
>
> **Rollback, only on Prime's word.**
> 1. Stop intern-decision first. The two services do not fit GPU 1 together next to scriberr.
> semif held 9.2 GB at rest and peaked at 12.9 GB.
> `cd /opt/docker/compose/intern-decision && docker compose stop`.
> 2. Start semif: `cd /opt/docker/compose/semif && docker compose start`.
> 3. Check nvidia-smi `Free` on GPU 1 against semif's 12.9 GB peak plus scriberr's 5.5 GB (its
> 120 s slices) before calling it done. If it does not cover both, semif collides with scriberr.
>
> Everything below describes the service as it was deployed.
**SemIf option-logit decisions** on **fv-ml1 GPU 1**, the utility card beside
`vllm-coder`, the erp/meromero seats and scriberr. Deployed 2026-09-27 at Prime's