docs: Jev candidate bench vs SemIf (fv-ml1 GPU 3) — Intern-Decision-4B is the replacement if SemIf is displaced

Operator ask relayed by brokkr-smithy-dev. Positive control (SemIf 187/231,
hard 0.613) reproduced exactly; negative control and a 4-restart noise floor
(0 flips) measured. On our replaced-baseline sets no candidate beats SemIf-with-
rotations beyond the ~4-pt floor; Intern-Decision-4B native matches it at one
ordering, is better on Wyrd, fits 9.7/10.3 GB and is 1.5-2.3x faster. JevBench
rank does not transfer. Raw per-item data kept out of git.
This commit is contained in:
vh
2026-09-30 05:00:45 -07:00
parent 9a6ac59da7
commit 475d6d6bcb
27 changed files with 19068 additions and 0 deletions
@@ -0,0 +1,10 @@
# Transient bench image (2026-09-30): semif-serve:0.1.4's exact torch 2.10.0+cu128 / transformers 5.17.0 /
# fla 0.5.2 / causal-conv1d 1.7.0 stack, plus what the candidates' own runtimes need. Removed after the bench.
FROM semif-serve:0.1.4
USER root
RUN uv pip install --python /app/.venv/bin/python --no-cache --no-deps \
"jevk5 @ git+https://github.com/allebee/jevk5@85238d7be5527370c43206fe54cd752eb3134c1b" \
&& uv pip install --python /app/.venv/bin/python --no-cache peft pillow python-multipart \
&& uv pip install --python /app/.venv/bin/python --no-cache --no-deps \
--index-url https://download.pytorch.org/whl/cu128 "torchvision==0.25.0"
USER semif