docs: Jev candidate bench vs SemIf (fv-ml1 GPU 3) — Intern-Decision-4B is the replacement if SemIf is displaced

Operator ask relayed by brokkr-smithy-dev. Positive control (SemIf 187/231,
hard 0.613) reproduced exactly; negative control and a 4-restart noise floor
(0 flips) measured. On our replaced-baseline sets no candidate beats SemIf-with-
rotations beyond the ~4-pt floor; Intern-Decision-4B native matches it at one
ordering, is better on Wyrd, fits 9.7/10.3 GB and is 1.5-2.3x faster. JevBench
rank does not transfer. Raw per-item data kept out of git.
This commit is contained in:
vh
2026-09-30 05:00:45 -07:00
parent 9a6ac59da7
commit 475d6d6bcb
27 changed files with 19068 additions and 0 deletions
+14
View File
@@ -0,0 +1,14 @@
#!/usr/bin/env bash
# Runs code/run.sh once per line of queue.txt, in order, one at a time. Lines can be appended while it
# runs. touch STOP to halt after the current step.
W=/tmp/jevbench-2026-09-30
cd $W
while [ ! -e $W/STOP ]; do
line=$(head -n 1 $W/queue.txt 2>/dev/null)
[ -z "$line" ] && { echo "$(date -u +%FT%TZ) queue empty" >> $W/logs/queue.out; break; }
tail -n +2 $W/queue.txt > $W/queue.txt.tmp && mv $W/queue.txt.tmp $W/queue.txt
echo "$(date -u +%FT%TZ) >>> $line" >> $W/logs/queue.out
# shellcheck disable=SC2086
$W/code/run.sh $line >> $W/logs/queue.out 2>&1
echo "$(date -u +%FT%TZ) <<< rc=$? $line" >> $W/logs/queue.out
done