docs: Jev candidate bench vs SemIf (fv-ml1 GPU 3) — Intern-Decision-4B is the replacement if SemIf is displaced
Operator ask relayed by brokkr-smithy-dev. Positive control (SemIf 187/231, hard 0.613) reproduced exactly; negative control and a 4-restart noise floor (0 flips) measured. On our replaced-baseline sets no candidate beats SemIf-with- rotations beyond the ~4-pt floor; Intern-Decision-4B native matches it at one ordering, is better on Wyrd, fits 9.7/10.3 GB and is 1.5-2.3x faster. JevBench rank does not transfer. Raw per-item data kept out of git.
This commit is contained in:
+14
@@ -0,0 +1,14 @@
|
||||
#!/usr/bin/env bash
|
||||
# Runs code/run.sh once per line of queue.txt, in order, one at a time. Lines can be appended while it
|
||||
# runs. touch STOP to halt after the current step.
|
||||
W=/tmp/jevbench-2026-09-30
|
||||
cd $W
|
||||
while [ ! -e $W/STOP ]; do
|
||||
line=$(head -n 1 $W/queue.txt 2>/dev/null)
|
||||
[ -z "$line" ] && { echo "$(date -u +%FT%TZ) queue empty" >> $W/logs/queue.out; break; }
|
||||
tail -n +2 $W/queue.txt > $W/queue.txt.tmp && mv $W/queue.txt.tmp $W/queue.txt
|
||||
echo "$(date -u +%FT%TZ) >>> $line" >> $W/logs/queue.out
|
||||
# shellcheck disable=SC2086
|
||||
$W/code/run.sh $line >> $W/logs/queue.out 2>&1
|
||||
echo "$(date -u +%FT%TZ) <<< rc=$? $line" >> $W/logs/queue.out
|
||||
done
|
||||
Reference in New Issue
Block a user