docs: Jev candidate bench vs SemIf (fv-ml1 GPU 3) — Intern-Decision-4B is the replacement if SemIf is displaced

Operator ask relayed by brokkr-smithy-dev. Positive control (SemIf 187/231,
hard 0.613) reproduced exactly; negative control and a 4-restart noise floor
(0 flips) measured. On our replaced-baseline sets no candidate beats SemIf-with-
rotations beyond the ~4-pt floor; Intern-Decision-4B native matches it at one
ordering, is better on Wyrd, fits 9.7/10.3 GB and is 1.5-2.3x faster. JevBench
rank does not transfer. Raw per-item data kept out of git.
This commit is contained in:
vh
2026-09-30 05:00:45 -07:00
parent 9a6ac59da7
commit 475d6d6bcb
27 changed files with 19068 additions and 0 deletions
+6
View File
@@ -151,6 +151,12 @@ the fast kernels autotune at startup. That is n=1 row across one restart, and th
floor is otherwise unmeasured. When comparing two versions, compare them against that floor
and not against zero.
**Update 2026-09-30 (the Jev bench on GPU 3, `docs/pfi/jev-candidates-bench-2026-09-30.md`):**
across **4 restarts** of the same image, SemIf changed **0 labels**, and its probabilities were
bit-identical on every set, the 144 authored rows included. So the near-tie flip described above
did not reproduce. The cross-restart floor is now measured at 0 for n=4 restarts on an idle
card. Treat the one flip on 2026-09-27 as possible but rare, not as the norm.
## Acceptance (2026-09-27, v0.1.3)
Raw: `services/semif-serve/acceptance/result-2026-09-27-v0.1.3.json`,