docs: Jev candidate bench vs SemIf (fv-ml1 GPU 3) — Intern-Decision-4B is the replacement if SemIf is displaced
Operator ask relayed by brokkr-smithy-dev. Positive control (SemIf 187/231, hard 0.613) reproduced exactly; negative control and a 4-restart noise floor (0 flips) measured. On our replaced-baseline sets no candidate beats SemIf-with- rotations beyond the ~4-pt floor; Intern-Decision-4B native matches it at one ordering, is better on Wyrd, fits 9.7/10.3 GB and is 1.5-2.3x faster. JevBench rank does not transfer. Raw per-item data kept out of git.
This commit is contained in:
@@ -151,6 +151,12 @@ the fast kernels autotune at startup. That is n=1 row across one restart, and th
|
||||
floor is otherwise unmeasured. When comparing two versions, compare them against that floor
|
||||
and not against zero.
|
||||
|
||||
**Update 2026-09-30 (the Jev bench on GPU 3, `docs/pfi/jev-candidates-bench-2026-09-30.md`):**
|
||||
across **4 restarts** of the same image, SemIf changed **0 labels**, and its probabilities were
|
||||
bit-identical on every set, the 144 authored rows included. So the near-tie flip described above
|
||||
did not reproduce. The cross-restart floor is now measured at 0 for n=4 restarts on an idle
|
||||
card. Treat the one flip on 2026-09-27 as possible but rare, not as the norm.
|
||||
|
||||
## Acceptance (2026-09-27, v0.1.3)
|
||||
|
||||
Raw: `services/semif-serve/acceptance/result-2026-09-27-v0.1.3.json`,
|
||||
|
||||
Reference in New Issue
Block a user