docs: Jev candidate bench vs SemIf (fv-ml1 GPU 3) — Intern-Decision-4B is the replacement if SemIf is displaced

Operator ask relayed by brokkr-smithy-dev. Positive control (SemIf 187/231, hard 0.613) reproduced exactly; negative control and a 4-restart noise floor (0 flips) measured. On our replaced-baseline sets no candidate beats SemIf-with- rotations beyond the ~4-pt floor; Intern-Decision-4B native matches it at one ordering, is better on Wyrd, fits 9.7/10.3 GB and is 1.5-2.3x faster. JevBench rank does not transfer. Raw per-item data kept out of git.
2026-09-30 05:00:45 -07:00
parent 9a6ac59da7
commit 475d6d6bcb
27 changed files with 19068 additions and 0 deletions
@@ -0,0 +1,14 @@
+import os, sys, time
+from huggingface_hub import snapshot_download
+PINS = [
+    ("crh225/plumb-4b", "55de037801a8a9b9de3db5c0e16cef86210c2186", None),
+    ("alibiserikbay/JevK5", "27d2d6b8d4714807f6293b0623bd7370b27e42f8", None),
+    ("internlm/Intern-Decision-4B", "0e5e6aa7d6d750e2b1504ba11a8136cb58aeb3cd", None),
+    ("mohit67890/imajev-4b", "c9e5f132465da85d31735ec502d5557982671a7d", ["mlx/*", "assets/*"]),
+]
+only = sys.argv[1:] or [p[0] for p in PINS]
+for repo, rev, ignore in PINS:
+    if repo not in only: continue
+    t = time.time()
+    path = snapshot_download(repo, revision=rev, ignore_patterns=ignore)
+    print(f"{repo}@{rev} -> {path} in {time.time()-t:.0f}s", flush=True)