vh
47cad33dd1
SemIf's shared scorer trims one token at the state boundary. When an object state's last value ends in ')', ';' or '}', the JSON that follows re-merges two tokens back, so score_shared refused the request with 422. The engine now wraps semif_phase1.shared._state_prefix to keep only the tokens the full prompts share. Each row scores the same token sequence; only the prefill/suffix split moves. Startup proves the fix is in effect, not just installed (heid bug hunt SKAL, folded). It checks that the hook is callable and is what score_shared resolves, that an ordinary state keeps upstream's whole prefix, and that a merge-prone state scores through the shared path. Real tokenizer: 154 states, 23 refused before and 0 after, with no ordinary or authored144 prefix changed. Acceptance: 144/144 parity. Shared vs direct 71/72; the miss is a bf16 tie that flipped across a plain restart (see README).
12 lines
441 B
Bash
12 lines
441 B
Bash
# semif — copy to /opt/docker/compose/semif/.env on fv-ml1 (mode 0600).
|
|
# Built on fv-ml1 from services/semif-serve (see README "Building").
|
|
IMAGE=semif-serve:0.1.4
|
|
PORT=8032
|
|
HOST_IP=10.251.50.54
|
|
# fv-ml1 GPU 1 = the utility card (vllm-coder, erp, meromero, scriberr).
|
|
GPU_ID=1
|
|
# HARD per-process cap, from the measured peak (README "VRAM").
|
|
VRAM_CAP_GIB=12
|
|
# >= 32 characters; source of truth: secret get semif/api-token
|
|
SEMIF_API_TOKEN=
|