fix(semif): 0.1.4 — object states ending in ) ; } no longer 422 (INV-7)

SemIf's shared scorer trims one token at the state boundary. When an object
state's last value ends in ')', ';' or '}', the JSON that follows re-merges two
tokens back, so score_shared refused the request with 422. The engine now wraps
semif_phase1.shared._state_prefix to keep only the tokens the full prompts
share. Each row scores the same token sequence; only the prefill/suffix split
moves.

Startup proves the fix is in effect, not just installed (heid bug hunt SKAL,
folded). It checks that the hook is callable and is what score_shared resolves,
that an ordinary state keeps upstream's whole prefix, and that a merge-prone
state scores through the shared path.

Real tokenizer: 154 states, 23 refused before and 0 after, with no ordinary or
authored144 prefix changed. Acceptance: 144/144 parity. Shared vs direct 71/72;
the miss is a bf16 tie that flipped across a plain restart (see README).
This commit is contained in:
vh
2026-09-27 10:23:14 -07:00
parent 7e11cf247b
commit 47cad33dd1
11 changed files with 384 additions and 20 deletions
+37 -10
View File
@@ -53,12 +53,15 @@ curl -s -H "Authorization: Bearer $T" http://10.251.50.54:8032/decide -d '{
644 suffix tokens, 5 calls after warm-up). One cold call at that size returned 503. The
VRAM table below covers binary decisions only. For many options, use one ordering or
shorter option text.
- ⚠ **`/decide/shared` refuses some object states.** If the state is an object whose
LAST value ends in `)`, `;` or `}`, the service returns 422 "The fixed state prefix
does not match every full prompt". The closing `"}` merges with that character into one
token. The same text as a plain string state works, and `.`, `!`, `?`, `]`, `…` and
`—` endings work. Not fixed yet. Callers that pass user text last should
append a full stop or send a string state.
- **Object states ending in `)`, `;` or `}` work as of 0.1.4.** Until then, if the state was
an object whose last value ended in one of those characters, the service returned 422 "The
fixed state prefix does not match every full prompt". SemIf trims one token at the state
boundary, and the JSON that follows re-merged two tokens back. The fix (contract INV-7)
moves only where the shared prefix ends; every row still scores the same tokens. Measured
with the real tokenizer: of 154 states (22 endings × 7 shapes) SemIf alone refused 23, and
with the fix none. None of the 131 ordinary states, nor any of the authored144 states,
changed its prefix. Startup proves the fix is live by scoring `{"person_said": "ok :)"}`
through the shared path.
## ⚠ Probabilities are uncalibrated
@@ -123,6 +126,21 @@ warm-up fails, and startup fails closed. Build without the kernels:
| 6 orderings, short | 118 ms | 81 ms |
| 3 rotations, ~2,000-token state | 200 ms | 158 ms |
## Acceptance (2026-09-27, v0.1.4)
Raw: `services/semif-serve/acceptance/result-2026-09-27-v0.1.4.json`. Parity with upstream
144/144 (identical prompt hashes, max prob gap 0.060). Deterministic within the process
(A-vs-A gap 0.0). The negative control fails as it should (14/144). Shared vs direct 71/72.
The one miss is an exact bf16 tie in the shared result (0.444/0.444). INV-7 did not move that
row's prefix. After a plain `docker restart` of the same image, the row read 0.369/0.537 and
agreed.
⚠ **So "deterministic" holds within one process, not across restarts.** Logits come in bf16
steps (0.125 here), and a near-tie can land differently after a restart, most likely because
the fast kernels autotune at startup. That is n=1 row across one restart, and the cross-restart
floor is otherwise unmeasured. When comparing two versions, compare them against that floor
and not against zero.
## Acceptance (2026-09-27, v0.1.3)
Raw: `services/semif-serve/acceptance/result-2026-09-27-v0.1.3.json`,
@@ -149,16 +167,25 @@ now matches the label. So the misses come from the numeric path, not the wrapper
speed figure includes one network round trip (~27 ms). The burst release costs ~16 ms
on it (0.1.1 measured 143 ms without the release).
## Building
## Building and deploying
```bash
# from nh3-dev
tar -C services/semif-serve -cf - --exclude=.venv --exclude=.pytest_cache --exclude=__pycache__ --exclude=acceptance . \
| ssh infra-ops@10.251.50.54 'mkdir -p /opt/docker/src/semif-serve-X.Y.Z && tar -x -C /opt/docker/src/semif-serve-X.Y.Z'
# from nh3-dev. /opt/docker/src is root-owned, so create the version dir with sudo first.
ssh infra-ops@10.251.50.54 'sudo -n install -d -o infra-ops -g infra-ops /opt/docker/src/semif-serve-X.Y.Z'
tar -C services/semif-serve -cf - --exclude=.venv --exclude=.pytest_cache --exclude=__pycache__ \
--exclude=acceptance --exclude=spike --exclude='*.egg-info' . \
| ssh infra-ops@10.251.50.54 'tar -x -C /opt/docker/src/semif-serve-X.Y.Z'
# on fv-ml1
cd /opt/docker/src/semif-serve-X.Y.Z && docker build -t semif-serve:X.Y.Z .
# ⚠ /opt/docker/compose/semif is root-owned, so `sed -i` cannot write its temp file there. .env
# itself is infra-ops's: rewrite it IN PLACE, which keeps its owner and mode (0600).
cd /opt/docker/compose/semif && new=$(sed 's/^IMAGE=.*/IMAGE=semif-serve:X.Y.Z/' .env) \
&& printf '%s\n' "$new" > .env && docker compose up -d
```
Startup fails closed (INV-3, INV-7), so a container that does not reach healthy did not pass
its own checks: read `docker logs semif`. Record the deploy with `scripts/ops-log`.
To move SemIf forward: bump the commit in `services/semif-serve/pyproject.toml`
(and `SEMIF_COMMIT` in `config.py`), run `uv lock`, rebuild, and **re-run the
acceptance**. Upstream is research code that changes weekly, which is why it is