vh
069725c4b3
services/semif-serve is a FastAPI wrapper around SemIf's direct and shared torch scorers (SemIf-OpenJev @ 23cf1f39, MIT). Upstream ships only a batch CLI. The wrapper loads the pinned Qwen3.5-4B (851bf6e8, BF16) once from the offline HF cache and returns SemIf's result dicts unchanged, with an optional per-workload temperature-calibrated view. Contract: semif-serve.contract.md. Built with a short contract, TDD (39 tests, fake engine and fake torch, no GPU) and a heid bug-hunt panel (pending). On the card: - torch 2.10.0+cu128 with sm_120 kernels, which is SemIf's own stack; - a hard 12 GiB VRAM cap. Two defects surfaced only on the card, and each fix is covered by a test: - 0.1.1: an OOM raised as a chained exception kept the failed request's tensors alive (11.9 GiB after the 503). It is now raised unchained, after gc. - 0.1.2: a large request left 12.6 GB reserved on the shared card. After each call, reserved memory over the baseline + 512 MiB is now released. Acceptance against SemIf's committed torch predictions (authored144): - 142/144 same top choice; both misses are exact bf16 ties; - 144/144 identical prompt hashes; - deterministic A-vs-A; - negative control 14/144; - shared vs direct 72/72. 21 binary criteria over one state take 159 ms. The shared-mode capacity table under the cap is in stacks/semif/README.md. The Dockerfile installs dependencies from a manifest with the project version blanked, so a version bump reuses the ~4 GB torch layer. Verified: 41 s rebuild, dependency layer CACHED. DNS: semif.fv.internal. Token: vault semif/api-token.
12 lines
441 B
Bash
12 lines
441 B
Bash
# semif — copy to /opt/docker/compose/semif/.env on fv-ml1 (mode 0600).
|
|
# Built on fv-ml1 from services/semif-serve (see README "Building").
|
|
IMAGE=semif-serve:0.1.2
|
|
PORT=8032
|
|
HOST_IP=10.251.50.54
|
|
# fv-ml1 GPU 1 = the utility card (vllm-coder, erp, meromero, scriberr).
|
|
GPU_ID=1
|
|
# HARD per-process cap, from the measured peak (README "VRAM").
|
|
VRAM_CAP_GIB=12
|
|
# >= 32 characters; source of truth: secret get semif/api-token
|
|
SEMIF_API_TOKEN=
|