fix(scriberr): patch 0004 — chat context window for gateway model names (LLM_DEFAULT_CONTEXT_WINDOW=131072); image ctxwin1 live, idle-gated deploy; svos v0.0.20 staged
This commit is contained in:
@@ -157,7 +157,7 @@ Every vLLM seat on a shared card (GPU 0, GPU 1) has byte-pinned KV (`--kv-cache-
|
||||
|
||||
### Scriberr (fv-ml1 GPU 3)
|
||||
|
||||
- **LIVE `scriberr:local-blackwell-a353078-dropout2`:** upstream a353078 plus patch 0001 (overlap slicer) and patch 0002 (gap retry, `PARAKEET_MODEL_PATH`), carried LOCALLY ONLY (Prime 2026-10-01: no upstream). `patches/0003` (pyannote env pins) is committed but in no image yet. `scripts/scriberr-rebuild` re-applies all. → `persistent-memory.d/2026-09-30-scriberr-slicer-gap-retry-gpu3.md`
|
||||
- **LIVE `scriberr:local-blackwell-a353078-ctxwin1` since 10-19 0051** (was `-dropout2`, kept for rollback via `.env.bak-ctxwin1`): upstream a353078 + patches 0001 (overlap slicer), 0002 (gap retry, `PARAKEET_MODEL_PATH`), 0003 (pyannote env pins; the live env was already hand-pinned and did not re-sync) and **0004 (chat: `LLM_DEFAULT_CONTEXT_WINDOW`, compose default 131072; upstream returned 4096 for every non-OpenAI model name, so chat refused any transcript over ~3.6k est. tokens — Prime 10-19 "fix the 4k issue")**, carried LOCALLY ONLY (Prime 2026-10-01: no upstream). Rebuild verdict PASS; deploy idle-GATED (`playbooks/scriberr-deploy-ctxwin1.yaml`). Post-deploy: all 7 backends initialized; pyannote smoke on the new container: AMI 19.5 s clip 2 speakers ×3 runs identical, SCOTUS 30 min 8 speakers / 196 segments in 52 s (functional check, not a quality comparison). Chat end-to-end not re-tried by infra-ops (Prime's account). ⚠ Prime uses Scriberr LIVE IN MEETINGS (diarized notes): every restart must pass the idle gate. Prime is weighing a from-scratch rebuild with Scriberr as the reference design (10-19; no decision). `scripts/scriberr-rebuild` re-applies all. → `persistent-memory.d/2026-09-30-scriberr-slicer-gap-retry-gpu3.md`
|
||||
- **Diarization works (fixed 2026-10-06):** model caches on /tank, `HF_TOKEN` a placeholder, saved profiles `WhisperX large-v3 · GPU · speakers` and `Parakeet · GPU · speakers`. A NEW gated model needs one download with vault `fv-ml1/hf-token`. → `persistent-memory.d/2026-10-06-scriberr-diarization-fixed.md`
|
||||
- **URL is https://scriberr.nh3.phasefinal.com since 2026-10-01** (fronted by the fleet TLS caddy on nh3-dev; Caddyfile backup `.bak-20261001-scriberr`). `getUserMedia` is refused on http://, so recording needs the HTTPS name, which is in `ALLOWED_ORIGINS` (`playbooks/scriberr-https-origin.yaml`). ⚠ `/opt/docker/compose/scriberr` is lkraven-owned, so `deploy-stack.sh` as infra-ops FAILS there; upload single files with `elway --upload --sudo --owner lkraven:lkraven`.
|
||||
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
# fv-ml1: Scriberr -> scriberr:local-blackwell-a353078-ctxwin1 (patches 0001-0004; 0004 = chat's
|
||||
# LLM_DEFAULT_CONTEXT_WINDOW, Prime 2026-10-19 "fix the 4k issue"; 0003's pyannote pins ride along).
|
||||
# Rebuild verdict PASS (scripts/scriberr-rebuild --suffix ctxwin1). Prime uses Scriberr live in meetings, so the
|
||||
# FIRST step is an idle gate that STOPS the run (exit 3) on any queued/processing job, a job start in the last
|
||||
# 2 min, or a compute process on GPU 3 - it gates, it does not just print (auto-memory
|
||||
# feedback_idle_check_must_gate_the_restart). Rollback: restore .env.bak-ctxwin1 (SCRIBERR_IMAGE=...-dropout2) and
|
||||
# `docker compose up -d scriberr`; the old image is kept.
|
||||
steps:
|
||||
- name: IDLE GATE - no job queued or running, no recent job start, GPU 3 empty
|
||||
sudo: true
|
||||
shell: |
|
||||
busy=$(python3 -c "
|
||||
import sqlite3
|
||||
c = sqlite3.connect('file:/tank/scriberr/data/scriberr.db?mode=ro', uri=True)
|
||||
print(c.execute(\"select count(*) from transcription_jobs where status not in ('completed','failed','cancelled','error')\").fetchone()[0])")
|
||||
recent=$(docker logs --since 2m scriberr 2>&1 | grep -c "Processing single-track job")
|
||||
gpu=$(nvidia-smi -i 3 --query-compute-apps=pid --format=csv,noheader | grep -c .)
|
||||
echo "jobs-not-done=$busy recent-starts=$recent gpu3-procs=$gpu"
|
||||
[ "$busy" = 0 ] && [ "$recent" = 0 ] && [ "$gpu" = 0 ] || { echo "BUSY - not deploying"; exit 3; }
|
||||
changed_when: "false"
|
||||
- name: Back up .env, then select the ctxwin1 image
|
||||
sudo: true
|
||||
shell: |
|
||||
set -e
|
||||
cd /opt/docker/compose/scriberr
|
||||
cp -p .env .env.bak-ctxwin1
|
||||
sed -i 's#^SCRIBERR_IMAGE=.*#SCRIBERR_IMAGE=scriberr:local-blackwell-a353078-ctxwin1#' .env
|
||||
chown lkraven:lkraven .env && chmod 600 .env # sed -i as root would leave it root-owned
|
||||
grep -n '^SCRIBERR_IMAGE=' .env; stat -c '%U:%G %a' .env
|
||||
- name: Recreate scriberr (compose config must parse first)
|
||||
sudo: true
|
||||
shell: |
|
||||
set -e
|
||||
cd /opt/docker/compose/scriberr
|
||||
docker compose config >/dev/null
|
||||
docker compose up -d scriberr 2>&1 | tail -3
|
||||
verify:
|
||||
- name: Running the new image, with the context-window env inside
|
||||
sudo: true
|
||||
shell: |
|
||||
docker inspect scriberr --format '{{.Config.Image}} {{.State.Status}} restarts={{.RestartCount}}'
|
||||
docker exec scriberr sh -c 'echo LLM_DEFAULT_CONTEXT_WINDOW=$LLM_DEFAULT_CONTEXT_WINDOW'
|
||||
docker inspect scriberr --format '{{.Config.Image}}' | grep -q ctxwin1
|
||||
changed_when: "false"
|
||||
@@ -0,0 +1,26 @@
|
||||
# nh3-svos: STAGE SVOS v0.0.20 (81cc5a2) beside the live release. Installs into /opt/svos/releases/81cc5a2 and runs the
|
||||
# new kernel's check-config against the live configuration. Does NOT switch current, stop or start anything.
|
||||
# svos-dev's request (thread 01M59JF8Z900KCHP5CJ98F90BW): UI 0.0.24 (U21 stronger by default; the /tasks draft waits 125 s); kernel 0.0.20 is a
|
||||
# version bump; everything else carried from 91e9f2d. Built from the LOCAL tag before the push; the publish
|
||||
# (nh3-svos-81cc5a2-v0.0.20.yaml) first needs origin's v0.0.20 to name the same commit.
|
||||
steps:
|
||||
- name: Private staging dir
|
||||
shell: install -d -m 0700 /tmp/svos-stage-81cc5a2
|
||||
- {name: stage release, upload: {src: /tmp/claude-1000/-home-lkraven-development-eshpfi-management/e0be18fd-0734-4786-b933-904f0a74d2a2/scratchpad/svos-81cc5a2/svos-release-81cc5a2.tar.gz, dest: /tmp/svos-stage-81cc5a2/release.tar.gz, mode: "0644"}}
|
||||
- {name: stage uv, upload: {src: /home/lkraven/bin/uv, dest: /tmp/svos-stage-81cc5a2/uv, mode: "0755"}}
|
||||
- {name: stage installer, upload: {src: /tmp/claude-1000/-home-lkraven-development-eshpfi-management/e0be18fd-0734-4786-b933-904f0a74d2a2/scratchpad/svos-81cc5a2/install-release.py, dest: /tmp/svos-stage-81cc5a2/install-release.py, mode: "0644"}}
|
||||
- name: Verify staged hashes
|
||||
shell: cd /tmp/svos-stage-81cc5a2 && printf '%s %s\n' e81fadd86bc409e3032584f13fb4df3b6c67b846e55011cc45c7688500e82614 release.tar.gz 83aa128da93e168283ed32cf75cd819fe17b135b3fc1f7ccb3c912eec198d9fb uv | sha256sum -c -
|
||||
changed_when: "false"
|
||||
- name: Install release 81cc5a2 beside the live one (current NOT switched)
|
||||
sudo: true
|
||||
shell: python3 /tmp/svos-stage-81cc5a2/install-release.py
|
||||
creates: /opt/svos/releases/81cc5a2/apps-env/pyvenv.cfg
|
||||
verify:
|
||||
- name: New kernel 0.0.20 accepts the live configuration as it is
|
||||
sudo: true
|
||||
shell: runuser -u svc-svos -- /opt/svos/releases/81cc5a2/kernel-env/bin/svos-kernel check-config --data-root /var/lib/svos
|
||||
changed_when: "false"
|
||||
- name: current still points at the live release
|
||||
shell: readlink /opt/svos/current
|
||||
changed_when: "false"
|
||||
@@ -0,0 +1,26 @@
|
||||
# nh3-svos -> SVOS v0.0.20 (81cc5a2): UI-only publish. UI 0.0.24 (U21: the working look at stronger by default;
|
||||
# the /tasks draft now waits 125 s instead of the 10 s abort). Kernel 0.0.20 = a version bump; no app change, no card.
|
||||
# svos-dev's request, thread 01M59JF8Z900KCHP5CJ98F90BW. PRE-REQS: the stage playbook; origin's v0.0.20 == 81cc5a21dfab
|
||||
# (git ls-remote on nh3-dev before this runs). current must move because svos-front-inventory generates from
|
||||
# /opt/svos/current/static; NOTHING restarts: the kernel, knowledge, judge and voice keep running 15c2d04's code and the
|
||||
# interface 6d4cb1c's (all identical in code), so keep both dirs until those processes restart.
|
||||
steps:
|
||||
- name: Pre-reqs present
|
||||
sudo: true
|
||||
shell: test -f /opt/svos/releases/81cc5a2/static/index.html && readlink /opt/svos/current
|
||||
changed_when: "false"
|
||||
- name: Switch current to 81cc5a2
|
||||
sudo: true
|
||||
shell: ln -sfn /opt/svos/releases/81cc5a2 /opt/svos/current.new && chgrp -h svc-svos /opt/svos/current.new && mv -T /opt/svos/current.new /opt/svos/current && readlink /opt/svos/current
|
||||
- name: Front door inventory (UI 0.0.24 live; 91e9f2d becomes previous)
|
||||
sudo: true
|
||||
shell: /usr/local/sbin/svos-front-inventory 2>&1 | grep -E 'GENERATED|updated|root \*|path /assets' | head -4
|
||||
verify:
|
||||
- name: Front door inventory current; health unchanged
|
||||
sudo: true
|
||||
shell: /usr/local/sbin/svos-front-inventory --check && curl -s http://127.0.0.1:8400/health && echo
|
||||
changed_when: "false"
|
||||
- name: Which release dir each running service maps (nothing restarted)
|
||||
sudo: true
|
||||
shell: for u in svos-kernel svos-interface svos-knowledge svos-judge svos-voice; do pid=$(systemctl show $u -p MainPID --value); printf '%s %s\n' $u "$(grep -oE '/opt/svos/releases/[0-9a-f]+' /proc/$pid/maps | sort -u | tr '\n' ' ')"; done
|
||||
changed_when: "false"
|
||||
@@ -54,3 +54,6 @@ SCRIBERR_SECURE_COOKIES=false
|
||||
# when unset; override only with a re-measured peak.
|
||||
# SCRIBERR_PARAKEET_CHUNK_SECS=120
|
||||
# SCRIBERR_PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
|
||||
|
||||
# Chat context window for gateway model names (patch 0004; needs an image built with it).
|
||||
# SCRIBERR_LLM_DEFAULT_CONTEXT_WINDOW=131072
|
||||
|
||||
@@ -65,6 +65,10 @@ services:
|
||||
# reached by host IP. Keep this in sync with how the app is reached.
|
||||
- ALLOWED_ORIGINS=${SCRIBERR_ALLOWED_ORIGINS}
|
||||
- NVIDIA_DRIVER_CAPABILITIES=compute,utility
|
||||
# Chat's context window for a model name Scriberr's OpenAI table does not know (patch 0004, 2026-10-19).
|
||||
# Upstream returns 4096 for every non-OpenAI name, so chat refused any transcript over ~3.6k estimated
|
||||
# tokens even on 262k-context gateway models (summarizer, gen-large). Half their real context.
|
||||
- LLM_DEFAULT_CONTEXT_WINDOW=${SCRIBERR_LLM_DEFAULT_CONTEXT_WINDOW:-131072}
|
||||
# Scriberr builds each model backend's Python env with `uv` at runtime.
|
||||
# uv's default link mode reflink/hardlinks out of its cache, which fails
|
||||
# on this overlayfs+ZFS combination with a misleading
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
Subject: [PATCH 0004] chat: LLM_DEFAULT_CONTEXT_WINDOW for model names the OpenAI table does not know
|
||||
|
||||
OpenAIService.GetContextWindow knows only OpenAI's own names (gpt-4o, gpt-4, gpt-3.5)
|
||||
and returns 4096 for anything else. Behind an OpenAI-compatible gateway every model is
|
||||
"anything else" (here LiteLLM's summarizer and gen-large, 262k-context vLLM seats), so
|
||||
chat refuses any transcript the handler estimates above 3,596 tokens (chars/4, minus 500
|
||||
kept for the reply): a 22k-character meeting was refused on 2026-10-19 with "limit 4096".
|
||||
The default branch now reads LLM_DEFAULT_CONTEXT_WINDOW (a positive integer) and keeps
|
||||
4096 when it is unset or invalid; the known OpenAI names are unchanged.
|
||||
|
||||
---
|
||||
--- a/internal/llm/openai.go
|
||||
+++ b/internal/llm/openai.go
|
||||
@@ -9,6 +9,8 @@
|
||||
"io"
|
||||
"log"
|
||||
"net/http"
|
||||
+ "os"
|
||||
+ "strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
@@ -337,7 +339,11 @@
|
||||
case strings.HasPrefix(model, "gpt-3.5-turbo"):
|
||||
return 16385, nil // Most recent gpt-3.5-turbo is 16k
|
||||
default:
|
||||
- // Default fallback
|
||||
+ // Not an OpenAI model name (a LiteLLM alias, a vLLM served name): the operator's
|
||||
+ // LLM_DEFAULT_CONTEXT_WINDOW when set to a positive integer, else upstream's 4096.
|
||||
+ if v, err := strconv.Atoi(strings.TrimSpace(os.Getenv("LLM_DEFAULT_CONTEXT_WINDOW"))); err == nil && v > 0 {
|
||||
+ return v, nil
|
||||
+ }
|
||||
return 4096, nil
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user