64 lines
4.3 KiB
Markdown
64 lines
4.3 KiB
Markdown
# Diariz (pilot) — live diarized meeting transcription on fv-ml1 GPU 3
|
|
|
|
**What:** [kenhayward/Diariz](https://github.com/kenhayward/Diariz) (AGPL-3.0). It does live speaker-labelled captions in the
|
|
browser, audio/video uploads with a full-recording re-pass, and summaries/minutes/actions through the LiteLLM gateway.
|
|
It is a pilot **alongside Scriberr, never instead of it**. Prime uses Scriberr live in meetings, and its idle gate stands.
|
|
Brokkr's request for Prime (2026-10-19, thread `01M5AH9QCVW7ZQ6VVXGWQGFPRS`). Prime chose GPU 3 at 1035 2026-10-19.
|
|
|
|
**URL:** https://diariz.nh3.phasefinal.com (fleet TLS caddy on nh3-dev → `fv-ml1:8081`). Login `lkraven@lkraven.com`,
|
|
password `secret get diariz/seed-password`.
|
|
|
|
## Layout on fv-ml1
|
|
|
|
| | |
|
|
|---|---|
|
|
| source | `/opt/docker/src/diariz-5a96cad`: a clean `git archive` of main `5a96cade77ba` (2026-10-16), owned by infra-ops. Upstream ships no images, so it is built locally |
|
|
| compose | upstream `deploy/docker-compose.yml` + our `deploy/docker-compose.override.yml` (this dir). `/opt/docker/compose/diariz` is a **symlink** to that deploy dir |
|
|
| .env | `deploy/.env` (0600). Rendered from the vault by `render-env.sh` and piped over ssh; never on disk on nh3-dev. Key list: `.env.example` |
|
|
| project | `diariz`: postgres (pgvector pg16), redis 8, SeaweedFS S3, api (.NET), web (nginx), worker, live-worker (profile on) |
|
|
| GPU | worker + live-worker pinned to **GPU 3** by `device_ids` (upstream reserves `count: all`). Scriberr shares the card (about 5.5 GB per job) |
|
|
| ports | web `0.0.0.0:8081`; api `127.0.0.1:18080` (upstream's 8080 is Scriberr's); postgres `127.0.0.1:5433`; S3 `127.0.0.1:9002` |
|
|
| data | named volumes `diariz_pgdata`, `diariz_s3data` (recordings), `diariz_workercache` (model weights), `diariz_apikeys`, `diariz_redisdata`. ⚠ Not in the restic set yet: it's a pilot |
|
|
|
|
## Fleet settings (the why)
|
|
|
|
- **Summaries:** `gen` (flash-next, 262,144 window) with `CHAT_CONTEXT_LENGTH=245760`, which leaves 16k for the reply. The key is LiteLLM alias **`diariz`**
|
|
(models `gen` + `whisper-1` only), vaulted as `diariz/litellm-key`.
|
|
- **Dictation** (chat voice input only): `whisper-1` = the gateway's Parakeet alias. Meeting transcription runs on the worker's own
|
|
WhisperX large-v3 and does **not** use the Parakeet seat.
|
|
- **Embeddings off.** Diariz's vector column is 768-d and the gateway's `qwen3-embedding` is 1024-d, so `EMBED_*` is unset and search is lexical.
|
|
- **MCP OAuth off** (`MCP_OAUTH_ENABLED=false`). Prime can turn it on.
|
|
- **HF token:** `fv-ml1/hf-token`, because `pyannote/speaker-diarization-3.1` is gated.
|
|
|
|
## Memory behaviour and the nightly restart
|
|
|
|
Read at source (`src/Diariz.Worker/pipeline.py`, `worker.py`, `gpu_memory.py`):
|
|
- The models load **lazily on the first job** and stay loaded. Each job's working memory is freed afterwards.
|
|
- The live worker drops its copy only when live transcription is switched off in the app.
|
|
|
|
So `diariz-nightly-restart.timer` (03:00) restarts both workers to drop the models. It is **idle-gated**: GPU 3 is sampled for
|
|
20 s, and any sample above 10% (a Diariz or a Scriberr job) means wait 15 min, retrying until 0500, then skip.
|
|
Playbook: `playbooks/fv-ml1-diariz-nightly-restart.yaml`. Log: `journalctl -u diariz-nightly-restart`.
|
|
|
|
## Operate
|
|
|
|
```bash
|
|
ssh infra-ops@10.251.50.54
|
|
cd /opt/docker/compose/diariz # = /opt/docker/src/diariz-5a96cad/deploy
|
|
docker compose ps
|
|
docker compose logs -f worker live-worker
|
|
docker compose up -d # after a .env change (re-render it from nh3-dev first)
|
|
```
|
|
|
|
**Upgrade:** a new `git archive` of a new pin into `/opt/docker/src/diariz-<sha7>`. Copy the override and re-render the `.env`,
|
|
`docker compose build`, `down` in the old dir, `up -d` in the new one (same project name, so the volumes carry over), then repoint the symlink.
|
|
Upstream ships very often: pin a commit and read its migrations notes first.
|
|
|
|
**Rollback / remove:** `docker compose down` in the deploy dir (add `-v` only to throw the data away). Disable the timer, then remove
|
|
the `@diariz` block from `configs/fleet-tls-caddy/Caddyfile`.
|
|
|
|
## Known limits (not deploy bugs)
|
|
|
|
No YouTube/URL ingest (files only). Upstream issue #736: speaker-count instability. Desktop capture apps are Windows/macOS only;
|
|
on Linux, capture runs in the browser. Quality is unmeasured, which is the point of the pilot.
|