From 377f8a43c8e5727614551b99036bbcf5b99acb21 Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Thu, 6 Aug 2026 10:49:50 -0700 Subject: [PATCH] docs(reranker): record cutover VERIFIED + v13 gate PASS, A2 teardown, throughput Brokkr independent verify clean (maxdiff 0.000000, no split). R42 v13 acceptance gate PASSES first time in its history: main+kb 56/90->90/90, evictions 33->0. A2 control torn down. A3 throughput characterized at ~34 req/s (graceful queueing), with A4/util-bump/replica as levers. --- docs/pfi/reranker-selection-ledger.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/docs/pfi/reranker-selection-ledger.md b/docs/pfi/reranker-selection-ledger.md index ec74d89..992681b 100644 --- a/docs/pfi/reranker-selection-ledger.md +++ b/docs/pfi/reranker-selection-ledger.md @@ -209,3 +209,22 @@ on ana-ml2; `/model/delete` the two aliases (DB-persisted); weights retained in - **Incumbent (`vllm-rerank` :8002)** — keep up as rollback until Brokkr/worldtree close the post-cutover watch; retire (not delete) only on explicit sign-off. - **`rerank_hybrid_floor`** — Brokkr routing to worldtree-dev directly (drop, don't re-tune). + +### 2026-08-06 — VERIFIED + A2 torn down + throughput characterized + +- **Brokkr independent verify: CUTOVER VERIFIED** — prod `reranker` == `reranker-a3-bge-v2-m3` + at maxdiff 0.000000 (5 samples, spread 0.000009), single backend, no split routing. +- **R42 v13 acceptance gate PASSES** — anchors_flip 4/4, no_regression 8/8, no_distractor_rise + TRUE, zero aborts. **First PASS in R42 history after 4 failed verdicts.** Production main+kb: + 56/90 → 90/90 top-10; evictions 33 → 0. +- **A2 torn down** (Brokkr signalled done): gateway alias `reranker-a2-qwen3-seqcls` deleted + (/model/delete 200) + container removed. ~3.5 GB freed on GPU1. Remaining: `vllm-rerank` + (incumbent, rollback), `vllm-rerank-a3` (prod), `vllm-rerank-a4` (fallback). +- **Throughput characterized (the one open risk):** A3 caps ~34 req/s — flat from 8→16 + concurrent while latency climbs gracefully (p50 214→332→456 ms; p99 525 ms @16-conc). It + QUEUES, doesn't cliff. ~40% below the incumbent's ~55 req/s. Likely fine for fleet rerank + QPS (internal, per-search), but if real p99/queue-depth bites: levers are (a) swap to A4 + (~51 req/s, but English-only), (b) raise A3 `--gpu-memory-utilization` for bigger batching + (recreate = brief blip), (c) run a 2nd A3 replica load-balanced behind `reranker` (~2× tput, + identical replicas so no split-measurement issue now the bake-off is closed). Brokkr will + re-run the grid against A4 if it bites — no intuition swaps.