diff --git a/archival-memory.md b/archival-memory.md index 6da9acf..0b8c84d 100644 --- a/archival-memory.md +++ b/archival-memory.md @@ -317,6 +317,34 @@ _Entries moved out of persistent-memory.md to keep the active file scannable. Re to latency budget), not fixed per-sentence splits. _Archived 2026-06-11._ +- `[2026-06-03]` **Declined worldtree v0.30.4 staging deploy** — that's worldtree-team's + CI/CD lane (a developer `staging/vX.Y.Z` git-tag promote), not infra-ops. They self-corrected + to the same conclusion independently. + _Archived 2026-06-13._ + +- `[2026-06-03]` **R14 scope = (a) provision-only.** infra-ops provides box + CUDA env + + engines + weights + NFS; brokkr/dev wires `arms.py` + runs — keeps infra-ops OFF the + VIVAE-processing path (VIVAE = Variably Intense Vocalizations of Affect/Emotion, CHARTER §4 + highest-liability; operator authorized R&D-eval-only, quarantined). Box = irv-ml1 (A6000 + free; ana-ml2 GPU-saturated). Per-engine venvs (divergent torch stacks); A6000 = `cuda:0` + NATIVE (≠ docker `=1`). + _Archived 2026-06-13._ + +- `[2026-06-03]` **yt-voice-clipper push-to-deploy via gitea webhook** (operator-directed, + after 6 manual rebuilds in ~40 min). Webhook (not poll) — gitea CAN reach the WG IP per the + operator. The proxy env + Homepage labels live in the **host-specific override** (untracked + → survive the auto-deploy's `git reset --hard`), NOT yt-voice-clipper-dev's image. Runbook + `d4f180d`. + _Archived 2026-06-13._ + +- `[2026-06-03]` **yt-voice-clipper bot-gate fix = route yt-dlp through NH3 residential + egress, NOT cookies/PO-token.** YouTube hard-flags the Irvine colo IP (LOGIN_REQUIRED on a + public video even with no cookies). Cookies + the bgutil PO-token + deno JS-runtime all + loaded fine — the gate is pure IP reputation. Operator chose proxy-via-nh3-dev → durable + dante proxy → proven. The egress proxy is a reusable fleet lever for any datacenter-IP-gated + service. + _Archived 2026-06-13._ + ## Tried and abandoned (archived) - `[2026-04-30]` task-board workflow with @@ -681,3 +709,30 @@ _Entries moved out of persistent-memory.md to keep the active file scannable. Re (4) Forcing `static_chunk_size>0` didn't stabilize it. True sub-second frame-streaming needs deep attention surgery with quality risk — not worth it. → adaptive-chunking instead. _Archived 2026-06-11._ + +- `[2026-06-03]` **`uv pip install .` fails on SmoothKen/knn-svc** (and similar script-repos) + — it's analysis scripts + a poetry pyproject, no buildable package (setuptools + package-discovery error). Install the pyproject deps directly, don't build the "package". + _Archived 2026-06-13._ + +- `[2026-06-03]` **Backgrounding `althing-cli monitor` with an inline shell `&` (instead of + the Bash-tool `run_in_background`) orphans it** — it survives the shell exit, holds the + per-handle flock UNTRACKED (won't notify the session), and `stop-monitor` doesn't detect it. + Fix: find + kill the orphan PID (verify cwd=this repo / handle first — nh3-dev is shared, other + agents' monitors run there too), then re-arm via run_in_background. Always re-arm tracked. + _Archived 2026-06-13._ + +- `[2026-06-03]` **torch-2.12 venvs need `uv pip install torchcodec`** — torchaudio 2.12 + defaults to the TorchCodec backend for `.load`; without it, real audio I/O throws "TorchCodec + is required" — and it ONLY surfaces at actual conversion, NOT at import/model-load. Lesson: + validate real I/O, not just import, when provisioning ML engine envs. (seed-vc on torch 2.4 + uses the legacy backend, exempt.) + _Archived 2026-06-13._ + +- `[2026-06-03]` **gitea webhook to a private IP is denied by `webhook.ALLOWED_HOST_LIST`** + (anti-SSRF; default `external` blocks private/loopback). Symptom: delivery shows + `dial tcp ...: webhook can only call allowed HTTP servers`. Fix = APPEND the target net to + ALLOWED_HOST_LIST in gitea's app.ini (keep `external`; scope tight, never `*`/`private`) + + restart gitea (act_runner job containers survive a restart). gitea runs as a container on + ana-docker (`gitea_gitea_data` volume, `/data/gitea/conf/app.ini`). + _Archived 2026-06-13._ diff --git a/persistent-memory.md b/persistent-memory.md index d48ac0e..d3186a0 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -1,6 +1,6 @@ # Persistent memory — eshpfi-management -_Last updated: 2026-06-11_ +_Last updated: 2026-06-13_ ## Repo purpose @@ -91,43 +91,100 @@ no longer deployed sidecars here. See Recent decisions.) ## Current state / in-flight -_As of 2026-06-11:_ +_As of 2026-06-13:_ -- **Mac Pro migration is the big open project** — `migration-plan.md` (repo root) frames a - WORKSTATION-ONLY migration of nh3-dev's dev env to an M2 Ultra Mac Pro Rack, racked in NH3 on - the 10.100 subnet. nh3-dev wears two hats; only **Hat 1** (the ~40 repos, all of `~/.claude`, - dotfiles, toolchain) moves. **Hat 2** fleet sidecars (egress SOCKS5 proxy, mead-hall, ttyd - seats, `althing-forseti`) STAY on the Linux VM. **Phase 0 is the only time-sensitive step**: - push the pushable repos + confirm which of ~6 no-upstream repos are local-only. Cutover = - `rsync` the working trees, **NOT re-clone** (re-clone would lose unsaved work in ~15 repos). - macOS gotcha pre-flagged: `/home/lkraven` → `/Users/lkraven` repath. Everything else waits on - the hardware landing in the rack. +- **ana-ml2 went Ada → Blackwell** (dual NVIDIA RTX PRO 6000 Blackwell Max-Q, 96 GB each, cc 12.0 / + sm_120 — was dual RTX 6000 Ada 48 GB / cc 8.9, confirmed live via `nvidia-smi`). GPU layout now: + **GPU 0** held free for large-model hot-loads (llama-swap pinned there, `edf0f91`); **GPU 1** is the + steady-tenant card — granite (131k ctx) + qwen3.5-vl (65k) + the embed/rerank/reward trio, ~3.5 GB + free after the rebalance. CLAUDE.md's server table + the granite/qwen "Ada (cc 8.9)" compose comments + are STALE — doc-fix offered, **pending operator go-ahead** (/snapshot doesn't auto-edit CLAUDE.md). -- **Two commits/branches committed-not-pushed** (operator's push call): `95b2701` (GLM thinking - config) on main; and the R17 v2 results branch `r17-v2-characterization` on the irv-ml1 - checkout (push HELD — see decisions). +- **R16 "vmoan" TTS LoRA POC — awaiting the operator's ear** (Brokkr's designated gate). Two auditions + on NFS: `/mnt/smithy/scratch/r16-vmoan-pilot/vmoan_audition_v1.wav` (42 s, run-on) vs `_v2.wav` (10 s, + tight). v2 fixed the run-on; Brokkr holds the pilot verdict on the operator's listen. Grab: + `scp irv-ml1:/mnt/smithy/scratch/r16-vmoan-pilot/vmoan_audition_v{1,2}.wav .`. LOCAL-ONLY corpus + (soundgasm-derived) — distribution barred; never echo transcripts to the bus. -- **R17 v2 corpus characterization DONE on irv-ml1** (brokkr, ~8.5h): 119 clips → **18 MULTI / - 101 ELIGIBLE** (Stage-0 drop list) + 19936 candidates + ASR speech + blend verdicts. Outputs - on the local branch; **push HELD** — no-push rule AND the `speech.json` are ASR transcripts of +- **R17 v2 corpus characterization still on the local irv-ml1 branch, push HELD** (`r17-v2-character + ization`, 18 MULTI / 101 ELIGIBLE + 19936 candidates + `speech.json` ASR). No-push rule AND ASR of the intimate-audio batch (content-exposure call). Awaiting brokkr-collect or explicit push approval. -- **pi coding agent + GLM 5.1 harness live on nh3-dev** — `glm` launcher runs pi against `glm-5.1` - (thinking-off) via the LiteLLM gateway; `glm-5.1-reasoning` for opt-in thinking. +- **Mac Pro migration is still the big open project** — `migration-plan.md` (repo root): WORKSTATION- + ONLY move of nh3-dev's Hat-1 dev env (~40 repos, all of `~/.claude`, dotfiles, toolchain) to an M2 + Ultra Mac Pro Rack on the 10.100 subnet. Hat-2 fleet sidecars (egress SOCKS5, mead-hall, ttyd seats, + `althing-forseti`) STAY on the Linux VM. **Phase 0** (push pushables + confirm the ~6 local-only repos) + is the only time-sensitive step. Cutover = `rsync` working trees, NOT re-clone (re-clone loses unsaved + work in ~15 repos). macOS `/home/lkraven`→`/Users/lkraven` repath. Everything else waits on hardware. -- **Worldtree config-propagation lane is mature + humming** — pre-merge delta-ping → sync-on-merge. - All of v0.33.x→v0.34.1 reconciled to the demo+personal bind-mounts; demo/personal both on v0.34.x, - pinned stays pre-cutover (full migration at its next re-image). +- **sglang-vs-vLLM bench stack staged but parked** (`5f049cb`) — the originating question (SGLang + RadixAttention caching) resolved without it: vLLM v1 already defaults prefix-caching ON, now pinned + explicit on granite+qwen. The NVFP4 chase is abandoned. Bench stack stays staged if ever revisited; + the MAX_JOBS-on-shared-prod foot-gun (see Tried) caps any future from-source build on ana-ml2. -- **Disclosed-keys hygiene queue** carries forward (rotate at convenience): HF token, the wt-personal - keys, Gitea runner reg token, MINIFLUX_PASSWORD. `ZAI_API_KEY` (z.ai MCP web-tools key) now also - lives in `~/.config/secrets/env.sh` (shared from Worldtree env for the Brokkr panel). +- **pi + GLM 5.1 harness live on nh3-dev** — `glm` launcher runs pi against `glm-5.1` (thinking-off) + via the LiteLLM gateway; `glm-5.1-reasoning` for opt-in thinking. -- **Still open from prior:** clean legacy `news-digest` on ana-docker; pin llama-swap to GPU 0; the - `docker push 60s ceiling` mystery uninstrumented. +- **Worldtree config-propagation lane is mature + humming** — pre-merge delta-ping → sync-on-merge to + the demo+personal bind-mounts; pinned stays pre-cutover (full migration at its next re-image). + +- **Disclosed-keys hygiene queue** (rotate at convenience): HF token, the wt-personal keys, Gitea + runner reg token, MINIFLUX_PASSWORD, ZAI_API_KEY. (The shared all-agents LiteLLM key is intentional, + not hygiene-debt — see decisions; rotatable via infra-ops only if it leaks.) + +- **Still open from prior:** clean legacy `news-digest` on ana-docker; the `docker push 60s ceiling` + mystery uninstrumented. (llama-swap GPU-0 pin — DONE this session, `edf0f91`.) ## Recent decisions +- `[2026-06-13]` **ana-ml2 upgraded Ada → dual RTX PRO 6000 Blackwell Max-Q** (96 GB each, cc 12.0 / + sm_120; was dual RTX 6000 Ada 48 GB / cc 8.9 — confirmed live via `nvidia-smi`). Unlocks NVFP4 (FP4 + tensor cores) and doubles VRAM headroom. CLAUDE.md's server table + the granite/qwen compose + "Ada (cc 8.9)" comments are now stale; doc-fix **offered, pending operator go-ahead** (snapshot doesn't + auto-edit CLAUDE.md). Tracking surface: this snapshot + commits `19a07b9`/`1e2a3a1` ("Blackwell 96GB"). + +- `[2026-06-13]` **NVFP4-W4A4 is infeasible for Granite — FP8 stays the Granite-on-Blackwell format.** + W4A4 (4-bit weights + activations) collapses at 30k context, proven **producer-independent** (modelopt + AND llm-compressor both clean-NONE from the same BF16 base + wikitext-2k calib). No 4-bit wins both + axes: W4A4 = quality collapse; W4A16-NVFP4/AWQ = weight-only dequant → bf16 (no FP4-core speedup), so + the tempting ~40% uplift doesn't materialize without quality loss. **30B retired** (not enough quality + for the VRAM/perf hit for our use case). (auto-memory `reference_nvfp4_w4a4_granite_infeasible`) + +- `[2026-06-13]` **Qwen3.5-9B VL (FP8) deployed on ana-ml2 GPU 1** — `qwen35-vl` stack, :8007, gateway + alias `qwen3.5-9b-fp8`. Clean FP8 (no NVFP4 for vision). **Pinned nightly digest, not `:latest`**: the + stable release quantizes the Qwen3.5-VL *vision tower* under `--quantization fp8` → garbage vision (LM + fine, "sees" noise); the nightly correctly excludes the vision tower. Re-pin to `:latest` + drop the + pin once that exclusion lands stable. (`2e3dcc2`) + +- `[2026-06-13]` **comfyui 325 G model tree migrated worktank → `/storetank/arbo`** (worktank 97% → 26%). + `arbo` is the consuming app (ComfyUI-backed prompt/image pipeline); models live under its name on the + bigger pool. Overlay bind-mount via `COMFYUI_MODELS_DIR` in the comfyui compose; inventory at + `docs/arbo-comfyui-model-catalog.md` (retain-large-portion-for-future-use decision). (`38186be`) + +- `[2026-06-13]` **GPU layout settled on the Blackwell box.** GPU 0 held free for large-model hot-loads + (llama-swap pinned there, `edf0f91`); GPU 1 is the steady-tenant card — granite 131k ctx (was 51k), + qwen 65k, embed/rerank/reward trio, ~3.5 GB free after rebalance (`1e2a3a1`, `19a07b9`; trio re-floored + for 96 GB, 20×-parallel-stable). embed/rerank left at floor — long docs are chunked BEFORE embedding, + so a longer embed ctx buys nothing. PagedAttention note: max-model-len is a ceiling, not a reservation, + so small requests aren't blocked by the big ceiling; concurrency = KV-pool-tokens / actual-request-size. + +- `[2026-06-13]` **Prefix caching pinned explicit on granite + qwen** — benched ~6.5× faster TTFT (45 ms + vs 292 ms) on a shared ~4.5k-token summarizer template; soft/evictable KV, neutral when prefixes don't + repeat. vLLM v1 `:latest` defaults it ON (granite) but the qwen nightly defaults it OFF — pin both so a + version flip can't silently disable it. (`a9a2be7`) + +- `[2026-06-13]` **granite-4.1-8b listed as the always-available summarizer/classifier endpoint + a + shared all-agents key minted** (operator-directed). Added to the global `~/.claude/CLAUDE.md` Global- + tools section; key alias `all-agents-local`, scoped to the FREE local models only (granite + qwen- + vision + embed/rerank, NOT the paid GLM), internal-gateway-only, rotatable. The standing "reach for + this before spending premium tokens on low-caliber high-volume work" lever. (auto-memory + `reference_litellm_gateway`) + +- `[2026-06-13]` **arbo engine + frontend stack stood up** (ADR-0001) — irv-ml1 co-located inference + engine (`ee57e69`), python-based healthcheck (the slim image ships no curl/wget, `bdb3312`), frontend + ro-mounted from the v0.11.2 checkout (`922e8ad`, ADR-0001 D2). arbo = the ComfyUI-consuming app whose + models now live at `/storetank/arbo`. + - `[2026-06-11]` **GLM thinking inverted at the LiteLLM gateway** (operator call): `glm-5.1` defaults thinking-OFF; new `glm-5.1-reasoning` alias = same z.ai upstream with thinking ON (opt-in). Mechanism: `litellm_params.extra_body:{thinking:{type:disabled}}` — LiteLLM `drop_params` STRIPS a @@ -172,12 +229,10 @@ _As of 2026-06-11:_ - `[2026-06-05]` **Granite 4.1 8B FP8 replaced phi4-mini as the production summarizer** (supersedes the 2026-06-04 phi4 decision below). Beat phi4 on precision in brokkr's R15 P03. **Staying FP8, not - Q4/AWQ** — primary workload (agent memory + summarization) is high-concurrency, where FP8-on-Ada - scales ~linearly (profiled 2010 tok/s @ C=32; single-stream 67.5 is batch-1 GEMV physics, not a - config bug — placement/kernel/contention all ruled out). vLLM `vllm-granite` :8004 GPU 1, official - IBM compressed-tensors FP8, CUDA graphs. **GPU-1 retune** (trio utils 0.2/0.2/0.3→0.07/0.07/0.18, - granite 0.36) freed ~10 GB → CUDA graphs + a Granite-text-LoRA hedge. nevermore repointed. (`34a43a0`, - auto-memory `reference_ana_ml2_vllm_granite`) + Q4/AWQ** — primary workload (agent memory + summarization) is high-concurrency, where FP8 scales + ~linearly (profiled 2010 tok/s @ C=32; single-stream 67.5 is batch-1 GEMV physics, not a config bug). + vLLM `vllm-granite` :8004 GPU 1, official IBM compressed-tensors FP8, CUDA graphs. (Then on Ada cc 8.9; + the box has since gone Blackwell — see 2026-06-13.) (`34a43a0`, auto-memory `reference_ana_ml2_vllm_granite`) - `[2026-06-05]` **Langfuse v3 stood up on ana-docker (:3001) as the gateway trace UI**; LiteLLM `success_callback:[langfuse]` live (project `gateway`). Pretty prompt/completion/reasoning traces + @@ -225,34 +280,38 @@ _As of 2026-06-11:_ - `[2026-06-04]` **`brokkr` user (no-sudo) on irv-ml1; R14/R15/R16 substrate moved to /home/brokkr.** Persistent box services there need SYSTEM systemd units (see Tried). -- `[2026-06-03]` **yt-voice-clipper bot-gate fix = route yt-dlp through NH3 residential - egress, NOT cookies/PO-token.** YouTube hard-flags the Irvine colo IP (LOGIN_REQUIRED on a - public video even with no cookies). Cookies + the bgutil PO-token + deno JS-runtime all - loaded fine — the gate is pure IP reputation. Operator chose proxy-via-nh3-dev → durable - dante proxy → proven. The egress proxy is a reusable fleet lever for any datacenter-IP-gated - service. - -- `[2026-06-03]` **yt-voice-clipper push-to-deploy via gitea webhook** (operator-directed, - after 6 manual rebuilds in ~40 min). Webhook (not poll) — gitea CAN reach the WG IP per the - operator. The proxy env + Homepage labels live in the **host-specific override** (untracked - → survive the auto-deploy's `git reset --hard`), NOT yt-voice-clipper-dev's image. Runbook - `d4f180d`. - -- `[2026-06-03]` **R14 scope = (a) provision-only.** infra-ops provides box + CUDA env + - engines + weights + NFS; brokkr/dev wires `arms.py` + runs — keeps infra-ops OFF the - VIVAE-processing path (VIVAE = Variably Intense Vocalizations of Affect/Emotion, CHARTER §4 - highest-liability; operator authorized R&D-eval-only, quarantined). Box = irv-ml1 (A6000 - free; ana-ml2 GPU-saturated). Per-engine venvs (divergent torch stacks); A6000 = `cuda:0` - NATIVE (≠ docker `=1`). - -- `[2026-06-03]` **Declined worldtree v0.30.4 staging deploy** — that's worldtree-team's - CI/CD lane (a developer `staging/vX.Y.Z` git-tag promote), not infra-ops. They self-corrected - to the same conclusion independently. - -_40 older entries archived to archival-memory.md._ +_44 older entries archived to archival-memory.md._ ## Tried and abandoned +- `[2026-06-13]` **Heavy from-source compile (`MAX_JOBS=128` sglang fork build) on the shared PROD GPU + box PINS it** — load hit 187, prod vLLM services restarted, killed an in-flight quant. ana-ml2 hosts + live inference; never run a big build there at full parallelism. Cap `MAX_JOBS≤32`, build off-box, or + cgroup-constrain. (Operator ran the kill; infra-ops NOPASSWD-sudo confirmed working on ana-ml2 — retry + infra-ops on an ssh-255 before concluding "no access.") + +- `[2026-06-13]` **`--quantization fp8` on a VL model can quantize the VISION TOWER → garbage vision** + (Qwen3.5-VL on the stable vLLM: gray-grid output; the LM answers text fine, so it "looks" healthy + until you actually feed it an image). The nightly excludes the vision tower. Lesson: validate the + VISION path on a quantized VLM, not just text — and pin the engine digest that has the exclusion. + +- `[2026-06-13]` **vLLM's `--gpu-memory-utilization` is checked against FREE VRAM at startup + (`free >= util*total`), not total** — on a shared card, growing one service before trimming a + co-tenant OOMs ("free 48.56 < desired 80.72" at util 0.85 on a half-occupied 96 GB card). Start-order + matters: trim the shrinking service FIRST, then grow the other. Size to the FREE budget, not the total. + +- `[2026-06-13]` **The `vllm/vllm-openai` entrypoint is already `["vllm","serve"]`** — the compose + `command:` supplies the model as the first POSITIONAL arg + flags; a second `serve` (or `--model X`) + yields "unrecognized arguments". Same-class gotchas this session: `tee` masks the real exit code (use + a `>` redirect to keep rc); HF `datasets` rejects bare `wikitext` (needs `Salesforce/wikitext`). + +- `[2026-06-13]` **Chatterbox-Turbo LoRA finetune: the repo's `setup.py` loads the WRONG tokenizer** — + `merge_and_save_turbo_tokenizer()` pulls gpt2-medium + a grapheme merge file (len mismatch) instead of + the chatterbox-turbo GPT2 tokenizer (vocab.json+merges.txt, len 50276). Fix = override with the correct + tokenizer + delete the grapheme `tokenizer.json`; `[vmoan]` token → new_vocab_size 50277 (1-row resize), + lora_r 64 / alpha 128, modules_to_save=[text_emb,text_head]. Also: a unique-stem corpus collision (53 + rows, 32 wavs) needs `{index}_{stem}` IDs. (irv-ml1 `~/r16-vmoan-harness`) + - `[2026-06-11]` **A completion-poll `while pgrep -f ` SELF-MATCHES its own remote shell argv** — the poll's command line contains the script name, so its own `pgrep -f` always finds itself → the loop never exits, the poll never fires. Use a match pattern ABSENT from the poll command (pgrep @@ -261,7 +320,7 @@ _40 older entries archived to archival-memory.md._ - `[2026-06-08]` **Demucs `uv pip install demucs` pulls torch 2.12/torchaudio 2.11 → `ta.save()` requires torchcodec → dies AFTER separating (0 stems written, rc=1).** Same class as the torch-2.12 - torchcodec foot-gun below. Fix = pin `torch==torchaudio==2.4.1` (pre-torchcodec save backend) + + torchcodec foot-gun. Fix = pin `torch==torchaudio==2.4.1` (pre-torchcodec save backend) + `UV_LINK_MODE=copy` for the EPERM-hardlink quirk. Lesson restated: validate the SAVE path, not just import + GPU inference, on a bleeding-edge torch. @@ -317,27 +376,4 @@ _40 older entries archived to archival-memory.md._ widget targets (retired ESH firewall IPs) hanging the node event loop into `exit_mmap`, needing a host reboot. Check homepage's siteMonitors against retired hosts. (`incident_esh_docker_nfs_boot_race`) -- `[2026-06-03]` **gitea webhook to a private IP is denied by `webhook.ALLOWED_HOST_LIST`** - (anti-SSRF; default `external` blocks private/loopback). Symptom: delivery shows - `dial tcp ...: webhook can only call allowed HTTP servers`. Fix = APPEND the target net to - ALLOWED_HOST_LIST in gitea's app.ini (keep `external`; scope tight, never `*`/`private`) + - restart gitea (act_runner job containers survive a restart). gitea runs as a container on - ana-docker (`gitea_gitea_data` volume, `/data/gitea/conf/app.ini`). - -- `[2026-06-03]` **torch-2.12 venvs need `uv pip install torchcodec`** — torchaudio 2.12 - defaults to the TorchCodec backend for `.load`; without it, real audio I/O throws "TorchCodec - is required" — and it ONLY surfaces at actual conversion, NOT at import/model-load. Lesson: - validate real I/O, not just import, when provisioning ML engine envs. (seed-vc on torch 2.4 - uses the legacy backend, exempt.) - -- `[2026-06-03]` **Backgrounding `althing-cli monitor` with an inline shell `&` (instead of - the Bash-tool `run_in_background`) orphans it** — it survives the shell exit, holds the - per-handle flock UNTRACKED (won't notify the session), and `stop-monitor` doesn't detect it. - Fix: find + kill the orphan PID (verify cwd=this repo / handle first — nh3-dev is shared, other - agents' monitors run there too), then re-arm via run_in_background. Always re-arm tracked. - -- `[2026-06-03]` **`uv pip install .` fails on SmoothKen/knn-svc** (and similar script-repos) - — it's analysis scripts + a poetry pyproject, no buildable package (setuptools - package-discovery error). Install the pyproject deps directly, don't build the "package". - -_44 older entries archived to archival-memory.md._ +_48 older entries archived to archival-memory.md._