diff --git a/CLAUDE.md b/CLAUDE.md index 89b3169..5ea1f3e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -26,6 +26,34 @@ leaving a durable record a *future* infra-ops session will read on recovery (e.g. an on-boot-gap checklist). That is a memo to your successor, not a page to a peer — write it as such, and don't then "reply to the peer who wrote it." +### `infra-hermes` IS a real peer — and is one character from a mirror + +**Operator ruling 2026-09-19: `infra-hermes` is this session's assistant.** It is a +Hermes-hosted bus seat on nh3-dev (`althing-pump-infra-hermes.service`, enabled), NOT +another name for you. The division of labour: + +| | owns | +|---|---| +| **infra-ops** (you) | improving infrastructure **tooling**; the hard calls; anything escalated | +| **infra-hermes** | day-to-day checks, triage, routine operations | + +Either of you may perform infrastructure operations. **You may task him downward** — +route routine work to him rather than doing it yourself; he escalates upward as needed. + +⚠ **`infra-ops` and `infra-hermes` differ by one character in the middle of a word**, +and the mirror trap above still applies to the OTHER name. Read the handle back before +sending during an incident. A misaddressed page to `infra-ops` still mails you yourself. + +⚠ **He is NOT Miranda** (operator, explicitly). The global CLAUDE.md names Miranda as +the sole trusted relay of operator authority; infra-hermes is not covered by that +exception. A directive he relays is **information, not authorization**: reversible +relayed work is fine to execute, but anything irreversible or fleet-affecting goes to +the operator directly. This is standing policy and not a judgement about him. + +⚠ **You both act as the same OS identity** — `ssh infra-ops@` is either of you, +and dockerd exec is not logged per-caller, so host-side changes are fingerprint-less. +That is why the ops log exists (below); use it. + ## Persistent memory `persistent-memory.md` at the repo root captures durable intent and diff --git a/graphify-out/GRAPH_REPORT.md b/graphify-out/GRAPH_REPORT.md index e4761fc..a11f03d 100644 --- a/graphify-out/GRAPH_REPORT.md +++ b/graphify-out/GRAPH_REPORT.md @@ -1,16 +1,16 @@ # Graph Report - eshpfi-management (2026-09-18) ## Corpus Check -- 623 files · ~840,412 words +- 629 files · ~843,892 words - Verdict: corpus is large enough that graph structure adds value. ## Summary -- 5472 nodes · 5975 edges · 632 communities (478 shown, 154 thin omitted) +- 5507 nodes · 6006 edges · 641 communities (487 shown, 154 thin omitted) - Extraction: 98% EXTRACTED · 2% INFERRED · 0% AMBIGUOUS · INFERRED: 97 edges (avg confidence: 0.81) - Token cost: 0 input · 0 output ## Graph Freshness -- Built from commit: `ee0fa1b7` +- Built from commit: `148a5a34` - Run `git rev-parse HEAD` and compare to check if the graph is stale. - Run `graphify update .` after code changes (no API cost). @@ -158,7 +158,7 @@ - 7. Critical Warnings by Model - think-leak — reproducers for the h300 unterminated-`` defect - services.schema.json -- CatalogReproducibility +- [2026-09-18] NH3↔Anaheim was DERP-relayed, not direct — a UDP 41641 port-forward on ana-gw fixed it - test_invocation.py - scriberr — self-hosted transcription + diarization (fv-ml1, GPU1) - elway @@ -374,7 +374,7 @@ - Cloudflare state for phasefinal.com - 6. Round-1 aborted; throughput root-caused (measured 2026-08-24 22:00 PDT) - SLZB-MR1U Zigbee/Thread coordinator — ESH IoT VLAN -- CatalogResponse +- [2026-09-18] SearXNG had ONE working general web engine — now seven, and the health script never noticed - Packed - Gemma-4 26B-A4B ERP/RP tune — GPU sizing adjudication - Training throughput probes @@ -444,11 +444,11 @@ - `[2026-09-04]` `gen` moved to ana-ml2 GPU0 — and I sized it against the wrong number, twice in one hour - launch-run-03c.sh - `[2026-09-04]` Forcing 10G on the ESH-Media DAC FAILED progressively — and I called the plateau too early -- .submit +- gpu.py - `[2026-09-04]` esh-nas SMB account for the AudioGridder box — and the NAS is effectively open to the whole LAN - remote-ssh - verify_corpus.py -- max +- [2026-09-18] althing search returned ZERO for every hyphenated query — silently, for everyone - booth - tok_repro.py - wrap_verbatim_html @@ -535,7 +535,7 @@ - 2026-09-08-yt-voice-clipper-back-up.md - 2026-09-09-erp-run-7-failed-the-safety.md - svos_miranda Hermes plugin — validation pass (2026-09-15) -- step +- [2026-09-18] FleetTools — one capability index, autoloaded by three agent families - Claude-Code-only tooling - dockge - fablefusion-charrp-probe — THROWAWAY evaluation seat (fv-ml1 GPU1) @@ -552,7 +552,7 @@ - irv-ml1 dead-address sweep — EXECUTED 2026-09-15 (operator-directed) - fv-mesh-watchdog.sh - grok-token-broker -- mime +- CatalogService - kb - 2026-09-10-an-author-voice-adapter-transfers-subject.md - 2026-09-10-babybronte-h02-adapter-the-voice-transferred.md @@ -578,7 +578,7 @@ - breeze-tts sizing and the fv-ml1 GPU 0 placement question (2026-09-15) - 2026-09-15-client-abandon-cancellation-boundary.md - gender_by_proximity.py -- mime_from_field +- accepted_types - verify_quant.py - [2026-08-24] ESH DNS fixed at the IPv6 layer, and the naming scheme went live - booth/app.py @@ -587,17 +587,17 @@ - description - score_beats.py - Direct inference seats + the Asset Engine control plane -- optional +- catalog_version - LiteLLM gateway — the fleet's OpenAI-compatible front door - Fleet SSH — the infra-ops identity and host map - build_beat_fixture.py - The Booth — ephemeral media board + operator decisions - options -- min -- $ref +- category +- properties - pin_known_gender.py - Speech — TTS and ASR endpoints -- output_field +- content_type - repair_corpus_bronte.py - 2026-09-16-grok-broker-shelved.md - 2026-09-17-lv-bronte-gate.md @@ -610,6 +610,8 @@ - voice-prep.py - althing — inter-agent message bus - sanitize_tts_text +- fields +- generated_at - apply_gender_overrides.py - rename.py - elway — SSH playbook runner (CHANGING things) @@ -620,6 +622,13 @@ - henge.md - playwright.md - _cli +- license_warning +- method +- streamable +- estimated_latency +- host +- lifecycle +- version ## God Nodes (most connected - your core abstractions) 1. ``[2026-09-01]` althing 3.2.0 deployed on nh3-dev — pane routes, and `uv tool install .` lies` - 36 edges @@ -642,13 +651,13 @@ stacks/zonos/adapter/server.py → services/coldfusion-abliteration/kl_divergence.py - `build_gallery()` --calls--> `is_ask_file()` [INFERRED] services/booth/booth/app.py → services/booth/booth/asks.py -- `list_booths()` --calls--> `is_ask_file()` [INFERRED] +- `build_gallery()` --calls--> `is_answer_file()` [INFERRED] services/booth/booth/app.py → services/booth/booth/asks.py ## Import Cycles - None detected. -## Communities (632 total, 154 thin omitted) +## Communities (641 total, 154 thin omitted) ### Community 0 - "quant_modelopt.py" Cohesion: 0.53 @@ -671,8 +680,8 @@ Cohesion: 0.07 Nodes (29): ana-docker (VM 10.250.50.70), ana-filebot (CT 112 on pfi-pve) — file automation, ana-nas (CT 109 on pfi-pve, 10.250.50.50), ana-wg (CT 113 on pfi-pve) — WireGuard VPN, Cross-cutting: what to check FIRST for ambiguous outages, esh-docker-vm + vm-esh-nas, esh-pve + esh-pve-nas (10.0.250.35 + 10.0.50.55), Fleet disaster-recovery runbook (+21 more) ### Community 5 - "properties" -Cohesion: 0.15 -Nodes (13): anyOf, default, title, anyOf, default, title, properties, audio_field (+5 more) +Cohesion: 0.08 +Nodes (25): anyOf, default, title, anyOf, default, title, properties, anyOf (+17 more) ### Community 6 - "1. Locked architectural decisions" Cohesion: 0.07 @@ -707,8 +716,8 @@ Cohesion: 0.11 Nodes (17): 1. Create rest-server-ana htpasswd entry, 2. Install secrets on esh-vm-db, 3. Initialize the repo, 4. Install prerequisites (restic, resticprofile, mongosh client), 5. Deploy profile + hook, 6. Schedule + verify, Deploy (one-time), Full host config (+9 more) ### Community 14 - "properties" -Cohesion: 0.18 -Nodes (11): anyOf, default, title, properties, default, title, title, type (+3 more) +Cohesion: 0.10 +Nodes (20): properties, default, title, anyOf, default, title, anyOf, default (+12 more) ### Community 15 - "Deploy (one-time setup)" Cohesion: 0.12 @@ -964,7 +973,7 @@ Nodes (23): Exception, bw(), BwError, cmd_backfill(), cmd_get(), cmd_list(), cmd ### Community 78 - "test_booth.py" Cohesion: 0.06 -Nodes (35): generate_pickup_id(), is_kept(), True if this booth carries the keep sentinel and must never be swept., A human-readable id like '4-wombat' or 'star-84'. `exists(name)->bool` gates…, Releasing something already released is a no-op, not a 500., Reversible: the release step must not be a one-way door., Both mutating routes must use resolve_booth, not raw path joining. A name…, test_booth_autogallery_renders_media() (+27 more) +Nodes (33): is_kept(), True if this booth carries the keep sentinel and must never be swept., Releasing something already released is a no-op, not a 500., Reversible: the release step must not be a one-way door., Both mutating routes must use resolve_booth, not raw path joining. A name…, It is a dotfile, so it must not inflate the item count or become a tile., test_booth_autogallery_renders_media(), test_delete_api_wipes() (+25 more) ### Community 79 - "pfi-tacticalrmm" Cohesion: 0.25 @@ -999,8 +1008,8 @@ Cohesion: 0.25 Nodes (7): API, Deploy, Gotchas, Switching to the 7B variant, VibeVoice, Voices, Why this stack exists ### Community 87 - "properties" -Cohesion: 0.05 -Nodes (42): properties, enum, title, type, default, title, type, title (+34 more) +Cohesion: 0.15 +Nodes (13): properties, title, type, $ref, title, type, endpoint, model (+5 more) ### Community 88 - "Dia / Dia2" Cohesion: 0.29 @@ -1083,8 +1092,8 @@ Cohesion: 0.29 Nodes (6): Adapter request (example), Deploy, Notes, Two surfaces: Gradio eval (8199) + REST adapter (8203), Why this stack exists, Zonos ### Community 108 - "test_asks.py" -Cohesion: 0.07 -Nodes (66): parametrize, ask_stem(), AskError, _blank(), _bool(), _clean_notes(), is_ask_file(), list_asks() (+58 more) +Cohesion: 0.06 +Nodes (68): parametrize, list_booths(), ask_stem(), AskError, _blank(), _bool(), _clean_notes(), is_answer_file() (+60 more) ### Community 109 - "Fleet bootstrap — read this first" Cohesion: 0.40 @@ -1199,8 +1208,8 @@ Cohesion: 0.19 Nodes (17): ClockFn, GenerateFn, ChunkConfig, _ema(), _est_gen_time(), plan_chunk(), protect_first_audio(), Split into sentence units, preserving punctuation. Whitespace-collapsed. (+9 more) ### Community 138 - "build_gallery" -Cohesion: 0.09 -Nodes (24): booth_image_names(), build_gallery(), classify(), doc_kind(), list_booths(), markdown' | 'text' | None — a booth file viewable as a readable page., (rendered, is_html). Markdown → HTML (fenced code, tables, sane lists); plain…, Image files in a booth, in gallery (sorted-rel) order — for viewer prev/next. (+16 more) +Cohesion: 0.13 +Nodes (15): build_gallery(), doc_kind(), markdown' | 'text' | None — a booth file viewable as a readable page., (rendered, is_html). Markdown → HTML (fenced code, tables, sane lists); plain…, Files in a booth as render items, with caption sidecars folded in. A…, render_doc(), test_gallery_hides_ask_and_answer_files(), test_build_gallery_folds_caption_sidecars() (+7 more) ### Community 139 - "lv-mccarthy — corpus → gate → pairs" Cohesion: 0.33 @@ -1215,12 +1224,12 @@ Cohesion: 0.22 Nodes (8): Did our abliteration cause it? No — the base did (~83% / ~17%), Resolution — Cold-Fusion abandoned, seat rolled back to `heresy` (2026-08-21), Scripts, The fix, The trigger is TEMPERATURE, not presence_penalty, think-leak — reproducers for the h300 unterminated-`` defect, Using these on any future seat, What actually happens ### Community 142 - "services.schema.json" -Cohesion: 0.10 -Nodes (19): additionalProperties, default, title, type, description, anyOf, default, title (+11 more) +Cohesion: 0.25 +Nodes (7): additionalProperties, description, $id, required, $schema, title, type -### Community 143 - "CatalogReproducibility" -Cohesion: 0.12 -Nodes (17): additionalProperties, properties, required, title, type, CatalogReproducibility, title, type (+9 more) +### Community 143 - "[2026-09-18] NH3↔Anaheim was DERP-relayed, not direct — a UDP 41641 port-forward on ana-gw fixed it" +Cohesion: 0.14 +Nodes (12): [2026-09-18] `.internal` DNS was failing ~10% of lookups — two causes, both fleet-wide, End-to-end, the original symptom, Fault 1 — a PUBLIC resolver was the fallback for a PRIVATE zone, Fault 2 — THE ROOT CAUSE: AdGuard rate-limiting, identical on all three, What this says about monitoring, [2026-09-18] NH3↔Anaheim was DERP-relayed, not direct — a UDP 41641 port-forward on ana-gw fixed it, How it surfaced, and how it did NOT, Measured before → after (+4 more) ### Community 144 - "test_invocation.py" Cohesion: 0.25 @@ -1339,8 +1348,8 @@ Cohesion: 0.15 Nodes (12): 1. The MTP head is abliterated *in-band*, 2. The vision tower is preserved byte-identical, Abliteration recipe — Qwen3.8-27B (MTP-aware, vision-preserving), How this maps onto our pipeline, Measured behaviour (their numbers, for reference), Provenance, Refusal-direction selection, The attention-sink dimension — the one that bricks the model (+4 more) ### Community 190 - "JobManager" -Cohesion: 0.20 -Nodes (7): Popen, Path, Job, JobManager, _pid_alive(), Singleton owner of the one active job + retained history. Thread-safe., Phase 2 publish step: COPY the succeeded LoRA into ComfyUI's loras search path a +Cohesion: 0.17 +Nodes (8): Popen, Path, Job, JobManager, _pid_alive(), Singleton owner of the one active job + retained history. Thread-safe., Start a train. Raises InvalidTrainRequest (→422) or Busy (→409)., Phase 2 publish step: COPY the succeeded LoRA into ComfyUI's loras search path a ### Community 193 - "LoRA training worker" Cohesion: 0.22 @@ -1444,7 +1453,7 @@ Nodes (29): base, compliance_rate, len, refused, text, gens, A train leaves at 1 ### Community 227 - "_board" Cohesion: 0.09 -Nodes (38): order_for_display(), parse_link_entries(), Board rows for the web view: pinned first, then newest-first in each group.…, Rows of the standing link board, newest last (posting order). Non-matching…, _board(), _body(), The whole reason for the sidecar: links.md stays untouched by a pin, so it…, Callers that want the file-order view (the CLI) must not see a `pinned` key… (+30 more) +Nodes (34): order_for_display(), parse_link_entries(), Board rows for the web view: pinned first, then newest-first in each group.…, Rows of the standing link board, newest last (posting order). Non-matching…, _board(), _body(), The whole reason for the sidecar: links.md stays untouched by a pin, so it…, Callers that want the file-order view (the CLI) must not see a `pinned` key… (+26 more) ### Community 228 - "~26k" Cohesion: 0.20 @@ -1755,8 +1764,8 @@ Cohesion: 0.70 Nodes (4): ask(), classify(), load_prompts(), main() ### Community 327 - "worker/app.py" -Cohesion: 0.15 -Nodes (13): get_gpu_status(), FastAPI surface for the LoRA training worker (§4.1 API — arbo is the client). E, Static configuration for the LoRA training worker. Everything load-bearing is a, _bus_id_by_index(), _cmdline(), gpu_status(), _query_compute_apps(), _query_devices() (+5 more) +Cohesion: 0.14 +Nodes (10): RuntimeError, get_gpu_status(), post_train(), FastAPI surface for the LoRA training worker (§4.1 API — arbo is the client). E, TrainRequest, Static configuration for the LoRA training worker. Everything load-bearing is a, LoRA training worker — a host service (runs as llmuser on irv-ml1) that runs sd-, Busy (+2 more) ### Community 329 - "Beszel fleet monitoring" Cohesion: 0.33 @@ -1787,8 +1796,8 @@ Cohesion: 0.25 Nodes (7): built_at, corpus, normalisation, source, total_chapters, total_words, works ### Community 337 - "properties" -Cohesion: 0.11 -Nodes (18): additionalProperties, properties, required, title, type, CatalogAuditEntry, title, type (+10 more) +Cohesion: 0.06 +Nodes (35): additionalProperties, properties, required, title, type, additionalProperties, properties, required (+27 more) ### Community 338 - "archival-memory.md" Cohesion: 0.10 @@ -1926,9 +1935,9 @@ Nodes (10): 6.1 Where the step time goes, 6.1a ⚠ 8.6% MFU was an accounting ar Cohesion: 0.18 Nodes (10): ⚠ Home Assistant CANNOT resolve `.internal` — give it the IP, mDNS crosses the VLAN boundary — and that has one sharp edge, ⚠ port_overrides is a WHOLE-ARRAY PUT, Reachability is already open, and was proven before the move, Resolver: ha-dev declined the `dns:` fix, deliberately, Rollback, SLZB-MR1U Zigbee/Thread coordinator — ESH IoT VLAN, The MAC is a derived ESP32 interface address — stable, not fragile (+2 more) -### Community 380 - "CatalogResponse" -Cohesion: 0.33 -Nodes (6): additionalProperties, description, required, title, type, CatalogResponse +### Community 380 - "[2026-09-18] SearXNG had ONE working general web engine — now seven, and the health script never noticed" +Cohesion: 0.17 +Nodes (10): [2026-09-18] SearXNG's ESH SOCKS5 egress — one day live, reverted, and it fixed nothing, The transferable lesson, What was built, because it was built correctly, Why it was reverted, and why the STATED reason was wrong, [2026-09-18] SearXNG had ONE working general web engine — now seven, and the health script never noticed, ⚠ Still open: the health script cannot see any of this, The fix, The root cause, which is subtler than "engines were disabled" (+2 more) ### Community 381 - "Packed" Cohesion: 0.09 @@ -2079,8 +2088,8 @@ Cohesion: 0.48 Nodes (6): expand_light(), font_faces(), main(), Concatenate the vendored token files, refusing to build if one moved., sha256(), vendored_css() ### Community 431 - "remove_link_entry" -Cohesion: 0.20 -Nodes (17): link_entry_id(), Path, Standing link board: parse and prune the multi-writer link log. STDLIB ONLY, ON…, Atomic replace of the pins file. Caller must hold the board lock., Pinned entry ids for a board. Missing file → empty set. Lock-free: a set read…, Flip one row's pinned state. Returns the NEW state (True = now pinned). Held…, Stable short id for a board row. Content-addressed, so it survives concurrent…, Remove one row by content id. Returns the removed entry, or None. Held under an… (+9 more) +Cohesion: 0.15 +Nodes (21): link_entry_id(), Path, Standing link board: parse and prune the multi-writer link log. STDLIB ONLY, ON…, Atomic replace of the pins file. Caller must hold the board lock., Pinned entry ids for a board. Missing file → empty set. Lock-free: a set read…, Flip one row's pinned state. Returns the NEW state (True = now pinned). Held…, Stable short id for a board row. Content-addressed, so it survives concurrent…, Remove one row by content id. Returns the removed entry, or None. Held under an… (+13 more) ### Community 432 - "OPNsense API — vendored reference + field-shape notes" Cohesion: 0.18 @@ -2111,8 +2120,8 @@ Cohesion: 0.40 Nodes (3): `[2026-09-05]` A peer's "2.7x stack effect" was a coin flip — the operator caught it and the arithmetic is worth keeping, `[2026-09-04/05]` vLLM RUNS on sm_121 — and the blocker was `ninja` not being on PATH, not the silicon, Run 4 — the corpus arm ### Community 443 - "FastAPI" -Cohesion: 0.16 -Nodes (13): ACEStepInput, ACEStepOutput, generate_audio(), initialize_pipeline(), Generate music; respond with the audio bytes inline. Pre-2026-05-11 this re, ACEStepPipeline, BaseModel, FastAPI (+5 more) +Cohesion: 0.17 +Nodes (12): ACEStepInput, ACEStepOutput, generate_audio(), initialize_pipeline(), Generate music; respond with the audio bytes inline. Pre-2026-05-11 this re, ACEStepPipeline, BaseModel, FastAPI (+4 more) ### Community 445 - "`[2026-09-15]` ESPHome modernised for ha-dev; `kb` search tool for the personal Worldtree KB" Cohesion: 0.50 @@ -2127,8 +2136,8 @@ Cohesion: 0.07 Nodes (26): `[2026-09-10]` MeroMero — both quants landed; the v2 dense took five attempts, A4B — SUCCEEDED, MTP — checked on the SOURCES, not inferred, Published quants: none of them are our scheme, Still owed, The A4B reached a live seat while broken — and looked healthy doing it, The attempt-4 trap — the toolchain moved, the config was fine, The family, because I got it wrong first (+18 more) ### Community 450 - "Current state / in-flight" -Cohesion: 0.12 -Nodes (16): Current state / in-flight, DONE: lv-mccarthy D4 pairs built — NEXT is `train_pairs_lora.py`, ESH is on Verizon failover — Cityside Fiber failed twice on 2026-09-16, Older deferred set, unchanged, OPEN for the operator, OPEN LOOP — `ravenpen.com`, and a reply hamr-dev is waiting on, ⛔ PARKED — `lv-krakauer` (operator, 2026-09-17), henge id **82**, Persistent memory — eshpfi-management (+8 more) +Cohesion: 0.15 +Nodes (12): Current state / in-flight, ⚠ FIRST — lv-mccarthy's run outcome is UNVERIFIED by this session, OPEN LOOP — `ravenpen.com`, and a reply hamr-dev is waiting on, Other standing items, unchanged, Persistent memory — eshpfi-management, PLANNED (operator) — move `dragonfireacoustics.com` to Namecheap, DNS to Cloudflare, Recent decisions, Repo purpose (+4 more) ### Community 451 - "`[2026-09-04]` `gen` moved to ana-ml2 GPU0 — and I sized it against the wrong number, twice in one hour" Cohesion: 0.50 @@ -2138,17 +2147,17 @@ Nodes (3): `[2026-09-04]` `gen` moved to ana-ml2 GPU0 — and I sized it against Cohesion: 0.50 Nodes (3): `[2026-09-04]` Forcing 10G on the ESH-Media DAC FAILED progressively — and I called the plateau too early, ⚠ IT DEGRADED, AND MY "PLATEAU" READ WAS WRONG, Method notes worth keeping -### Community 454 - ".submit" -Cohesion: 0.33 -Nodes (4): RuntimeError, Busy, Start a train. Raises InvalidTrainRequest (→422) or Busy (→409)., A second train was dispatched while one is active → HTTP 409. +### Community 454 - "gpu.py" +Cohesion: 0.42 +Nodes (8): _bus_id_by_index(), _cmdline(), gpu_status(), _query_compute_apps(), _query_devices(), GPU status for arbo's device-aware scheduler (GET /gpu-status). arbo steers a l, Return `{devices: [...], tts_on_3090: bool}`. Degrades to an error field on nvid, _run() ### Community 456 - "remote-ssh" Cohesion: 0.50 Nodes (3): SSH_MCP_STRICT_ALLOWLIST, node, remote-ssh -### Community 458 - "max" -Cohesion: 0.50 -Nodes (4): anyOf, default, title, max +### Community 458 - "[2026-09-18] althing search returned ZERO for every hyphenated query — silently, for everyone" +Cohesion: 0.29 +Nodes (6): [2026-09-18] althing search returned ZERO for every hyphenated query — silently, for everyone, Rollout mechanics worth keeping, Still open — two postbox fixes proposed to forseti, Symptom 1 — "never delivered": NOT A DEFECT, Symptom 2 — A REAL DEFECT, and far wider than one message, The fix, and the boundary that held ### Community 462 - "wrap_verbatim_html" Cohesion: 0.22 @@ -2240,7 +2249,7 @@ Nodes (21): describe(), die(), main(), Return (frontmatter_dict, body). Absent o ### Community 528 - "$defs" Cohesion: 0.12 -Nodes (16): additionalProperties, description, required, title, type, additionalProperties, title, type (+8 more) +Nodes (17): additionalProperties, description, required, title, type, additionalProperties, title, type (+9 more) ### Community 529 - "Does MTP actually hurt Qwen3.8-Flash-Next on one Blackwell card?" Cohesion: 0.29 @@ -2294,9 +2303,9 @@ Nodes (5): Also, Arbo — catalog-driven image generation engine, Character Stud Cohesion: 0.17 Nodes (11): A false finding I caught on myself, ⚠⚠ `agent.disabled_toolsets` is GLOBAL — it would have cost 26 tools fleet-wide, ENABLED in config 2026-09-15 (operator-directed), ✅✅ FULLY LIVE 2026-09-15 02:17 — SVOS restarted, roster verified both ends, ✅ LIVE 2026-09-15 02:10 — gateway restarted, plugin registered, Roster verified another way, svos_miranda Hermes plugin — validation pass (2026-09-15), The blocker (found, fixed by svos-dev at `c964e64`) (+3 more) -### Community 547 - "step" -Cohesion: 0.50 -Nodes (4): step, anyOf, default, title +### Community 547 - "[2026-09-18] FleetTools — one capability index, autoloaded by three agent families" +Cohesion: 0.33 +Nodes (5): [2026-09-18] FleetTools — one capability index, autoloaded by three agent families, Autoload — verified per family, not assumed, Rule zero, which is the load-bearing part, Shape, The CLAUDE.md swap ### Community 548 - "Claude-Code-only tooling" Cohesion: 0.33 @@ -2330,9 +2339,9 @@ Nodes (6): ⚠⚠ FOUR MORE LIVE BREAKAGES on OTHER hosts — the sweep's real f Cohesion: 0.29 Nodes (6): Current state — ⛔ SHELVED, deliberately, 2026-09-16, For a consumer, grok-token-broker, Invariants, The problem, ⛔ Why the loop is disarmed until measured -### Community 564 - "mime" -Cohesion: 0.50 -Nodes (4): anyOf, default, title, mime +### Community 564 - "CatalogService" +Cohesion: 0.40 +Nodes (5): additionalProperties, required, title, type, CatalogService ### Community 582 - "build_corpus_hemingway.py" Cohesion: 0.20 @@ -2354,9 +2363,9 @@ Nodes (8): breeze-tts sizing and the fv-ml1 GPU 0 placement question (2026-09-15 Cohesion: 0.31 Nodes (9): decide(), load_text(), main(), Path, D2b for BabyHemingway: resolve entity gender by MAJORITY VOTE over nearby…, One pass over the corpus for EVERY surface at once. The obvious shape -- rescan…, Score a name's local pronoun mix AGAINST THE CORPUS BASE RATE, not against…, score() (+1 more) -### Community 590 - "mime_from_field" +### Community 590 - "accepted_types" Cohesion: 0.50 -Nodes (4): anyOf, default, title, mime_from_field +Nodes (4): anyOf, default, title, accepted_types ### Community 591 - "verify_quant.py" Cohesion: 0.40 @@ -2367,8 +2376,8 @@ Cohesion: 0.40 Nodes (5): [2026-08-24] ESH DNS fixed at the IPv6 layer, and the naming scheme went live, Root cause, The naming scheme became real, The three wrong turns, What landed ### Community 593 - "booth/app.py" -Cohesion: 0.10 -Nodes (22): fixture, create_app(), _from_env(), human_dur(), The Booth — a standing web server that renders drop-folders as ephemeral media…, A Content-Disposition-safe `.zip` (strip quotes/control chars)., Reduce a client-supplied filename to a safe basename (no path, no hidden)., safe_upload_name() (+14 more) +Cohesion: 0.07 +Nodes (30): fixture, booth_image_names(), classify(), create_app(), generate_pickup_id(), human_dur(), The Booth — a standing web server that renders drop-folders as ephemeral media…, Image files in a booth, in gallery (sorted-rel) order — for viewer prev/next. (+22 more) ### Community 596 - "description" Cohesion: 0.50 @@ -2382,9 +2391,9 @@ Nodes (7): load(), main(), per_seed(), Score beat arms against a decision rule f Cohesion: 0.33 Nodes (5): Asset Engine — `http://10.250.50.70:8200`, Direct inference seats + the Asset Engine control plane, GPU hosts, ⚠ Query the live inventory, never a written list, Traps -### Community 599 - "optional" +### Community 599 - "catalog_version" Cohesion: 0.50 -Nodes (4): default, title, type, optional +Nodes (4): default, title, type, catalog_version ### Community 600 - "LiteLLM gateway — the fleet's OpenAI-compatible front door" Cohesion: 0.33 @@ -2402,21 +2411,21 @@ Nodes (4): Ask the operator to decide (no chat round-trip), Show the operator so Cohesion: 0.50 Nodes (4): anyOf, default, title, options -### Community 605 - "min" +### Community 605 - "category" Cohesion: 0.50 -Nodes (4): anyOf, default, title, min +Nodes (4): enum, title, type, category -### Community 606 - "$ref" -Cohesion: 0.14 -Nodes (14): items, title, type, $ref, fields, section_groups, services, items (+6 more) +### Community 606 - "properties" +Cohesion: 0.15 +Nodes (14): $ref, properties, reproducibility_audit, section_groups, services, items, title, type (+6 more) ### Community 608 - "Speech — TTS and ASR endpoints" Cohesion: 0.40 Nodes (4): ASR, Speech — TTS and ASR endpoints, Traps, TTS -### Community 609 - "output_field" +### Community 609 - "content_type" Cohesion: 0.50 -Nodes (4): anyOf, default, title, output_field +Nodes (4): default, title, type, content_type ### Community 612 - "2026-09-17-lv-bronte-gate.md" Cohesion: 0.05 @@ -2438,6 +2447,14 @@ Nodes (3): althing — inter-agent message bus, Commands, Traps Cohesion: 0.50 Nodes (3): Language-safe text sanitizer for the OmniVoice TTS wrapper. Strips artifacts th, Return ``text`` cleaned of artifacts that shouldn't be spoken, language-safe., sanitize_tts_text() +### Community 622 - "fields" +Cohesion: 0.50 +Nodes (4): items, title, type, fields + +### Community 623 - "generated_at" +Cohesion: 0.50 +Nodes (4): anyOf, default, title, generated_at + ### Community 625 - "rename.py" Cohesion: 0.60 Nodes (4): build_pool(), main(), R49 Stage D2 (final) + D3 — entity resolution and deterministic rename…, title_gender() @@ -2454,8 +2471,36 @@ Nodes (3): load_text(), main(), Audit a stoplist for entries that are actually C Cohesion: 0.25 Nodes (8): _cli(), Regression: the branch ended with `[ "$n" -eq 0 ] && echo ...`, so it returned…, Regression: ids are 8 hex chars and roughly one in forty is all digits. Those…, test_cli_links_exits_zero_on_a_NON_empty_board(), test_cli_unlink_by_id_even_when_the_id_is_all_digits(), test_cli_unlink_by_index(), test_cli_unlink_of_a_stale_id_leaves_the_board_alone(), test_cli_unlink_rejects_a_non_id_non_index() +### Community 634 - "license_warning" +Cohesion: 0.50 +Nodes (4): anyOf, default, title, license_warning + +### Community 635 - "method" +Cohesion: 0.50 +Nodes (4): default, title, type, method + +### Community 636 - "streamable" +Cohesion: 0.50 +Nodes (4): streamable, default, title, type + +### Community 637 - "estimated_latency" +Cohesion: 0.67 +Nodes (3): anyOf, default, estimated_latency + +### Community 638 - "host" +Cohesion: 0.67 +Nodes (3): title, type, host + +### Community 639 - "lifecycle" +Cohesion: 0.67 +Nodes (3): anyOf, default, lifecycle + +### Community 640 - "version" +Cohesion: 0.67 +Nodes (3): version, title, type + ## Knowledge Gaps -- **3022 isolated node(s):** `Why Headscale (decided 2026-09-06)`, `Site facts that drive placement`, `Topology`, `Access & credentials — all in hand (verified 2026-09-06)`, `Phases` (+3017 more) +- **3045 isolated node(s):** `Symptom 1 — "never delivered": NOT A DEFECT`, `Symptom 2 — A REAL DEFECT, and far wider than one message`, `The fix, and the boundary that held`, `Rollout mechanics worth keeping`, `Still open — two postbox fixes proposed to forseti` (+3040 more) These have ≤1 connection - possible missing edges or undocumented components. - **154 thin communities (<3 nodes) omitted from report** — run `graphify query` to explore isolated nodes. @@ -2463,16 +2508,16 @@ Nodes (8): _cli(), Regression: the branch ended with `[ "$n" -eq 0 ] && echo ... _Questions this graph is uniquely positioned to answer:_ - **Why does `create_app()` connect `booth/app.py` to `FastAPI`, `test_booth.py`, `Path`?** - _High betweenness centrality (0.004) - this node is a cross-community bridge._ -- **Why does ``[2026-09-01]` pfi-gx10 (ASUS Ascent GX10) onboarded headless — and it is the intended new home for run 3c` connect ``[2026-09-01]` pfi-gx10 (ASUS Ascent GX10) onboarded headless — and it is the intended new home for run 3c` to `archival-memory.md`?** + _High betweenness centrality (0.003) - this node is a cross-community bridge._ +- **Why does `$defs` connect `$defs` to `CatalogLifecycle`, `CatalogSectionGroup`, `services.schema.json`, `CatalogModel`, `properties`, `FieldOption`, `CatalogService`?** _High betweenness centrality (0.002) - this node is a cross-community bridge._ -- **Why does `CatalogService` connect `$defs` to `properties`?** - _High betweenness centrality (0.001) - this node is a cross-community bridge._ -- **What connects `Why Headscale (decided 2026-09-06)`, `Site facts that drive placement`, `Topology` to the rest of the system?** - _3022 weakly-connected nodes found - possible documentation gaps or missing edges._ +- **Why does `properties` connect `properties` to `version`, `content_type`, `license`, `category`, `status`, `properties`, `fields`, `CatalogModel`, `properties`, `description`, `CatalogService`, `license_warning`, `method`, `streamable`, `estimated_latency`, `host`, `lifecycle`?** + _High betweenness centrality (0.002) - this node is a cross-community bridge._ +- **What connects `Symptom 1 — "never delivered": NOT A DEFECT`, `Symptom 2 — A REAL DEFECT, and far wider than one message`, `The fix, and the boundary that held` to the rest of the system?** + _3045 weakly-connected nodes found - possible documentation gaps or missing edges._ - **Should `Status + Open Issues` be split into smaller, more focused modules?** _Cohesion score 0.05555555555555555 - nodes in this community are weakly interconnected._ - **Should `Fleet disaster-recovery runbook` be split into smaller, more focused modules?** _Cohesion score 0.06666666666666667 - nodes in this community are weakly interconnected._ -- **Should `1. Locked architectural decisions` be split into smaller, more focused modules?** - _Cohesion score 0.07407407407407407 - nodes in this community are weakly interconnected._ \ No newline at end of file +- **Should `properties` be split into smaller, more focused modules?** + _Cohesion score 0.08 - nodes in this community are weakly interconnected._ \ No newline at end of file diff --git a/persistent-memory.md b/persistent-memory.md index dd76a9f..780b376 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -1,6 +1,6 @@ # Persistent memory — eshpfi-management -_Last updated: 2026-09-18 ~15:05 PT (infrastructure day, no training work. THREE silent faults found and fixed: NH3↔Anaheim was DERP-relayed (now 6 ms direct), `.internal` DNS was failing ~10% of lookups (cross-site resolver ring + AdGuard ratelimit 0), and SearXNG had one working web engine (now seven). althing v3.6.3 deployed — hyphenated search works for the first time. FleetTools shipped and autoloaded for Codex/Grok. ⚠ lv-mccarthy's run outcome is UNVERIFIED by this session.)_ +_Last updated: 2026-09-19 ~04:55 PT (⭐ `infra-hermes` is now infra-ops' ASSISTANT — he takes day-to-day checks and triage, infra-ops keeps tooling and the hard calls, and infra-ops may task him downward. He is explicitly NOT Miranda: his relays are information, not authorization. **Building the ops log is assigned and not started** — two agents now share one fingerprint-less OS identity. Previous day: three silent fleet faults fixed (DERP relay, `.internal` DNS, SearXNG one-engine), althing v3.6.3, FleetTools. ⚠ lv-mccarthy's run outcome still UNVERIFIED.)_ > **Always check for `/tmp/infra-ops-handoff.md`** — if it exists and its > `Written:` stamp is under **8 hours** old, read it (it carries the in-flight @@ -115,7 +115,31 @@ no longer deployed sidecars here. See Recent decisions.) ## Current state / in-flight -_As of 2026-09-18 ~14:50 PT._ +_As of 2026-09-19 ~04:55 PT._ + +### ▶ ASSIGNED, NOT STARTED — build the ops log (operator, 2026-09-19) + +**The problem it solves:** infra-ops and infra-hermes both act as the SAME OS identity +(`ssh infra-ops@`), and dockerd exec is not logged per-caller, so host-side +changes are fingerprint-less. With one agent that was theoretical. With two doing +routine ops daily it is operational — when a host differs from expectation, neither of +us can tell whether the other did it, a prior session did, or something broke. + +⚠ **Not hypothetical.** On 2026-09-18 another session edited the searxng stack while +this one was deploying it, crash-looping fleet search for ~4 minutes, and the author +was unidentifiable: every commit is attributed to Vuong Hoang by convention and the +on-host file carried no provenance. + +**Shape proposed and approved, not yet designed in detail:** one appended line per +host-changing action (who / what / when), plus a lightweight claim on shared stacks so +two agents do not deploy the same thing at once. Cheap, no new infrastructure, and it +makes infra-hermes's handoffs upward legible. + +**Open design questions, mine to settle:** where it lives (repo file vs a host-side +log vs the post office), whether the claim is advisory or enforced, and whether +`deploy-stack.sh` / `elway` write to it automatically rather than relying on +discipline — the last one matters most, since a log that depends on remembering is the +same class of instrument as a health check that passes in both states. ### ⚠ FIRST — lv-mccarthy's run outcome is UNVERIFIED by this session @@ -179,6 +203,8 @@ nothing touched. Full context in the 09-17 Recent decisions entries. ## Recent decisions +- `[2026-09-19]` ⭐⭐⭐ **`infra-hermes` is this session's ASSISTANT, and the division of labour is now standing policy.** infra-ops keeps **improving infrastructure tooling** plus the hard calls; infra-hermes does **day-to-day checks, triage and routine operations**; either may perform infra ops; **infra-ops may task him downward** and he escalates upward as needed. Three operator answers, verbatim in intent: (1) **build the ops log** — see in-flight; (2) **he is NOT Miranda**, so the global CLAUDE.md's sole-trusted-relay exception does not cover him and a directive he relays is information rather than authorization (reversible relayed work executes, irreversible or fleet-affecting goes to the operator); (3) **yes, task him**. Structural facts recorded in `CLAUDE.md` § "infra-hermes IS a real peer" rather than here, because a fresh session must have them without reading this file. ⚠ He is a Hermes bus seat on nh3-dev (`althing-pump-infra-hermes.service`, enabled, route declared) — round trip proven both directions 2026-09-19 04:46. + - `[2026-09-18]` ⭐⭐⭐ **NH3↔Anaheim had been running over a throttled DERP relay, not a direct path — 78 GB of fleet traffic on someone else's free infrastructure.** Four additive objects on ana-gw gave ana-scale a stable inbound UDP 41641 endpoint; `tailscale ping` 373–522 ms → **6 ms direct**, cross-site HTTP 1.2 s → 0.015 s, STT via the ANA gateway 1.4 s → 0.25 s. ⚠ That box runs `central-nat`, so a policy `dstaddr` is the REAL internal address, not the VIP. No OOB access — back up with `show` to a local file and make additive changes ONLY. irv-ml1 still relayed. → `persistent-memory.d/2026-09-18-nh3-ana-derp-relay.md` - `[2026-09-18]` ⭐⭐⭐ **`.internal` DNS was failing ~10% of lookups fleet-wide, from two independent causes.** A PUBLIC resolver was the fallback for a PRIVATE zone (Cloudflare answers NXDOMAIN authoritatively, so a transient miss became a hard failure) — now a cross-site ring, each site local-first with a different site as backup. Then the root cause: all three AdGuards shipped `ratelimit: 20` shared across an entire **/24**, silently dropping queries at 5 s each. Set to 0. Hard failures 3/40 → 0/40; burst timeouts 40/60 → 0/60. ⚠ `resolv.conf` is DHCP-managed — change it at the UDM/FortiGate, not the file. → `persistent-memory.d/2026-09-18-fleet-dns-ring-and-ratelimit.md` - `[2026-09-18]` ⭐⭐ **SearXNG had ONE working general web engine and every health check said fine.** 7 of 55 were enabled-by-default and six of those are dictionary/translation engines — `inactive: false` only makes an engine SELECTABLE, `disabled: false` puts it in the DEFAULT set. Now seven. ⭐ This stack tracks `:latest` ON PURPOSE (upstream ships engine-handler fixes continuously; a pin freezes breakage). ⭐ The Brave key is committed in plaintext by explicit operator decision — scoped to one low-value credential, NOT a change to the no-secrets rule. → `persistent-memory.d/2026-09-18-searxng-one-engine-to-seven.md`