Commit Graph

380 Commits

Author SHA1 Message Date
vh 2185964a6a memory: gen seat RESOLVED — mixed FP8-attn build on pinned nightly, coherent through 60k
Supersedes the stale 'GEN SEAT = AEON' current-state line. Records the
final resolved config (JonathanColetti/Heretic mixed NVFP4+FP8 on pinned
vLLM nightly, MTP on), the two-real-causes root cause, the AEON purge, and
the pinned-nightly follow-up (move to stable once #51113 ships). A cold
session now reads the correct gen-seat state.
2026-08-17 02:13:42 -07:00
vh aba7cda33e fix(refusal-probe): retract an invalid cross-battery comparison; adopt the two-control pair
RETRACTION. I claimed my arm-3 92.5% 'exceeded' brokkr-smithy-dev's 62.5%.
That comparison was invalid. His diff against his own artifact shows my
battery-instruct.yaml reproduces only his CREATIVE class -- 8 of 16 axes --
dropping all 5 operational (violence/incite, crime/fraud, cyber/malware,
selfharm/methods, privacy/stalk) and all 3 meta (meta/sysprompt,
meta/ignore, meta/dan), while adding 2 controls he never had, at k=5
against his k=2. His figure pools 16 axes; mine is creative-only. Different
denominators, so there was never a delta to report.

Cause: I rebuilt his instrument from his MESSAGE, and the class field lives
in the artifact, not the prose. Lesson recorded -- reconstructing a peer's
instrument from their description reproduces what they described, not what
they ran; diff against the artifact before claiming comparability.

Adopts his amendment to my control fix, which is better than my proposal. I
argued a single UNWRAPPED control suffices since over-fire is a property of
the scorer, not the wrapper. True but incomplete: one control gives only a
scorer FPR baseline and cannot attribute anything to the wrapper. Two -- the
same benign scenario unwrapped AND wrapped -- make the wrapper the only
variable, so the delta between them is the wrapper's contribution to
classifier firing. Costs one prompt.

ictrl-reunion stays contaminated and unchanged, now with an explicit
do-not-fix marker: rewriting it would break comparability with results
already collected, and it is the cleanest live demonstration of the very
confound this battery measures.
2026-08-16 16:45:17 -07:00
vh 766c65801c memory: AEON accepted on the gen seat; canonical sampling; four-wrong-diagnoses lesson
Gen seat state, the abliteration catatonia signature, canonical Qwen3.8
sampling with the wrong-mode presence_penalty fix, the two real
gateway-chat defects, and the single-file bind-mount inode trap.

Also records the methodology failure honestly: four disproved hypotheses
on one bug, caused by a harness that varied the QUESTION along with the
conversation depth, so a narrower question drawing a shorter answer read
as degeneration. Banked as rules -- hold the final question fixed when
comparing across depth, do not infer trends from n=3 when identical
inputs span 25-465 words, and ask for the real failing transcript before
building a synthetic reproduction.
2026-08-16 16:31:24 -07:00
vh 0b95701173 memory: snapshot — FF A/B result, seat state, Qwen-thinking finding, esh hardening
Supersedes the 'blocked on a reproducing battery' entry: brokkr supplied
the framing, the A/B ran, and the refusal question is settled.

Most load-bearing fact for a cold session: Dark-Scarlett is DOWN and
Fable-Fusion is serving char-rp-reasoning on an evaluation window with no
permanent decision taken. Rollback recorded inline.
2026-08-16 10:29:39 -07:00
vh 09c56d51a9 memory: DS-replacement eval blocked on a reproducing battery; refusal-measurement lessons
Records the open loop with brokkr-smithy-dev (01M05KQAD55PYHGCP668AV06YR)
and three durable lessons: benign controls cannot validate a refusal
classifier on in-character RP prose, a 0% rate needs a classifier
self-test before it is believed, and an approved plan whose premise the
evidence invalidates should be surfaced rather than spent.
2026-08-16 08:41:57 -07:00
vh f83456a276 memory: snapshot for /clear — auto-archived 16 entries (Recent decisions 13, Tried and abandoned 3); index 303 -> 254 2026-08-15 19:55:40 -07:00
vh 4e74e0aefe memory: RP-seat direction — keep MeroMero, reject Artemis, Dark-Scarlett 3.8 queued as the MTP-restoring move 2026-08-15 19:50:11 -07:00
vh f338f228a6 memory: record the quantization playbook as the durable home for quant lessons 2026-08-15 10:02:24 -07:00
vh b9da05aeb2 memory: snapshot — overnight arc pushed; operator ruled the +1.7% PPL an acceptable trade (settled) 2026-08-15 08:14:05 -07:00
vh 930197a56a memory: record prefill result (roughly doubled) on the gen-seat mixed requant 2026-08-15 02:34:04 -07:00
vh fa4f652a39 memory: snapshot — gen seat mixed NVFP4+FP8 requant (+18%) + char-rp tool parser; both queued items closed 2026-08-15 02:22:58 -07:00
vh b1370e4b4d memory: snapshot — uncensored gen seat landed (Qwen3.8-27B-Uncensored, gen-seat/vllm-gen); next = chase W4A8 2026-08-15 00:32:50 -07:00
vh bf915e15f0 memory: snapshot — eRP dual-seat overhaul landed (MeroMero + Dark-Scarlett @ 256K); next = infra+access 2026-08-12 14:33:53 -07:00
vh 69597cb686 memory: wgtunnel Phase-0 DNS landed + infra-ops now holds fleet Cloudflare DNS-edit token
boring.phasefinal.com CNAME -> ana-srv1 (DNS-only) published + verified; unblocks
the wstunnel ACME cert. Operator handed over an all-zones Zone:DNS:Edit token, now
vaulted at nh3-dev/.config/cloudflare/infra-ops-dns-token — fleet DNS is self-serve.
2026-08-12 01:29:42 -07:00
vh a8c6d85df9 memory: wgtunnel moved to its own repo (vh/wgtunnel) — endpoint infra verified + pre-seeded
Flip the queued-project pointer: wgtunnel is now a standalone repo, pre-seeded
after a live verification pass (ana-wg UDP/31337 endpoint, traefik TLS edge on
ana-docker, direct-A public edge). Ground truth captured in the new repo;
/vor-plan is the next step. Do wgtunnel work in ~/development/wgtunnel, not here.
2026-08-11 23:48:17 -07:00
vh 3b7e10cd29 memory: snapshot for /clear — secrets-broker + stonehenge-park landed; wgtunnel queued next
Current state rewritten to session-end: secrets-broker LIVE (secret CLI + 25-item
backfill + rm/warn), stonehenge-park /park service repo stood up (vor-plan+vor-ui),
dots-tts extracted to tts-stack; NEXT PROJECT = wgtunnel (WireGuard over a plane's
wifi). Two detail files added; 2 oldest T&A entries archived. Handoff written to
/tmp/infra-ops-handoff.md for the post-clear pickup.
2026-08-11 23:25:31 -07:00
vh 62672c9850 refactor(dots-tts): extract TTS stack to tts-stack repo; pointer stub + move voices out
TTS development moves to a dedicated repo (~/development/tts-stack) so a separate
agent can own tuning/dev. Mirrors the chatterbox-fast extraction:

- stacks/dots-tts/ reduced to a pointer README (code/Dockerfile/compose/tests/env
  now canonical in tts-stack).
- voices/ canonical corpus moved out to tts-stack/voices/. Blast-radius checked:
  no eshpfi playbook/script reads the corpus (other voices/ refs are unrelated
  host paths under /worktank/...).
- persistent-memory updated: TTS dev extracted + stood down; reverses the earlier
  "corpus home = eshpfi voices/" call.

The ~15 experimental TTS compose wrappers stay here as reference (catalogued in
tts-stack/KNOWLEDGE.md). Live service on irv-ml1:8198 is unaffected (runs from a
copy on the host).
2026-08-11 07:46:11 -07:00
vh a80f6e958f fix(dots-tts): v3 — clause-break (; : em-dash) → period pause mapping
dots' prosody honors a pause only for ellipsis (~+0.43s) and period (~+0.3s);
comma/semicolon/colon/dash all run flat (~+0.03s vs no-punct), measured via a
duration-over-N-runs pause probe against the live service. Two sub-causes for
the flat clause reads: em-dashes regressed in v2 (the —→- fold made them read
as word-joiners), and semicolons were never honored by dots at all.

Operator ruled ellipsis "too much" → map semicolon, clause colon, and em-dash
to a period in _sanitize (believable ~0.3s clause pause). Guards, pinned by
tests: digit-guarded colon so times (3:45) and ratios (2:1) keep their colon;
en-dash kept folding to hyphen so numeric ranges (10–20) don't become "10.20";
a genuine ellipsis retains its strong pause.

Deployed to irv-ml1:8198 as local/dots-tts:v3 via the redeploy2 build →
:8199-test → pause-gate → cutover pattern (gate measured +0.427s, live healthy).
2026-08-10 21:54:35 -07:00
vh 944c22a95c memory: snapshot for /clear — dots.tts engine arc + LFM2.5 landed; in-flight refreshed to session end-state 2026-08-10 15:07:18 -07:00
vh 077570167f memory: dots-tts v2 consumer-confirmed clean (ratatoskr end-to-end verify) 2026-08-10 09:15:08 -07:00
vh 10d379db5b fix(dots-tts): v2 — curly-punctuation sanitize + sentence-chunking
Curly apostrophes (ratatoskr's LLM emits typographic punctuation) made dots
mispronounce contractions ("Donut's"->"donut ess"); fold curly->ASCII before
synth, keep normalize_text on. Add server-side sentence-chunking so long turns
stop truncating at dots' ~40s single-generate cap (verified full 160s Zev).
Dockerfile: pin dots.tts==0.2.1 + torch/torchaudio==2.8.0 (upstream constraints
now pin a phantom gradio==6.17.0; float torchaudio->2.11.0 crashes the load).
2026-08-10 09:11:56 -07:00
vh d3727dee53 feat(vllm): lfm2.5 reasoning-parser (deepseek_r1) — scoreable JSON for brokkr's bake-off
LFM2.5 is </think>-delimited (opening tag in prompt); deepseek_r1 splits
reasoning into reasoning_content so content is the clean post-</think>
answer. Re-smoke: content valid JSON + reasoning_content populated. License
production-cleared (operator <$10M ruling), still out of routing per the
measurement gate.
2026-08-10 07:27:36 -07:00
vh bb65f36f70 memory: chatterbox-fast :8197 reclaimed (dots migration fully closed) — 3090 freed ~7GB 2026-08-10 07:19:28 -07:00
vh fa6e9a3c69 memory: dots-tts ratatoskr cutover COMPLETE (operator ear-confirmed) — chatterbox :8197 reclaim pending operator go 2026-08-10 07:17:35 -07:00
vh b846ebf32e memory: ratatoskr dots-tts cutover shipped (v0.22.2) — hold :8197 rollback pending operator ear-check 2026-08-10 07:15:22 -07:00
vh edc9f42da1 feat(vllm,litellm): lfm2.5-2.6b non-prod bake-off alias for brokkr
vllm-lfm25 on ana-ml2 GPU1 :8021 (LiquidAI/LFM2.5-2.6B, util 0.09 into
unreserved slack, max-len 16384, no reasoning-parser so content is non-empty).
LiteLLM alias lfm2.5-2.6b with vendor sampling baked as default (temp 0.1;
top_k 50 + repetition_penalty 1.1 via extra_body). Eval-only, not in any
routing chain, pending operator ruling on LFM Open License production use.
2026-08-10 07:13:37 -07:00
vh c8acf60449 feat(dots-tts): ship OpenAI-compatible dots.tts TTS stack on irv-ml1:8198
Thin FastAPI wrapper over DotsTtsRuntime (soar, optimize=True, RTF ~0.22),
serialized single-consumer; OpenAI /v1/audio/speech (stream + non-stream),
voices from the voices/ corpus derived set. Live + healthy alongside
chatterbox-fast on the 3090; nothing repointed. Dockerfile needs
build-essential (torch.compile/inductor JITs via gcc at runtime) + persisted
inductor cache. Remaining Phase-2: ratatoskr client cutover.
2026-08-10 01:07:37 -07:00
vh fca1a545f1 feat(voices): canonical voice corpus + dots.tts-optimized refs
Engine-agnostic voice corpus: canonical source clip + transcript per voice,
per-engine reference sets derived by derive.py from engines.yaml profiles.
First residents donut/glados/emmie/miranda optimized + verified clean for
dots.tts (sentence-bounded ref + accurate transcript — dots leaks reference
audio into output otherwise). canonical/ + transcripts/ tracked; derived/
gitignored (regenerable). Records the dots.tts burn-in in persistent-memory.
2026-08-10 00:32:10 -07:00
vh 58b58d1401 memory: worldtree #400 closed — fiction-decomp snapshot cleared (208M); ratatoskr-dev knob revert now sole non-blocking await 2026-08-08 20:37:47 -07:00
vh ba4597b8f2 memory: snapshot for /clear — session arcs landed (kb sweep, muninn 0.1.6, chatterbox-fast tail-degradation fix, Zonos-down, #400 pull); in-flight trimmed to awaiting-peers 2026-08-07 19:15:03 -07:00
vh 6399a5a267 memory: record #400 personal-KB Chroma snapshot on nh3-dev (worldtree-dev pull)
Read-only .chroma persist store pulled to ~/snapshots/worldtree-400-fiction-decomp/
(fiction 1578 / kb 2876 post-#394 / main 3224), provenance-marked, keep until #400 done.
2026-08-07 18:53:12 -07:00
vh 6332f14af5 memory: chatterbox-fast tail-degradation diagnosed + fixed (max_chunk_chars=250 cap)
Long operator-driven diagnosis: the 'broken/German/dead-air' was the Turbo T3
model over-running its generation tail (garble in final ~2-3s, worse with
length + tight sampling), NOT a language leak or OOM. Fix = server-side
max_chunk_chars=250 cap (:v2), keeping 3-4 sentence clean chunks with smooth
joins. Method (amplitude-gated voiced-ZCR), foot-guns (tail-trim unreliable,
build-context/image drift), and the flat-vs-repo divergence recorded.
2026-08-07 11:40:05 -07:00
vh 2d7eb90cc3 memory: Zonos2 taken down on 3090 (operator-directed, for memory, temporary)
Freed ~17.4GB (3090 728MiB->18.2GB free) for co-resident chatterbox-fast
(was OOMing). Detached native process; GPU mem held by multiprocessing-fork
children that orphan to init - kill children explicitly. Restore cmd +
affected consumers (asset-engine, gateway-chat) recorded.
2026-08-07 10:29:48 -07:00
vh 2e0bb85906 memory: chatterbox-fast donut voice added + contract delivered to ratatoskr-dev
Operator-directed donut add (zonos Donut.wav -> chatterbox /refs/donut.wav,
live glob no restart, verified 7.5s synth). Answered ratatoskr's 8-Q contract
ask for their TTS migration off Zonos: POST /tts (not OpenAI), no affect dials
(Turbo no-ops), streaming shape identical to Zonos, 24kHz, English-only.
2026-08-07 08:41:15 -07:00
vh 9147bc9413 memory: muninn-dispatch 0.1.6 published to vh PyPI (worldtree-dev carried ask done)
Comment-only bump built from clean git archive of Worldtree origin/main
(578f8fc, on the b182 line). Index + clean-venv install + metadata==0.1.6
verified; muninn-dev + worldtree-dev pinged. Cleared from carried-pending.
2026-08-07 07:17:26 -07:00
vh 7ea8dd326b memory: personal-KB orphan sweep EXECUTED post-b182 (kb 8230->2876, orphan=0)
Operator-authorized (via worldtree-dev thread 01KZE6TGRHAW) cleanup of the WT
#394 contamination: b182 deployed onto personal, all 3 preconditions verified,
reconcile --repair swept the 5,354 orphan index rows. Fresh reconcile confirms
orphan=0/missing=0/stale=0. Evidence-hold index-row half lifted+done; on-disk
generation-dir file-retention hold still stands. reconcile --repair exited rc=1
on a non-blocking worldtree-side git-staging bug (index.add on a deleted
uncommitted path) — flagged to worldtree-dev, not fixed here.
2026-08-07 07:10:21 -07:00
vh 5616a9da35 memory: snapshot for /clear — fleet reranker cutover shipped+verified (R42 v13 PASS), WT#394 kb-contamination diagnosed (attribution UNRESOLVED, shared-identity gap parked), b182 recreate-verify in-flight 2026-08-07 06:32:20 -07:00
vh 6c6d3f2939 memory: snapshot for /clear — booth 3 features shipped, herald v2.1.2, CI-flip PARKED (runner-auth)
Session (2026-08-05): 3 Booth features live+tagged (verbatim-wrap chip, .md/.txt
doc-viewer, image prev/next arrows); worldtree herald re-nudge bug -> forseti
althing-core v2.1.2; fleet-CI-resilience flip attempted end-to-end and PARKED on
an act_runner->gitea action-fetch auth blocker (infra-ops to research, deferred).
Archived the 2026-07-15/16 recent-decisions batch (8 entries) to keep the index
under the soft cap.
2026-08-05 04:31:59 -07:00
vh b5ae9365ff memory: triton backend is a HARD NO on the Ampere comfy box (fp8e4nv), deferred to Ada
Recorded under Tried-and-abandoned: --enable-triton-backend on irv-ml1's
A6000 crashes every render (triton's fp8 dequant needs fp8e4nv/e4m3, which
sm_86 lacks; the global --fp8_e4m3fn-text-enc dequant dies at CLIPTextEncode,
taking fp8 and int8 alike). Reverted cleanly; cu130 rebuild won't fix a
hardware format limit; revisit at the Ada refresh. Also captures the
compose-env-needs-recreate-not-restart mechanics lesson.
2026-08-03 21:29:00 -07:00
vh d4d9956fed memory: correct extdev sudo status in 07-25 herald-install entry
The [2026-07-25] nh3-extdev herald-install entry claimed infra-ops was
sudo-less on extdev, explaining the use of the lkraven@ NOPASSWD path.
That parenthetical was wrong: infra-ops has had full NOPASSWD sudo on
extdev since 2026-06-25. Kept the historical action (lkraven@ was used)
and appended a dated correction so a future session self-serves as
infra-ops instead of taking the lkraven@ hop unnecessarily. Surfaced
during the 2026-08-03 herald.py:363 re-nudge-bug arc.
2026-08-03 08:01:17 -07:00
vh bc084edbee memory: snapshot for /clear — worldtree b168/#384/#385 arc complete
Collapsed the completed-arc bloat out of Current state / in-flight (line 125:
6487->1073 chars; runbooks moved to persistent-memory.d/2026-08-03-worldtree-b168-384-385-arc.md).
Added the 2026-08-03 arc Recent-decisions entry. Auto-archived 20 oldest
Recent-decisions entries (2026-07-08..07-14) to archival-memory.md (index 345->308
lines). Refreshed the b169-deploy watch, unpushed-count, and handoff.
2026-08-03 00:12:59 -07:00
vh 4be87f1a94 memory: b169 pull-fail was a shared-containerd race, NOT disk; dont prune in-use img
corviduo-dev demo+personal+pinned share one containerd; concurrent pull of b169's
torch layer failed personal mid-race while demo completed. Image 6e34a87 fully
valid (demo running it healthy). Fix = re-run. NEAR-MISS: prune/rmi 6e34a87 would
have downed demo. Lesson: docker ps running images before any prune cleanup.
2026-08-02 23:57:48 -07:00
vh 45594e3891 memory: config-delta pre-sync rule — only config/ is bind-mounted, agents/ in-image
Verified via docker inspect: worldtree containers bind-mount only config/ host-side
(providers/model_roles/matrix/policies/defaults/env.public = pre-syncable set);
agents/ (schemas.yaml) + code ship in-image. So b169's schemas.yaml delta needs
no host pre-sync; only config/*.yaml changes are boot-blocking-pre-syncable.
2026-08-02 23:40:39 -07:00
vh 523b28f12f memory: production sweep DONE — 785 April DCC dupes deleted, arc complete
Operator-approved: backed up 785 rows (reversible) -> deleted job_id=b59c147c5ce0
from main (4009->3224, target cleared) -> bounced api (HNSW reload) -> confirmed
search fiction-only. #384/#385 closed, worldtree arc complete.
2026-08-02 23:12:21 -07:00
vh b4ef0600d2 memory: BrokenPipe wing-git watch CLOSED (round-2 commits clean, chown enabled)
worldtree-dev closed the watch with git evidence: both round-2 wing commits
landed clean under exec -u 1000; loose-object chown enabled it. Self-healed as
predicted; last thread of the reindex-as-root fallout closed.
2026-08-02 22:54:32 -07:00
vh 3a829890c5 memory: worldtree #384/round-2 arc COMPLETE + #381 restart fired
Both books terminal (DCC 705 / P&P 667 indexed, 0 failures, #385 budget fix
validated); #381 restart-after-ingest fired, api healthy, wing retrievable
(search_library returns DCC+P&P from fiction). ratatoskr verify handed off.
Delete-sweep precondition now met (stale main rows = genuine duplicates).
2026-08-02 22:53:01 -07:00
vh d72597bade memory: /health W_OK blind-spot for root-owned job subdirs (muninn-dev)
os.access(ingestion_root, W_OK) tests only the root dir, so a root-owned job
subdir passes requeue guards + /health both while being unclaimable by uid 1000.
muninn-dev added an ownership column to the post-move check.
2026-08-02 22:47:51 -07:00
vh 8bc5be35ce memory: reindex-as-root contaminated KB tree; adopt docker exec -u 1000 rule
sudo docker exec --reindex ran as root (app=uid 1000), leaving a root-owned
.old- dir (blocked uid-1000 rmtree) + 60 root-owned git objects. Fixed: rm -rf
.old- (tar'd first) + chown 1000:1000 the objects. RULE: docker exec that writes
pipeline state uses -u 1000. GOTCHA: don't head a scope-defining find (.old- had
153 files, head-20 hid the 60 git objects).
2026-08-02 22:45:33 -07:00
vh 3790669fb5 memory: round-2 sequencing (mimir-dev drives, muninn-dev confirms, infra-ops #381)
muninn-dev HOLD LIFTED (guard-verified 02:36Z). Recorded who owns what:
round-2 requeue = mimir-dev browser flow (op ruling pending); muninn-dev = gate
confirmer / post-move board-check custody; infra-ops = #381 restart after both
terminal + later supervised sweep.
2026-08-02 19:40:04 -07:00
vh 9bd5f2a4b1 memory: mv -t is the real guard (trailing slash is a false guard); dont-tidy note
muninn-dev tested on coreutils 9.1: mv src failed/ with failed/ missing STILL
silently renames (rc=0) — trailing slash buys nothing. mv -t <dir> <src> refuses
a missing target loudly (rc=1). Adopted as house convention. Also: DCC+P&P rest
in failed/ with manifest=complete until round-2 (requeue keys on dir placement) —
deliberate, don't tidy.
2026-08-02 19:38:08 -07:00