boring.phasefinal.com CNAME -> ana-srv1 (DNS-only) published + verified; unblocks
the wstunnel ACME cert. Operator handed over an all-zones Zone:DNS:Edit token, now
vaulted at nh3-dev/.config/cloudflare/infra-ops-dns-token — fleet DNS is self-serve.
Flip the queued-project pointer: wgtunnel is now a standalone repo, pre-seeded
after a live verification pass (ana-wg UDP/31337 endpoint, traefik TLS edge on
ana-docker, direct-A public edge). Ground truth captured in the new repo;
/vor-plan is the next step. Do wgtunnel work in ~/development/wgtunnel, not here.
Current state rewritten to session-end: secrets-broker LIVE (secret CLI + 25-item
backfill + rm/warn), stonehenge-park /park service repo stood up (vor-plan+vor-ui),
dots-tts extracted to tts-stack; NEXT PROJECT = wgtunnel (WireGuard over a plane's
wifi). Two detail files added; 2 oldest T&A entries archived. Handoff written to
/tmp/infra-ops-handoff.md for the post-clear pickup.
TTS development moves to a dedicated repo (~/development/tts-stack) so a separate
agent can own tuning/dev. Mirrors the chatterbox-fast extraction:
- stacks/dots-tts/ reduced to a pointer README (code/Dockerfile/compose/tests/env
now canonical in tts-stack).
- voices/ canonical corpus moved out to tts-stack/voices/. Blast-radius checked:
no eshpfi playbook/script reads the corpus (other voices/ refs are unrelated
host paths under /worktank/...).
- persistent-memory updated: TTS dev extracted + stood down; reverses the earlier
"corpus home = eshpfi voices/" call.
The ~15 experimental TTS compose wrappers stay here as reference (catalogued in
tts-stack/KNOWLEDGE.md). Live service on irv-ml1:8198 is unaffected (runs from a
copy on the host).
dots' prosody honors a pause only for ellipsis (~+0.43s) and period (~+0.3s);
comma/semicolon/colon/dash all run flat (~+0.03s vs no-punct), measured via a
duration-over-N-runs pause probe against the live service. Two sub-causes for
the flat clause reads: em-dashes regressed in v2 (the —→- fold made them read
as word-joiners), and semicolons were never honored by dots at all.
Operator ruled ellipsis "too much" → map semicolon, clause colon, and em-dash
to a period in _sanitize (believable ~0.3s clause pause). Guards, pinned by
tests: digit-guarded colon so times (3:45) and ratios (2:1) keep their colon;
en-dash kept folding to hyphen so numeric ranges (10–20) don't become "10.20";
a genuine ellipsis retains its strong pause.
Deployed to irv-ml1:8198 as local/dots-tts:v3 via the redeploy2 build →
:8199-test → pause-gate → cutover pattern (gate measured +0.427s, live healthy).
Curly apostrophes (ratatoskr's LLM emits typographic punctuation) made dots
mispronounce contractions ("Donut's"->"donut ess"); fold curly->ASCII before
synth, keep normalize_text on. Add server-side sentence-chunking so long turns
stop truncating at dots' ~40s single-generate cap (verified full 160s Zev).
Dockerfile: pin dots.tts==0.2.1 + torch/torchaudio==2.8.0 (upstream constraints
now pin a phantom gradio==6.17.0; float torchaudio->2.11.0 crashes the load).
LFM2.5 is </think>-delimited (opening tag in prompt); deepseek_r1 splits
reasoning into reasoning_content so content is the clean post-</think>
answer. Re-smoke: content valid JSON + reasoning_content populated. License
production-cleared (operator <$10M ruling), still out of routing per the
measurement gate.
vllm-lfm25 on ana-ml2 GPU1 :8021 (LiquidAI/LFM2.5-2.6B, util 0.09 into
unreserved slack, max-len 16384, no reasoning-parser so content is non-empty).
LiteLLM alias lfm2.5-2.6b with vendor sampling baked as default (temp 0.1;
top_k 50 + repetition_penalty 1.1 via extra_body). Eval-only, not in any
routing chain, pending operator ruling on LFM Open License production use.
Long operator-driven diagnosis: the 'broken/German/dead-air' was the Turbo T3
model over-running its generation tail (garble in final ~2-3s, worse with
length + tight sampling), NOT a language leak or OOM. Fix = server-side
max_chunk_chars=250 cap (:v2), keeping 3-4 sentence clean chunks with smooth
joins. Method (amplitude-gated voiced-ZCR), foot-guns (tail-trim unreliable,
build-context/image drift), and the flat-vs-repo divergence recorded.
Freed ~17.4GB (3090 728MiB->18.2GB free) for co-resident chatterbox-fast
(was OOMing). Detached native process; GPU mem held by multiprocessing-fork
children that orphan to init - kill children explicitly. Restore cmd +
affected consumers (asset-engine, gateway-chat) recorded.
Comment-only bump built from clean git archive of Worldtree origin/main
(578f8fc, on the b182 line). Index + clean-venv install + metadata==0.1.6
verified; muninn-dev + worldtree-dev pinged. Cleared from carried-pending.
Operator-authorized (via worldtree-dev thread 01KZE6TGRHAW) cleanup of the WT
#394 contamination: b182 deployed onto personal, all 3 preconditions verified,
reconcile --repair swept the 5,354 orphan index rows. Fresh reconcile confirms
orphan=0/missing=0/stale=0. Evidence-hold index-row half lifted+done; on-disk
generation-dir file-retention hold still stands. reconcile --repair exited rc=1
on a non-blocking worldtree-side git-staging bug (index.add on a deleted
uncommitted path) — flagged to worldtree-dev, not fixed here.
Session (2026-08-05): 3 Booth features live+tagged (verbatim-wrap chip, .md/.txt
doc-viewer, image prev/next arrows); worldtree herald re-nudge bug -> forseti
althing-core v2.1.2; fleet-CI-resilience flip attempted end-to-end and PARKED on
an act_runner->gitea action-fetch auth blocker (infra-ops to research, deferred).
Archived the 2026-07-15/16 recent-decisions batch (8 entries) to keep the index
under the soft cap.
Recorded under Tried-and-abandoned: --enable-triton-backend on irv-ml1's
A6000 crashes every render (triton's fp8 dequant needs fp8e4nv/e4m3, which
sm_86 lacks; the global --fp8_e4m3fn-text-enc dequant dies at CLIPTextEncode,
taking fp8 and int8 alike). Reverted cleanly; cu130 rebuild won't fix a
hardware format limit; revisit at the Ada refresh. Also captures the
compose-env-needs-recreate-not-restart mechanics lesson.
The [2026-07-25] nh3-extdev herald-install entry claimed infra-ops was
sudo-less on extdev, explaining the use of the lkraven@ NOPASSWD path.
That parenthetical was wrong: infra-ops has had full NOPASSWD sudo on
extdev since 2026-06-25. Kept the historical action (lkraven@ was used)
and appended a dated correction so a future session self-serves as
infra-ops instead of taking the lkraven@ hop unnecessarily. Surfaced
during the 2026-08-03 herald.py:363 re-nudge-bug arc.
Collapsed the completed-arc bloat out of Current state / in-flight (line 125:
6487->1073 chars; runbooks moved to persistent-memory.d/2026-08-03-worldtree-b168-384-385-arc.md).
Added the 2026-08-03 arc Recent-decisions entry. Auto-archived 20 oldest
Recent-decisions entries (2026-07-08..07-14) to archival-memory.md (index 345->308
lines). Refreshed the b169-deploy watch, unpushed-count, and handoff.
corviduo-dev demo+personal+pinned share one containerd; concurrent pull of b169's
torch layer failed personal mid-race while demo completed. Image 6e34a87 fully
valid (demo running it healthy). Fix = re-run. NEAR-MISS: prune/rmi 6e34a87 would
have downed demo. Lesson: docker ps running images before any prune cleanup.
worldtree-dev closed the watch with git evidence: both round-2 wing commits
landed clean under exec -u 1000; loose-object chown enabled it. Self-healed as
predicted; last thread of the reindex-as-root fallout closed.
os.access(ingestion_root, W_OK) tests only the root dir, so a root-owned job
subdir passes requeue guards + /health both while being unclaimable by uid 1000.
muninn-dev added an ownership column to the post-move check.
muninn-dev tested on coreutils 9.1: mv src failed/ with failed/ missing STILL
silently renames (rc=0) — trailing slash buys nothing. mv -t <dir> <src> refuses
a missing target loudly (rc=1). Adopted as house convention. Also: DCC+P&P rest
in failed/ with manifest=complete until round-2 (requeue keys on dir placement) —
deliberate, don't tidy.
A misplaced ingestion-state move is silent (list_jobs OK, files inert; sole
symptom = job absent from board, job_row->None/404). Verify job is on the board
+ job.dispatch.json survived after any move; don't trust mv exit codes.
worldtree-dev round-2 unblock: mv job complete/->failed/ assumed failed/
existed; it didn't on personal, so mv renamed job1 to 'failed' + nested job2.
Caught + reconstructed via complete/ scratch, no data loss. Lesson: verify dir
exists with [ -d ] before mv-into (empty ls is ambiguous); guard peer fs cmds
with mkdir -p.
worldtree-dev accepted #384 recovery. Wing-git BrokenPipeError ruled a
watch-item (self-heals on DCC round-2 re-run; escalate only if 2-in-a-row).
Recorded a heads-up for a future production delete-by-job_id of 785 April-era
DCC orphans (job_id=b59c147c5ce0, 'main' collection) — to be handled supervised
(backup + scope-confirm + operator-in-loop) after ratatoskr's verify.
worldtree-dev pushed the train (main + tags v1.0.0b168/staging) and verified my
pre-sync matches the commit byte-for-byte. Recorded the next pending op: after
b168 CI-lands on personal, run --reindex of job mimir-6351554e8e8f + #381 restart
on their exact commands.
Synced the boot-gating summarization block + deep-reasoning desc to demo +
personal via deploy-wt-config (health-gated); replied GO. Recorded gotchas:
unpushed-commit fetch falls back to default branch (verify SHA exists), and
paste-sourced YAML needs indent/validate (8-space desc was invalid).
worldtree-dev advance notice: #384/#385 train adds a boot-blocking
'summarization' capability block (vastblueai-gateway/gen) that in-image
muninn/config.yaml references. Must pre-sync into providers.yaml on demo
+ personal via deploy-wt-config BEFORE the image deploy, from the exact
commit. ACKed; recorded as an open watch.
worldtree-dev gave advance notice: standing #381 restart of
worldtree-personal-worldtree-api-1 will be requested when DCC re-ingest
job mimir-6351554e8e8f (fiction wing) goes terminal in a few hours.
ACKed no scheduling constraints; recorded as an open watch so a fresh
session catches the request.
A/B (5 pinned-seed neutral pairs) showed original single-clip seg000
(16.3s) beats the 52s 4-take concat on timbre fidelity — reverted live +
build-source to seg000-alone. Recorded two craft lessons: more reference
audio is not automatically better when takes vary; emotion steering pulls
output away from a cloned voice fast (keep clones emotion-neutral).
vh