The in-flight section was a day stale: it still described run 3c as the
live subject on a box where nothing had moved. Rewritten around what is
actually true now -- v3 deployed fleet-wide, the post office relocated
to nh3-docker, sec serving on GPU0, run 3c still held on power.
Six new decision entries, three of which carry findings that outlive
their incident: the inbound half of the handle-resolution bug (a stale
ALTHING_HANDLE reads another agent's mailbox and reports it empty,
which is a second route into the failure v3 exists to prevent), the OOM
attribution to Claude Code sessions, and the operator's two explicit
belays recorded so a later session does not re-raise them as new.
Auto-archival fired at 301 lines and moved exactly one entry. Three
others were old enough and every one carries a still-open deferred
pointer -- the parked CI flip, muninn-gate's submit path, and the triton
backend deferred to the Ada refresh. Held back per the guards; an
over-cap file that keeps live decisions beats a scannable one that lost
them. The entry that did move had its deferred item closed today:
nh3-extdev's staged v2.1.2 wheel is moot now that the box runs 3.1.1.
Run 3 trained, gated and dispositioned do-not-serve on a measured 44pp self-harm
guardrail regression that its own preregistered rule passed -- a pooled preserve-list
test cannot see a single-axis collapse. Run 3c (lr 20x cut, single variable) launched,
killed by an Anaheim power-breaker trip at step 80, relaunched, then stopped by the
operator at step 22 pending a weekend power triage.
Also captured: the corpus mix was specified in a unit the optimiser never sees
(45.8% dialogue by context, 24.2% by loss); the dose-response says benefit and damage
are one direction in weight space, so the merge-back measures the problem rather than
fixing it; four guests including the storage SPOF had onboot unset and never came back
from the outage, now fixed with dependency ordering; and a transport failure that
enters a measurement as a value looks like whatever you hoped to find -- which found a
live defect in another agent's instrument an hour after it was reported.
Auto-archived 8 entries to archival-memory.md (Recent decisions: 8, Tried and
abandoned: 0); 4 held back on open deferred-work pointers.
Captures an evening that ran from an OOM crash-loop to a measured trainee base.
The durable lessons, none of which CLAUDE.md can carry: --gpu-memory-utilization
sizes the KV cache and does not cover CUDA context or graphs, which is half of
why a seat that fit on the 21st stopped fitting on the 24th; the other half is
that gen's footprint GROWS WITH UPTIME (38.5 GiB fresh against 45.6 GiB after
three days, same container, same flag), so headroom arithmetic against a
long-running gen measures a moving number. The stale-chat-template trap turned
out to be endemic across third-party Gemma-4 derivatives rather than a one-off,
and it is silent in both directions — wrong prompt when serving, train/serve
skew when tuning. And a benchmark finding was retracted because 12% on a
five-option task is below the 20% chance floor: a below-chance score indicts the
instrument before the model, and a preflight can be thorough while aimed in the
wrong direction.
Records the serving decision for the tuned model with its history intact:
LoRA-on-NVFP4 is preferred if it works, merged weights the expected fallback,
but the archived root-cause says the objection was never NVFP4-specific — vLLM
0.24.0's LoRA application was a silent no-op proven quant-agnostic, and ana-ml2
now runs 0.26.0. Retest before designing around merge; the answer changes what
Eitri's harness must emit, and he is still early.
Auto-archival moved 5 entries (Recent decisions) to archival-memory.md; the
guards held back the rest of the 78 age-eligible candidates because their bodies
carry open deferred-work language, per the keep-when-unsure rule. Index sits at
286 lines, above the ~250 target and reported rather than forced.
Rewrites the in-flight section (140 lines to 18) now that the AI-seat work
has settled, promoting the durable calls into the dated decision log where
they will not decay: DFlash2 measured and promoted to sec's compose stack,
the calibration-truncation defect fixed at source in the quant pipeline,
sec retuned after a runtime OOM, the Max-Q spread disproved by telemetry,
ESH dual-stack with the Cityside static proven unprovisioned, and speaches
live with the weak-gate finding.
Carries forward the two things a fresh session must not get wrong: the sec
degeneration question is confounded because engine and drafter changed
together, and neither the 2k nor the 10k observation is evidence.
Auto-archived one entry (Recent decisions: 1) to archival-memory.md.
Eight further candidates were held back by the guards -- open deferred work
or referenced by in-flight state.
Captures the session's real work as the in-flight focus: abliterating
DavidAU Cold-Fusion with the Robinson formula. fp32 capture succeeded
(finite direction, layer 22, sink-clean) but two-template agreement is
0.59 vs Robinson's 0.99 — calibration-set expansion is the next step.
New detail file records the full saga including the transformers/DeltaNet
bf16-NaN fight (fp32 fix, the causal-conv1d kernel gap, the seat-restart
VRAM-greed gotcha). Supersedes the earlier "watch for DavidAU's heretic
build" posture — we abliterate it ourselves.
Auto-archived 4 closed entries (Recent decisions: Booth-3-features 08-05,
worldtree-sdk 07-31; Tried and abandoned: containerd-race 08-03,
mv-rename 08-02) to archival-memory.md; the rest of the over-cap entries
are held back by the <14-day and open-deferred guards. Index 331 -> 327.
Captures a long infra session: fleet *.internal DNS (git-sourced, 42 names,
three resolvers including a new colo one), waterland studio containerised on
irv-ml1, Homepage cleaned up and themed with Australis Skyfall over an
Arbo-generated background, and four unmanaged stacks adopted into stacks/.
Four detail files added. Auto-archived 4 entries to archival-memory.md
(Recent decisions 2, Tried and abandoned 2); 5 held back by the open-deferred
guard rather than moved.
Also records three operator-owned open items: the colo DNS repoint, the
static-v6 convention, and the deliberately belayed AI-tab Dormant regrouping.
Session captured: ESH cut over to Cityside 2Gb symmetric fiber and was
fully provisioned on it; esh-pve-nas completed its ZFS-root migration off
the USB DOM and took its 225-package security backlog with the reboot
deferred; ESH<->colo IPsec was rebuilt as a dialup tunnel with NAT-T after
CGNAT broke the statically-pinned one; IPv6 was mapped across all three
sites.
Auto-archival fired at the soft cap: 7 entries moved to
archival-memory.md (Recent decisions 3, Tried and abandoned 4), all
verified-complete arcs, with two detail files moved and removed. The
remaining pre-Aug-05 entries were held back by the open-deferred-work
guard, so the index stays slightly over cap at 313 lines rather than
losing a live pointer.
Also records the one self-inflicted outage of the session (missing
--make-rslave on a chroot rbind) and that three long-dead things
surfaced incidentally: pvestatd down 82 days, a vzdump hung 126 days,
and a VM sitting in prelaunch for four months.
Current state rewritten to session-end: secrets-broker LIVE (secret CLI + 25-item
backfill + rm/warn), stonehenge-park /park service repo stood up (vor-plan+vor-ui),
dots-tts extracted to tts-stack; NEXT PROJECT = wgtunnel (WireGuard over a plane's
wifi). Two detail files added; 2 oldest T&A entries archived. Handoff written to
/tmp/infra-ops-handoff.md for the post-clear pickup.
Session (2026-08-05): 3 Booth features live+tagged (verbatim-wrap chip, .md/.txt
doc-viewer, image prev/next arrows); worldtree herald re-nudge bug -> forseti
althing-core v2.1.2; fleet-CI-resilience flip attempted end-to-end and PARKED on
an act_runner->gitea action-fetch auth blocker (infra-ops to research, deferred).
Archived the 2026-07-15/16 recent-decisions batch (8 entries) to keep the index
under the soft cap.
Collapsed the completed-arc bloat out of Current state / in-flight (line 125:
6487->1073 chars; runbooks moved to persistent-memory.d/2026-08-03-worldtree-b168-384-385-arc.md).
Added the 2026-08-03 arc Recent-decisions entry. Auto-archived 20 oldest
Recent-decisions entries (2026-07-08..07-14) to archival-memory.md (index 345->308
lines). Refreshed the b169-deploy watch, unpushed-count, and handoff.
9 Recent decisions + 6 Tried-and-abandoned (dates [2026-06-14]/[2026-06-15]) moved
non-destructively to archival-memory.md, each stamped _Archived 2026-06-19._. Kept
the active [2026-06-14] 'migrate ALL infra access to Claude-specific credentials'
standing directive. Back-ref counts: Recent decisions 79->88, Tried-and-abandoned
70->76. persistent-memory.md 397->363 lines.
Capture this session's durable state: Dia2 live as dia2-2b/dia2-1b (legacy
dia retired), dia2 catalog on /tts with full controls + stable-voice default,
zonos REST adapter (built, stack down), schema regen, all pushed. New
foot-guns: dia2 image-build quirks, predefined-voices-not-in-/tts, voice_mode
clone 'undefined' 404. Archived 6 older entries (4 decisions, 2 foot-guns).
vh