vh
b6340519bc
Session captured for a context reset. Six new detail entries. THE OPEN ITEM: Anaheim's IPsec tunnel delivers ~550 Mbit/s aggregate against a circuit measured at 2,153 Mbit/s. Not WireGuard (it is IPsec on ana-gw), not CPU (idle), not crypto exhaustion (NPU-offloaded), not the fibre. Both tunnels negotiate aes256-sha1; AES-GCM proposed. Operator signalled authorization; execution pending, untracked by operator choice. Also recorded: selene retired after losing a head-to-head on its own job with chat-judge moved to gen and the model name left to 404; the 7-alias collision on the gen seat that makes cross-alias corroboration an echo; hrafn adopted and its CI found to have been reporting green while deploying nothing for its whole life; all three Worldtree instances de-armed from a 69-day-stale :latest and the Matrix homeserver re-plumbed to personal; every secret-bearing .env on ana-docker tightened to 0600; the pfi org closing the repo-creation half of the credential-migration directive; booth kept-board deletion and link pruning. Two entries under Tried and abandoned: the CI checkout assertion that broke the pipeline twice and was removed, and my proposal to alias a retired model name at a different model, which the operator correctly overruled. Index 271 -> 282 lines, under the 300 cap, so no archival fired.
2.2 KiB
[2026-08-23] /mnt/smithy mounted on ana-ml2 — read-only and SOFT, deliberately not matching nh3-dev
brokkr-smithy-dev asked for 10.100.50.50:/volume1/smithy on ana-ml2 to run R47's
CPU-bound corpus pipeline on 96 idle EPYC cores instead of one nh3-dev vCPU. Granted,
with two deliberate deviations from what was requested.
sudo mount -t nfs4 -o ro,soft,timeo=30,retrans=3,proto=tcp,vers=4.1 \
10.100.50.50:/volume1/smithy /mnt/smithy
The export already permitted ana-ml2 — no DSM change needed. Write is genuinely refused.
Why soft, not hard
They asked to match nh3-dev's mount, which is hard. nh3-dev is same-site as the
NAS; ana-ml2 is not — this is cross-site NFS on the box running the fleet's
inference seats. A hard mount turns a link blip into unkillable D-state, and this
fleet has already lost a host that way (esh-docker-vm; only fix was a reboot). Soft
returns EIO, the batch job fails, you rerun it. The soft-mount corruption caveat is a
write hazard and this is read-only. Mirrors the existing ESH books mount.
Why not in fstab
Manual only, matching irv-ml1's /mnt/smithy precedent. A cross-site NFS entry in
fstab can hang boot on a GPU host with 71 days uptime. Needs remounting after a
reboot.
The performance reality, measured on the same file through the same mount
ana-ml2 (cross-site) nh3-dev (same-site)
sequential read 24.7 MB/s 98.3 MB/s
small-file rate 45.3 files/s 34.6 files/s
Two different stories, and file layout decides which you get:
- Many small records -> ana-ml2 wins on BOTH axes. That path is bound by per-file round-trips and NAS overhead, not bandwidth, and ana-ml2 is an idle 96-core box while nh3-dev is a loaded 16-vCPU VM.
- Bulk sequential streaming -> the link eats the win. 4x read penalty against a
6x CPU gain.
datasets/rawis 126 GB,datasets/derivedis 1.7 GB — which one the pipeline traverses changes the answer by two orders of magnitude. Staging a subset to ana-ml2 local disk (195 GB free) beats pulling it over the wire repeatedly.
The 24.7 MB/s is the Anaheim tunnel, not NFS and not the NAS — see 2026-08-23-anaheim-ipsec-tunnel-ceiling. No mount tuning will move it; parallelism will.