`[2026-09-25]` nh3-ml1 — second embed/rerank GPU LXC on nh3-pve (DEFERRED: Secure Boot)

Ask (Prime, ~1410 PT): "stand up an nh3-ml1 that takes advantage of that GPU" (the RTX 2000E Ada installed in nh3-pve that morning, 01:00.0).

Decisions taken on Prime's behalf (he said "clear to proceed" at ~1418)

Purpose: the second TEI embed/rerank backend. esh-ml1's README had been waiting on "the second RTX 2000" for failover. No reward-seat copy is planned, because that seat has no working consumer.
Placement: CT 109, 10.100.50.80/24, VLAN 50, gw .1, rootfs on local-zfs, 6 cores / 16 GB / 80 GB, startup order 30. The address came from the UDM (nh3-servers DHCP pool .150–.249; in use .40 .42 .45 .46 .50 .60 .71 .90). .80 mirrors esh-ml1. nh3-pve had 40 of 62 GB RAM free.
Not backed up, like esh-ml1. nh3-pve's vzdump job is all 1 (exclude 107), so gpu-lxc.yaml adds the CT to that job's exclude list. The guard and the step were tested against sample configs: one positive case, plus negatives for already excluded, a vmid-list job, a non-vzdump all 1 stanza and a missing file.
Playbooks made host-generic (bc278d4): esh-pve-nvidia-host → pve-nvidia-host, which gained headers/dkms/build-essential, a nouveau blacklist and an unload that refuses if nouveau has bound a device. esh-ml1-lxc → gpu-lxc, whose host vars have no defaults; elway aborts on an undefined {{ var }} (tested). Per-host --var lines are in its header.
stacks/embed-rerank Homepage labels take HOST_NAME / HOST_IP. The defaults render byte-identical to esh-ml1's live labels, so nothing is recreated there.

The first pve-nvidia-host run died on host-key verification for nh3-pve.nh3.internal. Every when: got ssh rc 255 and SKIPPED, and verify failed. I pinned the name after confirming its ed25519 key equals the known IP entry.
The second run installed headers 6.8.12-11 + proxmox-headers-6.8 + dkms + build-essential, blacklisted and unloaded nouveau (it had failed its probe with gsp ctor failed: -2 and was bound to nothing), then the DKMS module was refused at load: Secure Boot is ON (mokutil --sb-state enabled; lockdown integrity; the only MOK is the Proxmox Secure Boot CA). esh-pve, the same MS-01 and BIOS, has Secure Boot OFF. The installer rolled back: no dkms entry, no nvidia-smi. All guests were unaffected.
7ddd116 added a pre-flight that refuses when Secure Boot is on without an enrolled /var/lib/dkms/mok.pub. Tested: it refuses on nh3-pve and passes on esh-pve.

Secure Boot: (1) turn it off in the BIOS on the NH3 visit (recommended; one more toggle in the menu he is already in for IGFX, and it matches esh-pve); (2) keep it and enroll a DKMS MOK (prep remotely, then answer MokManager at boot; DKMS 3.x then signs rebuilds itself); (3) a VFIO VM (works without a visit, but reverses his 09-24 "LXC + host driver, not VFIO" rule, and the RTX is currently the boot VGA). Options 1 and 2 need the console at boot, which is blind until IGFX is set, so the visit carries both.
Gateway routing once parity passes: recommendation load-share. The two cards are identical, and the old "don't load-share" rule came from pairing esh-ml1 with the much faster fv-ml1.

The Current-state checklist, step 9. Nothing host-side exists yet for CT 109.