diff --git a/persistent-memory.md b/persistent-memory.md index 92b634f..a54ee87 100644 --- a/persistent-memory.md +++ b/persistent-memory.md @@ -139,9 +139,12 @@ NH3 DNS are all fine. Open items: **nh3-ml1 (Prime, 2026-09-25 ~1410): IN PROGRESS.** CT 109 on nh3-pve at 10.100.50.80 (VLAN 50, outside the UDM pool .150–.249), local-zfs, a second embed/rerank (TEI) backend on the new RTX. Playbooks are host-generic as of -`bc278d4`. **Blocked:** the permission classifier refused the host driver install -(`pve-nvidia-host.yaml` on nh3-pve), and nothing host-side has run. Prime has to -run it or allow it. After it: `gpu-lxc.yaml` with nh3 vars → embed-rerank with +`bc278d4`. Prime cleared the driver install at ~1418. **It then failed: nh3-pve has +SECURE BOOT ON** (esh-pve has it off), and the unsigned DKMS module was refused. +The installer rolled back. The playbook now refuses on this up front. Unblocking +needs the console at boot (MokManager enroll, or SB off in the BIOS), which ties +it to the IGFX/NanoKVM visit, unless Prime picks a VFIO VM. Waiting on his call. +After it: `gpu-lxc.yaml` with nh3 vars → embed-rerank with `HOST_NAME/HOST_IP` → parity vs esh-ml1 → monitoring and DNS → gateway routing (Prime's call). diff --git a/playbooks/pve-nvidia-host.yaml b/playbooks/pve-nvidia-host.yaml index 2cf61c6..74193c5 100644 --- a/playbooks/pve-nvidia-host.yaml +++ b/playbooks/pve-nvidia-host.yaml @@ -33,6 +33,13 @@ # module holds nothing and unloads cleanly. The unload step refuses if nouveau # has actually bound a device; then the blacklist plus a reboot is the only way. # +# ⚠ SECURE BOOT: a DKMS-built module only loads under Secure Boot if its signing +# key is enrolled as a MOK, and enrolling needs a person at the console at boot +# (MokManager). esh-pve has Secure Boot OFF. nh3-pve has it ON (lockdown = +# integrity), and the 2026-09-25 install failed there with "module failed to load". +# The installer rolled itself back. The first step below now refuses early +# instead of getting that far. +# # Upgrading the driver: bump driver_version + driver_sha256 here AND # driver_version in playbooks/gpu-lxc.yaml, run this on every GPU host, then # gpu-lxc.yaml for every GPU LXC. Until both have run, CUDA in the container @@ -50,6 +57,18 @@ vars: gpu_match: RTX 2000 steps: + - name: Pre-flight — Secure Boot off, or the DKMS signing key already enrolled + shell: | + if mokutil --sb-state 2>/dev/null | grep -q 'SecureBoot enabled'; then + if [ -f /var/lib/dkms/mok.pub ] && mokutil --test-key /var/lib/dkms/mok.pub 2>&1 | grep -q 'already enrolled'; then + exit 0 + fi + echo "Secure Boot is ON and no enrolled DKMS key: the nvidia module would be refused." >&2 + echo "Needs the console at boot: disable Secure Boot in the BIOS, or enroll /var/lib/dkms/mok.pub via MokManager." >&2 + exit 1 + fi + changed_when: "false" + # DKMS needs the headers for the RUNNING kernel (built now) and the series # meta-package (so each future kernel's headers arrive with it and DKMS # rebuilds on upgrade). nh3-pve had neither, nor dkms or a compiler. diff --git a/servers/nh3-pve/README.md b/servers/nh3-pve/README.md index f5d1bc6..9e032c9 100644 --- a/servers/nh3-pve/README.md +++ b/servers/nh3-pve/README.md @@ -90,6 +90,14 @@ not power back on by itself. Display = IGFX (or enable iGPU Multi-Monitor). Reaching the BIOS now needs a display on the RTX's mini-DP, for example the NanoKVM through a mini-DP→HDMI adapter. Verify with `lspci | grep 00:02.0` and `boot_vga` on `00:02.0`. +- ⚠ **Secure Boot is ON here** (`mokutil --sb-state`, lockdown `integrity`; + enrolled MOK = the Proxmox Secure Boot CA). esh-pve, the same model, has it + OFF. Any DKMS-built module, the NVIDIA driver included, is refused until its + key is enrolled through MokManager at boot or Secure Boot is turned off in the + BIOS. Both need the console, and the console is blind (below). The 2026-09-25 + NVIDIA install failed on this and rolled itself back. What it left, all + harmless: headers `6.8.12-11` plus the series meta, dkms and build-essential; + nouveau blacklisted and unloaded; the `.run` staged in `/root/nvidia`. - **OOB decision (Prime via Miranda, 2026-09-25): HOLD. nh3-pve stays console-blind until the next NH3 site visit.** No BIOS change and no reboot until then. The accepted risk: a boot without network means a site trip. **Target end state:**