52 lines
4.1 KiB
Markdown
52 lines
4.1 KiB
Markdown
# nh3-pve-2: Minisforum MS-03, Proxmox VE 9 (NH3)
|
||
|
||
The NH3 MS-03, twin of esh-pve-2. **Purpose is TBD on purpose** (Prime, 2026-10-02): a high-powered PVE host,
|
||
possibly a dev environment for security-software work. First guest: nh3-albok (2026-10-04, below). Standalone node, not clustered with nh3-pve.
|
||
|
||
- **Host:** `10.100.250.62` on nh3-mgmt (`nh3-pve-2.nh3.internal`, `nh3-pve-2.phasefinal.com` in /etc/hosts). Static
|
||
in `/etc/network/interfaces`, plus a UDM reservation on `38:05:25:3b:a0:a8`. `ssh infra-ops@10.100.250.62`
|
||
(fleet key, NOPASSWD sudo with `log_output`).
|
||
- PVE 9.2.21, installed 2026-10-03 on site to the right disk (the 256 GB Toshiba). **Follow-ups DONE 2026-10-04:** Prime rebooted it at 0724 onto 7.0.14-20; verified 1635: `.62` on vmbr0, `nic0` up at 1 Gb with `arp_ignore=8` + IPv6 off, MeshCentral shows `nh3-pve-2-amt` connected (CIRA, power on), ceph + pve enterprise sources `Enabled: false`, 0 upgradable, no failed units.
|
||
|
||
## Guests
|
||
|
||
| ID | guest | onboot | notes |
|
||
|---|---|---|---|
|
||
| 110 | **nh3-albok** (`10.100.50.44`, VLAN 50) | 1 | First guest, 2026-10-04 (albok-dev relaying Prime): runs only albok-service. `servers/nh3-albok/README.md` |
|
||
| 111 | **nh3-svos** (`10.100.50.43`, VLAN 50) | 1 | 2026-10-05 (svos-dev relaying Prime): the new SVOS OS. `servers/nh3-svos/README.md` |
|
||
|
||
VMIDs continue the NH3 numbering (nh3-pve holds 100–109) so a VMID names one guest on the site.
|
||
|
||
**Backups:** storage `pbs-ana` (10.250.50.90, datastore `backups`, **namespace `nh3-pve-2`**, token
|
||
`root@pam!fleet-vzdump`, secret vaulted as `pbs-ana/fleet-vzdump-token`) and vzdump job **21:30 nightly, `all 1`**, snapshot
|
||
mode, fleecing on `vmstore`, the same prune as nh3-pve. 30 min after nh3-pve's 21:00 so the two do not contend on pbs-ana.
|
||
pbs-nh3's sync (root namespace, recursive) mirrors the new namespace without any change on the DR side.
|
||
⚠ `pvesm add pbs` refuses without `--password` on the command line (it test-logs in), which would put the token in the
|
||
sudo log. The stanza was appended to `/etc/pve/storage.cfg` and the secret written to `/etc/pve/priv/storage/pbs-ana.pw`
|
||
(37 bytes, value + newline) with `sudo dd` instead.
|
||
|
||
| NIC | MAC | Chip | Use |
|
||
|---|---|---|---|
|
||
| `nic0` | `:a6` | I226-LM (igc) | **Intel AMT only.** UDM port 5, native nh3-mgmt, AMT DHCP `.63`. `auto nic0`, no address, `arp_ignore=8`, IPv6 off (`/etc/sysctl.d/90-amt-port.conf`) |
|
||
| `nic1` | `:a9` | RTL8127 10G | unused |
|
||
| `nic2` | `:a7` | X710 SFP+ | unused |
|
||
| `nic3` | `:a8` | X710 SFP+ | **vmbr0**. USW Pro 24 port 25 (native nh3-mgmt, tagged VLANs allowed) |
|
||
|
||
⚠ **Keep `auto nic0`.** If Linux leaves the AMT port down, igc powers off the PHY and AMT goes dark. That
|
||
happened 2026-10-03 when the bridge moved to the SFP+.
|
||
**`vmbr0` is VLAN-aware since 2026-10-04 2311** (`bridge-vlan-aware yes`, `bridge-vids 2-4094`, same as nh3-pve), applied by
|
||
`playbooks/nh3-pve-2-vmbr0-vlan-aware.yaml` + a reboot with no guests running. The host address stays untagged (PVID 1 =
|
||
the port's native nh3-mgmt); guests take `tag=<vlan>`. Pre-change file kept at `/etc/network/interfaces.pre-vlan-aware-2026-10-04`.
|
||
⚠ Change this bridge only by reboot, never a live `ifreload`: on 2026-10-03 a live ifreload that also re-IP'd read the new
|
||
vmbr0 as "not a bridge" and downed it (auto-memory `feedback_no_remote_reip_thrash_when_operator_has_console`).
|
||
⚠ `ifup -a -s` exits 1 on this host even for a good file: it warns on the PVE-default `bridge-fd 0`. Compare its output
|
||
against the previous file's rather than trusting the exit code.
|
||
|
||
**Disks:** boot = Toshiba 256 GB (`69CA12VQK65N`). **VM storage = LVM-thin `vmstore`, 913 GiB**, on the Crucial 1 TB
|
||
(`2610EAD05AB7`). It held the old wrong-drive install, which was wiped 2026-10-03 by `playbooks/nh3-pve-2-disk-prep.yaml`,
|
||
along with its stale firmware boot entry. Firmware boot order: `0000 proxmox` (256 GB), then its fallback `0006`.
|
||
Address disks by `/dev/disk/by-id/`, because nvme numbering can swap between boots (seen on esh-pve-2).
|
||
|
||
**AMT:** MeshCentral device `nh3-pve-2-amt` (phone-home, AMT 21.0.6, ACM; password vaulted `nh3-pve-2/amt-admin`).
|
||
If HW Connect sticks at "Setup…", `cira-tunnel-watchdog` on pfi-tacticalrmm clears it within ~4 min.
|