vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
d3f59aa2c541a8b571300ad536bbf15ed21fac2b
esh-pfi-infrastructure/servers/nh3-pve-2/README.md
T
vh d3f59aa2c5 feat(albok): move albok-service to a dedicated VM, nh3-albok (VM 110 on nh3-pve-2) 
albok-service leaves nh3-docker before unit 4 puts the personal vault and
licensed books into restricted wings. On nh3-docker, portainer, portainer_agent,
dockge (socket rw) and dockerd on plaintext tcp/2375 were each root over
/srv/albok. The new host runs only albok-service: no socket containers and no
TCP Docker API, so its Homepage tile is a static services.yaml entry.

- nh3-pve-2: vmbr0 made VLAN-aware (applied by reboot, no guests running);
  pbs-ana storage in the new namespace nh3-pve-2, plus a 21:30 vzdump job
  (pbs-nh3's recursive sync mirrors it)
- VM 110 nh3-albok, 10.100.50.44 on VLAN 50, Debian 13 cloud-init, 4 vCPU
  x86-64-v3 / 12 GiB / 100 GB; built to fleet conventions (vh 1000,
  infra-ops 850, docker 851)
- compose: mem_limit 4g -> 8g, homepage labels removed; host port 8392 kept
- cutover by tar stream, verified by digest (contents plus owner, group, mode,
  mtime and type identical); albok.nh3.internal now points at nh3-albok
- old /srv/albok stays on nh3-docker (container removed, compose renamed)
  until acceptance and the operator's approval to delete it
- sudo-io archive hosts and ops-log STACK_HOSTS include nh3-albok
2026-10-04 23:33:24 -07:00

4.0 KiB
Raw Blame History

nh3-pve-2: Minisforum MS-03, Proxmox VE 9 (NH3)

The NH3 MS-03, twin of esh-pve-2. Purpose is TBD on purpose (Prime, 2026-10-02): a high-powered PVE host, possibly a dev environment for security-software work. First guest: nh3-albok (2026-10-04, below). Standalone node, not clustered with nh3-pve.

  • Host: 10.100.250.62 on nh3-mgmt (nh3-pve-2.nh3.internal, nh3-pve-2.phasefinal.com in /etc/hosts). Static in /etc/network/interfaces, plus a UDM reservation on 38:05:25:3b:a0:a8. ssh infra-ops@10.100.250.62 (fleet key, NOPASSWD sudo with log_output).
  • PVE 9.2.21, installed 2026-10-03 on site to the right disk (the 256 GB Toshiba). Follow-ups DONE 2026-10-04: Prime rebooted it at 0724 onto 7.0.14-20; verified 1635: .62 on vmbr0, nic0 up at 1 Gb with arp_ignore=8 + IPv6 off, MeshCentral shows nh3-pve-2-amt connected (CIRA, power on), ceph + pve enterprise sources Enabled: false, 0 upgradable, no failed units.

Guests

ID guest onboot notes
110 nh3-albok (10.100.50.44, VLAN 50) 1 First guest, 2026-10-04 (albok-dev relaying Prime): runs only albok-service. servers/nh3-albok/README.md

VMIDs continue the NH3 numbering (nh3-pve holds 100–109) so a VMID names one guest on the site.

Backups: storage pbs-ana (10.250.50.90, datastore backups, namespace nh3-pve-2, token root@pam!fleet-vzdump, secret vaulted as pbs-ana/fleet-vzdump-token) and vzdump job 21:30 nightly, all 1, snapshot mode, fleecing on vmstore, the same prune as nh3-pve. 30 min after nh3-pve's 21:00 so the two do not contend on pbs-ana. pbs-nh3's sync (root namespace, recursive) mirrors the new namespace without any change on the DR side. ⚠ pvesm add pbs refuses without --password on the command line (it test-logs in), which would put the token in the sudo log. The stanza was appended to /etc/pve/storage.cfg and the secret written to /etc/pve/priv/storage/pbs-ana.pw (37 bytes, value + newline) with sudo dd instead.

NIC MAC Chip Use
nic0 :a6 I226-LM (igc) Intel AMT only. UDM port 5, native nh3-mgmt, AMT DHCP .63. auto nic0, no address, arp_ignore=8, IPv6 off (/etc/sysctl.d/90-amt-port.conf)
nic1 :a9 RTL8127 10G unused
nic2 :a7 X710 SFP+ unused
nic3 :a8 X710 SFP+ vmbr0. USW Pro 24 port 25 (native nh3-mgmt, tagged VLANs allowed)

⚠ Keep auto nic0. If Linux leaves the AMT port down, igc powers off the PHY and AMT goes dark. That happened 2026-10-03 when the bridge moved to the SFP+. vmbr0 is VLAN-aware since 2026-10-04 2311 (bridge-vlan-aware yes, bridge-vids 2-4094, same as nh3-pve), applied by playbooks/nh3-pve-2-vmbr0-vlan-aware.yaml + a reboot with no guests running. The host address stays untagged (PVID 1 = the port's native nh3-mgmt); guests take tag=<vlan>. Pre-change file kept at /etc/network/interfaces.pre-vlan-aware-2026-10-04. ⚠ Change this bridge only by reboot, never a live ifreload: on 2026-10-03 a live ifreload that also re-IP'd read the new vmbr0 as "not a bridge" and downed it (auto-memory feedback_no_remote_reip_thrash_when_operator_has_console). ⚠ ifup -a -s exits 1 on this host even for a good file: it warns on the PVE-default bridge-fd 0. Compare its output against the previous file's rather than trusting the exit code.

Disks: boot = Toshiba 256 GB (69CA12VQK65N). VM storage = LVM-thin vmstore, 913 GiB, on the Crucial 1 TB (2610EAD05AB7). It held the old wrong-drive install, which was wiped 2026-10-03 by playbooks/nh3-pve-2-disk-prep.yaml, along with its stale firmware boot entry. Firmware boot order: 0000 proxmox (256 GB), then its fallback 0006. Address disks by /dev/disk/by-id/, because nvme numbering can swap between boots (seen on esh-pve-2).

AMT: MeshCentral device nh3-pve-2-amt (phone-home, AMT 21.0.6, ACM; password vaulted nh3-pve-2/amt-admin). If HW Connect sticks at "Setup…", cira-tunnel-watchdog on pfi-tacticalrmm clears it within ~4 min.

Reference in New Issue View Git Blame Copy Permalink