Files
esh-pfi-infrastructure/servers/pfi-gx10/README.md
T
vh a0c5fc6ed5 feat(pfi-gx10): rack-move network playbook — VLAN 50, static 10.100.50.60
Target settled: nh3-servers VLAN 50, static 10.100.50.60. Clear of the four
existing statics and below the .150 DHCP pool where fleet statics live.

The playbook never leaves itself one path back. Wi-Fi stays up throughout while
the wired interface is configured beside it; the new address is verified from
outside before anything is torn down, and Wi-Fi teardown is explicitly a
separate later change. A botched netplan therefore costs a retry over Wi-Fi
rather than a trip to the rack — which is what substitutes for 'netplan try',
whose interactive rollback needs a TTY that elway cannot provide.

Two preconditions are asserted as steps rather than assumed: the interface must
have carrier (writing a static config for a dead NIC and reporting success is
the failure this avoids), and its MAC must match, since interface names can
renumber across kernels but MACs do not.

Requires nothing from the operator beyond racking the box. The wired NIC has a
distinct MAC from the Wi-Fi one, so the post-move address and switch port are
both discoverable from the UDM rather than needing to be relayed.
2026-09-01 11:41:43 -07:00

3.2 KiB
Raw Blame History

pfi-gx10 — ASUS Ascent GX10 (NVIDIA GB10)

Grace-Blackwell desktop supercomputer. Registered 2026-09-01.

GPU NVIDIA GB10, driver 580.173.02, compute capability 12.1 (sm_121)
CPU 20 cores, aarch64
Memory 121 GB unified (CPU and GPU share it — not 121 GB plus VRAM)
Storage 916 GB NVMe, 6% used
Kernel 6.17.0-1031-nvidia
Hostname pfi-gx10 (shipped with static gx10-a745, corrected)

⚠ The address in ssh-target is TEMPORARY

As of 2026-09-01 this box is on a desk, on Wi-Fi, holding a DHCP lease at 10.100.10.226 on nh3-userland (VLAN 10). Ethernet enP7s7 has no carrier.

It is going into the rack later. Target settled (operator, 2026-09-01): nh3-servers, VLAN 50, static 10.100.50.60 — clear of .40 nh3-docker, .42 nh3-extdev, .50 nh3-nas, .90 pbs-nh3, and below the .150 DHCP pool where fleet statics live.

Nothing was configured on the desk — an address that is about to be wrong is worse than DHCP. The move is playbooks/gx10-rack-network.yaml.

Nothing is needed from the operator beyond racking it. The wired NIC has its own MAC (30:c5:99:3d:a7:45, distinct from the Wi-Fi 50:bb:b5:a2:00:a8), so its post-move address is discoverable from the UDM without being told:

curl -sk "https://10.100.0.1/proxy/network/api/s/default/stat/sta" \
  -H "X-API-KEY: $(secret get unifi/pfi-udmse-api-key)" \
  | python3 -c "import json,sys;[print(c['ip'],c.get('sw_port')) for c in json.load(sys.stdin)['data'] if c['mac']=='30:c5:99:3d:a7:45']"

That also returns the switch port, which must be set to the nh3-servers network or the box lands back on VLAN 10.

Expect ~65–80 ms RTT until it is wired. That is Wi-Fi power-save, not a fault.

Access

infra-ops with NOPASSWD sudo (operator-bootstrapped). lkraven also has key auth but needs a password for sudo — automation must connect as infra-ops.

Headless conversion

playbooks/gx10-headless.yaml — run it with the infra-ops@ prefix, since elway's --sudo applies only to ad-hoc commands and playbook steps carry their own.

Ships booting to graphical.target with GDM and GNOME Remote Desktop. The playbook sets multi-user.target, stops the remote-desktop service, masks the sleep/suspend/hibernate targets, tells logind to ignore lid and idle, and adds sshd keepalives so a stalled link does not kill a long job.

⚠ GDM is static on Ubuntu — pulled in by display-manager.service, never "enabled". Guard and verify on is-active, not is-enabled; the latter passes trivially while the desktop is still running.

The playbook will not stop GDM while someone holds a seat session. Override with --var force_dm_stop=true, or just let the rack-install reboot handle it.

Relevance to Flash-Next

sm_121, not sm_120. The SGLang fork evaluated for ana-ml2 (henge item 49) narrows to exact SM120 and explicitly excludes SM121/GB10 — it does not apply here. This chip has its own path: the DGX Spark recipe, which mmaps the ~48 GiB PLE n-gram table from NVMe rather than holding it in memory. 121 GB unified and 822 GB of free NVMe make that viable on this box in a way it is not on a 96 GB discrete card.