vh/esh-pfi-infrastructure
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(elway): probe NOPASSWD with sudo -n true, never sudo -n -v

Browse Source 
`sudo -v` refreshes the auth timestamp, and a NOPASSWD-only rule creates
no timestamp to refresh, so on sudo >= 1.9.15 `sudo -n -v` returns
non-zero while every real command runs passwordless. Measured:

  pfi-gx10    sudo 1.9.15p5   sudo -n -v rc=1   sudo -n true rc=0
  nh3-docker  sudo 1.9.13p3   sudo -n -v rc=0   sudo -n true rc=0
  ana-docker  sudo 1.9.13p3   sudo -n -v rc=0   sudo -n true rc=0
  irv-ml1     sudo 1.9.13p3   sudo -n -v rc=0   sudo -n true rc=0

Only pfi-gx10 is new enough to hit it today, but every host does as it
moves past 1.9.13, and the failure mode is bad: elway prompts for a
password on a host with working NOPASSWD sudo, which in a
non-interactive run is an EOFError partway through a playbook.

The same probe in my own notes cost this session directly. gx10 looked
like a fleet exception with no NOPASSWD sudo when it had it from account
creation, and the operator was asked for a password that was never
needed. Corrected in auto-memory too.

Also lands the gx10 privileged outfit playbook, now green at 5/5:
NOPASSWD sudo, nvcc, docker group, a CUDA container seeing the GB10, and
the userspace torch stack still working afterward.
This commit is contained in:
vh
2026-09-01 23:37:42 -07:00
parent 0b48517909
commit c0e352a47b
3 changed files with 84 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/elway
+18 -2
View File
@@ -426,9 +426,25 @@ def ensure_sudo(ctx: SSHContext) -> None:
if ctx.sudo_probed:
return
ctx.sudo_probed = True
# Is sudo configured NOPASSWD for this user? `sudo -n -v` exits 0 if so.
# Is sudo configured NOPASSWD for this user?
#
# ⚠ This MUST be `sudo -n true`, never `sudo -n -v`. `-v` refreshes the
# auth timestamp, and a NOPASSWD-only rule creates no timestamp to refresh,
# so on **sudo >= 1.9.15** `-n -v` returns non-zero even though every real
# command runs passwordless. Measured 2026-09-01:
#
# pfi-gx10 sudo 1.9.15p5 sudo -n -v rc=1 sudo -n true rc=0
# nh3-docker sudo 1.9.13p3 sudo -n -v rc=0 sudo -n true rc=0
# ana-docker sudo 1.9.13p3 sudo -n -v rc=0 sudo -n true rc=0
# irv-ml1 sudo 1.9.13p3 sudo -n -v rc=0 sudo -n true rc=0
#
# The old probe cost a whole session on gx10: elway prompted for a password
# on a host that had working NOPASSWD sudo the entire time, and in a
# non-interactive run that prompt is an EOFError mid-playbook. Every host
# will hit this as it moves past sudo 1.9.13. Assert the effective
# behaviour (can I run a command?), not a proxy for it.
p = subprocess.run(
ctx.ssh_cmd("sudo -n -v"),
ctx.ssh_cmd("sudo -n true"),
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)