feat(ana-ml2): durable vm.overcommit_memory=1 sysctl playbook

ana-ml2 ran overcommit_memory=0 with zero swap, capping the CommitLimit at
~RAM/2 (~283 GB of 566 GB). The resident vLLM services commit ~224 GB, so a
large model-file mmap (the 50 GB NVFP4 shard during HF->native conversion, or
a vLLM model load) failed with ENOMEM despite ~393 GB of RAM actually free.

overcommit_memory=1 is the conventional setting for ML hosts that mmap large
files. A drop-in under /etc/sysctl.d/ makes it reboot-durable. Operator-directed
permanent (2026-06-17). Idempotent via when:; sudo tee for the root-owned path
(elway runs steps as the SSH user, so a shell > redirect can't write there).

playbooks/ana-ml2-overcommit-memory.yaml

@@ -0,0 +1,43 @@
+# Make vm.overcommit_memory=1 durable on ana-ml2 (GPU inference host).
+#
+# Why: ana-ml2 runs vm.overcommit_memory=0 (heuristic) with zero swap, so the
+# CommitLimit is ~RAM/2 (~283 GB of 566 GB). The resident vLLM services already
+# commit ~224 GB of address space, leaving < 60 GB of headroom. A large model-file
+# mmap (e.g. the 50 GB NVFP4 shard during HF->native conversion, or a vLLM model
+# load) then fails with ENOMEM despite ~393 GB of RAM actually being free — the
+# kernel rejects the *commit*, not the allocation.
+#
+# overcommit_memory=1 (always overcommit) is the conventional setting for ML hosts
+# that mmap large files: the real RAM is there to back the pages, and the heuristic
+# accounting is the only thing in the way. Operator-directed permanent + durable
+# (2026-06-17). A drop-in under /etc/sysctl.d/ applies at every boot.
+#
+# Run:  scripts/elway infra-ops@ana-ml2 --playbook playbooks/ana-ml2-overcommit-memory.yaml
+# Rerunnable: a second run shows the write step `skipped` (idempotent via when:).
+
+vars:
+  dropin: /etc/sysctl.d/99-overcommit-memory.conf
+  setting: "vm.overcommit_memory = 1"
+
+steps:
+  - name: Write durable overcommit sysctl drop-in
+    # elway runs steps as the SSH user, so a shell `>` redirect can't write a
+    # root-owned path — pipe through `sudo tee` (infra-ops has NOPASSWD sudo).
+    shell: |
+      printf '# GPU inference host: large model-file mmaps (NVFP4 native convert, vLLM loads)\n# exceed the heuristic CommitLimit (overcommit=0 + zero swap) despite ample free RAM.\n# Operator-directed permanent setting 2026-06-17.\n%s\n' '{{ setting }}' | sudo tee {{ dropin }} >/dev/null
+    # Skip the write if the drop-in already holds exactly this line.
+    when: "! grep -qxF '{{ setting }}' {{ dropin }} 2>/dev/null"
+
+  - name: Apply all sysctl drop-ins now
+    shell: sudo sysctl --system >/dev/null
+    # Applying is a no-op when the runtime value already matches.
+    changed_when: "false"
+
+verify:
+  - name: Runtime vm.overcommit_memory is 1
+    shell: test "$(cat /proc/sys/vm/overcommit_memory)" = "1"
+    changed_when: "false"
+
+  - name: Drop-in file persists the setting (survives reboot)
+    shell: grep -qxF '{{ setting }}' {{ dropin }}
+    changed_when: "false"