From ce7b07f7af5414312aa29020fb174de42443ca3f Mon Sep 17 00:00:00 2001 From: Vuong Hoang Date: Mon, 14 Sep 2026 13:36:33 -0700 Subject: [PATCH] fix(fleet): strip sudo+docker from llmuser; record the pgrep over-attribution trap MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Operator ruling: remove the groups and see what breaks. Nothing did. ana-docker llmuser sudo+docker -> none; irv-ml1 llmuser sudo -> none (it was never in docker there). 45 containers on ana-docker and 18 on irv-ml1 all still running with zero unhealthy, and lora-training-worker stayed active. Extended to irv-ml1 because it is the same account with the same defect and gpasswd -a reverses it in one command; ana-docker was only the host the audit happened to run against first. The durable lesson is why it was safe, and it is a measurement trap rather than a permissions one. reported 19 processes on ana-docker and 3 on irv-ml1, which reads as a busy service account. Nearly all of them were CONTAINER processes whose in-image UID is 1001 and therefore collides with llmuser on the host — /proc//cgroup shows docker-*.scope. A container's runtime UID is unrelated to host group membership, so the groups were buying those workloads nothing. The single real host workload sets User=/Group= explicitly through systemd, which does not consult the sudo group either. Recorded in the conventions doc so the next audit checks the cgroup before concluding a host account is busy — otherwise a UID collision blocks a cleanup that carries no risk. --- docs/pfi/fleet-conventions.md | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/docs/pfi/fleet-conventions.md b/docs/pfi/fleet-conventions.md index 41a8a33..580a9a0 100644 --- a/docs/pfi/fleet-conventions.md +++ b/docs/pfi/fleet-conventions.md @@ -71,8 +71,19 @@ useradd --system --uid 8NN --user-group --shell /usr/sbin/nologin \ (1000+) where `/etc/passwd` cannot distinguish them from people. That is the defect the convention fixes going forward. -⚠ **A service account must not hold interactive root.** As of 2026-09-14 -`llmuser` is in the `sudo` group on ana-docker. Flagged, not yet resolved. +⚠ **A service account must not hold interactive root.** Enforced 2026-09-14: +`llmuser` was in `sudo`+`docker` on ana-docker and `sudo` on irv-ml1; both +stripped. Nothing broke, and the reason is worth knowing — nearly every process +attributed to `llmuser` by `pgrep -u` was a **container** process whose in-image +UID happens to be 1001. A container's runtime UID has nothing to do with host +group membership, so the groups were buying it nothing. The one genuine host +workload (`lora-training-worker.service` on irv-ml1) sets `User=`/`Group=` +explicitly via systemd, which does not consult the `sudo` group either. + +**Generalisation for the next audit:** `pgrep -u ` on a Docker host +over-attributes badly. Check `/proc//cgroup` for a `docker-*.scope` before +concluding a host account is busy — otherwise a UID collision reads as an +active service account and blocks a cleanup that carries no risk. --- @@ -183,14 +194,16 @@ Anything already at `0600`/`0400` is deliberate. Leave it. | item | state | |---|---| -| `llmuser` holds `sudo` on ana-docker | flagged 2026-09-14, unresolved | | `/mnt/smithy` NFS export is `0777` throughout | blocked on § 3 UID alignment | | Synapse appservice tokens sit in plaintext on ana-docker | flagged 2026-09-14; belongs in the vault | | Legacy service accounts in the human UID range | not renamed, by decision | ## Changelog -- **2026-09-14** — ratified. Deleted the dormant `linus` account on ana-docker +- **2026-09-14** — ratified. `vh` confirmed as the standard for new hosts with + **no retro-renames**. Stripped `sudo`+`docker` from `llmuser` (ana-docker) and + `sudo` (irv-ml1) — see § 1.3; all 63 containers across both hosts stayed + healthy and `lora-training-worker` stayed active. Deleted the dormant `linus` account on ana-docker (passwordless root, last used 2026-04-11, archived to `/root/account-archive/linus-20260914.tar.gz`). Standardised `infra-ops` home to `0700` on five hosts. Normalised corviduo-dev's docker tree, the operator