Session captured: the FV outbound-NAT root cause and its diagnostic signature, the fv-ml1 dead man's switch, fleet identity/group/path conventions and the root:docker normalization, nh3-dev's ts-input reachability fix, ESPHome modernisation and the kb KB-search tool, and the Hermes bearer rotation release. Six new detail files. Tried-and-abandoned gains three: probing OPNsense endpoints by POSTing at them (which rebooted the FV firewall), advertising a /32 from nh3-dev, and the nh3-scale remote-site masquerade rules that fired but were not the fix. Housekeeping: 8 Recent-decisions entries archived to archival-memory.md, and 21 oversized inline entries split into detail files per the two-tier rule -- they had been sitting fully inline in the index, which is what the split exists to prevent. Two pointers to a detail file archived this run were repointed at archival-memory.md. The index is 389 lines, still over the ~300 soft cap. The archival guards stop it there: only 4 further entries are old enough to move and every one carries an open deferred-work pointer. An over-cap file that keeps live decisions beats a scannable one that lost a deferred call.
68 lines
3.3 KiB
Markdown
68 lines
3.3 KiB
Markdown
# `[2026-09-15]` Fleet identity/group/path conventions pinned + docker trees normalized
|
||
|
||
Operator ratified four conventions. `docs/pfi/fleet-conventions.md` is the pin;
|
||
`playbooks/audit-host-conventions.yaml` is its read-only instrument. Commits
|
||
`826a63b`, `abef67a`, `ce7b07f`.
|
||
|
||
## Pinned allocation map
|
||
|
||
Verified free on all eight surveyed hosts — dynamically-allocated system
|
||
accounts cluster in 989–999 and descend, so 800–899 is safe:
|
||
|
||
800–849 svc-* service accounts
|
||
850 infra-ops (uid + gid)
|
||
851 docker (gid)
|
||
852–899 reserved for fleet-wide groups
|
||
1000 the human account (vh)
|
||
|
||
**`vh` for new hosts, no retro-renames.** `lkraven` stays on the six legacy
|
||
hosts; renaming uid 1000 with populated homes, lingering systemd services and
|
||
live agent sessions is real blast radius for cosmetic gain — and the thing that
|
||
mattered (a personal username owning *shared* infrastructure) was removed by the
|
||
`root:docker` change below.
|
||
|
||
## Deploy trees → `root:docker 2775` setgid, all 5 hosts
|
||
|
||
Not a personal username and not a new admin account: the `docker` group already
|
||
existed on every host holding exactly `lkraven` + `infra-ops`. Cleared the
|
||
`0777` on nh3-docker and ana-docker (a 2024 `chmod -R 777` to get a git clone
|
||
working). 55 stack `.env` files → `root:docker 0640`, tightening 43
|
||
world-readable ones and opening 31 that were legible to only one of the two
|
||
deploy identities.
|
||
|
||
⚠ **This is NOT privilege separation.** `docker` membership is root-equivalent.
|
||
A future non-root deployer needs a dedicated `deploy` group.
|
||
|
||
⚠ **Deliberately not a recursive chmod.** Three `acme.json` files and an ssh
|
||
private key are mode `0600`, and traefik/ssh refuse to start if that widens —
|
||
which would fail at the *next restart*, weeks later. Protection is both
|
||
mode-based and name-based.
|
||
|
||
## Accounts
|
||
|
||
- `linus` on ana-docker **deleted** — passwordless root, last used 2026-04-11 to
|
||
set up a Synapse appservice, archived to `/root/account-archive/`.
|
||
⚠ I reported it "never logged in" off `lastlog`; it had a `.bash_history`.
|
||
`lastlog` is a bad instrument for that question.
|
||
- `llmuser` stripped of `sudo`+`docker` (ana-docker) and `sudo` (irv-ml1).
|
||
|
||
⭐ **The durable lesson is a measurement trap.** `pgrep -u llmuser` reported 19
|
||
processes — which reads as a busy service account and would stop a cleanup.
|
||
Nearly all were **container** processes whose in-image UID is 1001 and collides
|
||
with llmuser on the host (`/proc/<pid>/cgroup` shows `docker-*.scope`). A
|
||
container's runtime UID has nothing to do with host group membership. Check the
|
||
cgroup before concluding a host account is busy.
|
||
|
||
## deploy-stack.sh, fixed three times before the rule was written
|
||
|
||
`-a` is `-rlptgoD`, and a non-root identity cannot apply owner, group,
|
||
permissions **or** times to a root-owned tree. Each patch fixed one letter and
|
||
the next deploy failed on the next one, every time exiting 23 **after**
|
||
transferring content — a loud error on a deploy that had succeeded. The rule now
|
||
in the script: **the deploy syncs content, the conventions own metadata** —
|
||
`--no-o --no-g --no-perms --omit-dir-times`.
|
||
|
||
Open: `llmuser`/`sduser`/`brokkr`/`arbotrain`/`nas`/`deploy` keep their legacy
|
||
names by decision; `/mnt/smithy` NFS is `0777` throughout, blocked on UID
|
||
alignment; Synapse appservice tokens sit in plaintext on ana-docker.
|