esh-pfi-infrastructure/stacks/albok-service

albok-service — fleet knowledgebase service (nh3-docker)

The FastAPI process that owns Albok's buildings-and-wings store (vh/albok, packages/albok-service, contract docs/contracts/unit2_service.contract.md). It is the only writer of the store. Deployed 2026-10-02 at albok-dev's request (operator-approved).

• URL: http://albok.nh3.internal:8392 (= http://10.100.50.40:8392). Container port 8390. ⚠ Host port 8392, because 8390 on nh3-docker is the althing post office.
• Image: gitea.phasefinal.com/pfi/albok-service:0.1.2 @ sha256:6f91a33f… (pinned in compose.yaml), built 2026-10-03 0250 from vh/albok 544ecfd (tag albok-service/v0.1.2; albok core 0.1.2). ⚠ Built from albok-dev's LOCAL commit before the operator pushed it (main was 29 ahead of origin). After the push, check that origin's tag albok-service/v0.1.2 is still 544ecfd. Image labels carry the full SHA. History: 0.1.0 (0b37431) 2026-10-02 1754–1805; 0.1.1 (e349d50) 1805 to 2026-10-03 0251.
• Health: GET /health (no auth) → "status": "ok" on 0.1.1. (0.1.0 always said degraded: its canary reports alive and the check expected ok; fixed in 0.1.1.) The Docker healthcheck checks HTTP 200.
• 0.1.1 also accepts numeric default_gid / gid in the config, which would make the mounted /etc/group unnecessary. The current name-based config stays valid, so it is left as is.
• Why 0.1.2: 0.1.1 leaked a Chroma client (and its sqlite fds) on every health probe. After 9 h it held 1018 of 1024 fds, 478 of them deleted canary files, and /health and /search answered 500. 0.1.2 closes the client. Verified 0251: 30 /health calls left the fd count at 59, with 0 deleted. The same count had caught the leak on 0.1.1. 0.1.2 also stamps X-Albok-Service: <version> on every response.

Pieces and where they live

What	Where
store root (one git repo per wing)	/srv/albok/store — local ext4, albok:albok 0711
private root (lease, journal, tokens.db, per-wing chroma)	/srv/albok/private — local ext4, albok:albok 0700, single-attach
config (holds the embedder key)	/srv/albok/etc/albok.yaml — root:albok 0640; rendered from conf/albok.yaml.template
container /etc/group	/opt/docker/conf/albok-service/group (= conf/group)
compose	/opt/docker/compose/albok-service/compose.yaml

Identities (fixed numeric ids, created by playbooks/albok-service-host.yaml): user and group albok 1500:1500 (the image's user), read groups albok-read 1510 (building fleet) and albok-personal 1511 (building personal). albok-feedback 1512 = restricted wing personal/agent-feedback (2026-10-03, default-deny: no viewer is in it). Reserve 1513+ for the vault wing and any later restricted wing; add each to the host, conf/group and group_add.

Why the group file and group_add: the service resolves group NAMES with grp.getgrnam() inside the container and chgrps wing dirs as uid 1500. Without the names in the container's /etc/group it reports "group does not resolve", and without membership the chgrp is refused. Verified: wings came up drwxr-s--- albok:albok-read / albok:albok-personal, drift 0.

Secrets (vault): albok/litellm-key, a LiteLLM key scoped to qwen3-embedding only (alias albok-service; verified 200 on embeddings, 403 on any other model); albok/bootstrap-admin-token, the first-start admin token (also at /srv/albok/private/bootstrap-admin-token, 0600). Nemi (albok's tagger, runs on nh3-dev): albok/token-nemi, a service token minted 0255 with read+write on fleet/memory, fleet/stash and personal/agent-feedback; it sees exactly those three wings (personal/notes hidden). The previous token (01M40979…, without agent-feedback) is still live until albok-dev revokes it. albok/litellm-key-nemi, alias albok-nemi, scoped to gen-small, summarizer and qwen3-embedding. Verified 200 on all three and 403 on another model.

Backups: nh3-docker is VM 100 on nh3-pve, in the nightly 21:00 vzdump (snapshot mode → pbs-ana), so /srv/albok is covered whole-VM. No file-level restic job for it.

Deploy / redeploy

# 1. host prep (idempotent)
scripts/elway infra-ops@10.100.50.40 --playbook playbooks/albok-service-host.yaml
# 2. config: render the template with the vaulted key (never commit the rendered file)
umask 077; secret get albok/litellm-key | tr -d '\n' | python3 -c "import sys; k=sys.stdin.read(); \
  open('/tmp/albok.yaml','w').write(open('stacks/albok-service/conf/albok.yaml.template').read().replace('__ALBOK_LITELLM_KEY__', k))"
scripts/elway infra-ops@10.100.50.40 --upload /tmp/albok.yaml:/srv/albok/etc/albok.yaml:0640 --sudo --owner root:albok
# then delete /tmp/albok.yaml by its literal path
# 3. compose + conf, then start (root holds the registry login on nh3-docker)
scripts/deploy-stack.sh nh3-docker albok-service
ssh infra-ops@10.100.50.40 'cd /opt/docker/compose/albok-service && sudo docker compose up -d'

Image rebuild (from a clean archive, never a working tree):

git -C ~/development/albok archive <sha> | tar -x -C <scratch>
cd <scratch> && docker build -f packages/albok-service/Dockerfile -t gitea.phasefinal.com/pfi/albok-service:<ver> .
secret get nh3-dev/.config/claude-bot/gitea-token-sdkops | docker login gitea.phasefinal.com -u claude-bot --password-stdin
docker push gitea.phasefinal.com/pfi/albok-service:<ver>     # then pin the new digest in compose.yaml

Viewers (read-only access to the store)

Mount /srv/albok/store :ro, group_add the read group's numeric gid (1510 / 1511), set GIT_OPTIONAL_LOCKS=0, and add safe.directory for the path as the viewer sees it. Nothing else may mount the store read-write.