feat(albok-service): deploy the fleet knowledgebase service on nh3-docker

albok-service 0.1.0 (vh/albok 0b37431), image pfi/albok-service pinned by digest, published on host port 8392 because 8390 is the post office. Host prep playbook creates the fixed ids (albok 1500, albok-read 1510, albok-personal 1511) and the local store/private roots; the container gets a mounted /etc/group and group_add so the service can resolve and chgrp its wing dirs. The config carries a LiteLLM key scoped to qwen3-embedding and lives outside the deploy-synced conf dir. DNS name albok.nh3.internal.
2026-10-02 17:56:34 -07:00
parent 9c233ca255
commit d3958655c1
7 changed files with 248 additions and 0 deletions
@@ -0,0 +1,71 @@
+# albok-service — fleet knowledgebase service (nh3-docker)
+
+The FastAPI process that owns Albok's buildings-and-wings store (`vh/albok`,
+`packages/albok-service`, contract `docs/contracts/unit2_service.contract.md`). It is the
+**only writer** of the store. Deployed 2026-10-02 at albok-dev's request (operator-approved).
+
+- **URL:** `http://albok.nh3.internal:8392` (= `http://10.100.50.40:8392`). Container port 8390.
+  ⚠ Host port **8392**, because **8390 on nh3-docker is the althing post office**.
+- **Image:** `gitea.phasefinal.com/pfi/albok-service:0.1.0` @ `sha256:9ae2c94e…` (pinned in
+  `compose.yaml`), built from vh/albok `0b37431` (tag v0.1.1; albok core 0.1.1 inside).
+- **Health:** `GET /health` (no auth). ⚠ 0.1.0 reports `"status": "degraded"` even when everything
+  is fine: `health.py` treats canary values other than ok/healthy/ready as dead, and the canary
+  reports `"alive"`. Reported to albok-dev. Read the fields, not `status`. The Docker healthcheck
+  only checks for HTTP 200.
+
+## Pieces and where they live
+
+| What | Where |
+|---|---|
+| store root (one git repo per wing) | `/srv/albok/store` — local ext4, `albok:albok` 0711 |
+| private root (lease, journal, tokens.db, per-wing chroma) | `/srv/albok/private` — local ext4, `albok:albok` 0700, **single-attach** |
+| config (holds the embedder key) | `/srv/albok/etc/albok.yaml` — `root:albok` 0640; rendered from `conf/albok.yaml.template` |
+| container `/etc/group` | `/opt/docker/conf/albok-service/group` (= `conf/group`) |
+| compose | `/opt/docker/compose/albok-service/compose.yaml` |
+
+**Identities (fixed numeric ids, created by `playbooks/albok-service-host.yaml`):** user and group
+`albok` 1500:1500 (the image's user), read groups `albok-read` **1510** (building `fleet`) and
+`albok-personal` **1511** (building `personal`). Reserve 1512+ for restricted wings (agent-feedback,
+vault) and add each to the host, `conf/group` and `group_add`.
+
+**Why the group file and `group_add`:** the service resolves group NAMES with `grp.getgrnam()`
+inside the container and `chgrp`s wing dirs as uid 1500. Without the names in the container's
+`/etc/group` it reports "group does not resolve", and without membership the chgrp is refused.
+Verified: wings came up `drwxr-s--- albok:albok-read` / `albok:albok-personal`, drift 0.
+
+**Secrets (vault):** `albok/litellm-key`, a LiteLLM key scoped to `qwen3-embedding` only (alias
+`albok-service`; verified 200 on embeddings, 403 on any other model); `albok/bootstrap-admin-token`,
+the first-start admin token (also at `/srv/albok/private/bootstrap-admin-token`, 0600).
+
+**Backups:** nh3-docker is VM 100 on nh3-pve, in the nightly 21:00 vzdump (snapshot mode → pbs-ana),
+so `/srv/albok` is covered whole-VM. No file-level restic job for it.
+
+## Deploy / redeploy
+
+```bash
+# 1. host prep (idempotent)
+scripts/elway infra-ops@10.100.50.40 --playbook playbooks/albok-service-host.yaml
+# 2. config: render the template with the vaulted key (never commit the rendered file)
+umask 077; secret get albok/litellm-key | tr -d '\n' | python3 -c "import sys; k=sys.stdin.read(); \
+  open('/tmp/albok.yaml','w').write(open('stacks/albok-service/conf/albok.yaml.template').read().replace('__ALBOK_LITELLM_KEY__', k))"
+scripts/elway infra-ops@10.100.50.40 --upload /tmp/albok.yaml:/srv/albok/etc/albok.yaml:0640 --sudo --owner root:albok
+# then delete /tmp/albok.yaml by its literal path
+# 3. compose + conf, then start (root holds the registry login on nh3-docker)
+scripts/deploy-stack.sh nh3-docker albok-service
+ssh infra-ops@10.100.50.40 'cd /opt/docker/compose/albok-service && sudo docker compose up -d'
+```
+
+**Image rebuild** (from a clean archive, never a working tree):
+
+```bash
+git -C ~/development/albok archive <sha> | tar -x -C <scratch>
+cd <scratch> && docker build -f packages/albok-service/Dockerfile -t gitea.phasefinal.com/pfi/albok-service:<ver> .
+secret get nh3-dev/.config/claude-bot/gitea-token-sdkops | docker login gitea.phasefinal.com -u claude-bot --password-stdin
+docker push gitea.phasefinal.com/pfi/albok-service:<ver>     # then pin the new digest in compose.yaml
+```
+
+## Viewers (read-only access to the store)
+
+Mount `/srv/albok/store` **`:ro`**, `group_add` the read group's **numeric** gid (1510 / 1511), set
+`GIT_OPTIONAL_LOCKS=0`, and add `safe.directory` for the path as the viewer sees it. Nothing else
+may mount the store read-write.
@@ -0,0 +1,48 @@
+# albok-service — the fleet knowledgebase service (vh/albok, packages/albok-service). The only
+# writer of the buildings-and-wings store. Requested by albok-dev 2026-10-02 (operator-approved).
+#
+# Deploy:  scripts/elway infra-ops@10.100.50.40 --playbook playbooks/albok-service-host.yaml   (host prep, config)
+#          scripts/deploy-stack.sh nh3-docker albok-service                                    (this file + conf/)
+# Image:   built from a clean `git archive` of vh/albok, pushed to the pfi org (see README.md).
+#
+# ⚠ Host port 8392, NOT 8390: 8390 on nh3-docker is the althing post office.
+# ⚠ Exactly ONE instance per private root: a second one exits 75 on the lease. Never scale this.
+# ⚠ Nothing else may mount /srv/albok/store read-write. Viewers mount it :ro with group_add by gid
+#   (1510 albok-read, 1511 albok-personal) and need GIT_OPTIONAL_LOCKS=0 + a safe.directory entry.
+services:
+  albok-service:
+    image: gitea.phasefinal.com/pfi/albok-service:0.1.0@sha256:9ae2c94e39917d690aef2361e47abceff863f25338fd0ea2848b651bcb145ab6
+    container_name: albok-service
+    restart: unless-stopped
+    ports:
+      - "8392:8390"
+    # Membership the service needs to chgrp wing dirs to the read groups (it runs as uid 1500, not root).
+    group_add:
+      - "1510"
+      - "1511"
+    volumes:
+      - /srv/albok/store:/srv/albok/store
+      - /srv/albok/private:/srv/albok/private
+      - /srv/albok/etc/albok.yaml:/etc/albok/albok.yaml:ro
+      # the read groups must RESOLVE BY NAME inside the container (layout.py uses grp.getgrnam)
+      - /opt/docker/conf/albok-service/group:/etc/group:ro
+    mem_limit: 4g
+    healthcheck:
+      test: ["CMD", "python", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8390/health', timeout=5).status == 200 else 1)"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 60s
+    networks:
+      - tnet
+    labels:
+      - homepage.group=Agents (no UI)
+      - homepage.name=Albok
+      - homepage.icon=mdi-book-search
+      - homepage.description=Fleet knowledgebase service (albok-service) on nh3-docker
+      - homepage.href=http://10.100.50.40:8392/health
+
+networks:
+  tnet:
+    name: traefik-net
+    external: true
@@ -0,0 +1,45 @@
+# albok-service configuration (template). The REAL file is /srv/albok/etc/albok.yaml on nh3-docker,
+# rendered from this one with the embedder key filled in (vault `albok/litellm-key`) by
+# playbooks/albok-service-host.yaml. It is not under /opt/docker/conf because it holds that key and
+# deploy-stack would sync over it. Every path is absolute and used verbatim (nothing is expanded).
+# Shape: packages/albok-service/albok.example.yaml in vh/albok @ 0b37431 (albok-service 0.1.0).
+
+store_root: /srv/albok/store          # host bind mount, local ext4, owned by uid 1500
+private_root: /srv/albok/private      # host bind mount, local ext4, owned by uid 1500; single-attach (lease)
+
+git_identity:
+  name: Albok
+  email: albok@fleet.local
+
+embedding:                            # fleet LiteLLM gateway (Anaheim); TEI behind it
+  provider: litellm-ana
+  type: openai_compat
+  base_url: http://10.250.50.70:4000/v1
+  model: qwen3-embedding
+  api_key: "__ALBOK_LITELLM_KEY__"    # scoped key `albok-service`: qwen3-embedding only
+
+reranker: null
+
+ids_in_cap: 1000
+search_defaults:
+  top_k: 20
+  inline_budget_chars: 8000
+listen:
+  host: 0.0.0.0
+  port: 8390                          # container port; published on the host as 8392 (8390 is the post office)
+
+# First instance, per albok-dev's ask (2026-10-02): fleet/{memory,stash}, personal/notes.
+# Restricted wings (agent-feedback, vault) come later with their own groups (gids 1512+).
+buildings:
+  - id: fleet
+    default_group: albok-read         # host gid 1510
+    wings:
+      - id: memory
+        probe_object_id: null
+        probe_query: null
+      - id: stash
+        near_duplicate_threshold: null
+  - id: personal
+    default_group: albok-personal     # host gid 1511
+    wings:
+      - id: notes
@@ -0,0 +1,41 @@
+root:x:0:
+daemon:x:1:
+bin:x:2:
+sys:x:3:
+adm:x:4:
+tty:x:5:
+disk:x:6:
+lp:x:7:
+mail:x:8:
+news:x:9:
+uucp:x:10:
+man:x:12:
+proxy:x:13:
+kmem:x:15:
+dialout:x:20:
+fax:x:21:
+voice:x:22:
+cdrom:x:24:
+floppy:x:25:
+tape:x:26:
+sudo:x:27:
+audio:x:29:
+dip:x:30:
+www-data:x:33:
+backup:x:34:
+operator:x:37:
+list:x:38:
+irc:x:39:
+src:x:40:
+shadow:x:42:
+utmp:x:43:
+video:x:44:
+sasl:x:45:
+plugdev:x:46:
+staff:x:50:
+games:x:60:
+users:x:100:
+nogroup:x:65534:
+albok:x:1500:
+albok-read:x:1510:albok
+albok-personal:x:1511:albok