feat(albok): move albok-service to a dedicated VM, nh3-albok (VM 110 on nh3-pve-2)
albok-service leaves nh3-docker before unit 4 puts the personal vault and licensed books into restricted wings. On nh3-docker, portainer, portainer_agent, dockge (socket rw) and dockerd on plaintext tcp/2375 were each root over /srv/albok. The new host runs only albok-service: no socket containers and no TCP Docker API, so its Homepage tile is a static services.yaml entry. - nh3-pve-2: vmbr0 made VLAN-aware (applied by reboot, no guests running); pbs-ana storage in the new namespace nh3-pve-2, plus a 21:30 vzdump job (pbs-nh3's recursive sync mirrors it) - VM 110 nh3-albok, 10.100.50.44 on VLAN 50, Debian 13 cloud-init, 4 vCPU x86-64-v3 / 12 GiB / 100 GB; built to fleet conventions (vh 1000, infra-ops 850, docker 851) - compose: mem_limit 4g -> 8g, homepage labels removed; host port 8392 kept - cutover by tar stream, verified by digest (contents plus owner, group, mode, mtime and type identical); albok.nh3.internal now points at nh3-albok - old /srv/albok stays on nh3-docker (container removed, compose renamed) until acceptance and the operator's approval to delete it - sudo-io archive hosts and ops-log STACK_HOSTS include nh3-albok
This commit is contained in:
@@ -1,11 +1,13 @@
|
||||
# albok-service — fleet knowledgebase service (nh3-docker)
|
||||
# albok-service — fleet knowledgebase service (nh3-albok)
|
||||
|
||||
The FastAPI process that owns Albok's buildings-and-wings store (`vh/albok`,
|
||||
`packages/albok-service`, contract `docs/contracts/unit2_service.contract.md`). It is the
|
||||
**only writer** of the store. Deployed 2026-10-02 at albok-dev's request (operator-approved).
|
||||
|
||||
- **URL:** `http://albok.nh3.internal:8392` (= `http://10.100.50.40:8392`). Container port 8390.
|
||||
⚠ Host port **8392**, because **8390 on nh3-docker is the althing post office**.
|
||||
- **Host:** **nh3-albok** (VM 110 on nh3-pve-2, `10.100.50.44`) since 2026-10-04, a VM that runs nothing else
|
||||
(`servers/nh3-albok/README.md` says why and what must never be added there). Before that: nh3-docker, 2026-10-02 to 2026-10-04.
|
||||
- **URL:** `http://albok.nh3.internal:8392` (= `http://10.100.50.44:8392`). Container port 8390.
|
||||
Host port **8392** is kept from nh3-docker, where 8390 was the althing post office, so no client changed.
|
||||
- **Image:** `gitea.phasefinal.com/pfi/albok-service:0.1.8` @ `sha256:239f900e…` (pinned in `compose.yaml`), built
|
||||
2026-10-04 ~1115 from vh/albok `e653c29` (tag albok-service/v0.1.8; heid re-pass fixes over 0.1.7: keyword-index
|
||||
lock mismatch, journal-stranding replay path, bypassable 400, link-kind handling in the collapse). fleet/memory
|
||||
@@ -65,8 +67,10 @@ The previous token (`01M40979…`, without agent-feedback) is still live until a
|
||||
`albok/litellm-key-nemi`, alias `albok-nemi`, scoped to `gen-small`, `summarizer` and `qwen3-embedding`. Verified 200 on
|
||||
all three and 403 on another model.
|
||||
|
||||
**Backups:** nh3-docker is VM 100 on nh3-pve, in the nightly 21:00 vzdump (snapshot mode → pbs-ana),
|
||||
so `/srv/albok` is covered whole-VM. No file-level restic job for it.
|
||||
**Backups:** nh3-albok is VM 110 on nh3-pve-2, in that node's nightly **21:30** vzdump (snapshot mode → pbs-ana,
|
||||
namespace `nh3-pve-2`; pbs-nh3's 06:30 sync mirrors it), so `/srv/albok` is covered whole-VM. No file-level restic
|
||||
job. ⚠ The PBS datastores are not client-side encrypted: anyone with PBS or NAS root can read the restricted wings
|
||||
out of a backup. That matters once unit 4 ingests the personal vault; unit 6's encrypted wings close it.
|
||||
|
||||
## Deploy / redeploy
|
||||
|
||||
@@ -77,15 +81,15 @@ work and the service commits per batch, but avoidable. Check first, on nh3-dev a
|
||||
|
||||
```bash
|
||||
# 1. host prep (idempotent)
|
||||
scripts/elway infra-ops@10.100.50.40 --playbook playbooks/albok-service-host.yaml
|
||||
scripts/elway infra-ops@10.100.50.44 --playbook playbooks/albok-service-host.yaml
|
||||
# 2. config: render the template with the vaulted key (never commit the rendered file)
|
||||
umask 077; secret get albok/litellm-key | tr -d '\n' | python3 -c "import sys; k=sys.stdin.read(); \
|
||||
open('/tmp/albok.yaml','w').write(open('stacks/albok-service/conf/albok.yaml.template').read().replace('__ALBOK_LITELLM_KEY__', k))"
|
||||
scripts/elway infra-ops@10.100.50.40 --upload /tmp/albok.yaml:/srv/albok/etc/albok.yaml:0640 --sudo --owner root:albok
|
||||
scripts/elway infra-ops@10.100.50.44 --upload /tmp/albok.yaml:/srv/albok/etc/albok.yaml:0640 --sudo --owner root:albok
|
||||
# then delete /tmp/albok.yaml by its literal path
|
||||
# 3. compose + conf, then start (root holds the registry login on nh3-docker)
|
||||
scripts/deploy-stack.sh nh3-docker albok-service
|
||||
ssh infra-ops@10.100.50.40 'cd /opt/docker/compose/albok-service && sudo docker compose up -d'
|
||||
# 3. compose + conf, then start (root holds the registry login on nh3-albok: claude-bot, vaulted nh3-dev/.config/claude-bot/gitea-token)
|
||||
scripts/deploy-stack.sh nh3-albok albok-service
|
||||
ssh infra-ops@10.100.50.44 'cd /opt/docker/compose/albok-service && sudo docker compose up -d'
|
||||
```
|
||||
|
||||
**Image rebuild** (from a clean archive, never a working tree):
|
||||
|
||||
@@ -1,11 +1,14 @@
|
||||
# albok-service — the fleet knowledgebase service (vh/albok, packages/albok-service). The only
|
||||
# writer of the buildings-and-wings store. Requested by albok-dev 2026-10-02 (operator-approved).
|
||||
# Host: nh3-albok (VM 110 on nh3-pve-2, 10.100.50.44) since 2026-10-04, a VM that runs ONLY this.
|
||||
# It moved off nh3-docker, where portainer/dockge/the agent and dockerd on tcp/2375 were all root
|
||||
# over the store (servers/nh3-albok/README.md).
|
||||
#
|
||||
# Deploy: scripts/elway infra-ops@10.100.50.40 --playbook playbooks/albok-service-host.yaml (host prep, config)
|
||||
# scripts/deploy-stack.sh nh3-docker albok-service (this file + conf/)
|
||||
# Deploy: scripts/elway infra-ops@10.100.50.44 --playbook playbooks/albok-service-host.yaml (host prep, config)
|
||||
# scripts/deploy-stack.sh nh3-albok albok-service (this file + conf/)
|
||||
# Image: built from a clean `git archive` of vh/albok, pushed to the pfi org (see README.md).
|
||||
#
|
||||
# ⚠ Host port 8392, NOT 8390: 8390 on nh3-docker is the althing post office.
|
||||
# ⚠ Host port 8392 (kept from nh3-docker, where 8390 was the post office) so clients did not change.
|
||||
# ⚠ Exactly ONE instance per private root: a second one exits 75 on the lease. Never scale this.
|
||||
# ⚠ Nothing else may mount /srv/albok/store read-write. Viewers mount it :ro with group_add by gid
|
||||
# (1510 albok-read, 1511 albok-personal) and need GIT_OPTIONAL_LOCKS=0 + a safe.directory entry.
|
||||
@@ -28,7 +31,7 @@ services:
|
||||
- /srv/albok/etc/albok.yaml:/etc/albok/albok.yaml:ro
|
||||
# the read groups must RESOLVE BY NAME inside the container (layout.py uses grp.getgrnam)
|
||||
- /opt/docker/conf/albok-service/group:/etc/group:ro
|
||||
mem_limit: 4g
|
||||
mem_limit: 8g # 4g on nh3-docker (7.7 GiB shared host); 12 GiB VM to itself now
|
||||
healthcheck:
|
||||
test: ["CMD", "python", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8390/health', timeout=5).status == 200 else 1)"]
|
||||
interval: 30s
|
||||
@@ -37,12 +40,8 @@ services:
|
||||
start_period: 60s
|
||||
networks:
|
||||
- tnet
|
||||
labels:
|
||||
- homepage.group=Agents (no UI)
|
||||
- homepage.name=Albok
|
||||
- homepage.icon=mdi-book-search
|
||||
- homepage.description=Fleet knowledgebase service (albok-service) on nh3-docker
|
||||
- homepage.href=http://10.100.50.40:8392/health
|
||||
# No homepage.* labels: Homepage discovers containers over a host's tcp/2375, and this host must not
|
||||
# expose its Docker API. The Albok tile is a static entry in stacks/homepage/conf/services.yaml.
|
||||
|
||||
networks:
|
||||
tnet:
|
||||
|
||||
@@ -85,6 +85,16 @@
|
||||
# well or it renders twice. To move a service between AI groups, change the
|
||||
# label on its compose file and recreate the container — labels are read at
|
||||
# creation, so `restart` will not do it.
|
||||
- Agents (no UI):
|
||||
# Manual entry — albok-service moved to nh3-albok (2026-10-04), a host that deliberately does NOT
|
||||
# expose its Docker API on tcp/2375 (it holds restricted wings), so it is not in docker.yaml and its
|
||||
# container carries no homepage labels. Do not add the host to docker.yaml to "fix" discovery.
|
||||
- Albok:
|
||||
href: http://10.100.50.44:8392/health
|
||||
icon: mdi-book-search
|
||||
siteMonitor: http://10.100.50.44:8392/health
|
||||
description: Fleet knowledgebase service (albok-service) on nh3-albok
|
||||
|
||||
- AI - Studios:
|
||||
# Manual entry — Voice Design Studio is a user-level systemd service on
|
||||
# irv-ml1, not a Docker-labeled stack, so it cannot auto-discover. It sits
|
||||
|
||||
Reference in New Issue
Block a user