feat(muninn-gate): WG-internal Muninn ingestion front door stack (#377)

Deployed on corviduo-dev, co-located with the worldtree-personal muninn watcher. ingestion_root=/data/state/ingestion (shared state volume, byte- identical to the watcher); runs as uid 1000 to write the queue; staging bound :ro at the ratified /mnt/muninn-staging/mimir-inbox (local placeholder until the shared mount + mimir-inbox writer land). Boot verified: /ping {"service":"ok"}, /health watcher.running=true (byte-identity proven). Image built out-of-band with the Gitea read token as a BuildKit secret. Real config (bearer keys) lives on-server at /opt/docker/conf 0600.
2026-07-30 21:33:29 -07:00
parent 17d776fc90
commit 786462ac9c
4 changed files with 185 additions and 0 deletions
@@ -0,0 +1,27 @@
+# muninn-gate — WG-internal HTTP front door for the Muninn ingestion queue (#377).
+# No Dockerfile in the upstream repo (vh/muninn-gate); this is infra-ops's.
+# muninn-dispatch==0.1.4 comes from the internal Gitea PyPI index "gitea";
+# the read token is passed as a BuildKit secret (never baked into a layer).
+#
+#   DOCKER_BUILDKIT=1 docker build \
+#     --secret id=gitea_pw,src=<read-token-file> \
+#     -t muninn-gate:0.0.14 .
+FROM python:3.11-slim
+
+RUN pip install --no-cache-dir uv==0.11.* \
+ && useradd --system --uid 10014 --create-home --home-dir /home/gate gate
+
+WORKDIR /app
+COPY pyproject.toml README.md uv.lock ./
+COPY src ./src
+
+# Install the app + deps. muninn-dispatch resolves only from the "gitea" index
+# (pinned via [tool.uv.sources]); everything else from PyPI.
+RUN --mount=type=secret,id=gitea_pw \
+    UV_INDEX_GITEA_USERNAME=vh \
+    UV_INDEX_GITEA_PASSWORD="$(cat /run/secrets/gitea_pw)" \
+    uv pip install --system --no-cache .
+
+# The launcher (compose) owns the bind — no CMD here. The app carries `bind`
+# for observability only; --host/--port passed by compose are authoritative.
+USER gate
@@ -0,0 +1,56 @@
+# muninn-gate
+
+WG-internal HTTP front door for the Muninn ingestion queue (Worldtree #377).
+Path-addressed submit, list/status, cancel/retry, watcher-liveness health.
+Upstream: `vh/muninn-gate` (no Dockerfile there — this stack owns containerization).
+
+## Where it runs
+
+**corviduo-dev (10.250.50.152)**, co-located with the worldtree-personal muninn
+watcher (`worldtree-personal-worldtree-muninn-1`). infra-ops-managed stack,
+separate from the worldtree CI/CD compose. Bind: `10.250.50.152:8090`, `network_mode: host`.
+
+## Key wiring (why it's shaped this way)
+
+- **`ingestion_root: /data/state/ingestion`** — the `worldtree-personal_worldtree-state`
+  volume, mounted at `/data/state`. Byte-identical path to the watcher's view; the
+  dispatch records absolute paths and both processes read each other's.
+  Acceptance: `/health` → `watcher.running: true` **proves** this byte-identity
+  (gate is reading the heartbeat the watcher writes). `no_heartbeat` with the
+  watcher up = root mismatch.
+- **`user: "1000:1000"`** — the ingestion dir is `vh:vh 0755`; the gate must run as
+  uid 1000 to write the queue (non-root, least-privilege). Overrides the image user.
+- **staging** — `/mnt/muninn-staging/mimir-inbox`, same absolute path bound `:ro` in
+  BOTH the gate and the watcher (worldtree-dev's b162). Currently a LOCAL placeholder
+  dir; becomes the shared mount when mimir-inbox (the writer) lands. **No gate-side
+  check can tell a real share from an empty dir** (docker fabricates a missing bind
+  source) — the guard is operational: confirm the host mount exists before wiring/
+  repointing a bind. Repoint needs a restart (config is read once at boot).
+- **health probe uses `/ping`, not `/health`** — `/health` is always-200 by design
+  (watcher-down is report *content*, not a status code), so it must never be the
+  liveness probe or it would restart the wrong container.
+
+## Config (single-writer, infra-ops)
+
+Real config with bearer-key secrets lives on the server at
+`/opt/docker/conf/muninn-gate/muninn-gate.yaml` (`1000:1000`, `0600`, gitignored).
+Redacted schema: `conf/muninn-gate.example.yaml`. Schema is CLOSED — any unknown
+field is a boot failure. Keys are flat-scope (`read | submit | control`, no
+inheritance); `name` is the non-secret `submitted_by` identity.
+
+## Build + deploy
+
+Image builds out-of-band — the Gitea read token (for `muninn-dispatch==0.1.4` from
+the internal index) rides as a BuildKit secret, never a layer:
+
+```bash
+DOCKER_BUILDKIT=1 docker build --secret id=gitea_pw,src=<gitea-read-token> -t muninn-gate:0.0.14 .
+docker compose up -d          # after installing the config to /opt/docker/conf/muninn-gate/
+```
+
+## Status
+
+Booted + healthy 2026-07-31; `/ping` + `/health` (watcher:running:true,
+ingestion_root_writable:true) verified. **SUBMIT deferred** — returns `not_found`
+against the placeholder staging until the real shared mount + mimir-inbox writer
+exist; then repoint + the one-file path-agreement probe + the real acceptance run.
@@ -0,0 +1,70 @@
+# muninn-gate — WG-internal HTTP front door for the Muninn ingestion queue (#377).
+# Deployed on corviduo-dev (10.250.50.152), co-located with the worldtree-personal
+# muninn watcher (worldtree-personal-worldtree-muninn-1). infra-ops-managed stack,
+# separate from the worldtree CI/CD compose.
+#
+# Image is built out-of-band (the Gitea read token rides as a BuildKit secret, so
+# it never lands in a layer) — see README.md:
+#   DOCKER_BUILDKIT=1 docker build --secret id=gitea_pw,src=<token> -t muninn-gate:0.0.14 .
+#
+# Real config (with bearer-key secrets) lives on the server at
+# /opt/docker/conf/muninn-gate/muninn-gate.yaml (gitignored); repo carries the
+# redacted conf/muninn-gate.example.yaml.
+services:
+  muninn-gate:
+    image: muninn-gate:0.0.14
+    container_name: muninn-gate
+    restart: unless-stopped
+    # Run as the ingestion-owner uid (vh, 1000:1000) so the gate can WRITE the
+    # queue (ingestion dir is 1000:1000 0755). Overrides the image's build user;
+    # non-root, least-privilege (no root needed — binds :8090, reads config +
+    # staging, writes only ingestion). The mounted config is chowned 1000:1000 0600.
+    user: "1000:1000"
+    # WG-internal bind straight to the host WG address; the launcher owns the bind
+    # (the app's `bind` field is observability-only). Host networking so --host/--port
+    # land directly on corviduo-dev.
+    network_mode: host
+    command:
+      - uvicorn
+      - muninn_gate.entrypoint:app_factory
+      - --factory
+      - --host
+      - "10.250.50.152"
+      - --port
+      - "8090"
+    environment:
+      MUNINN_GATE_CONFIG: /app/config/muninn-gate.yaml
+    volumes:
+      # ingestion_root = /data/state/ingestion — the SAME volume the personal muninn
+      # watcher uses, byte-identical path in both containers (absolute-path agreement).
+      # NOTE (hardening candidate): whole state volume per muninn-dev's spec; a subpath
+      # mount of just `ingestion` -> /data/state/ingestion would be tighter (gate only
+      # needs RW on ingestion). Flag to muninn-dev before adopting.
+      - worldtree-personal_worldtree-state:/data/state
+      # single-writer mounted config (bearer-key secrets), read-only:
+      - /opt/docker/conf/muninn-gate/muninn-gate.yaml:/app/config/muninn-gate.yaml:ro
+      # shared staging root — SAME absolute path as the watcher's bind, read-only
+      # (gate only resolves + reads staging; never writes there):
+      - /mnt/muninn-staging/mimir-inbox:/mnt/muninn-staging/mimir-inbox:ro
+    healthcheck:
+      # /ping is the one anonymous route; returns {"service":"ok"}. NB /health must
+      # NOT be the liveness probe — it is always-200 by design even when the watcher
+      # is down, so it would never restart the gate (it would wrongly signal the
+      # watcher's container).
+      test: ["CMD", "python", "-c", "import urllib.request,sys; sys.exit(0 if b'ok' in urllib.request.urlopen('http://10.250.50.152:8090/ping', timeout=3).read() else 1)"]
+      interval: 30s
+      timeout: 5s
+      retries: 3
+      start_period: 10s
+    labels:
+      - homepage.group=Worldtree
+      - homepage.name=Muninn Gate
+      - homepage.icon=mdi-gate-arrow-right
+      - homepage.description=Muninn ingestion queue front door (#377)
+      - homepage.href=http://10.250.50.152:8090/ping
+
+volumes:
+  # The worldtree-personal muninn watcher's state volume — external, managed by the
+  # worldtree-personal compose. We attach to it read/write for the ingestion queue.
+  worldtree-personal_worldtree-state:
+    external: true
@@ -0,0 +1,32 @@
+# muninn-gate mounted config — REDACTED EXAMPLE (committed).
+# The real file (with bearer-key secrets) lives on corviduo-dev at
+# /opt/docker/conf/muninn-gate/muninn-gate.yaml and is gitignored.
+#
+# Schema is CLOSED: an unknown field (top-level or per-key) is a BOOT FAILURE,
+# not a warning. Read once at boot; every validation failure aborts before the
+# socket binds. Four top-level fields only.
+
+# Absolute, existing directory, same volume as the personal muninn watcher.
+# Byte-identical to the watcher's view: /data/state/ingestion in both containers.
+ingestion_root: /data/state/ingestion
+
+# Submit-path allowlist (non-empty; each absolute, existing, a directory).
+# Canonicalized at boot. Ratified 2026-07-30. Currently a LOCAL placeholder dir
+# on corviduo-dev; becomes the shared mount when mimir-inbox (the writer) lands.
+staging_roots:
+  - /mnt/muninn-staging/mimir-inbox
+
+# Bearer credentials. `name` = non-secret caller identity, recorded as
+# `submitted_by`. Flat scopes (no hierarchy/inheritance): read | submit | control.
+# Names AND key values must both be unique; a padded key fails boot.
+keys:
+  - name: mimir-inbox
+    key: <64-hex-secret, provisioned by infra-ops>
+    scopes: [read, submit]
+  - name: ops-curl
+    key: <different 64-hex-secret>
+    scopes: [read, submit, control]
+
+# WG-internal address — observability only, NOT read by the app. Keep consistent
+# with the launcher's --host/--port (the launcher is authoritative).
+bind: 10.250.50.152:8090