feat(mimir-inbox): book-ingestion UI stack on corviduo-dev:8091 (#377)
WG-internal FastAPI+HTMX front end for large-document ingestion into the Muninn KB, over the muninn-gate API (browser -> mimir-inbox -> staging -> path-addressed POST /jobs). Co-located on corviduo-dev with the gate (:8090) and the worldtree-personal muninn watcher per the operator's 2026-08-01 co-location ruling (reversing the earlier off-box/NFS plan; worldtree-dev approved the box placement). - Dockerfile: python:3.11-slim + uv sync --no-dev --frozen (--no-dev is load-bearing; the dev group's muninn-dispatch path source is absent in-image and INV-MI-7 forbids importing it). Single-stage by design — src/ stays in the final image (uv installs the project editable-linked to src/). - compose.yaml: uid 1000, host-net bind 10.250.50.152:8091, staging :rw, TCP-liveness healthcheck (deliberately not coupled to gate reachability). - Built from vh/mimir-inbox HEAD c8ab38f; deployed + healthy. Records the open-in-place claim semantics (worldtree-dev, runner.py:362-367) and the INV-MI-19 retention rule (staged files persist until job terminal; gate retry returns a false-200 on a swept source) in persistent memory.
This commit is contained in:
@@ -0,0 +1,33 @@
|
||||
# mimir-inbox runtime config — copy to .env (gitignored on server) and fill the key.
|
||||
# The 7 RUNTIME vars ONLY. The two DEV-ONLY drift-guard vars
|
||||
# (WORLDTREE_CONFIG_PATH, MUNINN_GATE_REPO) MUST NOT be set here — they belong
|
||||
# only to the test environment. All are validated at boot; any failure aborts
|
||||
# startup with a secret-redacted message naming the variable.
|
||||
|
||||
# muninn-gate front door — the co-located gate on corviduo-dev (host network).
|
||||
MUNINN_GATE_URL=http://10.250.50.152:8090
|
||||
|
||||
# infra-ops-provisioned bearer (read+submit+control); SERVER-side only, never logged.
|
||||
MUNINN_GATE_BEARER_KEY=__FILL_ON_SERVER__
|
||||
|
||||
# Shared staging root — RATIFIED value (not an example). Must exist, be a dir, and
|
||||
# be writable by uid 1000; canonicalized at boot.
|
||||
MIMIR_INBOX_STAGING_ROOT=/mnt/muninn-staging/mimir-inbox
|
||||
|
||||
# The RETRY WINDOW (not a tidiness setting): staged files are retained until the
|
||||
# job is terminal + this many days. Shortening it shortens retryability
|
||||
# (INV-MI-19 / worldtree-dev open-in-place). Do not tune down without telling
|
||||
# mimir-dev.
|
||||
MIMIR_INBOX_STAGING_RETENTION_DAYS=7
|
||||
|
||||
# Wing allowlist — name:description pairs, ";"-separated (first ":" splits
|
||||
# name/description; a name is one path component; order = select order). Names
|
||||
# MUST be upstream Worldtree library.wings ids. Boot-fails if empty.
|
||||
MIMIR_INBOX_WINGS=fiction:Novels, novellas and short stories;main:General knowledge — non-fiction, manuals and technical reference
|
||||
|
||||
# Upload ceiling in MiB, enforced mid-stream.
|
||||
MIMIR_INBOX_MAX_UPLOAD_MB=512
|
||||
|
||||
# Unclean-terminal float window (days) — how long a failed / complete_with_warnings
|
||||
# job sorts above clean ones before settling into chronological order.
|
||||
MIMIR_INBOX_FLOAT_WINDOW_DAYS=7
|
||||
@@ -0,0 +1,37 @@
|
||||
# mimir-inbox — UI front end for large-document / book ingestion into the Muninn
|
||||
# KB, over the muninn-gate API (browser -> mimir-inbox -> staging -> path-addressed
|
||||
# POST /jobs to the gate; bytes never reach the gate). No Dockerfile upstream
|
||||
# (vh/mimir-inbox); this is infra-ops's, modeled on the muninn-gate one.
|
||||
#
|
||||
# All 7 RUNTIME deps are public PyPI — NO private index, NO BuildKit secret
|
||||
# (unlike muninn-gate, which needed muninn-dispatch at runtime).
|
||||
#
|
||||
# docker build -t mimir-inbox:0.0.1 .
|
||||
FROM python:3.11-slim
|
||||
|
||||
RUN pip install --no-cache-dir uv==0.11.*
|
||||
|
||||
WORKDIR /app
|
||||
COPY pyproject.toml uv.lock README.md ./
|
||||
COPY src ./src
|
||||
|
||||
# Install project + deps from the lockfile, EXCLUDING the dev group. --no-dev is
|
||||
# load-bearing, not just leaner: the dev group carries muninn-dispatch via a path
|
||||
# source ("../Worldtree/packages/muninn-dispatch") that is absent in the image,
|
||||
# AND INV-MI-7 asserts runtime code never imports muninn_dispatch. Runtime deps
|
||||
# resolve entirely from PyPI.
|
||||
RUN uv sync --no-dev --frozen --no-cache
|
||||
|
||||
# SINGLE-STAGE BY DESIGN — DO NOT convert to multi-stage that copies only .venv.
|
||||
# `uv sync` installs the project EDITABLE-linked back to /app/src/mimir_inbox (not
|
||||
# copied into site-packages), so mimir_inbox.__file__ — and the 16 templates + 442
|
||||
# static assets (vendored design CSS + htmx) it serves from — resolve out of src/.
|
||||
# Dropping src/ in a final layer would kill the app at import; if it somehow got
|
||||
# past import it would 404 every asset. src/ MUST remain alongside .venv.
|
||||
# (mimir-dev preflight, 2026-08-01: `uv sync --no-dev --frozen` boots green with
|
||||
# the path source absent; the alternative safe shape is `uv build --wheel` +
|
||||
# install the wheel, which copies assets into site-packages.)
|
||||
ENV PATH="/app/.venv/bin:$PATH"
|
||||
|
||||
# The launcher (compose) owns --host/--port; no CMD here. Runs as uid 1000 (set
|
||||
# in compose) — the sole writer to the staging root; /app is read-only to it.
|
||||
@@ -0,0 +1,76 @@
|
||||
# mimir-inbox
|
||||
|
||||
UI front end for large-document / novel / book ingestion into the Muninn KB —
|
||||
server-side FastAPI + HTMX over the **muninn-gate** API (#377). The submit path
|
||||
is: **browser → mimir-inbox `/upload` (writes to staging) → path-addressed
|
||||
`POST /jobs` to the gate**. Bytes never reach the gate; the gate references the
|
||||
staged file by absolute path, and the watcher opens it **in place** at claim.
|
||||
|
||||
- **Upstream:** `git@gitea.phasefinal.com:vh/mimir-inbox.git` (infra-ops owns
|
||||
containerization — no Dockerfile upstream).
|
||||
- **Host:** corviduo-dev `10.250.50.152:8091`, co-located with muninn-gate
|
||||
(:8090) and the worldtree-personal muninn watcher. Operator ruled co-location
|
||||
2026-08-01 (reversing the earlier off-box/NFS plan); worldtree-dev approved
|
||||
the box placement.
|
||||
- **Posture:** LAN/WG-internal, no TLS, no public exposure (same as the gate).
|
||||
No datastore, no migrations, no writable path other than the staging root — a
|
||||
restart loses nothing.
|
||||
|
||||
## Why co-located (not off-box/NFS)
|
||||
|
||||
The watcher opens the staged file **in place** at claim (`core/muninn/runner.py`),
|
||||
so staging's only hard constraint is **path identity** — the writer, gate, and
|
||||
watcher must resolve the same absolute string to the same bytes. Co-location on
|
||||
corviduo-dev buys that outright with one local filesystem and sidesteps the NFS
|
||||
failure modes (path-identity drift, TOCTOU widening, stale handles, a hung mount
|
||||
blocking the async staging check on mimir-inbox's event loop). Multi-host stays
|
||||
viable later behind a same-path shared mount, but co-location is the cheapest
|
||||
way to path identity.
|
||||
|
||||
## Retention rule (load-bearing — INV-MI-19)
|
||||
|
||||
Because the watcher opens the file in place, **a staged file must persist from
|
||||
submit until the job is terminal** (complete, or failed-and-not-retried) — retry
|
||||
re-runs the structure phase, which re-opens the same staged path. Deleting on
|
||||
201-submit would kill every retried job at claim with a `not_found` that looks
|
||||
exactly like a namespace mismatch. `MIMIR_INBOX_STAGING_RETENTION_DAYS` is the
|
||||
**retry window**, not a tidiness knob; do not tune it down without telling
|
||||
mimir-dev.
|
||||
|
||||
## Config
|
||||
|
||||
All runtime config is environment variables — no config file. The real values
|
||||
(including the bearer-key secret) live in a gitignored `.env` next to
|
||||
`compose.yaml` on the server (`0600`); the repo carries `.env.example`. Two
|
||||
DEV-ONLY drift-guard vars (`WORLDTREE_CONFIG_PATH`, `MUNINN_GATE_REPO`) must
|
||||
**not** be set in the container.
|
||||
|
||||
## Deploy
|
||||
|
||||
```bash
|
||||
# 1. Fetch the source at the pinned tag/HEAD onto corviduo-dev, build the image
|
||||
# (all deps public PyPI; uv sync --no-dev is load-bearing — see Dockerfile):
|
||||
docker build -t mimir-inbox:0.0.1 .
|
||||
|
||||
# 2. Provision the shared staging root (mimir-inbox-writable, gate+watcher :ro):
|
||||
sudo install -d -o 1000 -g 1000 -m 0755 /mnt/muninn-staging/mimir-inbox
|
||||
|
||||
# 3. Drop the real .env (0600) next to compose.yaml with the bearer key filled
|
||||
# from the infra-ops key drop (~/.config/muninn-gate/mimir-inbox-env.sh).
|
||||
|
||||
# 4. Bring it up:
|
||||
docker compose up -d
|
||||
|
||||
# 5. muninn-dev's one-file path-agreement probe, then the acceptance run.
|
||||
```
|
||||
|
||||
**Operational guard:** docker fabricates a MISSING bind source as an empty dir
|
||||
that passes checks — confirm `/mnt/muninn-staging/mimir-inbox` exists on the host
|
||||
before `up -d` (step 2 does this). The container also fails boot if the staging
|
||||
root is not writable (`os.access(W_OK)`), so a perms miss surfaces loudly.
|
||||
|
||||
## Health
|
||||
|
||||
`GET /health-banner` is the HTMX health surface (reflects gate reachability).
|
||||
The container healthcheck is a **TCP-connect liveness** probe only — it must not
|
||||
couple to the gate's state, so a gate outage never restarts mimir-inbox.
|
||||
@@ -0,0 +1,59 @@
|
||||
# mimir-inbox — large-document / book ingestion UI over the muninn-gate API (#377).
|
||||
# Deployed on corviduo-dev (10.250.50.152), co-located with muninn-gate (:8090)
|
||||
# and the worldtree-personal muninn watcher. Operator ruled co-location 2026-08-01
|
||||
# (reversing the earlier off-box/NFS plan); worldtree-dev approved the box
|
||||
# placement. infra-ops-managed stack, separate from the worldtree CI/CD compose.
|
||||
#
|
||||
# Image built out-of-band (all deps public PyPI, no secret needed):
|
||||
# docker build -t mimir-inbox:0.0.1 .
|
||||
#
|
||||
# Runtime config (incl. the bearer-key secret) lives on the server in a gitignored
|
||||
# .env next to this compose; repo carries the redacted .env.example.
|
||||
services:
|
||||
mimir-inbox:
|
||||
image: mimir-inbox:0.0.1
|
||||
container_name: mimir-inbox
|
||||
restart: unless-stopped
|
||||
# uid 1000 (vh) — the SOLE writer to the staging root (INV-MI-3), which is
|
||||
# provisioned 1000:1000. Matches the corviduo-dev muninn stack (gate is 1000).
|
||||
user: "1000:1000"
|
||||
# Host networking: the WG-internal bind lands directly on corviduo-dev, and
|
||||
# MUNINN_GATE_URL=http://10.250.50.152:8090 reaches the co-located gate (whose
|
||||
# own bind is host-network on that address).
|
||||
network_mode: host
|
||||
# The 7 runtime vars incl. the secret. The two DEV-ONLY drift-guard vars
|
||||
# (WORLDTREE_CONFIG_PATH, MUNINN_GATE_REPO) are deliberately absent — they
|
||||
# must not be set in the container.
|
||||
env_file:
|
||||
- .env
|
||||
command:
|
||||
- uvicorn
|
||||
- --factory
|
||||
- mimir_inbox.app:create_app
|
||||
- --host
|
||||
- "10.250.50.152"
|
||||
- --port
|
||||
- "8091"
|
||||
volumes:
|
||||
# Shared staging root — mimir-inbox is the SOLE WRITER; the gate + watcher
|
||||
# bind this SAME absolute path :ro. A corviduo-dev-LOCAL dir (path identity
|
||||
# is the real constraint; the watcher opens staged files IN PLACE at claim
|
||||
# — worldtree-dev, core/muninn/runner.py:362-367 — so co-location buys path
|
||||
# identity outright and nothing crosses a device boundary).
|
||||
- /mnt/muninn-staging/mimir-inbox:/mnt/muninn-staging/mimir-inbox:rw
|
||||
healthcheck:
|
||||
# LIVENESS ONLY — a TCP connect proves uvicorn is bound, WITHOUT coupling
|
||||
# container health to gate reachability. A gate outage must NOT restart
|
||||
# mimir-inbox; the health banner surfaces that state instead (same lesson
|
||||
# as the gate's /ping-not-/health probe).
|
||||
test: ["CMD", "python", "-c", "import socket; socket.create_connection(('10.250.50.152',8091),3).close()"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 10s
|
||||
labels:
|
||||
- homepage.group=Worldtree
|
||||
- homepage.name=Mimir Inbox
|
||||
- homepage.icon=mdi-book-arrow-right
|
||||
- homepage.description=Large-document ingestion UI for the Muninn KB (#377)
|
||||
- homepage.href=http://10.250.50.152:8091/
|
||||
Reference in New Issue
Block a user