feat(mimir-inbox): book-ingestion UI stack on corviduo-dev:8091 (#377)
WG-internal FastAPI+HTMX front end for large-document ingestion into the Muninn KB, over the muninn-gate API (browser -> mimir-inbox -> staging -> path-addressed POST /jobs). Co-located on corviduo-dev with the gate (:8090) and the worldtree-personal muninn watcher per the operator's 2026-08-01 co-location ruling (reversing the earlier off-box/NFS plan; worldtree-dev approved the box placement). - Dockerfile: python:3.11-slim + uv sync --no-dev --frozen (--no-dev is load-bearing; the dev group's muninn-dispatch path source is absent in-image and INV-MI-7 forbids importing it). Single-stage by design — src/ stays in the final image (uv installs the project editable-linked to src/). - compose.yaml: uid 1000, host-net bind 10.250.50.152:8091, staging :rw, TCP-liveness healthcheck (deliberately not coupled to gate reachability). - Built from vh/mimir-inbox HEAD c8ab38f; deployed + healthy. Records the open-in-place claim semantics (worldtree-dev, runner.py:362-367) and the INV-MI-19 retention rule (staged files persist until job terminal; gate retry returns a false-200 on a swept source) in persistent memory.
This commit is contained in:
@@ -11,7 +11,11 @@ WG-internal HTTP front door for the Muninn ingestion queue (`vh/muninn-gate`, mu
|
||||
|
||||
**Verified boot:** `/ping` `{"service":"ok"}`; `/health` (ops-curl bearer, 200) `watcher.running:true` + `ingestion_root_writable:true`. muninn-dev independently poked the live gate — auth/route surface all held (401s w/ `WWW-Authenticate: Bearer`, the 4 FastAPI default routes gone, error-envelope-not-307 on trailing slashes = bug-hunt findings 5+6 confirmed outside pytest).
|
||||
|
||||
**DEFERRED (the submit path) — the mimir-inbox era:** SUBMIT returns `not_found` against the placeholder staging (correct, not a defect — muninn-dev confirmed) until (a) the real **shared mount** is provisioned (needs mimir-inbox's host — pre-implementation) and (b) a mimir-inbox writer exists. Operator ruled **shared mount** (mimir-inbox stays off-box, writes to a shared/NFS mount both gate + watcher bind at the same path). Then: repoint the gate's staging bind + **restart** (config is read ONCE at boot, no reload), muninn-dev's **one-file path-agreement probe** (the only check that can catch a gate/watcher path divergence), then the real acceptance run. **Hand mimir-dev the `mimir-inbox` key** (retrievable via `sudo` from the on-server config).
|
||||
**DEFERRED (the submit path) — the mimir-inbox era:** SUBMIT returns `not_found` against the placeholder staging (correct, not a defect — muninn-dev confirmed) until the real staging dir + a mimir-inbox writer exist. ~~Operator ruled shared mount (mimir-inbox stays off-box, writes to a shared/NFS mount both gate + watcher bind at the same path).~~ **SUPERSEDED 2026-08-01 — operator REVERSED to CO-LOCATE:** mimir-inbox runs ON corviduo-dev, alongside the gate + watcher, staging = a corviduo-dev-LOCAL dir (not NFS). Reason the off-box/NFS call fell: muninn-dev's code-check showed staging is NOT same-fs-constrained (gate reads staging metadata + passes path strings; `os.replace` is inside `ingestion_root`), so staging's real constraint is **path identity across writer/gate/watcher**, which co-location buys outright — and it sidesteps the NFS failure modes (path-identity break, TOCTOU widening, stale handles, a hung mount blocking `resolve(strict=True)` — the last of which blocks mimir-inbox's *event loop*, not just a threadpool worker, since its staging check is in an async handler). Ruling relayed 3× (muninn-dev ×2 w/ msg-id citations, mimir-dev ×2) + operator in-session; **mimir-inbox key handed over 2026-08-01** (bumped to [read,submit,control], 0600 drop on nh3-dev). Tail on the co-locate ruling: raise worldtree-dev (box-side provisioning + the watcher claim-semantics open Q) → provision the real corviduo-dev-local `/mnt/muninn-staging/mimir-inbox` (uid = mimir-inbox's runtime identity, rw-writer / ro-gate+watcher) → 0600 key drop on corviduo-dev → muninn-dev's **one-file path-agreement probe** → acceptance. NB gate submit surface = **`POST /jobs`** (path-addressed; NO `POST /upload` — upload deferred v0, gate never ingests bytes). `staging_roots` already allowlists the path (no gate-config change).
|
||||
|
||||
**RESOLVED 2026-08-01 (worldtree-dev, from source `core/muninn/runner.py:362-367`):** the watcher **OPENS the staged file in place** at claim (`parse_document(file_path)` on the dispatch-recorded absolute path) — it never moves/copies the source into the job dir (job dir holds DERIVED artifacts only). Consequences: (1) staging needs **PATH IDENTITY only**, so **co-location is a CONVENIENCE, not a requirement** — the parked multi-host option stays fully viable with a shared mount at the same absolute path on both hosts. (2) The real same-fs constraint is `.enqueue-tmp/` → `os.replace` into `pending/`, same-fs with `ingestion_root` — never staging (confirms muninn-dev). (3) **⚠️ OPERATIONAL RULE for mimir-inbox lifecycle (worldtree-dev):** open-in-place means the staged file MUST stay present+readable from submit **until the job is TERMINAL** (complete / failed-and-not-retried) — retry re-runs the structure phase, which re-opens the staged path. A cleanup that deletes on 201-submit kills every job at claim with a not-found that looks EXACTLY like the namespace-mismatch failure the bind exists to prevent. Relayed to mimir-dev for their cleanup design. **Gate-side edge (muninn-dev):** `POST /jobs/{id}/retry` returns `200 {requeued}` even for a job whose staged source was deleted — `muninn_dispatch.requeue` validates job STATE not file existence, and admission isn't re-run on retry (nothing re-stats files) → a FALSE success that dies at claim. Gate deliberately unguarded (re-admit re-resolves under a new clock, still races; lifecycle is the writer's), recorded as a gate compatibility constraint. So the retention rule isn't just "avoid claim-fail" — it's "retry will LIE with a 200 if the source is gone."
|
||||
|
||||
**worldtree-dev approved co-location** (2026-08-01): another small infra-ops-managed LAN/WG-internal service on corviduo-dev in the gate's posture is fine at their OS/app layer; port/supervision/identity mine to shape; staging-dir ownership flip (mimir-inbox-writable, gate+watcher :ro — b162 watcher bind already :ro) at my convenience. **NEXT: coordinate the mimir-inbox deploy inputs with mimir-dev** (image/build recipe — likely infra-ops containerizes like muninn-gate; app config/env; port), then provision staging dir + stand up the service (uid 1000, matching the corviduo-dev muninn stack) + 0600 key drop on corviduo-dev + muninn-dev's path-agreement probe + acceptance.
|
||||
|
||||
**Operational guard (no auto-check exists):** docker fabricates a MISSING bind source as an empty dir that passes every closed-config check → **confirm the host mount actually exists before wiring/repointing a bind** (`os.path.ismount` breaks on subdir roots; emptiness is normal pre-first-upload). This is why the gate/watcher path-agreement is an operational discipline, not a validated invariant.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user