Files

T

vh 66c860d6c1 fix(sweep): retire the dead 10.100.79.3 address across the fleet

Operator-directed. The wg0 lifeline retired at the 2026-09-06 headscale cutover is
on no interface anywhere, so anything pointing at it gets no route at all. Homepage
went from 9 dead cards to 0 of 112.

The load-bearing part is that there is no single right target: it depends on who
resolves it. The operator's browser and the Homepage and open-webui containers on
esh-docker-vm all resolve nh3.internal, so those get the name and survive the next
renumber. Containers on irv-ml1 and ana-docker cannot resolve it at all, so those
get the IP.

litellm on ana-docker looked like a counterexample and is not: it resolves the name
only through its own extra_hosts entry, while asset-engine on the same host fails on
it. Test from the container you are about to change, never from a neighbour. Before
committing to the name I confirmed the Homepage container actually fetches ytvc's
healthz through it in production rather than assuming resolution implies reach.

On irv-ml1, 24 files swept and 14 comment-only hits left as port-allocation history.
Seven running containers recreated so the labels took. Seven dormant ones carried
stale labels because editing a compose file does not touch an existing container
object - fixed with compose create --force-recreate, which rebuilds the container
without starting it, the right tool for a deliberately dormant stack.

The sweep's real find was off irv-ml1 entirely: four live values on two other hosts,
silently dead for nine days and alerting nobody. Open WebUI's read-aloud TTS,
asset-engine's inference host, and two skaldsong TTS URLs. Both running services were
recreated and verified reaching their targets afterwards rather than merely carrying
the new string.

One self-inflicted outage worth recording: I recreated breeze-tts for a cosmetic
label change and took ext-tts down for its ~90s CUDA-graph warm-up, returning 500. I
caught it only because I had taken a baseline before touching it. A label-only edit
still costs a full model reload on a GPU container.

2026-09-15 08:41:44 -07:00

.env.example

fix(sweep): retire the dead 10.100.79.3 address across the fleet

2026-09-15 08:41:44 -07:00

compose.yaml

chore(fleet): repoint stale irv-ml1 refs (10.100.79.3 -> irv-ml1.nh3.internal)

2026-09-07 15:08:56 -07:00

README.md

skaldsong: align env-var contract with app reality (pre-first-deploy)

2026-05-18 23:10:37 -07:00

README.md

skaldsong

Wizard + reader surface for multi-agent storytelling. Single FastAPI process serves both the SvelteKit-built SPA (static, baked into the image) and the wizard backend. Talks to Worldtree as a Bifrost consumer — Worldtree issues per-dispatch JWTs (post-v0.3) that skaldsong's /bifrost endpoint verifies against the shared HS256 secret.

Server: ana-docker URL: http://10.250.50.70:8300 (configurable via .env) Upstream repo: vh/skaldsong Image: gitea.phasefinal.com/vh/skaldsong:<sha> — built + pushed by vh/skaldsong's CI on every push to main and on tag.

Path layout (on ana-docker)

Host path	Container path	Purpose	Restic?
`/opt/docker/compose/skaldsong/`	—	compose.yaml + .env	included (via `/opt/docker`)
`/opt/docker/conf/skaldsong/db/`	`/app/state/db`	SQLite (`skaldsong-ui.db` + WAL/SHM) — wizard state, story rows	included
`/opt/docker/conf/skaldsong/runs/`	`/app/state/runs`	Per-story generation pipeline artifacts (spec, state, output, logs, manuscripts)	included

Network model

Internal tooling, LAN-only. Container's :8000 published on host :8300 (configurable via SKALDSONG_BIND / SKALDSONG_PORT); access direct at http://10.250.50.70:8300. No Traefik, no TLS terminator, no public hostname.

Skaldsong talks to two services on the LAN:

Worldtree at http://10.250.50.152:8080 (corviduo-dev, LAN-direct from ana-docker — same subnet).
Kokoro TTS at http://10.100.79.3:8193 (irv-ml1, reached via the WireGuard tunnel on ana-wg — same path asset-engine uses).

Worldtree calls back to skaldsong's /bifrost endpoint at http://10.250.50.70:8300 for tool-dispatch — that URL must be in Worldtree's BIFROST_CLIENT_ALLOWED_HOSTS. Host:port changes require a coordinated allowlist update on Worldtree-side (see corviduo-dev README for the docker-as-root edit pattern).

Deploy

Two paths — automated (preferred) and manual (escape hatch).

Automated (Gitea Actions, push-to-main)

vh/skaldsong ships .gitea/workflows/deploy.yaml. Push to main + manual workflow_dispatch triggers:

Build + push gitea.phasefinal.com/vh/skaldsong:${{ github.sha }} and :latest to the gitea registry.
Checkout this management repo using MGMT_REPO_TOKEN.
Run scripts/elway ana-docker --playbook playbooks/deploy-skaldsong.yaml --var ref=${{ github.sha }} via DEPLOY_SSH_KEY.

Three secrets required on vh/skaldsong's Actions settings:

DEPLOY_SSH_KEY — private key authorized on ana-docker for lkraven.
MGMT_REPO_TOKEN — Gitea PAT with read:repository on this repo.
REGISTRY_USER + REGISTRY_TOKEN — for the docker login step (token needs write:package scope).

Manual (elway from a workstation)

# Deploy a specific SHA (CI just pushed it):
scripts/elway ana-docker --playbook playbooks/deploy-skaldsong.yaml \
    --var ref=<sha>

# Or pin to a release tag:
scripts/elway ana-docker --playbook playbooks/deploy-skaldsong.yaml \
    --var ref=v0.1.0

Env-var contract

Var	In compose?	In `.env`?	Notes
`SKALDSONG_IMAGE`	✓ (image:)	✓	Full image URI; CI passes SHA-pin via elway `--var ref=<sha>` → playbook → this var.
`SKALDSONG_PORT`	✓	✓	Host-side port (8300 default). Internal always 8000.
`SKALDSONG_BIND`	✓	✓	Host bind address (0.0.0.0 default).
`SKALDSONG_DB_DIR`	✓ (mount)	✓	Host path for SQLite.
`SKALDSONG_RUNS_DIR_HOST`	✓ (mount)	✓	Host path for generation artifacts.
`SKALDSONG_BIFROST_JWT_KEY`	✓ (anchor)	✓	Shared HS256 secret w/ Worldtree's `WORLDTREE_SKALDSONG_USER_KEY`. Same value as `WORLDTREE_TOKEN`.
`SKALDSONG_HOST_BIFROST_ENDPOINT_URL`	✓ (anchor)	✓	URL Worldtree calls back to (must match allowlist).
`WORLDTREE_TOKEN`	✓ (anchor)	✓	Bearer for outbound HTTP to Worldtree. Same value as `SKALDSONG_BIFROST_JWT_KEY`.
`WORLDTREE_BASE_URL`	✓ (anchor)	✓	URL skaldsong calls Worldtree at.
`SKALDSONG_HOST_WIZARD_AGENT_ID`	✓ (anchor)	✓	Existing agent slot ID (`skaldsong:wizard-v2`). Blank burns a Heimdall quota slot on first boot.
`SKALDSONG_HOST_CORS_ORIGINS`	✓ (anchor)	✓	Comma-separated CORS origins.
`SKALDSONG_TTS_ENGINE`	✓ (anchor)	✓	`kokoro` (only engine in v1).
`SKALDSONG_TTS_BASE_URL`	✓ (anchor)	✓	Kokoro URL (WG-routed).

&skaldsong-env anchor declares all SKALDSONG_* vars via ${VAR:-} substitution. Same env-anchor pattern as Worldtree's &worldtree-env (see Worldtree #175 heuristic); new var additions must land in BOTH the .env AND the anchor.

Restart semantics

Per skaldsong's INV-006 (host CLAUDE.md): process restart marks any running generation rows partial and never auto-resumes. Deploy mid-run is safe — the SPA's reader treats partials like a power outage. Cancellation of in-flight runs on deploy is acceptable for v1.

docker compose up -d --force-recreate --pull always is the playbook's apply step (or --pull never when called with a specific --var ref=<sha> where the image was pre-pulled). Same kill-signal + graceful-shutdown pattern as Worldtree.

Worldtree allowlist coordination

The SKALDSONG_HOST_BIFROST_ENDPOINT_URL value MUST be in Worldtree's BIFROST_CLIENT_ALLOWED_HOSTS env var on corviduo-dev. As of deploy-time the value is 10.250.50.70:8300.

If you change SKALDSONG_PORT or move skaldsong off ana-docker, the allowlist needs a paired update on corviduo-dev. Reference: the edit-via-docker-as-root pattern in servers/corviduo-dev/README.md.