Files
ratatoskr/docs/canonical-sync.md
vh 9703eb2b6b init: seed Ratatoskr from corviduo-project-template + ship v0 scaffold
Worldtree Conversation API debug TUI. Multi-pane observability dashboard:
chat transcript + persona/Vili affect log + tool events + admin events +
Bifrost state + tool inventory + (opt-in) raw server log.

Design locked at docs/design-brief.md (originated as
brokkr-smithy/docs/ratatoskr-design-brief.md). Operator-locked decisions:

- Textual application-shell framework (multi-pane dashboard, not REPL).
- Separate repo + separate dev team (no Worldtree-source imports).
- httpx-sse for SSE consumption (reference Python SSE-resume impl).
- Triple version-skew mitigation: spec-pin in pyproject.toml + recorded
  SSE snapshot tests + conformance smoke. Initial pin: Worldtree v0.19.0
  at 55101e909abcd2219833266b6f905c5bc956e0f0.
- Persona pane: label-don't-refuse PII posture.
- Server-log pane: opt-in via --server-log <path>.
- Two-stage Ctrl-C (cancel then exit).
- Markdown rendering default-on; --raw opt-out.

In the box:

- docs/design-brief.md — the locked design with full rationale.
- docs/SPEC-PIN.md — Worldtree spec pin + bump procedure.
- docs/conversation-api-spec.md + docs/conversation_api.contract.md —
  vendored Worldtree spec snapshots at the pinned SHA.
- pyproject.toml — Python 3.12, hatchling, uv-managed, deps locked.
- src/ratatoskr/ — stub package (cli.py raises NotImplementedError).
- tests/test_no_worldtree_imports.py — boundary smoke test PASSING.
- tests/snapshots/README.md — recording convention for SSE snapshot tests.

Not in the box yet:

- Gitea remote (operator/infra-ops to register at vh/ratatoskr).
- Implementation — the dev team owns this; design brief is the spec.

Origin: althing thread 01KS3R34XD3N6HMK91VXESHGW7 (worldtree-dev →
brokkr-smithy-dev, 2026-05-20). Volva consulted via thread
01KS3VF6W33N3V5FNMGQ91YNVD.
2026-05-20 20:38:22 -07:00

11 KiB

Canonical-sync — the pattern, the tooling, and the documented adopters

Snapshot: 2026-05-18.

Canonical-sync is Corviduo's discipline for keeping a spec or piece of tooling byte-identical across the projects that depend on it, while making the points where they're not identical explicit and detectable in CI.

This file describes what canonical-sync is, why it exists, how to adopt it, and which Corviduo projects have adopted it. The pattern is older than this documentation — it's been in production across Worldtree, Galdrabok, and the template's own self-pins since 2026-05-15 — but this is the first dedicated home for it.

The hazard it closes

Silent spec drift. A consumer integrates against a canonical spec at version vN. The canonical's owner ships vN+1 with a wire-surface or contract change. The consumer keeps its old copy and continues to compile. Nothing detects the drift until something breaks at runtime (a 500, a {messages: []} defensive return, an architect sign-off that doesn't match the shipped behaviour).

The pattern's premise: the consumer should know its pin at every CI run, not at the next bug.

Mechanism

Two files + two scripts.

<project>-canonicals.toml (per-repo manifest)

A TOML file at the repo root declaring every canonical the project pins — either ones the project owns (self-pins) or ones it consumes from another canonical owner.

Each pin records:

[[pins]]
id = "<artifact-slug>-v<major>"
canonical_source = "<owner-repo-slug>"
canonical_path = "<path within the canonical-owner's repo>"
consumer_path = "<path within this repo>"
pinned_sha256_16 = "<16-hex-char prefix of sha256(canonical-file)>"
pinned_at = "<ISO-8601 timestamp of last sync>"

For self-pins (consumer = canonical), canonical_path == consumer_path and canonical_source is this repo's slug. For consumed canonicals, the consumer points at the canonical-owner's repo path and keeps a local mirror.

scripts/canonical_sync.py + scripts/canonical_drift.py

  • canonical_sync.py — surgical line-rewriter that updates pinned_sha256_16 + pinned_at from the canonical source. Preserves comments, key order, and blank lines. For consumed canonicals, also rewrites consumer_path content to match canonical.
  • canonical_drift.py — read-only verifier. Returns OK / WARN / STALE / DIVERGED / MISSING per pin. Suitable as a CI gate.

Both scripts are themselves canonicals (self-pinned in this template). They sync via the same mechanism they implement.

How to adopt

As a canonical publisher (your project owns a spec others should pin)

  1. Choose a stable path for the canonical artifact (versioned filename if appropriate: docs/spec-v0.3.md).
  2. Create .<project>-canonicals.toml at the repo root with a self-pin entry pointing at the artifact.
  3. Run canonical_sync.py to populate the SHA + timestamp.
  4. Add docs/canonicals.md (or similar) describing the pin for downstream consumers + the v → v<N+1> bump procedure.
  5. Announce on althing that the spec is canonical-sync ready; downstream consumers add an alias-table entry.

As a canonical consumer (you pin against someone else's spec)

  1. Add an entry to your project's .corviduo-canonicals.toml:
    [[pins]]
    id = "<publisher-artifact-slug>-v<major>"   # e.g., "bifrost-protocol-spec-v0.3"
    canonical_source = "<publisher-repo-slug>"
    canonical_path = "<path in publisher's repo>"
    consumer_path = "<your local mirror path>"
    pinned_sha256_16 = "<copy from publisher's manifest>"
    pinned_at = "<copy from publisher's manifest>"
    
  2. Optionally run canonical_sync.py to fetch the canonical content into consumer_path (rather than hand-mirror).
  3. Add python scripts/canonical_drift.py to CI; failure on drift gates merge.

Bump procedures

Three distinct procedures depending on how much the canonical change affects consumers. The classification axis is consumer-impact — not "how much did the publisher rewrite":

Path What changed Pin-id Path on disk Coordination
Spec-prose-only patch Typo / clarification / link / References-section / non-semantic prose unchanged unchanged publisher ships → publisher pings → consumers re-sync
Wire-surface patch at same major Semantic change consumers might notice but won't break under unchanged unchanged publisher pre-coordinates with steward → joint call on parallel-pin-within-major vs roll-to-major → only then ship
Major bump (v → v<N+1>) Shape change requiring consumer adaptation new (-v<N+1>) new path new spec at new path → publisher's manifest entry replaces v with v<N+1> → consumers parallel-pin during migration window

The pin-id naming convention (<artifact-slug>-v<major>) holds across patch bumps in both flavours; only major bumps move the pin-id.

Major bump (v → v<N+1>): new shape requires consumer adaptation

  1. Author the new version's artifact at a new path (spec-v<N+1>.md).
  2. Update the publisher's own <project>-canonicals.toml pin entry (new path, new SHA, new timestamp, new pin-id <artifact-slug>-v<N+1>).
  3. Announce on althing to alias-table consumers. Include: new spec path, new pin-id, what changed at the consumer-facing wire surface, and the migration window expectation.
  4. Consumers add a parallel pin for v<N+1> alongside their existing v pin during the migration window.
  5. Consumers drop the v pin once they've integrated v<N+1>.

The pattern doesn't dictate the migration window length; that's per-spec.

Spec-prose-only patch (cheap path)

The common case for bug fixes that don't touch wire-surface, prose clarifications, link fixes, or References-section updates. Path stays the same, pin-id stays the same, only pinned_sha256_16 + pinned_at move. Consumers don't need to adapt anything; they just track.

  1. Publisher updates the spec content at the same path (spec-v<N>.md).
  2. Publisher runs canonical_sync.py on their own <project>-canonicals.toml → refreshes own pinned_sha256_16 + pinned_at.
  3. Announce on althing to alias-table consumers with: the new SHA, the one-line "what changed in the spec" summary, and an explicit "prose-only, no wire-surface change" tag. Proactive notification is load-bearing — consumers' canonical_drift.py will catch the change reactively at next CI run, but the announcement makes the change consumer-tracked on the publisher's timeline, not on next-CI-fire.
  4. Consumers run canonical_sync.py on their manifests → drift-check surfaces the change; sync refreshes their pin's SHA + timestamp. Alias-table entries (id / canonical_source / canonical_path / consumer_path) are unchanged.

Wire-surface patch at same major (pre-coordinate)

The middle ground: a semantic change consumers might notice but won't break under (e.g., new error-code enum value, new optional field, broader response shape) — too small for a major bump, too consumer-visible for the cheap path. Most prone to going wrong under urgency (publisher ships fast because something's blocking; alias-table consumers find out reactively when CI fires).

  1. Publisher pre-coordinates with the canonical-sync steward (currently Brokkr-Smithy) before committing the spec change. Include: what wire-surface change is being made, what consumer-side adaptation (if any) would catch it, and the publisher's urgency context.
  2. Steward + publisher decide jointly: parallel-pin-within-major (publisher introduces an interim sub-version note in the spec; consumers add an alongside pin during the migration window — same path, same major id, but tracked) or roll to major (if consumer adaptation is non-trivial, escalate to a v<N+1> with the major-bump procedure).
  3. Publisher ships the agreed shape.
  4. Publisher announces on althing per the chosen shape (parallel-pin-within-major → consumers re-sync + note the interim shape; roll-to-major → major-bump path).

The pre-coordination step is the load-bearing one. By making it a hard step (not "ping after the patch ships"), the procedure prevents the worst version of patch-under-urgency: a wire-surface change shipped fast without anyone realising it broke consumers' assumptions. Don't ship unilaterally on this path.

Decision rule (which path?)

If unsure which patch flavour applies, ask:

  • Would a consumer pinned at the old SHA break under the new SHA's wire bytes? Yes → wire-surface (middle path). No → spec-prose-only (cheap path).
  • Would the consumer need to adapt their code, contract, or tests to integrate the change? Yes → major bump (or rare wire-surface-at-same-major if the change is genuinely backward-compatible). No → spec-prose-only.

A "patch" that breaks consumers is a major miscategorized — back out and re-bump as major with a new pin-id. The middle path (wire-surface at same major) exists specifically for the case where the wire surface evolves additively (new error code, new optional field) but pinned-against-old-SHA consumers continue to work — that's where pre-coordination buys the parallel-pin window without forcing a major.

Documented adopters

These Corviduo projects have adopted canonical-sync. Each is either a canonical publisher (owns a spec/tool that others pin), a canonical consumer (pins against someone else's canonical via alias-table), or both.

Project Role Adoption commit(s) Pinned canonicals (representative)
corviduo-project-template (this repo) Publisher (self) always contract-format-v2, principles-v1, methodology-v1, canonical-sync-v1, canonical-drift-v1, contract-parser-v2, contract-drift-check-v1
Worldtree Publisher + Consumer 40dd9bf (publisher init); cb01c6f (refinement); 60a8853 (Bifrost consumer) Publishes: conversation-api-client-spec-v2. Consumes: template canonicals (contract-format, methodology, etc.) + bifrost-protocol-spec-v0.3
Galdrabok Consumer 94b7ad4 Template canonicals
Bifrost Publisher a9359b4 (v0.3 release) Publishes: bifrost-protocol-spec-v0.3

New adopter? Open a PR adding a row to this table when your project's manifest lands on main.

Cross-references

  • .corviduo-canonicals.toml (this repo) — self-pin manifest for the template's own canonicals
  • scripts/canonical_sync.py — the sync mechanism
  • scripts/canonical_drift.py — the verification mechanism
  • docs/methodology.md § "What this file is, and isn't" — brief reference to canonical-sync in the broader methodology
  • Per-project docs/canonicals.md files (Bifrost, Worldtree, etc.) — operator-facing pin-setup + bump-procedure documentation specific to each canonical owner

Stewardship

Brokkr-Smithy is the current steward of the canonical-sync pattern and the canonical_sync.py / canonical_drift.py scripts. Stewardship is path-attributed (the tooling lives in corviduo-project-template), not agent-attributed — if stewardship shifts the path stays correct.