Commit Graph

77 Commits

Author SHA1 Message Date
vh e0f1dbfae6 fix(soong-lab-ci): sync web/ frontend on deploy (was serving stale web)
soong-dev found the studio serving a stale web/ (52015 vs 55025 bytes — missing the
01-Role section, favicon, thinking-status): the deploy rsynced backend/ but never web/,
so SOONG_LAB_WEB_DIR stayed pinned to the initial manual copy while the backend updated.
Deploy now rsyncs BOTH backend/->studio AND web/->SOONG_LAB_WEB_DIR (read from the env)
on every green run. Verified: served frontend now 55025 bytes, current.
2026-07-13 15:09:09 -07:00
vh cc0e3af87d feat(soong-lab-ci): red-run althing relay (nh3-dev poll -> ping soong-dev)
Per operator call (no gitea write token on the Worldtree-team VM): a 2-min systemd
--user timer on nh3-dev polls corviduo's last-deploy.json and pings soong-dev via
althing on a NEW red deploy (green stays silent). Delivers soong-dev's red-run
visibility without a credential on corviduo. Tested (red detect+format DRY, green quiet).
2026-07-13 14:28:39 -07:00
vh fb556586e3 feat(soong-lab-ci): green-gated push-to-deploy CI/CD for the soong-lab studio
Vuong-directed. gitea webhook (push→main) → HMAC listener on corviduo-dev:9010 →
clone (read-only deploy key) → uv sync + pytest → redeploy soong-lab-studio.service
ONLY on green (running studio untouched on red). Validated end-to-end 2026-07-13.
Canonical copies of the deploy script + listener + unit; runbook in docs/runbooks.
2026-07-13 14:16:26 -07:00
vh b95802efa4 docs(backup): hourly off-box ~/development backup to nh3-nas (runbook + script)
Adds the rsync --link-dest hourly snapshot job (nh3-dev:~/development ->
nh3-nas, 48-snapshot retention, secrets/build-dirs excluded) that closes the
no-off-box-backup gap exposed by the 2026-07-12 working-dir clobber. Script
mirrors the live ~/.config/dev-backup/dev-backup.sh; runbook covers restore.
2026-07-12 01:40:43 -07:00
vh 6d384dd361 catalog(asset-engine): add zonos-gateway service (full expressive dials)
New TTS service entry + reproducibility_audit row for the zonos-gateway
wrapper (irv-ml1:8890) — the ext-tts-aliased OpenAI facade over Zonos.
23 fields across Text&voice / Expression / Prosody / Quality / Sampling /
Output section groups; live voice dropdown from /v1/voices; response
format pcm|wav (audition UI forces wav). Distinct from the older down
zonos :8203 entry. jsonschema-validated.
2026-07-11 01:02:51 -07:00
vh 8894854127 docs(sampler-defaults): fix char-rp-reasoning seat → Deckard-PKD (was stale QwQ-RpR-v4)
The live gateway config has served char-rp-reasoning as deckard-pkd-27b (:8018)
since the 2026-07-08 A/B; the standalone doc had frozen on QwQ-RpR-v4. Corrects
seat 4 (backend + samplers + server-side DRY/reasoning-budget notes).

Also snapshots session state in persistent-memory.md: phantom-qwen verified
already-clean, ana-docker docker log-cap (logrotate copytruncate, no bounce),
and the granite→gen memory_extractor bind live on demo+personal.
2026-07-09 14:09:13 -07:00
vh aac4bcfa3e feat(litellm): wire canonical sampler defaults for all 4 gateway seats
dvalin-smithy canonical set, infra-ops triaged + char-rp A/B-validated on the live serve.

- gen (+summarizer-large twin): presence_penalty 1.0 -> 1.5 (Qwen3.6 non-thinking rec).
- gen-reasoning: temp 0.6 -> 1.0, presence 1.0 -> 1.5 (Qwen general-thinking profile;
  the old 0.6 was the coding sub-profile).
- char-rp: temp 1.0 -> 1.1, min_p 0.03 -> 0.10, top_k 0, NO rep. A/B on 2 dark-romantasy
  prompts: min_p 0.10 richened imagery; repeat_penalty 1.05 REJECTED (injected a stray
  markdown title, hurts Drummer/Magistral RP creativity per the card + dvalin's own note).
- char-rp-reasoning: add explicit top_p 0.95 (else per the RpR card: no rep/DRY/XTC).

Canonical reference: docs/pfi/model-sampler-defaults.md (mirrors dvalin's derivation).
2026-07-08 11:10:28 -07:00
vh 809c51e095 docs(pfi): sync recommended-model-settings KB to deployed gateway defaults
Add §9 "PFI LiteLLM Gateway — Deployed Sampling Defaults": the live fleet
sampling table (granite/qwen/judges/GLM) with provenance, overrideable-default
semantics, the GLM API-accepted-subset caveat, and the research-confirmed temp-0
rationale for granite + image-judge. Accepts the dvalin-smithy-dev recommendations
as deployed. §§1-8 vendor reference left intact.
2026-06-27 09:45:10 -07:00
vh 5a3a75b73d docs(backups): harden + live-activate /mnt/compose automount on ana-docker 2026-06-20 16:21:05 -07:00
vh 76b317ce3e feat(backups): freshness check + daily alert timer; record rest-server-ana recovery, fstab hardening, esh-pve-nas gap, worldtree admin-key provisioning 2026-06-20 16:13:13 -07:00
vh a7b4a82dec docs(backups): add backup architecture + freshness runbook; record rest-server-ana recovery + correct ana-docker sudo path 2026-06-20 15:56:41 -07:00
vh 981ae4e6a1 feat(omnivoice): expose full generation surface (voice-design, language, diffusion params)
Wrapper /v1/audio/speech now accepts OmniVoice's whole surface:
- voice (clone, now OPTIONAL) and/or instruct (voice DESIGN). instruct is a CONTROLLED
  vocabulary (gender/age/pitch/accent/whisper tags, comma-separated), not free prose —
  discoverable at the new /v1/audio/instruct-items endpoint (23 items).
- language (Auto + 647, new /v1/audio/languages endpoint), speed, duration.
- diffusion controls: num_step, guidance_scale, denoise, preprocess_prompt,
  postprocess_output; plus a generation_overrides JSON passthrough for expert
  GenerationConfig knobs (t_shift, layer_penalty_factor, position/class temperature,
  audio_chunk_*).
- at least one of voice/instruct required (else 400).

Catalog (services.yaml): omnivoice v1 -> v2, 13 schema-valid fields; instruct as a
controlled-vocab text field sourced from the items endpoint.

Verified live on irv-ml1: clone, voice-design (instruct-only), and tuned-param synths
all -> 24 kHz PCM_16 WAV; 647 languages; 23 instruct items.
2026-06-18 23:25:39 -07:00
vh 06eb487a26 feat(omnivoice): wire to asset-engine via FastAPI wrapper + reuse chatterbox voices
- app.py: thin FastAPI wrapper exposing OpenAI /v1/audio/speech (+ /v1/audio/voices,
  /healthz) around OmniVoice's Python API; precomputes a voice-clone prompt per voice
  at startup (loaded Whisper auto-transcribes each reference). Replaces the Gradio demo.
- Dockerfile/compose: run the uvicorn wrapper, /healthz healthcheck, project name pinned
  to "omnivoice" so the asset-engine liveness probe matches.
- deploy-omnivoice.yaml: stage chatterbox /refs/*.wav as clone voices (skip _* artifacts)
  + verify the API surface.
- services.yaml: catalog entry (id omnivoice, :8199/v1/audio/speech, voice list sourced
  live from /v1/audio/voices) + reproducibility_audit row.

Verified live on irv-ml1: /healthz ok, 33 voices loaded, test synth -> 24kHz PCM_16 WAV.
2026-06-18 23:03:20 -07:00
vh 6e58e57362 docs(orientation): gitea internal-route gotcha (fleet hosts -> 10.250.50.70:222)
Fleet/colo hosts must reach gitea over the internal route (ana-docker
container git-SSH at 10.250.50.70:222), not the public gitea.phasefinal.com:22
which fail2bans the host's egress IP and silently wedges webhook auto-deploys.
Bit irv-ml1's arbo deploy 2026-06-13.
2026-06-13 17:04:36 -07:00
vh 5007ec1236 docs(catalog): archive decommissioned + arbo refreshed (502 G post-migration)
storetank archive fully resolved (919 G -> 0): ~739 G killed (superseded/niche),
177 G migrated into arbo, rest dupes. Rewrite the archive doc as a decommission
record; refresh the arbo catalog to its post-migration 502 G state (+ SDXL/Pony
stack + 9 gen-agnostic utility categories).
2026-06-13 15:40:56 -07:00
vh 308ca6f5d2 docs(catalog): record llava_llama3 sweep (919->214 G, 705 G reclaimed)
Swept the orphaned llava_llama3 (HunyuanVideo text encoder, 23.5 G) after the
Hunyuan kill left it unreferenced. Update the curation table + remaining total.
2026-06-13 15:21:11 -07:00
vh 1902425682 docs(catalog): record storetank image-models curation + remaining inventory
Capture the 2026-06-13 archive curation pass (919->238 G, 681 G reclaimed:
Hunyuan + WAN2.1 + FLUX.1 + umt5 orphan, all superseded by arbo's current-gen
stack) and a detailed catalog of the remaining 238 G (SDXL/Pony stack, SD3.5/
Chroma, gen-agnostic utilities, shared encoders) for comfy-dev's migration
decisions into the active arbo set.
2026-06-13 15:18:21 -07:00
vh 38186be1a7 feat(comfyui): migrate 325G model tree worktank -> /storetank/arbo
ComfyUI's ~325G model tree moved off the near-full worktank NVMe (97%->26%,
342G free) to /storetank/arbo (roomy SATA SSD on irv-ml1), overlay-mounted
back at /basedir/models so ComfyUI behaviour is unchanged. rsync byte-verified
(src==dst), one comfyui restart, worktank original removed. Inventory of the
set in docs/arbo-comfyui-model-catalog.md for the retain decision. The older
919G /storetank/image-models/comfy archive is untouched (separate reclaim).
2026-06-13 03:31:56 -07:00
vh cf35adaf6d catalog(canonical): remove chatterbox dead levers + add csm-expressiva
The asset-engine catalog source of truth. Removes the chatterbox
exaggeration/cfg_weight sliders (proven Turbo no-ops) — reconciling the
canonical with the fix previously applied only to asset-engine's vendored
copy — and adds the csm-expressiva whisper TTS entry (irv-ml1:8198).
2026-06-04 18:02:13 -07:00
vh 83b2ec1a8a feat(litellm): add vLLM request/response logging gateway on ana-docker
LiteLLM proxy fronting the vLLM services on ana-ml2 so every request +
response is captured and inspectable in a browser Logs UI — the
visibility vLLM itself lacks (Dozzle shows only connection metadata).

- compose: litellm (proxy + /ui Logs) + litellm-db (Postgres store)
- conf/config.yaml: routes phi4-mini (chat, :8004), qwen3-embedding
  (:8001), qwen3-reranker (:8002); store_prompts_in_spend_logs persists
  full prompt/completion text. reward classifier (:8003) stays direct
  (no first-class LiteLLM route).
- Langfuse-ready: lean first cut intentionally skips Langfuse's heavy v3
  stack; graduating is one env-var + callback step, no re-architecture.
- roadmap: mark the vLLM-observability item's first cut as shipped.

Lean first cut of docs/roadmap.md "Observability for the vLLM stack".
2026-06-04 01:17:50 -07:00
vh 40a374b809 feat(vllm): add phi4-mini FP8 summarizer/dreamer on ana-ml2; retire granite from llama-swap
phi4-mini supersedes the granite-4-small llama-swap pin as the summarizer +
dreaming agent. New vllm-phi4 service: Phi-4-mini-instruct, vLLM-native FP8
(near-lossless on RTX 6000 Ada cc 8.9), 50K ctx, FP8 KV cache, GPU 1, :8004.

llama-swap: removed granite-4-small (depinned) + granite-4-micro config —
both superseded. CONFIG ONLY; the GGUFs stay on disk. Frees granite-4-small's
~24 GB (it was pinned at 120K ctx).

Placement: phi4 on GPU 1 with the embed/rerank/reward trio (~1.2 GB margin at
50K); keeps GPU 0 clear for llama-swap heavy models. docs/roadmap.md captures
the deferred vLLM observability (Langfuse req/resp tracing + Prometheus/Grafana).

Deploy order: llama-swap config (free granite) -> vllm-phi4 -> repoint nevermore.
2026-06-03 23:21:45 -07:00
vh d4f180d57f docs(runbook): yt-voice-clipper push-to-deploy (gitea webhook → irv-ml1)
Auto-deploy on push to main: gitea webhook → HMAC-validated listener on irv-ml1:9008
→ git fetch/reset + docker compose up -d --build. Documents the gitea-server
ALLOWED_HOST_LIST anti-SSRF relaxation (scoped to the WG net), the irv-ml1 components
(deploy key, git-clone deploy dir preserving the proxy override/secrets, listener +
user service), and verify/debug steps.
2026-06-03 01:25:20 -07:00
vh f27ee47fac feat(nh3-egress-proxy): durable SOCKS5 egress on nh3-dev for datacenter-IP-gated colo services
YouTube (and a growing set of services) hard-flag datacenter IPs, bot-gating
even public content regardless of cookies/PO-tokens. Origin case: yt-voice-clipper
on irv-ml1 (Irvine colo) — every yt-dlp fetch returned LOGIN_REQUIRED. Confirmed
pure IP reputation: the same public video fetches cleanly (no cookies) once routed
through nh3-dev's residential egress (70.230.226.88).

- scripts/setup-nh3-egress-proxy.sh: idempotent dante (SOCKS5) install + config.
  Internal-only ACL (10.100.0.0/16), bound to the WG interface, systemd-managed.
- docs/runbooks/nh3-egress-proxy.md: purpose, usage, security model, caveats.

Reusable fleet egress, not yt-voice-clipper-specific.
2026-06-02 22:37:08 -07:00
vh 099e1d7418 catalog: promote chatterbox-fast to ready; vibevoice->down; resolve voxtral 8197 collision
- chatterbox-fast experimental -> ready: browser audition verified end-to-end
  (operator confirmed progressive playback "excellent" 2026-06-02).
- vibevoice ready -> down: no container running on irv-ml1 (connection refused);
  catalog status was stale.
- voxtral: NOT a stale typo — its stack genuinely claimed :8197, the port now held
  by the live chatterbox-fast. voxtral is down, so moved IT to :8201 (catalog
  endpoint + source_url, stacks/voxtral/.env.example + README, host .env) rather
  than disturb the live service. No live clash existed (voxtral down) but it was a
  latent deploy-time collision I introduced by placing chatterbox-fast on 8197.

No catalog_version bump (status changes + endpoint correction, additive). Validates
against the schema.
2026-06-02 09:10:57 -07:00
vh e4b786d242 catalog(chatterbox-fast): add streaming TTS service entry (streamable:true)
Land chatterbox-fast in the asset-engine catalog as an additive service, per
asset-engine-dev's shipped streaming-audition path (asset-engine v0.1.17-19):
- streamable:true -> UI routes Generate to an ephemeral progressive-<audio>
  audition (no Job/Asset); re-run on `chatterbox` to keep output.
- New service-level `streamable` bool added to services.schema.json (additive,
  default false; mirrors the Pydantic model asset-engine-dev regenerates).
- Fields: text, voice (select via /voices), temperature/top_p/top_k/
  repetition_penalty/seed, format (pcm default; UI forces wav). exaggeration/
  cfg_weight omitted — Turbo ignores them.
- status experimental until the first real browser audition verifies progressive
  playback (the one thing asset-engine-dev couldn't machine-verify).
- reproducibility + audit entries added. No catalog_version bump (additive).
Validates against the updated schema.
2026-06-02 00:58:57 -07:00
vh 7cd39001b2 feat(chatterbox-fast): Phase 1 streaming server — adaptive-chunk scheduler
Build the streaming TTS server MVP per docs/design/chatterbox-fast-plan.md §4.

- scheduler.py: adaptive buffer-ratchet chunker (the meat) — GPU-free pure
  logic. First sentence emitted alone for low TTFA, then chunks ratchet ~3x by
  packing whole sentences to margin x buffered-audio; drives off measured RTF +
  sec/char (EMA). relieve_leader() clause-splits a too-big mid-stream sentence
  to avoid starvation (joins land on commas); a long comma-less sentence is the
  one honored-but-flagged limitation.
- test_scheduler.py: GPU-free simulation, 13 tests — asserts no-starvation
  (incl. overestimated RTF) and the ratchet.
- app.py: FastAPI model holder + POST /tts StreamingResponse (raw PCM s16le
  default, wav optional, stream/oneshot) + GET /health.
- bench.py: client — ground-truth TTFB + real 1x-consumer starvation check.

Live test on irv-ml1 (turbo, A6000, GLaDOS voice): streaming TTFB 499ms vs
oneshot 5230ms (~10x), stayed ahead of a 1x player (no starvation), ratchet
1.64->4.08->8.60->8.60s audio, measured RTF self-corrected 3.38->4.01.

Kill the superseded docs/design/chatterbox-fast.md — its §5 windowed-token
streaming was the abandoned native-frame-streaming arc; the adaptive-chunk plan
supersedes it. Repoint persistent-memory + README at the canonical plan.
2026-06-01 22:42:28 -07:00
vh 4b9bd109bf docs(chatterbox-fast): add executable plan-of-attack (durable, survives reboot vs /tmp)
Self-contained build plan for the chatterbox-fast streaming engine: the
adaptive buffer-ratchet chunking design, validated turbo API + facts, the
GPU-1 dev/test container pattern, 4 build phases, the base-fork A/B, and
watch-outs (incl. native-turbo-streaming is abandoned). Intended for a
fresh-context session to execute at full strength.
2026-06-01 22:23:17 -07:00
vh 19308ff5aa memory: snapshot — chatterbox-fast streaming build (adaptive buffer-ratchet chunking); native-turbo-streaming abandoned; fish reference_id fix + glados transcript + NFS/smithy + references-togglable; add design doc 2026-06-01 22:21:17 -07:00
vh c98a12baf4 catalog(fish-s2): opt references into togglable; catalog_version 1->2
asset-engine shipped the per-field enable-toggle (v0.1.14/.15) — the
durable fix for the "form submits untouched fields" family. A field
marked togglable:true renders with an OFF-by-default switch: while off
the control is disabled (excluded from submission) AND the server skips
injecting its default, so it is genuinely not sent until the user opts
in.

Per operator direction, opt fish-s2's `references` (inline-base64
Custom-clone) field in — it already satisfies the togglable-requires-
optional validator (optional:true, no default). The advanced clone
field now renders dormant and can never silently override the Voice
dropdown again.

This is a SCHEMA change (new CatalogField property), so:
- services.schema.json: add `togglable` (boolean, default false),
  mirroring the asset_engine Pydantic model that generates this schema.
- catalog_version 1 -> 2 (header: bump on schema changes).
- CATALOG-CONTRACT.md: consumer pin note -> catalog_version=2.

Scoped to `references` only. The chatterbox/dia2 clone fields are the
same family but NOT toggled: dia2 deliberately defaults voice_mode=clone
+ a clone ref as its stable out-of-box voice, and toggling that field
would change dia2's default-voice behavior (the earlier 404 fix).

Validated: jsonschema accepts togglable; additionalProperties:false
guard still rejects unknown props.
2026-06-01 18:05:56 -07:00
vh c5bbb90980 fix(fish-s2): reference_id was a silent no-op — populate per-voice dirs + guard the regression
reference_id=<name> resolves against the DIRECTORY references/<name>/
(audio + same-basename .lab), not a flat references/<name>.wav. Voices
were staged flat with the per-name dirs left empty, so every
reference_id resolved to nothing and Fish fell back to its default
speaker — every dropdown voice produced byte-identical audio (proven:
Abigail == Imogen == no-ref, same text+seed). This was the real "no
accent" root cause, independent of the asset-engine "undefined" select
bug.

Server fix (applied to irv-ml1): populated references/<name>/<name>.wav
+ <name>.lab for all 32 voices; re-test confirms Imogen/Eleanor/
Beatrice/Abigail/no-ref now all distinct.

Durable hardening + record correction:
- playbook: normalize-layout step (flat <name>.wav -> nested dir, cp -u
  idempotent, when-gated on count mismatch) + an A/B verify gate that
  hard-fails the deploy if two reference_ids yield identical output.
- services.yaml: correct the reference_id resolution doc (dir + .lab,
  not flat wav).
- README + persistent-memory: correct the "reference_id-by-name is THE
  working path, verified" claim — it was a no-op until this fix; the
  prior ECAPA 0.79 result came through the inline base64 path.
2026-06-01 16:42:30 -07:00
vh 38d9e3b7aa catalog(fish-s2): drop Imogen_Contralto (pitch-shift variants abandoned)
Pitch-shifted deepening (rubberband, formant on/off) sounded bad at every
depth (tuba / over-gravelly), so abandoned. Removed Imogen_Contralto from the
dropdown + deleted the staged variants (fish + chatterbox). Plain unmodified
Imogen remains. version 5->6.
2026-06-01 14:40:09 -07:00
vh 43e9616dd1 catalog(fish-s2): add Imogen_Contralto to voice dropdown
Imogen pitch-shifted -3 semitones (rubberband, formants follow) -> deeper
contralto register. Staged + verified cloning; added to reference_id options
(33 voices). version 4->5.
2026-06-01 13:34:29 -07:00
vh f52558dc0d catalog(fish-s2): add 3 British-female VCTK voices (Imogen/Eleanor/Beatrice)
Staged consenting VCTK Southern-England female speakers (p225/p228/p229, CC BY
4.0) as subtle-British-accent clone voices — repurposed from the on-host kyutai
tts-voices cache. Named neutrally; NOT modeled on or representing any public
figure. Added to the reference_id dropdown (32 voices total). version 3->4.
2026-06-01 13:24:50 -07:00
vh 284ec5b4c4 catalog(fish-s2): voice dropdown via reference_id (29 staged voices)
Staged 28 single-speaker dia voices + glados into /worktank/fish-s2/references/
(internal research use). Discovered the path-form references shape 500s on this
build; reference_id (by name) is the working voice path (verified live). So:
reference_id -> select 'Voice' with the 29 staged names (default Emily, female);
references demoted to advanced inline-base64 custom clones with the path->500
caveat documented. vram 10->24 (measured ~25GB in use); version 2->3.
2026-06-01 13:15:15 -07:00
vh f3c260e010 catalog(fish-s2): expose full control surface (was text + references only)
Fish-S2 rendered as an essentially blank form — only text + a references JSON
blob — despite being the fleet's richest-control engine. Expose the real
fish-speech ServeTTSRequest levers: temperature/top_p/repetition_penalty,
latency (normal|balanced), seed, format (wav/mp3/opus), + advanced cloning
(references/reference_id) and chunking (max_new_tokens/chunk_length/normalize/
use_memory_cache). Defaults from upstream schema.py, verified live 2026-06-01
(no /openapi.json; Kui server). Sections basic/sampling/advanced.

Also corrected: seedable false->true (/tts has seed); cold_start 8s->240s
(measured compile warmup); vram 4->10GB (~9GB BF16 weights); dropped the
misleading '~150ms TTFB' for honest ~realtime throughput. version 1->2.
2026-06-01 01:41:19 -07:00
vh 93cedd6951 catalog(chatterbox): route to /tts, expose emotion levers + sane defaults
Chatterbox was producing poor output because the catalog pointed at the thin
OpenAI /v1/audio/speech endpoint, which exposes none of Resemble's emotion/
pacing knobs — and the devnen server's shipped default exaggeration is 1.3
(tuned for its theatrical demo presets), which over-acts.

Re-point to the wrapper's richer /tts and expose the real control surface
(exaggeration, cfg_weight, temperature, speed_factor, seed, voice_mode),
mirroring the sibling dia stack (same devnen author). Defaults sourced live
2026-06-01: exaggeration + cfg_weight = 0.5 (Resemble README 'works well for
most prompts'), temperature 0.8 / speed 1.0 / seed 0 (server generation_
defaults). The shipped 1.3 exaggeration is deliberately NOT adopted.

Voices: expose the 28 built-in predefined voices via /get_predefined_voices
(default Emily.wav, the server default_voice_id) + clone via /get_reference_
files — replacing the wrong 'OpenAI aliases only' claim. Corrected seedable:
false -> true (/tts has seed) and image_tag_mutable -> true (:latest). Bumped
service version 1 -> 2 (breaking field-shape change); status down -> ready
(live + healthy). catalog_version unchanged (no new field types).
2026-06-01 01:18:59 -07:00
vh c54ab13d3c catalog: remove Sesame CSM stack — open csm-1b verified not deploy-worthy
Tear down the parked CSM stack (status: down, never successfully built).
Bring-up attempts failed at the image build: upstream
phildougherty/sesame_csm_openai pins no huggingface_hub version, which now
resolves to 1.17.0 where the `huggingface-cli` the Dockerfile relies on has
been removed (replaced by `hf`). Building would require vendoring + patching
the upstream Dockerfile.

Deep-research verdict (primary + community sourced) confirmed it isn't worth
that: the acclaimed Maya/Miles demo runs a fine-tuned, larger CSM variant
Sesame never open-sourced; the open csm-1b is the un-fine-tuned 1B base
(only the smallest of 1B/3B/8B shipped, no newer checkpoint as of mid-2026).
Ships no usable voices, can't generate text, English-only, can't stream
real-time out of the box; absent from current TTS leaderboards and dominated
by Kokoro/Dia2/Fish-S2/IndexTTS for narration.

Removes: stacks/csm/, playbooks/deploy-csm.yaml, the csm catalog service +
reproducibility_audit entries. Host state (compose dir, /worktank/csm) torn
down on irv-ml1; no container/image existed.
2026-06-01 00:58:56 -07:00
vh 5c4784339c catalog(dia2): default to a working stable voice (fix undefined-ref 404)
Reported: dia2-1b 404 'Reference audio file not found: undefined' when
accepting defaults — voice_mode=clone with no clone_reference_filename made
the UI submit the literal 'undefined'. Fix: voice_mode now defaults to clone
AND clone_reference_filename defaults to Abigail.wav (a staged voice), so the
out-of-box request is a valid, stable voice. Reproduced the 404 and verified
the Abigail.wav path returns 200. Folded into v2 (not yet consumed downstream).
2026-05-31 15:27:56 -07:00
vh 55602b7251 catalog(dia2): expose full /tts control surface + stable predefined voices
Repoint both dia2 entries from /v1/audio/speech to the wrapper's richer /tts
endpoint (CustomTTSRequest), exposing the levers that fix the random-voice
problem: voice_mode, clone_reference_filename, cfg_scale, temperature, top_p,
cfg_filter_top_k, speed_factor, seed, split_text, chunk_size, transcript,
max_tokens. All defaults are the wrapper's Pydantic blessed values (cfg 3.0 /
temp 1.3 / top_p 0.95 / top_k 35 / speed_factor 0.94 / chunk 300). Fields
grouped (basic/sampling/advanced). dia2 -> version 2 (field-shape change).

Voice stability: Dia2 samples a random speaker per call unless anchored. The
43 curated voices baked at /app/voices aren't reachable from /tts's clone path
(reference_audio dir only), so they're staged into reference_audio; the
clone_reference_filename picker now sources /get_reference_files. voice_mode=
clone + a reference filename pins voice/gender. Verified /tts clone end-to-end
(HTTP 200, Ogg/Opus 24 kHz). README documents the staging + two-instance shape.
2026-05-31 15:18:53 -07:00
vh e97d80cb8e catalog: re-source today's engine defaults from canonical upstreams
Replace the invented sample inputs / topk with canonically-sourced values
(per CATALOG-CONTRACT source precedence + cite-your-source rule), inline-cited:
  * dia2-2b/dia2-1b input -> '[S1] Hello Dia2!'  (nari-labs/dia2 README example)
  * zonos input -> 'Zonos uses eSpeak for text to phoneme conversion!'
      (upstream gradio_interface.py blessed Textbox default)
  * csm input -> 'Hello from Sesame.'  (sesame/csm-1b model card example)
  * csm topk -> default REMOVED (the card recommends no top-k default; the
    wrapper's internal default applies when omitted).
2026-05-31 14:52:16 -07:00
vh 373a0046b1 catalog: fill sensible field defaults for today's engines
Pre-fill the bare input textareas with format-demonstrating samples
(dia2-2b/dia2-1b: [S1]/[S2] dialogue + a nonverbal; csm: conversational;
zonos: expressive multilingual) and give csm.topk a standard default (50).
Addresses asset-engine-dev's best-practice-defaults request so the picker
forms pre-fill usefully instead of empty. No version bumps — these entries
ship fresh in this batch (no prior consumer-pinned shape).
2026-05-31 14:46:12 -07:00
vh db15638d8e catalog: replace dia with dia2-1b/dia2-2b; move zonos-api 8201->8203
Replace the single dia entry (legacy Dia 1.6B, retired) with two fixed-model
Dia2 entries (dia2-2b :8200, dia2-1b :8202), status ready (both exercised),
image local/dia:v2. Matching reproducibility_audit rows. catalog_version
unchanged (add/remove services = no vocab change).

Also fix a port collision I introduced earlier: the zonos-api adapter and
csm both claimed 8201 — move zonos-api to 8203 (catalog endpoint + voices
source_url, zonos .env.example, README).

NOTE FOR CONSUMERS: removing the dia id is a breaking catalog change for
asset_engine (it vendored dia in v0.1.4) — re-vendor + drop the dia tile,
add the two dia2 tiles.
2026-05-31 14:41:11 -07:00
vh 02b26f45a6 catalog: add zonos service entry; fix dia voice picker options
zonos: new tts entry routing to the REST adapter on 8201, JSON-envelope
response with reproducibility.seed_field=seed (seedable+deterministic),
lifecycle block, section groups. Matching reproducibility_audit row.

dia: voice select had a default (S1) but no options/source_url -> empty
picker; add options [S1, S2, dialogue] (per asset-engine-dev), bump dia
to version 2. Closed select drops clone-by-filename free-text; flagged
inline for a future source_url-backed picker. catalog_version unchanged
(add-service + field-options = no vocab change).
2026-05-31 13:59:10 -07:00
vh b7b2130784 catalog-schema: add CatalogLifecycle + reproducibility.seed_field
Resolves the stale-schema gap asset-engine-dev flagged (the published
schema rejected the lifecycle field 12/14 live services already use) and
adds reproducibility.seed_field so a seedable engine can declare which
response key carries the seed used. Authoritative regen path remains
dump_schema.py against asset_engine/catalog.py; reconcile there.
2026-05-31 13:59:10 -07:00
vh 8eac39b6ce catalog: add dia + csm service entries (status: down) + repro audits
dia (:8200) — OpenAI-compat /v1/audio/speech, seedable (not byte-exact),
Apache-2.0 weights. Clean catalog fit; flip to ready after first exercise.

csm (:8201) — OpenAI-compat, but NO seed + temperature-sampled =
non-reproducible (contract's fix-before-adding case), catalogued by
operator direction with a reproducibility caveat + gated-license warning;
belongs at experimental once running.

Fields read from each wrapper's API docs (2026-05-31), to confirm against
live OpenAPI/Pydantic at deploy. No catalog_version bump (add-service =
no bump). NOTE: services.schema.json is stale (pre-existing — 13 errors;
live catalog uses lifecycle, schema predates it); regen via dump_schema.py.
2026-05-31 11:18:46 -07:00
vh 54d1e7137e docs: archive asset-engine design-brief.md
Moved docs/asset-engine/design-brief.md → docs/archive/asset-engine/design-brief.md
with a 12-line archival header pointing at the live implementation
artifacts (vh/asset-engine source, stacks/asset-engine/ deploy,
CATALOG-CONTRACT.md, services.yaml).

The brief explicitly framed itself as a pre-implementation handoff
("Hand this to a design agent before any pixels"). Implementation
shipped 2026-05-12; the brief's role is past. Kept for the design
rationale it carries (why Asset is first-class, why v1 is synchronous,
v2/v3 seam reasoning) — future contributors benefit from finding it
when wondering "why is it this way."

Surfaced by /tend-docs audit 2026-05-14.
2026-05-14 15:33:15 -07:00
vh 7e7130172e vllm: rename stack from vllm-qwen3 → vllm + add Skywork reward classifier
Two related changes shipped together. The stack rename is independent
but adding `vllm-reward` to the existing `vllm-qwen3` would have made
that name actively misleading.

**Rename:** `stacks/vllm-qwen3/ → stacks/vllm/`. Updated all in-repo
references (README.md root, servers/ana-ml2/, stacks/llama-swap/,
configs/restic/ana-ml2/, docs/runbooks/disaster-recovery.md). Two
intentional history mentions retained (servers/ana-ml2 + stacks/vllm
README).

**Add `vllm-reward` service:** serves Skywork-Reward-V2-Llama-3.1-8B-AWQ
on port 8003. The AWQ output is a locally-quantized model (not from HF),
so bind-mounts `/tank/aimodels/llm:/local-models:ro` rather than the
shared HF cache. Model config.json declares LlamaForSequenceClassification
which vLLM's pooling runner picks up automatically — produces a single
reward score per input via /classify.

**Flag note:** the user's spec listed `--task classify`, but vLLM 0.19.1
deprecated --task in favor of --runner pooling (model architecture in
config.json drives the classification head). Compose uses --runner
pooling with a comment explaining the substitution.

**GPU memory:** no rebalance needed — production had already tuned
EMBED/RERANK down from 0.40 to 0.20 each (canonical .env.example now
matches reality). Adding REWARD at 0.30 totals 0.70, leaving ~14 GB
headroom on the 48 GB Ada.

**Server-side:** brought existing vllm-qwen3 down, mv'd
/opt/docker/compose/vllm-qwen3 → /opt/docker/compose/vllm, appended
REWARD_* lines to existing .env (preserving API_KEY/HF_TOKEN), deployed
new compose via scripts/deploy-stack.sh, brought all 3 services up.

**Smoke tests:**
- /health on 8001/8002/8003 → 200
- /v1/models on 8003 → lists Skywork/Skywork-Reward-V2-Llama-3.1-8B-AWQ
  with max_model_len 16384
- /classify with a sample conversation → returns LABEL_0 with prob 0.9999
  (single-output regression-style reward score, expected shape for a
  reward model)
2026-05-13 22:00:26 -07:00
vh 108ac507ec catalog: voxtral/qwen3-tts/kyutai-tts defaults from live server probes
Booted each service on irv-ml1 to capture the wrapper's actual voice
exposure, then took them back down. Initial state restored.

**Voxtral (v1 → v2)** — 20 native presets from live /v1/audio/voices:
neutral_{female,male}, casual_{female,male}, cheerful_female, plus
8 language-code variants ({ar,de,es,fr,hi,it,nl,pt}_{female,male}).
vLLM-Omni does NOT translate OpenAI aliases; `alloy` would 400. Default
flipped to `neutral_female` (matches Mistral docs).

**Qwen3-TTS (v1 → v2)** — 15 voices from live /v1/voices, NOT the 17
the upstream blog cites: 9 Qwen presets (vivian/serena/uncle_fu/ryan/
aiden/ono_anna/sohee/eric/dylan) + 6 OpenAI-compat aliases (alloy/
echo/fable/nova/onyx/shimmer). Default `vivian` matches the wrapper's
OpenAPI default. Catalog previously stated clone-only — wrong; the
wrapper does ship presets, the upstream blog list just doesn't match
the deployed wrapper. Cloning still works alongside via clone:<name>.

**Kyutai-TTS (v1 → v2)** — NillPointer wrapper does NOT expose any voice
listing endpoint (/v1/audio/voices 404; only /health + POST /v1/audio/
speech are wired). Voices are filesystem-discovered. Catalog now drops
the broken source_url, switches voice field to free-text with default
`unmute-prod-website/default_voice.wav` (upstream's named default).
Description lists the on-disk categories with counts + license posture
(vctk = CC BY 4.0 commercial-safe; expresso = CC BY-NC research-only).
2026-05-12 22:51:33 -07:00
vh 51c77370b3 catalog: defaults audit — Kokoro/SAO/VibeVoice/Fish-s2/ace-step
Six default + description tightenings from upstream-source research:

- **Kokoro voice**: af_bella → af_heart. Per upstream VOICES.md, af_heart
  is the only A-rated voice; also the kokoro-fastapi container's own
  default. Applied to both kokoro (v2 → v3) and kokoro-captioned (v1 → v2).
- **SAO negative_prompt**: "Low quality." → "low quality, average quality".
  Per diffusers official docs Tips section — the lowercase comma-separated
  shape is the explicit recommendation; the period form was a code-example
  string, never a documented default. SAO bumped v1 → v2.
- **SAO prompt description**: added the model card's canonical examples
  ("128 BPM tech house drum loop", "the sound of a hammer hitting a wooden
  surface") plus the Tips advice on descriptive prompts.
- **VibeVoice voice**: en-Carter_man → en-Alice_woman. Per upstream model
  card, Alice is the documented default. Description now flags the
  Alice-injects-BGM-for-intros foot-gun + the `_bgm`-suffix meaning.
  VibeVoice bumped v2 → v3.
- **Fish-s2 text description**: added multi-tag placeholder example
  combining emotion + physical tags, per upstream best-practice docs
  (physical tags "feel flat without emotional context").
- **ace-step prompt description**: added the upstream Gradio UI's
  pre-filled tag string as the canonical prompt-shape example.

Surfaced separately to lkraven (NOT applied here, need decisions):
- Voxtral voice "alloy" may need to become a native preset like
  "neutral_female" — depends on whether vLLM-Omni translates OpenAI
  aliases.
- Qwen3-TTS — research found 17 presets (Cherry, Ethan, ...) that the
  catalog currently says don't exist. Catalog says clone-only; needs
  wrapper-level verification before adding.
- Kyutai-TTS — service down, can't probe /v1/audio/voices. Default
  remains undefaulted.
2026-05-12 22:29:22 -07:00
vh 079c7b15e3 asset-engine + irv-ml1: scaffold orchestrator wiring
Three changes prepping infra for asset_engine's orchestration feature
(SSH-driven bring-up / bring-down of irv-ml1 inference services with
per-device VRAM gating, contract in vh/asset-engine commit 5a36f8c):

1. asset-engine compose + .env.example + playbook gain a read-only
   bind-mount for /app/runtime/ssh — the dedicated ed25519 keypair
   (generated on ana-docker, not in the repo) plus a pinned known_hosts
   for irv-ml1's host fingerprint. Env vars SSH_KEY_PATH and
   SSH_KNOWN_HOSTS are exposed for the app to consume.

2. docs/asset-engine/services.yaml gains a `lifecycle: { stack, vram_gb,
   gpu_device_id }` block on each of 12 orchestratable irv-ml1 services
   (kokoro, chatterbox, index-tts, qwen3-tts, cosyvoice, fish-s2,
   kyutai-tts, vibevoice, voxtral, parakeet, stable-audio-open, ace-step).
   VRAM numbers are estimates from model footprint at fp16 — tune from
   real nvidia-smi measurements once the gate is live. comfyui and
   kokoro-captioned are deliberately excluded (variable-VRAM and
   shared-container respectively).

3. servers/irv-ml1/README.md docker-stacks table now lists all 13
   inference stacks (was only dockge + agents + comfyui) with port +
   GPU pinning columns.

Pubkey deployed to ~lkraven/.ssh/authorized_keys on irv-ml1;
end-to-end SSH from ana-docker → irv-ml1 verified with strict
host-key checking.
2026-05-12 13:24:07 -07:00