catalog(fish-s2): expose full control surface (was text + references only)

Fish-S2 rendered as an essentially blank form — only text + a references JSON blob — despite being the fleet's richest-control engine. Expose the real fish-speech ServeTTSRequest levers: temperature/top_p/repetition_penalty, latency (normal|balanced), seed, format (wav/mp3/opus), + advanced cloning (references/reference_id) and chunking (max_new_tokens/chunk_length/normalize/ use_memory_cache). Defaults from upstream schema.py, verified live 2026-06-01 (no /openapi.json; Kui server). Sections basic/sampling/advanced. Also corrected: seedable false->true (/tts has seed); cold_start 8s->240s (measured compile warmup); vram 4->10GB (~9GB BF16 weights); dropped the misleading '~150ms TTFB' for honest ~realtime throughput. version 1->2.
2026-06-01 01:41:19 -07:00
parent 93cedd6951
commit f3c260e010
1 changed files with 120 additions and 16 deletions
@@ -616,13 +616,14 @@ services:
    name: Fish Audio S2-Pro
    description: >
      Fishaudio's richest-paralinguistic English TTS (15,000+ inline tags).
-      Trained 10M+ hours, dual-AR, ~150ms streaming TTFB. Released March 2026.
+      Trained 10M+ hours, dual-AR. Released March 2026. Heavy: ~240s compile
+      warmup on cold start, ~realtime throughput once warm.
    category: tts
-    version: 1
+    version: 2
    host: irv-ml1
    lifecycle:
      stack: fish-s2
-      vram_gb: 4
+      vram_gb: 10
      gpu_device_id: 1
    endpoint: http://10.100.79.3:8195/v1/tts
    method: POST
@@ -631,10 +632,19 @@ services:
      id: fishaudio/s2-pro
      revision: null
      image: local/fish-s2:v1
+    section_groups:
+      - id: basic
+        label: Text & output
+      - id: sampling
+        label: Sampling & latency
+        hint: fish-speech ServeTTSRequest defaults. Lower temperature/top_p for steadier reads; latency=balanced for lower latency.
+      - id: advanced
+        label: Advanced & voice cloning
    fields:
      - name: text
        type: textarea
        label: Text (with optional [tags])
+        section: basic
        required: true
        max_length: 5000
        description: >
@@ -644,29 +654,123 @@ services:
          without emotional context — pair with an emotion tag. Example:
          `[excited] Welcome back! [whisper] I have a secret. [laugh]
          Just kidding.`
+      - name: format
+        type: select
+        section: basic
+        options: [wav, mp3, opus]
+        default: wav
+        description: >
+          Output container (upstream also accepts raw `pcm`, omitted here).
+          wav + mp3 verified live 2026-06-01.
+      - name: temperature
+        type: slider
+        section: sampling
+        min: 0.1
+        max: 1.0
+        step: 0.05
+        default: 0.8
+        description: Sampling temperature; lower = steadier. fish-speech default 0.8.
+      - name: top_p
+        type: slider
+        section: sampling
+        min: 0.1
+        max: 1.0
+        step: 0.05
+        default: 0.8
+        description: Nucleus sampling cutoff. fish-speech default 0.8.
+      - name: repetition_penalty
+        type: slider
+        section: sampling
+        min: 0.9
+        max: 2.0
+        step: 0.05
+        default: 1.1
+        description: >
+          Penalizes repeats; raise if you hear stutters/loops. fish-speech
+          default 1.1.
+      - name: latency
+        type: select
+        section: sampling
+        options: [normal, balanced]
+        default: normal
+        description: >
+          `balanced` trades a little quality for lower latency — useful given
+          S2-Pro's ~realtime throughput. fish-speech default `normal`.
+      - name: seed
+        type: number
+        section: sampling
+        required: false
+        description: Omit for a random seed; a fixed integer repeats a take.
      - name: references
        type: json
-        label: Voice References
+        label: Voice References (clone)
+        section: advanced
        optional: true
        description: >
-          Array of {audio: "/app/references/<file>.wav", text: "transcript"}.
-          Files under /worktank/fish-s2/references/ on host.
+          Array of {audio: "/app/references/<file>.wav", text: "transcript"} for
+          zero-shot cloning. Files under /worktank/fish-s2/references/ on host.
+      - name: reference_id
+        type: text
+        label: Stored reference id
+        section: advanced
+        optional: true
+        description: >
+          Alternative to inline `references`: id of a server-stored reference
+          set (a folder under the server's references/). Leave blank to use
+          `references` or the default voice.
+      - name: max_new_tokens
+        type: number
+        section: advanced
+        required: false
+        default: 1024
+        description: Max audio tokens per chunk. fish-speech default 1024.
+      - name: chunk_length
+        type: slider
+        section: advanced
+        min: 100
+        max: 1000
+        step: 10
+        default: 200
+        description: Target chunk length in chars when splitting. fish-speech default 200.
+      - name: normalize
+        type: bool
+        section: advanced
+        default: true
+        description: Text normalization (numbers, punctuation). fish-speech default on.
+      - name: use_memory_cache
+        type: select
+        section: advanced
+        options: ["off", "on"]
+        default: "off"
+        description: >
+          `on` caches the reference encoding to speed up repeated calls with the
+          same voice. fish-speech default off.
    response:
      type: audio
-      mime: audio/wav
+      mime_from_field: format
    reproducibility:
-      seedable: false
-      deterministic: true
+      seedable: true
+      deterministic: false
+      seed_field: seed
+      notes: >
+        /v1/tts exposes `seed` (omit = random); a fixed seed repeats a take.
+        Temperature/top_p sampled → not guaranteed byte-exact. (Prior catalog
+        claimed no seed support — corrected against the upstream ServeTTSRequest
+        schema + a live generation 2026-06-01.)
    estimated_latency:
-      cold_start_s: 8
-      warm_per_unit: "~150ms TTFB"
+      cold_start_s: 240
+      warm_per_unit: "~realtime throughput on the A6000 (heavy). latency=balanced trades quality for lower latency."
    license: "Research-only (Fishaudio terms; non-commercial)"
    license_warning: |
      Research/internal only. Not clear for commercial use. Same flag as
      Voxtral and SAO — UI must surface when output destined for products.
    notes: |
-      NOT OpenAI-compatible: only /v1/tts. No /v1/audio/voices, no /v1/models.
-      Voice discovery is manual (host filesystem).
+      NOT OpenAI-compatible: only /v1/tts (no /v1/audio/voices, /v1/models, or
+      /openapi.json). Voice discovery is manual (host filesystem). Full control
+      surface (temperature/top_p/repetition_penalty/latency/seed/format + the
+      cloning fields) sourced from fish-speech's ServeTTSRequest schema and
+      verified live 2026-06-01; prior catalog exposed only text + references.
+      HEAVY: ~240s torch.compile warmup measured on cold start; budget the wait.

  - id: kyutai-tts
    name: Kyutai TTS 1.6B EN/FR
@@ -1860,10 +1964,10 @@ reproducibility_audit:
    image_tag_mutable: false
    notes: "instruct field broken for English; XML tags only."
  - service: fish-s2
-    seedable: false
-    model_deterministic: true
+    seedable: true
+    model_deterministic: false
    image_tag_mutable: false
-    notes: "Research-only license — non-commercial."
+    notes: "/v1/tts exposes seed (omit=random); temperature/top_p sampled, not byte-exact. Research-only license — non-commercial."
  - service: kyutai-tts
    seedable: false
    model_deterministic: true