From fd5717c7285b5bab55da1cb5b10c829a09a48e7c Mon Sep 17 00:00:00 2001
From: Vuong Hoang <vh@phasefinal.com>
Date: Mon, 27 Apr 2026 23:16:32 -0700
Subject: [PATCH] playbooks/deploy-fish-s2: pre-pull fishaudio/s2-pro
 checkpoint before container start
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Third deploy attempt got past the build but crashlooped at container
start: Fish's start_server.sh validates checkpoints/s2-pro/ exists
and exits cleanly (rc=0) if missing — no auto-download, no helpful
message. /worktank/fish-s2/checkpoints/ was empty, so the container
exited every ~52s under restart policy.

Added an idempotent pre-pull step using the same one-shot
python:3.12-slim + huggingface_hub.snapshot_download + hf_transfer
pattern we used for the Qwen 3.6 GGUFs earlier today. Pulls the 9
relevant files (~11 GB total: codec.pth + 2 safetensors shards +
config + tokenizer/template) directly into the bind-mount at
/worktank/fish-s2/checkpoints/s2-pro/ — gated by `creates:` on
codec.pth so the pre-pull step is a no-op on reruns.

~83 s wall-clock for the 11 GB pull on first deploy.
---
 playbooks/deploy-fish-s2.yaml | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/playbooks/deploy-fish-s2.yaml b/playbooks/deploy-fish-s2.yaml
index 267a84c..a6f13f2 100644
--- a/playbooks/deploy-fish-s2.yaml
+++ b/playbooks/deploy-fish-s2.yaml
@@ -63,6 +63,21 @@ steps:
       mode: "0644"
     when: "[ ! -f {{ compose_dir }}/.env ]"
 
+  # ── pre-pull model checkpoint ───────────────────────────────────────
+  # Fish doesn't auto-download on first run — start_server.sh validates
+  # checkpoints/s2-pro/ exists and exits cleanly (rc=0) if missing. So
+  # we pre-pull fishaudio/s2-pro into the bind-mount via a one-shot
+  # python container with hf_transfer (~9 GB at ~100 MB/s).
+  # Idempotent — `creates:` skips if the codec file is already there.
+
+  - name: Pre-pull fishaudio/s2-pro into checkpoints (~9 GB, ~90s)
+    shell: |
+      docker run --rm --user 1000:1000 \
+        -e HOME=/tmp/h -e HF_HUB_ENABLE_HF_TRANSFER=1 \
+        -v {{ checkpoints_dir }}:/dest \
+        python:3.12-slim sh -c 'set -e; mkdir -p /tmp/h /tmp/pip /tmp/site; PIP_CACHE_DIR=/tmp/pip pip install --quiet --target /tmp/site huggingface_hub hf_transfer; PYTHONPATH=/tmp/site python -c "from huggingface_hub import snapshot_download; snapshot_download(repo_id=\"fishaudio/s2-pro\", local_dir=\"/dest/s2-pro\", local_dir_use_symlinks=False, ignore_patterns=[\"*.md\",\"*.png\",\".gitattributes\"])"'
+    creates: "{{ checkpoints_dir }}/s2-pro/codec.pth"
+
   # ── build + bring up ────────────────────────────────────────────────
 
   - name: docker compose build (~10-15 min first time; cached after)