esh-pfi-infrastructure/docs/pfi/docker-stack.md

# PFI-ANA Docker Stack

## Overview

PFI-ANA (the Colo) runs Docker services managed through **Dockge**, a compose-aware Docker management UI. All services that require inbound HTTP/HTTPS routing join a shared external Docker network called `traefik-net`, allowing **Traefik** to act as a reverse proxy and handle TLS termination and routing.

## Conventions

### Network

| Network | Docker Name | Purpose |
|---|---|---|
| Traefik network | `traefik-net` | Shared external network. Services join as `tnet` so Traefik can discover them. |

Every compose file that needs to be reachable through Traefik **must** include:

```yaml
networks:
  tnet:
    name: traefik-net
    external: true
```

And the service must list `tnet` under its `networks` key.

### Storage Paths (Host)

| Host Path | Purpose |
|---|---|
| `/opt/docker/compose/<service>/` | Per-service compose files (managed by Dockge) |
| `/opt/docker/conf/<service>/` | Per-service configuration files (bind-mounted into containers) |
| `/tank/` | Large / persistent data storage (e.g., AI models, generated images, voice data) |

### GPU Support

Services requiring GPU access use the **NVIDIA Container Toolkit**:

```yaml
runtime: nvidia
```

or the more explicit device reservation:

```yaml
deploy:
  resources:
    reservations:
      devices:
        - driver: nvidia
          count: all
          capabilities: [gpu]
```

### Homepage Dashboard Labels

Several services include Docker labels for **Homepage** (a dashboard UI). The convention is:

```yaml
labels:
  - homepage.group=<Group Name>
  - homepage.name=<Display Name>
  - homepage.icon=<icon identifier>
  - homepage.description=<brief description>
  - homepage.href=http://<host>:<port>
```

All services reference the VM 102 host IP `10.250.50.70` (PFI-ANA_DOCKER).

### Compose File Location

Dockge expects compose files under `/opt/docker/compose/` on the Docker host. The live compose files are version-controlled in this project under `configs/pfi-ana/docker/compose/`. Configuration files that containers bind-mount live under `configs/pfi-ana/docker/conf/`.

---

## Live Services Index

| Service | Host Port | Container Port | GPU | Homepage Group | Status | Compose File | Config File |
|---|---|---|---|---|---|---|---|
| **Dockge** | 5001 | 5001 | No | PFI-ANA | ✅ Live | `compose/dockge/compose.yaml` | — |
| **llama-swap** | 9292 | 8080 | Yes (CUDA) | — | ✅ Live | `compose/llama-swap/compose.yaml` | `conf/llama-swap/config.yaml` |
| **ComfyUI** | 8188 | 8188 | Yes (all caps) | AI Systems | ✅ Live | `compose/comfyui/compose.yaml` | — |
| **VibeVoice** | 8745 | 8745 | Yes (gpu) | AI Systems | ✅ Live | `compose/vibevoice/compose.yaml` | — |
| **Parakeet STT** | 8300 | 8000 | Yes (gpu) | AI Systems | ✅ Live | `compose/parakeet/compose.yaml` | — |
| **ChromaDB** | 8000 | 8000 | No | AI Systems | ✅ Live | `compose/chromadb/compose.yaml` | `/opt/docker/conf/chromadb/auth_token` |

> All paths relative to `configs/pfi-ana/docker/`.

---

## Service Details

### 1. Dockge — Docker Compose Management UI

- **Image**: `louislam/dockge:latest`
- **Port**: 5001 → 5001
- **Restart policy**: `unless-stopped`
- **Homepage group**: PFI-ANA
- **Compose file**: `compose/dockge/compose.yaml`

**Volumes**:
| Host / Volume | Container | Purpose |
|---|---|---|
| `/var/run/docker.sock` | `/var/run/docker.sock` | Docker socket for managing containers |
| `dockge_data` (named volume) | `/app/data` | Dockge application data |
| `/opt/docker/compose` | `/opt/docker/compose` | Compose stack directory |

**Environment**:
- `DOCKGE_STACKS_DIR=/opt/docker/compose` — tells Dockge where to find/manage compose stacks

**Notes**: Dockge is the management interface for all other compose stacks on this host. It has full Docker daemon access via the socket mount.

---

### 2. llama-swap — Multi-Model LLM Gateway

- **Image**: `ghcr.io/mostlygeek/llama-swap:cuda`
- **Port**: 9292 → 8080
- **Runtime**: `nvidia` (CUDA)
- **Compose file**: `compose/llama-swap/compose.yaml`
- **Config file**: `conf/llama-swap/config.yaml`
- **Interactive**: `stdin_open: true`, `tty: true` (required by llama-swap)

**Volumes**:
| Host Path | Container Path | Purpose |
|---|---|---|
| `/opt/docker/conf/llama-swap/config.yaml` | `/app/config.yaml` | llama-swap configuration (models, groups, params) |
| `/tank/aimodels/llm` | `/models` | LLM model files (GGUF format) |

**Configured Models:** the live list (with quants, context, TTLs,
group memberships) is in
[`stacks/llama-swap/conf/config.yaml`](../../stacks/llama-swap/conf/config.yaml).
Don't mirror the table here — it drifted last time and immediately
went stale. Live HTTP listing:
`curl http://10.250.50.54:9292/v1/models`.

**Model Groups**:

| Group | Swap | Exclusive | Persistent | Members |
|---|---|---|---|---|
| `high-reasoning` | false | false | — | qwen3.5-35-a3b, qwen3.5-35-a3b-code, gemma4-31b-dense |
| `utility` | false | false | ✅ | embeddinggemma-300M, bge-reranker-v2-m3-0.6B |

**Global Settings**:
- `healthCheckTimeout`: 1200 seconds (20 minutes) — long timeout for large models
- `logLevel`: info
- `metricsMaxInMemory`: 1000

---

### 3. ComfyUI — Image Generation UI

- **Image**: `mmartial/comfyui-nvidia-docker:ubuntu24_cuda13.0-latest`
- **Port**: 8188 → 8188
- **Runtime**: `nvidia` with full device reservation (gpu, compute, utility capabilities)
- **Restart policy**: `unless-stopped`
- **Homepage group**: AI Systems
- **Compose file**: `compose/comfyui/compose.yaml`

**Volumes**:
| Host Path | Container Path | Purpose |
|---|---|---|
| `/tank/comfy/run` | `/comfy/mnt` | ComfyUI workspace / output directory |
| `/tank/aimodels/img/comfy` | `/basedir` | Image models and ComfyUI base directory |

**Environment**:
| Variable | Value | Purpose |
|---|---|---|
| `WANTED_UID` | 1001 | Run as user ID 1001 |
| `WANTED_GID` | 1002 | Run as group ID 1002 |
| `BASE_DIRECTORY` | /basedir | ComfyUI base directory path |
| `SECURITY_LEVEL` | weak | Relaxed security (private network) |
| `NVIDIA_VISIBLE_DEVICES` | all | Expose all GPUs |
| `NVIDIA_DRIVER_CAPABILITIES` | all | Enable all GPU capabilities |

**Notes**: Runs with user-mapped permissions (UID 1001 / GID 1002). The `basedir` points to the image model storage on `/tank`.

---

### 4. VibeVoice — Voice/Audio AI Service

- **Image**: `eworkerinc/vibevoice:latest`
- **Container name**: `vibevoice`
- **Port**: 8745 → 8745
- **GPU**: Yes (all devices, gpu capability)
- **Restart policy**: `unless-stopped`
- **Homepage group**: AI Systems
- **Compose file**: `compose/vibevoice/compose.yaml`

**Volumes**:
| Host Path | Container Path | Purpose |
|---|---|---|
| `/tank/vibevoice/hf` | `/root/.cache/huggingface` | HuggingFace model cache |
| `/tank/vibevoice/voices` | `/app/voices` | Voice data / presets |
| `/tank/vibevoice/state` | `/var/lib/eworker` | Application state persistence |

**Environment**:
| Variable | Value | Purpose |
|---|---|---|
| `ENABLE_1_5B` | true | Enable 1.5B parameter voice model |
| `ENABLE_LARGE` | true | Enable large voice model |
| `AUTH_REQUIRED` | true | Require authentication |
| `CORS_ENABLED` | true | Enable CORS headers |
| `ALLOWED_ORIGINS` | * | Allow all origins (development/private network) |

---

### 5. Parakeet STT — Speech-to-Text Service

- **Image**: `parakeet-stt` (locally built)
- **Port**: 8300 → 8000
- **GPU**: Yes (all devices, gpu capability)
- **Restart policy**: `unless-stopped`
- **Homepage group**: AI Systems
- **Compose file**: `compose/parakeet/compose.yaml`
- **Env file**: `.env` (not tracked in project — likely contains API keys or model config)

**Volumes**:
| Volume | Container Path | Purpose |
|---|---|---|
| `parakeet_cache` (named volume) | `/root/.cache` | Model download cache |

**Notes**: Uses a locally-built image (no registry prefix). The `.env` file is referenced but not stored in the project — it likely contains environment-specific configuration on the Docker host.

---

### 6. ChromaDB — Vector Database

- **Image**: `chromadb/chroma:latest`
- **Container name**: `chromadb`
- **Port**: 8000 → 8000
- **GPU**: No
- **Restart policy**: `unless-stopped`
- **Homepage group**: AI Systems
- **Compose file**: `compose/chromadb/compose.yaml`

**Volumes**:
| Host Path | Container Path | Mode | Purpose |
|---|---|---|---|
| `/opt/docker/conf/chromadb` | `/conf` | read-only | Config directory (contains auth_token) |
| `/tank/chromadb` | `/data` | read-write | Persistent vector data |

**Environment**:
| Variable | Value | Purpose |
|---|---|---|
| `CHROMA_SERVER_AUTHN_CREDENTIALS_FILE` | `/conf/auth_token` | Path to auth token file inside container |
| `CHROMA_SERVER_AUTHN_PROVIDER` | `chromadb.server.auth.token.TokenAuthenticationServerProvider` | Enable token-based authentication |
| `IS_PERSISTENT` | TRUE | Enable persistent storage |
| `PERSIST_DIRECTORY` | `/data` | Where vector data is stored inside container |
| `ANONYMIZED_TELEMETRY` | FALSE | Disable telemetry |

**Health Check**:
| Setting | Value |
|---|---|
| Test | `curl -f http://localhost:8000/api/v1/health` |
| Interval | 30s |
| Timeout | 10s |
| Retries | 3 |
| Start period | 40s |

**Authentication**: Token-based. The auth token is stored at `/opt/docker/conf/chromadb/auth_token` on the host (mode 600), generated with `openssl rand -hex 32`. Clients must supply this token to access the API.

**Notes**: CPU-only service (no GPU). Token is generated once at deploy with `openssl rand -hex 32` into `/opt/docker/conf/chromadb/auth_token` (mode 600); clients pass it via the chromadb client `Settings(chroma_client_auth_provider="...TokenAuthClientProvider", chroma_client_auth_credentials="...")`.

---

## Infrastructure Summary

```
┌──────────────────────────────────────────────────────────────┐
│                    PFI-ANA (10.250.50.70)               │
│                   Docker Host (PFI-ANA_DOCKER)          │
│                                                         │
│  ┌──────────┐    Manages all compose stacks              │
│  │  Dockge  │◄─── /opt/docker/compose/*                  │
│  │  :5001   │    /var/run/docker.sock                    │
│  └──────────┘                                           │
│                                                         │
│  ┌──────────────────┐   GPU via passthrough          │
│  │  llama-swap  :9292 (CUDA, multi-model gateway)   │   │
│  │  ├── 15 models (chat, code, embedding, reranker) │   │
│  │  ├── 2 groups (high-reasoning, utility)          │   │
│  │  └── Models from /tank/aimodels/llm              │   │
│  └──────────────────┘   GPU via passthrough          │
│                                                         │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  GPU via      │
│  │  ComfyUI     │  │  VibeVoice   │  │  Parakeet    │  passthrough  │
│  │  :8188       │  │  :8745       │  │  :8300       │              │
│  │  (GPU, img)  │  │  (GPU, voice)│  │  (GPU, STT)  │              │
│  └──────────┘  └──────────┘  └──────────┘              │
│                                                         │
│  ┌──────────┐                                       │
│  │  ChromaDB    │  Token auth, persistent vectors        │
│  │  :8000       │  /tank/chromadb (data)                 │
│  │  (CPU only)  │  /opt/docker/conf/chromadb (config)    │
│  └──────────┘                                       │
│                                                         │
│   ── All services on traefik-net (external) ──         │
│   ── /tank/* = persistent large data storage ──        │
│   ── /opt/docker/* = config + compose files ──        │
└──────────────────────────────────────────────────────────────┘
```

## Port Allocation

| Port | Service | Protocol |
|---|---|---|
| 5001 | Dockge | HTTP |
| 8000 | ChromaDB | HTTP |
| 8188 | ComfyUI | HTTP |
| 8300 | Parakeet STT | HTTP (→ container 8000) |
| 8745 | VibeVoice | HTTP |
| 9292 | llama-swap | HTTP (→ container 8080) |