vh
9171e6a20f
LLM observability for the fleet — pretty trace UI over the gateway: prompts, completions, reasoning, latency, token counts. The pretty layer LiteLLM's spend_logs lacked. - stacks/langfuse: v3 self-host stack (web/worker/postgres/clickhouse/redis/ minio) on ana-docker, adapted from upstream. UI on :3001 (gitea owns :3000). Project + API keys auto-provisioned via LANGFUSE_INIT_*. HOSTNAME=0.0.0.0 on langfuse-web so it's reachable via the published port while also on tnet. - litellm: enabled success_callback/failure_callback: ["langfuse"] (the passthrough env was already wired); keys + host go in the litellm .env. Verified: stack healthy, project keys authenticate, and a real gateway call landed a litellm-acompletion trace in Langfuse within ~6s. Secrets live only in the server .env (never committed).
Langfuse
LLM trace / observability for the fleet — the "pretty" layer over the LiteLLM
gateway: full prompt + completion, reasoning content, latency, token counts
(and tok/s), per-call trace explorer. Fronts LiteLLM via its native langfuse
success/failure callback.
Server: ana-docker (10.250.50.70) — 6 containers (langfuse-web, -worker,
postgres, clickhouse, redis, minio). v3, adapted from the official self-host
compose.
UI: http://10.250.50.70:3001 (host 3000 is taken by gitea → LANGFUSE_PORT=3001)
Login: lkraven@lkraven.com — password is in the server .env
(LANGFUSE_INIT_USER_PASSWORD); auto-provisioned on first boot.
How it's wired
- A project (
gateway/ orgpfi) + its API keys are auto-provisioned via theLANGFUSE_INIT_*vars in the server.env. - Those keys (
pk-lf-…/sk-lf-…) are set in the litellm stack's.envasLANGFUSE_PUBLIC_KEY/LANGFUSE_SECRET_KEY, withLANGFUSE_HOST=http://10.250.50.70:3001. - LiteLLM ships traces because
litellm_settings.success_callback: ["langfuse"]is enabled instacks/litellm/conf/config.yaml. Every consumer of the gateway is traced automatically — no consumer change.
Deploy / operate
scripts/deploy-stack.sh ana-docker langfuse # push compose
# real secrets live ONLY in /opt/docker/compose/langfuse/.env on the box
# (generated with openssl rand; NEVER committed). See .env.example.
ssh ana-docker 'cd /opt/docker/compose/langfuse && docker compose up -d'
# health: curl http://10.250.50.70:3001/api/public/health -> 200
Notes / gotchas
- HOSTNAME=0.0.0.0 is set on langfuse-web — the Next.js standalone server
otherwise binds only the default-network IP and is unreachable via the
published port once the container is also on
tnet. - The 3
LANGFUSE_S3_*_SECRET_ACCESS_KEYmust equalMINIO_ROOT_PASSWORD(MinIO is the S3 backend; access-key-id staysminio). - ClickHouse is the resource driver; ana-docker had ~77 GB of reclaimable docker cruft pruned (2026-06-05) to make room. Watch disk/RAM as trace volume grows.
- Internal services bind 127.0.0.1 only; just langfuse-web:3001 + minio:9090 are host-reachable on the internal network.