vh
9171e6a20f
LLM observability for the fleet — pretty trace UI over the gateway: prompts, completions, reasoning, latency, token counts. The pretty layer LiteLLM's spend_logs lacked. - stacks/langfuse: v3 self-host stack (web/worker/postgres/clickhouse/redis/ minio) on ana-docker, adapted from upstream. UI on :3001 (gitea owns :3000). Project + API keys auto-provisioned via LANGFUSE_INIT_*. HOSTNAME=0.0.0.0 on langfuse-web so it's reachable via the published port while also on tnet. - litellm: enabled success_callback/failure_callback: ["langfuse"] (the passthrough env was already wired); keys + host go in the litellm .env. Verified: stack healthy, project keys authenticate, and a real gateway call landed a litellm-acompletion trace in Langfuse within ~6s. Secrets live only in the server .env (never committed).
44 lines
2.2 KiB
Markdown
44 lines
2.2 KiB
Markdown
# Langfuse
|
|
|
|
LLM trace / observability for the fleet — the "pretty" layer over the LiteLLM
|
|
gateway: full prompt + completion, reasoning content, latency, token counts
|
|
(and tok/s), per-call trace explorer. Fronts LiteLLM via its native `langfuse`
|
|
success/failure callback.
|
|
|
|
**Server:** ana-docker (10.250.50.70) — 6 containers (langfuse-web, -worker,
|
|
postgres, clickhouse, redis, minio). v3, adapted from the official self-host
|
|
compose.
|
|
**UI:** http://10.250.50.70:3001 (host 3000 is taken by gitea → `LANGFUSE_PORT=3001`)
|
|
**Login:** `lkraven@lkraven.com` — password is in the server `.env`
|
|
(`LANGFUSE_INIT_USER_PASSWORD`); auto-provisioned on first boot.
|
|
|
|
## How it's wired
|
|
- A project (`gateway` / org `pfi`) + its API keys are auto-provisioned via the
|
|
`LANGFUSE_INIT_*` vars in the server `.env`.
|
|
- Those keys (`pk-lf-…` / `sk-lf-…`) are set in the **litellm** stack's `.env`
|
|
as `LANGFUSE_PUBLIC_KEY` / `LANGFUSE_SECRET_KEY`, with
|
|
`LANGFUSE_HOST=http://10.250.50.70:3001`.
|
|
- LiteLLM ships traces because `litellm_settings.success_callback: ["langfuse"]`
|
|
is enabled in `stacks/litellm/conf/config.yaml`. Every consumer of the gateway
|
|
is traced automatically — no consumer change.
|
|
|
|
## Deploy / operate
|
|
```bash
|
|
scripts/deploy-stack.sh ana-docker langfuse # push compose
|
|
# real secrets live ONLY in /opt/docker/compose/langfuse/.env on the box
|
|
# (generated with openssl rand; NEVER committed). See .env.example.
|
|
ssh ana-docker 'cd /opt/docker/compose/langfuse && docker compose up -d'
|
|
# health: curl http://10.250.50.70:3001/api/public/health -> 200
|
|
```
|
|
|
|
## Notes / gotchas
|
|
- **HOSTNAME=0.0.0.0** is set on langfuse-web — the Next.js standalone server
|
|
otherwise binds only the default-network IP and is unreachable via the
|
|
published port once the container is also on `tnet`.
|
|
- The 3 `LANGFUSE_S3_*_SECRET_ACCESS_KEY` must equal `MINIO_ROOT_PASSWORD`
|
|
(MinIO is the S3 backend; access-key-id stays `minio`).
|
|
- ClickHouse is the resource driver; ana-docker had ~77 GB of reclaimable docker
|
|
cruft pruned (2026-06-05) to make room. Watch disk/RAM as trace volume grows.
|
|
- Internal services bind 127.0.0.1 only; just langfuse-web:3001 + minio:9090 are
|
|
host-reachable on the internal network.
|