Skip to content

hashgraph-online/awesome-codex-plugins / plugins/avivsinai/langfuse-mcp/skills/langfuse

langfuse

Verified

Investigate AI traces, observations, exceptions, latency, sessions, prompts, datasets, annotation queues, and scores through Langfuse MCP. Use when the request names Langfuse or asks to diagnose recorded AI behavior.

FileKindSize
SKILL.mdskill md2.3 KB
references/setup.mdmarkdown5.4 KB
references/tool-reference.mdmarkdown26.2 KB
references/workflows.mdmarkdown7.1 KB

references/workflows.md

View on GitHub
# Langfuse Skill

Debug AI agents and LLM applications through Langfuse observability.

This skill is the agent-facing companion to `langfuse-mcp`. It tells Claude Code and Codex when to use Langfuse, which MCP tool to call first, and how to move from broad trace discovery to a concrete root-cause hypothesis.

**Triggers:** langfuse, traces, debug AI, find exceptions, set up langfuse, what went wrong, why is it slow, datasets, evaluation sets

## What This Skill Provides

- Setup steps for connecting `langfuse-mcp` to Claude Code or Codex.
- Playbooks for exception triage, trace inspection, latency analysis, sessions, prompts, and datasets.
- A quick reference for the highest-value MCP tools.
- Links to full setup and tool references for deeper troubleshooting.

Use the playbooks before guessing at individual tools. Start broad, identify the relevant trace/session/observation, then drill into the exact failure or slow path.

## Setup

**Step 1:** Get credentials from https://cloud.langfuse.com → Settings → API Keys

If self-hosted, use your instance URL for `LANGFUSE_HOST` and create keys there.

**Step 2:** Install MCP (pick one):

Requires Python 3.10 or newer. CI verifies Python 3.10 through 3.14.

```bash
# Claude Code (project-scoped, shared via .mcp.json)
claude mcp add \
  --scope project \
  --env LANGFUSE_PUBLIC_KEY=pk-... \
  --env LANGFUSE_SECRET_KEY=sk-... \
  --env LANGFUSE_HOST=https://cloud.langfuse.com \
  langfuse -- uvx langfuse-mcp

# Codex CLI (user-scoped, stored in ~/.codex/config.toml)
codex mcp add langfuse \
  --env LANGFUSE_PUBLIC_KEY=pk-... \
  --env LANGFUSE_SECRET_KEY=sk-... \
  --env LANGFUSE_HOST=https://cloud.langfuse.com \
  -- uvx langfuse-mcp
```

Add `--python 3.14` before `langfuse-mcp` if you want to pin a CI-verified interpreter explicitly.

**Step 3:** Restart CLI, verify with `/mcp` (Claude) or `codex mcp list` (Codex)

**Step 4:** Test: `fetch_traces(age=60)`

### Read-Only Mode

For safer observability without risk of modifying prompts or datasets, enable read-only mode:

```bash
# CLI flag
langfuse-mcp --read-only

# Or environment variable
LANGFUSE_MCP_READ_ONLY=true
```

This disables write tools: `create_text_prompt`, `create_chat_prompt`, `update_prompt_labels`, `create_dataset`, `create_dataset_item`, `delete_dataset_item`.

### Default Output Mode

If you want MCP clients to default to writing full payloads to files when they omit `output_mode`, configure:

```bash
langfuse-mcp --default-output-mode full_json_file

# Or via environment variable
LANGFUSE_MCP_DEFAULT_OUTPUT_MODE=full_json_file
```

For manual `.mcp.json` setup or troubleshooting, see `setup.md`.

---

## Playbooks

### "Where are the errors?"

```
find_exceptions(age=1440, group_by="file")
```
→ Shows error-level observation counts by recorded file. Pick the worst offender; each group carries a representative observation_id/trace_id.

```
find_exceptions_in_file(filepath="src/ai/chat.py", age=1440)
```
→ Lists specific exceptions. Grab a trace_id.

```
get_exception_details(trace_id="...")
```
→ Recorded error details (exception type/message, status message, filepath/function/line when recorded); stacktrace only when actually recorded. For `unknown` file groups, drill down with the representative trace_id directly — do not pass filepath="unknown".

---

### "What happened in this interaction?"

```
fetch_traces(age=60, user_id="...")
```
→ Find the trace. Note the trace_id.

If you don't know the user_id, start with:
```
fetch_traces(age=60)
```

```
fetch_trace(trace_id="...", include_observations=true)
```
→ See all LLM calls in the trace.

```
fetch_observation(observation_id="...")
```
→ Inspect a specific generation's input/output.

---

### "Why is it slow?"

```
fetch_observations(age=60, type="GENERATION")
```
→ Find recent LLM calls. Look for high latency.

```
fetch_observation(observation_id="...")
```
→ Check token counts, model, timing.

---

### "What's this user experiencing?"

```
get_user_sessions(user_id="...", age=1440)
```
→ List their sessions.

```
get_session_details(session_id="...")
```
→ See all traces in the session.

---

### "Manage datasets"

```
list_datasets()
```
→ See all datasets.

```
get_dataset(name="evaluation-set-v1")
```
→ Get dataset details.

```
list_dataset_items(dataset_name="evaluation-set-v1", page=1, limit=10)
```
→ Browse items in the dataset.

```
create_dataset(name="qa-test-cases", description="QA evaluation set")
```
→ Create a new dataset.

```
create_dataset_item(
  dataset_name="qa-test-cases",
  input={"question": "What is 2+2?"},
  expected_output={"answer": "4"}
)
```
→ Add test cases.

```
create_dataset_item(
  dataset_name="qa-test-cases",
  item_id="item_123",
  input={"question": "What is 3+3?"},
  expected_output={"answer": "6"}
)
```
→ Upsert: updates existing item by id or creates if missing.

---

### "Manage prompts"

```
list_prompts()
```
→ See all prompts with labels.

```
get_prompt(name="...", label="production")
```
→ Fetch current production version.

```
create_text_prompt(name="...", prompt="...", labels=["staging"])
```
→ Create new version in staging.

```
update_prompt_labels(name="...", version=N, labels=["production"])
```
→ Promote to production. (Rollback = re-apply label to older version)

---

## Quick Reference

| Task | Tool |
|------|------|
| List traces | `fetch_traces(age=N)` |
| Get trace details | `fetch_trace(trace_id="...", include_observations=true)` |
| List LLM calls | `fetch_observations(age=N, type="GENERATION")` |
| Get observation | `fetch_observation(observation_id="...")` |
| Error count | `get_error_count(age=N)` (exception_count is null; counts are error-level observations) |
| Find exceptions | `find_exceptions(age=N, group_by="file")` |
| List sessions | `fetch_sessions(age=N)` |
| User sessions | `get_user_sessions(user_id="...", age=N)` |
| List prompts | `list_prompts()` |
| Get prompt | `get_prompt(name="...", label="production")` |
| List datasets | `list_datasets()` |
| Get dataset | `get_dataset(name="...")` |
| List dataset items | `list_dataset_items(dataset_name="...", limit=N)` |
| Create/update dataset item | `create_dataset_item(dataset_name="...", item_id="...")` |

`age` = minutes to look back (max 10080 = 7 days)

---

## Troubleshooting

### MCP connection fails
- Verify credentials: check `LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY`, `LANGFUSE_HOST`
- Restart CLI after adding/updating MCP config
- Test MCP independently: `fetch_traces(age=60)` — if this fails, the issue is MCP, not the skill
- See `setup.md` for detailed troubleshooting

### No traces found
- Increase the `age` parameter (default lookback may be too short)
- Verify your application is sending traces to the correct Langfuse project
- Check `LANGFUSE_HOST` points to the right instance (cloud vs self-hosted)

### Permission denied
- Regenerate API keys from Langfuse dashboard
- Ensure keys have the required scopes for the operation
- Write operations require read-write keys (not read-only mode)

---

## References

- `tool-reference.md` — Full parameter docs, filter semantics, response schemas
- `setup.md` — Manual setup, troubleshooting, advanced configuration
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 469kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 321kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 321kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 321kUpdated Claude CodeCodexVerifiedInstall