Skip to content

a5c-ai/babysitter / library/specializations/ai-agents-conversational/skills/phoenix-arize-setup

phoenix-arize-setup

Verified

Arize Phoenix observability platform setup for LLM debugging and evaluation

FileKindSize
README.mdmarkdown502 B
SKILL.mdskill md1.5 KB
---
name: phoenix-arize-setup
description: Arize Phoenix observability platform setup for LLM debugging and evaluation
allowed-tools:
  - Read
  - Write
  - Edit
  - Bash
  - Glob
  - Grep
graph:
  domains: [domain:software-engineering]
  specializations: [specialization:ai-agents-conversational]
  skillAreas: [skill-area:agent-debugging-logging, skill-area:eval-driven-development]
  roles: [role:ml-engineer, role:backend-engineer]
  workflows: [workflow:ml-model-lifecycle, workflow:feature-development]

---

# Phoenix Arize Setup Skill

## Capabilities

- Set up Phoenix local server
- Configure tracing instrumentation
- Design evaluation experiments
- Implement embedding visualizations
- Set up retrieval analysis
- Create custom evaluations with LLM-as-judge

## Target Processes

- llm-observability-monitoring
- agent-evaluation-framework

## Implementation Details

### Core Features

1. **Tracing**: OpenTelemetry-based LLM traces
2. **Evals**: LLM-as-judge evaluations
3. **Embeddings**: Visualization and drift detection
4. **Retrieval**: RAG quality analysis
5. **Datasets**: Experiment management

### Instrumentation

- OpenAI auto-instrumentation
- LangChain instrumentation
- LlamaIndex instrumentation
- Custom span creation

### Configuration Options

- Phoenix server setup
- Trace sampling
- Evaluation metrics
- Embedding models
- Export settings

### Best Practices

- Comprehensive instrumentation
- Regular evaluation runs
- Monitor embedding drift
- Analyze retrieval quality

### Dependencies

- arize-phoenix
- openinference-instrumentation-openai
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall