Skip to content

a5c-ai/babysitter / library/specializations/ai-agents-conversational/skills/phoenix-arize-setup

phoenix-arize-setup

Verified

Arize Phoenix observability platform setup for LLM debugging and evaluation

Phoenix Arize Setup Skill

Capabilities

  • Set up Phoenix local server
  • Configure tracing instrumentation
  • Design evaluation experiments
  • Implement embedding visualizations
  • Set up retrieval analysis
  • Create custom evaluations with LLM-as-judge

Target Processes

  • llm-observability-monitoring
  • agent-evaluation-framework

Implementation Details

Core Features

  1. Tracing: OpenTelemetry-based LLM traces
  2. Evals: LLM-as-judge evaluations
  3. Embeddings: Visualization and drift detection
  4. Retrieval: RAG quality analysis
  5. Datasets: Experiment management

Instrumentation

  • OpenAI auto-instrumentation
  • LangChain instrumentation
  • LlamaIndex instrumentation
  • Custom span creation

Configuration Options

  • Phoenix server setup
  • Trace sampling
  • Evaluation metrics
  • Embedding models
  • Export settings

Best Practices

  • Comprehensive instrumentation
  • Regular evaluation runs
  • Monitor embedding drift
  • Analyze retrieval quality

Dependencies

  • arize-phoenix
  • openinference-instrumentation-openai
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 196kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 193kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 193kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 193kUpdated Claude CodeCodexVerifiedInstall