a5c-ai/babysitter / library/specializations/ai-agents-conversational/skills/langsmith-tracing
langsmith-tracing
VerifiedLangSmith tracing and debugging setup for LLM applications. Configure observability, capture traces, and enable debugging for LangChain/LangGraph agents.
langsmith-tracing
Configure LangSmith observability and tracing for LLM applications built with LangChain and LangGraph frameworks.
Overview
LangSmith is the managed observability suite by LangChain that provides:
- Dashboards and alerting for LLM applications
- Human-in-the-loop evaluation capabilities
- Deep LangChain/LangGraph integration
- Run Tree model for nested traces
- MCP connectivity to Claude, VSCode
Capabilities
Core Tracing Setup
- Initialize LangSmith client and API configuration
- Configure project/workspace settings
- Set up trace collection and sampling
- Enable debug logging for agent execution
Integration Patterns
- LangChain chain tracing with automatic instrumentation
- LangGraph workflow state tracking
- Custom span creation for non-LangChain code
- Parent-child trace relationships
Debugging Features
- Fetch execution traces for analysis
- Query run history and metadata
- Export traces for offline analysis
- Compare runs across different versions
Usage
Environment Setup
# Set required environment variables
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=<your-api-key>
export LANGCHAIN_PROJECT=<project-name>
Python Integration
from langsmith import Client, traceable
from langchain.callbacks.tracers import LangChainTracer
# Initialize client
client = Client()
# Use @traceable decorator for custom functions
@traceable(name="custom_operation")
def my_function(input_data):
# Your logic here
return result
# Initialize tracer for LangChain
tracer = LangChainTracer(project_name="my-project")
# Use with LangChain chains
chain.invoke(input, config={"callbacks": [tracer]})
Trace Retrieval
# Fetch traces from LangSmith
runs = client.list_runs(
project_name="my-project",
start_time=datetime.now() - timedelta(hours=24),
execution_order=1, # Root runs only
error=False, # Successful runs only
)
for run in runs:
print(f"Run ID: {run.id}")
print(f"Latency: {run.latency_p99}")
print(f"Tokens: {run.total_tokens}")
Task Definition
When used in a babysitter process, this skill produces:
const langsmithTracingTask = defineTask({
name: 'langsmith-tracing-setup',
description: 'Configure LangSmith tracing for the application',
inputs: {
projectName: { type: 'string', required: true },
apiKeyEnvVar: { type: 'string', default: 'LANGCHAIN_API_KEY' },
samplingRate: { type: 'number', default: 1.0 },
enableDebug: { type: 'boolean', default: false }
},
outputs: {
configured: { type: 'boolean' },
projectUrl: { type: 'string' },
artifacts: { type: 'array' }
},
async run(inputs, taskCtx) {
return {
kind: 'skill',
title: `Configure LangSmith tracing for ${inputs.projectName}`,
skill: {
name: 'langsmith-tracing',
context: {
projectName: inputs.projectName,
apiKeyEnvVar: inputs.apiKeyEnvVar,
samplingRate: inputs.samplingRate,
enableDebug: inputs.enableDebug,
instructions: [
'Verify LangSmith API credentials are available',
'Create or validate project configuration',
'Set up tracing instrumentation in codebase',
'Configure sampling rate and debug settings',
'Verify traces are being captured correctly'
]
}
},
io: {
inputJsonPath: `tasks/${taskCtx.effectId}/input.json`,
outputJsonPath: `tasks/${taskCtx.effectId}/result.json`
}
};
}
});
Applicable Processes
- llm-observability-monitoring
- agent-evaluation-framework
- react-agent-implementation
- conversation-quality-testing
- regression-testing-agent
External Dependencies
- LangSmith account and API key
- LangChain Python library
- langsmith Python package
References
Related Skills
- SK-OBS-002 langfuse-integration
- SK-OBS-003 phoenix-arize-setup
- SK-OBS-004 opentelemetry-llm
Related Agents
- AG-OPS-004 observability-engineer
- AG-SAF-004 agent-evaluator
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 469k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 321k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 321k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 321k | Updated | Claude CodeCodex | Verified | Install |