agentsope/SkillAlchemy / skills/agentsop-observability-setup
agentsop-observability-setup
VerifiedEnhancement-overlay skill — the DECISION + WIRING layer for LM observability that the single-backend skills [[langsmith]], [[phoenix]], [[mlflow]] do NOT cover. Each of those installs one backend; none of them help you DECIDE which backend fits your stack/scale/budget, nor give you a one-line autolog that turns it on fast. Use when starting any LM project, before the first deploy, or the moment someone asks "why did it do that?" and there are no traces to answer with. The skill picks a backend by stack (LangSmith for LangChain/LangGraph; Phoenix for OSS/local OpenTelemetry; MLflow for ML-shops already on MLflow; Langfuse for self-host), wires one-line autolog, verifies traces land, and adds eval hooks — instrumenting BEFORE you need it. Cross-links the first-debug-move skill [[agentsop-prompt-history-inspect]]. Do NOT activate to re-teach a backend you already chose (defer to its own skill), or for non-LM ML experiment tracking with no LLM calls (that is plain MLflow).
| File | Kind | Size |
|---|---|---|
| README.md | markdown | 2.3 KB |
| SKILL.md | skill md | 17.8 KB |
| intermediate/operation_candidates.json | config | 8.0 KB |
| references/R1-source-evidence.md | markdown | 5.1 KB |
README.md
View on GitHub# observability-setup (E3 — Enhancement overlay)
**One line:** the *decision + one-line-wiring* layer for LM observability. Picks **which** backend
(LangSmith / Phoenix / MLflow / Langfuse / OpenTelemetry) and turns it on fast — the gap the
single-backend skills leave open.
## Why this exists
The local skills [[langsmith]], [[phoenix]], and [[mlflow]] each install and teach **one** backend
deeply. None of them helps a coder **decide which backend** fits their stack/scale/budget, or gives
a single-line autolog to turn it on before the first deploy. All 7 Phase A SOPs reference
observability (langgraph's AP-15 "don't skip LangSmith", llamaindex's "LangSmith / Phoenix",
dspy's MLflow + LangFuse lines) but assume the choice is already made. This overlay fills that gap,
then hands off to the chosen backend's own skill.
## Type
ENHANCE overlay — sits one level above the per-backend skills. Cross-links:
`[[langsmith]]`, `[[phoenix]]`, `[[mlflow]]`, `[[agentsop-prompt-history-inspect]]`.
## When it fires
- Starting any LM project (no tracing wired yet)
- Before the first deploy (the AP-15 trap)
- "Why did it do that?" asked, and there are no traces to answer with
## When it does NOT fire
- Backend already chosen → use that backend's skill directly
- Need to read one prompt right now → [[agentsop-prompt-history-inspect]]
- Pure ML experiment tracking, no LLM calls → plain MLflow
## Core idea
> Instrument before you need it. The cheapest debugging is a trace you already have.
## SOP
pick backend by stack/scale/budget → one-line autolog → verify a trace landed → add eval hooks → hand off.
## Files
| File | Purpose |
|---|---|
| `SKILL.md` | The skill — 7 sections, decision table, 6 ops, 2 dilemma cases |
| `README.md` | This file |
| `references/R1-source-evidence.md` | Traceable citations from the Phase A SOPs + backend frontmatter |
| `intermediate/operation_candidates.json` | Machine-readable op + decision + anti-pattern inventory |
## Relationship to neighbors
- **[[agentsop-prompt-history-inspect]]** = reactive, single-prompt, no setup (first move on a bug).
- **observability-setup** (this) = proactive, persistent, decision + wiring (so the next "why" already has a trace).
- **[[langsmith]] / [[phoenix]] / [[mlflow]]** = the depth layer once a backend is chosen.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |