a5c-ai/babysitter / library/specializations/data-science-ml/skills/whylabs-monitor
whylabs-monitor
VerifiedWhyLabs integration skill for ML observability, profile logging, and anomaly detection.
whylabs-monitor
Overview
WhyLabs integration skill for ML observability, data profile logging, and anomaly detection in production ML systems.
Capabilities
- Data profile generation (whylogs)
- Profile upload to WhyLabs platform
- Anomaly detection and alerts
- Segment analysis for data subsets
- Performance monitoring dashboards
- Integration with ML pipelines
- Historical profile comparison
- Custom constraint validation
Target Processes
- Model Performance Monitoring and Drift Detection
- ML System Observability and Incident Response
Tools and Libraries
- whylogs
- WhyLabs Platform
- pandas
Input Schema
{
"type": "object",
"required": ["action"],
"properties": {
"action": {
"type": "string",
"enum": ["profile", "upload", "compare", "validate", "alert-config"],
"description": "WhyLabs action to perform"
},
"profileConfig": {
"type": "object",
"properties": {
"dataPath": { "type": "string" },
"datasetId": { "type": "string" },
"segments": { "type": "array", "items": { "type": "string" } },
"timestamp": { "type": "string" }
}
},
"uploadConfig": {
"type": "object",
"properties": {
"orgId": { "type": "string" },
"modelId": { "type": "string" },
"profilePath": { "type": "string" }
}
},
"compareConfig": {
"type": "object",
"properties": {
"baselineProfile": { "type": "string" },
"targetProfile": { "type": "string" },
"metrics": { "type": "array", "items": { "type": "string" } }
}
},
"validationConfig": {
"type": "object",
"properties": {
"constraints": {
"type": "array",
"items": {
"type": "object",
"properties": {
"column": { "type": "string" },
"constraint": { "type": "string" },
"value": { "type": "number" }
}
}
}
}
}
}
}
Output Schema
{
"type": "object",
"required": ["status", "action"],
"properties": {
"status": {
"type": "string",
"enum": ["success", "error", "warning"]
},
"action": {
"type": "string"
},
"profilePath": {
"type": "string"
},
"uploadId": {
"type": "string"
},
"dashboardUrl": {
"type": "string"
},
"comparison": {
"type": "object",
"properties": {
"driftScore": { "type": "number" },
"driftedFeatures": { "type": "array", "items": { "type": "string" } },
"alerts": { "type": "array" }
}
},
"validation": {
"type": "object",
"properties": {
"passed": { "type": "boolean" },
"failures": { "type": "array" }
}
}
}
}
Usage Example
{
kind: 'skill',
title: 'Profile and upload production data',
skill: {
name: 'whylabs-monitor',
context: {
action: 'profile',
profileConfig: {
dataPath: 'data/production_batch.parquet',
datasetId: 'fraud-detection',
segments: ['region', 'customer_type'],
timestamp: '2024-01-15T00:00:00Z'
},
uploadConfig: {
orgId: 'org-123',
modelId: 'model-fraud-v2'
}
}
}
}
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |