Skip to content

mikeparcewski/wicked-garden / skills/agentic/trust-and-safety

wicked-garden-agentic-trust-and-safety

Unverified

Trust, safety, and control patterns for production agentic systems with human-in-the-loop gates and guardrails. Use when: designing guardrails or human-in-the-loop gates for an agent, or hardening an agentic system against prompt injection.

refs/safety-checklist-advanced.md

View on GitHub
---
phase_relevance: ["design", "review"]
archetype_relevance: ["*"]
---
# Safety Checklist: Advanced

Advanced safety checklist covering monitoring, incident response, testing, and operational safety.

## Monitoring & Observability

### Logging

- [ ] **Comprehensive audit logging**
  - All agent decisions logged
  - All actions logged
  - User interactions logged

- [ ] **Structured logging**
  - JSON/structured format
  - Consistent fields
  - Correlation IDs

- [ ] **Log retention policy**
  - Retention period defined
  - Compliance with regulations
  - Secure log storage

- [ ] **Sensitive data not logged**
  - PII redacted from logs
  - Credentials not logged
  - Secrets not logged

### Monitoring

- [ ] **Real-time safety monitoring**
  - Anomaly detection
  - Unusual action patterns
  - Spike in errors/failures

- [ ] **Alerting configured**
  - Critical safety events trigger alerts
  - Alert routing to right team
  - Alert escalation policies

- [ ] **Dashboard for safety metrics**
  - Safety violations over time
  - Approval rates
  - Error rates by type

### Tracing

- [ ] **Distributed tracing implemented**
  - Request traced across agents
  - Agent-to-agent calls traced
  - External API calls traced

- [ ] **Tracing includes safety context**
  - Approval decisions in trace
  - Validation failures in trace
  - Safety gates in trace

## Incident Response

### Detection

- [ ] **Incident detection mechanisms**
  - Automated detection of safety violations
  - Anomaly detection
  - User reporting mechanism

- [ ] **Incident classification**
  - Severity levels defined
  - Classification criteria
  - Triage process

### Response

- [ ] **Incident response plan documented**
  - Roles and responsibilities
  - Communication plan
  - Escalation procedures

- [ ] **Kill switch mechanism**
  - Emergency stop capability
  - Clear activation criteria
  - Alert on activation

- [ ] **Rollback capability**
  - Can revert problematic changes
  - Rollback tested regularly
  - Data backup for rollback

### Recovery

- [ ] **Post-incident review process**
  - Root cause analysis
  - Lessons learned documented
  - Corrective actions tracked

- [ ] **Incident documentation**
  - Timeline of events
  - Actions taken
  - Impact assessment

## Testing & Validation

### Safety Testing

- [ ] **Adversarial testing performed**
  - Red team testing
  - Prompt injection testing
  - Jailbreak attempt testing

- [ ] **Edge case testing**
  - Invalid inputs
  - Boundary conditions
  - Race conditions

- [ ] **Safety regression testing**
  - Safety tests automated
  - Run on every change
  - No reduction in safety coverage

### Validation

- [ ] **Safety requirements documented**
  - Clear safety specifications
  - Acceptance criteria
  - Success metrics

- [ ] **Independent security review**
  - External security assessment
  - Penetration testing
  - Code review for security

## Operational Safety

### Deployment

- [ ] **Gradual rollout strategy**
  - Canary deployments
  - Percentage-based rollout
  - Automatic rollback on errors

- [ ] **Deployment gates**
  - All tests pass
  - Security scan clear
  - Approval obtained

### Runtime Safety

- [ ] **Circuit breakers implemented**
  - Prevent cascading failures
  - Automatic recovery
  - Alert on circuit open

- [ ] **Rate limiting active**
  - Per-user limits
  - Per-agent limits
  - Per-API limits

- [ ] **Graceful degradation**
  - Reduced functionality when components fail
  - System remains safe in degraded mode
  - User informed of degraded state

### Continuous Improvement

- [ ] **Safety metrics tracked**
  - Violations per day/week
  - False positive rate
  - Response time to incidents

- [ ] **Regular safety reviews**
  - Quarterly safety audits
  - Policy updates
  - New threat assessment

- [ ] **User feedback integration**
  - Safety concerns from users tracked
  - Feedback incorporated into improvements

## Scoring Your System

**Critical (must have):**
- Input validation
- Output validation
- Action whitelisting
- Human approval for high-stakes
- Audit logging
- Kill switch

**Important (should have):**
- PII detection/redaction
- Resource limits
- Circuit breakers
- Rate limiting
- Incident response plan

**Nice to have:**
- Advanced hallucination detection
- ML-based anomaly detection
- Automated adversarial testing

**Minimum viable safety:** All critical items + 80% of important items

**Production ready:** All critical, all important, 50% of nice-to-have

**Best in class:** 100% of all items
brainstormingobra/superpowersYou MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation).Stars: 323kUpdated Claude CodeCodexVerifiedInstall
brand-guidelinesanthropics/skillsApplies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
frontend-designanthropics/skillsGuidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
mcp-builderanthropics/skillsGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK).Stars: 198kUpdated Claude CodeCodexVerifiedInstall