datarobot-oss/datarobot-agent-skills / skills/datarobot-gap-analysis
datarobot-gap-analysis
FlaggedUse when the user wants to assess whether an AI agent repository (DataRobot-built or not) is enterprise-ready, run a gap analysis / readiness scan against security, governance, reliability, or regulatory standards, score an agent against EU AI Act requirements, or find and fix gaps (secrets, unpinned models, missing CI, missing guardrails, over-permissioned identity, etc.) before deploying an agent to production. Works on any repository, not only ones built with DataRobot.
scripts/policy/defaults.yaml
View on GitHub---
# Enterprise-Readiness Gap Analysis — Default Policy
#
# These are opinionated, sensible defaults. An IT admin / team supplies their
# own policy file (via --policy) which is DEEP-MERGED over these defaults:
# - scalars are overridden
# - lists are REPLACED (not concatenated) unless the key ends in `_add`
# - missing keys fall back to these defaults
#
# Semantics:
# libraries.allow -> empty list means "allow all"; non-empty = strict allowlist
# *.deny -> always enforced
# models.allow -> glob patterns ('*' wildcard) of approved model ids
it_admin:
python:
# ITA-001 — minimum acceptable Python version
min_version: "3.11"
libraries:
# ITA-002 — empty allow = allow-all; add entries to enforce a strict allowlist
allow: []
# Packages/usages that are never permitted
deny:
- "telnetlib" # insecure protocol
- "ftplib" # insecure protocol
- "pycrypto" # unmaintained, known CVEs (use 'cryptography')
# Packages that MUST be present
require: []
models:
# AIG-003 / ITA-003: approved LLM model ids (glob patterns). Models served by
# the org's DataRobot LLM Gateway (`dr llm-gateway list`) are approved as well
# when the `dr` CLI is available; the provider families below are the ones the
# gateway and DataRobot application templates use.
allow:
- "anthropic/claude-*"
- "datarobot/*"
- "datarobotdev/*"
- "azure/gpt-*"
- "bedrock/anthropic.claude-*"
- "bedrock/amazon.nova-*"
- "vertex_ai/gemini-*"
- "openai/gpt-5*"
- "openai/gpt-4.1*"
- "google/gemini-2.5*"
licenses:
# ITA-004 — denied OSS licenses (SPDX ids)
deny:
- "GPL-2.0"
- "GPL-3.0"
- "AGPL-3.0"
- "SSPL-1.0"
base_images:
# ITA-005 — approved container base images (glob patterns)
allow:
- "python:3.11*"
- "python:3.12*"
- "python:3.13*"
- "datarobot/*"
- "datarobotdev/*"
- "datarobot_notebook*" # local kernel image built by DataRobot notebook execution environments
regulatory:
# Which regulatory packs (Layer 4) to run. Extensible: add gdpr, soc2, nist_ai_rmf.
# The eu_ai_act pack means "ask DataRobot risk-management what it requires,"
# not "run a fixed set of checks." No DataRobot connection means no findings
# for this pack, not a fallback to a generic checklist.
packs:
- "eu_ai_act"
# Name (or id) of the DataRobot risk-management policy to look up for the
# eu_ai_act pack. Change this if the org's policy isn't named "EU AI Act"
# (e.g. a custom policy under a different name). On a name collision the
# org's own policy wins over the built-in; a policy id is the unambiguous
# form when several tenant policies share a name.
policy_name: "EU AI Act"
# Per-condition severity overrides (key = condition id)
severity_overrides: {}
# Example:
# SEC-004: critical
scan:
# Globs excluded from all scanning
exclude:
- "**/.git/**"
- "**/node_modules/**"
- "**/.venv/**"
- "**/venv/**"
- "**/dist/**"
- "**/build/**"
- "**/__pycache__/**"
- "**/*.min.js"
# Cap on file size (bytes) fed to the LLM per Layer-2 check
max_file_bytes: 200000
posture:
# Remediation-posture decision gate (patch vs re-platform). The score is the share of
# weighted risk that comes from STRUCTURAL gaps (architectural deficiencies that can't be
# surgically patched). See src/gap_analysis/posture.py.
patch_max: 0.25 # score <= this -> PATCH (patch in place)
replatform_min: 0.50 # score >= this -> RE-PLATFORM (migrate to af-components)
replatform_structural_count: 4 # this many high/critical structural gaps forces RE-PLATFORM
max_drivers: 8 # how many structural drivers to list in the report
report:
# Severities that make the CLI exit non-zero (CI gating)
fail_on:
- "critical"
- "high"
remediation:
# Fix tiers the engine applies itself. Only deterministic codemods ("auto") run;
# "assisted" findings ship an agent prompt in the report instead.
allow_fix_types:
- "auto"
# Risk classes that may be applied in a blanket "fix all" (no explicit ids). Business-
# logic-touching fixes are intentionally excluded so they are only applied when a human
# names the condition id — protecting business logic in someone else's agent.
# plumbing — config/manifests/scaffolding only (safe to sweep)
# business_logic — edits application code (require explicit per-finding opt-in)
auto_apply_risk:
- "plumbing"
# Fixes land on a gap-fixes/<timestamp> branch. The engine never pushes or
# opens a pull request; that stays a separate, explicit human step.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |