Skip to content

datarobot-oss/datarobot-agent-skills / skills/datarobot-gap-analysis

datarobot-gap-analysis

Flagged

Use when the user wants to assess whether an AI agent repository (DataRobot-built or not) is enterprise-ready, run a gap analysis / readiness scan against security, governance, reliability, or regulatory standards, score an agent against EU AI Act requirements, or find and fix gaps (secrets, unpinned models, missing CI, missing guardrails, over-permissioned identity, etc.) before deploying an agent to production. Works on any repository, not only ones built with DataRobot.

FileKindSize
SKILL.mdskill md17.2 KB
references/pillars-and-layers.mdmarkdown7.8 KB
references/policy-authoring.mdmarkdown6.3 KB
references/remediation-paths.mdmarkdown3.0 KB
scripts/gap_analysis/__init__.pyscript534 B
scripts/gap_analysis/_bootstrap.pyscript2.7 KB
scripts/gap_analysis/agent_prompt.pyscript2.5 KB
scripts/gap_analysis/cli.pyscript19.9 KB
scripts/gap_analysis/conformance.pyscript9.6 KB
scripts/gap_analysis/detect.pyscript24.4 KB
scripts/gap_analysis/docs.pyscript2.9 KB
scripts/gap_analysis/engine.pyscript9.3 KB
scripts/gap_analysis/ingest.pyscript3.4 KB
scripts/gap_analysis/inventory.pyscript28.0 KB
scripts/gap_analysis/llm.pyscript2.6 KB
scripts/gap_analysis/migrate.pyscript11.7 KB
scripts/gap_analysis/models.pyscript5.9 KB
scripts/gap_analysis/opencode.pyscript2.7 KB
scripts/gap_analysis/paths.pyscript1.1 KB
scripts/gap_analysis/policy.pyscript1.3 KB
scripts/gap_analysis/posture.pyscript9.6 KB
scripts/gap_analysis/remediate.pyscript20.1 KB
scripts/gap_analysis/report.pyscript20.8 KB
scripts/gap_analysis/report_html.pyscript41.0 KB
scripts/gap_analysis/risk_management.pyscript36.3 KB
scripts/gap_analysis/scanners.pyscript52.0 KB
scripts/gap_analysis/settings.pyscript2.2 KB
scripts/gap_analysis/taxonomy.pyscript5.3 KB
scripts/gap_analysis/variants.pyscript1.5 KB
scripts/policy/defaults.yamlconfig4.6 KB
scripts/prompts/_contract.mdmarkdown3.0 KB
scripts/prompts/_deployment_datarobot_app.mdmarkdown1.2 KB
scripts/prompts/_fix_contract.mdmarkdown1.3 KB
scripts/prompts/_migrate_contract.mdmarkdown1.6 KB
scripts/prompts/_verify.mdmarkdown2.2 KB
scripts/prompts/aig-001-guardrails.mdmarkdown475 B
scripts/prompts/aig-002-model-pinning.mdmarkdown446 B
scripts/prompts/aig-004-evals.mdmarkdown357 B
scripts/prompts/aig-005-human-in-loop.mdmarkdown370 B
scripts/prompts/aig-006-cost-controls.mdmarkdown368 B
scripts/prompts/aig-007-grounding.mdmarkdown406 B
scripts/prompts/aig-008-prompt-versioning.mdmarkdown363 B
scripts/prompts/fix-aig-001-guardrails.mdmarkdown441 B
scripts/prompts/fix-aig-002-model-pinning.mdmarkdown527 B
scripts/prompts/fix-aig-003-approved-model.mdmarkdown351 B
scripts/prompts/fix-aig-006-cost-controls.mdmarkdown354 B
scripts/prompts/fix-idn-003-narrow-scope.mdmarkdown415 B
scripts/prompts/fix-ita-005-base-image.mdmarkdown305 B
scripts/prompts/fix-ops-001-logging.mdmarkdown298 B
scripts/prompts/fix-ops-002-tracing.mdmarkdown375 B
scripts/prompts/fix-pol-pulumi.mdmarkdown3.1 KB
scripts/prompts/fix-rel-003-resilience.mdmarkdown361 B
scripts/prompts/fix-sec-004-redact-log.mdmarkdown418 B
scripts/prompts/fix-sec-011-injection.mdmarkdown598 B
scripts/prompts/fix-sec-012-validation.mdmarkdown374 B
scripts/prompts/fix-sec-013-encryption.mdmarkdown370 B
scripts/prompts/idn-001-human-identity.mdmarkdown548 B
scripts/prompts/idn-002-rotation.mdmarkdown410 B
scripts/prompts/idn-003-over-permissioned.mdmarkdown711 B
scripts/prompts/idn-004-rbac.mdmarkdown433 B
scripts/prompts/migrate-extract-spec.mdmarkdown1.9 KB
scripts/prompts/ops-001-logging.mdmarkdown396 B
scripts/prompts/ops-002-tracing.mdmarkdown351 B
scripts/prompts/ops-003-health.mdmarkdown363 B
scripts/prompts/rel-003-resilience.mdmarkdown401 B
scripts/prompts/rel-004-fallback.mdmarkdown410 B
scripts/prompts/risk-management-mitigation.mdmarkdown2.7 KB
scripts/prompts/sec-001-secret-in-context.mdmarkdown701 B
scripts/prompts/sec-011-injection.mdmarkdown711 B
scripts/prompts/sec-012-input-validation.mdmarkdown999 B
scripts/prompts/sec-013-encryption.mdmarkdown410 B
scripts/risk_management_mitigations.yamlconfig32.0 KB
scripts/run_gap_analysis.pyexecutablescript1.5 KB
scripts/taxonomy.yamlconfig28.8 KB
scripts/validate_risk_management_mapping.pyexecutablescript2.0 KB

scripts/gap_analysis/taxonomy.py

View on GitHub
# Copyright (c) 2026 DataRobot, Inc. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

"""Load and index the condition registry (taxonomy.yaml)."""

from __future__ import annotations

from dataclasses import dataclass, field
from pathlib import Path
from typing import Any

import yaml

from . import paths
from .models import Severity

# Blast-radius classification for the remediation phase.
#   plumbing       — surgical, deterministic; touches config/manifests/scaffolding only.
#   business_logic — edits application code; can alter behavior the author owns.
#   none           — advisory finding, no fix applied.
# Derived from fix_type by default (auto -> plumbing, assisted -> business_logic,
# advisory -> none); a condition may override via the `fix_risk:` key in taxonomy.yaml
# (e.g. assisted fixes that only swap a model id or base image are really plumbing).
_FIX_RISK_VALUES = {"plumbing", "business_logic", "none"}


def _derive_structural(declared: bool, fix_type: str, severity: Severity) -> bool:
    """Whether a condition flags an architectural deficiency that cannot be
    surgically patched: declared as such, or advisory at high/critical
    severity, where by definition no safe in-place fix exists."""
    return declared or (
        fix_type == "advisory" and severity in (Severity.CRITICAL, Severity.HIGH)
    )


def _derive_fix_risk(fix_type: str) -> str:
    if fix_type == "auto":
        return "plumbing"
    if fix_type == "assisted":
        return "business_logic"
    return "none"


@dataclass
class Condition:
    id: str
    pillar: str
    layer: int
    severity: Severity
    title: str
    description: str
    files_glob: list[str]
    relational: bool
    detector: str
    remediation: str
    fix_type: str
    fix_strategy: str | None
    fix_risk: str = "none"
    structural: bool = False
    # Kept so a severity override can re-derive `structural` without losing it.
    structural_declared: bool = False
    # "repo": the question is answered once for the whole codebase (is there
    # tracing anywhere?), so multiple hits collapse into one finding listing
    # its locations. "file": every hit is its own finding (an injection sink).
    scope: str = "file"
    # Runtime-behaviour checks skip IaC, migrations and CI files: a Pulumi
    # program never has OTel spans and should not be told it lacks them.
    runtime_only: bool = False
    # Regexes grepped across the whole repo; hits are shown to the model as
    # evidence its file selection may have missed (a health endpoint registered
    # one call away from main.py, a JSON log formatter in a telemetry module).
    hint_patterns: list[str] = field(default_factory=list)
    # Extra runtime context to attach to the prompt, e.g. "llm_gateway_catalog".
    context: str = ""

    @classmethod
    def from_dict(cls, d: dict[str, Any]) -> Condition:
        fix_type = d.get("fix_type", "advisory")
        fix_risk = d.get("fix_risk") or _derive_fix_risk(fix_type)
        if fix_risk not in _FIX_RISK_VALUES:
            fix_risk = _derive_fix_risk(fix_type)
        severity = Severity(d.get("severity", "medium"))
        declared = bool(d.get("structural", False))
        structural = _derive_structural(declared, fix_type, severity)
        return cls(
            id=d["id"],
            pillar=d["pillar"],
            layer=int(d["layer"]),
            severity=severity,
            title=d["title"],
            description=d.get("description", "").strip(),
            files_glob=list(d.get("files_glob", [])),
            relational=bool(d.get("relational", False)),
            detector=d.get("detector", ""),
            remediation=d.get("remediation", ""),
            fix_type=fix_type,
            fix_strategy=d.get("fix_strategy"),
            fix_risk=fix_risk,
            structural=structural,
            structural_declared=declared,
            scope=str(d.get("scope", "file")),
            runtime_only=bool(d.get("runtime_only", False)),
            hint_patterns=list(d.get("hint_patterns", []) or []),
            context=str(d.get("context", "") or ""),
        )


class Taxonomy:
    def __init__(self, conditions: list[Condition]):
        self.conditions = conditions
        self._by_id = {c.id: c for c in conditions}

    @classmethod
    def load(cls, path: str | Path | None = None) -> Taxonomy:
        p = Path(path) if path else paths.taxonomy_file()
        data = yaml.safe_load(p.read_text())
        return cls([Condition.from_dict(c) for c in data.get("conditions", [])])

    def get(self, condition_id: str) -> Condition | None:
        return self._by_id.get(condition_id)

    def by_layer(self, layer: int) -> list[Condition]:
        return [c for c in self.conditions if c.layer == layer]

    def by_detector(self, detector: str) -> list[Condition]:
        """All conditions whose detector starts with `detector` (e.g. a scanner name)."""
        return [c for c in self.conditions if c.detector.split("#")[0] == detector]

    def apply_severity_overrides(self, overrides: dict[str, str]) -> None:
        for cid, sev in (overrides or {}).items():
            c = self._by_id.get(cid)
            if c:
                c.severity = Severity(sev)
                c.structural = _derive_structural(
                    c.structural_declared, c.fix_type, c.severity
                )
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall