Skip to content

datarobot-oss/datarobot-agent-skills / skills/datarobot-gap-analysis

datarobot-gap-analysis

Flagged

Use when the user wants to assess whether an AI agent repository (DataRobot-built or not) is enterprise-ready, run a gap analysis / readiness scan against security, governance, reliability, or regulatory standards, score an agent against EU AI Act requirements, or find and fix gaps (secrets, unpinned models, missing CI, missing guardrails, over-permissioned identity, etc.) before deploying an agent to production. Works on any repository, not only ones built with DataRobot.

FileKindSize
SKILL.mdskill md17.2 KB
references/pillars-and-layers.mdmarkdown7.8 KB
references/policy-authoring.mdmarkdown6.3 KB
references/remediation-paths.mdmarkdown3.0 KB
scripts/gap_analysis/__init__.pyscript534 B
scripts/gap_analysis/_bootstrap.pyscript2.7 KB
scripts/gap_analysis/agent_prompt.pyscript2.5 KB
scripts/gap_analysis/cli.pyscript19.9 KB
scripts/gap_analysis/conformance.pyscript9.6 KB
scripts/gap_analysis/detect.pyscript24.4 KB
scripts/gap_analysis/docs.pyscript2.9 KB
scripts/gap_analysis/engine.pyscript9.3 KB
scripts/gap_analysis/ingest.pyscript3.4 KB
scripts/gap_analysis/inventory.pyscript28.0 KB
scripts/gap_analysis/llm.pyscript2.6 KB
scripts/gap_analysis/migrate.pyscript11.7 KB
scripts/gap_analysis/models.pyscript5.9 KB
scripts/gap_analysis/opencode.pyscript2.7 KB
scripts/gap_analysis/paths.pyscript1.1 KB
scripts/gap_analysis/policy.pyscript1.3 KB
scripts/gap_analysis/posture.pyscript9.6 KB
scripts/gap_analysis/remediate.pyscript20.1 KB
scripts/gap_analysis/report.pyscript20.8 KB
scripts/gap_analysis/report_html.pyscript41.0 KB
scripts/gap_analysis/risk_management.pyscript36.3 KB
scripts/gap_analysis/scanners.pyscript52.0 KB
scripts/gap_analysis/settings.pyscript2.2 KB
scripts/gap_analysis/taxonomy.pyscript5.3 KB
scripts/gap_analysis/variants.pyscript1.5 KB
scripts/policy/defaults.yamlconfig4.6 KB
scripts/prompts/_contract.mdmarkdown3.0 KB
scripts/prompts/_deployment_datarobot_app.mdmarkdown1.2 KB
scripts/prompts/_fix_contract.mdmarkdown1.3 KB
scripts/prompts/_migrate_contract.mdmarkdown1.6 KB
scripts/prompts/_verify.mdmarkdown2.2 KB
scripts/prompts/aig-001-guardrails.mdmarkdown475 B
scripts/prompts/aig-002-model-pinning.mdmarkdown446 B
scripts/prompts/aig-004-evals.mdmarkdown357 B
scripts/prompts/aig-005-human-in-loop.mdmarkdown370 B
scripts/prompts/aig-006-cost-controls.mdmarkdown368 B
scripts/prompts/aig-007-grounding.mdmarkdown406 B
scripts/prompts/aig-008-prompt-versioning.mdmarkdown363 B
scripts/prompts/fix-aig-001-guardrails.mdmarkdown441 B
scripts/prompts/fix-aig-002-model-pinning.mdmarkdown527 B
scripts/prompts/fix-aig-003-approved-model.mdmarkdown351 B
scripts/prompts/fix-aig-006-cost-controls.mdmarkdown354 B
scripts/prompts/fix-idn-003-narrow-scope.mdmarkdown415 B
scripts/prompts/fix-ita-005-base-image.mdmarkdown305 B
scripts/prompts/fix-ops-001-logging.mdmarkdown298 B
scripts/prompts/fix-ops-002-tracing.mdmarkdown375 B
scripts/prompts/fix-pol-pulumi.mdmarkdown3.1 KB
scripts/prompts/fix-rel-003-resilience.mdmarkdown361 B
scripts/prompts/fix-sec-004-redact-log.mdmarkdown418 B
scripts/prompts/fix-sec-011-injection.mdmarkdown598 B
scripts/prompts/fix-sec-012-validation.mdmarkdown374 B
scripts/prompts/fix-sec-013-encryption.mdmarkdown370 B
scripts/prompts/idn-001-human-identity.mdmarkdown548 B
scripts/prompts/idn-002-rotation.mdmarkdown410 B
scripts/prompts/idn-003-over-permissioned.mdmarkdown711 B
scripts/prompts/idn-004-rbac.mdmarkdown433 B
scripts/prompts/migrate-extract-spec.mdmarkdown1.9 KB
scripts/prompts/ops-001-logging.mdmarkdown396 B
scripts/prompts/ops-002-tracing.mdmarkdown351 B
scripts/prompts/ops-003-health.mdmarkdown363 B
scripts/prompts/rel-003-resilience.mdmarkdown401 B
scripts/prompts/rel-004-fallback.mdmarkdown410 B
scripts/prompts/risk-management-mitigation.mdmarkdown2.7 KB
scripts/prompts/sec-001-secret-in-context.mdmarkdown701 B
scripts/prompts/sec-011-injection.mdmarkdown711 B
scripts/prompts/sec-012-input-validation.mdmarkdown999 B
scripts/prompts/sec-013-encryption.mdmarkdown410 B
scripts/risk_management_mitigations.yamlconfig32.0 KB
scripts/run_gap_analysis.pyexecutablescript1.5 KB
scripts/taxonomy.yamlconfig28.8 KB
scripts/validate_risk_management_mapping.pyexecutablescript2.0 KB

scripts/gap_analysis/docs.py

View on GitHub
# Copyright (c) 2026 DataRobot, Inc. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

"""Resolve DataRobot documentation pages at run time.

docs.datarobot.com publishes an llms.txt index (one `- [Title](URL): summary`
line per page). Mitigations name a docs *topic*; the matching page is looked up
here so the report never carries a pinned URL that can go stale. Offline, the
topic is shown as a search hint instead of a link.
"""

from __future__ import annotations

import re
import urllib.request

LLMS_TXT_URL = "https://docs.datarobot.com/llms.txt"
_LINE_RE = re.compile(
    r"^- \[([^\]]+)\]\((https?://[^)]+)\)(?::\s*(.*))?$", re.MULTILINE
)
_WORD_RE = re.compile(r"[a-z0-9]+")
# Classic UI, API reference and release notes describe the feature but are not
# where a user goes to configure it; prefer the current product pages.
_DEPRIORITIZED_PATHS = ("/classic-ui/", "/api/", "/release/")
_STOP = {
    "the",
    "a",
    "an",
    "and",
    "or",
    "of",
    "for",
    "to",
    "in",
    "on",
    "with",
    "datarobot",
}

_index_cache: list[tuple[str, str, str]] | None = None


def parse_llms_txt(text: str) -> list[tuple[str, str, str]]:
    """[(title, url, summary)] for every page line in an llms.txt document."""
    return [
        (m.group(1).strip(), m.group(2).strip(), (m.group(3) or "").strip())
        for m in _LINE_RE.finditer(text)
    ]


def docs_index(offline: bool = False) -> list[tuple[str, str, str]]:
    """The live page index, fetched once per process; empty when `offline`."""
    global _index_cache
    if _index_cache is not None:
        return _index_cache
    _index_cache = []
    if not offline:
        try:
            with urllib.request.urlopen(LLMS_TXT_URL, timeout=8) as resp:
                _index_cache = parse_llms_txt(resp.read().decode("utf-8", "ignore"))
        except (OSError, ValueError):
            _index_cache = []
    return _index_cache


def _tokens(text: str) -> set[str]:
    return {w for w in _WORD_RE.findall(text.lower()) if w not in _STOP}


def resolve_docs(
    topic: str, index: list[tuple[str, str, str]] | None = None, offline: bool = False
) -> str:
    """URL of the docs page that best matches `topic`, or "" when nothing fits.

    Title matches count double; a page must cover at least half of the topic's
    words so a vague topic does not pick an arbitrary page.
    """
    want = _tokens(topic)
    if not want:
        return ""
    best_url, best_score = "", 0.0
    for title, url, summary in index if index is not None else docs_index(offline):
        title_hits = len(want & _tokens(title))
        score = 2.0 * title_hits + len(want & _tokens(summary))
        if title_hits == len(want):
            score += 3  # the title says exactly what the topic asks for
        if any(part in url for part in _DEPRIORITIZED_PATHS):
            score -= 1.5
        if score > best_score:
            best_url, best_score = url, score
    return best_url if best_score / (2 * len(want)) >= 0.5 else ""
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall