datarobot-oss/datarobot-agent-skills / skills/datarobot-gap-analysis
datarobot-gap-analysis
FlaggedUse when the user wants to assess whether an AI agent repository (DataRobot-built or not) is enterprise-ready, run a gap analysis / readiness scan against security, governance, reliability, or regulatory standards, score an agent against EU AI Act requirements, or find and fix gaps (secrets, unpinned models, missing CI, missing guardrails, over-permissioned identity, etc.) before deploying an agent to production. Works on any repository, not only ones built with DataRobot.
scripts/gap_analysis/llm.py
View on GitHub# Copyright (c) 2026 DataRobot, Inc. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
"""LLM client protocol for Layer-2/4 reasoning.
The CLI builds an opencode-backed client (see opencode.py) and passes it in; an
embedding may inject any callable(system, user) -> str. There is no standalone
fallback: without the DataRobot CLI the LLM layers are skipped and the report
says so.
"""
from __future__ import annotations
import json
from collections.abc import Callable
from typing import Any, Protocol
from .settings import DEFAULTS, Settings
class LLMClient(Protocol):
def complete(self, system: str, user: str) -> str: # pragma: no cover - interface
...
class InjectedClient:
"""Wraps a callable(system, user)->str, e.g. from af-component-llm."""
def __init__(self, fn: Callable[[str, str], str]):
self._fn = fn
def complete(self, system: str, user: str) -> str:
return str(self._fn(system, user))
def get_client(
injected: LLMClient | Callable[[str, str], str] | None = None,
settings: Settings = DEFAULTS,
) -> LLMClient | None:
"""Return an LLM client, or None if none is configured/available."""
del settings # the CLI decides the backend; nothing is auto-detected here
if injected is not None:
if hasattr(injected, "complete"):
return injected # type: ignore[return-value]
return InjectedClient(injected) # type: ignore[arg-type]
return None
def parse_json(text: str) -> dict[str, Any]:
"""The first JSON object in a model response, ignoring fences and any
commentary the model appended after the closing brace."""
text = text.strip()
if text.startswith("```"):
text = text.strip("`")
text = text.split("\n", 1)[1] if "\n" in text else text
try:
loaded = json.loads(text)
if not isinstance(loaded, dict):
# ValueError, not TypeError: callers retry on ValueError like a parse error.
raise ValueError("model response is not a JSON object") # noqa: TRY004
return loaded
except json.JSONDecodeError:
start = text.find("{")
if start < 0:
raise
obj, _end = json.JSONDecoder().raw_decode(text[start:])
if not isinstance(obj, dict):
raise ValueError("model response is not a JSON object") from None # noqa: TRY004
return obj
def brief_error(e: BaseException, limit: int = 300) -> str:
"""One line of an exception message, capped, for skip reasons and notes."""
text = " ".join(str(e).split())
return text if len(text) <= limit else text[: limit - 1] + "…"
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |