datarobot-oss/datarobot-agent-skills / skills/datarobot-gap-analysis
datarobot-gap-analysis
FlaggedUse when the user wants to assess whether an AI agent repository (DataRobot-built or not) is enterprise-ready, run a gap analysis / readiness scan against security, governance, reliability, or regulatory standards, score an agent against EU AI Act requirements, or find and fix gaps (secrets, unpinned models, missing CI, missing guardrails, over-permissioned identity, etc.) before deploying an agent to production. Works on any repository, not only ones built with DataRobot.
references/remediation-paths.md
View on GitHub# Remediation Path Determination
Fix-in-place vs. rebuild-from-scratch is a false binary. The unit of decision is the
**gap**, not the agent. Plumbing gaps (secrets, unpinned models, missing scaffolding)
patch safely; **structural** gaps (no observability, no guardrails, human/shared
identity, no resilience) cannot be surgically fixed, "fixing" them means restructuring
business logic someone else owns. When structural gaps dominate, lifting the business
logic into a fresh, conformant base is safer than in-place surgery.
## How a condition becomes "structural"
A condition in `taxonomy.yaml` is structural when either:
- It is explicitly flagged `structural: true`, or
- Its `fix_type` is `advisory` **and** its severity is `high` or `critical` (an advisory
finding has no automated fix by definition; at high/critical severity there is no
safe way to leave it alone either).
## Scoring
Each finding is weighted by severity (critical=4, high=3, medium=2, low=1). The
**structural score** is `structural_weight / total_weight` across all findings in the
run.
## The three postures
| Posture | Condition | Meaning |
|---|---|---|
| **PATCH** | Structural score ≤ 0.25 | Only a small share of weighted risk is structural. Fixes are surgical and low-risk to existing business logic. |
| **HYBRID** | Structural score between 0.25 and 0.50, and fewer than 4 high/critical structural findings | Mixed profile. Automated fixes handle the plumbing now; flagged structural items need targeted human review, or a later Re-platform pass, before a clean rescore. |
| **RE-PLATFORM** | Structural score ≥ 0.50, **or** 4+ high/critical structural findings (an absolute override, so a small repo that is almost entirely structural doesn't get missed by density alone) | Structural risk is pervasive. Patching means restructuring code you don't own. Extract the business logic into a fresh base instead of restructuring in place. |
Every posture ships with a plain-language rationale naming the count of structural
findings and the weighted-risk percentage, so the recommendation is never a bare label.
## Thresholds are policy, not code
`patch_max`, `replatform_min`, and the absolute structural-count override all live
under `posture:` in the policy file (see
[policy-authoring.md](policy-authoring.md)) and can be tuned per org without a code
change.
## Re-platform mechanics
1. **Extract** — the business logic (system prompt, tools, decision flow) is lifted
into a structured, human-readable `agent_spec.md`.
2. **Human review gate** — the spec, not a code diff, is the checkpoint. Nothing is
scaffolded until the developer approves it. This is what makes the path safe when
the codebase under analysis isn't one this skill (or the developer running it)
originally wrote.
3. **Scaffold** — on approval, the spec is the input to `datarobot-agent-assist`'s
coding flow, which scaffolds a governed agent from it using the DataRobot
application-framework stack.
4. **Rescore** — re-run the assessment against the scaffolded agent to confirm the
structural gaps are closed.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |