Skip to content

datarobot-oss/datarobot-agent-skills / skills/datarobot-gap-analysis

datarobot-gap-analysis

Flagged

Use when the user wants to assess whether an AI agent repository (DataRobot-built or not) is enterprise-ready, run a gap analysis / readiness scan against security, governance, reliability, or regulatory standards, score an agent against EU AI Act requirements, or find and fix gaps (secrets, unpinned models, missing CI, missing guardrails, over-permissioned identity, etc.) before deploying an agent to production. Works on any repository, not only ones built with DataRobot.

FileKindSize
SKILL.mdskill md17.2 KB
references/pillars-and-layers.mdmarkdown7.8 KB
references/policy-authoring.mdmarkdown6.3 KB
references/remediation-paths.mdmarkdown3.0 KB
scripts/gap_analysis/__init__.pyscript534 B
scripts/gap_analysis/_bootstrap.pyscript2.7 KB
scripts/gap_analysis/agent_prompt.pyscript2.5 KB
scripts/gap_analysis/cli.pyscript19.9 KB
scripts/gap_analysis/conformance.pyscript9.6 KB
scripts/gap_analysis/detect.pyscript24.4 KB
scripts/gap_analysis/docs.pyscript2.9 KB
scripts/gap_analysis/engine.pyscript9.3 KB
scripts/gap_analysis/ingest.pyscript3.4 KB
scripts/gap_analysis/inventory.pyscript28.0 KB
scripts/gap_analysis/llm.pyscript2.6 KB
scripts/gap_analysis/migrate.pyscript11.7 KB
scripts/gap_analysis/models.pyscript5.9 KB
scripts/gap_analysis/opencode.pyscript2.7 KB
scripts/gap_analysis/paths.pyscript1.1 KB
scripts/gap_analysis/policy.pyscript1.3 KB
scripts/gap_analysis/posture.pyscript9.6 KB
scripts/gap_analysis/remediate.pyscript20.1 KB
scripts/gap_analysis/report.pyscript20.8 KB
scripts/gap_analysis/report_html.pyscript41.0 KB
scripts/gap_analysis/risk_management.pyscript36.3 KB
scripts/gap_analysis/scanners.pyscript52.0 KB
scripts/gap_analysis/settings.pyscript2.2 KB
scripts/gap_analysis/taxonomy.pyscript5.3 KB
scripts/gap_analysis/variants.pyscript1.5 KB
scripts/policy/defaults.yamlconfig4.6 KB
scripts/prompts/_contract.mdmarkdown3.0 KB
scripts/prompts/_deployment_datarobot_app.mdmarkdown1.2 KB
scripts/prompts/_fix_contract.mdmarkdown1.3 KB
scripts/prompts/_migrate_contract.mdmarkdown1.6 KB
scripts/prompts/_verify.mdmarkdown2.2 KB
scripts/prompts/aig-001-guardrails.mdmarkdown475 B
scripts/prompts/aig-002-model-pinning.mdmarkdown446 B
scripts/prompts/aig-004-evals.mdmarkdown357 B
scripts/prompts/aig-005-human-in-loop.mdmarkdown370 B
scripts/prompts/aig-006-cost-controls.mdmarkdown368 B
scripts/prompts/aig-007-grounding.mdmarkdown406 B
scripts/prompts/aig-008-prompt-versioning.mdmarkdown363 B
scripts/prompts/fix-aig-001-guardrails.mdmarkdown441 B
scripts/prompts/fix-aig-002-model-pinning.mdmarkdown527 B
scripts/prompts/fix-aig-003-approved-model.mdmarkdown351 B
scripts/prompts/fix-aig-006-cost-controls.mdmarkdown354 B
scripts/prompts/fix-idn-003-narrow-scope.mdmarkdown415 B
scripts/prompts/fix-ita-005-base-image.mdmarkdown305 B
scripts/prompts/fix-ops-001-logging.mdmarkdown298 B
scripts/prompts/fix-ops-002-tracing.mdmarkdown375 B
scripts/prompts/fix-pol-pulumi.mdmarkdown3.1 KB
scripts/prompts/fix-rel-003-resilience.mdmarkdown361 B
scripts/prompts/fix-sec-004-redact-log.mdmarkdown418 B
scripts/prompts/fix-sec-011-injection.mdmarkdown598 B
scripts/prompts/fix-sec-012-validation.mdmarkdown374 B
scripts/prompts/fix-sec-013-encryption.mdmarkdown370 B
scripts/prompts/idn-001-human-identity.mdmarkdown548 B
scripts/prompts/idn-002-rotation.mdmarkdown410 B
scripts/prompts/idn-003-over-permissioned.mdmarkdown711 B
scripts/prompts/idn-004-rbac.mdmarkdown433 B
scripts/prompts/migrate-extract-spec.mdmarkdown1.9 KB
scripts/prompts/ops-001-logging.mdmarkdown396 B
scripts/prompts/ops-002-tracing.mdmarkdown351 B
scripts/prompts/ops-003-health.mdmarkdown363 B
scripts/prompts/rel-003-resilience.mdmarkdown401 B
scripts/prompts/rel-004-fallback.mdmarkdown410 B
scripts/prompts/risk-management-mitigation.mdmarkdown2.7 KB
scripts/prompts/sec-001-secret-in-context.mdmarkdown701 B
scripts/prompts/sec-011-injection.mdmarkdown711 B
scripts/prompts/sec-012-input-validation.mdmarkdown999 B
scripts/prompts/sec-013-encryption.mdmarkdown410 B
scripts/risk_management_mitigations.yamlconfig32.0 KB
scripts/run_gap_analysis.pyexecutablescript1.5 KB
scripts/taxonomy.yamlconfig28.8 KB
scripts/validate_risk_management_mapping.pyexecutablescript2.0 KB

scripts/prompts/_contract.md

View on GitHub
# Shared Detection Output Contract

Every Layer-2 / Layer-4 detection prompt MUST return a single JSON object and
nothing else. The orchestrator parses this; prose outside the JSON breaks it.

```json
{
  "condition_id": "<the id being checked, e.g. SEC-001>",
  "status": "found | not_found | skipped",
  "skip_reason": "<only when status=skipped, e.g. 'relational pair incomplete'>",
  "findings": [
    {
      "file": "relative/path.py",
      "line": 42,
      "evidence": "short code excerpt or description — NEVER include a real secret value",
      "explanation": "why this is a gap",
      "confidence": "high | medium | low",
      "root_cause": "short label shared by every location with the same underlying defect",
      "severity_adjustment": "keep | lower",
      "severity_reason": "<only when lower: why this instance is less severe than the condition's default>"
    }
  ]
}
```

Rules for every detection prompt:
- Report only the file, line, and code path. **Never echo an actual secret value** —
  describe it (e.g. "OpenAI-style key assigned to `OPENAI_API_KEY`").
- If a relational check is missing one of its required file groups, return
  `status: "skipped"` with a `skip_reason` rather than guessing.
- Prefer precision over recall: if unsure, use `confidence: "low"` rather than omitting.
- Return at most 10 findings, the most consequential first. Locations that share
  a `root_cause` are ONE finding: cite the clearest location and name the others
  in `evidence` ("same pattern in tools/b.py, tools/c.py"). Never enumerate every
  parameter or every call site; the reply must stay short enough to finish.
- `line` may be null when the finding is file-level (e.g. "no tests anywhere").
- Files are shown with each line prefixed `N| `. Report that N as `line`, and put
  the exact code of that line (without the prefix) in `evidence` so it can be
  checked mechanically.
- A large file may be shown as an EXCERPT: its header, its definition lines, and
  windows around the lines that matter for this condition, with
  `… lines A-B omitted …` markers. Line numbers are the file's real ones. Never
  report on an omitted range; judge only the lines shown.
- Read the enclosing block before claiming something is unguarded: a `try/except`,
  validator, or allowlist a few lines above the cited line means it is guarded.
- A comment that explains why a guard exists ("timeout added to prevent a hang")
  is evidence the guard exists, not evidence the bug is live.
- Compensating controls elsewhere in the same file lower severity; say so with
  `severity_adjustment: "lower"` instead of reporting the default severity.
- When a file only delegates (a 3-line entrypoint calling `create_app()`), do not
  assert absence of what you could not see; use `confidence: "low"` or skip.
- Group locations that share one underlying defect under the same `root_cause`;
  the report merges them into a single finding.
- The section `REPO-WIDE EVIDENCE HINTS` lists grep hits from files you were not
  shown. Treat them as evidence that may already satisfy the condition.
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall