Skip to content

BdM-15/capture-insights / skills/competitive-intel

competitive-intel

Verified

Federal capture competitor, incumbent, and obligation intelligence backed by live USAspending.gov data via the vendored `usaspending` MCP. USE WHEN the user asks "who's the incumbent on this contract?", "what was the last award value?", "give me a black-hat read on Bidder X", "who are the top three competitors for this NAICS?", "what's typical pricing for this work?", "pull award history for this agency / NAICS / PSC", "analyze burn rate", "obligation trend by year", or "start from this contract number". Supports both black-hat competitor research and contract-number-first obligation analysis, including standalone awards, parent IDIQs, and orders under IDIQs with parent/child rollups. DO NOT USE FOR proposal drafting (use `proposal-generator`), clause / FAR compliance audit (use `compliance-auditor`), or generic web search about a company (no MCP tool covers that yet).

FileKindSize
SKILL.mdskill md29.0 KB
evals/evals.jsonconfig11.7 KB
references/black_hat_template.mdmarkdown1.5 KB
references/data_sources.mdmarkdown10.2 KB

evals/evals.json

View on GitHub
{
  "skill": "competitive-intel",
  "version": "0.6.2",
  "notes": "Phase 4e+ — black-hat plus contract-number obligation modes. Workflow B now calls collect_competitive_obligation_intel with vehicle scope for parent-level synthesis. Workflow C calls the same collector with single_award scope so idiq_order artifacts stay order-only while still carrying compact parent-vehicle context. Run from a workspace whose KG has at least program/agency/NAICS context OR provide contract number inputs directly. Live MCP eval requires THESEUS_LIVE_MCP=1.",
  "evals": [
    {
      "id": "eval-001-incumbent-active-window",
      "prompt": "Who is the incumbent on this contract and when does their period of performance end?",
      "expectations": [
        {
          "kind": "artifact_exists",
          "path": "competitive_intel.json"
        },
        {
          "kind": "json_path_set",
          "path": "incumbent.name"
        },
        {
          "kind": "json_path_in_set",
          "path": "incumbent.incumbent_status",
          "values": ["current", "prior", "unknown"]
        },
        {
          "kind": "json_path_set",
          "path": "incumbent.flagship_award_id"
        },
        {
          "kind": "transcript_tool_called",
          "tool": "mcp__usaspending__search_awards"
        }
      ]
    },
    {
      "id": "eval-002-top-three-competitors",
      "prompt": "Give me a black-hat read on the top three likely competitors for this opportunity. I need predicted themes and ghost language hooks I can hand to my proposal lead.",
      "expectations": [
        {
          "kind": "artifact_exists",
          "path": "competitive_intel.json"
        },
        {
          "kind": "json_path_array_min_length",
          "path": "competitors",
          "min": 3
        },
        {
          "kind": "json_path_array_min_length_each",
          "path": "competitors[*].predicted_themes",
          "min": 1
        },
        {
          "kind": "json_path_array_min_length_each",
          "path": "competitors[*].ghost_language_hooks",
          "min": 1
        },
        {
          "kind": "json_path_array_contains",
          "path": "competitors[*].claim_gaps",
          "value_substring": "CPARS"
        }
      ]
    },
    {
      "id": "eval-003-pricing-benchmark",
      "prompt": "What's the typical award value range for this NAICS at this magnitude over the last 5 years? Be honest if the sample is too small to give a real range.",
      "expectations": [
        {
          "kind": "artifact_exists",
          "path": "competitive_intel.json"
        },
        {
          "kind": "json_path_in_set",
          "path": "pricing_benchmark.pricing_band",
          "values": ["valid", "insufficient sample"]
        },
        {
          "kind": "json_path_set",
          "path": "pricing_benchmark.sample_size"
        },
        {
          "kind": "transcript_tool_called",
          "tool": "mcp__usaspending__search_awards"
        }
      ]
    },
    {
      "id": "eval-004-missing-naics-fail-loud",
      "prompt": "Run competitive intel on this opportunity.",
      "workspace_hint": "use a workspace with NO regulatory_reference entities for NAICS/PSC",
      "expectations": [
        {
          "kind": "response_contains",
          "value": "GAP"
        },
        {
          "kind": "transcript_no_invention",
          "rule": "no competitor name should appear that wasn't returned by mcp__usaspending__search_awards"
        }
      ]
    },
    {
      "id": "competitive-intel-grounding-floor",
      "name": "Grounding-audit floor",
      "prompt": "Who is the incumbent on this contract and when does their period of performance end?",
      "expected_signals": [
        "tools/audit_skill_grounding.py --skill competitive-intel reports grounding_ratio >= 0.65 (black-hat envelope is mostly KG-grounded incumbent claims, but ~25% of cover-note prose is competitive-strategy reasoning; the on-disk JSON envelope source_chunks are the citation-of-record, the chat narrative is the human-readable digest).",
        "tools/audit_skill_grounding.py reports claims_exempt is at least 0.40 of (claims_total + claims_exempt).",
        "No regression below 0.6 across at least three consecutive runs of the same prompt."
      ]
    },
    {
      "id": "eval-005-obligation-standalone-contract",
      "prompt": "Start from contract number N00024-24-C-0085 and produce obligation intel. Tell me if this is standalone, parent IDIQ, or order, and give me by-year and by-modification obligations.",
      "expectations": [
        {
          "kind": "artifact_exists",
          "path": "competitive_intel_obligation.json"
        },
        {
          "kind": "transcript_tool_called",
          "tool": "collect_competitive_obligation_intel"
        },
        {
          "kind": "json_path_in_set",
          "path": "resolved.scenario",
          "values": ["standalone_contract", "parent_idiq", "idiq_order"]
        },
        {
          "kind": "json_path_set",
          "path": "obligations.by_fiscal_year"
        },
        {
          "kind": "json_path_set",
          "path": "obligations.by_transaction"
        },
        {
          "kind": "json_path_set",
          "path": "obligations.by_period_of_performance"
        },
        {
          "kind": "json_path_set",
          "path": "obligations.by_transaction[0].cumulative_obligated_usd",
          "notes": "Every by_transaction row must carry a running cumulative total"
        },
        {
          "kind": "json_path_set",
          "path": "obligations.by_transaction[0].inferred_pop_segment",
          "notes": "Every row must carry a POP segment label: base_year, option_year_N, supplemental, admin, or unknown"
        },
        {
          "kind": "json_path_set",
          "path": "obligations.rate_analysis",
          "notes": "rate_analysis block with monthly_burn_usd and by_option_year must be present"
        },
        {
          "kind": "json_path_set",
          "path": "obligations.rate_analysis.monthly_burn_usd"
        },
        {
          "kind": "json_path_set",
          "path": "obligations.rate_analysis.by_option_year"
        },
        {
          "kind": "json_path_set",
          "path": "insights.headline"
        },
        {
          "kind": "json_path_array_min_length",
          "path": "insights.blocks",
          "min": 2
        }
      ]
    },
    {
      "id": "eval-006-obligation-parent-idiq-rollup",
      "prompt": "Analyze this parent IDIQ for obligation burn and competitor completeness. Roll up all child orders, give a per-order breakdown, and tell me all order-holder competitors.",
      "expectations": [
        {
          "kind": "artifact_exists",
          "path": "competitive_intel_obligation.json"
        },
        {
          "kind": "transcript_tool_called",
          "tool": "collect_competitive_obligation_intel"
        },
        {
          "kind": "json_path_in_set",
          "path": "resolved.scenario",
          "values": ["parent_idiq"]
        },
        {
          "kind": "json_path_array_min_length",
          "path": "hierarchy.child_award_ids",
          "min": 1
        },
        {
          "kind": "json_path_array_min_length",
          "path": "obligations.by_child_order",
          "min": 1
        },
        {
          "kind": "json_path_set",
          "path": "obligations.by_period_of_performance"
        },
        {
          "kind": "json_path_array_min_length",
          "path": "competitor_discovery.order_holder_recipients",
          "min": 1
        },
        {
          "kind": "json_path_in_set",
          "path": "competitor_discovery.completeness_status",
          "values": ["high", "medium", "low"]
        },
        {
          "kind": "json_path_set",
          "path": "insights.headline"
        },
        {
          "kind": "json_path_array_min_length",
          "path": "insights.blocks",
          "min": 3
        }
      ]
    },
    {
      "id": "eval-007-obligation-idiq-order-sibling-competitors",
      "prompt": "I only have this order number. Resolve parent IDIQ, calculate this order's obligations, then tally full sibling-order vehicle obligations and identify competitors tied to the parent vehicle.",
      "expectations": [
        {
          "kind": "artifact_exists",
          "path": "competitive_intel_obligation.json"
        },
        {
          "kind": "transcript_tool_called",
          "tool": "collect_competitive_obligation_intel"
        },
        {
          "kind": "json_path_in_set",
          "path": "resolved.scenario",
          "values": ["idiq_order", "parent_idiq"]
        },
        {
          "kind": "json_path_set",
          "path": "hierarchy.parent_award_id"
        },
        {
          "kind": "json_path_array_min_length",
          "path": "hierarchy.child_award_ids",
          "min": 1
        },
        {
          "kind": "json_path_set",
          "path": "obligations.by_period_of_performance"
        },
        {
          "kind": "json_path_array_min_length",
          "path": "competitor_discovery.parent_holder_recipients",
          "min": 1
        },
        {
          "kind": "json_path_in_set",
          "path": "competitor_discovery.linkage_method_used",
          "values": ["parent_child", "piid_family", "solicitation_fallback"]
        },
        {
          "kind": "json_path_set",
          "path": "insights.headline"
        },
        {
          "kind": "json_path_array_min_length",
          "path": "insights.blocks",
          "min": 3
        }
      ]
    },
    {
      "id": "eval-008-single-order-burn-intel",
      "prompt": "I only care about this one order. Give me a clean single-award burn read with grouped mods, POP, burn rate, and only brief parent vehicle context.",
      "expectations": [
        {
          "kind": "artifact_exists",
          "path": "competitive_intel_obligation.json"
        },
        {
          "kind": "transcript_tool_called",
          "tool": "collect_competitive_obligation_intel"
        },
        {
          "kind": "json_path_array_min_length",
          "path": "obligations.by_award",
          "min": 1
        },
        {
          "kind": "json_path_set",
          "path": "obligations.by_award[0].by_transaction"
        },
        {
          "kind": "json_path_set",
          "path": "obligations.by_award[0].net_obligated_usd"
        },
        {
          "kind": "json_path_in_set",
          "path": "scope",
          "values": ["single_award"]
        },
        {
          "kind": "json_path_set",
          "path": "insights.headline"
        },
        {
          "kind": "json_path_array_min_length",
          "path": "insights.blocks",
          "min": 2
        },
        {
          "kind": "json_path_set",
          "path": "vehicle_context.parent_award_id"
        }
      ]
    },
    {
      "id": "eval-009-order-only-explicit-contract-only",
      "prompt": "Provide me an obligation rate on this contract only: FA805122F0001",
      "expectations": [
        {
          "kind": "artifact_exists",
          "path": "competitive_intel_obligation.json"
        },
        {
          "kind": "transcript_tool_called",
          "tool": "collect_competitive_obligation_intel"
        },
        {
          "kind": "json_path_in_set",
          "path": "scope",
          "values": ["single_award"]
        },
        {
          "kind": "json_path_in_set",
          "path": "resolved.scenario",
          "values": ["idiq_order", "standalone_contract"]
        },
        {
          "kind": "json_path_in_set",
          "path": "obligations.by_award[0].role",
          "values": ["current_order", "resolved_award"]
        },
        {
          "kind": "json_path_set",
          "path": "vehicle_context.child_order_count"
        }
      ]
    }
  ]
}
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall