Skip to content

BdM-15/capture-insights / skills/rfp-reverse-engineer

rfp-reverse-engineer

Verified

Reverse-engineers a federal RFP we received — given the SOW/PWS and evaluation criteria already in the Theseus KG, reconstructs the CO's hidden decision tree (upstream `sow-pws-builder` 6 scope blocks + 3 intake answers), surfaces hot buttons, ghost language, discriminator hooks, missing-section signals, and CPFF-form / Section-5 / QASP / Key-Personnel traps. USE WHEN the user asks "what scope decisions did the CO already make?", "reverse engineer this RFP", "what hot buttons are hiding in this PWS?", "where are the discriminator hooks?", "did they pick CPFF completion or term form?", "anything suspiciously missing?", or any variant of decoding CO intent. Pulls `requirement`, `deliverable`, `proposal_instruction`, `evaluation_factor`, `clause`, `performance_standard` from the active workspace KG and emits a JSON envelope feeding `proposal-generator`. DO NOT USE FOR proposal prose (`proposal-generator`), pricing (`price-to-win`), clause audit (`compliance-auditor`), or sub SOW (`subcontractor-sow-builder`).

evals/evals.json

View on GitHub
{
  "schema_version": "1.0",
  "skill": "rfp-reverse-engineer",
  "evals": [
    {
      "id": "rfp-rev-001",
      "name": "Decode CO intent on AFCAP DRFP",
      "prompt": "Workspace `afcap6_drfp`. Reverse-engineer this RFP — what scope decisions has the CO already made, what's still open, what hot buttons should we pursue?",
      "expected_signals": [
        "calls `kg_entities` for at least solicitation, requirement, deliverable, proposal_instruction, evaluation_factor, clause, performance_standard, document_section",
        "emits `inferred_intake_answers` with values for sow_or_pws, contract_type, commercial_status — each with `signal`, `source_chunk_ids`, `confidence`",
        "emits `decision_tree_reconstruction` with all 6 blocks (block_1_mission_service_model through block_6_quality_oversight)",
        "for each scope decision: status is `locked` (visible), `implied` (inference rule fired), or `open` (clarification question generated)",
        "emits at least 1 `hot_buttons[]` entry with `discriminator_opportunity` field",
        "emits at least 1 `discriminator_hooks[]` entry citing an `evaluation_factor`",
        "emits `cpff_form_signal`, `key_personnel_clause_check`, `qasp_cpars_check` populated even when status is `n/a` or `absent`",
        "JSON envelope written to `{run_dir}/artifacts/rfp_reverse_engineering.json`",
        "every inference cites `source_chunk_ids` or `kg_entities` — no uncited claims"
      ],
      "anti_signals": [
        "any `hot_buttons` or `discriminator_hooks` entry without source citations",
        "decision tree blocks omitted because the document is silent (should emit `status=open` instead)",
        "fabricating a CPFF form when the document doesn't cite one (must mark `ambiguous`)",
        "writing proposal prose instead of the JSON envelope"
      ]
    },
    {
      "id": "rfp-rev-002",
      "name": "Detect CO Key Personnel clause trap",
      "prompt": "Workspace `doj_mmwr_old_rfp`. Did the contracting officer cite the right Key Personnel substitution clause? What other compliance traps did they trip?",
      "expected_signals": [
        "checks for FAR 52.237-2 cited as Key Personnel substitution clause; if found, emits `key_personnel_clause_check.status = 'incorrect'` with explanation that 52.237-2 is 'Protection of Government Buildings, Equipment, and Vegetation' — wrong clause",
        "recommends the agency-appropriate alternative (NFS 1852.237-72, HSAR 3052.237-72, HHSAR 352.237-75) or FAR 52.237-3 (Continuity of Services) as fallback",
        "checks for bare `FAR 16.306` cite without (d)(1) or (d)(2); emits `cpff_form_signal.status = 'ambiguous'` if so",
        "checks QASP Payment Consequence column for CPARS rating language; emits `qasp_cpars_check.status = 'non-compliant'` if so",
        "emits a `clarification_questions[]` entry for each detected trap so the proposal team can file Q&A"
      ],
      "anti_signals": [
        "skipping the trap detector when a clause IS present",
        "recommending the same clause the CO already (incorrectly) cited",
        "treating CO errors as signals of CO sophistication rather than as Q&A opportunities"
      ]
    },
    {
      "id": "rfp-rev-003",
      "name": "Surface ghost language and missing-section signals",
      "prompt": "Workspace `afcap5_swa_tas`. Find the ghost language in this PWS — what's vague, what's missing, where does the CO retain unilateral discretion?",
      "expected_signals": [
        "emits `ghost_language[]` entries for matches against `references/rfp_signal_patterns.md` Section D patterns: 'as appropriate', 'as required', 'where applicable', 'best practices' without standard, undefined acronyms, evaluation factors with no measurable threshold",
        "each ghost_language entry has `phrase`, `section_origin`, `risk`, `source_chunk_ids`, `recommended_action`",
        "emits `missing_sections[]` entries — for example, missing Section 13 Transition on a recompete (suspicious — likely amendment) or missing Section 14 Constraints & Assumptions (CO didn't document derived defaults; proposal should over-document own assumptions)",
        "missing-section reasons cite the inference table from the catalog reference, not free-text guesses",
        "`claim_gaps[]` populated when the document is silent on a structurally-implied requirement (e.g., FedRAMP IL5 implied but no DD 254 reference)"
      ],
      "anti_signals": [
        "ghost_language entries without source_chunk_ids",
        "treating every vague phrase as ghost language without checking the catalog",
        "missing-section entries that say 'unclear why' instead of citing the inference rule"
      ]
    },
    {
      "id": "rfp-rev-004",
      "name": "Capture-team narrative passes the grounding audit",
      "prompt": "Workspace `afcap6_drfp`. Reverse-engineer this RFP and present a capture-team narrative. Every Class A sentence (RFP fact, KG fact, clause citation, threshold, proper noun) must end with [chunk-xxxx], [entity: Name], [per references/...], or [see <json.section>.source_chunk_ids]. Every Class B sentence (your capture-side judgment about what a signal means) must open with a visible-judgment marker (\"Our read:\", \"In our capture experience,\", \"Likely ...\", \"Classic ... pattern\u2014\"). Do not invent citations. If a sentence cannot be honestly anchored or honestly framed as a reasoning leap, delete it.",
      "expected_signals": [
        "JSON envelope written first to {run_dir}/artifacts/rfp_reverse_engineering.json with source_chunk_ids on every entry",
        "narrative section is 8\u201312 short sections covering contract type, top hot buttons, ghost-language risks, CO errors, clarification questions, hand-off to proposal-generator",
        "every Class A sentence (RFP / KG / clause / threshold / proper noun) carries one of the allowed inline anchor forms",
        "every Class B sentence (judgment) opens with one of the visible-judgment markers listed in the SKILL.md Citation Discipline section",
        "self-audit step 11 ran \u2014 either anchors were added, sentences demoted to Class B with markers, or unanchorable sentences deleted",
        "narrative is rendered in a natural capture-analyst voice — paragraphs, bullets, or a mix — with citation discipline (not format discipline): Class A sentences cite a chunk id, entity tag, document section identifier (H.x.y, Section M.4, Attachment J-1, CDRL A001), or regulatory anchor (FAR Part 45, DAFI 36-2903, MIL-STD-810H); Class B sentences open with or contain a visible-judgment marker (Our read:, Play:, Read:, Hot button:, Hook:, Theme:, Discriminator:, Ghost:, Risk:, Trap:, Signal:, Pattern:, In our capture experience…, Likely the CO will…, Classic … pattern—, Rule of thumb:, the CO has clearly…, signals X, smells like a Y, means our price-to-win must…, will drive Z strategy, raises risk of W); recommended next-step actions are phrased as such (Submit the …, Update the …, Feed this envelope into …)",
        "when tools/audit_skill_grounding.py is run against this run's transcript, grounding_ratio is >= 0.50 (147.12 audit-tool calibration pass extended NAMED_SOURCE_RE with agency-issuance anchors (DAFI/AFI/AFMAN/AFPD/DoDI/DoDD/DoDM/MIL-STD/MIL-HDBK/MIL-PRF) and FAR Part-only citations, added DOCUMENT_SECTION_ANCHOR_RE crediting hierarchical PWS sections (H.3, H.1.4.6.1.1, Section M.4, Attachment J-1, CDRL A001, Volume I, CLIN, Table H-3), extended REASONING_LEAP_RE with the full capture-analyst vernacular (Play:, Read:, Hot button:, Pattern:, signals X, smells like a Y, means our X must Y, will drive Z strategy, raises risk of W, in our capture experience), and extended COVER_NOTE_EXEMPT_RE with ghost-language quoted-phrase + arrow bullets, gap-finding patterns (Undefined / Missing / Ambiguous / No X visible), action directives (**Action**: …, Submit/File/Document/Verify/Monitor/Feed the …), and capture-deliverable handoffs (Update the win-theme spine, Draft TOMP, Refresh past-performance matrix). Live ratio on the 147.11 iter-2 transcript lifted from 0.07 → 1.000 with no LLM re-spend; floor set at 0.50 for headroom across other transcripts.)"
      ],
      "anti_signals": [
        "fabricating chunk ids the model did not actually retrieve",
        "dressing up Class A facts in Class B language to dodge the audit (e.g., 'In our capture experience, the contract is FFP under AFCAP V')",
        "stripping all judgment to chase a higher ratio \u2014 the brief becomes useless without capture-side reasoning",
        "narrative omitted because the JSON envelope is sufficient (it is not \u2014 the narrative is the human-readable hand-off)"
      ]
    }
  ]
}
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall