Skip to content

alexgreensh/repo-forensics / skills/repo-forensics

repo-forensics

Flagged

Security forensics for git repos, AI skills, and MCP servers. Audits dependencies, detects prompt injection, credential theft, runtime dynamism, manifest drift, known CVEs, CISA KEV (actively exploited) vulns, and 2026 attack patterns. Not for fixing vulnerabilities or pentesting.

FileKindSize
.gitignoreother193 B
SKILL.mdskill md43.4 KB
checksums.jsonconfig10.8 KB
checksums.json.sigother64 B
data/README.mdmarkdown1.3 KB
data/compromised_versions.jsonconfig188.3 KB
data/rule_ids.csvother35.3 KB
data/rulepacks/.gitkeepother221 B
data/rulepacks/dead_anchors.jsonconfig9.0 KB
data/rulepacks/mcp_security.jsonconfig31.2 KB
data/rulepacks/runtime_dynamism.jsonconfig46.7 KB
data/rulepacks/sast.jsonconfig107.6 KB
data/rulepacks/secrets.jsonconfig30.4 KB
data/rulepacks/shared.jsonconfig4.4 KB
data/rulepacks/skill_threats.jsonconfig95.4 KB
data/runtime_floors.jsonconfig2.3 KB
data/yara/cryptominers.yarother1.8 KB
data/yara/hacktools.yarother2.2 KB
data/yara/malware.yarother2.2 KB
data/yara/manifest.jsonconfig8.0 KB
data/yara/webshells.yarother2.5 KB
references/cursor-hook-contract.mdmarkdown5.6 KB
references/kernel_privesc_sandbox_pack.mdmarkdown5.0 KB
references/mcp-attack-patterns.mdmarkdown8.7 KB
references/research_sources.mdmarkdown8.0 KB
references/threat_patterns.mdmarkdown20.3 KB
scripts/_context_gate.pyscript21.7 KB
scripts/_ed25519.pyscript5.1 KB
scripts/_pyc_unmarshal.pyscript17.1 KB
scripts/_shared_patterns.pyscript4.8 KB
scripts/adjudication.pyscript16.4 KB
scripts/adjudication_bridge.pyscript1.0 KB
scripts/aggregate_json.pyscript69.2 KB
scripts/auto_scan.pyexecutablescript152.1 KB
scripts/corpus_sync.pyscript10.0 KB
scripts/cursor_pre_scan.pyscript22.5 KB
scripts/dast_sandbox.sbother1.6 KB
scripts/dead_anchors_extract.pyscript14.8 KB
scripts/dead_anchors_probe.pyscript27.1 KB
scripts/forensics_core.pyscript176.5 KB
scripts/gen_rule_ids.pyscript8.6 KB
scripts/hook_adapter.pyscript19.4 KB
scripts/ioc_manager.pyscript49.6 KB
scripts/parse_pnpm_lock.pyscript7.9 KB
scripts/pre_scan.pyscript143.5 KB
scripts/pypi_pins.pyscript8.1 KB
scripts/refresh_controller.pyexecutablescript48.4 KB
scripts/refresh_threat_dbs.pyexecutablescript19.4 KB
scripts/rule_loader.pyscript44.8 KB
scripts/rulepack_feed.pyscript26.4 KB
scripts/run_forensics.shexecutablescript20.3 KB
scripts/sarif_export.pyscript18.6 KB
scripts/scan_agent_skills.pyscript33.2 KB
scripts/scan_archive.pyscript29.1 KB
scripts/scan_ast.pyscript44.9 KB
scripts/scan_binary.pyscript10.9 KB
scripts/scan_bytecode.pyscript28.2 KB
scripts/scan_dast.pyscript28.8 KB
scripts/scan_dataflow.pyscript15.4 KB
scripts/scan_dead_anchors.pyscript15.9 KB
scripts/scan_decode.pyscript37.5 KB
scripts/scan_dependencies.pyscript124.5 KB
scripts/scan_devcontainer.pyscript8.5 KB
scripts/scan_entropy.pyscript15.6 KB
scripts/scan_entrypoint.pyscript34.3 KB
scripts/scan_git_config.pyscript96.3 KB
scripts/scan_git_forensics.pyscript24.1 KB
scripts/scan_history.pyscript14.5 KB
scripts/scan_infra.pyscript54.3 KB
scripts/scan_integrity.pyscript41.0 KB
scripts/scan_lifecycle.pyscript65.1 KB
scripts/scan_manifest_drift.pyscript47.0 KB
scripts/scan_mcp_security.pyscript42.3 KB
scripts/scan_oversize.pyscript11.7 KB
scripts/scan_post_incident.pyscript36.9 KB
scripts/scan_provenance.pyscript19.4 KB
scripts/scan_runtime_dynamism.pyscript23.5 KB
scripts/scan_runtime_floors.pyscript33.5 KB
scripts/scan_sast.pyscript59.1 KB
scripts/scan_secrets.pyscript11.9 KB
scripts/scan_skill_threats.pyscript102.4 KB
scripts/scan_splitstream.pyscript29.5 KB
scripts/scan_yara.pyscript19.7 KB
scripts/session_scan.pyscript54.1 KB
scripts/suppressions.pyscript15.4 KB
scripts/validate_manifests.pyscript13.7 KB
scripts/verify_install.pyscript26.2 KB
scripts/vuln_feed.pyscript31.1 KB
tests/conftest.pyscript13.4 KB
tests/corpus/DEAD_ANCHORS_CORPUS.mdmarkdown2.3 KB
tests/corpus/budgets.jsonconfig5.6 KB
tests/cursor_helpers.pyscript12.6 KB
tests/fixtures/legacy_main_blocks.jsonconfig10.3 KB
tests/fixtures/powershell_verdict_corpus.jsonconfig114.9 KB
tests/golden/parity_mcp_security.jsonconfig17.5 KB
tests/golden/parity_runtime_dynamism.jsonconfig31.7 KB
tests/golden/parity_sast.jsonconfig59.8 KB
tests/golden/parity_secrets.jsonconfig18.8 KB
tests/golden/parity_skill_threats.jsonconfig48.8 KB
tests/parity_corpus.pyscript6.3 KB
tests/powershell_hook.pyscript8.2 KB
tests/report_builders.pyscript5.0 KB
tests/schemas/sarif-schema-2.1.0.jsonconfig109.1 KB
tests/shell_compat.pyscript2.2 KB
tests/test_adjudication_bridge.pyscript832 B
tests/test_adjudication_containment.pyscript1.4 KB
tests/test_adjudication_output.pyscript20.0 KB
tests/test_adjudication_suppressions.pyscript18.7 KB
tests/test_advisory_adjudication.pyscript1.3 KB
tests/test_agent_relay_rules.pyscript7.8 KB
tests/test_aggregate_json.pyscript10.0 KB
tests/test_auto_scan_patterns.pyscript10.5 KB
tests/test_benign_corpus.pyscript16.4 KB
tests/test_benign_new_scanners.pyscript6.1 KB
tests/test_build_output_skip.pyscript10.6 KB
tests/test_capability_ledger.pyscript18.8 KB
tests/test_codex_install.pyscript10.0 KB
tests/test_codex_marketplace.pyscript3.3 KB
tests/test_comment_false_positives.pyscript2.2 KB
tests/test_composer_dependencies.pyscript4.0 KB
tests/test_compromised_versions.pyscript51.2 KB
tests/test_context_gate.pyscript32.4 KB
tests/test_coverage_status.pyscript7.6 KB
tests/test_css_compounding_shrink.pyscript20.8 KB
tests/test_cursor_adapter.pyscript7.8 KB
tests/test_cursor_benign.pyscript3.4 KB
tests/test_cursor_degrade.pyscript12.7 KB
tests/test_cursor_install.pyscript17.1 KB
tests/test_cursor_latency.pyscript7.2 KB
tests/test_cursor_parity.pyscript5.8 KB
tests/test_cursor_redteam_fn.pyscript4.9 KB
tests/test_cursor_tamper_modify.pyscript14.2 KB
tests/test_cursor_wiring.pyscript32.3 KB
tests/test_d1_yara_correlation.pyscript9.2 KB
tests/test_dead_anchors_extract.pyscript7.6 KB
tests/test_dead_anchors_probe.pyscript19.7 KB
tests/test_evidence_engine_parity.pyscript8.1 KB
tests/test_evidence_model.pyscript11.4 KB
tests/test_forensics_core.pyscript79.2 KB
tests/test_fp_hardening_r3.pyscript9.7 KB
tests/test_fp_hardening_r4.pyscript18.8 KB
tests/test_fp_regression_pairs.pyscript62.5 KB
tests/test_fp_structural.pyscript21.2 KB
tests/test_freshness.pyscript6.2 KB
tests/test_frontmatter_validity.pyscript7.0 KB
tests/test_gpg_rce_hotfix.pyscript4.8 KB
tests/test_hook_inert_data.pyscript6.1 KB
tests/test_hook_scripts.pyscript14.0 KB
tests/test_inspection_cpu_budget.pyscript11.6 KB
tests/test_ioc_manager.pyscript55.4 KB
tests/test_kernel_privesc_rules.pyscript27.8 KB
tests/test_legit_precision_regressions.pyscript6.8 KB
tests/test_lethal_trifecta.pyscript20.7 KB
tests/test_manifest_signature.pyscript1.5 KB
tests/test_manifest_validity.pyscript2.9 KB
tests/test_monotonic_trust_e2e.pyscript4.4 KB
tests/test_non_breaking_contract.pyscript22.7 KB
tests/test_openclaw_install.pyscript6.6 KB
tests/test_pack_parity.pyscript24.8 KB
tests/test_parse_pnpm_lock.pyscript10.7 KB
tests/test_plugin_checkout_provenance.pyscript11.1 KB
tests/test_powershell_hook_wire.pyscript10.6 KB
tests/test_powershell_verdict_corpus.pyscript6.2 KB
tests/test_pr44_hardening.pyscript6.5 KB
tests/test_pre_scan.pyscript19.6 KB
tests/test_pre_scan_composer.pyscript7.4 KB
tests/test_precondition_fields.pyscript4.6 KB
tests/test_provenance_offline.pyscript2.5 KB
tests/test_published_rulepack_feed_current.pyscript1.4 KB
tests/test_py38_compat.pyscript3.3 KB
tests/test_pypi_pinned_versions.pyscript19.3 KB
tests/test_pyproject_string_forms.pyscript2.2 KB
tests/test_redteam_fn_regression.pyscript32.5 KB
tests/test_reflective_fetch_execute.pyscript23.5 KB
tests/test_reflective_review_fixes.pyscript17.8 KB
tests/test_refresh_controller.pyscript19.9 KB
tests/test_refresh_rulepacks.pyscript12.4 KB
tests/test_review_gate_regressions.pyscript9.3 KB
tests/test_rule_loader.pyscript23.2 KB
tests/test_rulepack_feed.pyscript38.1 KB
tests/test_run_forensics_json.pyscript1012 B
tests/test_run_scanner_fail_loud.pyscript6.5 KB
tests/test_runtime_floors.pyscript19.8 KB
tests/test_sandbox_probing_rules.pyscript9.6 KB
tests/test_sarif_export.pyscript19.9 KB
tests/test_scan_agent_skills.pyscript31.3 KB
tests/test_scan_archive.pyscript11.6 KB
tests/test_scan_ast.pyscript12.8 KB
tests/test_scan_binary.pyscript15.3 KB
tests/test_scan_bytecode.pyscript21.5 KB
tests/test_scan_dast.pyscript32.1 KB
tests/test_scan_dataflow.pyscript7.1 KB
tests/test_scan_dead_anchors.pyscript11.5 KB
tests/test_scan_decode.pyscript28.0 KB
tests/test_scan_dependencies.pyscript37.8 KB
tests/test_scan_devcontainer.pyscript8.5 KB
tests/test_scan_entropy.pyscript11.2 KB
tests/test_scan_entrypoint.pyscript19.5 KB
tests/test_scan_git_config.pyscript58.5 KB
tests/test_scan_git_config_review_fixes.pyscript33.3 KB

scripts/gen_rule_ids.py

View on GitHub
#!/usr/bin/env python3
"""
gen_rule_ids.py - Dev helper: mint canonical rule ids for the extraction work.

The U3/U4/U5 extraction units move ~500 hardcoded patterns into JSON rule packs.
Those run in parallel, so they need a single collision-free id namespace agreed
up front. This script reads the existing scanner source with `ast` (no imports,
no execution of scanner code), finds the module-level pattern TABLES (lists /
dicts whose names look like `*_PATTERNS`, `*_KEYWORDS`, `HOMOGLYPHS`, etc.),
counts the entries in each, and emits a canonical

    <SCANNER>-<CATEGORY>-<NNN>

id map to a committed CSV. Extractors (U3-U5) assign ids straight from this CSV,
so two parallel branches never collide and the parity harness can assert every
pack-driven finding carries a non-empty rule_id.

Pure stdlib, dependency-free, side-effect-free (it never imports the scanners —
it parses them as text via ast, so a hostile pattern table can't run).

Usage:
    python3 gen_rule_ids.py                 # writes data/rule_ids.csv
    python3 gen_rule_ids.py --out path.csv  # custom output
    python3 gen_rule_ids.py --print         # stdout only, no file write

CSV columns:
    rule_id, scanner, category, source_file, table_name, index, severity_hint

`index` is the 0-based position of the entry inside its source table, so an
extractor can map "the 3rd item of PROMPT_INJECTION_PATTERNS" to its id
deterministically.
"""

import os
import sys
import ast
import csv

_SCRIPTS_DIR = os.path.dirname(os.path.realpath(__file__))
_DATA_DIR = os.path.normpath(os.path.join(_SCRIPTS_DIR, "..", "data"))
_DEFAULT_OUT = os.path.join(_DATA_DIR, "rule_ids.csv")

# Per-scanner file -> abbreviation used in rule ids. Mirrors the
# <SCANNER> half of the id convention (e.g. ST = skill_threats).
_SCANNER_ABBREV = {
    "scan_secrets.py": "SC",
    "scan_sast.py": "SA",
    "scan_skill_threats.py": "ST",
    "scan_mcp_security.py": "MC",
    "scan_agent_skills.py": "AS",
    "scan_runtime_dynamism.py": "RD",
    "scan_infra.py": "IN",
    "scan_lifecycle.py": "LC",
    "scan_dependencies.py": "DP",
    "scan_devcontainer.py": "DC",
    "_shared_patterns.py": "SH",
    # dead_anchors authors all its rule ids directly in
    # data/rulepacks/dead_anchors.json (no module-level *_PATTERNS tables to
    # scrape), so this abbrev emits ZERO rows in rule_ids.csv — registration for
    # completeness only. Expected, not a bug.
    "scan_dead_anchors.py": "DA",
    # yara authors all its rule ids directly in data/yara/manifest.json
    # (rule_meta[].id), so this abbrev emits ZERO scraped rows — registration
    # for namespace completeness only. The minted YR-* ids are appended to
    # rule_ids.csv manually. Expected, not a bug.
    "scan_yara.py": "YR",
    # git_config authors its rule ids as module constants (R_*) with no
    # *_PATTERNS tables to scrape, so this abbrev emits ZERO scraped rows —
    # registration for namespace completeness only, so a future rule_ids.csv
    # regen does not silently drop the 8 GC rows. The GC-* ids are appended
    # to rule_ids.csv manually. Expected, not a bug.
    "scan_git_config.py": "GC",
}

# Table-name fragment -> CATEGORY abbreviation. First matching fragment wins.
# Extend as extraction proceeds; unknown tables fall back to "GEN".
_CATEGORY_RULES = [
    ("PROMPT_INJECTION", "PI"),
    ("INJECTION", "PI"),
    ("HOMOGLYPH", "HG"),
    ("EXFIL", "EX"),
    ("CREDENTIAL", "CR"),
    ("PERSISTENCE", "PE"),
    ("SCOPE", "SP"),
    ("STEALTH", "ST"),
    ("CLICKFIX", "CF"),
    ("PREREQUISITE", "PR"),
    ("TOOL_INJECTION", "TI"),
    ("SUB_AGENT", "SA"),
    ("AUTHORITY", "AU"),
    ("SAFETY_THEATER", "ST"),
    ("TRUST_ESCALATION", "TE"),
    ("UPDATE_CHANNEL", "UC"),
    ("RAT_BINARY", "RB"),
    ("MALICIOUS_AUTHOR", "MA"),
    ("SAST", "SAST"),
    ("PATTERNS", "GEN"),
    ("KEYWORDS", "KW"),
]

# Table names to ignore: IOC fallbacks (live-fed, not rules-as-data) and
# anything that is clearly not a behavioral-pattern table.
_SKIP_FRAGMENTS = ("_FALLBACK_", "_IPS", "_DOMAINS", "MAP")

# Table-name suffixes / shapes we treat as pattern tables.
_TABLE_SUFFIXES = ("_PATTERNS", "_KEYWORDS", "_PATHS")
_TABLE_EXACT = {"HOMOGLYPHS"}


def _category_for(table_name):
    for frag, abbrev in _CATEGORY_RULES:
        if frag in table_name:
            return abbrev
    return "GEN"


def _is_pattern_table(name):
    if any(skip in name for skip in _SKIP_FRAGMENTS):
        return False
    if name in _TABLE_EXACT:
        return True
    return any(name.endswith(suf) for suf in _TABLE_SUFFIXES)


def _count_entries(node):
    """Return the number of top-level entries in a List/Tuple/Dict/Set literal,
    or None if the value is not a simple collection literal."""
    if isinstance(node, (ast.List, ast.Tuple, ast.Set)):
        return len(node.elts)
    if isinstance(node, ast.Dict):
        return len(node.keys)
    return None


def _severity_hint(node, index):
    """Best-effort severity extraction from a dict-shaped entry, else ''."""
    if not isinstance(node, (ast.List, ast.Tuple)):
        return ""
    if index >= len(node.elts):
        return ""
    elt = node.elts[index]
    if isinstance(elt, ast.Dict):
        for k, v in zip(elt.keys, elt.values):
            if isinstance(k, ast.Constant) and k.value == "severity" \
                    and isinstance(v, ast.Constant) and isinstance(v.value, str):
                return v.value
    return ""


def collect_tables(scripts_dir=None):
    """Parse every known scanner and yield (scanner_file, table_name, node).

    Reads source as text and parses with ast — never imports/executes it.
    """
    scripts_dir = scripts_dir or _SCRIPTS_DIR
    for fname in sorted(_SCANNER_ABBREV):
        path = os.path.join(scripts_dir, fname)
        if not os.path.isfile(path):
            continue
        with open(path, "r", encoding="utf-8") as f:
            src = f.read()
        try:
            tree = ast.parse(src, filename=path)
        except SyntaxError:
            print(f"[gen_rule_ids] skip (parse error): {fname}", file=sys.stderr)
            continue
        for node in tree.body:
            if not isinstance(node, ast.Assign):
                continue
            for target in node.targets:
                if not isinstance(target, ast.Name):
                    continue
                if not _is_pattern_table(target.id):
                    continue
                yield fname, target.id, node.value


def build_id_map(scripts_dir=None):
    """Build the full list of rule-id rows.

    Returns list[dict] with keys: rule_id, scanner, category, source_file,
    table_name, index, severity_hint. Sequence numbers (<NNN>) are assigned
    per (scanner, category) in stable file+table+index order.
    """
    rows = []
    seq = {}  # (scanner_abbrev, category) -> next int
    for fname, table_name, node in collect_tables(scripts_dir):
        count = _count_entries(node)
        if not count:
            continue
        scanner = _SCANNER_ABBREV[fname]
        category = _category_for(table_name)
        for i in range(count):
            key = (scanner, category)
            n = seq.get(key, 0) + 1
            seq[key] = n
            rule_id = f"{scanner}-{category}-{n:03d}"
            rows.append({
                "rule_id": rule_id,
                "scanner": scanner,
                "category": category,
                "source_file": fname,
                "table_name": table_name,
                "index": i,
                "severity_hint": _severity_hint(node, i),
            })
    return rows


_FIELDS = ["rule_id", "scanner", "category", "source_file",
           "table_name", "index", "severity_hint"]


def write_csv(rows, out_path):
    """Write rows to CSV at out_path (creates parent dir if needed)."""
    os.makedirs(os.path.dirname(os.path.abspath(out_path)), exist_ok=True)
    with open(out_path, "w", encoding="utf-8", newline="") as f:
        writer = csv.DictWriter(f, fieldnames=_FIELDS)
        writer.writeheader()
        writer.writerows(rows)


def main(argv=None):
    argv = argv if argv is not None else sys.argv[1:]
    out = _DEFAULT_OUT
    print_only = False
    i = 0
    while i < len(argv):
        if argv[i] == "--out" and i + 1 < len(argv):
            out = argv[i + 1]
            i += 2
        elif argv[i] == "--print":
            print_only = True
            i += 1
        else:
            print(f"[gen_rule_ids] unknown arg: {argv[i]}", file=sys.stderr)
            return 2
    rows = build_id_map()
    if print_only:
        w = csv.DictWriter(sys.stdout, fieldnames=_FIELDS)
        w.writeheader()
        w.writerows(rows)
    else:
        write_csv(rows, out)
        print(f"[gen_rule_ids] wrote {len(rows)} rule ids -> {out}")
    return 0


if __name__ == "__main__":
    sys.exit(main())
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall