Skip to content

alexgreensh/repo-forensics / skills/repo-forensics

repo-forensics

Flagged

Security forensics for git repos, AI skills, and MCP servers. Audits dependencies, detects prompt injection, credential theft, runtime dynamism, manifest drift, known CVEs, CISA KEV (actively exploited) vulns, and 2026 attack patterns. Not for fixing vulnerabilities or pentesting.

FileKindSize
.gitignoreother193 B
SKILL.mdskill md43.4 KB
checksums.jsonconfig10.8 KB
checksums.json.sigother64 B
data/README.mdmarkdown1.3 KB
data/compromised_versions.jsonconfig188.3 KB
data/rule_ids.csvother35.3 KB
data/rulepacks/.gitkeepother221 B
data/rulepacks/dead_anchors.jsonconfig9.0 KB
data/rulepacks/mcp_security.jsonconfig31.2 KB
data/rulepacks/runtime_dynamism.jsonconfig46.7 KB
data/rulepacks/sast.jsonconfig107.6 KB
data/rulepacks/secrets.jsonconfig30.4 KB
data/rulepacks/shared.jsonconfig4.4 KB
data/rulepacks/skill_threats.jsonconfig95.4 KB
data/runtime_floors.jsonconfig2.3 KB
data/yara/cryptominers.yarother1.8 KB
data/yara/hacktools.yarother2.2 KB
data/yara/malware.yarother2.2 KB
data/yara/manifest.jsonconfig8.0 KB
data/yara/webshells.yarother2.5 KB
references/cursor-hook-contract.mdmarkdown5.6 KB
references/kernel_privesc_sandbox_pack.mdmarkdown5.0 KB
references/mcp-attack-patterns.mdmarkdown8.7 KB
references/research_sources.mdmarkdown8.0 KB
references/threat_patterns.mdmarkdown20.3 KB
scripts/_context_gate.pyscript21.7 KB
scripts/_ed25519.pyscript5.1 KB
scripts/_pyc_unmarshal.pyscript17.1 KB
scripts/_shared_patterns.pyscript4.8 KB
scripts/adjudication.pyscript16.4 KB
scripts/adjudication_bridge.pyscript1.0 KB
scripts/aggregate_json.pyscript69.2 KB
scripts/auto_scan.pyexecutablescript152.1 KB
scripts/corpus_sync.pyscript10.0 KB
scripts/cursor_pre_scan.pyscript22.5 KB
scripts/dast_sandbox.sbother1.6 KB
scripts/dead_anchors_extract.pyscript14.8 KB
scripts/dead_anchors_probe.pyscript27.1 KB
scripts/forensics_core.pyscript176.5 KB
scripts/gen_rule_ids.pyscript8.6 KB
scripts/hook_adapter.pyscript19.4 KB
scripts/ioc_manager.pyscript49.6 KB
scripts/parse_pnpm_lock.pyscript7.9 KB
scripts/pre_scan.pyscript143.5 KB
scripts/pypi_pins.pyscript8.1 KB
scripts/refresh_controller.pyexecutablescript48.4 KB
scripts/refresh_threat_dbs.pyexecutablescript19.4 KB
scripts/rule_loader.pyscript44.8 KB
scripts/rulepack_feed.pyscript26.4 KB
scripts/run_forensics.shexecutablescript20.3 KB
scripts/sarif_export.pyscript18.6 KB
scripts/scan_agent_skills.pyscript33.2 KB
scripts/scan_archive.pyscript29.1 KB
scripts/scan_ast.pyscript44.9 KB
scripts/scan_binary.pyscript10.9 KB
scripts/scan_bytecode.pyscript28.2 KB
scripts/scan_dast.pyscript28.8 KB
scripts/scan_dataflow.pyscript15.4 KB
scripts/scan_dead_anchors.pyscript15.9 KB
scripts/scan_decode.pyscript37.5 KB
scripts/scan_dependencies.pyscript124.5 KB
scripts/scan_devcontainer.pyscript8.5 KB
scripts/scan_entropy.pyscript15.6 KB
scripts/scan_entrypoint.pyscript34.3 KB
scripts/scan_git_config.pyscript96.3 KB
scripts/scan_git_forensics.pyscript24.1 KB
scripts/scan_history.pyscript14.5 KB
scripts/scan_infra.pyscript54.3 KB
scripts/scan_integrity.pyscript41.0 KB
scripts/scan_lifecycle.pyscript65.1 KB
scripts/scan_manifest_drift.pyscript47.0 KB
scripts/scan_mcp_security.pyscript42.3 KB
scripts/scan_oversize.pyscript11.7 KB
scripts/scan_post_incident.pyscript36.9 KB
scripts/scan_provenance.pyscript19.4 KB
scripts/scan_runtime_dynamism.pyscript23.5 KB
scripts/scan_runtime_floors.pyscript33.5 KB
scripts/scan_sast.pyscript59.1 KB
scripts/scan_secrets.pyscript11.9 KB
scripts/scan_skill_threats.pyscript102.4 KB
scripts/scan_splitstream.pyscript29.5 KB
scripts/scan_yara.pyscript19.7 KB
scripts/session_scan.pyscript54.1 KB
scripts/suppressions.pyscript15.4 KB
scripts/validate_manifests.pyscript13.7 KB
scripts/verify_install.pyscript26.2 KB
scripts/vuln_feed.pyscript31.1 KB
tests/conftest.pyscript13.4 KB
tests/corpus/DEAD_ANCHORS_CORPUS.mdmarkdown2.3 KB
tests/corpus/budgets.jsonconfig5.6 KB
tests/cursor_helpers.pyscript12.6 KB
tests/fixtures/legacy_main_blocks.jsonconfig10.3 KB
tests/fixtures/powershell_verdict_corpus.jsonconfig114.9 KB
tests/golden/parity_mcp_security.jsonconfig17.5 KB
tests/golden/parity_runtime_dynamism.jsonconfig31.7 KB
tests/golden/parity_sast.jsonconfig59.8 KB
tests/golden/parity_secrets.jsonconfig18.8 KB
tests/golden/parity_skill_threats.jsonconfig48.8 KB
tests/parity_corpus.pyscript6.3 KB
tests/powershell_hook.pyscript8.2 KB
tests/report_builders.pyscript5.0 KB
tests/schemas/sarif-schema-2.1.0.jsonconfig109.1 KB
tests/shell_compat.pyscript2.2 KB
tests/test_adjudication_bridge.pyscript832 B
tests/test_adjudication_containment.pyscript1.4 KB
tests/test_adjudication_output.pyscript20.0 KB
tests/test_adjudication_suppressions.pyscript18.7 KB
tests/test_advisory_adjudication.pyscript1.3 KB
tests/test_agent_relay_rules.pyscript7.8 KB
tests/test_aggregate_json.pyscript10.0 KB
tests/test_auto_scan_patterns.pyscript10.5 KB
tests/test_benign_corpus.pyscript16.4 KB
tests/test_benign_new_scanners.pyscript6.1 KB
tests/test_build_output_skip.pyscript10.6 KB
tests/test_capability_ledger.pyscript18.8 KB
tests/test_codex_install.pyscript10.0 KB
tests/test_codex_marketplace.pyscript3.3 KB
tests/test_comment_false_positives.pyscript2.2 KB
tests/test_composer_dependencies.pyscript4.0 KB
tests/test_compromised_versions.pyscript51.2 KB
tests/test_context_gate.pyscript32.4 KB
tests/test_coverage_status.pyscript7.6 KB
tests/test_css_compounding_shrink.pyscript20.8 KB
tests/test_cursor_adapter.pyscript7.8 KB
tests/test_cursor_benign.pyscript3.4 KB
tests/test_cursor_degrade.pyscript12.7 KB
tests/test_cursor_install.pyscript17.1 KB
tests/test_cursor_latency.pyscript7.2 KB
tests/test_cursor_parity.pyscript5.8 KB
tests/test_cursor_redteam_fn.pyscript4.9 KB
tests/test_cursor_tamper_modify.pyscript14.2 KB
tests/test_cursor_wiring.pyscript32.3 KB
tests/test_d1_yara_correlation.pyscript9.2 KB
tests/test_dead_anchors_extract.pyscript7.6 KB
tests/test_dead_anchors_probe.pyscript19.7 KB
tests/test_evidence_engine_parity.pyscript8.1 KB
tests/test_evidence_model.pyscript11.4 KB
tests/test_forensics_core.pyscript79.2 KB
tests/test_fp_hardening_r3.pyscript9.7 KB
tests/test_fp_hardening_r4.pyscript18.8 KB
tests/test_fp_regression_pairs.pyscript62.5 KB
tests/test_fp_structural.pyscript21.2 KB
tests/test_freshness.pyscript6.2 KB
tests/test_frontmatter_validity.pyscript7.0 KB
tests/test_gpg_rce_hotfix.pyscript4.8 KB
tests/test_hook_inert_data.pyscript6.1 KB
tests/test_hook_scripts.pyscript14.0 KB
tests/test_inspection_cpu_budget.pyscript11.6 KB
tests/test_ioc_manager.pyscript55.4 KB
tests/test_kernel_privesc_rules.pyscript27.8 KB
tests/test_legit_precision_regressions.pyscript6.8 KB
tests/test_lethal_trifecta.pyscript20.7 KB
tests/test_manifest_signature.pyscript1.5 KB
tests/test_manifest_validity.pyscript2.9 KB
tests/test_monotonic_trust_e2e.pyscript4.4 KB
tests/test_non_breaking_contract.pyscript22.7 KB
tests/test_openclaw_install.pyscript6.6 KB
tests/test_pack_parity.pyscript24.8 KB
tests/test_parse_pnpm_lock.pyscript10.7 KB
tests/test_plugin_checkout_provenance.pyscript11.1 KB
tests/test_powershell_hook_wire.pyscript10.6 KB
tests/test_powershell_verdict_corpus.pyscript6.2 KB
tests/test_pr44_hardening.pyscript6.5 KB
tests/test_pre_scan.pyscript19.6 KB
tests/test_pre_scan_composer.pyscript7.4 KB
tests/test_precondition_fields.pyscript4.6 KB
tests/test_provenance_offline.pyscript2.5 KB
tests/test_published_rulepack_feed_current.pyscript1.4 KB
tests/test_py38_compat.pyscript3.3 KB
tests/test_pypi_pinned_versions.pyscript19.3 KB
tests/test_pyproject_string_forms.pyscript2.2 KB
tests/test_redteam_fn_regression.pyscript32.5 KB
tests/test_reflective_fetch_execute.pyscript23.5 KB
tests/test_reflective_review_fixes.pyscript17.8 KB
tests/test_refresh_controller.pyscript19.9 KB
tests/test_refresh_rulepacks.pyscript12.4 KB
tests/test_review_gate_regressions.pyscript9.3 KB
tests/test_rule_loader.pyscript23.2 KB
tests/test_rulepack_feed.pyscript38.1 KB
tests/test_run_forensics_json.pyscript1012 B
tests/test_run_scanner_fail_loud.pyscript6.5 KB
tests/test_runtime_floors.pyscript19.8 KB
tests/test_sandbox_probing_rules.pyscript9.6 KB
tests/test_sarif_export.pyscript19.9 KB
tests/test_scan_agent_skills.pyscript31.3 KB
tests/test_scan_archive.pyscript11.6 KB
tests/test_scan_ast.pyscript12.8 KB
tests/test_scan_binary.pyscript15.3 KB
tests/test_scan_bytecode.pyscript21.5 KB
tests/test_scan_dast.pyscript32.1 KB
tests/test_scan_dataflow.pyscript7.1 KB
tests/test_scan_dead_anchors.pyscript11.5 KB
tests/test_scan_decode.pyscript28.0 KB
tests/test_scan_dependencies.pyscript37.8 KB
tests/test_scan_devcontainer.pyscript8.5 KB
tests/test_scan_entropy.pyscript11.2 KB
tests/test_scan_entrypoint.pyscript19.5 KB
tests/test_scan_git_config.pyscript58.5 KB
tests/test_scan_git_config_review_fixes.pyscript33.3 KB

tests/test_aggregate_json.py

View on GitHub
"""Unit tests for aggregate_json.py."""

import json
import os

import aggregate_json as module


def _write(path, content):
    os.makedirs(os.path.dirname(path), exist_ok=True)
    with open(path, "w", encoding="utf-8") as handle:
        handle.write(content)


def test_parse_scanner_payload_empty_output():
    findings, error = module.parse_scanner_payload("")
    assert findings == []
    assert error is None


def test_parse_scanner_payload_valid_json_list():
    raw = json.dumps([{"severity": "high", "title": "A"}], indent=2)
    findings, error = module.parse_scanner_payload(raw)
    assert error is None
    assert findings[0]["title"] == "A"


def test_parse_scanner_payload_with_leading_banner_fails():
    raw = "[*] Scanning repo...\n" + json.dumps([{"severity": "high", "title": "A"}], indent=2)
    findings, error = module.parse_scanner_payload(raw)
    assert findings == []
    assert "Invalid JSON output" in error


def test_parse_scanner_payload_malformed_json():
    findings, error = module.parse_scanner_payload("{not json")
    assert findings == []
    assert "Invalid JSON output" in error


def test_build_report_multiple_scanners_with_mixed_results(tmp_path):
    _write(tmp_path / "a.out", json.dumps([{"severity": "high", "title": "A"}]))
    _write(tmp_path / "a.err", "")
    _write(tmp_path / "a.exit", "1")

    _write(tmp_path / "b.out", json.dumps([{"severity": "critical", "title": "B"}]))
    _write(tmp_path / "b.err", "debug warning")
    _write(tmp_path / "b.exit", "2")

    report = module.build_report(str(tmp_path), "/repo", "false")

    assert report["mode"] == "full"
    assert report["scanner_count"] == 2
    assert report["summary"]["critical"] == 1
    assert report["summary"]["high"] == 1
    assert report["exit_code"] == 2
    assert report["findings"][0]["severity"] == "critical"
    assert report["scanners"][1]["stderr"] == "debug warning"


def test_trifecta_raw_leaf_is_internal_only(tmp_path):
    repo = tmp_path / "repo"
    results = tmp_path / "results"
    repo.mkdir()
    results.mkdir()
    (repo / "build.py").write_text("import subprocess\nsubprocess.run(['make'])\n")

    report = module.build_report(str(results), str(repo), "false")

    assert not any(f.get("scanner") == "trifecta_raw" for f in report["findings"])
    assert not any(s.get("name") == "trifecta_raw" for s in report["scanners"])
    assert report["summary"]["total"] == 0


def test_build_report_malformed_scanner_output_surfaces_parse_error(tmp_path):
    _write(tmp_path / "bad.out", "{oops")
    _write(tmp_path / "bad.err", "")
    _write(tmp_path / "bad.exit", "1")

    report = module.build_report(str(tmp_path), "/repo", "true")

    assert report["mode"] == "skill"
    assert report["summary"]["total"] == 0
    assert report["exit_code"] == 99
    assert report["scanners"][0]["parse_error"] == "Invalid JSON output: Expecting property name enclosed in double quotes"


def test_build_report_missing_output_with_nonzero_exit_fails_closed(tmp_path):
    _write(tmp_path / "bad.out", "")
    _write(tmp_path / "bad.err", "Traceback")
    _write(tmp_path / "bad.exit", "1")

    report = module.build_report(str(tmp_path), "/repo", "false")

    assert report["exit_code"] == 99
    assert report["scanners"][0]["parse_error"] == "No JSON output captured from scanner"


# ---------------------------------------------------------------------------
# U1: verdict tiers
# ---------------------------------------------------------------------------


def test_verdict_tier_thresholds():
    assert module.verdict_tier(0.95) == "block"
    assert module.verdict_tier(0.92) == "block"
    assert module.verdict_tier(0.75) == "warn"
    assert module.verdict_tier(0.60) == "warn"
    assert module.verdict_tier(0.45) == "info"
    assert module.verdict_tier(0.30) == "info"
    assert module.verdict_tier(0.20) == "suppressed"


def test_verdict_tier_user_suppressed_overrides():
    assert module.verdict_tier(0.99, suppressed=True) == "suppressed"


def test_build_verdicts_counts_by_tier():
    active = [
        {"severity": "critical", "confidence": 0.95},
        {"severity": "high", "confidence": 0.75},
        {"severity": "medium", "confidence": 0.45},
        {"severity": "low", "confidence": 0.20},
    ]
    verdicts = module.build_verdicts(active, [])
    assert verdicts == {"block": 1, "warn": 1, "info": 1, "suppressed": 1}


def test_build_verdicts_includes_user_suppressed():
    active = [{"severity": "critical", "confidence": 0.95}]
    suppressed = [{"severity": "high", "confidence": 0.80}, {"severity": "low"}]
    verdicts = module.build_verdicts(active, suppressed)
    assert verdicts["block"] == 1
    assert verdicts["suppressed"] == 2


def test_finding_confidence_fills_from_severity():
    assert module._finding_confidence({"severity": "critical"}) == 0.95
    assert module._finding_confidence({"severity": "low", "confidence": 0.0}) == 0.40
    assert module._finding_confidence({"severity": "low", "confidence": 0.7}) == 0.7


def test_report_has_verdicts_and_suppressed_keys(tmp_path):
    _write(tmp_path / "a.out", json.dumps([
        {"scanner": "secrets", "severity": "critical", "title": "Key",
         "file": "x.py", "line": 1, "confidence": 0.95},
    ]))
    _write(tmp_path / "a.err", "")
    _write(tmp_path / "a.exit", "2")
    report = module.build_report(str(tmp_path), str(tmp_path), "false")
    assert "verdicts" in report
    assert "suppressed" in report
    assert report["verdicts"]["block"] >= 1
    assert report["suppressed"] == []


# ---------------------------------------------------------------------------
# U1: per-finding suppression + abuse guards
# ---------------------------------------------------------------------------


def _scan_dir(tmp_path):
    """A separate empty repo dir used as the scan target (no trifecta noise)."""
    d = tmp_path / "repo"
    d.mkdir()
    return d


def test_suppression_excludes_from_summary_and_exit_but_keeps_audit(tmp_path):
    repo = _scan_dir(tmp_path)
    (repo / ".forensicsignore").write_text("rule:SC-KEY-001:src/**\n")
    _write(tmp_path / "a.out", json.dumps([
        {"scanner": "secrets", "severity": "high", "title": "Suppress me",
         "file": "src/app.py", "line": 1, "rule_id": "SC-KEY-001", "confidence": 0.80},
        {"scanner": "secrets", "severity": "critical", "title": "Stays",
         "file": "src/db.py", "line": 2, "rule_id": "SC-CRIT-001", "confidence": 0.95},
    ]))
    _write(tmp_path / "a.err", "")
    _write(tmp_path / "a.exit", "2")
    report = module.build_report(str(tmp_path), str(repo), "false")

    # Suppressed finding gone from summary + active findings
    assert report["summary"]["high"] == 0
    suppressed_titles = [f["title"] for f in report["suppressed"]]
    assert "Suppress me" in suppressed_titles
    active_titles = [f["title"] for f in report["findings"]]
    assert "Suppress me" not in active_titles
    # Critical still fires => exit 2
    assert "Stays" in active_titles
    assert report["exit_code"] == 2
    assert report["summary"]["critical"] == 1


def test_suppression_glob_scopes_to_path(tmp_path):
    repo = _scan_dir(tmp_path)
    (repo / ".forensicsignore").write_text("rule:SC-KEY-001:tests/**\n")
    _write(tmp_path / "a.out", json.dumps([
        {"scanner": "secrets", "severity": "low", "title": "In tests",
         "file": "tests/fixture.py", "line": 1, "rule_id": "SC-KEY-001", "confidence": 0.40},
        {"scanner": "secrets", "severity": "low", "title": "In src",
         "file": "src/app.py", "line": 1, "rule_id": "SC-KEY-001", "confidence": 0.40},
    ]))
    _write(tmp_path / "a.err", "")
    _write(tmp_path / "a.exit", "1")
    report = module.build_report(str(tmp_path), str(repo), "false")

    suppressed_titles = [f["title"] for f in report["suppressed"]]
    active_titles = [f["title"] for f in report["findings"]]
    assert "In tests" in suppressed_titles
    assert "In src" in active_titles


def test_suppressing_critical_rule_raises_tampering_finding(tmp_path):
    repo = _scan_dir(tmp_path)
    (repo / ".forensicsignore").write_text("rule:SC-CRIT-001\n")
    _write(tmp_path / "a.out", json.dumps([
        {"scanner": "secrets", "severity": "critical", "title": "Critical thing",
         "file": "src/db.py", "line": 2, "rule_id": "SC-CRIT-001", "confidence": 0.95},
    ]))
    _write(tmp_path / "a.err", "")
    _write(tmp_path / "a.exit", "2")
    report = module.build_report(str(tmp_path), str(repo), "false")

    titles = [f["title"] for f in report["findings"]]
    assert ".forensicsignore: Critical Rule Suppression" in titles
    tamper = [f for f in report["findings"]
              if f["title"] == ".forensicsignore: Critical Rule Suppression"][0]
    assert tamper["severity"] == "critical"
    # The original critical finding is suppressed; the tampering guard keeps exit 2
    assert report["exit_code"] == 2


def test_mass_suppression_raises_high_finding(tmp_path):
    repo = _scan_dir(tmp_path)
    lines = "".join(f"rule:R-LOW-{i:03d}\n" for i in range(1, 7))  # 6 rule: lines
    (repo / ".forensicsignore").write_text(lines)
    _write(tmp_path / "a.out", json.dumps([
        {"scanner": "secrets", "severity": "low", "title": "noise",
         "file": "src/a.py", "line": 1, "rule_id": "R-LOW-001", "confidence": 0.40},
    ]))
    _write(tmp_path / "a.err", "")
    _write(tmp_path / "a.exit", "1")
    report = module.build_report(str(tmp_path), str(repo), "false")

    titles = [f["title"] for f in report["findings"]]
    assert ".forensicsignore: Mass Rule Suppression" in titles
    mass = [f for f in report["findings"]
            if f["title"] == ".forensicsignore: Mass Rule Suppression"][0]
    assert mass["severity"] == "high"


def test_no_suppression_file_no_suppressed_findings(tmp_path):
    repo = _scan_dir(tmp_path)
    _write(tmp_path / "a.out", json.dumps([
        {"scanner": "secrets", "severity": "high", "title": "x",
         "file": "src/a.py", "line": 1, "rule_id": "SC-KEY-001", "confidence": 0.80},
    ]))
    _write(tmp_path / "a.err", "")
    _write(tmp_path / "a.exit", "1")
    report = module.build_report(str(tmp_path), str(repo), "false")
    assert report["suppressed"] == []
    assert report["summary"]["high"] == 1
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall