Skip to content

hint-shu/deep-research / skills/osint-research

osint-research

Unverified

OSINT recon skill — passive entity discovery for domain / IP / email / person / company / github user. Builds a hybrid findings/dossier/graph report from free public sources (Whois, DNS, crt.sh, Wayback, Shodan InternetDB, GitHub code search, Tavily/Firecrawl/Exa/Perplexity dorks, optional theHarvester/subfinder). No subscription dependencies. Disclaimer is non-removable; secrets are redacted; blocklisted dump sites are filtered at outbound and inbound levels.

FileKindSize
SKILL.mdskill md6.1 KB
channels/crtsh.shexecutablescript1.8 KB
channels/dorks.shexecutablescript3.5 KB
channels/github-leaks.shexecutablescript2.0 KB
channels/shodan-idb.shexecutablescript2.2 KB
channels/subfinder-wrap.shscript1.8 KB
channels/theharvester-wrap.shexecutablescript1.8 KB
channels/wayback.shexecutablescript1.5 KB
channels/whois-dns.shexecutablescript2.2 KB
lib/entity-classifier.shexecutablescript1.4 KB
lib/findings-extractor.shexecutablescript1.6 KB
lib/inbound-filter.shscript6.9 KB
lib/secret-patterns.txtmarkdown1.1 KB
lib/secret-redactor.shexecutablescript2.3 KB
templates/graph.mmd.tplother53 B
templates/osint-report.md.tplother1.1 KB

channels/crtsh.sh

View on GitHub
#!/usr/bin/env bash
# skills/osint-research/channels/crtsh.sh
# Queries crt.sh for Certificate Transparency entries → unique subdomains.

set -uo pipefail

ENTITY_TYPE=""
TARGET=""
FIXTURE=""
while [ $# -gt 0 ]; do
    case "$1" in
        --type) ENTITY_TYPE="$2"; shift 2;;
        --target) TARGET="$2"; shift 2;;
        --fixture) FIXTURE="$2"; shift 2;;
        *) echo "crtsh: unknown flag: $1" >&2; exit 1;;
    esac
done

if [ -z "$TARGET" ]; then
    echo "crtsh: empty target" >&2
    exit 1
fi

case "$ENTITY_TYPE" in
    domain|company) ;;
    *) echo "crtsh: not applicable for $ENTITY_TYPE" >&2; exit 0;;
esac

if [ -n "$FIXTURE" ]; then
    json=$(cat "$FIXTURE")
else
    json=$(curl -fsSL --max-time 30 "https://crt.sh/?q=%25.${TARGET}&output=json" 2>/dev/null || echo "[]")
fi

# Extract unique subdomain names. name_value can contain newline-separated multi-SAN entries.
# Write python script to a temp file so we can feed json via stdin (heredoc conflicts with pipe).
_py_tmp=$(mktemp /tmp/crtsh_XXXXXX.py)
trap 'rm -f "$_py_tmp"' EXIT
cat > "$_py_tmp" <<'PY'
import json, sys
target = sys.argv[1]
try:
    data = json.loads(sys.stdin.read())
except Exception:
    sys.exit(0)
seen = set()
for entry in data:
    nv = entry.get("name_value", "")
    for name in nv.split("\n"):
        name = name.strip().lstrip("*.")
        if name and name not in seen and (name == target or name.endswith('.' + target)):
            seen.add(name)
            url = f"https://crt.sh/?q=%25.{target}"
            out = {
                "channel": "crtsh",
                "record_type": "subdomain",
                "url": url,
                "content": name,
                "metadata": {"target": target}
            }
            print(json.dumps(out, separators=(',', ':')))
PY
printf '%s' "$json" | python3 "$_py_tmp" "$TARGET"
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 196kUpdated Claude CodeCodexVerifiedInstall