Skip to content

hint-shu/deep-research / skills/osint-research

osint-research

Unverified

OSINT recon skill — passive entity discovery for domain / IP / email / person / company / github user. Builds a hybrid findings/dossier/graph report from free public sources (Whois, DNS, crt.sh, Wayback, Shodan InternetDB, GitHub code search, Tavily/Firecrawl/Exa/Perplexity dorks, optional theHarvester/subfinder). No subscription dependencies. Disclaimer is non-removable; secrets are redacted; blocklisted dump sites are filtered at outbound and inbound levels.

FileKindSize
SKILL.mdskill md6.1 KB
channels/crtsh.shexecutablescript1.8 KB
channels/dorks.shexecutablescript3.5 KB
channels/github-leaks.shexecutablescript2.0 KB
channels/shodan-idb.shexecutablescript2.2 KB
channels/subfinder-wrap.shscript1.8 KB
channels/theharvester-wrap.shexecutablescript1.8 KB
channels/wayback.shexecutablescript1.5 KB
channels/whois-dns.shexecutablescript2.2 KB
lib/entity-classifier.shexecutablescript1.4 KB
lib/findings-extractor.shexecutablescript1.6 KB
lib/inbound-filter.shscript6.9 KB
lib/secret-patterns.txtmarkdown1.1 KB
lib/secret-redactor.shexecutablescript2.3 KB
templates/graph.mmd.tplother53 B
templates/osint-report.md.tplother1.1 KB

channels/shodan-idb.sh

View on GitHub
#!/usr/bin/env bash
# skills/osint-research/channels/shodan-idb.sh
# Queries free InternetDB endpoint. No auth required.

set -uo pipefail

TARGET=""
FIXTURE=""
FIXTURE_STATUS=200
while [ $# -gt 0 ]; do
    case "$1" in
        --target) TARGET="$2"; shift 2;;
        --fixture) FIXTURE="$2"; shift 2;;
        --fixture-status) FIXTURE_STATUS="$2"; shift 2;;
        *) echo "shodan-idb: unknown flag: $1" >&2; exit 1;;
    esac
done

[ -z "$TARGET" ] && { echo "shodan-idb: empty target" >&2; exit 1; }

if [ -n "$FIXTURE" ]; then
    json=$(cat "$FIXTURE")
    status="$FIXTURE_STATUS"
else
    body=$(curl -sS -w '\n%{http_code}' --max-time 20 "https://internetdb.shodan.io/${TARGET}" 2>/dev/null || echo $'\n000')
    json=$(printf '%s' "$body" | sed '$d')
    status=$(printf '%s' "$body" | tail -n1)
fi

# 404 = "no data on this IP", not an error — silent success.
[ "$status" = "404" ] && exit 0

# Plan defect #1 fix: heredoc + pipe conflict — write script to tmp file instead.
_py_tmp=$(mktemp)
trap 'rm -f "$_py_tmp"' EXIT
cat > "$_py_tmp" <<'PY'
import json, sys
target = sys.argv[1]
try:
    data = json.loads(sys.stdin.read())
except Exception:
    sys.exit(0)
if "detail" in data and "ports" not in data:
    sys.exit(0)
url = f"https://internetdb.shodan.io/{target}"
# Plan defect #2 fix: compact separators throughout to match test assertions.
SEP = (',', ':')
for port in data.get("ports", []):
    print(json.dumps({
        "channel": "shodan-idb",
        "record_type": "open_port",
        "url": url,
        "content": f"port {port} open on {target}",
        "metadata": {"target": target, "port": port}
    }, separators=SEP))
for vuln in data.get("vulns", []):
    print(json.dumps({
        "channel": "shodan-idb",
        "record_type": "cve",
        "url": url,
        "content": f"{vuln} detected on {target}",
        "metadata": {"target": target, "cve": vuln}
    }, separators=SEP))
for h in data.get("hostnames", []):
    print(json.dumps({
        "channel": "shodan-idb",
        "record_type": "hostname",
        "url": url,
        "content": h,
        "metadata": {"target": target, "hostname": h}
    }, separators=SEP))
PY
printf '%s' "$json" | python3 "$_py_tmp" "$TARGET"
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 196kUpdated Claude CodeCodexVerifiedInstall