Skip to content

hint-shu/deep-research / skills/osint-research

osint-research

Unverified

OSINT recon skill — passive entity discovery for domain / IP / email / person / company / github user. Builds a hybrid findings/dossier/graph report from free public sources (Whois, DNS, crt.sh, Wayback, Shodan InternetDB, GitHub code search, Tavily/Firecrawl/Exa/Perplexity dorks, optional theHarvester/subfinder). No subscription dependencies. Disclaimer is non-removable; secrets are redacted; blocklisted dump sites are filtered at outbound and inbound levels.

FileKindSize
SKILL.mdskill md6.1 KB
channels/crtsh.shexecutablescript1.8 KB
channels/dorks.shexecutablescript3.5 KB
channels/github-leaks.shexecutablescript2.0 KB
channels/shodan-idb.shexecutablescript2.2 KB
channels/subfinder-wrap.shscript1.8 KB
channels/theharvester-wrap.shexecutablescript1.8 KB
channels/wayback.shexecutablescript1.5 KB
channels/whois-dns.shexecutablescript2.2 KB
lib/entity-classifier.shexecutablescript1.4 KB
lib/findings-extractor.shexecutablescript1.6 KB
lib/inbound-filter.shscript6.9 KB
lib/secret-patterns.txtmarkdown1.1 KB
lib/secret-redactor.shexecutablescript2.3 KB
templates/graph.mmd.tplother53 B
templates/osint-report.md.tplother1.1 KB

lib/secret-redactor.sh

View on GitHub
#!/usr/bin/env bash
# skills/osint-research/lib/secret-redactor.sh
# Reads stdin, redacts known secret patterns, writes stdout.
# Pattern source: skills/osint-research/lib/secret-patterns.txt

set -uo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PATTERNS_FILE="${OSINT_SECRET_PATTERNS:-$SCRIPT_DIR/secret-patterns.txt}"

if [ ! -r "$PATTERNS_FILE" ]; then
    echo "secret-redactor: patterns file not readable: $PATTERNS_FILE" >&2
    exit 2
fi

# Read all of stdin into a variable (acceptable for typical OSINT payloads <10MB).
input="$(cat)"

# Build a sed expression set, one per pattern. We use perl for richer regex support
# and easy callback-style replacement.
perl_script='
my %patterns;
while (<STDIN>) {
    chomp;
    next if /^\s*#/ || /^\s*$/;
    my ($label, $re) = split /\|/, $_, 2;
    $patterns{$label} = $re if defined $re;
}
my $text = do { local $/; <DATA> };
for my $label (keys %patterns) {
    my $re = $patterns{$label};
    $text =~ s/($re)/_truncate($1, $label)/ge;
}
print $text;

sub _truncate {
    my ($match, $label) = @_;
    return "<REDACTED:$label>" if length($match) < 12;
    return substr($match, 0, 4) . "..." . substr($match, -4);
}
'

# Pass patterns via stdin and the input via DATA section (joined with __DATA__).
# Implementation: use a here-doc trick.
{ cat "$PATTERNS_FILE"; printf '\n__OSINT_REDACTOR_BOUNDARY_8f3a2c1d_NEVER_IN_INPUT__\n'; printf '%s' "$input"; } | \
perl -e '
my %patterns;
my $section = 0;
my $text = "";
while (<STDIN>) {
    if (/^__OSINT_REDACTOR_BOUNDARY_8f3a2c1d_NEVER_IN_INPUT__$/) { $section = 1; next; }
    if ($section == 0) {
        chomp;
        next if /^\s*#/ || /^\s*$/;
        my ($label, $re) = split /\|/, $_, 2;
        $patterns{$label} = $re if defined $re;
    } else {
        $text .= $_;
    }
}
my @errs;
for my $label (keys %patterns) {
    my $re = $patterns{$label};
    eval { $text =~ s/($re)/_t($1, $label)/ge; 1 } or do {
        push @errs, "secret-redactor: bad pattern '"'"'$label'"'"': $@";
    };
}
if (@errs) {
    print STDERR "$_\n" for @errs;
    exit 2;
}
print $text;

sub _t {
    my ($m, $l) = @_;
    return "<REDACTED:$l>" if length($m) < 12;
    # PEM markers are always fully redacted (they contain no extractable entropy after label).
    return "<REDACTED:$l>" if $l eq "private_key_pem";
    return substr($m, 0, 4) . "..." . substr($m, -4);
}
'
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 196kUpdated Claude CodeCodexVerifiedInstall