Skip to content

Bhanunamikaze/Agentic-SEO-Skill

Deterministic LLM-first SEO audits for websites, blog posts, and GitHub repositories. Use this when the user asks to "perform SEO analysis", "run SEO audit", "analyze SEO", "check technical SEO", "review schema", "Core Web Vitals", "E-E-A-T", "hreflang", "GEO", "AEO", or GitHub repository SEO optimization. For full/page/repo audits, run bundled scripts for evidence and return prioritized, confidence-labeled fixes.

FileKindSize
SKILL.mdskill md19.4 KB
scripts/__init__.pyscript34 B
scripts/a11y_seo_checker.pyscript3.4 KB
scripts/ai_crawler_policy_matrix.pyscript2.2 KB
scripts/analyze_visual.pyexecutablescript6.5 KB
scripts/anchor_text_audit.pyscript7.7 KB
scripts/answer_block_scanner.pyscript3.9 KB
scripts/article_seo.pyscript27.1 KB
scripts/audit_runner.pyscript3.1 KB
scripts/broken_links.pyscript8.3 KB
scripts/cache_compression_checker.pyscript6.7 KB
scripts/canonical_checker.pyscript3.4 KB
scripts/capture_screenshot.pyexecutablescript4.4 KB
scripts/citation_readiness.pyscript5.1 KB
scripts/collection_page_checker.pyscript4.2 KB
scripts/competitor_gap.pyscript10.2 KB
scripts/content_decay_detector.pyscript6.2 KB
scripts/content_intent_matcher.pyscript5.4 KB
scripts/crawl_audit.pyscript7.2 KB
scripts/critical_request_chain.pyscript4.8 KB
scripts/duplicate_content.pyscript14.0 KB
scripts/eeat_signal_checker.pyscript5.5 KB
scripts/entity_checker.pyscript19.4 KB
scripts/env_loader.pyscript3.6 KB
scripts/external_link_quality.pyscript6.6 KB
scripts/faceted_nav_audit.pyscript3.3 KB
scripts/fetch_page.pyexecutablescript3.6 KB
scripts/finding_verifier.pyscript4.9 KB
scripts/font_audit.pyscript5.9 KB
scripts/freshness_checker.pyscript6.2 KB
scripts/generate_report.pyscript80.7 KB
scripts/github_api.pyscript12.8 KB
scripts/github_community_health.pyscript8.9 KB
scripts/github_competitor_research.pyscript17.2 KB
scripts/github_readme_lint.pyscript14.9 KB
scripts/github_repo_audit.pyscript21.0 KB
scripts/github_search_benchmark.pyscript7.7 KB
scripts/github_seo_report.pyscript32.0 KB
scripts/github_traffic_archiver.pyscript6.3 KB
scripts/github_weekly_scorecard.pyscript4.4 KB
scripts/gsc_checker.pyscript11.4 KB
scripts/hreflang_checker.pyscript24.2 KB
scripts/hreflang_sitemap_validator.pyscript6.4 KB
scripts/image_inventory.pyscript2.8 KB
scripts/image_weight_audit.pyscript4.8 KB
scripts/indexability_matrix.pyscript3.5 KB
scripts/indexnow_checker.pyscript9.4 KB
scripts/internal_links.pyscript10.5 KB
scripts/javascript_render_audit.pyscript2.9 KB
scripts/lcp_subparts.pyscript7.3 KB
scripts/lib/__init__.pyscript46 B
scripts/lib/safe_http.pyscript6.1 KB
scripts/lighthouse_runner.pyscript6.0 KB
scripts/link_profile.pyscript12.5 KB
scripts/llms_txt_checker.pyscript7.5 KB
scripts/llms_txt_generator.pyscript2.9 KB
scripts/local_seo_checker.pyscript3.7 KB
scripts/log_file_analyzer.pyscript6.0 KB
scripts/mobile_render_checker.pyscript6.9 KB
scripts/orphan_pages_from_sitemap.pyscript6.2 KB
scripts/pagespeed.pyscript12.9 KB
scripts/parse_html.pyexecutablescript12.2 KB
scripts/pre_commit_seo_check.shexecutablescript2.9 KB
scripts/product_schema_checker.pyscript4.9 KB
scripts/readability.pyscript14.5 KB
scripts/redirect_backlink_reclaim.pyscript5.7 KB
scripts/redirect_checker.pyscript6.2 KB
scripts/reference_freshness.pyexecutablescript2.9 KB
scripts/repo_docs_site_checker.pyscript5.7 KB
scripts/repo_file_inventory.pyscript3.6 KB
scripts/repo_release_seo.pyscript6.6 KB
scripts/repo_social_preview_checker.pyscript4.1 KB
scripts/repo_topic_suggester.pyscript6.3 KB
scripts/review_schema_checker.pyscript4.0 KB
scripts/rich_results_guard.pyscript3.4 KB
scripts/robots_checker.pyscript7.2 KB
scripts/robots_path_tester.pyscript1.8 KB
scripts/schema_diff.pyscript3.1 KB
scripts/schema_required_props.pyscript8.1 KB
scripts/schema_template_generator.pyscript5.4 KB
scripts/security_headers.pyscript6.8 KB
scripts/seo_common.pyscript14.1 KB
scripts/sitemap_checker.pyscript6.5 KB
scripts/sitemap_generator.pyscript3.4 KB
scripts/social_meta.pyscript8.7 KB
scripts/third_party_script_audit.pyscript4.7 KB
scripts/topical_cluster_mapper.pyscript6.4 KB
scripts/url_quality.pyscript2.5 KB
scripts/validate_schema.pyexecutablescript4.3 KB
scripts/validate_skill_inventory.pyexecutablescript4.8 KB
scripts/video_schema_checker.pyscript3.6 KB
scripts/visual_regression_snapshot.pyscript5.3 KB
scripts/x_robots_header_checker.pyscript2.0 KB

scripts/competitor_gap.py

View on GitHub
#!/usr/bin/env python3
"""
Competitor Topic Gap Analyzer

Crawls competitor sitemaps / pages, extracts H-tag topics, and identifies
content gaps by comparing against your site's topic coverage.

No API keys required — uses sitemaps and on-page content only.

Usage:
    python competitor_gap.py https://yoursite.com --competitor https://competitor.com --json
    python competitor_gap.py https://yoursite.com --competitor https://c1.com --competitor https://c2.com
"""

import argparse
import json
import re
import sys
import time
from collections import Counter, defaultdict
from urllib.parse import urlparse, urljoin

try:
    from bs4 import BeautifulSoup
except ImportError:
    print("Error: beautifulsoup4 required. Install with: pip install beautifulsoup4")
    sys.exit(1)

try:
    from lib.safe_http import safe_get
except ImportError:
    from scripts.lib.safe_http import safe_get

STOP_WORDS = {
    "the", "a", "an", "and", "or", "but", "in", "on", "at", "to", "for",
    "with", "by", "of", "from", "as", "is", "are", "was", "were", "be",
    "been", "this", "that", "these", "those", "it", "he", "she", "they",
    "we", "you", "your", "my", "their", "our", "its", "which", "who",
    "what", "where", "when", "why", "how", "all", "any", "both", "each",
    "few", "more", "most", "other", "some", "such", "no", "not", "only",
    "own", "same", "so", "than", "too", "very", "can", "will", "just",
    "should", "have", "has", "had", "do", "does", "did", "about", "into",
    "would", "could", "here", "there", "also", "get", "got", "make", "use",
}


# ---------------------------------------------------------------------------
# Fetch helpers
# ---------------------------------------------------------------------------

def fetch_page(url: str, timeout: int = 10) -> str:
    try:
        return safe_get(url, timeout=timeout).text
    except Exception:
        return ""


def extract_sitemap_urls(site_url: str, limit: int = 100) -> list:
    """Extract URLs from sitemap.xml."""
    parsed = urlparse(site_url)
    sitemap_url = f"{parsed.scheme}://{parsed.netloc}/sitemap.xml"

    html = fetch_page(sitemap_url)
    if not html:
        return []

    urls = re.findall(r"<loc>([^<]+)</loc>", html)

    # Handle sitemap index (sitemaps pointing to other sitemaps)
    if any("sitemap" in u.lower() and u.endswith(".xml") for u in urls[:5]):
        expanded = []
        for sub_url in urls[:10]:
            time.sleep(0.3)
            sub_html = fetch_page(sub_url)
            if sub_html:
                expanded.extend(re.findall(r"<loc>([^<]+)</loc>", sub_html))
        urls = expanded

    return urls[:limit]


# ---------------------------------------------------------------------------
# Topic extraction
# ---------------------------------------------------------------------------

def extract_topics(html: str) -> dict:
    """Extract topics from H-tags and title of a page."""
    soup = BeautifulSoup(html, "html.parser")

    title = ""
    title_tag = soup.find("title")
    if title_tag:
        title = title_tag.get_text(strip=True)

    topics = {
        "title": title,
        "h1": [],
        "h2": [],
        "h3": [],
    }

    for tag in ["h1", "h2", "h3"]:
        for h in soup.find_all(tag):
            text = h.get_text(strip=True)
            if text and len(text) > 3:
                topics[tag].append(text)

    return topics


def normalize_topic(text: str) -> str:
    """Normalize a topic string for comparison."""
    words = re.findall(r"\b[a-z]+\b", text.lower())
    return " ".join(w for w in words if w not in STOP_WORDS and len(w) > 2)


def extract_topic_phrases(topics: dict) -> set:
    """Extract normalized topic phrases from H-tags."""
    phrases = set()
    for source in ["title", "h1", "h2", "h3"]:
        items = topics[source]
        if isinstance(items, str):
            items = [items]
        for text in items:
            norm = normalize_topic(text)
            if norm and len(norm.split()) >= 2:
                phrases.add(norm)
    return phrases


# ---------------------------------------------------------------------------
# Site crawl (lightweight — sitemap + top pages)
# ---------------------------------------------------------------------------

def crawl_site_topics(site_url: str, max_pages: int = 50) -> dict:
    """
    Crawl a site's sitemap and extract topics from each page.
    Returns {url: {title, h1, h2, h3}} and a set of all topic phrases.
    """
    urls = extract_sitemap_urls(site_url, limit=max_pages)

    if not urls:
        # Fallback: crawl homepage and extract internal links
        homepage = fetch_page(site_url)
        if homepage:
            soup = BeautifulSoup(homepage, "html.parser")
            base_domain = urlparse(site_url).netloc
            for a in soup.find_all("a", href=True):
                full = urljoin(site_url, a["href"])
                if urlparse(full).netloc == base_domain:
                    urls.append(full)
            urls = list(set(urls))[:max_pages]

    page_topics = {}
    all_phrases = set()

    for url in urls:
        time.sleep(0.3)
        html = fetch_page(url)
        if not html:
            continue

        topics = extract_topics(html)
        page_topics[url] = topics
        all_phrases.update(extract_topic_phrases(topics))

    return {
        "pages_crawled": len(page_topics),
        "page_topics": page_topics,
        "all_phrases": all_phrases,
    }


# ---------------------------------------------------------------------------
# Gap analysis
# ---------------------------------------------------------------------------

def find_topic_gaps(your_phrases: set, competitor_data: dict) -> dict:
    """
    Compare your topic coverage vs competitors.
    Returns gaps (topics competitors cover that you don't).
    """
    gaps = []
    competitor_only = set()

    for comp_url, comp in competitor_data.items():
        comp_phrases = comp["all_phrases"]
        comp_unique = comp_phrases - your_phrases

        for phrase in comp_unique:
            competitor_only.add(phrase)
            gaps.append({
                "topic": phrase,
                "competitor": comp_url,
                "competitor_pages": comp["pages_crawled"],
            })

    # Deduplicate and count coverage
    topic_coverage = Counter()
    for gap in gaps:
        topic_coverage[gap["topic"]] += 1

    # Sort by coverage (topics covered by more competitors = higher priority)
    prioritized = []
    seen = set()
    for topic, count in topic_coverage.most_common():
        if topic not in seen:
            sources = [g["competitor"] for g in gaps if g["topic"] == topic]
            prioritized.append({
                "topic": topic,
                "covered_by_competitors": count,
                "competitors": sources,
                "priority": "High" if count >= 2 else "Medium",
            })
            seen.add(topic)

    # Your unique topics (competitive advantage)
    your_unique = your_phrases - competitor_only

    return {
        "gaps": prioritized[:50],
        "your_unique_topics": len(your_unique),
        "competitor_unique_topics": len(competitor_only),
        "overlap_topics": len(your_phrases & competitor_only),
        "total_your_topics": len(your_phrases),
    }


# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------

def main():
    parser = argparse.ArgumentParser(
        description="Competitor Topic Gap Analyzer — finds content gaps from competitor sitemaps/pages"
    )
    parser.add_argument("url", help="Your site URL")
    parser.add_argument("--competitor", action="append", required=True, help="Competitor URL (can specify multiple)")
    parser.add_argument("--max-pages", type=int, default=50, help="Max pages to crawl per site (default: 50)")
    parser.add_argument("--json", action="store_true", help="Output as JSON")
    args = parser.parse_args()

    print(f"Crawling your site: {args.url}...", file=sys.stderr)
    your_data = crawl_site_topics(args.url, max_pages=args.max_pages)

    competitor_data = {}
    for comp_url in args.competitor:
        print(f"Crawling competitor: {comp_url}...", file=sys.stderr)
        competitor_data[comp_url] = crawl_site_topics(comp_url, max_pages=args.max_pages)

    print("Analyzing topic gaps...", file=sys.stderr)
    gap_report = find_topic_gaps(your_data["all_phrases"], competitor_data)

    report = {
        "your_site": args.url,
        "your_pages_crawled": your_data["pages_crawled"],
        "your_topics": len(your_data["all_phrases"]),
        "competitors": {
            url: {"pages_crawled": d["pages_crawled"], "topics": len(d["all_phrases"])}
            for url, d in competitor_data.items()
        },
        "analysis": gap_report,
    }

    if args.json:
        # Convert sets to lists for JSON serialization
        report_json = json.loads(json.dumps(report, default=lambda x: list(x) if isinstance(x, set) else str(x)))
        print(json.dumps(report_json, indent=2))
        return

    print(f"\nTopic Gap Analysis")
    print("=" * 60)
    print(f"Your Site          : {args.url} ({your_data['pages_crawled']} pages, {len(your_data['all_phrases'])} topics)")
    for url, d in competitor_data.items():
        print(f"Competitor         : {url} ({d['pages_crawled']} pages, {len(d['all_phrases'])} topics)")

    ga = gap_report
    print(f"\nCoverage Summary:")
    print(f"  Your unique topics             : {ga['your_unique_topics']}")
    print(f"  Competitor-only topics (GAPS)   : {ga['competitor_unique_topics']}")
    print(f"  Shared topics                  : {ga['overlap_topics']}")

    if ga["gaps"]:
        print(f"\nTop Content Gaps ({len(ga['gaps'])} found):")
        for gap in ga["gaps"][:20]:
            icon = "🔴" if gap["priority"] == "High" else "⚠️"
            comp_str = ", ".join(urlparse(c).netloc for c in gap["competitors"])
            print(f"  {icon} \"{gap['topic']}\" — covered by {gap['covered_by_competitors']} competitor(s): {comp_str}")

        print("\nRecommendation: Create content targeting the high-priority gap topics above.")
        print("Start with topics covered by 2+ competitors (highest opportunity).")
    else:
        print("\n✅ No significant topic gaps found — your content coverage is strong!")


if __name__ == "__main__":
    main()
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall