Skip to content

nexscope-ai/nexscope-ecommerce-skills / ecommerce-geo-score-check

ecommerce.geo-score-check

Verified

Score an ecommerce URL for SEO + GEO readiness and AI citation visibility. Triggers on "geo score", "SEO + GEO score", "check ecommerce GEO", "Amazon/Shopify GEO", "GEO readiness audit". Covers product pages, store pages, marketplace listings, category pages, and homepages, scored from crawlable evidence (not live AI polling).

FileKindSize
.nexscope-manifest.jsonconfig218 B
SKILL.mdskill md14.9 KB
geo-ai-visibility-report-optimized.htmlother86.9 KB
geo-eval-ecommerce/assets/config.jsonconfig388 B
geo-eval-ecommerce/assets/prompts/evaluator.mdmarkdown7.8 KB
geo-eval-ecommerce/profiles/_template/product.mdmarkdown914 B
geo-eval-ecommerce/profiles/_template/queries.csvother1.7 KB
geo-eval-ecommerce/profiles/_template/query_rules.mdmarkdown1.3 KB
geo-eval-ecommerce/profiles/airscape-planetary-design-kilo-coffee-st/product.mdmarkdown1.9 KB
geo-eval-ecommerce/profiles/airscape-planetary-design-kilo-coffee-st/queries.csvother3.2 KB
geo-eval-ecommerce/profiles/airscape-planetary-design-kilo-coffee-st/query_rules.mdmarkdown525 B
geo-eval-ecommerce/profiles/aisi-hair-ombre-blonde-wig-glueless-curl/product.mdmarkdown1.2 KB
geo-eval-ecommerce/profiles/aisi-hair-ombre-blonde-wig-glueless-curl/queries.csvother2.6 KB
geo-eval-ecommerce/profiles/aisi-hair-ombre-blonde-wig-glueless-curl/query_rules.mdmarkdown428 B
geo-eval-ecommerce/profiles/apple-macbook-air-13-m5-2026/product.mdmarkdown1.4 KB
geo-eval-ecommerce/profiles/apple-macbook-air-13-m5-2026/queries.csvother2.3 KB
geo-eval-ecommerce/profiles/apple-macbook-air-13-m5-2026/query_rules.mdmarkdown746 B
geo-eval-ecommerce/profiles/boat-neck-sleeveless-tank-top-shirred-wa/product.mdmarkdown1.1 KB
geo-eval-ecommerce/profiles/boat-neck-sleeveless-tank-top-shirred-wa/queries.csvother2.5 KB
geo-eval-ecommerce/profiles/boat-neck-sleeveless-tank-top-shirred-wa/query_rules.mdmarkdown761 B
geo-eval-ecommerce/profiles/coffee-canister/product.mdmarkdown1.4 KB
geo-eval-ecommerce/profiles/coffee-canister/queries.csvother1.3 KB
geo-eval-ecommerce/profiles/coffee-canister/query_rules.mdmarkdown356 B
geo-eval-ecommerce/profiles/hanging-egg-swing-chair-with-stand-white/product.mdmarkdown1.1 KB
geo-eval-ecommerce/profiles/hanging-egg-swing-chair-with-stand-white/queries.csvother2.7 KB
geo-eval-ecommerce/profiles/hanging-egg-swing-chair-with-stand-white/query_rules.mdmarkdown766 B
geo-eval-ecommerce/profiles/iphone-17-pro/product.mdmarkdown1.4 KB
geo-eval-ecommerce/profiles/iphone-17-pro/queries.csvother2.1 KB
geo-eval-ecommerce/profiles/iphone-17-pro/query_rules.mdmarkdown731 B
geo-eval-ecommerce/profiles/nike-shox-r4-se-fade/product.mdmarkdown1.3 KB
geo-eval-ecommerce/profiles/nike-shox-r4-se-fade/queries.csvother2.3 KB
geo-eval-ecommerce/profiles/nike-shox-r4-se-fade/query_rules.mdmarkdown738 B
geo-eval-ecommerce/profiles/ninja-10-in-1-double-oven-with-flexdoor-/product.mdmarkdown1.8 KB
geo-eval-ecommerce/profiles/ninja-10-in-1-double-oven-with-flexdoor-/queries.csvother2.7 KB
geo-eval-ecommerce/profiles/ninja-10-in-1-double-oven-with-flexdoor-/query_rules.mdmarkdown764 B
geo-eval-ecommerce/profiles/rare-beauty/product.mdmarkdown1.5 KB
geo-eval-ecommerce/profiles/rare-beauty/queries.csvother1.6 KB
geo-eval-ecommerce/profiles/rare-beauty/query_rules.mdmarkdown1.1 KB
geo-eval-ecommerce/profiles/rhode-peptide-lip-tint-sweet-pea/product.mdmarkdown1.2 KB
geo-eval-ecommerce/profiles/rhode-peptide-lip-tint-sweet-pea/queries.csvother2.3 KB
geo-eval-ecommerce/profiles/rhode-peptide-lip-tint-sweet-pea/query_rules.mdmarkdown750 B
geo-eval-ecommerce/profiles/roc-inflatable-stand-up-paddle-board-10f/product.mdmarkdown1.5 KB
geo-eval-ecommerce/profiles/roc-inflatable-stand-up-paddle-board-10f/queries.csvother2.7 KB
geo-eval-ecommerce/profiles/roc-inflatable-stand-up-paddle-board-10f/query_rules.mdmarkdown781 B
geo-eval-ecommerce/profiles/skullcandy-crusher-evo-wireless-headphon/product.mdmarkdown1.2 KB
geo-eval-ecommerce/profiles/skullcandy-crusher-evo-wireless-headphon/queries.csvother2.4 KB
geo-eval-ecommerce/profiles/skullcandy-crusher-evo-wireless-headphon/query_rules.mdmarkdown760 B
geo-eval-ecommerce/scripts/__init__.pyscript0 B
geo-eval-ecommerce/scripts/_next_steps.pyscript5.1 KB
geo-eval-ecommerce/scripts/api_capture_runner.pyscript9.0 KB
geo-eval-ecommerce/scripts/api_client_base.pyscript8.2 KB
geo-eval-ecommerce/scripts/cli.pyscript10.0 KB
geo-eval-ecommerce/scripts/evaluator.pyscript6.9 KB
geo-eval-ecommerce/scripts/html_report.pyscript39.8 KB
geo-eval-ecommerce/scripts/loader.pyscript4.4 KB
geo-eval-ecommerce/scripts/normalizer.pyscript3.0 KB
geo-eval-ecommerce/scripts/preaggregate.pyscript7.0 KB
geo-eval-ecommerce/scripts/profile_builder.pyscript9.5 KB
geo-eval-ecommerce/scripts/send_report_email.pyscript4.4 KB
geo-eval-ecommerce/scripts/task_builder.pyscript2.9 KB
geo-eval-ecommerce/scripts/test_raw_api.pyscript2.6 KB
geo-eval-report-tool.jsscript7.8 KB
references/report-template.htmlother22.7 KB

geo-eval-ecommerce/scripts/evaluator.py

View on GitHub
"""Run evaluator prompt against normalized responses via API."""
import json, sys, os, re, time
from pathlib import Path
from concurrent.futures import ThreadPoolExecutor, as_completed

sys.path.insert(0, str(Path(__file__).parent))
from api_client_base import GATEWAY_BASE_URL, GATEWAY_API_KEY, CURRENT_YEAR
import requests


def _headers():
    return {"Authorization": f"Bearer {GATEWAY_API_KEY}", "Content-Type": "application/json"}


def _load_evaluator_prompt(base_dir, profile):
    """Load evaluator prompt template with product info injected."""
    prompt_path = Path(base_dir) / "assets" / "prompts" / "evaluator.md"
    product_path = Path(base_dir) / "profiles" / profile / "product.md"

    template = prompt_path.read_text(encoding="utf-8") if prompt_path.exists() else ""
    product_info = product_path.read_text(encoding="utf-8") if product_path.exists() else ""
    return template.replace("{product_info}", product_info)


def _extract_json(text):
    """Extract JSON object from model response text."""
    text = text.strip()
    if text.startswith("```"):
        text = re.sub(r"^```(?:json)?\n?", "", text)
        text = re.sub(r"\n?```$", "", text)
        text = text.strip()
    try:
        return json.loads(text)
    except json.JSONDecodeError:
        match = re.search(r"\{[\s\S]*\}", text)
        if match:
            try:
                return json.loads(match.group())
            except json.JSONDecodeError:
                pass
    return None


PRIMARY_SIGNALS = ["best overall", "top pick", "top choice", "#1 pick", "i recommend", "i\'d buy", "i\'d choose", "my top", "best option"]

def _fix_recommendation_strength(eval_result):
    """Override recommendation_strength to primary when quote clearly indicates first-choice."""
    if eval_result.get("recommendation_strength") == "primary":
        return
    quote = (eval_result.get("recommendation_quote") or "").lower()
    position = eval_result.get("position")
    if not quote:
        return
    for signal in PRIMARY_SIGNALS:
        if signal in quote:
            if position == 1 or "best overall" in quote:
                eval_result["recommendation_strength"] = "primary"
                return


def evaluate_single(normalized_path, eval_prompt, model="global.anthropic.claude-sonnet-4-6", timeout=90):
    """Evaluate a single normalized response."""
    normalized = json.loads(Path(normalized_path).read_text(encoding="utf-8"))

    query_text = normalized.get("messages", [{}])[0].get("content", "") if normalized.get("messages") else ""
    response_text = normalized.get("response_content", "")
    citations = normalized.get("citations", [])

    citation_text = ""
    if citations:
        citation_text = "\n\nCitations/URLs found in response:\n"
        for c in citations[:20]:
            citation_text += f"- [{c.get('title', '')}]({c.get('url', '')})\n"

    user_message = (
        f"## User Query\n\n{query_text}\n\n"
        f"## AI Engine Response ({normalized.get('engine', 'unknown')})\n\n{response_text}"
        f"{citation_text}\n\n"
        f"Now evaluate this response. Output JSON only."
    )

    payload = {
        "model": model,
        "messages": [
            {"role": "system", "content": eval_prompt},
            {"role": "user", "content": user_message}
        ],
        "temperature": 0.2,
        "max_tokens": 4096
    }

    resp = requests.post(f"{GATEWAY_BASE_URL}/chat/completions", headers=_headers(), json=payload, timeout=timeout)
    if resp.status_code != 200:
        return None, f"HTTP {resp.status_code}"

    data = resp.json()
    content = data["choices"][0]["message"].get("content", "")
    eval_result = _extract_json(content)

    if eval_result is None:
        return None, "Failed to parse JSON from response"

    # Ensure required fields
    eval_result["schema_version"] = "eval.v1"
    eval_result["query_id"] = normalized.get("query_id", "")
    eval_result["engine"] = normalized.get("engine", "")
    if "search_queries" not in eval_result:
        eval_result["search_queries"] = normalized.get("search_queries", [])

    # Post-process: fix recommendation_strength when quote contains clear primary signals
    _fix_recommendation_strength(eval_result)

    return eval_result, None


def run_eval(base_dir, run_dir, profile, parallel=4, model="global.anthropic.claude-sonnet-4-6", max_retries=2):
    """Run evaluator on all normalized files in a run with auto-resume and retry."""
    normalized_dir = Path(run_dir) / "normalized"
    eval_dir = Path(run_dir) / "eval"
    eval_dir.mkdir(parents=True, exist_ok=True)

    eval_prompt = _load_evaluator_prompt(base_dir, profile)
    normalized_files = sorted(normalized_dir.glob("*.json"))

    # Skip already evaluated (resume support)
    pending = []
    for nf in normalized_files:
        eval_path = eval_dir / nf.name
        if not eval_path.exists():
            pending.append(nf)

    if not pending:
        print(f"[eval] All {len(normalized_files)} files already evaluated")
        return {"total": len(normalized_files), "success": len(normalized_files), "failed": 0}

    print(f"[eval] Evaluating {len(pending)} of {len(normalized_files)} files (parallel={parallel}, retries={max_retries})...")
    success = 0
    failed_files = list(pending)

    for attempt in range(max_retries + 1):
        if attempt > 0:
            # Re-check which files still need eval
            still_pending = [nf for nf in failed_files if not (eval_dir / nf.name).exists()]
            if not still_pending:
                break
            failed_files = still_pending
            print(f"[eval] Retry {attempt}/{max_retries}: {len(still_pending)} files remaining...")
            time.sleep(2)
        else:
            still_pending = failed_files

        batch_failed = []
        with ThreadPoolExecutor(max_workers=parallel) as executor:
            futures = {executor.submit(evaluate_single, str(nf), eval_prompt, model): nf for nf in still_pending}
            for future in as_completed(futures):
                nf = futures[future]
                try:
                    result, error = future.result()
                    if result:
                        eval_path = eval_dir / nf.name
                        eval_path.write_text(json.dumps(result, ensure_ascii=False, indent=2), encoding="utf-8")
                        print(f"  [OK] {nf.stem}")
                        success += 1
                    else:
                        print(f"  [FAIL] {nf.stem}: {error}")
                        batch_failed.append(nf)
                except Exception as e:
                    print(f"  [ERR] {nf.stem}: {str(e)[:80]}")
                    batch_failed.append(nf)

        failed_files = batch_failed
        if not failed_files:
            break

    final_failed = len(failed_files)
    print(f"[eval] Done: {success} success, {final_failed} failed (of {len(pending)} pending)")
    return {"total": len(pending), "success": success, "failed": final_failed}
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall