alirezarezvani/claude-skills / engineering/universal-scraping-architect/skills/universal-scraping-architect
universal-scraping-architect
VerifiedBuild a scraper or extractor for sites, PDFs or paginated APIs and validate the fields (Firecrawl or local Python). Load whenever the user wants data pulled from pages, files or an API, or a scraper broke.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 5.0 KB |
| references/firecrawl-technical-guide.md | markdown | 2.7 KB |
| references/local-extraction-patterns.md | markdown | 2.5 KB |
| references/scraping-ethics-security.md | markdown | 2.5 KB |
| requirements.txt | manifest | 300 B |
| scripts/firecrawl_example.py | script | 7.7 KB |
| scripts/local_bs4_example.py | script | 10.0 KB |
| scripts/validate_extraction.py | script | 1.3 KB |
scripts/validate_extraction.py
View on GitHub#!/usr/bin/env python3
"""
validate_extraction.py
Intelligent stdlib-only validation for JSON structures.
"""
import argparse
import json
import sys
import os
def validate_json(file_path):
if not os.path.exists(file_path):
return {"status": "error", "message": f"File {file_path} not found."}
try:
with open(file_path, 'r', encoding='utf-8') as f:
data = json.load(f)
# Basic validation: ensure it's not empty and is a list/dict
if not data:
return {"status": "warning", "message": "JSON file is empty."}
return {"status": "ok", "message": "JSON is valid and well-formed."}
except json.JSONDecodeError as e:
return {"status": "error", "message": f"Invalid JSON: {str(e)}"}
def main():
parser = argparse.ArgumentParser(description="Standard Library JSON Validator")
parser.add_argument("file", help="Path to JSON file to validate")
parser.add_argument("--json", action="store_true", help="Output results in JSON format")
args = parser.parse_args()
result = validate_json(args.file)
if args.json:
print(json.dumps(result, indent=2))
else:
print(f"[{result['status'].upper()}] {result['message']}")
sys.exit(0 if result['status'] == "ok" else 1)
if __name__ == "__main__":
main()
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| receiving-code-reviewobra/superpowersUse when receiving code review feedback, before implementing suggestions, especially if feedback seems unclear or technically questionable - requires technical rigor and verification, not performative agreement or blind implementation | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| requesting-code-reviewobra/superpowersUse when completing tasks, implementing major features, or before merging to verify work meets requirements | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| using-git-worktreesobra/superpowersUse when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallback | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| verification-before-completionobra/superpowersUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |