archibate/dotfiles-opencode / skills/scrapling
scrapling
UnverifiedScrape web pages using Scrapling with anti-bot bypass (like Cloudflare Turnstile), stealth headless browsing, spiders framework, adaptive scraping, and JavaScript rendering. This skill should be used when asked to scrape, crawl, or extract data from websites; web_fetch fails; the site has anti-bot protections; write Python code to scrape/crawl; write spiders.
| File | Kind | Size |
|---|---|---|
| LICENSE.txt | markdown | 1.5 KB |
| SKILL.md | skill md | 19.3 KB |
| examples/.DS_Store | binary | 6.0 KB |
| examples/01_fetcher_session.py | script | 855 B |
| examples/02_dynamic_session.py | script | 951 B |
| examples/03_stealthy_session.py | script | 952 B |
| examples/04_spider.py | script | 1.9 KB |
| examples/README.md | markdown | 1.7 KB |
| references/fetching/choosing.md | markdown | 6.3 KB |
| references/fetching/dynamic.md | markdown | 20.4 KB |
| references/fetching/static.md | markdown | 17.2 KB |
| references/fetching/stealthy.md | markdown | 20.1 KB |
| references/mcp-server.md | markdown | 10.6 KB |
| references/migrating_from_beautifulsoup.md | markdown | 11.0 KB |
| references/parsing/adaptive.md | markdown | 10.5 KB |
| references/parsing/main_classes.md | markdown | 26.7 KB |
| references/parsing/selection.md | markdown | 22.1 KB |
| references/spiders/advanced.md | markdown | 10.5 KB |
| references/spiders/architecture.md | markdown | 6.7 KB |
| references/spiders/getting-started.md | markdown | 4.6 KB |
| references/spiders/proxy-blocking.md | markdown | 8.7 KB |
| references/spiders/requests-responses.md | markdown | 8.9 KB |
| references/spiders/sessions.md | markdown | 8.5 KB |
examples/02_dynamic_session.py
View on GitHub"""
Example 2: Python - DynamicSession (Playwright browser automation, visible)
Scrapes all 10 pages of quotes.toscrape.com using a persistent browser session.
The browser window stays open across all page requests for efficiency.
Best for: JavaScript-heavy pages, SPAs, sites with dynamic content loading.
Set headless=True to run the browser hidden.
Set disable_resources=True to skip loading images/fonts for a speed boost.
"""
from scrapling.fetchers import DynamicSession
all_quotes = []
with DynamicSession(headless=False, disable_resources=True) as session:
for i in range(1, 11):
page = session.fetch(f"https://quotes.toscrape.com/page/{i}/")
quotes = page.css(".quote .text::text").getall()
all_quotes.extend(quotes)
print(f"Page {i}: {len(quotes)} quotes (status {page.status})")
print(f"\nTotal: {len(all_quotes)} quotes\n")
for i, quote in enumerate(all_quotes, 1):
print(f"{i:>3}. {quote}")
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 466k | Updated | Claude CodeCodex | Verified | Install |
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 466k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |