code-yeongyu/oh-my-openagent / packages/shared-skills/skills/ultimate-browsing
ultimate-browsing
FlaggedReaches web pages the user's browser cannot open: WAF-blocked and bot-scored hosts, platform-native readers (Chinese and social platforms), archive snapshots with provenance, and a browser you own for stealth or throwaway profiles. Not for plain search, an unblocked static fetch, or a page the user's browser can open.
engine/url_transforms.py
View on GitHub"""Generic URL transforms for the fetch grid.
Transforms are domain-agnostic *rules*. They never reference a specific
site by name. A transform either applies (returns a new URL) or is skipped
(returns None). Callers iterate transforms in order.
Empirically useful transforms (see observations/):
* mobile_subdomain — `www.example.com` → `m.example.com`
Strong win on SSR sites with mobile-first serving. Loss on SPA shells
(some mobile sites return tiny bootstrap HTML).
* am_prefix — `example.com` (no www) → `m.example.com`
* drop_www — occasionally unblocks hosts that gate www but not apex.
Adding new transforms: prove they help on ≥2 unrelated sites first
(cross-site validation — bias check).
"""
from __future__ import annotations
from typing import Callable, Optional
from urllib.parse import urlsplit, urlunsplit
def _replace_host(url: str, new_host: str) -> str:
parts = urlsplit(url)
return urlunsplit(parts._replace(netloc=new_host))
def _original(url: str) -> Optional[str]:
return url
def _mobile_subdomain(url: str) -> Optional[str]:
"""`https://www.example.com/a` → `https://m.example.com/a` (only if host starts with www.)."""
parts = urlsplit(url)
host = parts.hostname or ""
if not host.startswith("www."):
return None
new_host = "m." + host[4:]
if parts.port:
new_host = f"{new_host}:{parts.port}"
return _replace_host(url, new_host)
def _am_prefix(url: str) -> Optional[str]:
"""`https://example.com/a` → `https://m.example.com/a` (only if host has no subdomain)."""
parts = urlsplit(url)
host = parts.hostname or ""
if not host or host.startswith("m."):
return None
# Only apply to apex-like hosts (≤2 dot-separated labels).
if host.count(".") >= 2 and not host.startswith("www."):
return None
if host.startswith("www."):
return None # handled by mobile_subdomain
return _replace_host(url, "m." + host)
def _drop_www(url: str) -> Optional[str]:
parts = urlsplit(url)
host = parts.hostname or ""
if not host.startswith("www."):
return None
return _replace_host(url, host[4:])
TRANSFORMS: dict[str, Callable[[str], Optional[str]]] = {
"original": _original,
"mobile_subdomain": _mobile_subdomain,
"am_prefix": _am_prefix,
"drop_www": _drop_www,
}
def apply_transform(name: str, url: str) -> Optional[str]:
"""Apply one transform by name. Returns transformed URL or None if skipped."""
fn = TRANSFORMS.get(name)
if fn is None:
raise ValueError(f"Unknown transform: {name!r}. Known: {list(TRANSFORMS)}")
return fn(url)
def iter_transformed(url: str, order: list[str]) -> list[tuple[str, str]]:
"""Yield (transform_name, transformed_url) pairs for a given order.
Skips transforms that return None (not applicable) and deduplicates
URLs (so `original` and `drop_www` of `https://example.com` don't double-run).
"""
seen: set[str] = set()
out: list[tuple[str, str]] = []
for name in order:
new_url = apply_transform(name, url)
if new_url is None:
continue
if new_url in seen:
continue
seen.add(new_url)
out.append((name, new_url))
return out
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 197k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |