Skip to content

aiskillstore/marketplace / skills/21pounder/web-scrape

web-scrape

Verified

Intelligent web scraper with content extraction, multiple output formats, and error handling

FileKindSize
SKILL.mdskill md5.8 KB
scripts/html_clean.jsscript1.5 KB
skill-report.jsonconfig43.9 KB

scripts/html_clean.js

View on GitHub
const fs = require('fs');
const cheerio = require('cheerio');
const TurndownService = require('turndown');
const { resolve } = require('path');

const readStdin = () => {
    return new Promise((resolve, reject) => {
        let data = '';
        process.stdin.setEncoding('utf8');
        process.stdin.on('data', chunk => data += chunk);
        process.stdin.on('end', () => resolve(data));
        process.stdin.on('error', error => reject(error));
    });
};
(async () => {
    try {
        const html = await readStdin();
        if (!html) {
            console.error('No input provided via stdin');
            process.exit(1);
        }
        const $ = cheerio.load(html);
        const noiseSelectors = [
            'script', 'style', 'noscript', 'iframe', 'svg',
            'nav', 'footer', 'header',
            '.ad', '.ads', '.advertisement',
            '#sidebar', '.sidebar',
            '.comment', '.comments',
            '[role="alert"]', '[role="banner"]', '[role="navigation"]'
        ];
        $(noiseSelectors.join(',')).remove();

        const turndownService = new TurndownService({
            headingStyle: 'atx',
            codeBlockStyle: 'fenced'
        });

        let contentHtml = $('article').html() || $('main').html() || $('body').html();
        const markdown = turndownService.turndown(contentHtml || '');

        console.log(markdown);

    } catch (error) {
        console.error('Error cleaning HTML:', error.message);
        process.exit(1);
    }
})();
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall