Web Scraping & Data Collection Claude 스킬

Firecrawl Web Scraper

Converts any website into clean, LLM-ready markdown or structured data using the Firecrawl API.

Crawl4AI Web Scraper

Extracts clean markdown and structured data from any website, including JavaScript-heavy single-page applications.

Documentation Crawler

Automates the process of crawling, extracting, and converting documentation websites into structured Markdown or Claude Code skills.

Video & Transcript Downloader

Downloads videos, audio, and clean paragraph-style transcripts from YouTube and other yt-dlp supported platforms.

Web Search

Retrieves real-time information, news, images, and videos from the web using DuckDuckGo to provide up-to-date data and resources.

Video Subtitle Extractor

Extracts and cleans transcripts from global video platforms like YouTube and Bilibili into readable text files.

Obsidian Web Clipper Template Creator

Generates structured JSON templates for the Obsidian Web Clipper by analyzing web page metadata and verifying CSS selectors.

YouTube Video Downloader

Downloads YouTube videos and audio with customizable quality and format settings using the powerful yt-dlp engine.

YouTube Transcript Downloader

Extracts and cleans transcripts, subtitles, and captions from YouTube videos using yt-dlp and OpenAI Whisper for AI-powered transcription.

Tavily Web Search & Extraction

Provides real-time web search, content extraction, and automated site crawling capabilities powered by the Tavily API.

Tavily Web Search

Empowers Claude with real-time web search, automated content extraction, and deep research capabilities using the Tavily API.

Tavily Web Search

Empowers Claude with real-time web search, content extraction, and advanced crawling capabilities using the Tavily API.

Firecrawl Web Scraper

Extracts clean web content, captures screenshots, and parses PDFs using the powerful Firecrawl API.

Web Research Agent

Executes multi-perspective web searches and generates structured research reports with source attribution and uncertainty analysis.

Firecrawl Scraper

Automates complex web data extraction, page crawling, and document parsing using the Firecrawl API.

Article Extractor

Extracts clean, readable text from web articles and blog posts by removing ads, navigation, and clutter.

WASDE Data Ingestor

Automates the extraction and parsing of monthly USDA WASDE reports into standardized datasets for agricultural market analysis.

Google Trends ATH Detector

Detects and analyzes Google Trends search spikes and all-time highs using automated browser simulation and statistical modeling.

China Macro News Aggregator

Aggregates and summarizes real-time China macro-economic news from premium financial sources into professional magazine-style reports.

RuTracker Torrent Manager

Automates searching, filtering, and downloading torrents from RuTracker using Playwright and aria2c.

Media Downloader

Downloads high-quality video and audio from over 1,000 platforms with automated metadata extraction and QR code support.

Market Recon

Conducts automated community sentiment analysis and market research via Reddit to identify developer pain points and adoption trends.

Playbooks Fetch

Converts any URL into clean, structured Markdown with YAML metadata for efficient content consumption and LLM processing.

CASS Freight Inflation Detector

Monitors the CASS Freight Index to identify economic cycle turns and predict US inflation trends through logistics data analysis.

Rolex Market Liquidity Proxy Analyzer

Analyzes global liquidity and risk-on sentiment by using the Rolex Market Index as a high-beta proxy for financial conditions.

Market Research Setup

Configures the essential Brave Search MCP integration and environment settings required for automated market research workflows.

Firecrawl Best Practices & Pitfalls

Identifies and avoids common Firecrawl integration mistakes and anti-patterns to optimize web scraping efficiency.

API Scraper Architect

Provides standardized architectural patterns and Pydantic models for building robust API documentation scrapers.

Web Research & Source Attribution

Conducts multi-page web research to synthesize information with full source attribution and conflict detection.

WeChat Article Fetcher & Research Analyst

Fetches and ranks WeChat articles based on research interests with seamless Obsidian integration.