Web Scraping & Data Collection Claude 技能

Defuddle Web Content Extractor

Extracts clean, clutter-free Markdown from web pages to optimize token usage and context relevance.

FireCrawl Rate Limit Manager

Implements robust rate limiting, exponential backoff, and idempotency patterns for resilient FireCrawl API interactions.

Exa Neural Search Core Workflow

Integrates Exa's neural search API to perform semantic web queries and retrieve contextually relevant data for RAG pipelines.

AI Twitter Radar

Monitors Twitter/X for real-time AI news, trending tools, and developer insights using the Bird CLI.

YouTube Video to Audio

Downloads and converts YouTube videos into high-quality audio files using yt-dlp and ffmpeg.

Ebook Text Extractor

Extracts clean, plain text from EPUB, MOBI, and PDF files for analysis and data processing.

Scanner Excel Extraction

Extracts, transforms, and structures data from complex Excel files into JSON or CSV formats.

Scanner PDF Analysis

Analyzes PDF documents to extract structured data, including tables, section headers, and metadata, while providing automated summaries.

Firecrawl Best Practices & Pitfall Prevention

Identifies and resolves common Firecrawl integration mistakes, anti-patterns, and resource management issues during code reviews.

Add Awesome Tool

Analyzes AI tool URLs to extract metadata and automatically categorizes and adds them to the awesome-ai-tools repository.

MCP Opportunity Pipeline

Automates the end-to-end lifecycle of discovering, validating, building, and publishing Model Context Protocol (MCP) servers and automation tools.

Research Reporter

Automates information gathering from web searches and authoritative sources to generate and save structured research reports.

Competitive Research with Bright Data

Conducts enterprise-grade company research, competitive analysis, and market intelligence using professional web scraping and search tools.

YouTube Info MCP Installer

Installs and configures the YouTube Info MCP server to enable automated extraction of video metadata and details within Claude Code.

Pure.md MCP Installer

Automates the installation and configuration of the Pure.md MCP server for seamless web-to-markdown conversion within Claude Code.

Korean Job Hunter & Web Scraper

Automates web scraping workflows to collect and analyze job postings from major Korean and international job boards.

Competition Scraper & GitHub Persister

Scrapes Australian creative writing competitions and automatically manages them as structured GitHub issues with intelligent duplicate detection.

Stalker OSINT Research

Conducts deep, multi-level OSINT research on people, companies, domains, and concepts using automated investigative techniques.

LinkedIn Intelligence

Automates LinkedIn profile discovery and data extraction using optimized search queries and structured result parsing.

YT-DLP Video Downloader

Downloads audio and video from thousands of websites with advanced control over formats, subtitles, and metadata.

Crawl4AI

Crawls entire websites and extracts clean, structured content into markdown files with AI-enriched metadata.

Genome Assembly Dataset Curator

Automates the end-to-end processing and metadata curation of genome assembly datasets for VEuPathDB resources.

LinkedIn Resume & Job Optimizer

Automates LinkedIn job searching and generates ATS-optimized resumes with integrated skill gap analysis and interview preparation.