Web Scraping & Data Collection Claude 스킬

FireCrawl Core Workflow B

Executes secondary FireCrawl workflows to complement primary data collection and automated web scraping tasks.

Crypto News Aggregator

Aggregates real-time cryptocurrency news from over 50 authoritative sources with advanced filtering and relevance scoring.

FireCrawl Rate Limit Handler

Implements robust rate limiting, exponential backoff, and idempotency patterns for FireCrawl API integrations.

Scrapling Web Extractor

Automates web data extraction using the Scrapling CLI to capture HTML, Markdown, and text from static or dynamic webpages.

YouTube Transcript Extractor

Extracts and saves YouTube video subtitles or transcripts to local text files using command-line tools or automated browser interaction.

Z.AI Multimodal CLI

Integrates vision analysis, real-time web search, and GitHub exploration capabilities into Claude Code workflows.

Daily Tech News Reporter

Generates curated daily technical reports by scraping high-quality news and papers from customizable URL lists.

ZAI CLI Integration

Enhances Claude with real-time web search, vision-based image analysis, and advanced GitHub repository exploration.

Vault Protocol Logo Extractor

Extracts and organizes brand logos for DeFi vault protocols by identifying homepage links and automating asset retrieval.

Financial Data Collector

Automates the collection and validation of standardized financial data for US publicly traded companies from free public sources.

Agent Reach

Performs comprehensive multi-platform internet searches and content retrieval with intelligent tool fallback and zero external dependencies.

Canonical Event Deduplication

Normalizes and merges duplicate data from multiple sources using reputation scoring and semantic hash-based grouping.

Twitter Reader

Fetches Twitter/X post content and metadata into clean Markdown format using the Jina.ai API to bypass JavaScript restrictions.

Web to Markdown Converter

Transforms web pages into clean, readable Markdown files optimized for AI ingestion and local documentation.

Perplexity AI Search

Performs real-time AI web searches with citations using Perplexity models to provide up-to-date information and scientific literature.

Regex vs LLM Structured Text Parser

Optimizes text parsing workflows by combining efficient Regex patterns with LLM-based validation for high-accuracy, cost-effective data extraction.

Dev Opinions Scanner

Aggregates and synthesizes real-world developer perspectives from Hacker News, Reddit, and major technical communities.

Ark Research

Researches technical solutions and gathers cross-platform evidence to inform architecture and implementation decisions.

Reverse API Engineer

Transforms browser traffic into production-ready Python API clients through automated HAR analysis and code generation.

Community Discourse Search

Scans developer communities like Hacker News, Reddit, and Lobsters to gather real-world experience reports and technology opinions.

GitHub Code Search

Searches GitHub for production-ready implementations, libraries, and code patterns to accelerate development research.

Recent Topic Research & Sentiment

Researches and synthesizes real-world community discussions from the last 30 days across Reddit, X, and the web.

X Research

Conducts real-time agentic research and sentiment analysis across X/Twitter to gather developer insights and industry trends.

Google Search & Web Access

Enables Claude to search the live web and fetch content from specific URLs to provide up-to-date information.

Tavily Web Search & Extraction

Equips Claude with high-performance web search capabilities and deep content extraction tools powered by the Tavily API.

arXiv Search & Metadata Retrieval

Automates the retrieval and normalization of academic paper metadata from arXiv to support research pipelines and literature reviews.

Reddit Data Connector

Extracts and analyzes Reddit content including posts, comments, subreddits, and user profiles using the public JSON API.

Tavily Web Search & Research

Performs real-time web searches, content extraction, and deep research to provide LLM-optimized data for coding and analysis tasks.

Tavily Web Crawler

Crawls websites and extracts content from multiple pages into structured JSON or local markdown files.

Tavily Site Mapping

Maps entire website structures and discovers specific URLs rapidly without extracting full page content.