Web Scraping & Data Collection Claude 스킬

Perplexity AI Search

Performs real-time AI web searches with citations using Perplexity models to provide up-to-date information and scientific literature.

Dev Opinions Scanner

Aggregates and synthesizes real-world developer perspectives from Hacker News, Reddit, and major technical communities.

Ark Research

Researches technical solutions and gathers cross-platform evidence to inform architecture and implementation decisions.

Reverse API Engineer

Transforms browser traffic into production-ready Python API clients through automated HAR analysis and code generation.

Recent Topic Research & Sentiment

Researches and synthesizes real-world community discussions from the last 30 days across Reddit, X, and the web.

X Research

Conducts real-time agentic research and sentiment analysis across X/Twitter to gather developer insights and industry trends.

Google Search & Web Access

Enables Claude to search the live web and fetch content from specific URLs to provide up-to-date information.

Tavily Web Search & Extraction

Equips Claude with high-performance web search capabilities and deep content extraction tools powered by the Tavily API.

arXiv Search & Metadata Retrieval

Automates the retrieval and normalization of academic paper metadata from arXiv to support research pipelines and literature reviews.

Reddit Data Connector

Extracts and analyzes Reddit content including posts, comments, subreddits, and user profiles using the public JSON API.

Tavily Web Search & Research

Empowers Claude with real-time web search, content extraction, and deep research capabilities using the Tavily API.

MarkItDown File Converter

Converts diverse file formats including PDFs, Office documents, and media into structured, token-efficient Markdown for LLM processing.

YouTube Transcript Extractor

Extracts text transcripts and captions from YouTube videos for content analysis, summarization, and documentation.

YouTube Transcript Extractor

Extracts and formats transcripts from YouTube videos using URLs or video IDs.

Tavily Web Search & Research

Enables high-quality web search, content extraction, and deep research capabilities using the Tavily API.

Apify Brand Reputation Monitoring

Monitor brand reputation and sentiment by scraping reviews and mentions across major social and review platforms.

Apify Market Research

Analyzes market trends, competitive pricing, and consumer behavior by extracting real-time data from major social and location-based platforms.

Apify Trend Analysis

Monitors and extracts emerging trends from Google, Instagram, Facebook, YouTube, and TikTok to power data-driven content strategies.

Apify Ultimate Web Scraper

Automates data extraction from over 55 major platforms including Instagram, TikTok, and Google Maps using AI-driven Apify Actors.

Competitor Intelligence by Apify

Extracts and analyzes competitor data across social media, maps, and travel platforms using specialized Apify Actors.

Apify Actorization

Converts existing applications into serverless Apify Actors for scalable web scraping and data processing workflows.

Apify Actor Development

Develops, debugs, and deploys high-performance serverless actors for web scraping, data processing, and automation on the Apify platform.

Tavily Web Search

Integrates the Tavily Search API to provide real-time, clean, and LLM-optimized web search results for RAG pipelines.

Structured Data Extraction

Transforms unstructured files like PDFs, Word documents, and presentations into structured Pydantic models using LlamaExtract services.

PDF Processing

Parses and extracts structured content from complex PDF documents using LlamaParse and agentic OCR capabilities.

Firecrawl Web Scraper

Converts websites into LLM-ready markdown and structured data using the Firecrawl API.

Exa Semantic Search

Performs semantic and neural web searches to find content based on context and meaning rather than simple keywords.

Deep Research Workflow

Orchestrates multi-agent parallel workflows to conduct comprehensive research and generate structured, insight-driven reports.

YouTube Downloader

Downloads high-quality videos and HLS streams from platforms like YouTube, Vimeo, and Mux using optimized workflows for yt-dlp and ffmpeg.

Website Clone & Replicate

Replicates existing websites into production-ready Next.js 16 and Tailwind CSS v4 codebases using Firecrawl MCP.