Web Scraping & Data Collection Agent Skills

Topic Collector

Automates the gathering of AI industry trends, product launches, and developer insights from multiple high-signal web sources.

Markdown Web Scraper

Extracts web page content and converts it into clean, readable Markdown for seamless AI analysis and data collection.

YouTube Transcript to Chinese

Extracts YouTube subtitles and generates formatted Chinese transcripts with optional translation and timestamp support.

YouTube Collector

Manages YouTube channel tracking by automating video content collection, transcript retrieval, and structured summary generation.

DOCX Advanced Patterns

Extracts text and structural data from complex Microsoft Word documents containing nested tables, checkboxes, and multi-layered cell layouts.

Sketch Checker: Metal Band Vetting

Conducts deep-dive research into metal bands to identify far-right, NSBM, or fascist ties using a multi-phase investigative methodology.

GitHub Source Fetcher

Extracts clean source code from GitHub file URLs using the GitHub CLI to bypass web scraping noise and HTML clutter.

Twitter Data Collector

Automates the collection, filtering, and processing of Twitter search results into structured link lists and databases.

Gemini Web Search

Empowers Claude with real-time internet research capabilities by integrating Gemini's Google Search tool directly into the terminal workflow.

Authenticated Web Scraper

Extracts content from password-protected websites and internal documentation sites by leveraging the Windows Edge browser via Chrome DevTools Protocol.

Snowflake Documentation Scraper

Extracts and organizes Snowflake documentation into structured Markdown format with intelligent caching and configurable spider depth.

Local Media Source Manager

Lists and manages configured event sources for Instagram accounts and web aggregators used in newsletter generation.

Data Sourcing & Provider Optimization

Optimizes B2B data enrichment through intelligent provider selection, waterfall logic, and credit-efficient routing.

Browser Content Capture

Extracts data from JavaScript-heavy websites, authenticated pages, and complex documentation using advanced browser automation.

Exa AI Web Search & Research

Powers Claude with real-time web searches, deep company research, and high-quality programming documentation retrieval using Exa AI.

Firecrawl Web Extraction

Extracts high-quality, LLM-optimized web data and performs advanced crawling through a powerful CLI integration.

News Extractor

Extracts structured content from popular Chinese news platforms and converts it into JSON and Markdown formats.

Hacker News Agent

Integrates real-time Hacker News data streams into AI agents for automated tech news monitoring and community trend analysis.

Asteroid Tracking Agent

Monitors near-Earth asteroids and hazardous space objects using real-time NASA NeoWs data and integrated x402 payment processing.

Web Search

Searches the web using Exa AI to provide real-time information retrieval and up-to-date data for AI coding workflows.

Web Content Fetcher

Extracts clean, markdown-formatted content and metadata from any URL using the Jina Reader API for LLM consumption.

Firecrawl Web Scraper

Converts entire websites into LLM-ready markdown and structured data with advanced anti-bot bypass and JavaScript rendering.

URL Summarization Engine

Extracts and summarizes web content using quote-grounding and structured reporting to ensure high technical fidelity.

Google Export

Downloads and converts public Google Docs, Sheets, and Slides into local formats for direct analysis and integration.

Trend Discovery & Market Analysis

Identifies high-potential trending topics and data gaps on X and the web to surface monetization opportunities for AI agents.

Tavily Web Crawler

Crawls and extracts website content into structured markdown files or context-optimized chunks for AI analysis.

Tavily AI Search

Integrates the Tavily API to perform live web searches and structured data retrieval for RAG-augmented workflows.

Hacker News Reader

Fetches and analyzes real-time stories, comments, and user data from Hacker News using the official API.

Apify Web Scraping & Automation

Automates web data collection and browser tasks using pre-built Actors for popular sites like Amazon, Google, and LinkedIn.

Brave Search

Integrates privacy-focused web, image, video, and news search capabilities directly into Claude Code via the Brave Search API.