Web Scraping & Data Collection Agent Skills

AI Tech Digest

Curates specialized AI technology news and technical insights using targeted search strategies and quality filtering rules.

FireCrawl Pitfall Guard

Audits and optimizes FireCrawl implementations by identifying common anti-patterns and security vulnerabilities.

YouTube Video Downloader

Downloads YouTube videos and audio with customizable quality and format settings directly through the command line.

FireCrawl Rate Limit Handler

Implements resilient FireCrawl API interactions using exponential backoff, jitter, and idempotency patterns.

FireCrawl Secondary Workflow B

Executes optimized secondary FireCrawl data extraction tasks to complement primary web scraping workflows.

FireCrawl Reference Architecture

Implements production-ready architectural patterns and structured project layouts for FireCrawl web scraping integrations.

Google Trends ATH Detector

Detects and analyzes Google Trends search spikes and all-time highs using automated browser simulation and statistical modeling.

Zhipu AI Search

Performs intelligent web searches via the Zhipu search engine with automated relative date resolution.

GitHub Harvester

Extracts and processes comprehensive data from GitHub repositories for ingestion into RAG pipelines and LLM knowledge bases.

USPTO Patent and Trademark Database

Accesses USPTO APIs to perform comprehensive patent and trademark searches, analyze prosecution history, and track intellectual property assignments.

Exa Semantic Search

Powers Claude Code with semantic search, similar content discovery, and structured research capabilities via the Exa API.

Article Extractor

Extracts clean, readable text from web articles and blog posts by removing ads, navigation, and clutter.

Wget URL Reader

Fetches and downloads content from any URL using the powerful wget command-line utility.

Exa API Troubleshooting

Diagnoses and resolves common Exa API errors and integration issues to ensure reliable AI search functionality.

FireCrawl Reliability Patterns

Implements robust error handling and fault-tolerance patterns for stable FireCrawl web scraping integrations.

Threads Scraper

Scrapes and extracts post data from Threads profiles using automated browser navigation and authentication.

Brave Search Integration

Performs headless web searches and extracts readable markdown content using the Brave Search API without requiring a browser.

Blockchain Data Pipeline Validation

Validates blockchain data collection pipelines using a systematic 5-step empirical workflow to ensure data integrity and storage efficiency.

Hot List Data Enrichment

Automates company data enrichment for investment dashboards by fetching employee counts, job postings, and news mentions.

WASDE Data Ingestor

Automates the extraction and parsing of monthly USDA WASDE reports into standardized datasets for agricultural market analysis.

Exa Semantic Web Search

Empowers Claude with semantic, neural search capabilities and specialized web filtering using the Exa API.

API Scraper Architect

Provides standardized architectural patterns and Pydantic models for building robust API documentation scrapers.

YouTube Video to Audio

Downloads and converts YouTube videos into high-quality audio files using yt-dlp and ffmpeg.

Ebook Text Extractor

Extracts clean, plain text from EPUB, MOBI, and PDF files for analysis and data processing.

Deep Research & Analysis

Conducts exhaustive, multi-threaded investigations using parallel agent teams to produce high-confidence synthesized reports from 20+ sources.

FireCrawl Installation and Authentication

Installs and configures the FireCrawl SDK and CLI with secure API authentication for web scraping workflows.

Scanner Excel Extraction

Extracts, transforms, and structures data from complex Excel files into JSON or CSV formats.

Scanner PDF Analysis

Analyzes PDF documents to extract structured data, including tables, section headers, and metadata, while providing automated summaries.

Google Places Integration

Queries the Google Places API to retrieve business locations, venue details, and reviews directly through the CLI.

FireCrawl Performance Tuning

Optimizes FireCrawl API integrations using advanced caching, request batching, and connection pooling strategies.