Web Scraping & Data Collection Agent Skills

Competition Scraper & GitHub Persister

Scrapes Australian creative writing competitions and automatically manages them as structured GitHub issues with intelligent duplicate detection.

Brave Research

Conducts comprehensive web research and content extraction using the Brave Search API and multi-step deep-dive workflows.

Rust Web Crawler

Extracts website content and structure at high speeds to generate LLM-ready Markdown, interactive site maps, and structured data for RAG pipelines.

YT-DLP Video Downloader

Downloads audio and video from thousands of websites with advanced control over formats, subtitles, and metadata.

Web Search

Retrieves real-time information, news, images, and videos from the web using DuckDuckGo to provide up-to-date data and resources.

Web Research & Synthesis

Conducts structured, multi-threaded web research by spawning specialized subagents to gather and synthesize information.

Structured Web Research

Orchestrates parallel subagents to conduct systematic, well-documented web research and synthesize findings into comprehensive reports.

Video Downloader

Automates video and audio downloads from YouTube and other platforms with customizable quality and format options.

Video Downloader

Downloads high-quality videos and audio from YouTube and other platforms for offline access, archival, and content repurposing.

Video Downloader

Downloads videos and audio from YouTube and other platforms for offline viewing, archiving, and media editing.

LinkedIn Resume & Job Optimizer

Automates LinkedIn job searching and generates ATS-optimized resumes with integrated skill gap analysis and interview preparation.

XLSForm Designer

Simplifies the creation of standardized data collection forms in Excel for mobile survey platforms like ODK and KoBoToolbox.

Web Research Pro

Verifies technical requirements and gathers real-time information through structured web searches and documentation analysis.

Gemini Web Search

Enhances Claude with advanced web search capabilities using Gemini to retrieve and synthesize real-time information from the internet.

RSS Feed Operations

Reads and processes RSS/Atom feeds from any blog URL with intelligent automatic feed discovery.

Persistent Sector Caching

Optimizes yfinance data retrieval by implementing a permanent SQLite caching layer to bypass API rate limits and 401 errors.

Competitive Ads Extractor

Extracts and analyzes competitor advertisements from major ad libraries to provide actionable messaging and creative insights.

FireCrawl Pitfall Guard

Audits and optimizes FireCrawl implementations by identifying common anti-patterns and security vulnerabilities.

Web Scraper Quality Assurance

Enforces a rigorous, test-driven development workflow for building and maintaining web scrapers and data extraction agents.

YouTube Transcript Extractor

Extracts high-quality subtitles and transcripts from YouTube videos in multiple formats and languages for analysis and automation.

Video Downloader

Automates video and audio downloads from various web platforms with customizable quality and format settings using yt-dlp.

APOD Downloader

Downloads NASA's Astronomy Picture of the Day along with detailed metadata directly to your desktop.

Web Content Fetcher

Scrapes web pages and WeChat articles to produce clean, noise-free Markdown content for processing, translation, or archival.

XML Element Extractor

Isolates specific XML elements from large source files while maintaining structural integrity and formatting.

Insight Extractor

Extracts valuable insights and key information from URLs or text documents to streamline information processing.

Crawl4AI

Crawls entire websites and extracts clean, structured content into markdown files with AI-enriched metadata.

Rolex Market Liquidity Proxy Analyzer

Analyzes global liquidity and risk-on sentiment by using the Rolex Market Index as a high-beta proxy for financial conditions.

CASS Freight Inflation Detector

Monitors the CASS Freight Index to identify economic cycle turns and predict US inflation trends through logistics data analysis.

Gemini Web Search

Integrates Google Gemini's web search capabilities into Claude Code for comprehensive, real-time information retrieval.

Genome Assembly Dataset Curator

Automates the end-to-end processing and metadata curation of genome assembly datasets for VEuPathDB resources.