Web Scraping & Data Collection Agent Skills

Web Search

Retrieves real-time information, news, images, and videos from the web using DuckDuckGo to provide up-to-date data and resources.

Video Subtitle Extractor

Extracts and cleans transcripts from global video platforms like YouTube and Bilibili into readable text files.

Unified Research

Aggregates data from multiple search APIs like Tavily, SerpAPI, and Perplexity to provide comprehensive, fact-based research summaries.

Article Extractor

Extracts clean, readable text from web articles and blog posts by removing ads, navigation, and clutter.

Firecrawl Real Estate Scraper

Extracts structured property data from real estate listing URLs using Firecrawl AI for market analysis and content creation.

JRA-VAN Keiba Data Pipeline

Automates the retrieval of JRA-VAN horse racing data and facilitates seamless loading into PostgreSQL databases.

Web Research Agent

Executes multi-perspective web searches and generates structured research reports with source attribution and uncertainty analysis.

RuTracker Torrent Manager

Automates searching, filtering, and downloading torrents from RuTracker using Playwright and aria2c.

Markdown URL Extractor

Converts web URLs into clean, raw markdown to minimize context window usage and eliminate HTML noise.

YouTube Video Downloader

Downloads YouTube videos and audio with customizable quality and format settings directly through the terminal.

Market Recon

Conducts automated community sentiment analysis and market research via Reddit to identify developer pain points and adoption trends.

Playbooks Fetch

Converts any URL into clean, structured Markdown with YAML metadata for efficient content consumption and LLM processing.

Market Research Setup

Configures the essential Brave Search MCP integration and environment settings required for automated market research workflows.

Web Research & Source Attribution

Conducts multi-page web research to synthesize information with full source attribution and conflict detection.

WeChat Article Fetcher & Research Analyst

Fetches and ranks WeChat articles based on research interests with seamless Obsidian integration.

Web Search

Performs structured web searches via DuckDuckGo to retrieve real-time documentation, library information, and technical solutions.

YouTube Transcript Downloader

Extracts and cleans transcripts, subtitles, and captions from YouTube videos using yt-dlp and OpenAI Whisper for AI-powered transcription.

HTML Parser Rule Writer

Generates and validates robust regex-based HTML parsing rules to extract article titles, links, and metadata from webpages.

Reddit Research

Researches Reddit communities, threads, and wikis to gather community insights and generate structured research reports.

YouTube Transcript & Search

Fetches YouTube video transcripts, searches for content, and retrieves channel updates directly within the AI workflow.

YouTube Playlist & Transcripts

Browses YouTube playlists and extracts video transcripts for automated content analysis and summarization.

YouTube Transcript Fetcher

Retrieves comprehensive transcripts and metadata from YouTube videos for AI-driven summarization, research, and content analysis.

YouTube Video Transcripts

Extracts high-quality transcripts and metadata from YouTube videos for analysis and summarization within Claude.

YouTube Channel Explorer

Provides comprehensive tools to interact with YouTube channels, including handle resolution, upload browsing, and channel-specific searching.

YouTube Captions & Subtitles

Extracts closed captions and multi-language subtitles from YouTube videos for accessibility, quoting, and content analysis.

YouTube API & Transcripts

Accesses YouTube transcripts, video metadata, and channel data without requiring a Google API key or quota management.

Perplexity Scientific Search

Performs real-time, AI-powered web searches and scientific literature discovery using Perplexity models via OpenRouter.

USPTO Database & IP Analysis

Accesses USPTO APIs for comprehensive patent and trademark searches, examination history, and intellectual property analysis.

Brave Research

Conducts comprehensive web research and content extraction using the Brave Search API and multi-step deep-dive workflows.

Idea Analyzer

Evaluates business ideas from multiple media sources to filter hype, assess market viability, and determine strategic fit.