Scraper
Optimizes web content for Large Language Models by filtering HTML and converting it to markdown or plain text, drastically reducing token usage.
About
Scraper is a powerful Model Context Protocol (MCP) server designed to streamline web content processing for AI tools. It tackles the inefficiency of feeding raw HTML to LLMs by intelligently pre-processing web pages server-side. Through features like CSS selector filtering, HTML-to-markdown/text conversion, and smart link extraction, Scraper significantly reduces LLM token usage by 70-90%, allowing AI models to focus on relevant data and process 18x more content within the same context window. It also offers robust infrastructure for scraping, including batch operations, intelligent caching, and a real-time monitoring dashboard with an interactive API playground.
Key Features
- Real-time dashboard for monitoring server health, request statistics, and cache metrics
- Interactive API playground for testing scraping tools directly from a browser
- Context optimization via CSS selector filtering and HTML-to-markdown/text conversion
- Three-tier intelligent caching system to minimize redundant requests
- 4 GitHub stars
- Batch operations for concurrent URL processing with retry logic
Use Cases
- Obtain clean plain text content from web pages with minimal formatting
- Extract links for navigation analysis, sitemap generation, or deep crawling
- Efficiently scrape articles, documentation, and blog posts for LLM consumption
