Web Analyzer

概要

Web Analyzer is an enterprise-grade Model Context Protocol (MCP) server engineered for high-performance web analysis. Built upon FastMCP and FastAPI, it delivers a robust suite of features including asynchronous task processing, efficient caching, and comprehensive monitoring capabilities, making it ideal for production environments. The tool excels at systematically exploring websites to discover subpages, leveraging AI for intelligent web page content summarization, and extracting structured information specifically tailored for Retrieval Augmented Generation (RAG) workflows.

主な機能

• AI-powered web page summarization and key information extraction
• Systematic discovery of website subpages and links
• Structured content extraction optimized for Retrieval Augmented Generation (RAG)
• Asynchronous and distributed task processing using Celery and Redis
• Integrated support for OpenAI/Anthropic APIs and robust API key/JWT-based authentication
• 0 GitHub stars

ユースケース

• Automating web content analysis and summarization workflows
• Populating Retrieval Augmented Generation (RAG) systems with structured web data
• Integrating advanced web scraping and AI content processing into custom applications