Tomoshibi
Provides a self-hosted server for web scraping and search, transforming any website content into LLM-ready markdown quickly and efficiently.
Tomoshibi is a high-performance, self-hosted search and scraping server designed specifically for AI agents. Built with Go, it rapidly converts web pages into clean, LLM-ready markdown, often in as little as 200ms. It addresses common pain points of hosted scraping APIs by offering a cost-effective, single-binary solution that runs efficiently on minimal resources (e.g., $5/month hosting). Featuring a 'smart mode' for handling both static and dynamic JavaScript-heavy sites, it ensures accurate content extraction while stripping away noise like ads and navigation. Tomoshibi also boasts a monolithic architecture for simplified deployment, offering API compatibility with Firecrawl's schema, making it an ideal choice for building Retrieval Augmented Generation (RAG) systems without incurring high cloud bills.